\documentclass[../../book.tex]{subfiles}
\graphicspath{{\subfix{.}}}
\begin{document}
\chapter{Handing dates and time with datetime}
\setcounter{section}{-1}  % Start subsections from 0

You will often  desire to keep track of not only \textit{what} you did, but \textit{when} you did it. For instance, you might want to report a spectrum with the date it was acquired.  Or you might have acquired multiple data points, and want to look at the dependence of the values on the time between measurements. While you can record times in a notebook, and type them into a program, temporal information is often included in data files. Thus, if you can extract this in a meaningful way, you might \textit{automatically} get date and time information.  And even if you do have to type in temporal data, it might be nice to have Python handle calculating time differences.  For instance: how many days are between February 13 and June 23? This is a question a computer can answer much more quickly for you, as long as you can communicate the dates in a way that it can operate on. 
Finally, if you have many different dates, and want to change the format for how you report them, you might find it nice to have Python change all the formatting. 

The \lstinline{datetime} library in Python makes the extraction, calculation, and formatting of temporal information simple. But, before we discuss the library's usage, we need to understand the various ways that we represent time. 


\section{Understanding date and time conventions}
\setcounter{subsection}{-1}  % Start subsections from 0

\subsection{Gregorian Calendar}
This is the calendar that most of the Western world is familiar with. When someone says: I was born on on July 7th, they are using the Gregorian Calendar. In this calendar, years start on January 1st and last for 365.2425 days. The fractional days means that we have leap years every 4 years, but not every 100 years (unless it is also divisible by 400).  

\subsection{12 hour versus 24 hour time}
We all know that there are 24 hours in the day. Though people in the USA often represent this by two 12 hour cycles (with AM and PM designations), many places in the world simply use a 24 hour clock.  The day starts at 00:00 and ends at 23:59, where the first pair of numbers designate the hours, and the latter pair of numbers designate the minutes. 

\subsection{Timezones and UTC}
If I am in State College, PA and I want to call a friend in Oslo, Norway, it is not enough to tell that friend "I will call you at noon." Instead, the question becomes am I calling at noon PA time, or noon Oslo time? Once that decision is made, the other person must translate 'noon' into their own time.  This can be done by counting the time zones between us.  Currently the standard is to use \textbf{Coordinated Universal Time} (UTC) zones. In this system, the timezone containing the town of Greenwich, England is the `base' time. As you move west, you subtract an hour per timezone.  As you go east, you add one hour per time zone. Timezones are often expressed in terms of offsets between their local time and that of Greenwich.  In our example above, State College, PA is at UTC offset of -5, and Oslo, Norway is at a UTC offset of +1. Thus, there are 6 hours between Oslo and PA, or PA has an offset of -6 compared to Oslo.  Thus, if it is noon in PA, it is 6pm in Oslo. \footnote{Of course, this is a simple example that ignores complications like daylight savings time.}

\subsubsection*{ISO 8601}
ISO stands for ``International Organization for Standardization." It attempts to establish standards for various things---including counting and reporting time. 

The ISO 8601 standard for time includes: 
\begin{itemize}
    \item When representing a year, we use four digits, running from 0000 to 9999, with 0000 being equal to 1BC, 0001 being equal to 1AD.  However, years before 1583 (the Gregorian calendar was first introduced October 15, 1582) are not automatically included.  Thus, the standard actually runs from 1583 to 9999. It is worth noting that this is the official \textit{standard}. However, Python extends this standard infinitly, both forwards and backwards. 
    
    \item We specify a particular month, then we using YYYY-MM, where YYYY is a four digit year that follows the same rules as above, and the MM are two digits representing a month. Months start at 01 (January) and run through 12 (December). You may find other formats for the month specification online, but they are old, and since the year 2000, the \textit{only} way to specify a month is YYYY-MM.

    \item We specify a day, using either YYYY-MM-DD or YYYYMMDD, where YYYY and MM are the same as above, and DD is a two digit day, starting at 01 (the first day of the month) and running through the last day of the month. Since there is only one way to specify months, we are going to recommend you use the same format for days: YYYY-MM-DD.

    \item There are several ways to specify time, but we recommend using Thh:mm:ss:sss. The first ``T" indicates what follows is time.  'hh' are hours with values from 00 (midnight) to 24.\footnote{Both 00 and 24 are midnight, but 00 is the midnight at the start of the day, and 24 is midnight at the end of the day.  Thus 24 of one day \textit{is the same time} as 00 on the next day.} 'mm' are minutes with values from 00 to  59. 'ss' are seconds, with values from 00 to 60 (60 is only used for leap-seconds). 'sss' specifies milliseconds, and has values from 000 to 999. 

    When specifying time, you can omit any of the above values, as long as any lower order time element is omitted as well.  Thus, if you omit seconds, you must also omit milliseconds. 

    You may also add decimals to the end of the lowest order time element. Thus, 10:09:30 is the same as 10:09.5.  There is no limit to the precision of the decimal fraction, which is how precision of the ISO 8601 can be extended beyond milliseconds.

    \item If you want to specify the time zone, then you must specify this using the same hh:mm designation used for time, but appended by a "+z" or "-z", where z is the time offset.   The first draft of this paragraph was written in Pennsylvania, at 7:22pm.  Pennsylvania is in UTC -5, thus  we could represent this time as 19:22-05.\footnote{Here we have not used the minutes designation, though there are a few timezones that are off by 30 minutes, so you might need them eventually. } If you are in the UTC time zone, you simply use +00:00 or +00.  That is, the offset for Greenwich is always positive.\footnote{You can also use the designator ``Z", which stands for "Zulu". }

    \item If you want to specify a date and time (also called datetime) you can simply combine the date and time elements.  For example, the first draft of this paragraph was written 2023-10-6T19:25:20-05. Recall that the capital 'T' indicates we are moving from talking about dates to talking about time. 

    \item If you want to designate just an amount of time something has lasted---in other words a duration---you use the following general format: P[n]Y[n]M[n]DT[n]H[n]M[n]S. Here the leading ``P" indicates that this is a period of time. The n's are numbers, and the Y, M, D, H, M, and S, indicate year, month, day, hour, minute, and second, respectively. Also, the ``T" \textit{must always} be included, since we can omit the longer times if they are zero.  Thus, a duration of 1 month and 1 hour is P1M1H, but a duration of 1 minute is PT1M.
\end{itemize}
Within ISO 8601, there are other representations that are possible, such as specifying the week of the year. However, we believe the above will provide a solution for most of the needs of our readers. More information on the various designations within ISO 8601 can be found on the Wikipedia article for ISO 8601.

\subsubsection*{Epoch (Unix) time (POSIX timestamp)}
While there is no `natural' convenient zero point for time,\footnote{A case might be made for the start of the Universe, but this seems a bit inconvenient.} there are a few common choices. We have already seen that the Gregorian calendar sets the years at the transition between BC and AD as the zero point. Another common choice (especially when working with computers) is the so called "Unix time" or "Epoch time."  This is defined as is the amount of time that has elapsed since January 1, 1970 (midnight UTC+0), not counting leap seconds.  This is often expressed in units of microseconds. This choice of units, of course, leads to very large numbers. While such large numbers might be difficult for humans to read, computers have little trouble with them.\footnote{One note of caution: as time differences get large, we can run into issues with 32-bit integers.}

\subsubsection*{Resolution}
Though scientists have developed the means to measure time to at least attoseconds, the fundamental unit of time in Python (and many programming languages) is the microsecond.  Thus, unless you do something to circumvent this standard, times and time differences will `naturally' be reported to the nearest microsecond.  Though there are ways to deal with time on finer resolutions, we anticipate you will not have such a need, and so we will accept this resolution. 


\section{Data types in Python's \texttt{datetime} library}

In order to understand how best to use the datetime library, it is important to understand the data types within it, as well as how they are used.  Also, because of how the library is structured, we will need to introduce objects in an order that may, at first, appear strange. 

\begin{itemize}
    \item \texttt{timedelta} This object stores a duration of time. You can create a timedelta object using the \lstinline{timedelta()} function, and supplying the time difference using keywords: days, minutes, seconds, microseconds, and even things like weeks.  However, the difference is stored only in terms of days and seconds. Each of these can be positive or negative differences. 

    \begin{lstlisting}
from datetime import timedelta
td = timedelta(days = -5, hours = 1, minutes = -23, seconds = 16)
td
    \end{lstlisting}

\begin{pyout}
datetime.timedelta(days=-5, seconds=2236)
\end{pyout}

    However, if we ask for a print of this timedelta, we have:
\begin{pyin}
    print(td)
\end{pyin}
\begin{pyout}
    -5 days, 0:37:16
\end{pyout}

    So that you can see we have a difference of -5 days, 0 hours, 37 minutes, and 16 seconds.

    Additionally, a timedelta object is automatically created when you calculate the difference between between two date objects, time objects, or datetime objects. These objects will be described below, and then an example of this usage given in the section after this one. 

    \item \texttt{timezone} The timezone object is used to specify an offset from UTC. You need to provide a value of the timedelta type in order to use it. 

    \begin{lstlisting}
        from datetime import timedelta, timezone
        td = timedelta(hours = -5)
        tz = timezone(td)
        tz
    \end{lstlisting}
\begin{pyout}
    datetime.timezone(datetime.timedelta(days=-1, seconds=68400))
\end{pyout}
    So you can see that, internally, the timezone object references the timedelta object.  

    But, as above, you can print the object to get a more human formatted result. 
\begin{pyin}
print(tz)
\end{pyin}
\begin{pyout}
UTC-05:00
\end{pyout}    

    Which is a result that is more similar to how we discuss timezones. 


\item  \texttt{time} The time object holds time in terms of hours, minutes, seconds, and microseconds. You can generate a time object by providing an ISO 8601 formatted string, with the important difference being that the ISO standard is to use milliseconds, but the datetime library expects microseconds. This difference is due to the very wide use of microseconds in computation and computers. Specifically, since we often also work with Unix time (see farther below), which uses microseconds as the base unit it is convenient to also use microseconds in datetime.

    \begin{lstlisting}
from datetime import time
t = time(12, 13, 28, 123456)

    \end{lstlisting}

    Again, this is held in an object, whose structure you can find by asking for the value.

    \begin{pyin}
        t
    \end{pyin}
    \begin{pyout}
        datetime.time(12, 13, 28, 123456)
    \end{pyout}

    You can print the time to get a formatted output.

    \begin{pyin}
        print(t)
    \end{pyin}
    \begin{pyout}
        12:13:28.123456
    \end{pyout}

    Here you see that the microseconds are represented as a decimal number with the seconds. This is another difference between how time is handled in the datetime and the ISO standard. 

    The hour, minute, seconds, and microseconds can be accessed by using the attributes. 

    \begin{pyin}
        t.hour
    \end{pyin}
    \begin{pyout}
        12
    \end{pyout}

    \begin{pyin}
        t.min
    \end{pyin}
    \begin{pyout}
        13
    \end{pyout}
    \begin{pyin}
        t.second
    \end{pyin}
    \begin{pyout}
        28
    \end{pyout}
    \begin{pyin}
        t.microsecond
    \end{pyin}
    \begin{pyout}
        123456
    \end{pyout}

    One thing to note here is that we can specify only partial times, for instance, only hours and minutes, without seconds and microseconds. However, the time object always must have at least hours and minutes.  If you try to specify only the hours, the datetime library will accept this, but assign a value of 0  to the minutes. 

    You can associated a timezone offset with the time object, in two ways.  Both require a timezone object.  You can provide this to the keyword \lstinline{tzinfo} during the reaction of the time object, or you can use the \lstinline{replace()} method of the time object to add the timezone to the time object, using the \lstinline{tzinfo} keyword.



    \item \texttt{date} represents year, month, and day in reference to the Gregorian calendar. It follows the ISO 8601 standard, with one difference: the calendar is extended indefinitely in either direction. The object has methods that can interpret dates that come ISO 8601 format.

    \begin{lstlisting}
from datetime import date
d = date(2024, 4, 17,)
    \end{lstlisting}

    This creates a date object that holds the information on the date. 

    If you simply ask for the value of this, you will see that the structure of the object is such that it holds the year, month, and day inside the object. 

    \begin{pyin}
    d
    \end{pyin}
    \begin{pyout}
    datetime.date(2024, 4, 17)
    \end{pyout}
    This also indicates that the date object belongs to a datetime object. 

    Python also realizes that many people want dates formatted, and so if you print the date object, you will get a formatted output. 

    \begin{pyin}
    print(d)
    \end{pyin}
    \begin{pyout}
    2024-04-17
    \end{pyout}

    The date object also has attributes that allow access to the year, month, and day separately.

    \begin{pyin}
        d.year
    \end{pyin}
    \begin{pyout}
        2024
    \end{pyout}

    \begin{pyin}
        d.month
    \end{pyin}
    \begin{pyout}
        4
    \end{pyout}

    \begin{pyin}
        d.day
    \end{pyin}
    \begin{pyout}
        17
    \end{pyout}

    One thing to note is that, when creating a date object, it must always have a year, month, and day.  You cannot specify only a year, or only a year and month, despite these being valid ISO formats. 

    

    \item \texttt{datetime} Similarly to how dates and times can be combined in ISO 8601, we can combine them in Python for the datetime object. 

    \begin{lstlisting}
from datetime import datetime
dt = datetime(2024, 4, 19, 12, 34, 56)
dt\end{lstlisting}
    
\begin{pyout}
datetime.datetime(2024, 4, 19, 12, 34, 56)
\end{pyout}

As with all the other examples, printing the datetime object will provide a better formatted output. 

\begin{pyin}
2024-04-19 12:34:56
\end{pyin}
\begin{pyout}
2024-04-19 12:34:56
\end{pyout}

All the methods that we used for date and time objects to get the components can be used with datetime objects as well. 
     
\end{itemize}





\section{Reading and writing temporal data}

There are times you wish to simply work with text that specifies time. For instance, you might have a date and/or time encoded within a file or a file name, and you wish to extract that so that you have the datetime information for that data. 

You can use the \lstinline{fromisoformat()} functions of the date, time, and datetime classes to interpret ISO formatted text as either dates, times, or both, respectively. In the most general case, we have both date and time information:

\begin{lstlisting}
from datetime import datetime

iso_string = "2024-04-19T12:34:56.789"

dt = datetime.fromisoformat(iso_string)

dt\end{lstlisting}

\begin{pyout}
datetime.datetime(2024, 4, 19, 12, 34, 56, 789000)
\end{pyout}

So, you can see that we have generated a datetime object.

Of course, it is quite likely that you will encounter strings of date and time information that are not ISO formatted.  In this case, it might be simpler to use the \lstinline{strptime()} function.  You can think of this function as being `\textbf{str}ing \textbf{p}arse \textbf{time},' which is a function that can take more arbitrary formatted strings and parse them into date, time, or datetime objects. When using this function, you must supply two positional arguments: the first is the string you wish to parse, and the second is the format of this string:
\begin{lstlisting}
from datetime import datetime

date_string = "19-04-2024 12:34:56"
format_string = "%d-%m-%Y %H:%M:%S"
dt = datetime.strptime(date_string, format_string)
dt\end{lstlisting}
\begin{pyout}
datetime.datetime(2024, 4, 19, 12, 34, 56)
\end{pyout}

In this usage you can see that we represent an expected number as \lstinline{%<X>}, where `X' indicates the component to associated the number with, and the character between them are the characters in the string that separate them. In this way, we can parse arbitrarily formatted strings. 




Once you have a date, time, or datetime object, and you simply want to write out this information into a file. There is a corresponding function to \lstinline{strptime()}, method of a date, time, or datetime object: \lstinline{strftime()}. This can be thought of as `\textbf{str}ing \textbf{f}rom \textbf{time}' and requires a positional argument that is the formatting of the string.  This formatting is the same as used in \lstinline{strptime()} For instance, if we worked with the datetime object we created above, we can specify a new format: 

\begin{pyin}
dt.strftime("%Y/%m/%d %H.%M.%S")
\end{pyin}
\begin{pyout}
'2024/04/19 12.34.56'
\end{pyout}

And, if you simply want ISO formatted text, you can use the \lstinline{isoformat()} method of a date, time, or datetime object.  This will directly give you the corresponding ISO formatted string:

\begin{pyin}
dt.isoformat()
\end{pyin}
\begin{pyout}
'2024-04-19T12:34:56'
\end{pyout}


\section{Other operations in the datetime library}

There are times you want to get the current date, time, or both.  This is easily done using the \lstinline{now()} method of the object you wish to create. For instance, if I wanted the current datetime as I write this, I could use:
\begin{lstlisting}
from datetime import datetime
current = datetime.now()
print(current)
\end{lstlisting}
\begin{pyout}
2024-04-22 09:01:15.979583
\end{pyout}
Where you see that you get the current date and time down to the microsecond. 

One very useful thing about having date, time, or datetime objects is that you can do math on them.  For instance, imagine you wanted to know how long it has taken me to consider what I want to say and write this sentence from the prior example...

\begin{lstlisting}
endtime = datetime.now()
timetowrite = endtime - current
timetowrite
\end{lstlisting}

\begin{pyout}
datetime.timedelta(seconds=125, microseconds=776147)
\end{pyout}

The first line created a new datetime object, based on the current time. The second line asked to subtract the current datetime object from the timetowrite datetime object.  When we ask for the result, we see this is a timedelta object. 


\end{document}