Here at Counsyl we use Python, Django and Postgres as key components of our software technology stack. As part of modeling and tracking a complex business process we often need to answer the question so often and so eloquently asked by the philosopher Morris Day: What Time Is It? And also, what time was it? When did that happen? When does this need to happen?
The three software packages mentioned above provide tools for tracking time accurately, but there are some traps you can fall into if you are not careful. I recently investigated a time-related anomaly in a subsystem I maintain and discovered some interesting facts about Django time settings. The results of that investigation are now available as a simple Django application in the Counsyl GitHub repository.
This particular problem revolves around time zones. A time zone is a set of rules for determining a clock reading, say 11:00 am on July 24, 2011, given a fixed point in time. Or, given a clock reading, determining the fixed point in time it refers to. Just knowing the clock reading isn’t enough because that clock reading occurred multiple times around the world.
Let’s call any representation of a fixed point in time a timestamp, even if part of the representation is implicit (maybe we just happen to know that some clock reading is for Pacific Standard Time). As a timestamp makes its way from a Python application using Django to Postgres and back again it passes through the application code, the Django ORM, the Python database adaptor and finally Postgres before returning back again in the reverse trip.
So that gives us three timestamps we might care about:
- The timestamp we started with.
- The timestamp we stored in Postgres.
- The timestamp we loaded from Postgres.
Of course, for a particular timestamp in step 1 we’d want the other two to be equivalent, i.e., to refer to the same fixed point in time even if they might use different representations. But, if you are not careful, all three could be different from each other.
The README file in the Django application has the full explanation and the application itself has the details, but here is a high-level summary.
Python represents timestamps with datetime objects which come in two flavors, respectively called aware and naive, but which could also be called ‘explicit time zone’ and ‘implicit time zone’. An aware datetime in Python is a clock reading and a particular time zone representation. Such a datetime is an unambiguous representation of some fixed point in time. A naive datetime has no explicit time zone reference, we just happen to know (hopefully) what time zone it happens to be in.
The trouble comes in with naive datetimes and daylight savings time. If you are using an implicit time zone that has daylight savings time, then there are naive datetimes that can mean two different fixed points in time. That’s because when you have a daylight savings “fall back” event, where for example the clocks might go from 2:00am to 1:00am, then the same set of clock readings repeat themselves on the same day one hour apart. If the time zone is implicit, you cannot distinguish between datetimes in the first hour and datetimes in the second.
There is a related but subtler trap when it comes to daylight savings “fall forward” events where the clocks might go from 1:00am to 3:00am and skip all the 2:00am times. For the full explanation see the example Django application above. That application stores and then loads two such troublesome timestamps using different methods and then checks to see that the timestamps we stored and loaded matched the ones we saved. The application has all the details, but here are the results:
In either case the solution, and the moral of the story, comes straight from the Zen of Python: explicit is better than implicit.
Dave Peticolas is a software engineer at Counsyl. Prior to that he was a software engineer at Lucasfilm for ten years. He has contributed to numerous open source projects including GnuCash and Twisted, and is the author of a popular introduction to Twisted and asynchronous programming at http://krondo.com. He’ll be giving a talk “Django, Postgres, Timezones, and You” at the Django Meetup in San Francisco on 10/9/13.
December 23, 2020
Myriad’s 7 Greatest Hits in 2020
Before we say goodbye to 2020, we’d like to thank and recognize our partners. Together, we ensure patients have access…Read more about Myriad’s 7 Greatest Hits in 2020
December 4, 2020
Knowledge is Power: How PKU Affected the Ahern Family
December 3rd is National PKU Day. PKU, short for phenylketonuria, is an inherited genetic condition in which the body cannot properly…Read more about Knowledge is Power: How PKU Affected the Ahern Family
November 3, 2020
Simplify Patient Identification
Streamlining Cancer Risk Assessment with Digital Tools November is Family Health History Month and is a good time to bring…Read more about Simplify Patient Identification