I think summary here is that if you are storing an exact moment in time then UTC does the job. If however you are storing a future date and time in a certain location, due to the uncertainty about future daylight saving time changes you have to save it with time zone information in order to adjust for future changes to that system.
I suppose it’s the difference between exact time and social/political time (there is probably a better term for that).
Yeah, this is exactly the difference between what I’d call “timestamps” (something that represents a particular instant in time), and what I’d call “datetimes”, things like “midnight on Jan 1, 2019”.
I worked at an insurance startup that stored coverage period start/ends as timestamps, which ended up being semantically wrong. The coverage ends at midnight of the next year, wherever _you_ (technically the provider, I think) are. So there is no one single instant in time when coverage ends; a timestamp is the wrong data type for representing this.
If you've worked on a distributed team that had members in the UK/EU and the USA then you've run into this before when scheduling meetings.
A 10am PDT America/Los_Angeles meeting is different from a 6pm BST Europe/London meeting which is different from a 5pm GMT meeting and those will jump around by an hour depending on time changes. If the bulk of your team is in Seattle then you probably want to pick the US timezone for the meeting and not GMT or London. If you have a manager in London, though, who sets up the meeting time in their local timezone then Seattle employees will notice tomorrow that the meeting is a hour later in Seattle than it normally is.
If everyone shitcanned clocks jumping around twice a year this craziness would end.
I think this is the inverse problem. In this case, timestamp is the appropriate data structure for the meeting and each person's client needs to convert it back to local time, accounting for things like daylight savings time and so on.
Recurring meetings cannot be expressed as timestamps (unless you want all the humans to hate you). The recurrence has to be expressed in terms of local time (and date/day of week) in some (ideally per-event) timezone.
It's fine as long as all the meetings are conventionally expressed in one time zone.
If some meetings are created in some US tz, some other meetings in some EU timezones, then when the biyearly DSTgeddon week(s) come, you not only have meetings shifting unexpectedly but you also have conflicts since some meetings changed and others didn't!
Such a mess. But it doesn't happen frequently enough for it to be worthwhile fixing/regulating/whatever
> If some meetings are created in some US tz, some other meetings in some EU timezones, then when the biyearly DSTgeddon week(s) come, you not only have meetings shifting unexpectedly but you also have conflicts since some meetings changed and others didn't!
But this is a problem in the real world, not a software problem. Choosing to use UTC when that's not how the business actually thinks about schedules will lead to you just doing your own DST math, which is harder than just storing the correct time zones from the get go.
Sure it is a real world problem.
And no, I wasn't advocating for UTC which would not work for half a year, instead of causing a few days of minor headaches.
It's caused by the fact that different branches of a large corp have genuine situations where some meetings are between local employees and other situations where you invite a mix of distributed people.
Thus there is no "natural default" as to which timezone is "correct".
For a repeating meeting I think the construct you want is actually a datetime, generally.
E.g., a weekly meeting is not a meeting that happens on one particular timestamp and then repeats every 24*7 hours. If you hit daylight savings time, you want a 9am meeting to still be on the "new" 9am, not shift to 10am or 8am.
Future times are tricky. Some weekly meetings are 24 hours * 7 days apart. Some are at the same time in (fixed) LocationA, 7 days later; some in (fixed) LocationB. Some in whatever location the organizer is (maybe a personal reminder rather than a meeting with others).
It would be a lot easier if locations changing offsets to other locations was a lot less frequent.
Every calendar app I know asks for a timezone when scheduling a meeting. So if you schedule it in Europe/London, it will move in UTC. If you schedule it in GMT+1 (which what Europe/London is in summer), it won't.
IIRC in Saudi Arabia the start and end of daylight savings is decided on each year, so not only do you need a historical record, but there's no way to future-proof it without integrating some external source of data, otherwise you'd need to manually add the dates for each year at some point.
We adopted a policy the a `date` type is always interpreted in the reader's local timezone, which is sort how humans think of it anyway; put differently, a `date` type is always relative to the person's watch that is reading said value.
There are basically three types of time: coordinated time, based on atomic clocks, zoned time, derived from coordinated time by applying a time zone rule to calculate an offset, and wall clock time. The latter two are often lumped together as local time but are not in practice the same thing.
TZ rules are one way, you can derive zoned time reliably from utc, but not the other way around unless you already know the offset. That’s why seemingly storing utc is best, because you can apply whatever tz rule you want. But, because tz rules are decided by politicians (dst, ramadan, date line shifts, it gets pretty weird, …) and often with little advance notice, tz rule databases are often wrong in practice, which means storing only utc and getting zoned time back out of it reliably is hard. That’s why storing iso timestamps with offsets is best. It preserves both utc and intended local (zoned) time at the time of storing, even if the tz database then changes.
But this is not the author’s problem. They’re dealing with wall clock time. They want to store 9 am and have it always be 9 am, no matter what happens with time zones. That is easy: store the timestamp without tz info. But, the hard part is knowing the instant in time (in coordinated time). Wall clock times may occur twice, so there is no single reliable way to get utc out of it, and even when calculating utc the result can be wrong or become wrong due to the political insanity that is zoned time so it is dangerous to store it and rely on it. You see this problem with anything that schedules people or resources. Something scheduled at 9 am is always 9 am, but if the system needs to send a reminder at 9 am, when do you schedule it? There are no solutions I’m aware of without edge cases.
I solved this for a reservation system by storing wall clock time without offsets and the time zone (id) of the resource separately, and then calculating zoned time for that zone and comparing it with the stored wall clock time every time the instant in time mattered. It worked but was tricky to code.
I think it's even more complicated than that. Human time is much richer even than wall-clock time. E.g., if I go out for a dawn run every day, to me that's the same time. Or think of the Jewish Sabbath, which runs from sundown Friday to after dark on Saturday. Lunchtime. Teatime. Lunar calendars. Whatever people will do when they live on Mars. Or the way wall-clock time existed before railroads changed it.
It's often our job to cram human concepts into a small number of bits, so I get why we do what we do. But I think it's sometimes worth being explicit that the anthropological concepts we are dealing with are very complex, and we may leave a lot out when we simplify them enough to make sense to the sort of machine that can be cheaply built in its era.
Wall clock is solved by having a camera pointed at local live news where local time is shown, and having your system updated periodically where this period is arbitrarily short.
A similar issue can get developers in trouble with dates, specifically the YYYY-MM-DD for a given event. In C# for instance, there used to only be the DateTime, which stored a date AND a time, which was set to 00:00:00. It was on you to know to ignore the time in comparisons and/or always use the DateTime.Date method to compare with a 00:00 DateTime. This was fixed with the introduction of DateOnly (https://docs.microsoft.com/en-us/dotnet/api/system.dateonly?...).
Databases usually have types to represent just dates or dates/times. Always use the lowest resolution type to represent what you actually are trying to capture.
Rule number one and the only rule about wall clock times is that they are display style only and must never be stored. Local times must use 24 hour format.
This isn't what they're talking about, though. Say your boss says "your work shift starts at 9am". That isn't an instant in time, it's all about the display rule: clock says 9am and you're not at work, you're fired.
We don't typically care about this because we work 9-5 M-F, and daylight savings time conveniently happens at 2am on Sunday morning. But someone was told to come in at 2am this morning and couldn't actually do that because 1:59:59 became 3:00:00. How do you handle that? It's not a code problem, it's a business rules problem.
No, this is not a display rule. This is the domain data type "local time", which stores time component without offset or time zone.
Data stored this way is basically telling that regardless of current UTC offset, locally on the clock there will be always the same time shown.
Display rules are saying how exactly this time will be displayed on the clock (i.e. locale, 12/24 hour format etc). Wall clocks usually do not show "am/pm" information, so the same time indeed may occur twice on them.
I do not think this is what they meant, 24h or 12h+am/pm are equivalent amount of information.
I believe they were referring to the fact that because of DST and other time adjustments, 24h wall clock times can in fact occur twice (or not at all, like this morning).
That is still either display rule or scheduling process topic, rather than a data type issue.
Few examples:
1. Local public transportation systems must commit to certain schedule of departures from every stop and make it unambiguous. This means that they have to always use local time in publicly communicated schedule, but plan the time table in UTC assuming that offsets will not change. Usually results in departures happening twice on the same time, requiring some extra resource planning.
2. Long-range trains and flights must (and usually do) operate in UTC, communicating arrival and departure times via display rules.
3. Shift planning in 24-hour working processes: offsets complicate the planning process and add unnecessary risk of breaking compliance, so they better be avoided and UTC must be used in combination with display rules.
4. Execution of scheduled jobs: depends on recurrence setup, for which display rules must distinguish between "once per day" (local time + take first occurrence on spring offset change) and "every 24 hours" (UTC + display times of occurrence).
It seems to me that you are mostly arguing about what kind of time to use for a given business problem here. This is actually an interesting problem, but it is a business problem.
For example where I live trains will have a 1h delay (sometimes there are buffers that you can use to reduce the delay, or you cancel trains that anyway only run in the time window that is missing) when “springing forward” and will take a 1h break in the middle of the night when “falling back.” The schedule itself doesn’t change. Seems weird to me, but it is how the business rule is.
But what if the business rule is to use wall clock time? And that is actually a sensible choice for things like concerts or dinner appointments in a calendar.
When you use wall clock time (I assume that is what you mean when you call it local time) in example 4 you choose
> local time + take first occurrence on spring offset change
which is _one_ possible way to handle duplicate wall clock times; wall clock times are underspecified which you implicitly admit by choosing the first occurrence. So it _is_ a data type issue, unless you define the wall clock time data type to contain the occurrence number (e.g. something like `2022-10-30 02:32 II`).
To me that’s not a wall clock time but a wall clock time _with occurrence_ (or whatever name you want to give it) and it’s probably not a concept the user will know, so it needs some UX tweaks (e.g. hide the occurrence when there are no duplicate times and when there are notify the user about the ambiguity, offer a default and explicitly state it, maybe allow to change it to the 2nd occurrence, or maybe disallow duplicate times and ask for another time).
Another problem is that when switching forward there are holes in the time line, but that is easier to solve (it’s an invalid input).
> For example where I live trains will have a 1h delay (sometimes there are buffers that you can use to reduce the delay, or you cancel trains that anyway only run in the time window that is missing) when “springing forward” and will take a 1h break in the middle of the night when “falling back.” The schedule itself doesn’t change. Seems weird to me, but it is how the business rule is.
The ferries near me essentially keep the offset for the whole sailing day. The sailing day starts around 5 am, and has a number of trips; service has been reduced during covid, so typical last sailing times are around midnight or 1 AM, but when service continued through DST changes, a 2:30 AM ferry would sail an hour after 1:30 AM, regardless of what local time was then, and so on. The time between the end of Saturday's sailing day and the start of Sunday's sailing day varies due to the time change however. But we don't have 24 hour ferry coverage, so it's easy to keep the schedule aligned without adjusting service.
What really bothers me is that the tools we use (databases, time data libraries) have really poor support for wall time. Most of the times, there is no support at all.
That's why people go and store times in UTC (with a separated time zone), and then everything breaks because some tool always expects non-tz data to be on the local tz, or the original system converts tzs behind the scene, or whatever.
> There’s really no such thing as exact time. The closest is UT1, and it’s not really practical for normal applications.
I would have defined TAI (International Atomic Time) as "exact time". Of course with it being an average of clocks it only really exists in hindsight and isn't perfect either, but if you only need accuracy on the order of hundreds of nanoseconds and are on earth's surface it's very practical and easy to get anywhere with a GPS receiver.
And UTC is only slightly more messy, with its offset from TAI to keep it in sync with unpredictable UT1.
Right. It's crazy that we welded civil time to UT1 rather than TAI. TAI is ours, we made it, here on this rock we're all stuck on. Because time is necessarily relative to your position that's the best we could hope for.
The astronomers are welcome to UT1, it's useful for them and I have no problem with that, but it's crazy to keep tweaking UTC to match UT1 rather than just letting it plod along mechanically forever with TAI. Abolish leap seconds.
This will create problems in the long term, because the delta-T due to the deceleration of earth’s rotation due to tidal friction increases quadratically [1]. Not saying it wouldn’t be practical, but that’s the debate, along with astronomer’s needs, satellite navigation etc. You can’t just ignore the relationship with earth’s rotation.
> You can’t just ignore the relationship with earth’s rotation.
Doing so is, in fact, the point of abolishing leap seconds.
Imagine that we have leap seconds, and the very worst case conceivable happens for an entire century. 100 summer leap seconds and 100 winter leap seconds, that's a total of just over three minutes by 2122.
Can we tolerate such a disruption to the lives of ordinary people? Try asking anybody who lives in a place with Daylight Saving or "Summer" Time. They have twenty times more disruption and that happens twice every single year.
Right, but (a) that only defers the problem to the future (centuries), which will have ever faster increasing delta-T, and (b) in the short-term you'd have to change all the existing infrastructure for astronomy and satellites which rely on the broadcast (by NTP or whatever) clock signals to be UTC. There's too much that could break; therefore there's a preference for "never change a running system". Of course, there's disagreement on how well the current system "runs". But it's not as simple as saying "oh, let's just drop leap seconds and everything will be fine".
An increasing delta-T creates more problems for UTC than TAI. UTC can continue to be used for user-facing applications, or ones directly related to e.g. astronomy, but for most unstances of simple timing and time-keeping TAI is what you actually want. Remote systems can gradually be transitioned to TAI clock signals, and not have to bother with leap seconds ever again.
Issue with that is borders move, and with that time zones. So you may be in the “Europe/London” time zone now. But the north of England could annex itself and create a new time zone “Europe/Manchester”…
In the great schism of England, the meeting organizer fled to central Europe, and one of the primary stakeholders died. The meeting had to be rescheduled.
Time zone identifiers are created once an area stops matching the previously best-matched identifier.
Like right now maybe all of France uses Europe/Paris, but if southern France decides to secede and change their timezone by an hour, they would all have to re-configure from Europe/Paris to Europe/Toulouse. The existence of a bunch of currently seemingly-redundant time zone identifiers are a testament to the history of this.
So the only 100% future-proof solution is geographical coordinates that can be looked up in a continually-updated database.
So, what do you do for a remote meeting with participants in different places? You’d have to decide on one location. Specifying Paris when you are really in Toulouse isn’t much different. Yes, your meeting is now bound to Paris time, but at least it remains consistent with that.
Furthermore, having had to work with call routing based on geolocation, I can tell you that geo coordinates aren’t always enough, due to inaccurate zip code and area code divisions and other factors.
I wander if we need a new version of ISO8601 which includes the TZ location rather than just an offset. Something like:
2022-03-13T07:21:39@Europe/London
I suppose that’s also problematic if the location changes to a different time zone due to boundary changes. Maybe we need a coordinate (and planetary body) system:
2022-03-13T07:21:39@Earth/51.5055853,-0.1014699
We need that full frame of reference to be sure we will be correct in future.
I think tzdata timezone locations (like Europe/London) are fully sufficient, since they are tracking the clock everyone uses in that location. If the boundaries change, then so will the clock for everyone in that boundary, so the time probably won't be right regardless.
I think coordinates are not completely right, since the time context is mostly political/cultural, not geographic. I don't have examples but I can easily imagine a border region specifying some event in a time that is referencing the region across the border.
> I think coordinates are not completely right, since the time context is mostly political/cultural, not geographic.
My point was that geopolitical boundaries change and so what the “major” city near by for reference is now, may be wrong in the future. Coordinates ensure it’s completely future proof from boundary changes and correct to an exact location.
There will always be outliers. If you're scheduling a meeting in a small town on the border of 2 countries that honor different time-zones, then you're probably just going to have to pick one to be the source of truth, and hope it's the other side causing the disruption (changing their clocks) and they'll pay for the administrative overhead.
Well, there's also continental drift :-) honestly everything we do in IT is approximation - some of them are worth accounting for, many are not - at least for most of us.
Sometimes local date-time (with optionally a location) is better. If I tag a photo as Christmas morning and upload it to a global photosharing site, I'd expect something that says when I took it in my local time and optionally location. I don't care about UTC or the photo's timestamp in the viewers timezone. If you want to make it complicated store UTC + offset (not timezone).
Yeah, human time is much more complicated than the strict linear clock. For certain past periods the former can generally be destructively mapped onto the latter. But as with a lot of things, programmers tend to confuse human intent with the post-intent "objective" record of the thing. Which I get, as the latter is much easier to deal with.
Yeah, also you can't compare UTC times without an almanac generated by a random natural process which means even without the cultural/political problem UTC is still by definition uncertain for future dates. AST is arguably better for this but doesn't get around the political problem.
I probably should read into details, but I don't get it from the summary at least.
How would future time zone change or DST change affect it at all? Is it about your pre-defined time would no longer be aligned with a nice "local time" like 0:00?
Because things would still happen at the exactly same "moment" (absolute time or time delta from now) in UTC regardless any TZ change isn't it?
The difference is something like storing your comment was made at 2022-03-13+10:40. Which the time zone has been set and will always be the same at as it's in the past vs "set an alarm for 2023-03-13+6:00" I want the to happen at 6:00am whether or not my country continues to observe day light saving time.
You have a timestamp in the future for example the time when a session at a conference starts. If to want to store this as UTC you will have to convert it from the local timezone of the conference location. You can always go back from the UTC timestamp to the local timestamp by performing the reverse conversion EXCEPT if the conversion involves daylight savings and the rule changes between now and the event in the future. While rare, daylight savings rules sometimes do change. I guess even local timezone offsets might change once in a while. When this happens the conversion back from UTC to the local time will be wrong unless you keep track of more than the UTC timestamp.
I've said for ages that programmers only need two subjects to learn. Date and time programming and debugging. The first one leads naturally to the second one.
The rules I follow:
1. If you're recording an event in the past, store it in UTC.
2. If you need to record the local time for something either in the past or the future, store it as the local time as it was (for the past), or the local time as it will be (for the future), plus a timezone (not an offset).
3. If you're displaying a timestamp, a) if you can, display UTC, b) if needed, display in local time using current conversion rules
4. If you are getting user input, a) if you can, get it in UTC, b) if not, get it using a timezone (not an offset)
5. If you are storing a date, then that is not a timestamp. A date is a period, so either just store the date without a timezone, or, if it's a date period in a local area, store the start and end timestamps with their timezones.
> 1. If you're recording an event in the past, store it in UTC.
Although it is unlikely for a timezone change to be issued after its begin date, it is likely for a timezone change to be entered into a database after its begin date. If you preserve the original local time you can always convert to UTC accurately.
> 1. If you're recording an event in the past, store it in UTC.
Why not epoch? An integer always means epoch. A human readable timestamp, unless it's exactly ISO8601, is pretty much always ambiguous.
> 2. [...] plus a timezone (not an offset).
Why not offset? ISO8601 is a good standard.
And it takes a very special use case to care about leap seconds. For logging "leap smear" is clearly a better way to solve it, and for time measurements you should always use a monotonic clock, not wall clock.
Oh, and with timezone I assume you mean something like "Europe/New_York", not ambiguous like "EST".
> A date is a period, so either just store the date without a timezone
Of course some dates don't exist in some timezones (and I mean more modern times than gregorian/julian). So you can't count days between two dates without knowing the timezone.
In my opinion there are only two ways to store timestamps that are actually correct: epoch, or iso8601. Everything else will eventually lead to data corruption.
An offset would have exactly the same issues as UTC. You need the time zone location to correct for future changes. Knowing it’s UTC+2 doesn’t help with that.
Ultimately it comes down to intent, either relative to space time, or the socially excepted time in a particular location in the future. A time zone represented as a location/city name is the best we currently have, but even can can be wrong if the time zone boundary’s change.
Where do you anchor your future times. Is the meeting literally two months, four days, three hours, 32 minutes, and 14 seconds from now, or is it two motnhs, four days from now, when a local clock in New York is 15:30?
I was thinking international meetings where "your country changed: you move", which in retrospect is not realistic.
Which epoch? There are many, it’ll change depending on your system.
Also what unit? Seconds, milliseconds, nanoseconds. That will also change depending on your system.
If someone is saying store UTC then they 100% mean store it in ISO8601 or an equivalent standard.
> Why not offset? ISO8601 is a good standard.
Because the offset for timezones change (as is explained in the article). If you just store an offset, then you have no way of knowing which timezone the user meant, and thus you can’t compensate for changes in the timezone definition (again, the primary thrust of the article).
> In my opinion there are only two ways to store timestamps that are actually correct: epoch, or iso8601. Everything else will eventually lead to data corruption.
The entire article is describing why this approach isn’t enough.
> Which epoch? There are many, it’ll change depending on your system.
That's missing the point. That's like arguing if DD/MM/YYYY or YYYY-MM-DD is the format.
That's not where the hard part of programming with time is.
> Also what unit? Seconds, milliseconds, nanoseconds.
Just implementation details that don't affect anything.
> If you just store an offset, then you have no way of knowing which timezone the user meant
True. But it depends what you mean. If you scheduled a world wide meeting then you probably want offset. If the meeting is defined as "five minutes past noon in New York" then UTC and offset be damned.
So it depends. A cross-timezone event needs to live somewhere. It can live as offset from UTC, or physical location, but it all depends what the intention is.
> The entire article is describing why this approach isn’t enough.
> That's missing the point. That's like arguing if DD/MM/YYYY or YYYY-MM-DD is the format.
Not really. We’re talking about unambiguous storage and transmission of date time. Epoch has the big problem that it only makes sense on the system that generated. By itself an epoch doesn’t contain enough data to decode it correctly, and be certain your interpretation is correct. With a date-time format like ISO 8601 you need no further information to decode the time correctly to a human relevant time reference. Even if you don’t know the exact format used, it’s usually possible to figure it out if your have enough examples, the same isn’t true of epoch.
> Just implementation details that don't affect anything
Hahaha, oh god. Those entire article and discussion is all about implementation detail of date time storage, and why it does effect things. For the epoch unit it matters is your want two systems to communicate with each other. It’s a total pain in the arse working with epoch because every platform and language handles epoch times subtly different. Units in particular and really mess you up, especially when multiple systems have different levels of precision.
> True. But it depends what you mean. If you scheduled a world wide meeting then you probably want offset. If the meeting is defined as "five minutes past noon in New York" then UTC and offset be damned.
Every meeting involving humans involves a physical location (the humans have to at least physically occupy the space in front of cameras) the same . So you need pick somewhere, the host location would be the obvious choice.
> Like I said, you still need to be careful.
Great, so your contribution to this topic is to state the obvious, then say be careful, edge cases exist. How does that add to a conversation about an article that’s actually exploring those edge cases?
Offsets can change. Turkey switched from GMT+2 with DTS to GMT+3 without DTS a couple of years ago. If you were to store +2, you wouldn't be able to reconstruct the current date from it, while having Europe/Istanbul would give you either +2 or +3 depending on the revision.
So a better option is to store UTC + timezone + revision (i.e. timezone as it was in 2014 to figure out the offset later)
Indeed. So when the Turkey/France meeting moves, what then?
But I get you. You want the point in time to stay "the same" in the local "common sense" time, whereas I was thinking international cooperation in which case anchoring it in Turkey could hurt more than help.
> So a better option is to store UTC + timezone + revision
Well that means that you've decided on an exact number of seconds from now, no matter how timezones change. So what's the point?
Why not just have UTC+offset, then? timezone+revision is exactly equal to offset.
> Why not epoch? An integer always means epoch. A human readable timestamp, unless it's exactly ISO8601, is pretty much always ambiguous.
Because in practice, the only ubiquitous APIs for getting time since an epoch don't actually model an elapsed time standard. The odd one out is POSIX time.
Wall clock time should never be used to measure time when these seconds matter. Like I said in grandparent comment:
> for time measurements you should always use a monotonic clock, not wall clock.
POSIX time isn't a time elapsed standard. Yes. But neither is iso8601, timestamp+timezone, everyday speak time "tomorrow at noon", or any of these other options.
So I'm not sure what your point is, other than to say that a hammer is not a screwdriver.
With epoch, from whose perspective are units counted? Mine? Someone in Greenwich? Still needs a timezone so determine when something did or will happen.
> With epoch, from whose perspective are units counted? Mine? Someone in Greenwich?
That's the great part of about (at least UNIX) epochs. It's the same so it doesn't matter.
> Still needs a timezone so determine when something did or will happen.
Nope, it does not. That's exactly why it's better. At least for describing exact timestamps, like "when an HTTP request came in". It's less good at describing "no matter what changes are made in political circles, I want to represent noon in this native american tribe in Arkansas today three years from now.
I agree, storing with an offset is better here, e.g. prefer `2022-03-13T08:00-04:00` over `2022-03-13T12:00Z` when the regional time zone of the event location was UTC-4 when the event occurred.
Why? As instants they are both equivalent, but one carries additional information, namely the time that was on the wall where and when the event happened.
Fixed offsets will never change, so this really is a frozen snapshot
Then you run into all the problems in the article. Out of date time zones because they changed. Bad initial conversions because of time zone mismatches. Etc.
Not when it's an event in the past. When the event in the past you know which TZ rules to apply for conversion. The issues arise for the future because you don't know what change may occur.
I’ve had bugs with number 5 when I wanted to encode historical dates with a pre-Gregorian calendar. Even timezones on their own don’t help when you need to account for differing calendars.
So many issues you can run into with time! One that really annoys me is the Health app on iPhone, where it records step count and other data, and does it well if you stay in the same timezone.
But when you change timezone, the data is shown in the zone you’re in now (I assume it’s stored in UTC and then converted to the current timezone). Which is technically correct, but then looking back, it looks like I was doing a whole lot of walking from 9pm to 11am for a few weeks while I was on a trip (it was actually just a regular daily schedule), which then messes up the daily average step count, because the day boundaries aren’t what they actually were there.
I’d really rather have an option to show it in the timezone that I was in at the time, and then on the daily resolution graphs mark where the time zone changed (with the hours in the difference either skipped or repeated depending on which way).
Here is how I deal with the Time in the systems I design:
1. Always store time in UTC
2. If the time is tied to a fixed place, like start of a Football match, store the timezone of the place in another column, for example, Asia/Kolkata, America/New_York, etc
3. Always store user's timezone as part of their preferences. Helps in use cases like send the reminder to the user at 8:00AM in their timezone.
4. All APIs always return time in UTC and user's or the places timezone in the output.
5. It's the job of the frontend to convert the UTC time to proper timezone and display it to the user.
Never had a use case I couldn't solve with these rules.
I believe this is not correct for future scheduled events for humans. When a human sets an event for 8AM on some day 3 years in the future (maybe through a recurring weekly event), what they mean is "8AM local time in that jurisdiction". When a law passes that changes the daylight saving time amount or DST switch date or just sets the timezone to be different or the jurisdiction is merged into another or whatever, humans expect future scheduled events to adjust accordingly. Imagine a clock tower in the town square. If something political happens, it will be adjusted, and a meeting set to 8AM local time should occur according to that (possibly virtual) clock tower clock.
That is why logs (moment of time in the past) should be stored in UTC (possibly with local or user's timezone name for UI), timers (short offsets into the future from a moment of time in the past) should be stored in UTC like logs, and future scheduled events between humans should be stored in local time with the name of the timezone. The name of a timezone is never the offset, it is always the jurisdiction that can change the rules and be merged into another, etc.
When answering the question "what is the UTC time of a future scheduled appointment?", which is equivalent to "does it intersect a future scheduled appointment set in a different timezone on the same or adjacent dates?", which is also equivalent to the question "how many seconds in the future is this from right now?", you must assume the timezone rules set by politicians will not change between now and the appointment time. If the appointment time is soon (e.g. under 3 months), you indeed can assume this, because legislating an unexpected timezone change that will take effect "soon" is stupid and will usually not be done. But for a scheduled appointment a year from now? You honestly don't know. You can guess, but you must assume this can change by a TZ database update and act accordingly. If you have converted from localtime to UTC when the appointment was created, it is difficult to adjust correctly when TZ db is updated. If you translate to UTC only "just in time" when rendering or detecting collisions, everything will work correctly.
Actually, logs should be Unix timestamps in my opinion. It is a record of an event that occurred at a specific instant, regardless of how the user wants to see it formatted.
In most jurisdictions the DST rules and timezone offset change rarely. In some jurisdictions, they change more frequently. Over the whole world, they change often (dozens of times per year).
Worldwide there are several each year IIRC. There are a lot of small regional timezones you have to account for - at least two Native reservations have their own timezones, each offset by 15 minutes from the timezone of their surrounding region I believe, or at least they did a decade ago when I worked with all this.
Also political changes often bring TZ changes, most obviously when larger countries split into smaller ones. But changing trade patterns can lead to TZ and DST changes, such as when Samoa skipped forward 24 hours, skipping Dec 30 2011 completely, putting them in the same timezone as Australia and New Zealand.
That just means your data should have more context than Xinjiang. This is like saying the continent of Europe doesnt use the same tz, since the geo data logged is only in "continent" units
I am currently working on a product which followed what you mentioned above. The problem we ran into is what the author of the article highlights. You cannot convert stored UTC times to local timezones reliably wihtout additional information.
For example, for a conference whose starting time is stored as 14:00 Hrs UTC (0900 Hrs America/New_York) before DST, will suddenly start at 10:00 Hrs America/New_York after DST. Coincidentally, we just redid our implementation to follow Option 3 before New York's DST kicks in and it works as expected so far.
Please correct me if my understanding of your method is incorrect.
The standard date time library of your programming language should be able to handle the DST changes when you specify the timezone changes when you store it as "America/New_York". Java (which I use primarily) Date Time lib handles this for you.
Those are invalid times. Your date time validation should report an error to the user. For recurring scheduled reminders, I would remind them at 1:00AM along with explanation why they are getting that reminder early. There is no way to automatically handle this to my knowledge. So it's an edge case one must handle.
You could argue that the case of the clock jumping forward is an invalid time. But the case of the clock jumping back is in fact a valid time, even if it is non-unique.
How would you handle reporting, where you would for example want to aggregate by hour of day, localized by time zone?
To me it seems impossible to do that kind of reporting, unless the backend is able to use the time zone to accurately convert the UTC time to hour-of-day, including DST shenanigans and whatnot.
I agree that it's not an edge case for devs, but I do think it's an edge case for POs in the EU.
My friend was even struggling with conversations like "so what if the user hits save exactly when data coverage drops". Not an edge case for techies, but likely an edge case for POs.
> My friend was even struggling with conversations like "so what if the user hits save exactly when data coverage drops".
My favourite PO response to a question like that is "Oh, I suppose that could happen, but it would be very rare so don't worry about that", at which point you need to re-iterate that even if rare, we still have to do decide what should happen, even if it's just showing an error to try again.
> "Never ask the product owner what to do in an edge case."
Could you expand on that? I've found it best to keep PMs informed even about edge cases, but I'm curious to know what your friend meant?
I usually don't ask, but say something like "We store time as UTC for future events, but apparently this is not the best practice because when timezone rules change, the local time is not updated. Setting up that refactor will be medium complexity. I recommend we take the time to fix it now, so that we will not have awkward moments for event organizers as people show up an hour early."
The PM never has a good answer, and never has any idea on the problems his answer will cause. If he had an answer, you would receive the situation as a requirement, not as an edge case.
So, if you go ask him, you could as well ask some random strange on the street and it would be as useful.
Some times the end user knows the answer, but very often their opinion is equally useless.
Hmm. Perhaps I've had good luck with PMs. I've actually generally been impressed by some of the insights I've received from PMs about how to prioritize business interests. Most recently, there was a question of what to do in the case that data coming from the client is inconsistent with what is on the server. The proposal was to throw an error. PM said that, whatever we do, it should not interrupt or block communication, which answer simplified the remaining discussion.
Well, notice that the answer came in the form of a requirement. The PM just forgot to tell you that one up-front.
Also, your conversation doesn't reflect you asking him what to do. You made a proposal, and he clarified the requirements. He didn't tell you what to do.
I wasn't very clear on my post, but the way you did is the one that works. You go to the PM with an answer, and he says if it's acceptable (or why not). You know how to weight the technical details, he doesn't, he knows about the business details. So your proposal should be reasonable, but there isn't any expectation about any one he comes up with.
These conversations are part of the PM being considered one of the team, being part of the daily stand-up and such. If there's tension between the devs and the PM, then the incentive structure is off, somehow.
He was referring to cases like the PM being a highly skilled person with clinical trials. The PM already knows a lot about regulations, customers, what problems can be "automated" away with software, etc. The PM is likely trained in life science or medicine, and never coded themselves.
One might be tempted to ask the PM to decide on the following: What should happen if the user clicks "save" but the network went down? Shall we retry a few times? How many times? With what backoff? Or shall the application be offline-first? How should potential conflicts between the server data and the client data be resolved?
As you can imagine, the PM has already enough on their plate, so, in my experience, it's best to not overwhelm the PM with such questions and take a decision based on "best guess" on what would maximize customer success.
Another anecdote: A PM had to prioritize "integrate with Lufthansa" and "migrate to Java 8". Clearly the latter issue is at the wrong level of detail for a PM.
The problem here is a confusion between 2 different data types and attempting to use one type instead of the other correct type, which is a trivial programming mistake.
A future time moment may be a true time, known exactly, which must be stored as an UTC time.
Otherwise, it may be an event defined in the official time of some place. This must be a distinct data type, because as mentioned in the article, the real time that corresponds to it cannot be known in advance, because the legislation defining the official time may change before the event takes place.
Obviously this second data type must store the complete information that determines the event time, i.e. both the official local time and the place, so that, a short time before the event, the correct conversion to real time, i.e. UTC, can be done, and then the different local times for each participant may be computed, to be displayed in their schedules.
Of course, for displaying, an estimated UTC time for the event can be computed since the beginning, but it should be updated when the event approaches.
The only problem here is when you fail to recognize that the "time" for a future event that is scheduled in local time is not a time, and you attempt to store it using the wrong data type.
Also one must keep in mind that for this second data type, for future events scheduled in local time, no arithmetic operations can be defined, like the difference between two times. The only operation that can be defined for this data type is conversion to and from UTC.
I am a proponent of just standardising on UTC everywhere and making the humans deal with it at this point. No DST, no time zones. If you're in the east coast US then you have breakfast at 11:00 UTC. It's just a number!
This realisation came from trying to work out how the hell to write an international scheduling system where the humans couldn't even work out how to schedule a meeting across three time zones.
My main gripe with this article that I've thought about for years now after initially reading it is that the example ignores reality.
In the current system, how do I know what time it is in Australia? Two ways: I either look it up, or I know how many hours it is relative to me. Living in Germany now, I know my parents are -8 and some friends are -9. I still always have to do the math. This is still apriori knowledge necessary that the author just skims over
Compare this to if there were no time zones. I would still need to know when their solar noon was. The amount of information I would need to retain is the exact same. Looking it up would also be just the same, except the lookup would tell us the sun position instead of the local time.
Don't get me wrong, I mostly agree with the article's point. The example just never felt adequate.
Not a great article. It's basically a giant straw man perched on top of zoneinfo which is a massive database showing exactly what a shit show we got ourselves into as a species with the conclusion that it's going to hurt to fix it. Yeah it will. But it's death by a thousand paper cuts at the moment. I'm not sure we should even have a zoneinfo database or have that level of complexity to manage by chance.
Case in point: Europe abolished DST absolutely fine to decrease complexity.
But this is not a technical concern. This is a human concern.
Reality is watching one dude in Australia meet his friend an hour early by accident because they couldn't even work it out with local domain knowledge. This happens all the time. The amount of Zoom meetings I get into because no one can work it out or they had their Outlook set up in the wrong time zone because they were travelling is insane.
The only viable solution to these issues is a common, unified zone for scheduling which probably should leverage UTC.
Not everywhere. It works only on the planet surface. With the expansion of the humanity, even remote satellite orbits will benefit from the new time representation standard. Not to say about Moon, Mars, various interplanetary objects. We need new standard and very soon.
It's hard enough to agree what a "meeting next Friday" means. Is "next Friday" three days from now, or ten days from now? Everyone knows, and feels strongly, but people don't agree.
This would now introduce ambiguities in "see you tomorrow for dinner". I thought you said tonight? Yes, tonight is tomorrow. But we can start today and party until dinnertime tomorrow.
But you'll lose consistency of everyday human communication. Phrases like 'I slept until 11am', or 'I obviously don't want to schedule a meeting before 8am' will become meaningless. And anyone who travels far away will have to relearn what time they want to wake up to stay in tune with the local sun day.
The situation the author is considering, is a future event in some geographical location in which the rules for the timezone change.
In my honest opinion this is a non-solvable problem. So yes, storing in UTC or storing in local time is not a silver bullet. Because, depending on the event, there is a different wherewolve.
If the event is e.g. the opening hours of a store, chances are that local time works. Indeed: if e.g. the Netherlands change the time zone rules, the local shop may continue to use the 09:00 hour local time as the time to opening the shops. But if it is e.g. the time that the last ‘window’ of delivering a package to the delivery company such that it will be on time for delivery to neighbouring country (say) Germany, then this will depend on the (changes to the rules of the) time zone in that country. Which may be different then the (changes to the rules of the) local time zone in which the event occurs.
So, when planning ahead, one takes into account the knowledge of that time to make a assumption of when the event will take place. If the timezone rules change, one should reexamine which assumption works best.
If the event is e.g. the time the moon will be eclipsed, UTC works best (without fails). If it is some local event not depended on anything outside the area in which the time zone rules apply, local time probably works best.
But really, really knowing for certain? I would rather store (future) time in UTC and then consider whether changing time zone rules should or should not apply, as opposed to designing a system that could do this correctly in any situation…
Or... we could store Terrestrial Time and treat leap seconds as part of the timezone data.
This isn't a complete panacea, there would still be clock drift and updates could still cause time to be non-monotonic. But a timestamp would have a consistent meaning, and the TT->UTC conversion could be taken care of as just another step in localization.
Why on Earth was the decision taken in the first place to have NTP operate off UTC?
Not very often that system timestamp and UTC related posts sometimes appear on HN front page
I think 3 days ago I was checking on it again, because I was wondering how websites put timestamp in their published RSS/atom feed
https://hackerdaily.io RSS feed made me confused, to select timezone for the RSS feed as they want to think "Yesterday" means different thing for different timezone, not sure why, but I don't want to confused myself thinking how timezone and "Yesterday" work together
HN is using UTC, I realized that by hovering the mouse pointer on the "2 hours ago"
Again, if people don't want to use UTC for future events it's their choice. Personally I would advocate to use UTC and manually calculate the local datetime for future event, even if local timezone changes
You know, I wonder how watches work in this case, how do you alter the time on your watches according to the DST changes? The mechanical engineer of the watch takes this bug into account?
Interestingly, technically the time of the event changed with the hypothetical TZ law changes. The event was going to be n hours from now, but with the TZ law change, it got moved to be n+1 hours away.
If the convention hall could be booked hourly and had different events every hour, suddenly a slot would've opened up, at the 2 AM on the DST-change Sunday, the hour which would've not existed if the daylight savings law stayed the same...
> If the convention hall could be booked hourly and had different events every hour, suddenly a slot would've opened up, at the 2 AM on the DST-change Sunday, the hour which would've not existed if the daylight savings law stayed the same...
I did not think of this before...interesting....leave these problems to politicians, software engineers and end users to accommodate, I just want to be a naive programmer
The reason for a timezone specific RSS feed is because, like the site itself, the RSS feed only updates once a day at midnight. And _when_ the day changes depends on the timezone. Therefore we created different RSS feeds, one for each time zone.
The tricky part is the conversion process from the server to the client. As JavaScript doesn't have any way to make dates with specific timezones other than the local timezone, you have to do some things to make up for it.
In my last project, I convert the time using the timezone information I have stored by doing t at time zone x in postgres, so that it shows the time at that location, but then when it gets to the client, the actual Date object underneath is in the local timezone, even though it is displaying the time in a different timezone. It doesn't really matter because it is just for display purposes essentially.
When I send the time back to the server, I strip the timezone information from the date object and add in my own timezone information to a newly created string in a postgres-specific format and send it back.
Ok here's a little bit of a guilty confession.... we _love_ MySQL's `timestamp null` type. MySQL certainly has some areas that aren't pretty, but we use the hell of out this.
The `timestamp null` time automatically adjusts for the connecting user's timezone. Internally, IIRC it stores the value as UTC, but then converts it on the fly for wherever you're sitting. Why is this awesome? Well, because not everyone is a programmer or expert in time zone, daylight savings, understands UTC, etc. When our non-tech CEO wants to look at a report, he expects it to be relative to the time shown on his watch, which is exactly what this type does.
Is it perfect? No. But does it solve a simple problem eloquently and prevent non-technical users from making bone-headed mistakes? yep.
That sounds horrible, unless the CEO is writing and executing SQL statements on his own, directly against the database.
If I'm sitting in Seattle VPN'd into a network that geolocates to South Carolina, and I send an HTTP request that terminates on a front-end box in Texas and sends a SQL query to a database in Ireland, what timezone will be used in the result?
And why is this better than just making the same calculation in the UI layer?
In that scenario, your browser communicates the timezone to the box in Texas, which sets the DB connection timezone to Seattle and the Irish database does the conversion for you. The VPN doesn't really factor in.
Example: An order comes in 22:00 UTC, he's sitting in Seattle talking to a parts supplier. Said supplier want to know when the last order came in, so CEO pulls a sales report. It appears on the report as 3pm. This is intuitive and works every single time.
Ironically, that scenario you describe would be quite complex to implement with a ui tweak. What if your data entry is a webapp, the CEO is using Metabase, and your data analyst is using Tableau? You know have to implement timezones 'correctly' in three different pieces of software, sometimes with software you didn't write.
You don’t need to store timestamps “in UTC”. You just need to store the time zone offset. That extra few bytes of precision representing where the timestamp sender is makes the sun location absolute.
Some databases let you input values like “2022-03-13T08:33:26-06:00” and then display them in UTC or your local time zone or whatever time zone you configured. BugQuery at one time was toying with the idea of taking a few additional bytes of information to represent the preferred display timezone of the timestamp, but I have no idea if that ever made it to beta.
For a vast majority of use cases, UTC storage is sufficient. For everything else, you need both the UTC timestamp and some other piece of knowledge depending on the scenario at hand. In many cases, its as simple as a User.UtcOffsetMinutes fact. In other more paranoid scenarios, you may be inclined to store the original timezone/offset in which the UTC fact was originally valid.
Regardless of the scenario, there is no situation where I would want to store a non-UTC timestamp. Everything is UTC with some optional extras.
A few companies ago, we were (essentially) a TSDB marketed toward network performance in particular. We had some large telecom companies as customers. We stored all the time-series in UTC, but the analytics data shifted by an hour twice a year. This caused a whole lot of false alarms for about a week following both time changes.
This was a solid ten years ago. I don't think the problem was ever solved in the general case. We wrote them a script that support would manually run and shift all the analytics forward or back by an hour.
The answer is simple, store the time July 10th 2022 7am utc on march 27th. If you are using a countdown timer, reload the zoneinfo database everytime it changes, so that the countdown clock recalculates remaining time correctly. This article is about the difficulty of internationalizing countdowns, not the problem of storing times as UTC.
... for the purpose of this blog post I’ll assume that each member state has to decide whether they will “spring forward” one last time on March 28th 2021, then staying in permanent “summer time”, or “fall back” one last time on October 31st 2021, then staying in permanent “winter time”.
Relativity is even trickier because you need to account for the total maneuvering since you left your reference frame. Realistically you'd log ship time and datum points when you sync'd clocks with other reference frames, but establishing a continuous relationship is probably close to impossible.
Seems like we agree it's going to take some serious equipment to generate the sync data between reference frames. But between hackers, isn't data integrity worth it?
If you have a time for an event, then there's an encoded assumption about what you're trying to determine. Very few - I would say zero - events we want to know when they happened but don't care about where in anyway.
For example when planning meeting times, the reason local time comes up is ultimately some determinant about when it's going to be day or night - that's the actual metric that meeting planning generally tries to take into account.
Strictly speaking I would say this also applies for storing data about machine to machine events which might be normally considered to be purely sequence based. If we're storing times to determine processing order for example, then there'd be value in storing the lat/lon of the machines generating them, because it's an extra datum which can resolve expected ordering (i.e. based back-calculating latency windows).
So if I (in Australia) have a meeting with people in Thailand and Slovakia over Google meet, which is absolutely an "event we want to know when they happened but don't care about where in anyway", the local time(s) are all important when scheduling it for the future, but as a record of the event, it happened in all 3 locations simultaneously.
Sure, but if you're trying to derive any useful information from that record as opposed to some CYA audit then you really want to have some local situational context for it.
Personally when I'm arranging a meeting I want to know the local cultural context, and I want to know the relative times I'm asking for things there.
Getting stuff done 2 hours later may or may not be possible based on the local time of day, calendar day, and time of year.
For past events, I always beg people to somehow explicitly include the timezone in an encoding or storage, even if it's just UTC. i.e. never be tz naive! A tz naive PG column is a mess waiting to happen.
Semi-serious question: At what speed does time propagate away from UTC? Do systems take into account relativistic effects when synchronizing across the earth, given that “Now” does not propagate instantaneously?
This reminds me that Atlassian still doesn't support per-user date time formatting for some of their products (I use iso8601 where possible to avoid confusion in multinational settings). ಠ︵ಠ
good article! I have been using their Option 3: Store what the clock should say on the wall at either the physical location or the organizers desk if it is a call. Then store the timezone string for that location and calc the UTC from that to give local start times for other users.
Some apps store that UTC in the row and get re-calc’d as needed.
With other apps we create views or materialized views and update rows for UTC to give a single source for sorting, etc.
Storing the rules version is an improvement I will look into!
TL;dr: an instant in time and a future time based on a wall clock in a location aren't the same thing.
But in my view, it's okay to pretend they are the same thing and model them all as instants.
Even in the current climate of potentially large changes coming to the TZ database, I doubt there are many developers who will come out ahead trying to model this difference correctly vs manually handling any edge case misses. More likely, you'll have bugs in your implementation because time logic is awful.
I suppose it’s the difference between exact time and social/political time (there is probably a better term for that).