Time Programming for Lawyers and Jurors

I often like to have streams of trials playing in the background when I am working, and the recent trial of interest was the trial of Karen Read, who was accused of murder. One issue in this case was a conflict between two timestamps logged by different apps on a potential suspect’s phone. Apple health had logged the person climbing flights of stairs at a given time, and at the same timestamp, a log from the app Waze indicated that the person was driving. This seems impossible, but these apps used different time sources on the phone.

I wrote recently about engineering for correctness. As software engineers, we have some idea of how correct and how incorrect our work is, but the public perception is that our work is 100% correct all the time. In fact, this is a legal presumption in some jurisdictions. When this piece of contradictory evidence came up, the lawyers, and even their hired experts, did not seem to know what to do about it. This was due to a misunderstanding in how computers tell time.

How Computers Tell Time

Computers (from smart fridges to phones to servers) all have an internal clock that runs computations. Part of this clock’s job is to run a time counter. This counter simply counts up, one increment per clock tick. This happens regardless of frequency. When combined with a reference, you can use this counter to tell time. However, this clock is not very precise. It is often only stable in its frequency to 100 parts per million, which sounds okay, but means gaining or losing about 8.5 seconds per day or about a minute per week compared to an accurate counter. It also has a frequency that depends on temperature and on electrical interference, so the frequency is somewhat variable (within ~100 ppm) over time.

Computers can also have a separate clock based on a quartz crystal, which runs a lot slower than the CPU clock and is somewhat more precise, but still not up to the standard we expect from computers. This clock nominally counts up $2^{15}$ times per second, similar to the quartz clock in a watch. The tick rate of this clock is more uniform and more stable, generally varying by 20-50 ppm or less, but not very granular: a lot of things on computers happen faster than that. Quartz clocks are also used to keep track of things in low-power sleep modes. Still, 20 ppm of drift is over 1 second per day.

It is up to the operating system to translate these timers into a useful wall clock. The simplest way to do this is to have the user set the time, and just track the counters (either from the internal clock or the quartz clock) as they count up, assuming that they run at their nominal frequency. This would be simple, but is imprecise, so computers only operate in a mode like this when they are not able to get a source of more accurate time.

Getting Accurate Time from the Internet

Almost every computer has an internet connection, and this is how phones and computers have clocks that are so accurate. The internet connection allows your computer to talk to other computers with more accurate clocks, and they specifically use internet-connected atomic clocks that companies and governments host as a public service. There is a standard protocol, called NTP, to talk to these time servers to get the current time. NTP uses a bit of trickery to remove communication delay from the timestamp you get, so by using NTP with these atomic clocks, your computer can get a very accurate idea of the exact time the NTP message was sent.

Your operating system does two things with the time it gets from a time server. First, it jumps the clock to the accurate time, and second, it adjusts its idea of how fast its internal clocks are running. This second process is called “phaselocking” and allows you to turn a combination of an inaccurate clock and an accurate reference into an accurate clock. The phaselocking algorithm first measures the difference between the reference from the time server and your computer’s internal time. If the internal time is ahead of the time server, we can reduce our estimate of the frequency of the computer’s clock, and if the internal time is behind, we increase it. Increases and decreases happen proportionally to the error. This process allows us to “discipline” the time counter to the reference from the time server, and also allows your clock to stay accurate as conditions change. This process happens roughly once per minute, so at 100 ppm of error, your clock will only be 6 milliseconds off before being corrected.

GPS receivers and mobile baseband modems can also get precise time from different sources, specifically the GPS satellite cluster (a group of flying atomic clocks) or your carrier’s mobile network. These systems use an internal time as part of their protocols, so they prefer to provide their own reference. In practice, since these are also driven by atomic clocks, the time you get from GPS and the time you get from NTP are going to be very close. Usually, these devices will keep track of their own time separate from your computer’s core clock, and have access for programmers to get their timestamps when desired. These time sources may be different than the device’s internal time because there is a different network linking them back to a different atomic clock, but won’t be far.

Clocks for Software

After all of this has happened, and we have an accurate time and an accurate measurement of our clock frequency, the operating system just has to present the clock to the user. It does this in two ways:

• A real-time clock (CLOCK_REALTIME) for you to use to get the current time
• A monotonic clock (CLOCK_MONOTONIC) for you to use to measure durations

The real-time clock behaves as you would expect. It displays the most accurate estimate of the current time. The monotonic clock, however, is a clock that only goes up. It completely ignores NTP time adjustments. It’s monotonic, so it only goes up.

This seems like a useless construct, but it is helpful if you want to measure durations, and can also be helpful when you want to create an ordering of events that corresponds to the order in which they happened in the real world. Since the real-time clock can jump, using it for durations means that sometimes you will have events whose end time is before they started, and some measurements will seem a lot longer than they should be. Similarly, databases with any time dependency would very much prefer if timestamps were in order, and the monotonic clock is the way to get that guarantee.

If a computer clock is used as evidence, a real-time clock, like the clock shown on the phone display, is the most authoritative source of time of an event. If a duration is in question, a monotonic clock is the authoritative source, and a real-time clock may be off.

On many unix-based systems (Apple is also unix-based), CLOCK_MONOTONIC starts from 0 or a random number on first boot and just counts up. If it were directly translated to a time, you would get nonsense. Some programmers try to get something that looks like real time from the monotonic clock by adding the boot time to the value of the monotonic clock or “correcting” it with a single check against real time. This is useful for things like database entries, where you may want to know approximately when bugs happen, but should not be considered an accurate time. Every time an NTP adjustment causes your clock to adjust, this “monotonic real-time clock” will diverge more and more from real time. Clock adjustments happen every minute, so some drift between the two clocks is inherent. However, the real-time clock is authoritative.

The Karen Read Log Entries

It appeared that either Waze or iOS did this to keep the monotonic clock “close enough” to real time. This isn’t a “bug” necessarily, as there are several engineering reasons to do so. However, this kind of clock has the flaw that its error is unbounded and grows as the phone or computer stays on. While the programmers who wrote the logging code almost certainly knew that—which is why they included two other timestamps in the log entry—the investigators and experts in the case appear to have been confused by this. One side treated the monotonic clock as authoritative of the time, and the other side’s counter was:

• “There are three clocks on an iPhone” (No, there are a lot more than that on a BSD-based operating system like iOS with both a GPS and a mobile radio, but also—why is there more than one clock? Why would the log entry have three of the computer’s clocks tagging it?)
• “and some of them aren’t accurate” (Yes, but why would that be the case? Can you elaborate further?)

These two statements are incomplete and less than persuasive. We are all attuned to the idea that what a computer or algorithm says is correct, so saying “the computer is wrong” with no explanation does not resonate well. You have to overcome this prior belief if you want to argue that a computer has done something wrong. Time is a great place to do that—it is one of the places where computers actually can have bad answers, at least when precision is required, and where it’s easy with some digging to show that the system of clocks on computers a marvel of engineering that needs some delicacy. Unfortunately, even the experts in the trial didn’t seem to know this.

Conclusions

As an engineer, this is a stark reminder of Hyrum’s Law:

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

Because the monotonic clock appears to be relatively close to the other clocks, it is treated as an authoritative source of the absolute time of an event. Usually, Hyrum’s Law applies to someones misuse of an API, but in this case, it is being applied to a murder trial. If you do not want lawyers misusing your API and treating that misuse as authoritative, design it so that they can’t misinterpret it.

It was also interesting to look into the power of overconfidence in—and overreliance on—the idea of expertise. Not once on the stand did any of the three computer experts say “I don’t know” when it came to their technical knowledge, and the tricky subject of time programming was not the limit of this ignorance. As a result, what should have been a clear-cut technical issue became a quagmire.

As for the trial, in the end, as of July 1, 2024, a deadlocked jury could not reach a conclusion in the case and a mistrial was declared. A new trial will start in a few months. Time will tell if this issue gets cleared up.