[Python-Dev] Store timestamps as decimal.Decimal objects

[Nick Coghlan]

On Wed, Feb 1, 2012 at 6:03 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> 2012/2/1 Nick Coghlan <ncoghlan at gmail.com>:
>> The secret to future-proofing such an API while only using integers
>> lies in making the decimal exponent part of the conversion function
>> signature:
>>
>>    def from_components(integer, fraction=0, exponent=-9):
>>        return Decimal(integer) + Decimal(fraction) * Decimal((0,
>> (1,), exponent))
>
> The fractional part is not necessary related to a power of 10. An
> earlier version of my patch used also powers of 10, but it didn't work
> (loose precision) for QueryPerformanceCounter() and was more complex
> than the new version. NTP timestamp uses a fraction of 2**32.
> QueryPerformanceCounter() (used by time.clock() on Windows) uses the
> CPU frequency.

If a callback protocol is used at all, there's no reason those details
need to be exposed to the callbacks. Just choose an appropriate
exponent based on the precision of the underlying API call.

> We may need more information when adding a new timestamp formats
> later. If we expose the "internal structure" used to compute any
> timestamp format, we cannot change the internal structure later
> without breaking (one more time) the API.

You're assuming we're ever going to want timestamps that are something
more than just a number. That's a *huge* leap (much bigger than
increasing the precision, which is the problem we're dealing with
now).

With arbitrary length integers available, "integer, fraction,
exponent" lets you express numbers to whatever precision you like,
just as decimal.Decimal does (more on that below).

> My patch is similar to your idea except that everything is done
> internally to not have to expose internal structures, and it doesn't
> touch decimal or datetime modules. It would be surprising to add a
> method related to timestamp to the Decimal class.

No, you wouldn't add a timestamp specific method to the Decimal class
- you'd add one that let you easily construct a decimal from a fixed
point representation (i.e. integer + fraction*10**exponent)

>> This strategy would have negligible performance impact
>
> There is no such performance issue: time.time() performance is exactly
> the same using my patch. Depending on the requested format, the
> performance may be better or worse. But even for Decimal, I think that
> the creation of Decimal is really "fast" (I should provide numbers
> :-)).

But this gets us to my final question. Given that Decimal supports
arbitrary precision, *why* increase the complexity of the underlying
API by supporting *other* output types? If you're not going to support
arbitrary callbacks, why not just have a "high precision" flag to
request Decimal instances and be done with it? datetime, timedelta and
so forth would be able to get everything they needed from the Decimal
value.

As I said in my last message, both a 3-tuple (integer, fraction,
exponent) based callback protocol effectively supporting arbitrary
output types and a boolean flag to request Decimal values make sense
to me and I could argue in favour of either of them. However, I don't
understand the value you see in this odd middle ground of "instead of
picking 1 arbitrary precision timestamp representation, whether an
integer triple or decimal.Decimal, we're going to offer a few
different ones and make you decide which one of them you actually want
every time you call the API". That's seriously ducking our
responsibilities as language developers - it's our job to make that
call, not each user's.

Given the way the discussion has gone, my preference is actually
shifting strongly towards just returning decimal.Decimal instances
when high precision timestamps are requested via a boolean flag. The
flag isn't pretty, but it works, and the extra flexibility of a "type"
parameter or a callback protocol doesn't really buy us anything once
we have an output type that supports arbitrary precision.

FWIW, I did a quick survey of what other languages seem to offer in
terms of high resolution time interfaces:

- Perl appears to have Time::HiRes (it seems to use floats in the API
though, so I'm not sure how that works in practice)
- C# (and the CLR) don't appear to care about POSIX and just offer 100
nanosecond resolution in their DateTime libraries
- Java appears to have System.nanoTime(), no idea what they do for
filesystem times

However, I don't know enough about how the APIs in those languages
work to do sensible searches. It doesn't appear to be a cleanly solved
problem anywhere, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia