[Python-Dev] Store timestamps as decimal.Decimal objects

[Nick Coghlan]

On Wed, Feb 1, 2012 at 8:58 AM, Mark Shannon <mark at hotpy.org> wrote:
> Why not add a new function rather than modifying time.time()?
> (after all its just a timestamp, does it really need nanosecond precision?)
>
> For those who do want super-accuracy then add a new function
> time.picotime() (it could be nanotime but why not future proof it :) )
> which returns an int represent the number of picoseconds since the
> epoch. ints never loose precision and never overflow.

Because the problem is broader than that - it affects os.stat(), too,
along with a number of the other time module APIs that produce
timestamp values.

That's where Alexander's suggestion of a separate "hirestime" module
comes in - it would be based on the concept of *always* using a high
precision type in the API (probably decimal.Decimal()). Conceptually,
it's a very clean approach, and obviously has zero performance impact
on existing APIs, but the idea of adding
yet-another-time-related-module to the standard library is rather
questionable. Such an approach is also likely to lead to a lot of
duplicated code.

Victor's current approach, unfortunately, is a bit of a
"worst-of-both-worlds" approach. It couples the time and os modules to
various other currently unrelated modules (such as datetime and
decimal), but still doesn't provide a particularly extensible API
(whether indicated by flags or strings, each new supported output type
must be special cased in time and os).

Perhaps more fruitful would be to revisit the original idea from the
tracker of defining a conversion function protocol for timestamps
using some basic fixed point arithmetic. The objection to using a
conversion function that accepts a POSIX-style seconds+nanoseconds
timespec is that it isn't future-proof - what if at some point in the
future, nanonsecond resolution is considered inadequate?

The secret to future-proofing such an API while only using integers
lies in making the decimal exponent part of the conversion function
signature:

    def from_components(integer, fraction=0, exponent=-9):
        return Decimal(integer) + Decimal(fraction) * Decimal((0,
(1,), exponent))

    >>> from_components(100)
    Decimal('100.000000000')
    >>> from_components(100, 100)
    Decimal('100.000000100')
    >>> from_components(100, 100)
    Decimal('100.000000100')
    >>> from_components(100, 100, -12)
    Decimal('100.000000000100')

Such a protocol can easily be extended to any other type - the time
module could provide conversion functions for integers and float
objects (meaning results may have lower precision than the underlying
system calls), while the existing "fromtimestamp" APIs in datetime can
be updated to accept the new optional arguments (and perhaps an
appropriate class method added to timedelta, too). A class method
could also be added to the decimal module to construct instances from
integer components (as shown above), since that method of construction
isn't actually specific to timestamps.

With this approach, API usage might end up looking something like:

   >>> time.time()
   1328006975.681211
   >>> time.time(convert=time.as_float)
   1328006975.681211
   >>> time.time(convert=time.as_int)
   1328006979
   >>> time.time(convert=time.as_tuple)
   (1328006975, 681211, -9)
   >>> time.time(convert=decimal.Decimal.from_components)
   Decimal('1328006983.761119000')
   >>> time.time(convert=datetime.datetime.fromtimestamp)
   datetime.datetime(2012, 1, 31, 11, 49, 49, 409831)
   >>> time.time(convert=datetime.datetime.utcfromtimestamp)
   datetime.datetime(2012, 1, 31, 11, 49, 49, 409831)
   >>> time.time(convert=datetime.date.fromtimestamp)
   datetime.date(2012, 1, 31)
   >>> print(time.time(convert=datetime.timedelta.fromtimestamp))
   15370 days, 10:49:52.842116

This strategy would have negligible performance impact in already
supported cases (just an extra check to determine that no callback was
provided), and offer a very simple, yet fully general and
future-proof, integer based callback protocol when you want your
timestamps in a different format.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia