Re: [Python-Dev] proposal: add basic time type to the standard library
Jim Fulton wrote:
I would be willing to make the mxDateTime types subtypes of whatever Fredrik comes up with. The only requirement I have is that the binary footprint of the types needs to match todays layout of mxDateTime types since I need to maintain binary compatibility.
The binary footprint of your types, not the standard base class, right? I don't see a problem with that,
Fredrik's solution only provides an abstract base type with no additional parameters in the type object (only an interface definition on top of it) -- this would work nicely.
The other possibility would be adding a set of new types to mxDateTime which focus on low memory requirements rather than data roundtrip safety and speed.
What is data roundtrip safety?
Roundtrip safety means that e.g. if you take a COM date value from a ADO and create a DateTime object with it, you can be sure to get back the exact same value via the COMDate() method. The same is true for broken down values and, of course, the internal values .absdate and .abstime. This may not be too important for most applications, but it certainly is for database related ones, since rounding can cause e.g. 14:00:00.00 to become 13:59:59.99 and that's not what you want if you transfer data from one database to another.
I rarely do date-time arithmetic, but I often do date-time-part extraction. I think that mxDateTime is optimized for arithmetic, whereas, I'd prefer a type more focussed on extraction efficiency, and memory usage, and that effciently supports time zones. This is, of course, no knock on mxDateTime. I also want fast comparisons, which I presume mxDateTime provides.
DateTime objects use .abstime and .absdate for doing arithmetic since these provides the best accuracy. The most important broken down values are calculated once at creation time; a few others are done on-the-fly. I suppose that I could easily make a few calculation lazy to enhance speed; memory footprint would not change though. It's currently at 56 bytes per DateTime object and 36 bytes per DateTimeDelta object. To get similar accuracy in Python, you'd need a float and an integer per object, that's 16 bytes + 12 bytes == 28 bytes + malloc() overhead for the two and the wrapping instance which gives another 32 bytes (provided you store the two objects in slots)... >60 bytes per Python based date time object. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
[M.-A. Lemburg]
The other possibility would be adding a set of new types to mxDateTime which focus on low memory requirements rather than data roundtrip safety and speed.
[Jim Fulton]
What is data roundtrip safety?
[MAL]
Roundtrip safety means that e.g. if you take a COM date value from a ADO and create a DateTime object with it, you can be sure to get back the exact same value via the COMDate() method.
The same is true for broken down values and, of course, the internal values .absdate and .abstime.
I'm unsure whether Jim is aware of this, but if not he should be: the non-trivial time I spent over the last week repairing test failures in Zope's current DateTime.py was all spent finding & repairing basic roundtrip failures. These came in two flavors: 1. dt = DateTime() dt2 = DateTime( dt.year(), dt.month(), dt.day(), dt.hour(), dt.minute(), dt.second()) assert dt == dt2 could fail, depending on the exact time you tried it. This was the root cause of many test failures (they were usually reported as failures after doing some sort of arithmetic first, but the arithmetic was actually irrelevant: when these failed, the base objects didn't match from the start). 2. Failure of roundtrip conversion between DateTime objects D and repr(D), than back again, to reproduce the original DateTime or string. "Floating-point" got the generic blame for these things under the covers, but it was really peoples' spectacular inability to deal with *binary* floating-point that caused all the problems. People just can't help but see, e.g., "50.327" as an exact decimal value, so just can't help writing code that assumes false correlates (such as, e.g., that int((50.327 - 50) * 1000) will return 327; but it doesn't; it returns 326). If we were using decimal floating-point instead, the numerically naive code here would have worked fine.
... DateTime objects use .abstime and .absdate for doing arithmetic since these provides the best accuracy. The most important broken down values are calculated once at creation time; a few others are done on-the-fly.
Note that 2.2 properties allow natural support of calculated attributes, and that a computed attribute can easily arrange to cache its value.
I suppose that I could easily make a few calculation lazy to enhance speed; memory footprint would not change though. It's currently at 56 bytes per DateTime object and 36 bytes per DateTimeDelta object.
I'm assuming that counts Python object header overhead, but does not count hidden malloc overhead. Switching to pymalloc would slash the latter.
To get similar accuracy in Python, you'd need a float and an integer per object, that's 16 bytes + 12 bytes == 28 bytes + malloc() overhead for the two and the wrapping instance which gives another 32 bytes (provided you store the two objects in slots)... >60 bytes per Python based date time object.
Just noting that a Zope DateTime instance is huge, with a dozen named attributes, one of which holds a Python long (unbounded integer).
participants (2)
-
M.-A. Lemburg -
Tim Peters