Re: [Python-ideas] millisecond and microsecond times without floats
On Mon, Jun 22, 2015, at 19:15, Paul Sokolovsky wrote:
What about having a fixed-point decimal numeric type to be used for this purpose? Allowing time (and stat, and the relevant functions of the datetime module) to return any real numeric type rather than being required to use float would be a useful extension.
On 26 June 2015 at 04:06, <random832@fastmail.us> wrote:
It isn't the data type that's the problem per se, it's the additional abstraction layers - the time module assumes it's dealing with operating system provided timing functionality, rather than accessing timer hardware directly. Folks tend to think that the os and time modules are low level, but there's a wonderful saying that asks "How do you tell the difference between a software developer and a computer systems engineer?" Answer: Software developer: "In a *low* level language like C..." Computer systems engineer: "In a *high* level language like C..." Paul, what do you think of the idea of trying to come up with a "hwclock" module for MicroPython that aims to expose very, very low level timing functionality as you describe, and reserving that name for independent distribution on PyPI? Such a module could even eventually grow a plugin system to provide access to various real time clock modules in addition to the basic counter based clocks you're interested in right now. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hello, On Fri, 26 Jun 2015 18:00:44 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Just to clarify, I tried to pose problem exactly as capturing right level of abstraction, but implementation-wise, data type is important for us (MicroPython) too. Specifically small integer is different from most other types is that it's value type, not reference type, and so it doesn't require memory allocation (will never trigger garbage collection => no unpredictable pauses), and faster to work with (uPy has ahead-of-type machine code compiler -> operations on word-sized values approach (unoptimized) C performance). But those are implementation details hidden by formulation of the original task - we need integer-based time (Python has integers, so no problems with that), and that time may and will wrap around at implementation-specific intervals (so a particular implementation may choose to represent it with efficient "small integer" type if it have one). Note that I also don't try to bloat the problem space and say "Guys, why don't we have unsigned integers in Python?" or "Let's have a generic builtin modular arithmetics module". None of those needed here.
Well, so on MicroPython side, having extra modules is expensive (defining a module costs 100+ bytes, OMG! ;-)) That's why we have catch-all "pyb" module so far, and see ways to put sensible extra stuff into existing modules. So, my concern is function, not module, names and sensible semantics of those functions. To come up with general-purpose "hwclock" module, there would need to be bigger cooperation from various parties and stakeholders. Neither myself nor other MicroPython developers can lead that effort, unfortunately. But it's my hope that if someone starts that effort, they will grep Python lists first for prior art, and maybe fall into arguments presented here, and select compatible API, then for us, compatibility will be easy: --- hwclock.py --- from utime import * ------------------ And even if someone selects other API, we'll know that ours is the most efficient building blocks we can have on our side, and can implement compatibility layer in their terms.
[] -- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Thu, 25 Jun 2015 14:06:38 -0400 random832@fastmail.us wrote:
The problem is actually not even in floating point per se. For example, reference hardware board for MicroPython is built on a relatively powerful microcontroller which has hardware floating point. But only single-precision floating point (IEEE 32-bit). And you won't read it at https://docs.python.org/3/library/time.html or PEP418 - it's just implied - that you don't just need floating point for those functions, it should be floating point of specific mantissa requirements. Let's count: time.time() returns the time in seconds since the epoch (1 Jan 1970) as a floating point number. That means that mantissa already should be at least 32 bits. Single-precision FP has 23 mantissa bits, so it's already ruled out from suitable representation of time.time() value. Then, we need 10 extra mantissa bits for each SI decimal subunit (2^10 == 1024). Double-precision FP has 52 mantissa bits. We can store millisecond precision there, we can store microsecond precision there. But - oops - going further, we hit the same problem as MicroPython hit right away: it's not possible to represent the same calendar data range as Unix time() call, but with higher resolution than microsecond. Of course, PEP418 provides one direction to work that around - by using other epochs than Jan 1, 1970 with all these new functions like monotonic() (which is specified as "reference point of the returned value is undefined"). It still implicitly assumes there's enough bits so wrap-arounds can be ignored. My proposal works around issue in another direction - by embracing the fact that any fixed-size counter wraps around and preparing to deal with that. All that in turn enables to use just integer values for representing times. And why it's useful (implementation-wise) to be able to use integer values, I elaborated in another recent mail. [] -- Best regards, Paul mailto:pmiscml@gmail.com
On 26 June 2015 at 04:06, <random832@fastmail.us> wrote:
It isn't the data type that's the problem per se, it's the additional abstraction layers - the time module assumes it's dealing with operating system provided timing functionality, rather than accessing timer hardware directly. Folks tend to think that the os and time modules are low level, but there's a wonderful saying that asks "How do you tell the difference between a software developer and a computer systems engineer?" Answer: Software developer: "In a *low* level language like C..." Computer systems engineer: "In a *high* level language like C..." Paul, what do you think of the idea of trying to come up with a "hwclock" module for MicroPython that aims to expose very, very low level timing functionality as you describe, and reserving that name for independent distribution on PyPI? Such a module could even eventually grow a plugin system to provide access to various real time clock modules in addition to the basic counter based clocks you're interested in right now. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hello, On Fri, 26 Jun 2015 18:00:44 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Just to clarify, I tried to pose problem exactly as capturing right level of abstraction, but implementation-wise, data type is important for us (MicroPython) too. Specifically small integer is different from most other types is that it's value type, not reference type, and so it doesn't require memory allocation (will never trigger garbage collection => no unpredictable pauses), and faster to work with (uPy has ahead-of-type machine code compiler -> operations on word-sized values approach (unoptimized) C performance). But those are implementation details hidden by formulation of the original task - we need integer-based time (Python has integers, so no problems with that), and that time may and will wrap around at implementation-specific intervals (so a particular implementation may choose to represent it with efficient "small integer" type if it have one). Note that I also don't try to bloat the problem space and say "Guys, why don't we have unsigned integers in Python?" or "Let's have a generic builtin modular arithmetics module". None of those needed here.
Well, so on MicroPython side, having extra modules is expensive (defining a module costs 100+ bytes, OMG! ;-)) That's why we have catch-all "pyb" module so far, and see ways to put sensible extra stuff into existing modules. So, my concern is function, not module, names and sensible semantics of those functions. To come up with general-purpose "hwclock" module, there would need to be bigger cooperation from various parties and stakeholders. Neither myself nor other MicroPython developers can lead that effort, unfortunately. But it's my hope that if someone starts that effort, they will grep Python lists first for prior art, and maybe fall into arguments presented here, and select compatible API, then for us, compatibility will be easy: --- hwclock.py --- from utime import * ------------------ And even if someone selects other API, we'll know that ours is the most efficient building blocks we can have on our side, and can implement compatibility layer in their terms.
[] -- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Thu, 25 Jun 2015 14:06:38 -0400 random832@fastmail.us wrote:
The problem is actually not even in floating point per se. For example, reference hardware board for MicroPython is built on a relatively powerful microcontroller which has hardware floating point. But only single-precision floating point (IEEE 32-bit). And you won't read it at https://docs.python.org/3/library/time.html or PEP418 - it's just implied - that you don't just need floating point for those functions, it should be floating point of specific mantissa requirements. Let's count: time.time() returns the time in seconds since the epoch (1 Jan 1970) as a floating point number. That means that mantissa already should be at least 32 bits. Single-precision FP has 23 mantissa bits, so it's already ruled out from suitable representation of time.time() value. Then, we need 10 extra mantissa bits for each SI decimal subunit (2^10 == 1024). Double-precision FP has 52 mantissa bits. We can store millisecond precision there, we can store microsecond precision there. But - oops - going further, we hit the same problem as MicroPython hit right away: it's not possible to represent the same calendar data range as Unix time() call, but with higher resolution than microsecond. Of course, PEP418 provides one direction to work that around - by using other epochs than Jan 1, 1970 with all these new functions like monotonic() (which is specified as "reference point of the returned value is undefined"). It still implicitly assumes there's enough bits so wrap-arounds can be ignored. My proposal works around issue in another direction - by embracing the fact that any fixed-size counter wraps around and preparing to deal with that. All that in turn enables to use just integer values for representing times. And why it's useful (implementation-wise) to be able to use integer values, I elaborated in another recent mail. [] -- Best regards, Paul mailto:pmiscml@gmail.com
participants (3)
-
Nick Coghlan -
Paul Sokolovsky -
random832＠fastmail.us