millisecond and microsecond times without floats

Hello from MicroPython, a lean Python implementation scaling down to run even on microcontrollers (https://github.com/micropython/micropython). Our target hardware base oftentimes lacks floating point support, and using software emulation is expensive. So, we would like to have versions of some timing functions, taking/returning millisecond and/or microsecond values as integers. The most functionality we're interested in: 1. Delays 2. Relative time (from an arbitrary starting point, expected to be wrapped) 3. Calculating time differences, with immunity to wrap-around. The first presented assumption is to use "time.sleep()" for delays, "time.monotonic()" for relative time as the base. Would somebody gave alternative/better suggestions? Second question is how to modify their names for millisecond/microsecond versions. For sleep(), "msleep" and "usleep" would be concise possibilities, but that doesn't map well to monotonic(), leading to "mmonotonic". So, better idea is to use "_ms" and "_us" suffixes: sleep_ms() sleep_us() monotonic_ms() monotonic_us() Point 3 above isn't currently addressed by time module at all. https://www.python.org/dev/peps/pep-0418/ mentions some internal workaround for overflows/wrap-arounds on some systems. Due to lean-ness of our hardware base, we'd like to make this matter explicit to the applications and avoid internal workarounds. Proposed solution is to have time.elapsed(time1, time2) function, which can take values as returned by monotonic_ms(), monotonic_us(). Assuming that results of both functions are encoded and wrap consistently (this is reasonable assumption), there's no need for 2 separate elapsed_ms(), elapsed_us() function. So, the above are rough ideas we (well, I) have. We'd like to get wider Python community feedback on them, see if there're better/alternative ideas, how Pythonic it is, etc. To clarify, this should not be construed as proposal to add the above functions to CPython. -- Best regards, Paul mailto:pmiscml@gmail.com

On Mon, Jun 22, 2015 at 4:15 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
If you're going to add new function names, going with the _unit suffix seems best. Another option to consider: keyword only arguments. time.sleep(ms=31416) time.sleep(us=31415927) time.sleep(ns=31415296536) # We could use the long form names milliseconds, microseconds and nanoseconds but i worry with those that people would inevitably confuse ms with microseconds as times and APIs usually given the standard abbreviations rather than spelled out. time.monotonic(return_int_ns=True) ? # This seems ugly. time.monotonic_ns() seems better. These should be acceptable to add to Python 3.6 for consistency. I do not think we should have functions for each ms/us/ns unit if adding functions. Just choose the most useful high precision unit and let people do the math as needed for the others. Point 3 above isn't currently addressed by time module at all.
Reading the PEP my takeaway is that wrap-around of underlying deficient system APIs should be handled by the Python VM for the user. It sounds like we should explicitly spell this out though. I don't think time.elapsed() could ever provide any utility in either case, just use subtraction. time.elapsed() wouldn't know when and where the time values came from and magically be able to apply wrap around or not to them. -gps So, the above are rough ideas we (well, I) have. We'd like to get wider

For new functions altogether, maybe namespaces could be the a nice option -- from time.milliseconds import sleep, monotonic Named parameters would be a better way to implement it,though - I just don't know if having to go through the function that does have to be ready to handle floats anyway won't be in the way of the desired optimization On 22 June 2015 at 21:39, Gregory P. Smith <greg@krypto.org> wrote:

Hello, On Tue, 23 Jun 2015 00:03:14 +0000 "Gregory P. Smith" <greg@krypto.org> wrote: []
That doesn't immediately map to usage for monotonic(), as you mention below. Another issue is that keywords arguments on average (and for MicroPython all the time) are less efficient than positional. Put it other way, t = monotonic_ns() t = monotonic_ns() - t is going to give lower number than t = monotonic(ns=True) t = monotonic(ns=True) - t , and the closer it to 0, the better.
Another issue is that full spellings are rather long. Logistically, while function names can be expected to have autocompletion support, keyword arguments not necessarily.
Well, as I mentioned, I'm personally not looking for this to be implemented in CPython right away. Ideally, this should be tested by >1 independent "embedded" Python implementation first, and only then, based on the actual experience, submitted as a PEP. That's rather better than "desktop" CPython, which doesn't care about all the subtle "embedded" aspects "forced" a way to implement it.
Well, that's one of examples of that "desktop" thinking ;-). Consider for example that 2^32 microseconds is just over an hour, so expressing everything in microseconds would require arbitrary-precision integers, which may be just the same kind of burden for an embedded system as floats.
Point 3 above isn't currently addressed by time module at all. https://www.python.org/dev/peps/pep-0418/ mentions some internal
[]
This is another point which is overlooked by "desktop" programmers - time counters can, will, and do wrap around. Big OSes try hard to to hide this fact, and indeed succeed well enough, so in cases when they do fail, it has shattering effect (at least PR-wise) - Y2K, Y2K38 problems. For an embedded programmer wrapping counters is objective reality, and we wouldn't like to hide that fact in MicroPython (of course, only for these, newly introduced real-time precision time functions).
I don't think time.elapsed() could ever provide any utility in either case, just use subtraction.
Can't work. Previous value of monotonic_us() is 65530, next value is 10, what does it tell you?
Well, as I mentioned, it's an API contract that elapsed() takes values of monotonic_ms(), monotonic_us(), etc. functions, and knows law how their values change (likely, apply unsigned power-of-2 modular arithmetics). There's additional restriction that this change law for all of monotonic_ms(), monotonic_us() is the same, but I personally find this an acceptable restriction to not bloat API even further. (But it is a restriction, for example, if nano/microsecond time source is 24-bit counter, than millisecond time is limited to 24 bits too).
-gps
-- Best regards, Paul mailto:pmiscml@gmail.com

On Tue, Jun 23, 2015 at 1:25 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
I know. I was actually hoping you'd respond on that point because I haven't used micropython yet. I assumed it had bignum, or at least fixed "big" 64-bit number, support. But if it does not, having specific functions for the needed resolutions makes a lot of sense.
I still don't see how an elapsed() function taking two arbitrary integer arguments could work in a meaningful manner. Even if you assume they are the same units, the only assumption that can be made is that if the second int is lower than the first, at least one wraparound occurred.
At least one wrap around occurred. without more information you cannot know how many.
I guess what I'm missing is how you intend to tell elapsed() which of the _ms vs _us vs _ns functions the values came from. I'm assuming that all functions are likely to exist at once rather than there being only one high resolution integer time function. Given that, yes, you can make elapsed() do what you want. But I really think you should call it something more specific than elapsed if the function is serving as a common source of information on how a particular type of timer on the system works. monotonic_elapsed() perhaps? etc.. Also, agreed, we don't need these in 3.6. I'm not seeing anything really objectionable for inclusion in a future 3.x which is all I'm really looking out for. It sounds like avoiding keyword arguments and adding _ms _us and _ns variants of functions is the practical solution for micropython. -gps (awaiting his WiPys :)

On Tue, Jun 23, 2015 at 4:32 PM, Gregory P. Smith <greg@krypto.org> wrote:
Assuming you have an n-bit clock: (1) if you have arbitrary storage and the ability to do some sort of interrupt handling at least once per wraparound period, then you can reliably measure any duration. (2) if you don't have that, but can assume that at most one wraparound has occurred, then you can reliably measure any duration up to 2**n time units. (3) if you can't even make that assumption, then you can't reliably measure any duration whatsoever, so there's no point in even having the clock. I guess micropython is targeting platforms that can't afford option (1), but would like to at least take advantage of option (2)? -n -- Nathaniel J. Smith -- http://vorpus.org

On Tue, 23 Jun 2015 23:25:00 +0300 Paul Sokolovsky <pmiscml@gmail.com> wrote:
I'd like to suggest micropython first acquire the ability to handle 64-bit numbers (or something close to that, e.g. 60-bit, if it likes to use tags for typing), if it wants to become appropriate for precise datetime computations. That should be less of a heavy requirement than arbitrary-precision ints. Regards Antoine.

Hello, On Wed, 24 Jun 2015 09:19:55 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote:
MicroPython has such support. Long integers can be implemented either as variable-size arbitrary-precisions integers or and C long long type. But that doesn't change the fact that 64-bit values still overflow, or that we don't want to force need for any kind of long integer on any particular implementation. We don't even want to mask the fact that fixed-size (time) counters overflow - for various reasons, including the fact that we want to follow Python's tradition of being nice teaching/learning language, and learning embedded programming means learning to deal with timer, etc. overflows. So, the question is not how to "appropriate for precise datetime computations" - MicroPython inherits that ability by being a Python, but how to scale into the opposite direction, how to integrate into stdlib "realtime" time handling, which is simple, fast (getting timing value itself is low-overhead) and modular-arithmetic by its nature.
-- Best regards, Paul mailto:pmiscml@gmail.com

Hello, On Wed, 24 Jun 2015 12:40:10 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote:
Because MicroPython stays close (== may stay close) to hardware and does not depend on any OS (even those smaller embedded OSes, which are called RTOS'es). Then, it's usual case for embedded hardware to have hardware timers of the same size or smaller as the architecture machine word. For example, on a 32-bit CPU, timers are usually 32-, 24-, or 16- bit. On 16-bit CPUs, timers are 16- or 8-bit. Put it otherwise way, there's simply nowhere to get 64-bit time value from, except by building software abstractions, and MicroPython does not *require* them (if they exist - good, they will be helpful for other things, if not - MicroPython can still run and do a large subset of useful things). Another reason is that MicroPython exactly uses tagged pointers scheme, and small integers are value, not reference, objects. Dealing with them is largely faster (MicroPython easily beats CPython on (small) integer performance), and doesn't require memory allocation (the latter is another important feature for embedded systems).
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com

Hello, On Wed, 24 Jun 2015 13:03:38 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote: []
They don't, that was said in the very first message. They do have their place in MicroPython's stdlib and arguably in any other embedded Python's stdlib. There're number of embedded Python ports, I don't know if they tried to address to wider Python community regarding aspects peculiar to them. As you can see, we try to do the homework on our side.
Don't you have an additional namespace for micropython-specific features?
I treat it as a good sign that it's ~8th message in the thread and it's only the first time we get a hint that we should get out with our stuff into a separate namespace ;-). But of course, digging own hole and putting random stuff in there is everyone's first choice. And MicroPython has its "catch-all" module for random stuff imaginatively called "pyb", and in (user-friendly) embedded, the de-facto API standard is Arduino's, so that's what taken as a base for function names. So, MicroPython currently has: pyb.delay(ms) pyb.udelay(us) pyb.millis() pyb.micros() pyb.elapsed_millis() pyb.elapsed_micros() As can be seen, while these deal with time measurement/delays, they have little in common with how Python does it. And the main question we seek to answer is - what's more beneficial: to keep digging own hole or try to take Python's API as a close affinity (while still adhering to requirements posed by embedded platforms).
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com

On 24 June 2015 at 21:38, Paul Sokolovsky <pmiscml@gmail.com> wrote:
We hadn't previously gotten to the fact that part of your motivation was helping folks learn the intricacies of low level fixed width time measurement, though. That's actually a really cool idea - HC11 assembly programming and TI C5420 DSP programming are still two of my favourite things I've ever done, and it would be nice if folks could more easily start exploring the mindset of the embedded microprocessor world without having to first deal with the incidental complexity of emulators or actual embedded hardware (even something like programming an Arduino directly is more hassle than remote controlling one from a Raspberry Pi or PC). Unfortunately, I can't think of a good alternative name that isn't ambiguous at the CPython layer - embedded CPython is very different from an embedded microprocessor, utime is taken, and microtime is confusable with microseconds. I'm tempted to suggest calling it "qtime", and using TI's Q notation to denote the formats of numbers: https://en.wikipedia.org/wiki/Q_%28number_format%29 That would conflict with your notion of making the APIs agnostic as to the exact bitwidth used, though, as well as with the meaning of the "q" prefix in qmath: https://pypi.python.org/pypi/qmath Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hello, On Thu, 25 Jun 2015 22:56:59 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Well, Python is nice teaching language, and a lot of embedded programming can be done with just GPIO control (i.e. being able to control 1/0 digital signal) and (properly) timed delays (this approach is know as https://en.wikipedia.org/wiki/Bit_banging). Then if keeping it simple and performant we let people do more rather than less with that. So, yes, learning/experimentation is definitely in scope of this effort. People may ask what all that has to do with very high-level language Python, but I have 2 answers: 1. Languages like JavaScript or Lua have much more limited type model, e.g. they don't even have integer numeric type per se (only float), and yet their apologists don't feel too shy to push to use them for embedded hardware programming. Certainly, Python is not less, only more suited for that with its elaborated type model and stricter typedness overall. 2. When I started with Python 1.5, I couldn't imagine there would be e.g. memoryview's. And yet they're there. So, Python (and people behind it) do care about efficiency, so it shouldn't come as surprise special-purpose Python implementation cares about efficiency in its niche either.
Thanks, and yes, that's the idea behind MicroPython - that people new embedded programming could starter easier, while having chance to learn really cool language, and be able to go into low-level details and optimize (in my list, Python is as friendly to that as VHLL may be). And yet another idea to make MicroPython friendly to people who know Python and wanted to play with embedded. Making it play well for these 2 user groups isn't exactly easy, but we'd like to try.
You mean POSIX utime() function, right? How we have namespacing structured currently is that we have "u" prefix for all important buildin modules, e.g. uos, utime, etc. They contain bare minimum, and then fuller standard modules can be coded in Python. So, formally speaking, on our side it will go into separate namespace anyway. It's just we treat "utime" just as an alias for "time", and wouldn't like to put there something which couldn't be with clean conscience submitted as a PEP (in some distant future).
Yes, exact fixed-point nature of "Q" numbers doesn't help here, but as a designator of a special format it's pretty close to the original idea to use "_ms" and "_us" suffixes, so I treat that as a sign that we're on the right track. I however was thinking about our exchange with Antoine, and his surprise that we don't want to use 64-bit value. I guess I nailed the issue: I selected "monotonic()" because it seemed the closest to what we need, and in my list, our stuff is still "monotonic" in a sense that it goes only forward at constant pace. It just wraps around because so is the physical nature of the underlying fixes-size counter. Apparently, such "extended" treatment of "monotonic" is confusing for people who know time.monotonic() and PEP418. So, looks like we'll need to call our stuff different, I'm going to propose ticks_ms() and ticks_us() for MicroPython (hopefully "ticks" it's a well-known embedded term, and intuitive enough for other folks, at the very least, it's better than Linux kernel's jiffies ;-) ).
Cheers, Nick.
-- Best regards, Paul mailto:pmiscml@gmail.com

On 27 June 2015 at 05:14, Paul Sokolovsky <pmiscml@gmail.com> wrote:
I like it - as you say, ticks is already a common term for this, and it's clearly distinct from anything else in the time module if we ever decide to standardise it. It also doesn't hurt that "tick" is the term both LabVIEW (http://zone.ni.com/reference/en-XX/help/371361J-01/glang/tick_count_ms/) and Simulink (http://au.mathworks.com/help/stateflow/examples/using-absolute-time-temporal...) use for the concept. As a terminology/API suggestion, you may want to go with: tick_ms() - get the current tick with 1 millisecond between ticks tick_overflow_ms() - get the overflow period of the millisecond tick counter ticks_elapsed_ms(start, end) - get the number of millisecond ticks elapsed between two points in time (assuming at most one tick counter overflow between the start and end of the measurement) tick_us() - get the current tick with 1 microsecond between ticks tick_overflow_us() - get the overflow period of the microsecond tick counter ticks_elapsed_us(start, end) - get the number of microsecond ticks elapsed between two points in time (assuming at most one tick counter overflow between the start and end of the measurement) The problem I see with "ticks_ms()" and "ticks_us()" specifically is that the plural in the name implies "ticks elapsed since a given reference time". Since the tick counter can wrap around, there's no reference time - the current tick count is merely an opaque token allowing you to measure elapsed times up to the duration of the tick counter's overflow period. I also don't think you want to assume the overflow periods of the millisecond timer and the microsecond timer are going to be the same, hence the duplication of the other APIs as well. Something else you may want to consider is the idea of a "system tick", distinct from the fixed duration millisecond and microsecond ticks: tick() - get the current tick in system ticks tick_overflow() - get the overflow period of the system tick counter ticks_elapsed(start, end) - get the number of system ticks elapsed between two points in time (assuming at most one tick counter overflow between the start and end of the measurement) tick_duration() - get the system tick duration in seconds as a floating point number On platforms without a real time clock, the millisecond and microsecond ticks may then be approximations based on the system tick counter - that's actually the origin of my suggestion to expose completely separate APIs for the millisecond and microsecond versions, as if those are derived by dividing a fast system tick counter appropriately, they may wrap more frequently than every 2**32 or 2**64 ticks. Depending on use case, there may also be value in exposing the potential degree of jitter in the *_ms() and *_us() tick counters. I'm not sure if that would be best expressed in absolute or relative terms, though, so I'd suggest leaving that aspect undefined unless/until you have a specific use case in mind. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 23.06.2015 01:15, Paul Sokolovsky wrote:
You may want to use a similar approach as I have used in mxDateTime to express date/time values: http://www.egenix.com/products/python/mxBase/mxDateTime/ It uses an integer to represent days and a float to represent seconds since midnight (i.e. time of day). The concept has worked out really well and often makes date/time calculations a lot easier than trying to stuff everything into a single number and then having to deal things like leap seconds and rounding errors. In your case you'd use integers for both and nanoseconds as basis for the time of day integer. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 24 2015)
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78 2015-07-20: EuroPython 2015, Bilbao, Spain ... 26 days to go 2015-07-29: Python Meeting Duesseldorf ... 35 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

On Mon, Jun 22, 2015 at 4:15 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
If you're going to add new function names, going with the _unit suffix seems best. Another option to consider: keyword only arguments. time.sleep(ms=31416) time.sleep(us=31415927) time.sleep(ns=31415296536) # We could use the long form names milliseconds, microseconds and nanoseconds but i worry with those that people would inevitably confuse ms with microseconds as times and APIs usually given the standard abbreviations rather than spelled out. time.monotonic(return_int_ns=True) ? # This seems ugly. time.monotonic_ns() seems better. These should be acceptable to add to Python 3.6 for consistency. I do not think we should have functions for each ms/us/ns unit if adding functions. Just choose the most useful high precision unit and let people do the math as needed for the others. Point 3 above isn't currently addressed by time module at all.
Reading the PEP my takeaway is that wrap-around of underlying deficient system APIs should be handled by the Python VM for the user. It sounds like we should explicitly spell this out though. I don't think time.elapsed() could ever provide any utility in either case, just use subtraction. time.elapsed() wouldn't know when and where the time values came from and magically be able to apply wrap around or not to them. -gps So, the above are rough ideas we (well, I) have. We'd like to get wider

For new functions altogether, maybe namespaces could be the a nice option -- from time.milliseconds import sleep, monotonic Named parameters would be a better way to implement it,though - I just don't know if having to go through the function that does have to be ready to handle floats anyway won't be in the way of the desired optimization On 22 June 2015 at 21:39, Gregory P. Smith <greg@krypto.org> wrote:

Hello, On Tue, 23 Jun 2015 00:03:14 +0000 "Gregory P. Smith" <greg@krypto.org> wrote: []
That doesn't immediately map to usage for monotonic(), as you mention below. Another issue is that keywords arguments on average (and for MicroPython all the time) are less efficient than positional. Put it other way, t = monotonic_ns() t = monotonic_ns() - t is going to give lower number than t = monotonic(ns=True) t = monotonic(ns=True) - t , and the closer it to 0, the better.
Another issue is that full spellings are rather long. Logistically, while function names can be expected to have autocompletion support, keyword arguments not necessarily.
Well, as I mentioned, I'm personally not looking for this to be implemented in CPython right away. Ideally, this should be tested by >1 independent "embedded" Python implementation first, and only then, based on the actual experience, submitted as a PEP. That's rather better than "desktop" CPython, which doesn't care about all the subtle "embedded" aspects "forced" a way to implement it.
Well, that's one of examples of that "desktop" thinking ;-). Consider for example that 2^32 microseconds is just over an hour, so expressing everything in microseconds would require arbitrary-precision integers, which may be just the same kind of burden for an embedded system as floats.
Point 3 above isn't currently addressed by time module at all. https://www.python.org/dev/peps/pep-0418/ mentions some internal
[]
This is another point which is overlooked by "desktop" programmers - time counters can, will, and do wrap around. Big OSes try hard to to hide this fact, and indeed succeed well enough, so in cases when they do fail, it has shattering effect (at least PR-wise) - Y2K, Y2K38 problems. For an embedded programmer wrapping counters is objective reality, and we wouldn't like to hide that fact in MicroPython (of course, only for these, newly introduced real-time precision time functions).
I don't think time.elapsed() could ever provide any utility in either case, just use subtraction.
Can't work. Previous value of monotonic_us() is 65530, next value is 10, what does it tell you?
Well, as I mentioned, it's an API contract that elapsed() takes values of monotonic_ms(), monotonic_us(), etc. functions, and knows law how their values change (likely, apply unsigned power-of-2 modular arithmetics). There's additional restriction that this change law for all of monotonic_ms(), monotonic_us() is the same, but I personally find this an acceptable restriction to not bloat API even further. (But it is a restriction, for example, if nano/microsecond time source is 24-bit counter, than millisecond time is limited to 24 bits too).
-gps
-- Best regards, Paul mailto:pmiscml@gmail.com

On Tue, Jun 23, 2015 at 1:25 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
I know. I was actually hoping you'd respond on that point because I haven't used micropython yet. I assumed it had bignum, or at least fixed "big" 64-bit number, support. But if it does not, having specific functions for the needed resolutions makes a lot of sense.
I still don't see how an elapsed() function taking two arbitrary integer arguments could work in a meaningful manner. Even if you assume they are the same units, the only assumption that can be made is that if the second int is lower than the first, at least one wraparound occurred.
At least one wrap around occurred. without more information you cannot know how many.
I guess what I'm missing is how you intend to tell elapsed() which of the _ms vs _us vs _ns functions the values came from. I'm assuming that all functions are likely to exist at once rather than there being only one high resolution integer time function. Given that, yes, you can make elapsed() do what you want. But I really think you should call it something more specific than elapsed if the function is serving as a common source of information on how a particular type of timer on the system works. monotonic_elapsed() perhaps? etc.. Also, agreed, we don't need these in 3.6. I'm not seeing anything really objectionable for inclusion in a future 3.x which is all I'm really looking out for. It sounds like avoiding keyword arguments and adding _ms _us and _ns variants of functions is the practical solution for micropython. -gps (awaiting his WiPys :)

On Tue, Jun 23, 2015 at 4:32 PM, Gregory P. Smith <greg@krypto.org> wrote:
Assuming you have an n-bit clock: (1) if you have arbitrary storage and the ability to do some sort of interrupt handling at least once per wraparound period, then you can reliably measure any duration. (2) if you don't have that, but can assume that at most one wraparound has occurred, then you can reliably measure any duration up to 2**n time units. (3) if you can't even make that assumption, then you can't reliably measure any duration whatsoever, so there's no point in even having the clock. I guess micropython is targeting platforms that can't afford option (1), but would like to at least take advantage of option (2)? -n -- Nathaniel J. Smith -- http://vorpus.org

On Tue, 23 Jun 2015 23:25:00 +0300 Paul Sokolovsky <pmiscml@gmail.com> wrote:
I'd like to suggest micropython first acquire the ability to handle 64-bit numbers (or something close to that, e.g. 60-bit, if it likes to use tags for typing), if it wants to become appropriate for precise datetime computations. That should be less of a heavy requirement than arbitrary-precision ints. Regards Antoine.

Hello, On Wed, 24 Jun 2015 09:19:55 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote:
MicroPython has such support. Long integers can be implemented either as variable-size arbitrary-precisions integers or and C long long type. But that doesn't change the fact that 64-bit values still overflow, or that we don't want to force need for any kind of long integer on any particular implementation. We don't even want to mask the fact that fixed-size (time) counters overflow - for various reasons, including the fact that we want to follow Python's tradition of being nice teaching/learning language, and learning embedded programming means learning to deal with timer, etc. overflows. So, the question is not how to "appropriate for precise datetime computations" - MicroPython inherits that ability by being a Python, but how to scale into the opposite direction, how to integrate into stdlib "realtime" time handling, which is simple, fast (getting timing value itself is low-overhead) and modular-arithmetic by its nature.
-- Best regards, Paul mailto:pmiscml@gmail.com

Hello, On Wed, 24 Jun 2015 12:40:10 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote:
Because MicroPython stays close (== may stay close) to hardware and does not depend on any OS (even those smaller embedded OSes, which are called RTOS'es). Then, it's usual case for embedded hardware to have hardware timers of the same size or smaller as the architecture machine word. For example, on a 32-bit CPU, timers are usually 32-, 24-, or 16- bit. On 16-bit CPUs, timers are 16- or 8-bit. Put it otherwise way, there's simply nowhere to get 64-bit time value from, except by building software abstractions, and MicroPython does not *require* them (if they exist - good, they will be helpful for other things, if not - MicroPython can still run and do a large subset of useful things). Another reason is that MicroPython exactly uses tagged pointers scheme, and small integers are value, not reference, objects. Dealing with them is largely faster (MicroPython easily beats CPython on (small) integer performance), and doesn't require memory allocation (the latter is another important feature for embedded systems).
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com

Hello, On Wed, 24 Jun 2015 13:03:38 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote: []
They don't, that was said in the very first message. They do have their place in MicroPython's stdlib and arguably in any other embedded Python's stdlib. There're number of embedded Python ports, I don't know if they tried to address to wider Python community regarding aspects peculiar to them. As you can see, we try to do the homework on our side.
Don't you have an additional namespace for micropython-specific features?
I treat it as a good sign that it's ~8th message in the thread and it's only the first time we get a hint that we should get out with our stuff into a separate namespace ;-). But of course, digging own hole and putting random stuff in there is everyone's first choice. And MicroPython has its "catch-all" module for random stuff imaginatively called "pyb", and in (user-friendly) embedded, the de-facto API standard is Arduino's, so that's what taken as a base for function names. So, MicroPython currently has: pyb.delay(ms) pyb.udelay(us) pyb.millis() pyb.micros() pyb.elapsed_millis() pyb.elapsed_micros() As can be seen, while these deal with time measurement/delays, they have little in common with how Python does it. And the main question we seek to answer is - what's more beneficial: to keep digging own hole or try to take Python's API as a close affinity (while still adhering to requirements posed by embedded platforms).
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com

On 24 June 2015 at 21:38, Paul Sokolovsky <pmiscml@gmail.com> wrote:
We hadn't previously gotten to the fact that part of your motivation was helping folks learn the intricacies of low level fixed width time measurement, though. That's actually a really cool idea - HC11 assembly programming and TI C5420 DSP programming are still two of my favourite things I've ever done, and it would be nice if folks could more easily start exploring the mindset of the embedded microprocessor world without having to first deal with the incidental complexity of emulators or actual embedded hardware (even something like programming an Arduino directly is more hassle than remote controlling one from a Raspberry Pi or PC). Unfortunately, I can't think of a good alternative name that isn't ambiguous at the CPython layer - embedded CPython is very different from an embedded microprocessor, utime is taken, and microtime is confusable with microseconds. I'm tempted to suggest calling it "qtime", and using TI's Q notation to denote the formats of numbers: https://en.wikipedia.org/wiki/Q_%28number_format%29 That would conflict with your notion of making the APIs agnostic as to the exact bitwidth used, though, as well as with the meaning of the "q" prefix in qmath: https://pypi.python.org/pypi/qmath Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hello, On Thu, 25 Jun 2015 22:56:59 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Well, Python is nice teaching language, and a lot of embedded programming can be done with just GPIO control (i.e. being able to control 1/0 digital signal) and (properly) timed delays (this approach is know as https://en.wikipedia.org/wiki/Bit_banging). Then if keeping it simple and performant we let people do more rather than less with that. So, yes, learning/experimentation is definitely in scope of this effort. People may ask what all that has to do with very high-level language Python, but I have 2 answers: 1. Languages like JavaScript or Lua have much more limited type model, e.g. they don't even have integer numeric type per se (only float), and yet their apologists don't feel too shy to push to use them for embedded hardware programming. Certainly, Python is not less, only more suited for that with its elaborated type model and stricter typedness overall. 2. When I started with Python 1.5, I couldn't imagine there would be e.g. memoryview's. And yet they're there. So, Python (and people behind it) do care about efficiency, so it shouldn't come as surprise special-purpose Python implementation cares about efficiency in its niche either.
Thanks, and yes, that's the idea behind MicroPython - that people new embedded programming could starter easier, while having chance to learn really cool language, and be able to go into low-level details and optimize (in my list, Python is as friendly to that as VHLL may be). And yet another idea to make MicroPython friendly to people who know Python and wanted to play with embedded. Making it play well for these 2 user groups isn't exactly easy, but we'd like to try.
You mean POSIX utime() function, right? How we have namespacing structured currently is that we have "u" prefix for all important buildin modules, e.g. uos, utime, etc. They contain bare minimum, and then fuller standard modules can be coded in Python. So, formally speaking, on our side it will go into separate namespace anyway. It's just we treat "utime" just as an alias for "time", and wouldn't like to put there something which couldn't be with clean conscience submitted as a PEP (in some distant future).
Yes, exact fixed-point nature of "Q" numbers doesn't help here, but as a designator of a special format it's pretty close to the original idea to use "_ms" and "_us" suffixes, so I treat that as a sign that we're on the right track. I however was thinking about our exchange with Antoine, and his surprise that we don't want to use 64-bit value. I guess I nailed the issue: I selected "monotonic()" because it seemed the closest to what we need, and in my list, our stuff is still "monotonic" in a sense that it goes only forward at constant pace. It just wraps around because so is the physical nature of the underlying fixes-size counter. Apparently, such "extended" treatment of "monotonic" is confusing for people who know time.monotonic() and PEP418. So, looks like we'll need to call our stuff different, I'm going to propose ticks_ms() and ticks_us() for MicroPython (hopefully "ticks" it's a well-known embedded term, and intuitive enough for other folks, at the very least, it's better than Linux kernel's jiffies ;-) ).
Cheers, Nick.
-- Best regards, Paul mailto:pmiscml@gmail.com

On 27 June 2015 at 05:14, Paul Sokolovsky <pmiscml@gmail.com> wrote:
I like it - as you say, ticks is already a common term for this, and it's clearly distinct from anything else in the time module if we ever decide to standardise it. It also doesn't hurt that "tick" is the term both LabVIEW (http://zone.ni.com/reference/en-XX/help/371361J-01/glang/tick_count_ms/) and Simulink (http://au.mathworks.com/help/stateflow/examples/using-absolute-time-temporal...) use for the concept. As a terminology/API suggestion, you may want to go with: tick_ms() - get the current tick with 1 millisecond between ticks tick_overflow_ms() - get the overflow period of the millisecond tick counter ticks_elapsed_ms(start, end) - get the number of millisecond ticks elapsed between two points in time (assuming at most one tick counter overflow between the start and end of the measurement) tick_us() - get the current tick with 1 microsecond between ticks tick_overflow_us() - get the overflow period of the microsecond tick counter ticks_elapsed_us(start, end) - get the number of microsecond ticks elapsed between two points in time (assuming at most one tick counter overflow between the start and end of the measurement) The problem I see with "ticks_ms()" and "ticks_us()" specifically is that the plural in the name implies "ticks elapsed since a given reference time". Since the tick counter can wrap around, there's no reference time - the current tick count is merely an opaque token allowing you to measure elapsed times up to the duration of the tick counter's overflow period. I also don't think you want to assume the overflow periods of the millisecond timer and the microsecond timer are going to be the same, hence the duplication of the other APIs as well. Something else you may want to consider is the idea of a "system tick", distinct from the fixed duration millisecond and microsecond ticks: tick() - get the current tick in system ticks tick_overflow() - get the overflow period of the system tick counter ticks_elapsed(start, end) - get the number of system ticks elapsed between two points in time (assuming at most one tick counter overflow between the start and end of the measurement) tick_duration() - get the system tick duration in seconds as a floating point number On platforms without a real time clock, the millisecond and microsecond ticks may then be approximations based on the system tick counter - that's actually the origin of my suggestion to expose completely separate APIs for the millisecond and microsecond versions, as if those are derived by dividing a fast system tick counter appropriately, they may wrap more frequently than every 2**32 or 2**64 ticks. Depending on use case, there may also be value in exposing the potential degree of jitter in the *_ms() and *_us() tick counters. I'm not sure if that would be best expressed in absolute or relative terms, though, so I'd suggest leaving that aspect undefined unless/until you have a specific use case in mind. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 23.06.2015 01:15, Paul Sokolovsky wrote:
You may want to use a similar approach as I have used in mxDateTime to express date/time values: http://www.egenix.com/products/python/mxBase/mxDateTime/ It uses an integer to represent days and a float to represent seconds since midnight (i.e. time of day). The concept has worked out really well and often makes date/time calculations a lot easier than trying to stuff everything into a single number and then having to deal things like leap seconds and rounding errors. In your case you'd use integers for both and nanoseconds as basis for the time of day integer. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 24 2015)
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78 2015-07-20: EuroPython 2015, Bilbao, Spain ... 26 days to go 2015-07-29: Python Meeting Duesseldorf ... 35 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
participants (9)
-
Alexander Belopolsky
-
Antoine Pitrou
-
Gregory P. Smith
-
Joao S. O. Bueno
-
M.-A. Lemburg
-
Nathaniel Smith
-
Nick Coghlan
-
Paul Sokolovsky
-
Serhiy Storchaka