Victor Stinner schrieb am 13.10.2017 um 16:12:

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

I might have missed it while skipping through your post, but could you quickly explain why improving the precision of time.time() itself wouldn't help already? Would double FP precision not be accurate enough here? Stefan

On Fri, 13 Oct 2017 16:57:28 +0200 Stefan Behnel wrote:

Victor Stinner schrieb am 13.10.2017 um 16:12:

...

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

I might have missed it while skipping through your post, but could you quickly explain why improving the precision of time.time() itself wouldn't help already? Would double FP precision not be accurate enough here?

To quote Victor's message: « The problem is that Python returns time as a floatting point number which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days. » Regards Antoine.

On Sat, 14 Oct 2017 10:49:11 +0300 Serhiy Storchaka wrote:

13.10.17 17:12, Victor Stinner пише:

...

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

It would enhance the time.time() clock resolution. In my experience, it decreases the minimum non-zero delta between two clock by 3 times, new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in Python.

The question of this long email is if it's worth it to add more "_ns" time functions than just time.time_ns()?

I would like to add:

* time.time_ns() * time.monotonic_ns() * time.perf_counter_ns() * time.clock_gettime_ns() * time.clock_settime_ns()

time(), monotonic() and perf_counter() clocks are the 3 most common clocks and users use them to get the best available clock resolution. clock_gettime/settime() are the generic UNIX API to access these clocks and so should also be enhanced to get nanosecond resolution.

I don't like the idea of adding a parallel set of functions.

In the list of alternatives in PEP 410 there is no an idea about fixed precision float type with nanoseconds precision. It can be implemented internally as a 64-bit integer, but provide all methods required for float-compatible number. It would be simpler and faster than general Decimal.

I agree a parallel set of functions is not ideal, but I think a parallel set of functions is still more appropriate than a new number type specific to the time module. Also, if you change existing functions to return a new type, you risk breaking compatibility even if you are very careful about designing the new type. Regards Antoine.

On Sun, 15 Oct 2017 01:46:50 +1000 Nick Coghlan wrote:

Might it make more sense to have a parallel *module* that works with a different base data type rather than parallel functions within the existing API?

That is, if folks wanted to switch to 64-bit nanosecond time, they would use:

* time_ns.time() * time_ns.monotonic() * time_ns.perf_counter() * time_ns.clock_gettime() * time_ns.clock_settime()

The idea here would be akin to the fact we have both math and cmath as modules, where the common APIs conceptually implement the same algorithms, they just work with a different numeric type (floats vs complex numbers).

-1 from me. The math/cmath separation isn't even very well grounded, it just mirrors the C API that those two modules reflect. But regardless, the *operations* in math and cmath are different and operate in different domains (try e.g. ``sqrt(-1)``), which is not the case here. Regards Antoine.

(That being said, I'm a big fan of fractions, so I wonder if a Fraction with a constant nano denominator wouldn't fit in here...)

It was discussed in depth in PEP 410, and the PEP was rejected. Guido voted for nanoseconds as int, when os.stat_result.st_mtime_ns was added. Victor

I'd like to have time.time_ns() -- this is most parallel to st_mtime_ns. On Sun, Oct 15, 2017 at 2:59 AM, Victor Stinner wrote:

...

Might it make more sense to have a parallel *module* that works with a different base data type rather than parallel functions within the existing API?

I asked about adding new functions to 4 different modules: os, resource, signal, time.

For example, I dislike the idea of having os and os_ns modules. We already have os.stat() which returns time as seconds and nanoseconds (both at the same time). There is also os.utime() which accepts time as seconds *or* nanoseconds: os.utime (path, times=seconds) or os.utime(path, ns=nanoseconds).

If we had a time_ns module, would it only contain 4 clocks or does it have to duplicate the full API? If yes, it is likely to be a mess to maintain them. How will user choose between time and time_ns? What if tomorrow clocks get picosecond resolution? (CPU TSC also has sub-nanosecond resolution, but OS API uses timespec, 1 ns res.) Add a third module?

I prefer to leave all "time functions" in the "time module".

For example, I don't think that we need to add time.nanosleep() or time.sleep_ns(), since the precision loss starts after a sleep of 104 days. Who cares of 1 nanosecond after a sleep of 104 days?

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Since new APIs are expensive and we'd like to be future-proof, why not move to picoseconds? That would be safe until clocks reach the THz barrier, which is quite far away from us. Regards Antoine. On Fri, 13 Oct 2017 16:12:39 +0200 Victor Stinner wrote:

Hi,

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

It would enhance the time.time() clock resolution. In my experience, it decreases the minimum non-zero delta between two clock by 3 times, new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in Python.

The question of this long email is if it's worth it to add more "_ns" time functions than just time.time_ns()?

I would like to add:

* time.time_ns() * time.monotonic_ns() * time.perf_counter_ns() * time.clock_gettime_ns() * time.clock_settime_ns()

time(), monotonic() and perf_counter() clocks are the 3 most common clocks and users use them to get the best available clock resolution. clock_gettime/settime() are the generic UNIX API to access these clocks and so should also be enhanced to get nanosecond resolution.

== Nanosecond resolution ==

More and more clocks have a frequency in MHz, up to GHz for the "TSC" CPU clock, and so the clocks resolution is getting closer to 1 nanosecond (or even better than 1 ns for the TSC clock!).

The problem is that Python returns time as a floatting point number which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days. Conversion from nanoseconds (int) to seconds (float) and then back to nanoseconds (int) to check if conversions loose precision:

# no precision loss

...

...

...

x=2**52+1; int(float(x * 1e-9) * 1e9) - x 0 # precision loss! (1 nanosecond) x=2**53+1; int(float(x * 1e-9) * 1e9) - x -1 print(datetime.timedelta(seconds=2**53 / 1e9)) 104 days, 5:59:59.254741

While a system administrator can be proud to have an uptime longer than 104 days, the problem also exists for the time.time() clock which returns the number of seconds since the UNIX epoch (1970-01-01). This clock started to loose nanoseconds since mid-May 1970 (47 years ago):

...

...

...

import datetime print(datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=2**53 / 1e9)) 1970-04-15 05:59:59.254741

== PEP 410 ==

Five years ago, I proposed a large and complex change in all Python functions returning time to support nanosecond resolution using the decimal.Decimal type:

https://www.python.org/dev/peps/pep-0410/

The PEP was rejected for different reasons:

* it wasn't clear if hardware clocks really had a resolution of 1 nanosecond, especially when the clock is read from Python, since reading a clock in Python also takes time...

* Guido van Rossum rejected the idea of adding a new optional parameter to change the result type: it's an uncommon programming practice (bad design in Python)

* decimal.Decimal is not widely used, it might be surprised to get such type

== CPython enhancements of the last 5 years ==

Since this PEP was rejected:

* the os.stat_result got 3 fields for timestamps as nanoseconds (Python int): st_atime_ns, st_ctime_ns, st_mtime_ns

* Python 3.3 got 3 new clocks: time.monotonic(), time.perf_counter() and time.process_time()

* I enhanced the private C API of Python handling time (API called "pytime") to store all timings as the new _PyTime_t type which is a simple 64-bit signed integer. The unit of _PyTime_t is not part of the API, it's an implementation detail. The unit is currently 1 nanosecond.

This week, I converted one of the last clock to new _PyTime_t format: time.perf_counter() now has internally a resolution of 1 nanosecond, instead of using the C double type.

XXX technically https://github.com/python/cpython/pull/3983 is not merged yet :-)

== Clocks resolution in Python ==

I implemented time.time_ns(), time.monotonic_ns() and time.perf_counter_ns() which are similar of the functions without the "_ns" suffix, but return time as nanoseconds (Python int).

I computed the smallest difference between two clock reads (ignoring a differences of zero):

Linux:

* time_ns(): 84 ns <=== !!! * time(): 239 ns <=== !!! * perf_counter_ns(): 84 ns * perf_counter(): 82 ns * monotonic_ns(): 84 ns * monotonic(): 81 ns

Windows:

* time_ns(): 318000 ns <=== !!! * time(): 894070 ns <=== !!! * perf_counter_ns(): 100 ns * perf_counter(): 100 ns * monotonic_ns(): 15000000 ns * monotonic(): 15000000 ns

The difference on time.time() is significant: 84 ns (2.8x better) vs 239 ns on Linux and 318 us (2.8x better) vs 894 us on Windows. The difference will be larger next years since every day adds 864,00,000,000,000 nanoseconds to the system clock :-) (please don't bug me with leap seconds! you got my point)

The difference on perf_counter and monotonic clocks are not visible in this quick script since my script runs less than 1 minute, my computer uptime is smaller than 1 weak, ... and Python internally starts these clocks at zero *to reduce the precision loss*! Using an uptime larger than 104 days, you would probably see a significant difference (at least +/- 1 nanosecond) between the regular (seconds as double) and the "_ns" (nanoseconds as int) clocks.

== How many new nanosecond clocks? ==

The PEP 410 proposed to modify the following functions:

* os module: fstat(), fstatat(), lstat(), stat() (st_atime, st_ctime and st_mtime fields of the stat structure), sched_rr_get_interval(), times(), wait3() and wait4()

* resource module: ru_utime and ru_stime fields of getrusage()

* signal module: getitimer(), setitimer()

* time module: clock(), clock_gettime(), clock_getres(), monotonic(), time() and wallclock() ("wallclock()" was finally called "monotonic", see PEP 418)

According to my tests of the previous section, the precision loss starts after 104 days (stored in nanoseconds). I don't know if it's worth it to modify functions which return "CPU time" or "process time" of processes, since most processes live shorter than 104 days. Do you care of a resolution of 1 nanosecond for the CPU and process time?

Maybe we need 1 nanosecond resolution for profiling and benchmarks. But in that case, you might want to implement your profiler in C rather in Python, like the hotshot module, no? The "pytime" private API of CPython gives you clocks with a resolution of 1 nanosecond.

== Annex: clock performance ==

To have an idea of the cost of reading the clock on the clock resolution in Python, I also ran a microbenchmark on *reading* a clock. Example:

$ ./python -m perf timeit --duplicate 1024 -s 'import time; t=time.time' 't()'

Linux (Mean +- std dev):

* time.time(): 45.4 ns +- 0.5 ns * time.time_ns(): 47.8 ns +- 0.8 ns * time.perf_counter(): 46.8 ns +- 0.7 ns * time.perf_counter_ns(): 46.0 ns +- 0.6 ns

Windows (Mean +- std dev):

* time.time(): 42.2 ns +- 0.8 ns * time.time_ns(): 49.3 ns +- 0.8 ns * time.perf_counter(): 136 ns +- 2 ns <=== * time.perf_counter_ns(): 143 ns +- 4 ns <=== * time.monotonic(): 38.3 ns +- 0.9 ns * time.monotonic_ns(): 48.8 ns +- 1.2 ns

Most clocks have the same performance except of perf_counter on Windows: around 140 ns whereas other clocks are around 45 ns (on Linux and Windows): 3x slower. Maybe the "bad" perf_counter performance can be explained by the fact that I'm running Windows in a VM, which is not ideal for benchmarking. Or maybe my C implementation of time.perf_counter() is slow?

Note: I expect that a significant part of the numbers are the cost of Python function calls. Reading these clocks using the Python C functions are likely faster.

Victor _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, Oct 15, 2017 at 8:17 PM, Antoine Pitrou wrote:

Since new APIs are expensive and we'd like to be future-proof, why not move to picoseconds? That would be safe until clocks reach the THz barrier, which is quite far away from us.

​I somewhat like the thought, but would everyone then end up thinking about what power of 1000 they need to multiply with? -- Koos​

Regards

Antoine.

On Fri, 13 Oct 2017 16:12:39 +0200 Victor Stinner wrote:

...

Hi,

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

It would enhance the time.time() clock resolution. In my experience, it decreases the minimum non-zero delta between two clock by 3 times, new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in Python.

The question of this long email is if it's worth it to add more "_ns" time functions than just time.time_ns()?

I would like to add:

* time.time_ns() * time.monotonic_ns() * time.perf_counter_ns() * time.clock_gettime_ns() * time.clock_settime_ns()

time(), monotonic() and perf_counter() clocks are the 3 most common clocks and users use them to get the best available clock resolution. clock_gettime/settime() are the generic UNIX API to access these clocks and so should also be enhanced to get nanosecond resolution.

== Nanosecond resolution ==

More and more clocks have a frequency in MHz, up to GHz for the "TSC" CPU clock, and so the clocks resolution is getting closer to 1 nanosecond (or even better than 1 ns for the TSC clock!).

The problem is that Python returns time as a floatting point number which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days. Conversion from nanoseconds (int) to seconds (float) and then back to nanoseconds (int) to check if conversions loose precision:

# no precision loss

...

...

...

x=2**52+1; int(float(x * 1e-9) * 1e9) - x 0 # precision loss! (1 nanosecond) x=2**53+1; int(float(x * 1e-9) * 1e9) - x -1 print(datetime.timedelta(seconds=2**53 / 1e9)) 104 days, 5:59:59.254741

While a system administrator can be proud to have an uptime longer than 104 days, the problem also exists for the time.time() clock which returns the number of seconds since the UNIX epoch (1970-01-01). This clock started to loose nanoseconds since mid-May 1970 (47 years ago):

...

...

...

import datetime print(datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=2**53 / 1e9)) 1970-04-15 05:59:59.254741

== PEP 410 ==

Five years ago, I proposed a large and complex change in all Python functions returning time to support nanosecond resolution using the decimal.Decimal type:

https://www.python.org/dev/peps/pep-0410/

The PEP was rejected for different reasons:

* it wasn't clear if hardware clocks really had a resolution of 1 nanosecond, especially when the clock is read from Python, since reading a clock in Python also takes time...

* Guido van Rossum rejected the idea of adding a new optional parameter to change the result type: it's an uncommon programming practice (bad design in Python)

* decimal.Decimal is not widely used, it might be surprised to get such type

== CPython enhancements of the last 5 years ==

Since this PEP was rejected:

* the os.stat_result got 3 fields for timestamps as nanoseconds (Python int): st_atime_ns, st_ctime_ns, st_mtime_ns

* Python 3.3 got 3 new clocks: time.monotonic(), time.perf_counter() and time.process_time()

* I enhanced the private C API of Python handling time (API called "pytime") to store all timings as the new _PyTime_t type which is a simple 64-bit signed integer. The unit of _PyTime_t is not part of the API, it's an implementation detail. The unit is currently 1 nanosecond.

This week, I converted one of the last clock to new _PyTime_t format: time.perf_counter() now has internally a resolution of 1 nanosecond, instead of using the C double type.

XXX technically https://github.com/python/cpython/pull/3983 is not merged yet :-)

== Clocks resolution in Python ==

I implemented time.time_ns(), time.monotonic_ns() and time.perf_counter_ns() which are similar of the functions without the "_ns" suffix, but return time as nanoseconds (Python int).

I computed the smallest difference between two clock reads (ignoring a differences of zero):

Linux:

* time_ns(): 84 ns <=== !!! * time(): 239 ns <=== !!! * perf_counter_ns(): 84 ns * perf_counter(): 82 ns * monotonic_ns(): 84 ns * monotonic(): 81 ns

Windows:

* time_ns(): 318000 ns <=== !!! * time(): 894070 ns <=== !!! * perf_counter_ns(): 100 ns * perf_counter(): 100 ns * monotonic_ns(): 15000000 ns * monotonic(): 15000000 ns

The difference on time.time() is significant: 84 ns (2.8x better) vs 239 ns on Linux and 318 us (2.8x better) vs 894 us on Windows. The difference will be larger next years since every day adds 864,00,000,000,000 nanoseconds to the system clock :-) (please don't bug me with leap seconds! you got my point)

The difference on perf_counter and monotonic clocks are not visible in this quick script since my script runs less than 1 minute, my computer uptime is smaller than 1 weak, ... and Python internally starts these clocks at zero *to reduce the precision loss*! Using an uptime larger than 104 days, you would probably see a significant difference (at least +/- 1 nanosecond) between the regular (seconds as double) and the "_ns" (nanoseconds as int) clocks.

== How many new nanosecond clocks? ==

The PEP 410 proposed to modify the following functions:

* os module: fstat(), fstatat(), lstat(), stat() (st_atime, st_ctime and st_mtime fields of the stat structure), sched_rr_get_interval(), times(), wait3() and wait4()

* resource module: ru_utime and ru_stime fields of getrusage()

* signal module: getitimer(), setitimer()

* time module: clock(), clock_gettime(), clock_getres(), monotonic(), time() and wallclock() ("wallclock()" was finally called "monotonic", see PEP 418)

According to my tests of the previous section, the precision loss starts after 104 days (stored in nanoseconds). I don't know if it's worth it to modify functions which return "CPU time" or "process time" of processes, since most processes live shorter than 104 days. Do you care of a resolution of 1 nanosecond for the CPU and process time?

Maybe we need 1 nanosecond resolution for profiling and benchmarks. But in that case, you might want to implement your profiler in C rather in Python, like the hotshot module, no? The "pytime" private API of CPython gives you clocks with a resolution of 1 nanosecond.

== Annex: clock performance ==

To have an idea of the cost of reading the clock on the clock resolution in Python, I also ran a microbenchmark on *reading* a clock. Example:

$ ./python -m perf timeit --duplicate 1024 -s 'import time; t=time.time' 't()'

Linux (Mean +- std dev):

* time.time(): 45.4 ns +- 0.5 ns * time.time_ns(): 47.8 ns +- 0.8 ns * time.perf_counter(): 46.8 ns +- 0.7 ns * time.perf_counter_ns(): 46.0 ns +- 0.6 ns

Windows (Mean +- std dev):

* time.time(): 42.2 ns +- 0.8 ns * time.time_ns(): 49.3 ns +- 0.8 ns * time.perf_counter(): 136 ns +- 2 ns <=== * time.perf_counter_ns(): 143 ns +- 4 ns <=== * time.monotonic(): 38.3 ns +- 0.9 ns * time.monotonic_ns(): 48.8 ns +- 1.2 ns

Most clocks have the same performance except of perf_counter on Windows: around 140 ns whereas other clocks are around 45 ns (on Linux and Windows): 3x slower. Maybe the "bad" perf_counter performance can be explained by the fact that I'm running Windows in a VM, which is not ideal for benchmarking. Or maybe my C implementation of time.perf_counter() is slow?

Note: I expect that a significant part of the numbers are the cost of Python function calls. Reading these clocks using the Python C functions are likely faster.

Victor _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- + Koos Zevenhoven + http://twitter.com/k7hoven +

I proposed to use nanoseconds because UNIX has 1 ns resolution in timespec, the most recent API, and Windows has 100 ns. Using picoseconds would confuse users who may expect sub-nanosecond resolution, whereas no OS support them currently. Moreover, nanoseconds as int already landed in os.stat and os.utime. Last but not least, I already strugle in pytime.c to prevent integer overflow with 1 ns resolution. It can quickly become much more complex if there is no native C int type supporting a range large enough to more 1 picosecond resolution usable. I really like using int64_t for _PyTime_t, it's well supported, very easy to use (ex: "t = t2 - t1"). 64-bit int supports year after 2200 for delta since 1970. Note: I only know Ruby which chose picoseconds. Victor Le 15 oct. 2017 19:18, "Antoine Pitrou" a écrit :

Since new APIs are expensive and we'd like to be future-proof, why not move to picoseconds? That would be safe until clocks reach the THz barrier, which is quite far away from us.

Regards

Antoine.

On Fri, 13 Oct 2017 16:12:39 +0200 Victor Stinner wrote:

...

Hi,

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

It would enhance the time.time() clock resolution. In my experience, it decreases the minimum non-zero delta between two clock by 3 times, new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in Python.

The question of this long email is if it's worth it to add more "_ns" time functions than just time.time_ns()?

I would like to add:

* time.time_ns() * time.monotonic_ns() * time.perf_counter_ns() * time.clock_gettime_ns() * time.clock_settime_ns()

time(), monotonic() and perf_counter() clocks are the 3 most common clocks and users use them to get the best available clock resolution. clock_gettime/settime() are the generic UNIX API to access these clocks and so should also be enhanced to get nanosecond resolution.

== Nanosecond resolution ==

More and more clocks have a frequency in MHz, up to GHz for the "TSC" CPU clock, and so the clocks resolution is getting closer to 1 nanosecond (or even better than 1 ns for the TSC clock!).

The problem is that Python returns time as a floatting point number which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days. Conversion from nanoseconds (int) to seconds (float) and then back to nanoseconds (int) to check if conversions loose precision:

# no precision loss

...

...

...

x=2**52+1; int(float(x * 1e-9) * 1e9) - x 0 # precision loss! (1 nanosecond) x=2**53+1; int(float(x * 1e-9) * 1e9) - x -1 print(datetime.timedelta(seconds=2**53 / 1e9)) 104 days, 5:59:59.254741

While a system administrator can be proud to have an uptime longer than 104 days, the problem also exists for the time.time() clock which returns the number of seconds since the UNIX epoch (1970-01-01). This clock started to loose nanoseconds since mid-May 1970 (47 years ago):

...

...

...

import datetime print(datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=2**53 / 1e9)) 1970-04-15 05:59:59.254741

== PEP 410 ==

Five years ago, I proposed a large and complex change in all Python functions returning time to support nanosecond resolution using the decimal.Decimal type:

https://www.python.org/dev/peps/pep-0410/

The PEP was rejected for different reasons:

* it wasn't clear if hardware clocks really had a resolution of 1 nanosecond, especially when the clock is read from Python, since reading a clock in Python also takes time...

* Guido van Rossum rejected the idea of adding a new optional parameter to change the result type: it's an uncommon programming practice (bad design in Python)

* decimal.Decimal is not widely used, it might be surprised to get such type

== CPython enhancements of the last 5 years ==

Since this PEP was rejected:

* the os.stat_result got 3 fields for timestamps as nanoseconds (Python int): st_atime_ns, st_ctime_ns, st_mtime_ns

* Python 3.3 got 3 new clocks: time.monotonic(), time.perf_counter() and time.process_time()

* I enhanced the private C API of Python handling time (API called "pytime") to store all timings as the new _PyTime_t type which is a simple 64-bit signed integer. The unit of _PyTime_t is not part of the API, it's an implementation detail. The unit is currently 1 nanosecond.

This week, I converted one of the last clock to new _PyTime_t format: time.perf_counter() now has internally a resolution of 1 nanosecond, instead of using the C double type.

XXX technically https://github.com/python/cpython/pull/3983 is not merged yet :-)

== Clocks resolution in Python ==

I implemented time.time_ns(), time.monotonic_ns() and time.perf_counter_ns() which are similar of the functions without the "_ns" suffix, but return time as nanoseconds (Python int).

I computed the smallest difference between two clock reads (ignoring a differences of zero):

Linux:

* time_ns(): 84 ns <=== !!! * time(): 239 ns <=== !!! * perf_counter_ns(): 84 ns * perf_counter(): 82 ns * monotonic_ns(): 84 ns * monotonic(): 81 ns

Windows:

* time_ns(): 318000 ns <=== !!! * time(): 894070 ns <=== !!! * perf_counter_ns(): 100 ns * perf_counter(): 100 ns * monotonic_ns(): 15000000 ns * monotonic(): 15000000 ns

The difference on time.time() is significant: 84 ns (2.8x better) vs 239 ns on Linux and 318 us (2.8x better) vs 894 us on Windows. The difference will be larger next years since every day adds 864,00,000,000,000 nanoseconds to the system clock :-) (please don't bug me with leap seconds! you got my point)

The difference on perf_counter and monotonic clocks are not visible in this quick script since my script runs less than 1 minute, my computer uptime is smaller than 1 weak, ... and Python internally starts these clocks at zero *to reduce the precision loss*! Using an uptime larger than 104 days, you would probably see a significant difference (at least +/- 1 nanosecond) between the regular (seconds as double) and the "_ns" (nanoseconds as int) clocks.

== How many new nanosecond clocks? ==

The PEP 410 proposed to modify the following functions:

* os module: fstat(), fstatat(), lstat(), stat() (st_atime, st_ctime and st_mtime fields of the stat structure), sched_rr_get_interval(), times(), wait3() and wait4()

* resource module: ru_utime and ru_stime fields of getrusage()

* signal module: getitimer(), setitimer()

* time module: clock(), clock_gettime(), clock_getres(), monotonic(), time() and wallclock() ("wallclock()" was finally called "monotonic", see PEP 418)

According to my tests of the previous section, the precision loss starts after 104 days (stored in nanoseconds). I don't know if it's worth it to modify functions which return "CPU time" or "process time" of processes, since most processes live shorter than 104 days. Do you care of a resolution of 1 nanosecond for the CPU and process time?

Maybe we need 1 nanosecond resolution for profiling and benchmarks. But in that case, you might want to implement your profiler in C rather in Python, like the hotshot module, no? The "pytime" private API of CPython gives you clocks with a resolution of 1 nanosecond.

== Annex: clock performance ==

To have an idea of the cost of reading the clock on the clock resolution in Python, I also ran a microbenchmark on *reading* a clock. Example:

$ ./python -m perf timeit --duplicate 1024 -s 'import time; t=time.time' 't()'

Linux (Mean +- std dev):

* time.time(): 45.4 ns +- 0.5 ns * time.time_ns(): 47.8 ns +- 0.8 ns * time.perf_counter(): 46.8 ns +- 0.7 ns * time.perf_counter_ns(): 46.0 ns +- 0.6 ns

Windows (Mean +- std dev):

* time.time(): 42.2 ns +- 0.8 ns * time.time_ns(): 49.3 ns +- 0.8 ns * time.perf_counter(): 136 ns +- 2 ns <=== * time.perf_counter_ns(): 143 ns +- 4 ns <=== * time.monotonic(): 38.3 ns +- 0.9 ns * time.monotonic_ns(): 48.8 ns +- 1.2 ns

Most clocks have the same performance except of perf_counter on Windows: around 140 ns whereas other clocks are around 45 ns (on Linux and Windows): 3x slower. Maybe the "bad" perf_counter performance can be explained by the fact that I'm running Windows in a VM, which is not ideal for benchmarking. Or maybe my C implementation of time.perf_counter() is slow?

Note: I expect that a significant part of the numbers are the cost of Python function calls. Reading these clocks using the Python C functions are likely faster.

Victor _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 10/15/2017 3:13 PM, Stephan Houben wrote:

Hi all,

I propose multiples of the Planck time, 5.39 × 10 ^−44 s. Unlikely computers can be more accurate than that anytime soon.

On a more serious note, I am not sure what problem was solved by moving from double to a fixed-precision format.

I do know that it now introduced the issue of finding the correct base unit of the fixed-precision format.

From Victor's original message, describing the current functions using 64-bit binary floating point numbers (aka double). They lose precision: "The problem is that Python returns time as a floatting point number which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days." Eric.

On Sun, Oct 15, 2017 at 08:04:25PM +0300, Koos Zevenhoven wrote:

On Sun, Oct 15, 2017 at 6:58 PM, Guido van Rossum wrote:

...

I'd like to have time.time_ns() -- this is most parallel to st_mtime_ns.

​ Welcome to the list Guido! You sound like a C programmer. For many people, that was the best language they knew of when they learned to program. But have you ever tried Python? You should give it a try!

You seem to be making a joke, but I have *no idea* what the joke is. Are you making fun of Guido's choice of preferred name? "time_ns" instead of "time_nanoseconds" perhaps? Other than that, I cannot imagine what the joke is about. Sorry for being so slow. -- Steve

It could be: time.time(ns=True) On Sun, Oct 15, 2017 at 7:44 PM, Guido van Rossum wrote:

Sorry, that's an in-joke. Koos is expressing his disappointment in the rejection of PEP 555 in a way that's only obvious if you're Dutch.

On Sun, Oct 15, 2017 at 3:16 PM, Steven D'Aprano wrote:

...

...

On Sun, Oct 15, 2017 at 6:58 PM, Guido van Rossum wrote:

...

I'd like to have time.time_ns() -- this is most parallel to st_mtime_ns.

​ Welcome to the list Guido! You sound like a C programmer. For many

On Sun, Oct 15, 2017 at 08:04:25PM +0300, Koos Zevenhoven wrote: people,

...

that was the best language they knew of when they learned to program. But have you ever tried Python? You should give it a try!

You seem to be making a joke, but I have *no idea* what the joke is.

Are you making fun of Guido's choice of preferred name? "time_ns" instead of "time_nanoseconds" perhaps?

Other than that, I cannot imagine what the joke is about. Sorry for being so slow.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Juancarlo *Añez*

On Sun, Oct 15, 2017 at 8:40 PM, Nick Coghlan wrote:

Hopefully by the time we decide it's worth worrying about picoseconds in "regular" code, compiler support for decimal128 will be sufficiently ubiquitous that we'll be able to rely on that as our 3rd generation time representation (where the first gen is seconds as a 64 bit binary float and the second gen is nanoseconds as a 64 bit integer).

I hope we'll never see time_ns() and friends as the second generation -- it's a hack that hopefully we can retire in those glorious days of hardware decimal128 support. -- --Guido van Rossum (python.org/~guido)

On Mon, Oct 16, 2017 at 08:40:33AM +0200, Stephan Houben wrote:

"The problem is that Python returns time as a floatting point number

...

which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days."

Do we realize that at this level of accuracy, relativistic time dilatation due to continental drift starts to matter?

tai64na has supported attoseconds for quite some time: https://cr.yp.to/libtai/tai64.html The relativity issue is declared to be out of scope for the document. :-) Stefan Krah

Stephan Houben wrote:

Do we realize that at this level of accuracy, relativistic time dilatation due to continental drift starts to matter?

Probably also want an accurate GPS position for your computer so that variations in the local gravitational field can be taken into account. -- Greg

Hi,

What if we had a class, say time.time_base. The user could specify the base units (such as "s", "ns", 1e-7, etc) and the data type ('float', 'int', 'decimal', etc.) when the class is initialized. (...)

It's easy to invent various funny new types for arbitrary precision. But I prefer reusing the standard Python int type since it's very well known and very easy to manipulate. There is not need to modify the whole stdlib to support a new type. Moreover, we don't have to "implement a new type".

The other advantage is that third-party module could potentially subclass this with additional options, such as an astronomy module providing an option to choose between sidereal time vs. solar time, without having to duplicate the entire API.

Using nanoseconds as int doesn't prevent you to convert it to your own nice class. Victor

I'm completely +1 to Victor. nanosecond integer timestamp is used these days. And no complex newtype or newmodule is not needed for supporting it. INADA Naoki On Fri, Oct 13, 2017 at 11:12 PM, Victor Stinner wrote:

Hi,

I would like to add new functions to return time as a number of nanosecond (Python int), especially time.time_ns().

It would enhance the time.time() clock resolution. In my experience, it decreases the minimum non-zero delta between two clock by 3 times, new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in Python.

The question of this long email is if it's worth it to add more "_ns" time functions than just time.time_ns()?

I would like to add:

* time.time_ns() * time.monotonic_ns() * time.perf_counter_ns() * time.clock_gettime_ns() * time.clock_settime_ns()

time(), monotonic() and perf_counter() clocks are the 3 most common clocks and users use them to get the best available clock resolution. clock_gettime/settime() are the generic UNIX API to access these clocks and so should also be enhanced to get nanosecond resolution.

== Nanosecond resolution ==

More and more clocks have a frequency in MHz, up to GHz for the "TSC" CPU clock, and so the clocks resolution is getting closer to 1 nanosecond (or even better than 1 ns for the TSC clock!).

The problem is that Python returns time as a floatting point number which is usually a 64-bit binary floatting number (in the IEEE 754 format). This type starts to loose nanoseconds after 104 days. Conversion from nanoseconds (int) to seconds (float) and then back to nanoseconds (int) to check if conversions loose precision:

# no precision loss

...

...

...

x=2**52+1; int(float(x * 1e-9) * 1e9) - x 0 # precision loss! (1 nanosecond) x=2**53+1; int(float(x * 1e-9) * 1e9) - x -1 print(datetime.timedelta(seconds=2**53 / 1e9)) 104 days, 5:59:59.254741

While a system administrator can be proud to have an uptime longer than 104 days, the problem also exists for the time.time() clock which returns the number of seconds since the UNIX epoch (1970-01-01). This clock started to loose nanoseconds since mid-May 1970 (47 years ago):

...

...

...

import datetime print(datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=2**53 / 1e9)) 1970-04-15 05:59:59.254741

== PEP 410 ==

Five years ago, I proposed a large and complex change in all Python functions returning time to support nanosecond resolution using the decimal.Decimal type:

https://www.python.org/dev/peps/pep-0410/

The PEP was rejected for different reasons:

* it wasn't clear if hardware clocks really had a resolution of 1 nanosecond, especially when the clock is read from Python, since reading a clock in Python also takes time...

* Guido van Rossum rejected the idea of adding a new optional parameter to change the result type: it's an uncommon programming practice (bad design in Python)

* decimal.Decimal is not widely used, it might be surprised to get such type

== CPython enhancements of the last 5 years ==

Since this PEP was rejected:

* the os.stat_result got 3 fields for timestamps as nanoseconds (Python int): st_atime_ns, st_ctime_ns, st_mtime_ns

* Python 3.3 got 3 new clocks: time.monotonic(), time.perf_counter() and time.process_time()

* I enhanced the private C API of Python handling time (API called "pytime") to store all timings as the new _PyTime_t type which is a simple 64-bit signed integer. The unit of _PyTime_t is not part of the API, it's an implementation detail. The unit is currently 1 nanosecond.

This week, I converted one of the last clock to new _PyTime_t format: time.perf_counter() now has internally a resolution of 1 nanosecond, instead of using the C double type.

XXX technically https://github.com/python/cpython/pull/3983 is not merged yet :-)

== Clocks resolution in Python ==

I implemented time.time_ns(), time.monotonic_ns() and time.perf_counter_ns() which are similar of the functions without the "_ns" suffix, but return time as nanoseconds (Python int).

I computed the smallest difference between two clock reads (ignoring a differences of zero):

Linux:

* time_ns(): 84 ns <=== !!! * time(): 239 ns <=== !!! * perf_counter_ns(): 84 ns * perf_counter(): 82 ns * monotonic_ns(): 84 ns * monotonic(): 81 ns

Windows:

* time_ns(): 318000 ns <=== !!! * time(): 894070 ns <=== !!! * perf_counter_ns(): 100 ns * perf_counter(): 100 ns * monotonic_ns(): 15000000 ns * monotonic(): 15000000 ns

The difference on time.time() is significant: 84 ns (2.8x better) vs 239 ns on Linux and 318 us (2.8x better) vs 894 us on Windows. The difference will be larger next years since every day adds 864,00,000,000,000 nanoseconds to the system clock :-) (please don't bug me with leap seconds! you got my point)

The difference on perf_counter and monotonic clocks are not visible in this quick script since my script runs less than 1 minute, my computer uptime is smaller than 1 weak, ... and Python internally starts these clocks at zero *to reduce the precision loss*! Using an uptime larger than 104 days, you would probably see a significant difference (at least +/- 1 nanosecond) between the regular (seconds as double) and the "_ns" (nanoseconds as int) clocks.

== How many new nanosecond clocks? ==

The PEP 410 proposed to modify the following functions:

* os module: fstat(), fstatat(), lstat(), stat() (st_atime, st_ctime and st_mtime fields of the stat structure), sched_rr_get_interval(), times(), wait3() and wait4()

* resource module: ru_utime and ru_stime fields of getrusage()

* signal module: getitimer(), setitimer()

* time module: clock(), clock_gettime(), clock_getres(), monotonic(), time() and wallclock() ("wallclock()" was finally called "monotonic", see PEP 418)

According to my tests of the previous section, the precision loss starts after 104 days (stored in nanoseconds). I don't know if it's worth it to modify functions which return "CPU time" or "process time" of processes, since most processes live shorter than 104 days. Do you care of a resolution of 1 nanosecond for the CPU and process time?

Maybe we need 1 nanosecond resolution for profiling and benchmarks. But in that case, you might want to implement your profiler in C rather in Python, like the hotshot module, no? The "pytime" private API of CPython gives you clocks with a resolution of 1 nanosecond.

== Annex: clock performance ==

To have an idea of the cost of reading the clock on the clock resolution in Python, I also ran a microbenchmark on *reading* a clock. Example:

$ ./python -m perf timeit --duplicate 1024 -s 'import time; t=time.time' 't()'

Linux (Mean +- std dev):

* time.time(): 45.4 ns +- 0.5 ns * time.time_ns(): 47.8 ns +- 0.8 ns * time.perf_counter(): 46.8 ns +- 0.7 ns * time.perf_counter_ns(): 46.0 ns +- 0.6 ns

Windows (Mean +- std dev):

* time.time(): 42.2 ns +- 0.8 ns * time.time_ns(): 49.3 ns +- 0.8 ns * time.perf_counter(): 136 ns +- 2 ns <=== * time.perf_counter_ns(): 143 ns +- 4 ns <=== * time.monotonic(): 38.3 ns +- 0.9 ns * time.monotonic_ns(): 48.8 ns +- 1.2 ns

Most clocks have the same performance except of perf_counter on Windows: around 140 ns whereas other clocks are around 45 ns (on Linux and Windows): 3x slower. Maybe the "bad" perf_counter performance can be explained by the fact that I'm running Windows in a VM, which is not ideal for benchmarking. Or maybe my C implementation of time.perf_counter() is slow?

Note: I expect that a significant part of the numbers are the cost of Python function calls. Reading these clocks using the Python C functions are likely faster.

Victor _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

w/r relativistic effects and continental drift - not really. The speed is about 1cm/yr or v = 1e-18 c. Relativistic effect would go like 0.5 * (v/c)**2, so more like 5E-37 in relative rate of proper time. You can just barely capture a few minutes of that even with int128 resolution. As for financial incentive for int128, considering the super-rich will get exponentially more rich while inflation devalues the possession of the rest, the super-rich will eventually have demand for int128 to be able to count their wealth down to the last ct. It's a lot for money, though. A lot more than the net value of all real things on the planet. But the net value of assets on people's bank accounts already exceeds that number by some factor 50-100, and this factor will grow exponentially as more financial "products" are being created. When I was a student (~1990) we had to create new bookkeeping software using int64 ("comp") because the investment company could not deal with billions of dollars worth of Italian Lira down to centimos otherwise. Accounting needs will get us int128, sooner than you think, don't worry. Yes, it would be good to create routines than can handle time in units provided, maybe as a string. Or you could just create a time object that handles this internally, e.g., algebraic operations, similar to numpy, at the accuracy available. You can access its value by providing the desired divisor, and you may inquires the available precision. If you want to handle this properly, it may get you back to interval arithmetics. Regarding concerns about the effect of gravity on your computer, there will be a press release in a few h that may be of interest to some https://www.ligo.caltech.edu/news/ligo20171011 -Alexander On 16 October 2017 at 18:53, Greg Ewing wrote:

Stephan Houben wrote:

...

Do we realize that at this level of accuracy, relativistic time dilatation due to continental drift starts to matter?

Probably also want an accurate GPS position for your computer so that variations in the local gravitational field can be taken into account.

-- Greg

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi, FYI I proposed the PEP 564 on python-dev. "PEP 564 -- Add new time functions with nanosecond resolution" https://www.python.org/dev/peps/pep-0564/ Victor

2017-10-16 9:46 GMT+02:00 Stephan Houben :

Hi all,

I realize this is a bit of a pet peeve of me, since in my day job I sometimes get people complaining that numerical data is "off" in the sixteenth significant digit (i.e. it was stored as a double). (...)

Oh. As usual, I suck at explaining the rationale. I'm sorry about that. The problem is not to know the number of nanoseconds since 1970. The problem is that you lose precision even on short durations, say less than 1 minute. The precision loss depends on the reference of the clock, which can be the UNIX epoch, a reference based on the computer boot time, a reference based on the process startup time, etc. Let's say that your web server runs since 105 days, now you want to measure how much time it took to render a HTML template and this time is smaller than 1 ms. In such case, the benchmark lose precision just because of the float type, not because of the clock resolution. That's one use case. Another use case is when you have two applications storing time. A program A writes a timestamp, another program B compares the timestamp to the current time. To explain the issue, let's say that the format used to store the timestamp and clock used by program A have a resolution of 1 nanosecond, whereas the clock used by program B has a resolution of 1 second. In that case, there is a window of 1 second where the time is seen as "created in the future". For example, the GNU tar program emits a warning in that case ("file created in the future", or something like that). More generally, if you store time with a resolution N and your clock has resolution P, it's better to have N >= P to prevent bad surprises. More and more databases and filesystems support storing time with nanosecond resolution. Victor

On Mon, Oct 16, 2017 at 4:10 PM, Victor Stinner wrote:

2017-10-16 9:46 GMT+02:00 Stephan Houben :

...

Hi all,

I realize this is a bit of a pet peeve of me, since in my day job I sometimes get people complaining that numerical data is "off" in the sixteenth significant digit (i.e. it was stored as a double). (...)

Oh. As usual, I suck at explaining the rationale. I'm sorry about that.

The problem is not to know the number of nanoseconds since 1970. The problem is that you lose precision even on short durations, say less than 1 minute. The precision loss depends on the reference of the clock, which can be the UNIX epoch, a reference based on the computer boot time, a reference based on the process startup time, etc.

Let's say that your web server runs since 105 days, now you want to measure how much time it took to render a HTML template and this time is smaller than 1 ms. In such case, the benchmark lose precision just because of the float type, not because of the clock resolution.

That's one use case.

Another use case is when you have two applications storing time. A program A writes a timestamp, another program B compares the timestamp to the current time. To explain the issue, let's say that the format used to store the timestamp and clock used by program A have a resolution of 1 nanosecond, whereas the clock used by program B has a resolution of 1 second. In that case, there is a window of 1 second where the time is seen as "created in the future". For example, the GNU tar program emits a warning in that case ("file created in the future", or something like that).

More generally, if you store time with a resolution N and your clock has resolution P, it's better to have N >= P to prevent bad surprises. More and more databases and filesystems support storing time with nanosecond resolution.

​Indeed. And some more on where the precision loss comes from: When you measure time starting from one point, like 1970, the timer reaches large numbers today, like 10**9 seconds. Tiny fractions of a second are especially tiny when compared to a number like that. You then need log2(10**9) ~ 30 bits of precision just to get a one-second resolution in your timer. A double-precision (64bit) floating point number has 53 bits of precision in the mantissa, so you end up with 23 bits of precision left for fractions of a second, which means you get a resolution of 1 / 2**23 seconds, which is about 100 ns, which is well in line with the data that Victor provided (~100 ns + overhead = ~200 ns). —Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven +

On 2017-10-16 16:42, MRAB wrote:

On 2017-10-16 13:30, Greg Ewing wrote:

...

Stephan Houben wrote:

...

Interestingly, that 2.2e-16 pretty much aligns with the accuracy of the cesium atomic clocks which are currently used to *define* the second. So we move to this new API, we should provide our own definition of the second, since those rough SI seconds are just too imprecise for that.

The Python second: 1.1544865564196655e-06 of the time taken for an unladen swallow to fly from Cape Town to London.

Is that an African or a European swallow?

It starts African, but it flips and can be observed as either African or European in London. It's a neutrino swallow. -- Thomas Jollans

2017-10-16 18:13 GMT+02:00 Todd :

I am not suggesting implementing a new numeric data type. People wouldn't use the class directly like they would an int or float, they would simply use it to define the the precision and numeric type (float, int, decimal). Then they would have access to the entire "time" API as methods. So for example you could do something like:

I tried to include your idea in a more general description of "different API": https://www.python.org/dev/peps/pep-0564/#different-api

...

...

...

import time

ns_time_int = time.time_base(units='ns', type=int) ms_time_dec = time.time_base(units=1e-6, type='Decimal') s_time_float= time.time_base(units=1, type=float) # This is identical to the existing "time" functions

ns_time_int.clock() 4978480000 ms_time_dec.clock() Decimal('5174165.999999999') s_time_float.clock() 5.276855

*I* dislike this API. IMHO it's overcomplicated just to get the current time :-( For example, I wouldn't like to have to teach to a newbie "how to get the current time" with such API. I also expect that the implementation will be quite complicated. Somewhere, you will need an internal "standard" type to store time, a "master type" used to build all other types. Without that, you would have to duplicate code and it would be a mess. You have many options for such master time. For the private C API of CPython, I already "implemented" such "master type": I decided to use C int64_t type: it's a 64-bit signed integer. There is an API on top of it which is unit agnostic, while it uses nanoseconds in practice. The private "pytime" API supports many timestamps conversions to adapt to all funny operating system functions: * from seconds: C int * from nanoseconds: C long long * from seconds: Python float or int * from milliseconds: Python float or int * to seconds: C double * to milliseconds: _PyTime_t * to microseconds: _PyTime_t * to nanoseconds: Python int * to timeval (struct timeval) * to timeval (time_t seconds, int us) * to timespec (struct timespec) At the end, I think that it's better to only provide the "master type" at the Python level, so nanoseconds as Python int. You can *easily* implement your API on top of the PEP 564. You will be limited to nanosecond resolution, but in practice, all clocks are already limited to this resolution through operating system APIs: https://www.python.org/dev/peps/pep-0564/#sub-nanosecond-resolution Victor

On Mon, Oct 16, 2017 at 6:12 PM, Guido van Rossum wrote:

Sorry about your frustration. I should not have given you hope about PEP 555 -- it was never going to make it, so I should just have spared you the effort. Do you want to withdraw it or do I have to actually reject it?

​That's definitely not the reason for the frustration. You already told me earlier that you did not really like it. I think I know why you didn't, but we now have more reasons than we had then. We're definitely in a different place from where we started. And I still succeeded in the most important thing that needed to be done. The only thing is that, at least for now, nobody *really* needed the awesome, clean, flexible and super performant solution that I had to come up with to solve a wide class of problems which I thought many people would actually have. But that's not where the frustration came from either. I'll finish the PEP for archiving, and for throwing it on the pile of other PEPs that have tried to tackle these issues. -- Koos P.S. Of course this discussion really does not belong here, but at least the time_ns discussion is now on python-dev anyway.

On Mon, Oct 16, 2017 at 1:56 AM, Koos Zevenhoven wrote:

...

On Mon, Oct 16, 2017 at 2:44 AM, Guido van Rossum wrote:

...

Sorry, that's an in-joke. Koos is expressing his disappointment in the rejection of PEP 555 in a way that's only obvious if you're Dutch.

Hmm, or more accurately, it has to do with me going crazy because of the frustration of how the PEP 555 discussions went. We could have arrived at the same conclusion [*] in much less time and with less pulling one's hair out.

But this discrepancy probably comes from the fact that we're not dealing with the most pure kind of being Dutch here.

—Koos

​[*] Although right now different people still have slightly different ideas about what that conclusion is. ​

...

On Sun, Oct 15, 2017 at 3:16 PM, Steven D'Aprano wrote:

...

...

On Sun, Oct 15, 2017 at 6:58 PM, Guido van Rossum wrote:

...

I'd like to have time.time_ns() -- this is most parallel to st_mtime_ns.

​ Welcome to the list Guido! You sound like a C programmer. For many

On Sun, Oct 15, 2017 at 08:04:25PM +0300, Koos Zevenhoven wrote: people,

...

that was the best language they knew of when they learned to program. But have you ever tried Python? You should give it a try!

You seem to be making a joke, but I have *no idea* what the joke is.

Are you making fun of Guido's choice of preferred name? "time_ns" instead of "time_nanoseconds" perhaps?

Other than that, I cannot imagine what the joke is about. Sorry for being so slow.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- + Koos Zevenhoven + http://twitter.com/k7hoven +

-- --Guido van Rossum (python.org/~guido)

-- + Koos Zevenhoven + http://twitter.com/k7hoven +

Antoine Pitrou schrieb am 16.10.2017 um 10:20:

On Sun, 15 Oct 2017 22:00:10 -0700 Guido van Rossum wrote:

...

On Sun, Oct 15, 2017 at 8:40 PM, Nick Coghlan wrote:

...

Hopefully by the time we decide it's worth worrying about picoseconds in "regular" code, compiler support for decimal128 will be sufficiently ubiquitous that we'll be able to rely on that as our 3rd generation time representation (where the first gen is seconds as a 64 bit binary float and the second gen is nanoseconds as a 64 bit integer).

I hope we'll never see time_ns() and friends as the second generation -- it's a hack that hopefully we can retire in those glorious days of hardware decimal128 support.

Given the implementation costs, hardware decimal128 will only become mainstream if there's a strong incentive for it, which I'm not sure exists or will ever exist ;-)

Then we shouldn't implement the new nanosecond API at all, in order to keep pressure on the hardware developers. Stefan :o)

Hi all, Please excuse me for getting a bit off-topic, but I would like to point out that except for bean-counters who need to be bug-compatible with accounting standards, decimal floating point is generally a bad idea. That is because the worst-case bound on the rounding error grows linear with the base size. So you really want to choose a base size as small as possible, i.e., 2. This is not really related to the fact that computers use base-2 arithmetic, that is just a happy coincidence. If we used ternary logic for our computers, FP should still be based on base-2 and computer architects would complain about the costly multiplication and division with powers of two (just as they have historically complained about the costly implementation of denormals, but we still got that, mostly thanks to prof. Kahan convincing Intel). Worse, a base other than 2 also increases the spread in the average rounding error. This phenomenon is called "wobble" and adds additional noise into calculations. The ultimate problem is that the real number line contains quite a few more elements than our puny computers can handle. There is no 100% solution for this, but of all the possible compromises, floating-point forms a fairly optimum point in the design space for a wide range of applications. Stephan Op 20 okt. 2017 3:13 p.m. schreef "Victor Stinner"

:

Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi francis, Le 20 oct. 2017 18:42, "francismb" a écrit : Hi Victor, On 10/13/2017 04:12 PM, Victor Stinner wrote:

I would like to add:

* time.time_ns() * time.monotonic_ns() * time.perf_counter_ns() * time.clock_gettime_ns() * time.clock_settime_ns() due nano vs. pico vs. ...

why not something like (please change '_in' for what you like): time.time_in(precision) time.monotonic_in(precision) If you are not aware yet, I wrote a full PEP: PEP 564. The two discussed idea are alreafy listed in the PEP, configurable precision and sub-nanosecond resolution. I tried to explain why I proposed time.time_ns() in detail in the PEP: https://www.python.org/dev/peps/pep-0564/#sub-nanosecond-resolution You may want to join the discussion on python-dev. Victor