If the precision is available via OS APIs, this is mostly an issue+PR away from being implemented by someone who cares. FAT32 has a two billion nanosecond resolution IIRC. :P -gps On Mon, Dec 7, 2020 at 8:22 AM David Mertz wrote:

Are there any filesystems that can actually record a meaningful ns modification time? I find discussions claiming this:

- XFS and EXT3: second precision - EXT4: millisecond precision - NTFS: 100ns precision - APFS: 1 ns precision

But also notes that the precision is likely to exceed the accuracy by many times on real systems.

On Mon, Dec 7, 2020 at 2:34 PM Mats Wichmann wrote:

...

there are stat fields now for ns precision, e.g. st_mtime now has an analogue st_mtime_ns. But os.path didn't grow corresponding methods - there's an os.path.getmtime but not _ms. Was that intentional? The wrappers in genericpath.py are trivial and arguably aren't particularly needed, but still curious... _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CK3S2DYI... Code of Conduct: http://python.org/psf/codeofconduct/

-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZAPQSEQH... Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Dec 7, 2020 at 10:52 AM Antoine Pitrou wrote:

On Mon, 7 Dec 2020 16:19:02 +0000 David Mertz wrote:

...

Are there any filesystems that can actually record a meaningful ns modification time? I find discussions claiming this:

- XFS and EXT3: second precision - EXT4: millisecond precision - NTFS: 100ns precision - APFS: 1 ns precision

But also notes that the precision is likely to exceed the accuracy by many times on real systems.

Even if the accuracy is much lower than that, it can be important to reproduce exact timestamps.

EG GNU tar --diff

Le lun. 7 déc. 2020 à 17:22, David Mertz a écrit :

Are there any filesystems that can actually record a meaningful ns modification time? I find discussions claiming this:

EXT4: millisecond precision

EXT4 and BTRFS have a resolution of 1 nanosecond.

XFS and EXT3: second precision

XFS is going to support a resolution of 1 nanosecond in the incoming Linux kernel 5.10: https://www.phoronix.com/scan.php?page=news_item&px=XFS-Linux-5.10

NTFS: 100ns precision APFS: 1 ns precision

But also notes that the precision is likely to exceed the accuracy by many times on real systems.

On my Fedora 33 with Linux 5.9 and Python 3.9, time.time_ns() has an effective resolution around 71 nanoseconds (smallest difference which can be measured in user space): ----------------- import time t0 = time.time_ns() endtime = t0 + 5 * 1000_000_000 previous = t0 res = None while True: t1, t2 = (time.time_ns(), time.time_ns()) if t1 >= endtime: break dt = t2 - t1 if not dt: dt = t1 - previous if not dt: continue if res is None or dt < res: res = dt previous = t2 print(f"time.time_ns() resolution: {res} nanoseconds") ----------------- *Reading* the system clock in Python (as nanoseconds) already takes around 50 nanoseconds! $ python3 -m pyperf timeit -s 'import time; func=time.time_ns' --duplicate=1024 'func()' Mean +- std dev: 49.8 ns +- 1.2 ns The IEEE-754 floating point number format is a weird beast. A number "resolution" depends on its value. IMO this article well explains that: https://fabiensanglard.net/floating_point_visually_explained/ Python float resolution (using Unix epoch, 1970-01-01 at 00:00): * 1970-01-01 00:00 (0 seconds) => res = 4.9e-315 ns * 1970-01-02 00:00 (1 day) => res = 0.01 ns * 1970-01-01 00:00 (31 days) => res = 0.5 ns * 1970-04-09 00:00 (98 days) => res = 1.9 ns * 2020-12-07 00:00 (18,603 days) => res = 238.4 ns <=== TODAY Note: Windows uses 1601-01-01 Epoch for its FILETIME timestamp, but hopefully os.stat() shifts the value to the Unix Epoch (otherwise, the resolution would be way worse on Windows!). The resolution got worse than 1 nanosecond since April 9th, 1970. Today a timestamp using the Unix epoch has a resolution worse than 238 ns. See PEP 564 which added "_ns" functions to the time module to Python 3.7, like time.time_ns(): https://www.python.org/dev/peps/pep-0564/

...

...

import datetime, match epoch=datetime.datetime(1970, 1, 1, tzinfo=datetime.timezone.utc) t=epoch; ts=t.timestamp(); (t, ts, math.ulp(ts)*1e9) (datetime.datetime(1970, 1, 1, 0, 0, tzinfo=datetime.timezone.utc), 0.0, 4.94065646e-315) t=epoch+datetime.timedelta(days=1); ts=t.timestamp(); (t, ts, math.ulp(ts)*1e9) (datetime.datetime(1970, 1, 2, 0, 0, tzinfo=datetime.timezone.utc), 86400.0, 0.014551915228366852) t=epoch+datetime.timedelta(days=31); ts=t.timestamp(); (t, ts, math.ulp(ts)*1e9) (datetime.datetime(1970, 2, 1, 0, 0, tzinfo=datetime.timezone.utc), 2678400.0, 0.46566128730773926) t=epoch+datetime.timedelta(days=98); ts=t.timestamp(); (t, ts, math.ulp(ts)*1e9) (datetime.datetime(1970, 4, 9, 0, 0, tzinfo=datetime.timezone.utc), 8467200.0, 1.862645149230957) t=datetime.datetime(2020, 12, 7, tzinfo=datetime.timezone.utc); ts=t.timestamp(); (t, ts, math.ulp(ts)*1e9) (datetime.datetime(2020, 12, 7, 0, 0, tzinfo=datetime.timezone.utc), 1607299200.0, 238.4185791015625)

Conclusion: store timestamp as an integer number of nanoseconds, not as a float ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death.