Hi all,
I hope I’m not in danger of abusing this mailing list - please let me know
if this is too much noise.
I’m very happy to share that 3 more pathlib PRs have landed - #18909
https://github.com/python/cpython/pull/18909, #25240
https://github.com/python/cpython/pull/25240 and #25277
https://github.com/python/cpython/pull/25277. Thanks for the reviews - I
appreciate reviewer time is scarce!
I may be pushing my luck here, but there are *just two* PRs outstanding
before work on AbstractPath can begin in earnest.
I would love for these to land before the 3.10 feature freeze so that we
can spend the entire 3.11 development cycle getting the
design/implementation of AbstractPath exactly right. The PRs are:
- #25271 https://github.com/python/cpython/pull/25271 / bpo-42998
https://bugs.python.org/issue42998 - Add ‘user’ parameter to
Path.home()
- A simple patch. No reviewers yet
- #25264 https://github.com/python/cpython/pull/25264 / bpo-43757
https://bugs.python.org/issue43757 - Make Path.resolve() use
os.path.realpath() to resolve all symlinks in a path
- A more complex patch with several contributions from reviewers
- Adds keyword-only ‘strict’ param to realpath()
- Now backwards compatible! No longer changes default realpath()
behaviour
If anyone can spare the time over the next week to help get these patches
over the line, I would really appreciate it!
A quick elevator pitch for why AbstractPath is an important future feature:
- pathlib is a wonderful library that beautifully shows off Python’s
“everything is an object” approach. It’s rightly front-and-centre of many
Python beginner tutorials. If we can extend that “everything” to include
files in tarballs, zip files, disc images, FTP servers, S3, etc, it allows
Python users to easily apply their pathlib knowledge to new (and common!)
tasks.
- In DevOps it’s common to deal with remote or embedded filesystems.
There’s a need for pathlib-like interfaces, and that need is already met in
popular third-party packages either by *emulating* Path APIs (often with
subtle differences) or *hackily extending* Path/_Accessor/_Flavour
(which is brittle and totally unsupported). Even in the standard library,
zipfile.Path is a mostly-compatible emulation of pathlib.Path. Providing a
supported route for extending an AbstractPath saves a bunch of heartache
and makes the wider pathlib ecosystem more robust.
Thanks again to everyone who has contributed their time and expertise up to
this point.
Cheers
Barney
On Fri, 9 Apr 2021 at 01:17, Barney Gale
Thanks so much to everyone who discussed and reviewed the code and made suggestions.
The bulk of these patches have now landed. For those following along at home, here’s a summary of the remaining PRs/bugs :
- #18909 https://github.com/python/cpython/pull/18909 / bpo-39950 https://bugs.python.org/issue39950: Add Path.hardlink_to() method that supersedes link_to() - Note: already under core review - #25271 https://github.com/python/cpython/pull/25271 / bpo-42998 https://bugs.python.org/issue42998: Add ‘user’ parameter to Path.home() - #25240 https://github.com/python/cpython/pull/25240 / bpo-40107 https://bugs.python.org/issue40107: Stop using os.open() to implement Path.open() - #25264 https://github.com/python/cpython/pull/25264 / bpo-43757 https://bugs.python.org/issue43757: Make Path.resolve() use os.path.realpath() to resolve all symlinks in a path - Note: changes realpath() behaviour when symlink loops are encountered (!), and adds a keyword-only 'strict' param. - #25277 https://github.com/python/cpython/pull/25277 / bpo-39899 https://bugs.python.org/issue39899: Follow-up fix for oversight in #18841 https://github.com/python/cpython/pull/18841
I believe this is a complete account of the outstanding prep work. Once these land, pathlib’s *path*, *flavour* and *accessor* abstractions will be less leaky, which allows work on pathlib.AbstractPath to proceed.
Thanks again to Antoine Pitrou, Brett Cannon, Serhiy Storchaka, Steve Dower and Eryk Sun for their invaluable contributions.
Barney
On Sat, 3 Apr 2021 at 03:02, Barney Gale
wrote: I’m working towards supporting subclassing pathlib.Path for things like zip/tar archives, iso files, S3, etc. This was first brought up a few years ago in bpo-24132 https://bugs.python.org/issue24132 and discussed further on the python-ideas forum https://discuss.python.org/t/make-pathlib-extensible/3428.
Before I can approach the meat of the issue, I'd like to land a handful of backwards-compatible patches that tidy up the pathlib internals. These are:
- #19220 https://github.com/python/cpython/pull/19220 / bpo-39924 https://bugs.python.org/issue39924: Handle missing os functions more consistently in pathlib - #18838 https://github.com/python/cpython/pull/18838 / bpo-39895 https://bugs.python.org/issue39895: Move pathlib.Path.touch() implementation into the path accessor - #19342 https://github.com/python/cpython/pull/19342 / bpo-40038 https://bugs.python.org/issue40038: Remove partial support for preserving accessor when modifying a path - #18841 https://github.com/python/cpython/pull/18841 / bpo-39899 https://bugs.python.org/issue39899: Make pathlib use os.path.expanduser() to expand home directories - #18834 https://github.com/python/cpython/pull/18834 / bpo-39659 https://bugs.python.org/issue39659: Route calls from pathlib.Path to os.getcwd() via the path accessor - #18864 https://github.com/python/cpython/pull/18864 / bpo-39906 https://bugs.python.org/issue39906: Add follow_symlinks parameter to pathlib.Path.stat() and chmod() - #24294 https://github.com/python/cpython/pull/24294 / bpo-42999 https://bugs.python.org/issue42999: Expand and clarify pathlib.Path.link_to() documentation
Could I please request review of these patches? A couple may already be ready to land.
(I appreciate they’re a little dry. This is all prep work for the introduction of a shiny new class, tentatively named ‘AbstractPath’).
Thanks!
Barney