Hi all,

I hope I’m not in danger of abusing this mailing list - please let me know if this is too much noise.

I’m very happy to share that 3 more pathlib PRs have landed - #18909, #25240 and #25277. Thanks for the reviews - I appreciate reviewer time is scarce!

I may be pushing my luck here, but there are just two PRs outstanding before work on AbstractPath can begin in earnest.

I would love for these to land before the 3.10 feature freeze so that we can spend the entire 3.11 development cycle getting the design/implementation of AbstractPath exactly right. The PRs are:

#25271 / bpo-42998 - Add ‘user’ parameter to Path.home()
- A simple patch. No reviewers yet
#25264 / bpo-43757 - Make Path.resolve() use os.path.realpath() to resolve all symlinks in a path
- A more complex patch with several contributions from reviewers
- Adds keyword-only ‘strict’ param to realpath()
- Now backwards compatible! No longer changes default realpath() behaviour

If anyone can spare the time over the next week to help get these patches over the line, I would really appreciate it!

A quick elevator pitch for why AbstractPath is an important future feature:

pathlib is a wonderful library that beautifully shows off Python’s “everything is an object” approach. It’s rightly front-and-centre of many Python beginner tutorials. If we can extend that “everything” to include files in tarballs, zip files, disc images, FTP servers, S3, etc, it allows Python users to easily apply their pathlib knowledge to new (and common!) tasks.
In DevOps it’s common to deal with remote or embedded filesystems. There’s a need for pathlib-like interfaces, and that need is already met in popular third-party packages either by emulating Path APIs (often with subtle differences) or hackily extending Path/_Accessor/_Flavour (which is brittle and totally unsupported). Even in the standard library, zipfile.Path is a mostly-compatible emulation of pathlib.Path. Providing a supported route for extending an AbstractPath saves a bunch of heartache and makes the wider pathlib ecosystem more robust.

Thanks again to everyone who has contributed their time and expertise up to this point.

Cheers

Barney

On Fri, 9 Apr 2021 at 01:17, Barney Gale <barney.gale@gmail.com> wrote:

Thanks so much to everyone who discussed and reviewed the code and made suggestions.

The bulk of these patches have now landed. For those following along at home, here’s a summary of the remaining PRs/bugs :

#18909 / bpo-39950: Add Path.hardlink_to() method that supersedes link_to()

Note: already under core review

#25271 / bpo-42998: Add ‘user’ parameter to Path.home()

#25240 / bpo-40107: Stop using os.open() to implement Path.open()

#25264 / bpo-43757: Make Path.resolve() use os.path.realpath() to resolve all symlinks in a path

Note: changes realpath() behaviour when symlink loops are encountered (!), and adds a keyword-only 'strict' param.

#25277 / bpo-39899: Follow-up fix for oversight in #18841

I believe this is a complete account of the outstanding prep work. Once these land, pathlib’s path, flavour and accessor abstractions will be less leaky, which allows work on pathlib.AbstractPath to proceed.

Thanks again to Antoine Pitrou, Brett Cannon, Serhiy Storchaka, Steve Dower and Eryk Sun for their invaluable contributions.

Barney

On Sat, 3 Apr 2021 at 03:02, Barney Gale <barney.gale@gmail.com> wrote:
I’m working towards supporting subclassing pathlib.Path for things like zip/tar archives, iso files, S3, etc. This was first brought up a few years ago in bpo-24132 and discussed further on the python-ideas forum.

Before I can approach the meat of the issue, I'd like to land a handful of backwards-compatible patches that tidy up the pathlib internals. These are:

#19220 / bpo-39924: Handle missing os functions more consistently in pathlib

#18838 / bpo-39895: Move pathlib.Path.touch() implementation into the path accessor

#19342 / bpo-40038: Remove partial support for preserving accessor when modifying a path

#18841 / bpo-39899: Make pathlib use os.path.expanduser() to expand home directories

#18834 / bpo-39659: Route calls from pathlib.Path to os.getcwd() via the path accessor

#18864 / bpo-39906: Add follow_symlinks parameter to pathlib.Path.stat() and chmod()

#24294 / bpo-42999: Expand and clarify pathlib.Path.link_to() documentation

Could I please request review of these patches? A couple may already be ready to land.

(I appreciate they’re a little dry. This is all prep work for the introduction of a shiny new class, tentatively named ‘AbstractPath’).

Thanks!

Barney