Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
On Mar 23, 2018, at 7:26 AM, Steve Dower <steve.dower@python.org> wrote: I had a colleague complaining to me the other day about having to search multiple packages for the right function to move a file (implying: with the same semantics as drag-drop). Thanks Steve — I know there isn’t any kind of consensus about exactly what should be part of a “most file operations” package/class, but I’ve found it odd that there doesn’t even seem to be acknowledgment among longtime python users that the current hodgepodge is pretty dysfunctional for new users. One thing that’s been hammered home for me teaching newbies is that people get very far with computing these days without ever having touched a command line — certainly not a *nix one. Even the concept of a working directory is foreign to many. So telling them that they can create and manipulate paths in nifty ways with the Path object, but then telling them to go to the os module to do simple things like rename or delete a file, and then looking at the docs for the os module, which starts with a bunch of references to other modules you need for not quite as simple things (shutil) or more complex things (tempfile). Then you look through the module and see a LOT of low level cryptic and semi-platform dependent functions — not newbie friendly. Heck, I’ve been using Python for almost 20 years and I still have go look up whether it’s “delete” or “remove” or “unlink”. And there is so much in os that ipython or IDE completion doesn’t help much. So as not to only write a critical rant — here is a proposal of sorts: The python standard library should have one-stop shopping for the basic file system manipulations. “Basic” could be guided by “what users typically do with a GUI file manager” This could be yet another module, but given that navigating the file system ( I.e. path manipulation) is one of the things people typically do with a file manager, it makes sense to add it all to pathlib, maybe even all to the Path object. ( though maybe more advanced classes/functions could be added to pathlib as way to put it all in one place, while not expanding the Path namespace too much) Of course, this will require a substantial amount of work to work out the details in a PEP. I don’t have the time to do that, but if the OP or someone else does, I’ll help. -CHB If there isn’t a pathtools library on PyPI yet, this would certainly be valuable for newer developers. My view on Path is to either have everything on it or nothing on it (without removing what’s already there, of course), and since everything is so popular we should at least put everything in the one place. Top-posted from my Windows phone *From: *Mike Miller <python-ideas@mgmiller.net> *Sent: *Monday, March 19, 2018 10:51 *To: *python-ideas@python.org *Subject: *Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2 On 2018-03-18 10:55, Paul Moore wrote:
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a
path). Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc? _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I find this odd too. There have been a few comments along the lines of this being a problem for newbies or for *some* people (not sure who *some* is referring to), but I think that having these functions in different modules is weird for everyone. It just so happens that if you use file-operations a lot that you've gotten used to the system and know where to look, but it's still weird to have to use 2-3 different modules. Additionally, if you know where to look then it isn't really a hassle to import another module, whereas it is a hassle if you are less familiar with the functionality (regardless of whether you are a newbie or not).
Me too, although I've only been using python for ~ 10 years.
I'd be happy to help as well, and if I can find the time I'll write the PEP (although I'd rather just contribute to it). Figuring out what to include and how to change the methods' APIs seems like one of the easier PEPs to write in terms of required knowledge (I don't know much about python's C code), so hopefully I would have a chance at doing an adequate job. If you think not, please tell me and I won't bother :) - Jason
On 2018-03-23 13:47, Jason Maldonis wrote:
I'll acknowledge it too, just that after X years it becomes second nature as you know. Tinkered around at lunchtime with a module to bring them under one roof. Named it "f2" because "fs" was taken on PyPI and thought the name should be very short since it could potentially be used in every script, like os and sys are. https://github.com/mixmastamyk/f2 Just poking around these questions came up about what to include: - Include file descriptors apis? - chroot? - l- functions to not follow links? - Unix dev files? obscure - pathconf? - supports_* api? - Only scandirs or walk too? - Should unpack os.path functions into the root? Handy but many of them. - path and Path in same module could confuse. - Unpack file-related shutil functions? Or leave as submodule? Perhaps this is enough to generate some comments. -Mike
Here's a comparison table of os, os.path, shutil, pathlib, and path.py. The full version is at https://github.com/westurner/pyfilemods (README.rst) and at https://westurner.github.io/pyfilemods. I ran a few set intersections and went ahead and wrote a report to compare function/method signatures and sources. (cc'ed ironically, for the last time, I promise, from the other fork of this thread: https://mail.python.org/pipermail/python-ideas/2018-March/049375.html ) ... trio wraps pathlib methods with async; which also might as well be done in pathlib? - Does it make sense to copy the docstrings at import time every time? attr table ========== ================== == ======= ====== ======= ======= attr os os.path shutil pathlib path.py ================== == ======= ====== ======= ======= `__div__`_ X `__rdiv__`_ X `absolute`_ X `abspath`_ X X `access`_ X X `altsep`_ X X `anchor`_ X `as_posix`_ X `as_uri`_ X `atime`_ X `basename`_ X X `bytes`_ X `capitalize`_ X `casefold`_ X `cd`_ X `center`_ X `chdir`_ X X `chmod`_ X X X `chown`_ X X X `chroot`_ X X `chunks`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copy2`_ X X `copyfile`_ X X `copymode`_ X X `copystat`_ X X `copytree`_ X X `count`_ X `ctime`_ X `curdir`_ X X `cwd`_ X `defpath`_ X X `devnull`_ X X `dirname`_ X X `dirs`_ X `drive`_ X X `encode`_ X `endswith`_ X `exists`_ X X X `expand`_ X `expandtabs`_ X `expanduser`_ X X X `expandvars`_ X X `ext`_ X `extsep`_ X X `files`_ X `find`_ X `fnmatch`_ X X `format`_ X `format_map`_ X `get_owner`_ X `getatime`_ X X `getctime`_ X X `getcwd`_ X X `getmtime`_ X X `getsize`_ X X `glob`_ X X `group`_ X `home`_ X `in_place`_ X `index`_ X `is_absolute`_ X `is_block_device`_ X `is_char_device`_ X `is_dir`_ X `is_fifo`_ X `is_file`_ X `is_reserved`_ X `is_socket`_ X `is_symlink`_ X `isabs`_ X X `isalnum`_ X `isalpha`_ X `isdecimal`_ X `isdigit`_ X `isdir`_ X X `isfile`_ X X `isidentifier`_ X `islink`_ X X `islower`_ X `ismount`_ X X `isnumeric`_ X `isprintable`_ X `isspace`_ X `istitle`_ X `isupper`_ X `iterdir`_ X `join`_ X X `joinpath`_ X X `lchmod`_ X `lexists`_ X `lines`_ X `link`_ X X `listdir`_ X X `ljust`_ X `lower`_ X `lstat`_ X X X `lstrip`_ X `makedirs`_ X X `makedirs_p`_ X `maketrans`_ X `match`_ X `merge_tree`_ X `mkdir`_ X X X `mkdir_p`_ X `module`_ X `move`_ X X `mtime`_ X `name`_ X X X `namebase`_ X `normcase`_ X X `normpath`_ X X `open`_ X X X `os`_ X X `owner`_ X X `pardir`_ X X `parent`_ X X `parents`_ X `partition`_ X `parts`_ X `pathconf`_ X X `pathsep`_ X X `read_bytes`_ X `read_hash`_ X `read_hexhash`_ X `read_md5`_ X `read_text`_ X `readlink`_ X X `readlinkabs`_ X `realpath`_ X X `relative_to`_ X `relpath`_ X X `relpathto`_ X `remove`_ X X `remove_p`_ X `removedirs`_ X X `removedirs_p`_ X `rename`_ X X X `renames`_ X X `replace`_ X X X `resolve`_ X `rfind`_ X `rglob`_ X `rindex`_ X `rjust`_ X `rmdir`_ X X X `rmdir_p`_ X `rmtree`_ X X `rmtree_p`_ X `root`_ X `rpartition`_ X `rsplit`_ X `rstrip`_ X `samefile`_ X X X `sameopenfile`_ X `samestat`_ X `sep`_ X X `size`_ X `special`_ X `split`_ X X `splitall`_ X `splitdrive`_ X X `splitext`_ X X `splitlines`_ X `splitpath`_ X `splitunc`_ X `startswith`_ X `stat`_ X X X X X `statvfs`_ X X `stem`_ X X `strip`_ X `stripext`_ X `suffix`_ X `suffixes`_ X `swapcase`_ X `symlink`_ X X `symlink_to`_ X `text`_ X `title`_ X `touch`_ X X `translate`_ X `uncshare`_ X `unlink`_ X X X `unlink_p`_ X `upper`_ X `using_module`_ X `utime`_ X X `walk`_ X X `walkdirs`_ X `walkfiles`_ X `with_name`_ X `with_suffix`_ X X `write_bytes`_ X X `write_lines`_ X `write_text`_ X X `zfill`_ X ================== == ======= ====== ======= ======= On Fri, Mar 23, 2018 at 6:22 PM, Mike Miller <python-ideas@mgmiller.net> wrote:
On Fri, Mar 23, 2018 at 3:38 PM, Wes Turner <wes.turner@gmail.com> wrote:
Here's a comparison table of os, os.path, shutil, pathlib, and path.py.
Darn, that's a big list -- exactly what we want to avoid :-( Though there are bunch of string methods in there that we can dump right off the bat.
... trio wraps pathlib methods with async; which also might as well be done in pathlib?
hmm --that's s touch one -- a full set of async file operations would be great. But where to put them???? and most of what we are talking about doesn't need asnyc, does it? deleting/renaming a single, file, etc. So my thought is that an asnyc lib should mirror / overwrite this file-operations-api. Which makes it a problem for the async lib(s) later on down the road. BTW -- this is a nice argument for making this API in the first place -- then we have a single API to make an async version of! - Does it make sense to copy the docstrings at import time every time? copy from where? is this for the async version? or are you thinking that the new methods will simply be wrappers for the olds ones -- in which case, I think no -- this should be a new API with its own docs, whether or not a particular call is any different that the "old" one. The big challenge is to identity what is "basic" or "normal" ratehr than advanced. here's a quick take on trimming down that whole table -- jsut my quick HO: attr table ========== ================== == ======= ====== ======= ======= attr os os.path shutil pathlib path.py ================== == ======= ====== ======= ======= `absolute`_ X `abspath`_ X X `access`_ X X `atime`_ X `basename`_ X X `cd`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copytree`_ X X `curdir`_ X X `dirname`_ X X `drive`_ X X `exists`_ X X X `expanduser`_ X X X `expandvars`_ X X `ext`_ X `fnmatch`_ X X `get_owner`_ X `getatime`_ X X `getctime`_ X X `getcwd`_ X X `getmtime`_ X X `getsize`_ X X `glob`_ X X `home`_ X `is_absolute`_ X `is_dir`_ X `is_symlink`_ X `isabs`_ X X `isdir`_ X X `isfile`_ X X `islink`_ X X `iterdir`_ X `lchmod`_ X `link`_ X X `listdir`_ X X `lstat`_ X X X `mkdir`_ X X X `makedirs`_ X X `match`_ X `merge_tree`_ X `move`_ X X `mtime`_ X `normcase`_ X X `normpath`_ X X `open`_ X X X `owner`_ X X `parent`_ X X `parents`_ X `relative_to`_ X `relpath`_ X X `relpathto`_ X `remove`_ X X `removedirs`_ X X `rename`_ X X X `replace`_ X X X `resolve`_ X `rglob`_ X `root`_ X `samefile`_ X X X `size`_ X `suffix`_ X `suffixes`_ X `symlink`_ X X `symlink_to`_ X `touch`_ X X `utime`_ X X `walk`_ X X `with_name`_ X `with_suffix`_ X X `write_bytes`_ X X `write_text`_ X X These are about permissions -- challenging how to do that in a X-platform way... And it would be good to group this sort of thing together. `chmod`_ X X X `chown`_ X X X ================== == ======= ====== ======= ======= Now that I've done that, I think it would be better to be systematic: 1) keep every method that pathlib.Path has 2) dump every method this is only a string method (and maybe everything that path has that nothign else has -- most of those are string methods 3) match the ones that are the same thing, but different names 4) match the ones that are almost the same thing:remove and removedir and removedirs - then we can hash out whether to join with flags, or keep separate 5) group by "class of functionality: - file nameing, etc - seraching (glob) - permission handling - etc. That'll get it structured to capture the discussion. -CHB
This is a great idea to have a prototype of sorts, but I do'nt think we should take the approach of: make it, put it on pypi, and see if it gains traction -- this isn't adding anythign new, very few folks are going to want to crate a dependency to get a more discoverable api, and no one going to point newbies at it. Just poking around these questions came up about what to include:
- Include file descriptors apis?
no -- unless in an "advanced" api somewhere - chroot?
no
- l- functions to not follow links?
maybe adanced, or optional flags
- Unix dev files? obscure
don't even know what this is, so no :-) - pathconf?
again, what is this? probabaly no.
- supports_* api?
no.
- Only scandirs or walk too?
walk is really useful and not trivial to write, so yes.
- Should unpack os.path functions into the root? Handy but many of them.
no way.
- Unpack file-related shutil functions? Or leave as submodule?
make them methods on Path. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi folks, according to comments it seems, it would better to propose a new module. I start to work out a new proposal. I created a BitBucket repo to be able to work together with folks who want help ;-) the repo address is: https://bitbucket.org/GeorgeFischhof/pep_for_filesystem_operations.git -- o -- shutil and path object relation: the documentation laks the information. For functions of os module it is written that functions updated in 3.6 and from that point they can use pathlib objects. I will check the status. (if the work together I will submit an issue report to update the documentation accordingly) BR, George
On 23 March 2018 at 22:22, Mike Miller <python-ideas@mgmiller.net> wrote:
It's certainly easy to get so used to the current state of affairs that you forget that it's not "obvious" to a newcomer. I've been teaching a colleague Python and it's certainly something he would have problems with.
Yeah, this is the real issue. It's not that the general principle "have a single place where all filesystem stuff is found" is a particularly bad one, it's more that it's genuinely hard to know what the right answer is here. I'll add some other questions: - Should the module only contain cross-platform functions, or are platform-specific ones OK? - How about operations that appear on multiple levels (Steve Dower mentioned copy "that works like drag and drop", there's also about delete that handles recycle bins vs lower-level delete)? and the big one: - How will we handle backward compatibility? This is why a generic proposal like the one that started this thread is pretty useless in practice - it prompts debate, but doesn't really answer any of the important questions. Discussing functions on a case by case basis *will*, but it'll take a long time, and may miss the "big picture". A 3rd party module, like the f2 module you have created, is another approach - try it out and refine it over time, then when the various questions have been resolved, propose it for merging into the stdlib. There may be other ways to make progress, too. So I'm also OK with the general idea, I just don't see the value in simply rehashing all the old questions - we need something actionable if this is to go anywhere (and the current proposal simply isn't). Paul
On Fri, Mar 23, 2018 at 1:47 PM, Jason Maldonis
I can find the time I'll write the PEP (although I'd rather just
Most of PEP writing (at least for a PEP like this) is capturing the debate and conclusions and capturing and moderating the discussion to some extent -- technical knowledge is the least of it. I expect you'll get plenty of help on the technical issues if there are any (I'm say if, 'cause at this point it doesn't look like anyone is proposing any *new* functionality, it's really just moving and renaming things. I suggest by starting with the a PEP outpine from a previous PEP, getting it into a a gitHub repo (it can start in your own account somewhere, then go through this thread to start sorting out the issues. I say gitHub repo 'cause then it's easy to publish and collect comments and improvements in a central place - CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
I find this odd too. There have been a few comments along the lines of this being a problem for newbies or for *some* people (not sure who *some* is referring to), but I think that having these functions in different modules is weird for everyone. It just so happens that if you use file-operations a lot that you've gotten used to the system and know where to look, but it's still weird to have to use 2-3 different modules. Additionally, if you know where to look then it isn't really a hassle to import another module, whereas it is a hassle if you are less familiar with the functionality (regardless of whether you are a newbie or not).
Me too, although I've only been using python for ~ 10 years.
I'd be happy to help as well, and if I can find the time I'll write the PEP (although I'd rather just contribute to it). Figuring out what to include and how to change the methods' APIs seems like one of the easier PEPs to write in terms of required knowledge (I don't know much about python's C code), so hopefully I would have a chance at doing an adequate job. If you think not, please tell me and I won't bother :) - Jason
On 2018-03-23 13:47, Jason Maldonis wrote:
I'll acknowledge it too, just that after X years it becomes second nature as you know. Tinkered around at lunchtime with a module to bring them under one roof. Named it "f2" because "fs" was taken on PyPI and thought the name should be very short since it could potentially be used in every script, like os and sys are. https://github.com/mixmastamyk/f2 Just poking around these questions came up about what to include: - Include file descriptors apis? - chroot? - l- functions to not follow links? - Unix dev files? obscure - pathconf? - supports_* api? - Only scandirs or walk too? - Should unpack os.path functions into the root? Handy but many of them. - path and Path in same module could confuse. - Unpack file-related shutil functions? Or leave as submodule? Perhaps this is enough to generate some comments. -Mike
Here's a comparison table of os, os.path, shutil, pathlib, and path.py. The full version is at https://github.com/westurner/pyfilemods (README.rst) and at https://westurner.github.io/pyfilemods. I ran a few set intersections and went ahead and wrote a report to compare function/method signatures and sources. (cc'ed ironically, for the last time, I promise, from the other fork of this thread: https://mail.python.org/pipermail/python-ideas/2018-March/049375.html ) ... trio wraps pathlib methods with async; which also might as well be done in pathlib? - Does it make sense to copy the docstrings at import time every time? attr table ========== ================== == ======= ====== ======= ======= attr os os.path shutil pathlib path.py ================== == ======= ====== ======= ======= `__div__`_ X `__rdiv__`_ X `absolute`_ X `abspath`_ X X `access`_ X X `altsep`_ X X `anchor`_ X `as_posix`_ X `as_uri`_ X `atime`_ X `basename`_ X X `bytes`_ X `capitalize`_ X `casefold`_ X `cd`_ X `center`_ X `chdir`_ X X `chmod`_ X X X `chown`_ X X X `chroot`_ X X `chunks`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copy2`_ X X `copyfile`_ X X `copymode`_ X X `copystat`_ X X `copytree`_ X X `count`_ X `ctime`_ X `curdir`_ X X `cwd`_ X `defpath`_ X X `devnull`_ X X `dirname`_ X X `dirs`_ X `drive`_ X X `encode`_ X `endswith`_ X `exists`_ X X X `expand`_ X `expandtabs`_ X `expanduser`_ X X X `expandvars`_ X X `ext`_ X `extsep`_ X X `files`_ X `find`_ X `fnmatch`_ X X `format`_ X `format_map`_ X `get_owner`_ X `getatime`_ X X `getctime`_ X X `getcwd`_ X X `getmtime`_ X X `getsize`_ X X `glob`_ X X `group`_ X `home`_ X `in_place`_ X `index`_ X `is_absolute`_ X `is_block_device`_ X `is_char_device`_ X `is_dir`_ X `is_fifo`_ X `is_file`_ X `is_reserved`_ X `is_socket`_ X `is_symlink`_ X `isabs`_ X X `isalnum`_ X `isalpha`_ X `isdecimal`_ X `isdigit`_ X `isdir`_ X X `isfile`_ X X `isidentifier`_ X `islink`_ X X `islower`_ X `ismount`_ X X `isnumeric`_ X `isprintable`_ X `isspace`_ X `istitle`_ X `isupper`_ X `iterdir`_ X `join`_ X X `joinpath`_ X X `lchmod`_ X `lexists`_ X `lines`_ X `link`_ X X `listdir`_ X X `ljust`_ X `lower`_ X `lstat`_ X X X `lstrip`_ X `makedirs`_ X X `makedirs_p`_ X `maketrans`_ X `match`_ X `merge_tree`_ X `mkdir`_ X X X `mkdir_p`_ X `module`_ X `move`_ X X `mtime`_ X `name`_ X X X `namebase`_ X `normcase`_ X X `normpath`_ X X `open`_ X X X `os`_ X X `owner`_ X X `pardir`_ X X `parent`_ X X `parents`_ X `partition`_ X `parts`_ X `pathconf`_ X X `pathsep`_ X X `read_bytes`_ X `read_hash`_ X `read_hexhash`_ X `read_md5`_ X `read_text`_ X `readlink`_ X X `readlinkabs`_ X `realpath`_ X X `relative_to`_ X `relpath`_ X X `relpathto`_ X `remove`_ X X `remove_p`_ X `removedirs`_ X X `removedirs_p`_ X `rename`_ X X X `renames`_ X X `replace`_ X X X `resolve`_ X `rfind`_ X `rglob`_ X `rindex`_ X `rjust`_ X `rmdir`_ X X X `rmdir_p`_ X `rmtree`_ X X `rmtree_p`_ X `root`_ X `rpartition`_ X `rsplit`_ X `rstrip`_ X `samefile`_ X X X `sameopenfile`_ X `samestat`_ X `sep`_ X X `size`_ X `special`_ X `split`_ X X `splitall`_ X `splitdrive`_ X X `splitext`_ X X `splitlines`_ X `splitpath`_ X `splitunc`_ X `startswith`_ X `stat`_ X X X X X `statvfs`_ X X `stem`_ X X `strip`_ X `stripext`_ X `suffix`_ X `suffixes`_ X `swapcase`_ X `symlink`_ X X `symlink_to`_ X `text`_ X `title`_ X `touch`_ X X `translate`_ X `uncshare`_ X `unlink`_ X X X `unlink_p`_ X `upper`_ X `using_module`_ X `utime`_ X X `walk`_ X X `walkdirs`_ X `walkfiles`_ X `with_name`_ X `with_suffix`_ X X `write_bytes`_ X X `write_lines`_ X `write_text`_ X X `zfill`_ X ================== == ======= ====== ======= ======= On Fri, Mar 23, 2018 at 6:22 PM, Mike Miller <python-ideas@mgmiller.net> wrote:
On Fri, Mar 23, 2018 at 3:38 PM, Wes Turner <wes.turner@gmail.com> wrote:
Here's a comparison table of os, os.path, shutil, pathlib, and path.py.
Darn, that's a big list -- exactly what we want to avoid :-( Though there are bunch of string methods in there that we can dump right off the bat.
... trio wraps pathlib methods with async; which also might as well be done in pathlib?
hmm --that's s touch one -- a full set of async file operations would be great. But where to put them???? and most of what we are talking about doesn't need asnyc, does it? deleting/renaming a single, file, etc. So my thought is that an asnyc lib should mirror / overwrite this file-operations-api. Which makes it a problem for the async lib(s) later on down the road. BTW -- this is a nice argument for making this API in the first place -- then we have a single API to make an async version of! - Does it make sense to copy the docstrings at import time every time? copy from where? is this for the async version? or are you thinking that the new methods will simply be wrappers for the olds ones -- in which case, I think no -- this should be a new API with its own docs, whether or not a particular call is any different that the "old" one. The big challenge is to identity what is "basic" or "normal" ratehr than advanced. here's a quick take on trimming down that whole table -- jsut my quick HO: attr table ========== ================== == ======= ====== ======= ======= attr os os.path shutil pathlib path.py ================== == ======= ====== ======= ======= `absolute`_ X `abspath`_ X X `access`_ X X `atime`_ X `basename`_ X X `cd`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copytree`_ X X `curdir`_ X X `dirname`_ X X `drive`_ X X `exists`_ X X X `expanduser`_ X X X `expandvars`_ X X `ext`_ X `fnmatch`_ X X `get_owner`_ X `getatime`_ X X `getctime`_ X X `getcwd`_ X X `getmtime`_ X X `getsize`_ X X `glob`_ X X `home`_ X `is_absolute`_ X `is_dir`_ X `is_symlink`_ X `isabs`_ X X `isdir`_ X X `isfile`_ X X `islink`_ X X `iterdir`_ X `lchmod`_ X `link`_ X X `listdir`_ X X `lstat`_ X X X `mkdir`_ X X X `makedirs`_ X X `match`_ X `merge_tree`_ X `move`_ X X `mtime`_ X `normcase`_ X X `normpath`_ X X `open`_ X X X `owner`_ X X `parent`_ X X `parents`_ X `relative_to`_ X `relpath`_ X X `relpathto`_ X `remove`_ X X `removedirs`_ X X `rename`_ X X X `replace`_ X X X `resolve`_ X `rglob`_ X `root`_ X `samefile`_ X X X `size`_ X `suffix`_ X `suffixes`_ X `symlink`_ X X `symlink_to`_ X `touch`_ X X `utime`_ X X `walk`_ X X `with_name`_ X `with_suffix`_ X X `write_bytes`_ X X `write_text`_ X X These are about permissions -- challenging how to do that in a X-platform way... And it would be good to group this sort of thing together. `chmod`_ X X X `chown`_ X X X ================== == ======= ====== ======= ======= Now that I've done that, I think it would be better to be systematic: 1) keep every method that pathlib.Path has 2) dump every method this is only a string method (and maybe everything that path has that nothign else has -- most of those are string methods 3) match the ones that are the same thing, but different names 4) match the ones that are almost the same thing:remove and removedir and removedirs - then we can hash out whether to join with flags, or keep separate 5) group by "class of functionality: - file nameing, etc - seraching (glob) - permission handling - etc. That'll get it structured to capture the discussion. -CHB
This is a great idea to have a prototype of sorts, but I do'nt think we should take the approach of: make it, put it on pypi, and see if it gains traction -- this isn't adding anythign new, very few folks are going to want to crate a dependency to get a more discoverable api, and no one going to point newbies at it. Just poking around these questions came up about what to include:
- Include file descriptors apis?
no -- unless in an "advanced" api somewhere - chroot?
no
- l- functions to not follow links?
maybe adanced, or optional flags
- Unix dev files? obscure
don't even know what this is, so no :-) - pathconf?
again, what is this? probabaly no.
- supports_* api?
no.
- Only scandirs or walk too?
walk is really useful and not trivial to write, so yes.
- Should unpack os.path functions into the root? Handy but many of them.
no way.
- Unpack file-related shutil functions? Or leave as submodule?
make them methods on Path. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi folks, according to comments it seems, it would better to propose a new module. I start to work out a new proposal. I created a BitBucket repo to be able to work together with folks who want help ;-) the repo address is: https://bitbucket.org/GeorgeFischhof/pep_for_filesystem_operations.git -- o -- shutil and path object relation: the documentation laks the information. For functions of os module it is written that functions updated in 3.6 and from that point they can use pathlib objects. I will check the status. (if the work together I will submit an issue report to update the documentation accordingly) BR, George
On 23 March 2018 at 22:22, Mike Miller <python-ideas@mgmiller.net> wrote:
It's certainly easy to get so used to the current state of affairs that you forget that it's not "obvious" to a newcomer. I've been teaching a colleague Python and it's certainly something he would have problems with.
Yeah, this is the real issue. It's not that the general principle "have a single place where all filesystem stuff is found" is a particularly bad one, it's more that it's genuinely hard to know what the right answer is here. I'll add some other questions: - Should the module only contain cross-platform functions, or are platform-specific ones OK? - How about operations that appear on multiple levels (Steve Dower mentioned copy "that works like drag and drop", there's also about delete that handles recycle bins vs lower-level delete)? and the big one: - How will we handle backward compatibility? This is why a generic proposal like the one that started this thread is pretty useless in practice - it prompts debate, but doesn't really answer any of the important questions. Discussing functions on a case by case basis *will*, but it'll take a long time, and may miss the "big picture". A 3rd party module, like the f2 module you have created, is another approach - try it out and refine it over time, then when the various questions have been resolved, propose it for merging into the stdlib. There may be other ways to make progress, too. So I'm also OK with the general idea, I just don't see the value in simply rehashing all the old questions - we need something actionable if this is to go anywhere (and the current proposal simply isn't). Paul
On Fri, Mar 23, 2018 at 1:47 PM, Jason Maldonis
I can find the time I'll write the PEP (although I'd rather just
Most of PEP writing (at least for a PEP like this) is capturing the debate and conclusions and capturing and moderating the discussion to some extent -- technical knowledge is the least of it. I expect you'll get plenty of help on the technical issues if there are any (I'm say if, 'cause at this point it doesn't look like anyone is proposing any *new* functionality, it's really just moving and renaming things. I suggest by starting with the a PEP outpine from a previous PEP, getting it into a a gitHub repo (it can start in your own account somewhere, then go through this thread to start sorting out the issues. I say gitHub repo 'cause then it's easy to publish and collect comments and improvements in a central place - CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (7)
-
Chris Barker -
Chris Barker - NOAA Federal
-
George Fischhof -
Jason Maldonis -
Mike Miller -
Paul Moore -
Wes Turner