New PEP proposal -- Pathlib Module Should Contain All File Operations -- version 2
Hi folks, I added the list of functions to the proposal, here is the new version. George PEP: 9999 Title: Pathlib Module Should Contain All File Operations Author: George Fischhof <george at fischhof.hu> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Mar-2018 Python-Version: 3.8 Post-History: 12-Mar-2018, 17-Mar-2018 Abstract ======== This PEP proposes pathlib module to be a centralized place for all file-system related operations. Rationale ========= Right now we have several modules that contain functions related to file-system operations mainly the os, pathlib and shutil. For beginners it is quite hard to remember where can he / she find a function (copy resides in shutil, but the remove function can be found in the os module. (And sometimes developers with moderate experience have to check the documentation as well.) After the release of version 3.6 several methods became aware of path-like object. There are only a few ones which does not support the path-like object. After making these methods path-like object aware, these functions could be added to pathlib. With functions in pathlib the developers should not have to think on which method (function) can be found in which module. Makes the life easier. Implementation ============== For compatibility reasons the pathlib should contain wrappers to the original functions. The original functions should remain at their original place. (Or if pathlib contains the function, the original modules should have a wrapper to it.) List of mentioned functions =========================== * os.link => path.hardlink_to (should be named similarly to path.softlink_to) * os.mkfifo * os.readlink * os.remove * os.removedirs (shutil.rmtree has the same functionalaty) * os.truncate * shutil.copyfileobj * shutil.copyfile * shutil.copymode * shutil.copystat * shutil.copy * shutil.copy2 * shutil.copytree with shutil.ignore_patterns * shutil.move * shutil.disk_usage * shutil.chown All functions from os module accept path-like objects, and none of the shutil functions. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
I think you will find these to are a lot different.
* os.truncate
* shutil.copyfileobj
This function does not take paths as arguments. I guess it does not belong here.
Is yout plan to also expose all the constants need to use chmod etc from pathlib?
All functions from os module accept path-like objects, and none of the shutil functions.
Should shutils be improved to accept path-like first? Barry
2018. márc. 17. 21:34 ezt írta ("Barry" <barry@barrys-emacs.org>): On 17 Mar 2018, at 10:42, George Fischhof <george@fischhof.hu> wrote: Hi folks, I added the list of functions to the proposal, here is the new version. George PEP: 9999 Title: Pathlib Module Should Contain All File Operations Author: George Fischhof <george at fischhof.hu> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Mar-2018 Python-Version: 3.8 Post-History: 12-Mar-2018, 17-Mar-2018 Abstract ======== This PEP proposes pathlib module to be a centralized place for all file-system related operations. Rationale ========= Right now we have several modules that contain functions related to file-system operations mainly the os, pathlib and shutil. For beginners it is quite hard to remember where can he / she find a function (copy resides in shutil, but the remove function can be found in the os module. (And sometimes developers with moderate experience have to check the documentation as well.) After the release of version 3.6 several methods became aware of path-like object. There are only a few ones which does not support the path-like object. After making these methods path-like object aware, these functions could be added to pathlib. With functions in pathlib the developers should not have to think on which method (function) can be found in which module. Makes the life easier. Implementation ============== For compatibility reasons the pathlib should contain wrappers to the original functions. The original functions should remain at their original place. (Or if pathlib contains the function, the original modules should have a wrapper to it.) List of mentioned functions =========================== * os.link => path.hardlink_to (should be named similarly to path.softlink_to) * os.mkfifo * os.readlink * os.remove * os.removedirs (shutil.rmtree has the same functionalaty) I think you will find these to are a lot different.
From high level, it is only question of implementation: the main goal is to remove a chain of directories
* os.truncate * shutil.copyfileobj This function does not take paths as arguments. I guess it does not belong here. No path, it is true (right now), but it has sense to have this function among the others. * shutil.copyfile * shutil.copymode * shutil.copystat * shutil.copy * shutil.copy2 * shutil.copytree with shutil.ignore_patterns * shutil.move * shutil.disk_usage * shutil.chown Is yout plan to also expose all the constants need to use chmod etc from pathlib? Of course, because this PEP is about conveniece and easyness. All functions from os module accept path-like objects, and none of the shutil functions. Should shutils be improved to accept path-like first? Maybe. The PEP should show the goal. This is just a task among others to achieve the goal. But it can be done simultanuosly. George Barry Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 17 March 2018 at 22:49, George Fischhof <george@fischhof.hu> wrote:
So you're proposing to include these how? As standalone functions in pathlib? Strong -1 on that. I thought the intention here was to add methods to path objects (which I still have reservations about). But I'm definitely against adding standalone functions to pathlib - I see pathlib as where Path objects are defined, not as "the place where all path-related and other file operations should live"). Paul
It seems that the main quest to answer first is this. Should Path() have methods to access all file operations? Maybe it was a mistake to add the ones that are already there. Especially in light of the fspath protocol that now exists. If yes proceed to details the methods and their semantics. If no then concentrate on making shutil etc easy to use this Path() args. Maybe do this anyway. Barry
On 18 March 2018 at 14:44, Barry Scott <barry@barrys-emacs.org> wrote:
It seems that the main quest to answer first is this.
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a path).
Maybe it was a mistake to add the ones that are already there.
Seems unlikely, as Path objects are pretty useful in their current form. Even if it were, in some cases, backward compatibility dictates keeping them so who cares?
Especially in light of the fspath protocol that now exists.
Again, *maybe* the arguments for some of the existing methods were weakened by the existence of the fspath protocol, but not enough to override backward compatibility, so who cares?
If yes proceed to details the methods and their semantics.
Not applicable, see above.
If no then concentrate on making shutil etc easy to use this Path() args.
Wrong conclusion - just because it shouldn't have methods for *all* doesn't imply that *nothing* new warrants being a Path method. Just that cases should be considered on their individual merits.
Maybe do this anyway.
Certainly this is a reasonable thing to propose, Probably doesn't even need much discussion, an issue on bpo should be sufficient. Paul
On 2018-03-18 10:55, Paul Moore wrote:
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a path).
Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc?
I had a colleague complaining to me the other day about having to search multiple packages for the right function to move a file (implying: with the same semantics as drag-drop). If there isn’t a pathtools library on PyPI yet, this would certainly be valuable for newer developers. My view on Path is to either have everything on it or nothing on it (without removing what’s already there, of course), and since everything is so popular we should at least put everything in the one place. Top-posted from my Windows phone From: Mike Miller Sent: Monday, March 19, 2018 10:51 To: python-ideas@python.org Subject: Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2 On 2018-03-18 10:55, Paul Moore wrote:
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a path).
Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc? _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Sat, Mar 17, 2018 at 10:42 AM, George Fischhof <george@fischhof.hu> wrote:
All functions from os module accept path-like objects, and none of the shutil functions.
shutil indirectly supports __fspath__ paths via os and os.path. One exception is shutil.disk_usage() on Windows, which only supports str strings. This is fixed in 3.7, in resolution of issue 32556. Maybe it should be backported to 3.6. I like the idea of a high-level library that provides a subset of commonly used os, io, and shutil functionality in one place. But maybe a new module isn't required. shutil could be extended since its design goal is to provide "high-level operations on files and collections of files". That said, pathlib's goal to support "concrete paths [that] provide I/O operations" does seem incomplete. It should support copy, copytree, rmtree, and move methods. Also, a `parents` option should be added to Path.rmdir to implement removedirs, which mirrors how Path.mkdir implements makedirs.
os.link => path.hardlink_to
I'm surprised this doesn't already exist.
I think you will find these to are a lot different.
* os.truncate
* shutil.copyfileobj
This function does not take paths as arguments. I guess it does not belong here.
Is yout plan to also expose all the constants need to use chmod etc from pathlib?
All functions from os module accept path-like objects, and none of the shutil functions.
Should shutils be improved to accept path-like first? Barry
2018. márc. 17. 21:34 ezt írta ("Barry" <barry@barrys-emacs.org>): On 17 Mar 2018, at 10:42, George Fischhof <george@fischhof.hu> wrote: Hi folks, I added the list of functions to the proposal, here is the new version. George PEP: 9999 Title: Pathlib Module Should Contain All File Operations Author: George Fischhof <george at fischhof.hu> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Mar-2018 Python-Version: 3.8 Post-History: 12-Mar-2018, 17-Mar-2018 Abstract ======== This PEP proposes pathlib module to be a centralized place for all file-system related operations. Rationale ========= Right now we have several modules that contain functions related to file-system operations mainly the os, pathlib and shutil. For beginners it is quite hard to remember where can he / she find a function (copy resides in shutil, but the remove function can be found in the os module. (And sometimes developers with moderate experience have to check the documentation as well.) After the release of version 3.6 several methods became aware of path-like object. There are only a few ones which does not support the path-like object. After making these methods path-like object aware, these functions could be added to pathlib. With functions in pathlib the developers should not have to think on which method (function) can be found in which module. Makes the life easier. Implementation ============== For compatibility reasons the pathlib should contain wrappers to the original functions. The original functions should remain at their original place. (Or if pathlib contains the function, the original modules should have a wrapper to it.) List of mentioned functions =========================== * os.link => path.hardlink_to (should be named similarly to path.softlink_to) * os.mkfifo * os.readlink * os.remove * os.removedirs (shutil.rmtree has the same functionalaty) I think you will find these to are a lot different.
From high level, it is only question of implementation: the main goal is to remove a chain of directories
* os.truncate * shutil.copyfileobj This function does not take paths as arguments. I guess it does not belong here. No path, it is true (right now), but it has sense to have this function among the others. * shutil.copyfile * shutil.copymode * shutil.copystat * shutil.copy * shutil.copy2 * shutil.copytree with shutil.ignore_patterns * shutil.move * shutil.disk_usage * shutil.chown Is yout plan to also expose all the constants need to use chmod etc from pathlib? Of course, because this PEP is about conveniece and easyness. All functions from os module accept path-like objects, and none of the shutil functions. Should shutils be improved to accept path-like first? Maybe. The PEP should show the goal. This is just a task among others to achieve the goal. But it can be done simultanuosly. George Barry Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 17 March 2018 at 22:49, George Fischhof <george@fischhof.hu> wrote:
So you're proposing to include these how? As standalone functions in pathlib? Strong -1 on that. I thought the intention here was to add methods to path objects (which I still have reservations about). But I'm definitely against adding standalone functions to pathlib - I see pathlib as where Path objects are defined, not as "the place where all path-related and other file operations should live"). Paul
It seems that the main quest to answer first is this. Should Path() have methods to access all file operations? Maybe it was a mistake to add the ones that are already there. Especially in light of the fspath protocol that now exists. If yes proceed to details the methods and their semantics. If no then concentrate on making shutil etc easy to use this Path() args. Maybe do this anyway. Barry
On 18 March 2018 at 14:44, Barry Scott <barry@barrys-emacs.org> wrote:
It seems that the main quest to answer first is this.
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a path).
Maybe it was a mistake to add the ones that are already there.
Seems unlikely, as Path objects are pretty useful in their current form. Even if it were, in some cases, backward compatibility dictates keeping them so who cares?
Especially in light of the fspath protocol that now exists.
Again, *maybe* the arguments for some of the existing methods were weakened by the existence of the fspath protocol, but not enough to override backward compatibility, so who cares?
If yes proceed to details the methods and their semantics.
Not applicable, see above.
If no then concentrate on making shutil etc easy to use this Path() args.
Wrong conclusion - just because it shouldn't have methods for *all* doesn't imply that *nothing* new warrants being a Path method. Just that cases should be considered on their individual merits.
Maybe do this anyway.
Certainly this is a reasonable thing to propose, Probably doesn't even need much discussion, an issue on bpo should be sufficient. Paul
On 2018-03-18 10:55, Paul Moore wrote:
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a path).
Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc?
I had a colleague complaining to me the other day about having to search multiple packages for the right function to move a file (implying: with the same semantics as drag-drop). If there isn’t a pathtools library on PyPI yet, this would certainly be valuable for newer developers. My view on Path is to either have everything on it or nothing on it (without removing what’s already there, of course), and since everything is so popular we should at least put everything in the one place. Top-posted from my Windows phone From: Mike Miller Sent: Monday, March 19, 2018 10:51 To: python-ideas@python.org Subject: Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2 On 2018-03-18 10:55, Paul Moore wrote:
Should Path() have methods to access all file operations?
No, (Counterexample, having a Path operation to set Windows ACLs for a path).
Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc? _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Sat, Mar 17, 2018 at 10:42 AM, George Fischhof <george@fischhof.hu> wrote:
All functions from os module accept path-like objects, and none of the shutil functions.
shutil indirectly supports __fspath__ paths via os and os.path. One exception is shutil.disk_usage() on Windows, which only supports str strings. This is fixed in 3.7, in resolution of issue 32556. Maybe it should be backported to 3.6. I like the idea of a high-level library that provides a subset of commonly used os, io, and shutil functionality in one place. But maybe a new module isn't required. shutil could be extended since its design goal is to provide "high-level operations on files and collections of files". That said, pathlib's goal to support "concrete paths [that] provide I/O operations" does seem incomplete. It should support copy, copytree, rmtree, and move methods. Also, a `parents` option should be added to Path.rmdir to implement removedirs, which mirrors how Path.mkdir implements makedirs.
os.link => path.hardlink_to
I'm surprised this doesn't already exist.
participants (7)
-
Barry -
Barry Scott
-
eryk sun -
George Fischhof -
Mike Miller -
Paul Moore -
Steve Dower