Currently, pathlib supports pretty much all common filesystem operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that. Whatever the original intent might have been behind pathlib, it is now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying. So I think it would make a lot of sense to include copying inside pathlib. I propose adding a `copy` method to `pathlib.Path` (for concrete paths). The specific call signature would be: copy(dst, *, follow_symlinks=True, recursive=True, dir_exist_ok=True) This will call `shutil.copytree` for directories if recursive is True, or `copy2` if recursive if False. For files it will call `copy2` always. An alternative would be to have separate `copy` and `copytree` methods, but in most situations I think copying on directories would want copying a tree.
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that.
Whatever the original intent might have been behind pathlib, it is now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference. ChrisA
On 18 Oct 2022, at 21:59, Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that.
Whatever the original intent might have been behind pathlib, it is now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference.
I tend to think that if move is in pathlib then copy makes sense. As with all things pathlib it is a reasonable question to ask where should additions stop. Barry
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DD36UO... Code of Conduct: http://python.org/psf/codeofconduct/
improve discoverability with a
docs reference.
This is a great idea — I suggest a PR would be in order, and the good news is that a doc addition could be back-ported to all supported versions. However, I don’t agree that copy does not belong on pathlib. I get the point, but practically beats purity and all that. I think there is an active effort to make pathlib more full featured and perhaps make shutils redundant. I don’t know the status, but I don’t think it’s been rejected. I suggest you try to find that discussion and join in. I’d look in the archives of this list, gitHub issues, and of course discuss.python.org, which is where much (most) of these kinds of discussion have moved. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Tue, Oct 18, 2022 at 5:00 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem
operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that.
Whatever the original intent might have been behind pathlib, it is now
effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference.
ChrisA
How is it any less of a "path operation" than moving files, reading and writing files, making directories, and deleting files?
On Wed, 19 Oct 2022 at 09:17, Todd <toddrjen@gmail.com> wrote:
On Tue, Oct 18, 2022 at 5:00 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that.
Whatever the original intent might have been behind pathlib, it is now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference.
ChrisA
How is it any less of a "path operation" than moving files, reading and writing files, making directories, and deleting files?
At a file system level, those are nothing more than directory entry manipulation. (Reading and writing might count as slightly more than that, but practicality beats purity.) Copying a file involves a lot more work, and is not a simple operation done in the directory entry, so it's more of a separate tool. Also, not everyone agrees on what "copying" means, so there'll be various platform-specific concepts. In theory, you could take literally every possible action that could be done on a file and make it into a Path method. Do we need a method to read a .WAV file and return its headers, or is that better left in the 'wave' module? There's a reason that huge slabs of Python are built on protocols rather than methods. Assuming that shutil.copyfile accepts Path objects (and if it doesn't, that should definitely be fixed), is it really a problem for the Path objects to not have a copy method? I agree with you about discoverability; since copying IS a common operation, it would be great to have a docs reference pointing people to shutil. But I don't think we need every single operation as a Path method, and the easiest place to draw the line is at directory entry operations. ChrisA
On Tue, Oct 18, 2022 at 6:26 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 09:17, Todd <toddrjen@gmail.com> wrote:
On Tue, Oct 18, 2022 at 5:00 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem
Whatever the original intent might have been behind pathlib, it is
now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since
operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that. there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference.
ChrisA
How is it any less of a "path operation" than moving files, reading and writing files, making directories, and deleting files?
At a file system level, those are nothing more than directory entry manipulation. (Reading and writing might count as slightly more than that, but practicality beats purity.) Copying a file involves a lot more work, and is not a simple operation done in the directory entry, so it's more of a separate tool.
Yes, and I would say practically copying a file is at least as much a valid part of pathlib as basic file/io. File i/o is a builtin for python, so putting it pathlib is even less useful than copying, which requires a separate import.
Also, not everyone agrees on what "copying" means, so there'll be various platform-specific concepts.
But Python already has such a tool, we are simply exposing it in a more convenient way.
In theory, you could take literally every possible action that could be done on a file and make it into a Path method. Do we need a method to read a .WAV file and return its headers, or is that better left in the 'wave' module? There's a reason that huge slabs of Python are built on protocols rather than methods. Assuming that shutil.copyfile accepts Path objects (and if it doesn't, that should definitely be fixed), is it really a problem for the Path objects to not have a copy method?
Operating on specific file types (other than text files) is not considered a core filesystem operation in any operating system I know. In contrast copying is, such as in UNIX 1, Multics, PC-DOS 1.0, and OS/2. Again, whatever the original intent behind pathlib was, its effective use today is as the primary recommended way to deal with basic file operations. From what I can tell, copying is pretty much universally treated as a basic file operation in pretty much every major operating system.
For my edification, how is copy in pathlib proposed to be implemented? Just call out to the OS to do the copy? Do the copy by reading and writing? On Tue, 2022-10-18 at 19:24 -0400, Todd wrote:
On Tue, Oct 18, 2022 at 6:26 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 09:17, Todd <toddrjen@gmail.com> wrote:
On Tue, Oct 18, 2022 at 5:00 PM Chris Angelico <rosuav@gmail.com>
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem
operations. You can create, move, and delete both files and
wrote: directories. One big omission is copying. You need shutil for that.
Whatever the original intent might have been behind pathlib,
it is now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference.
ChrisA
How is it any less of a "path operation" than moving files, reading and writing files, making directories, and deleting files?
At a file system level, those are nothing more than directory entry manipulation. (Reading and writing might count as slightly more than that, but practicality beats purity.) Copying a file involves a lot more work, and is not a simple operation done in the directory entry, so it's more of a separate tool.
Yes, and I would say practically copying a file is at least as much a valid part of pathlib as basic file/io. File i/o is a builtin for python, so putting it pathlib is even less useful than copying, which requires a separate import.
Also, not everyone agrees on what "copying" means, so there'll be various platform-specific concepts.
But Python already has such a tool, we are simply exposing it in a more convenient way.
In theory, you could take literally every possible action that could be done on a file and make it into a Path method. Do we need a method to read a .WAV file and return its headers, or is that better left in the 'wave' module? There's a reason that huge slabs of Python are built on protocols rather than methods. Assuming that shutil.copyfile accepts Path objects (and if it doesn't, that should definitely be fixed), is it really a problem for the Path objects to not have a copy method?
Operating on specific file types (other than text files) is not considered a core filesystem operation in any operating system I know. In contrast copying is, such as in UNIX 1, Multics, PC-DOS 1.0, and OS/2.
Again, whatever the original intent behind pathlib was, its effective use today is as the primary recommended way to deal with basic file operations. From what I can tell, copying is pretty much universally treated as a basic file operation in pretty much every major operating system. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VWTK6H... Code of Conduct: http://python.org/psf/codeofconduct/
pyfilemods is a generated reference of file/path modules and their methods. From https://github.com/westurner/pyfilemods#copy (w/ all due respect to licenses) : Modules +++++++++ - os - Source: https://github.com/python/cpython/tree/3.6/Lib/os.py - Docs: https://docs.python.org/3/library/os.html - os.path - Source: https://github.com/python/cpython/blob/3.6/Lib/posixpath.py - Source: https://github.com/python/cpython/tree/3.6/Lib/ntpath.py - Source: https://github.com/python/cpython/tree/3.6/Lib/macpath.py - Docs: https://docs.python.org/3/library/os.path.html - shutil - Source: https://github.com/python/cpython/tree/3.6/Lib/shutil.py - Docs: https://docs.python.org/3/library/shutil.html - pathlib - Source: https://github.com/python/cpython/blob/3.6/Lib/pathlib.py - Docs: https://docs.python.org/3/library/pathlib.html - pathpy - Src: https://github.com/jaraco/path.py - Source: https://github.com/jaraco/path.py/blob/master/path.py - Docs: https://pathpy.readthedocs.io/en/latest/ - trio - Src: https://github.com/python-trio/trio - Source: https://github.com/python-trio/trio/blob/master/trio/_path.py - Docs: https://trio.readthedocs.io/en/latest/reference-io.html#trio.Path Sets ++++ attr table ========== ================== == ======= ====== ======= ======= attr os os.path shutil pathlib path.py ================== == ======= ====== ======= ======= `__div__`_ X `__rdiv__`_ X `absolute`_ X `abspath`_ X X `access`_ X X `altsep`_ X X `anchor`_ X `as_posix`_ X `as_uri`_ X `atime`_ X `basename`_ X X `bytes`_ X `capitalize`_ X `casefold`_ X `cd`_ X `center`_ X `chdir`_ X X `chmod`_ X X X `chown`_ X X X `chroot`_ X X `chunks`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copy2`_ X X `copyfile`_ X X `copymode`_ X X `copystat`_ X X `copytree`_ X X `count`_ X [...] ``copy`` ========= | **shutil.copy**\ ``(src, dst, *, follow_symlinks=True)`` | **pathpy.copy**\ ``(src, dst, *, follow_symlinks=True)`` | **shutil.copy**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://docs.python.org/3/library/shutil.html#shutil.copy>`__ `source (shutil.py) < https://github.com/python/cpython/tree/3.6/Lib/shutil.py>`__ .. code:: python def copy(src, dst, *, follow_symlinks=True): """Copy data and mode bits ("cp src dst"). Return the file's destination. The destination may be a directory. If follow_symlinks is false, symlinks won't be followed. This resembles GNU's "cp -P src dst". If source and destination are the same file, a SameFileError will be raised. """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) copyfile(src, dst, follow_symlinks=follow_symlinks) copymode(src, dst, follow_symlinks=follow_symlinks) return dst | **pathpy.copy**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://pathpy.readthedocs.io/en/latest/api.html#path.Path.copy>`__ `source (path.py) <https://github.com/jaraco/path.py/blob/master/path.py>`__ .. code:: python def copy(src, dst, *, follow_symlinks=True): """Copy data and mode bits ("cp src dst"). Return the file's destination. The destination may be a directory. If follow_symlinks is false, symlinks won't be followed. This resembles GNU's "cp -P src dst". If source and destination are the same file, a SameFileError will be raised. """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) copyfile(src, dst, follow_symlinks=follow_symlinks) copymode(src, dst, follow_symlinks=follow_symlinks) return dst ``copy2`` ========== | **shutil.copy2**\ ``(src, dst, *, follow_symlinks=True)`` | **pathpy.copy2**\ ``(src, dst, *, follow_symlinks=True)`` | **shutil.copy2**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://docs.python.org/3/library/shutil.html#shutil.copy2>`__ `source (shutil.py) < https://github.com/python/cpython/tree/3.6/Lib/shutil.py>`__ .. code:: python def copy2(src, dst, *, follow_symlinks=True): """Copy data and all stat info ("cp -p src dst"). Return the file's destination." The destination may be a directory. If follow_symlinks is false, symlinks won't be followed. This resembles GNU's "cp -P src dst". """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) copyfile(src, dst, follow_symlinks=follow_symlinks) copystat(src, dst, follow_symlinks=follow_symlinks) return dst | **pathpy.copy2**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://pathpy.readthedocs.io/en/latest/api.html#path.Path.copy2>`__ `source (path.py) <https://github.com/jaraco/path.py/blob/master/path.py>`__ .. code:: python def copy2(src, dst, *, follow_symlinks=True): """Copy data and all stat info ("cp -p src dst"). Return the file's destination." The destination may be a directory. If follow_symlinks is false, symlinks won't be followed. This resembles GNU's "cp -P src dst". """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) copyfile(src, dst, follow_symlinks=follow_symlinks) copystat(src, dst, follow_symlinks=follow_symlinks) return dst ``copyfile`` ============= | **shutil.copyfile**\ ``(src, dst, *, follow_symlinks=True)`` | **pathpy.copyfile**\ ``(src, dst, *, follow_symlinks=True)`` | **shutil.copyfile**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://docs.python.org/3/library/shutil.html#shutil.copyfile>`__ `source (shutil.py) < https://github.com/python/cpython/tree/3.6/Lib/shutil.py>`__ .. code:: python def copyfile(src, dst, *, follow_symlinks=True): """Copy data from src to dst. If follow_symlinks is not set and src is a symbolic link, a new symlink will be created instead of copying the file it points to. """ if _samefile(src, dst): raise SameFileError("{!r} and {!r} are the same file".format(src, dst)) for fn in [src, dst]: try: st = os.stat(fn) except OSError: # File most likely does not exist pass else: # XXX What about other special files? (sockets, devices...) if stat.S_ISFIFO(st.st_mode): raise SpecialFileError("`%s` is a named pipe" % fn) if not follow_symlinks and os.path.islink(src): os.symlink(os.readlink(src), dst) else: with open(src, 'rb') as fsrc: with open(dst, 'wb') as fdst: copyfileobj(fsrc, fdst) return dst | **pathpy.copyfile**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://pathpy.readthedocs.io/en/latest/api.html#path.Path.copyfile>`__ `source (path.py) <https://github.com/jaraco/path.py/blob/master/path.py>`__ .. code:: python def copyfile(src, dst, *, follow_symlinks=True): """Copy data from src to dst. If follow_symlinks is not set and src is a symbolic link, a new symlink will be created instead of copying the file it points to. """ if _samefile(src, dst): raise SameFileError("{!r} and {!r} are the same file".format(src, dst)) for fn in [src, dst]: try: st = os.stat(fn) except OSError: # File most likely does not exist pass else: # XXX What about other special files? (sockets, devices...) if stat.S_ISFIFO(st.st_mode): raise SpecialFileError("`%s` is a named pipe" % fn) if not follow_symlinks and os.path.islink(src): os.symlink(os.readlink(src), dst) else: with open(src, 'rb') as fsrc: with open(dst, 'wb') as fdst: copyfileobj(fsrc, fdst) return dst ``copymode`` ============= | **shutil.copymode**\ ``(src, dst, *, follow_symlinks=True)`` | **pathpy.copymode**\ ``(src, dst, *, follow_symlinks=True)`` | **shutil.copymode**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://docs.python.org/3/library/shutil.html#shutil.copymode>`__ `source (shutil.py) < https://github.com/python/cpython/tree/3.6/Lib/shutil.py>`__ .. code:: python def copymode(src, dst, *, follow_symlinks=True): """Copy mode bits from src to dst. If follow_symlinks is not set, symlinks aren't followed if and only if both `src` and `dst` are symlinks. If `lchmod` isn't available (e.g. Linux) this method does nothing. """ if not follow_symlinks and os.path.islink(src) and os.path.islink(dst): if hasattr(os, 'lchmod'): stat_func, chmod_func = os.lstat, os.lchmod else: return elif hasattr(os, 'chmod'): stat_func, chmod_func = os.stat, os.chmod else: return st = stat_func(src) chmod_func(dst, stat.S_IMODE(st.st_mode)) | **pathpy.copymode**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://pathpy.readthedocs.io/en/latest/api.html#path.Path.copymode>`__ `source (path.py) <https://github.com/jaraco/path.py/blob/master/path.py>`__ .. code:: python def copymode(src, dst, *, follow_symlinks=True): """Copy mode bits from src to dst. If follow_symlinks is not set, symlinks aren't followed if and only if both `src` and `dst` are symlinks. If `lchmod` isn't available (e.g. Linux) this method does nothing. """ if not follow_symlinks and os.path.islink(src) and os.path.islink(dst): if hasattr(os, 'lchmod'): stat_func, chmod_func = os.lstat, os.lchmod else: return elif hasattr(os, 'chmod'): stat_func, chmod_func = os.stat, os.chmod else: return st = stat_func(src) chmod_func(dst, stat.S_IMODE(st.st_mode)) ``copystat`` ============= | **shutil.copystat**\ ``(src, dst, *, follow_symlinks=True)`` | **pathpy.copystat**\ ``(src, dst, *, follow_symlinks=True)`` | **shutil.copystat**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://docs.python.org/3/library/shutil.html#shutil.copystat>`__ `source (shutil.py) < https://github.com/python/cpython/tree/3.6/Lib/shutil.py>`__ .. code:: python def copystat(src, dst, *, follow_symlinks=True): """Copy all stat info (mode bits, atime, mtime, flags) from src to dst. If the optional flag `follow_symlinks` is not set, symlinks aren't followed if and only if both `src` and `dst` are symlinks. """ def _nop(*args, ns=None, follow_symlinks=None): pass # follow symlinks (aka don't not follow symlinks) follow = follow_symlinks or not (os.path.islink(src) and os.path.islink(dst)) if follow: # use the real function if it exists def lookup(name): return getattr(os, name, _nop) else: # use the real function only if it exists # *and* it supports follow_symlinks def lookup(name): fn = getattr(os, name, _nop) if fn in os.supports_follow_symlinks: return fn return _nop st = lookup("stat")(src, follow_symlinks=follow) mode = stat.S_IMODE(st.st_mode) lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns), follow_symlinks=follow) try: lookup("chmod")(dst, mode, follow_symlinks=follow) except NotImplementedError: # if we got a NotImplementedError, it's because # * follow_symlinks=False, # * lchown() is unavailable, and # * either # * fchownat() is unavailable or # * fchownat() doesn't implement AT_SYMLINK_NOFOLLOW. # (it returned ENOSUP.) # therefore we're out of options--we simply cannot chown the # symlink. give up, suppress the error. # (which is what shutil always did in this circumstance.) pass if hasattr(st, 'st_flags'): try: lookup("chflags")(dst, st.st_flags, follow_symlinks=follow) except OSError as why: for err in 'EOPNOTSUPP', 'ENOTSUP': if hasattr(errno, err) and why.errno == getattr(errno, err): break else: raise _copyxattr(src, dst, follow_symlinks=follow) | **pathpy.copystat**\ ``(src, dst, *, follow_symlinks=True)``: | `docs <https://pathpy.readthedocs.io/en/latest/api.html#path.Path.copystat>`__ `source (path.py) <https://github.com/jaraco/path.py/blob/master/path.py>`__ .. code:: python def copystat(src, dst, *, follow_symlinks=True): """Copy all stat info (mode bits, atime, mtime, flags) from src to dst. If the optional flag `follow_symlinks` is not set, symlinks aren't followed if and only if both `src` and `dst` are symlinks. """ def _nop(*args, ns=None, follow_symlinks=None): pass # follow symlinks (aka don't not follow symlinks) follow = follow_symlinks or not (os.path.islink(src) and os.path.islink(dst)) if follow: # use the real function if it exists def lookup(name): return getattr(os, name, _nop) else: # use the real function only if it exists # *and* it supports follow_symlinks def lookup(name): fn = getattr(os, name, _nop) if fn in os.supports_follow_symlinks: return fn return _nop st = lookup("stat")(src, follow_symlinks=follow) mode = stat.S_IMODE(st.st_mode) lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns), follow_symlinks=follow) try: lookup("chmod")(dst, mode, follow_symlinks=follow) except NotImplementedError: # if we got a NotImplementedError, it's because # * follow_symlinks=False, # * lchown() is unavailable, and # * either # * fchownat() is unavailable or # * fchownat() doesn't implement AT_SYMLINK_NOFOLLOW. # (it returned ENOSUP.) # therefore we're out of options--we simply cannot chown the # symlink. give up, suppress the error. # (which is what shutil always did in this circumstance.) pass if hasattr(st, 'st_flags'): try: lookup("chflags")(dst, st.st_flags, follow_symlinks=follow) except OSError as why: for err in 'EOPNOTSUPP', 'ENOTSUP': if hasattr(errno, err) and why.errno == getattr(errno, err): break else: raise _copyxattr(src, dst, follow_symlinks=follow) ``copytree`` ============= | **shutil.copytree**\ ``(src, dst, symlinks=False, ignore=None, copy_function=<function copy2 at 0x...>, ignore_dangling_symlinks=False)`` | **pathpy.copytree**\ ``(src, dst, symlinks=False, ignore=None, copy_function=<function copy2 at 0x...>, ignore_dangling_symlinks=False)`` | **shutil.copytree**\ ``(src, dst, symlinks=False, ignore=None, copy_function=<function copy2 at 0x...>, ignore_dangling_symlinks=False)``: | `docs <https://docs.python.org/3/library/shutil.html#shutil.copytree>`__ `source (shutil.py) < https://github.com/python/cpython/tree/3.6/Lib/shutil.py>`__ .. code:: python def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False): """Recursively copy a directory tree. The destination directory must not already exist. If exception(s) occur, an Error is raised with a list of reasons. If the optional symlinks flag is true, symbolic links in the source tree result in symbolic links in the destination tree; if it is false, the contents of the files pointed to by symbolic links are copied. If the file pointed by the symlink doesn't exist, an exception will be added in the list of errors raised in an Error exception at the end of the copy process. You can set the optional ignore_dangling_symlinks flag to true if you want to silence this exception. Notice that this has no effect on platforms that don't support os.symlink. The optional ignore argument is a callable. If given, it is called with the `src` parameter, which is the directory being visited by copytree(), and `names` which is the list of `src` contents, as returned by os.listdir(): callable(src, names) -> ignored_names Since copytree() is called recursively, the callable will be called once for each directory that is copied. It returns a list of names relative to the `src` directory that should not be copied. The optional copy_function argument is a callable that will be used to copy each file. It will be called with the source path and the destination path as arguments. By default, copy2() is used, but any function that supports the same signature (like copy()) can be used. """ names = os.listdir(src) if ignore is not None: ignored_names = ignore(src, names) else: ignored_names = set() os.makedirs(dst) errors = [] for name in names: if name in ignored_names: continue srcname = os.path.join(src, name) dstname = os.path.join(dst, name) try: if os.path.islink(srcname): linkto = os.readlink(srcname) if symlinks: # We can't just leave it to `copy_function` because legacy # code with a custom `copy_function` may rely on copytree # doing the right thing. os.symlink(linkto, dstname) copystat(srcname, dstname, follow_symlinks=not symlinks) else: # ignore dangling symlink if the flag is on if not os.path.exists(linkto) and ignore_dangling_symlinks: continue # otherwise let the copy occurs. copy2 will raise an error if os.path.isdir(srcname): copytree(srcname, dstname, symlinks, ignore, copy_function) else: copy_function(srcname, dstname) elif os.path.isdir(srcname): copytree(srcname, dstname, symlinks, ignore, copy_function) else: # Will raise a SpecialFileError for unsupported file types copy_function(srcname, dstname) # catch the Error from the recursive copytree so that we can # continue with other files except Error as err: errors.extend(err.args[0]) except OSError as why: errors.append((srcname, dstname, str(why))) try: copystat(src, dst) except OSError as why: # Copying file access times may fail on Windows if getattr(why, 'winerror', None) is None: errors.append((src, dst, str(why))) if errors: raise Error(errors) return dst | **pathpy.copytree**\ ``(src, dst, symlinks=False, ignore=None, copy_function=<function copy2 at 0x...>, ignore_dangling_symlinks=False)``: | `docs <https://pathpy.readthedocs.io/en/latest/api.html#path.Path.copytree>`__ `source (path.py) <https://github.com/jaraco/path.py/blob/master/path.py>`__ .. code:: python def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False): """Recursively copy a directory tree. The destination directory must not already exist. If exception(s) occur, an Error is raised with a list of reasons. If the optional symlinks flag is true, symbolic links in the source tree result in symbolic links in the destination tree; if it is false, the contents of the files pointed to by symbolic links are copied. If the file pointed by the symlink doesn't exist, an exception will be added in the list of errors raised in an Error exception at the end of the copy process. You can set the optional ignore_dangling_symlinks flag to true if you want to silence this exception. Notice that this has no effect on platforms that don't support os.symlink. The optional ignore argument is a callable. If given, it is called with the `src` parameter, which is the directory being visited by copytree(), and `names` which is the list of `src` contents, as returned by os.listdir(): callable(src, names) -> ignored_names Since copytree() is called recursively, the callable will be called once for each directory that is copied. It returns a list of names relative to the `src` directory that should not be copied. The optional copy_function argument is a callable that will be used to copy each file. It will be called with the source path and the destination path as arguments. By default, copy2() is used, but any function that supports the same signature (like copy()) can be used. """ names = os.listdir(src) if ignore is not None: ignored_names = ignore(src, names) else: ignored_names = set() os.makedirs(dst) errors = [] for name in names: if name in ignored_names: continue srcname = os.path.join(src, name) dstname = os.path.join(dst, name) try: if os.path.islink(srcname): linkto = os.readlink(srcname) if symlinks: # We can't just leave it to `copy_function` because legacy # code with a custom `copy_function` may rely on copytree # doing the right thing. os.symlink(linkto, dstname) copystat(srcname, dstname, follow_symlinks=not symlinks) else: # ignore dangling symlink if the flag is on if not os.path.exists(linkto) and ignore_dangling_symlinks: continue # otherwise let the copy occurs. copy2 will raise an error if os.path.isdir(srcname): copytree(srcname, dstname, symlinks, ignore, copy_function) else: copy_function(srcname, dstname) elif os.path.isdir(srcname): copytree(srcname, dstname, symlinks, ignore, copy_function) else: # Will raise a SpecialFileError for unsupported file types copy_function(srcname, dstname) # catch the Error from the recursive copytree so that we can # continue with other files except Error as err: errors.extend(err.args[0]) except OSError as why: errors.append((srcname, dstname, str(why))) try: copystat(src, dst) except OSError as why: # Copying file access times may fail on Windows if getattr(why, 'winerror', None) is None: errors.append((src, dst, str(why))) if errors: raise Error(errors) return dst On Tue, Oct 18, 2022, 7:42 PM Paul Bryan <pbryan@anode.ca> wrote:
For my edification, how is copy in pathlib proposed to be implemented? Just call out to the OS to do the copy? Do the copy by reading and writing?
On Tue, 2022-10-18 at 19:24 -0400, Todd wrote:
On Tue, Oct 18, 2022 at 6:26 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 09:17, Todd <toddrjen@gmail.com> wrote:
On Tue, Oct 18, 2022 at 5:00 PM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, 19 Oct 2022 at 06:50, Todd <toddrjen@gmail.com> wrote:
Currently, pathlib supports pretty much all common filesystem
Whatever the original intent might have been behind pathlib, it is
now effectively the standard tool for filesystem operations. As such, excluding copying alone from basic operations creates an unnecessary hurdle for users. It also increases the difficulty of doing such operations since
operations. You can create, move, and delete both files and directories. One big omission is copying. You need shutil for that. there is no hint within the pathlib documentation on how to copy, new users basically need to google it to find out. That is fine for less common operations, but far from ideal from a basic one like copying.
Ah. I would look at solving this the other way: since this really isn't a path operation (in the same sense that moving/renaming and deleting are), keep it in shutil, but improve discoverability with a docs reference.
ChrisA
How is it any less of a "path operation" than moving files, reading and writing files, making directories, and deleting files?
At a file system level, those are nothing more than directory entry manipulation. (Reading and writing might count as slightly more than that, but practicality beats purity.) Copying a file involves a lot more work, and is not a simple operation done in the directory entry, so it's more of a separate tool.
Yes, and I would say practically copying a file is at least as much a valid part of pathlib as basic file/io. File i/o is a builtin for python, so putting it pathlib is even less useful than copying, which requires a separate import.
Also, not everyone agrees on what "copying" means, so there'll be various platform-specific concepts.
But Python already has such a tool, we are simply exposing it in a more convenient way.
In theory, you could take literally every possible action that could be done on a file and make it into a Path method. Do we need a method to read a .WAV file and return its headers, or is that better left in the 'wave' module? There's a reason that huge slabs of Python are built on protocols rather than methods. Assuming that shutil.copyfile accepts Path objects (and if it doesn't, that should definitely be fixed), is it really a problem for the Path objects to not have a copy method?
Operating on specific file types (other than text files) is not considered a core filesystem operation in any operating system I know. In contrast copying is, such as in UNIX 1, Multics, PC-DOS 1.0, and OS/2.
Again, whatever the original intent behind pathlib was, its effective use today is as the primary recommended way to deal with basic file operations. From what I can tell, copying is pretty much universally treated as a basic file operation in pretty much every major operating system. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VWTK6H... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/72GMTH... Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Oct 18, 2022, 19:39 Paul Bryan <pbryan@anode.ca> wrote:
For my edification, how is copy in pathlib proposed to be implemented? Just call out to the OS to do the copy? Do the copy by reading and writing?
Just a thin wrapper around shutil.copy2 and shutil.copytree. Most non-parsing pathlib operations are wrappers around corresponding operations from other modules.
On 10/18/22, Todd <toddrjen@gmail.com> wrote:
How is it any less of a "path operation" than moving files, reading and writing files, making directories, and deleting files?
Path-related operations involve creating, linking, symlinking, and listing directories and files, and peripherally also accessing file metadata such as size, timestamps, attributes, and permissions (i.e. filesystem indexing and bookkeeping). Reading and writing are I/O data operations on the contents of files. Copying a file is a path operation in that a new file gets created in the filesystem, but it's primarily an I/O operation, as are the read_text(), read_bytes(), write_text() and write_bytes() methods of Path objects. The ship sailed a long time ago. Path objects support I/O.
On 10/18/22, Todd <toddrjen@gmail.com> wrote:
So I think it would make a lot of sense to include copying inside pathlib. I propose adding a `copy` method to `pathlib.Path` (for concrete paths).
The specific call signature would be:
copy(dst, *, follow_symlinks=True, recursive=True, dir_exist_ok=True)
This will call `shutil.copytree` for directories if recursive is True, or `copy2` if recursive if False. For files it will call `copy2` always.
FYI, Barney Gale also proposed implementing copy() and copytree() methods recently. Barney is working on a significant restructuring of pathlib. https://discuss.python.org/t/incrementally-move-high-level-path-operations-f...
participants (7)
-
Barry -
Chris Angelico -
Christopher Barker -
Eryk Sun -
Paul Bryan -
Todd -
Wes Turner