[Python-Dev] (Licensing question) backport of shutil.copyfile() functionality

Giampaolo Rodola' g.rodola at gmail.com
Tue Mar 12 17:55:09 EDT 2019

On Tue, Mar 12, 2019 at 3:01 AM Glenn Linderman <v+python at g.nevcal.com>

> On 3/11/2019 4:35 PM, Giampaolo Rodola' wrote:
> Hello,
> some time ago I contributed a couple of patches to speedup shutil.copy*()
> functions:
> https://bugs.python.org/issue33671
> https://bugs.python.org/issue33695
> I would like to backport both functionalities so that they can be used on
> Python 2.7 and <3.8 and put it on PYPI. In order to do so I will basically
> have to copy some parts of shutil module (copytree() function + the
> unit-tests I added in BPO-33671 and a couple of other things). Are there
> constraints regarding this in terms of license? Am I supposed to use GPL?
> (I was thinking about using MIT)
> Note: in this package called "zerocopy" I will probably want to expose
> other functionalities such as tee(), splice() and CopyFileEx and
> TransmitFile on Windows, so it's probably gonna be half a backport and half
> a brand new project.
> Thanks.
> Thanks for the contributions. I don't know about the licensing.
> I wonder if you should make two packages, though... one just exactly a
> backport of the shutil speedups, and the second containing the new
> functionalities.

That was my initial thought as well (a "backports.shutil_copy" module
targeting copy* functions only), but (especially after playing with this
today) I think I have something a bit more ambitious in mind. I'm currently
experimenting with different things which could be baked in a third-party
lib and possibly contributed back to Python later on:

1) on OSX we could use f/copyfile() syscall to copy file attrs/metadata;
that may be useful to speedup shutil.copystat() and shutil.copymode()
2) copytree() on OSX could take advantage of f/copyfile() +
COPYFILE_RECURSIVE (which is probably too platform-specific for inclusion)
3) on Linux we could use copy_file_range() as a replacement for
os.sendfile() in shutil.copyfile() (it's supposed to be faster)
4) on Linux ioctl() + FICLONE could be used to implement CoW (copy on
write) instantaneous copies, and could be added as shutil.cow_copyfile()
(haven't look into Windows yet)
5) I was thinking about backporting socket.socket.sendfile() as well, which
uses os.sendfile() on POSIX but not TransmitFile on Windows (asyncio does
6)  another idea (but I'm not sure if it's possible, as I still have to dig
into that) is a new socket recvfile() function boosted up by tee() /
splice() on Linux, which maybe could be contributed back as

Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190312/abd71102/attachment.html>

More information about the Python-Dev mailing list