[New-bugs-announce] [issue38906] copy2 doesn't copy metadata on Windows and MacOS

[Gregg Tavares]

New submission from Gregg Tavares <github at greggman.com>:

MacOS have extended file attributes. Windows has both extended file attributes and alternate streams. In both OSes copy/cp and of course the Finder and Windows Explorer copy all this data. Python copy2 does not. 

On Windows it seems like CopyFileW needs to be called to actually do a full copy.

https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-copyfilew

On MacOS it appears to require copyItem

https://developer.apple.com/documentation/foundation/filemanager/1412957-copyitem

It's kind of unexpected to call a function to copy a file and have it not actually copy the file and have the user lose data

Windows example

> dir


    Directory: C:\Users\gregg\temp\test


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       11/24/2019   8:58 PM             28 original.txt


> Set-Content -Path original.txt -Stream FooBar

cmdlet Set-Content at command pipeline position 1
Supply values for the following parameters:
Value[0]: python should copy this too
Value[1]:

> Get-Content -Path original.txt -Stream FooBar
python should copy this too
> copy .\original.txt .\copied-with-copy.txt
> Get-Content -Path copied-with-copy.txt -Stream FooBar
python should copy this too
> C:\Users\gregg\AppData\Local\Programs\Python\Python38\python.exe
Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copy2("original.txt", "copied-with-python3.txt")
>>> exit()
> Get-Content -Path copied-with-python3.txt -Stream FooBar
> Get-Content : Could not open the alternate data stream 'FooBar' of the file 'C:\Users\gregg\temp\test\copied-with-python3.txt'.
At line:1 char:1
+ Get-Content -Path copied-with-python3.txt -Stream FooBar
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     + CategoryInfo          : ObjectNotFound: (C:\Users\gregg\...ith-python3.txt:String) [Get-Content], FileNotFoundException
     + FullyQualifiedErrorId : GetContentReaderFileNotFoundError,Microsoft.PowerShell.Commands.GetContentCommand





MacOS example

$ ls -l -@
total 1120
-rw-r--r--@ 1 gregg  staff  571816 Nov 24 18:48 original.jpg
	com.apple.lastuseddate#PS	    16 
	com.apple.macl	    72 
	com.apple.metadata:kMDItemWhereFroms	   530 
	com.apple.quarantine	    57 
$ cp original.jpg copied-with.cp 
$ ls -l -@                      
total 2240
-rw-r--r--@ 1 gregg  staff  571816 Nov 24 18:48 copied-with.cp
	com.apple.lastuseddate#PS	    16 
	com.apple.macl	    72 
	com.apple.metadata:kMDItemWhereFroms	   530 
	com.apple.quarantine	    57 
-rw-r--r--@ 1 gregg  staff  571816 Nov 24 18:48 original.jpg
	com.apple.lastuseddate#PS	    16 
	com.apple.macl	    72 
	com.apple.metadata:kMDItemWhereFroms	   530 
	com.apple.quarantine	    57
$python3
Python 3.8.0 (default, Nov 24 2019, 18:48:01) 
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copy2('original.jpg', 'copied-with-python3.jpg')
'copied-with-python3.jpg'
>>> exit()
$ ls -l -@
total 3360
-rw-r--r--@ 1 gregg  staff  571816 Nov 24 18:48 copied-with-python3.jpg
	com.apple.quarantine	    57 
-rw-r--r--@ 1 gregg  staff  571816 Nov 24 18:48 copied-with.cp
	com.apple.lastuseddate#PS	    16 
	com.apple.macl	    72 
	com.apple.metadata:kMDItemWhereFroms	   530 
	com.apple.quarantine	    57 
-rw-r--r--@ 1 gregg  staff  571816 Nov 24 18:48 original.jpg
	com.apple.lastuseddate#PS	    16 
	com.apple.macl	    72 
	com.apple.metadata:kMDItemWhereFroms	   530 
	com.apple.quarantine	    57

----------
components: Library (Lib), Windows, macOS
messages: 357395
nosy: greggman, ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: copy2 doesn't copy metadata on Windows and MacOS
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38906>
_______________________________________