[Distutils] setuptools and sdist on Mac OS X, tarfile flukes

[Hanno Schlichting]

Hi,

Phillip J. Eby wrote:
> At 05:38 PM 2/15/2007 +0000, Sidnei da Silva wrote:
>> Not sure if anyone reported a similar issue here since I'm not subscribed, but
>> here it goes.
>>
>> Some of the new plone.* eggs are being built on OS X by the developers, they
>> usually do 'setup.py sdist bdist_egg upload'. Sometimes they only do 'setup.py
>> sdist upload'
>>
>> Turns out that if you try to easy_install and there's only the sdist package,
>> but no egg, easy_install uses tarfile to unpack the package, build an egg and
>> install the egg. However, for some reason some of those nasty '._<filename>'
>> files that OS X creates end up in the tarball. And tarfile then fails to 
>> unpack
>> the tarball. One such package is 'plone.app.controlpanel 1.0a2'
>> (http://cheeseshop.python.org/packages/source/p/plone.app.controlpanel/plone.app.controlpanel-1.0a2.tar.gz#md5=2b2afeaba0d067f6b1a6707c6d69311d),
>> with the following traceback:
>> ...
>> I'm left wondering if the issue is:
>>
>>  a) Windows
>>  b) The nasty OS X file
>>  c) The Python 'tarfile' module
>>  d) setuptools
>>  e) None of the above :)
> 
> It appears to be a combination of "a" and "c".  Windows Python 2.5 does not 
> have an issue with the above file, nor does Cygwin Python 2.3.  Windows 
> Python 2.3 and 2.4 do have the problem.
> 
>> Thoughts?
> 
> Dunno.  It sounds like the people making these distributions have an 
> over-eager MANIFEST.in or an over-eager source control system.  They should 
> probably add a MANIFEST.in rule to exclude the files from the source release.

I looked into this a bit more and it seems you cannot fix this issue by
using a MANIFEST.in. The problem is that those resource forks aren't
found as regular files and so in return you cannot exclude them via a
filter.

According to
http://forums.macosxhints.com/archive/index.php/t-43243.html those
resource forks are created by the Mac OS tar program on the fly, which
is used internally by distutils.archive_util.

There is no command line switch to disable this behavior but some brave
soul looked through the source code of the modified tar and found that
setting an environment variable called
'COPY_EXTENDED_ATTRIBUTES_DISABLE' to any value should disable this
behavior.

I just tried to do a 'python setup.py sdist' for plone.app.controlpanel
and a 'tar tzvf plone.app.controlpanel-1.0b3dev-r13784.tar.gz' shows me
a ._setup.py file in there.

If I first do 'export COPY_EXTENDED_ATTRIBUTES_DISABLE=true' and then
rerun the sdist command I don't see the ._setup.py file anymore in the
tarball.

I wonder if this should be fixed on the distutils level? Does including
those resource forks make any sense for source tarballs in some
situation? Or could the archive_util set this environment variable on
Mac OS X prior to calling tar and reset it afterwards?

Hanno