[Pythonmac-SIG] py2app corrupting libraries -- patch for bug in py2app 2.5

[Kent Quirk]

This is a patch and explanation (written by one of my co-workers)
related to a question I posted to this list on 5/10. 


We have a Python application with a number of C++ extension modules,
some of
which dynamically load other C++ modules, all of which reference
additional
C++ frameworks we've built. The module dependency graph gets complex.

We've recently begun building those C++ pieces as Universal, and have
upgraded
to py2app 2.5 to construct a Universal app bundle. We're passing:

setup(
    app=['blobs.py'],
    data_files=[...],
    options=dict(
        py2app=dict(
            archs="ppc,i386",
            ...)))

However, we're getting an error of the form:

Thinning /Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/Frameworks/Events.framework/Versions/A/Events to ppc,
i386
/usr/bin/lipo: -extract ppc specified but fat file:
/Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/Frameworks/Events.framework/Versions/A/Events does
not contain that architecture

Examining previous output shows this some 25 lines above:

Thinning /Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/MacOS/../Frameworks/Events.framework/Versions/A/Event
s to i386

That shouldn't be happening. We inserted a test in util.thin_to_archs()
to
detect the case of a call that fails to specify both 'ppc' and 'i386',
and
tracked it back up into MachOStandalone.run(). That function constructs
an
archs dict that maps filenames to lists of architecture keywords. Then
it
looks up the architecture list for each file of interest.

Among the entries in that dict are:

 '/Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/Frameworks/Events.framework/Versions/A/Events':
  ['ppc', 'i386'],
 ...
 '/Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/MacOS/../Frameworks/Events.framework/Versions/A/Event
s': ['i386'],

with other similar duplications.

I'm guessing that it should work better to call os.path.realpath() on
the
filenames, both when constructing and when accessing that archs dict.
For
instance:

***
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/py2app/macholib/MachOStandalone.py~	Thu May 18 14:26:59 2006
---
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/py2app/macholib/MachOStandalone.py	Mon May 22 15:14:43 2006
***************
*** 117,130 ****
              mm = self.mms[arch]
              for node in mm.flatten(has_filename_filter):
                  machfiles[node.filename] = node
!                 archs.setdefault(node.filename, [])
!                 archs[node.filename].append(arch)
                  dest = os.path.join(contents,
node.filename[len(skipcontents):])
                  changemap[node.filename] = dest
  
          for node in machfiles.itervalues():
              filename = node.filename
!             nodearchs = archs[node.filename]
              changed = False
              for arch in nodearchs:
                  mm = self.mms[arch]
--- 117,129 ----
              mm = self.mms[arch]
              for node in mm.flatten(has_filename_filter):
                  machfiles[node.filename] = node
!                 archs.setdefault(os.path.realpath(node.filename),
[]).append(arch)
                  dest = os.path.join(contents,
node.filename[len(skipcontents):])
                  changemap[node.filename] = dest
  
          for node in machfiles.itervalues():
              filename = node.filename
!             nodearchs = archs[os.path.realpath(node.filename)]
              changed = False
              for arch in nodearchs:
                  mm = self.mms[arch]

Unfortunately this starts producing:

Thinning /Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/Frameworks/MathUtils.framework/Versions/A/MathUtils
to ppc, i386, i386
Thinning /Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/MacOS/../MacOS/../Frameworks/Python.framework/Version
s/2.4/Python to ppc, ppc, ppc, i386, i386, i386
/usr/bin/lipo: -extract ppc specified multiple times
ValueError: Error 1 returned by: '/usr/bin/lipo'
'/Users/nat/cbtr/blobs/Python/dist/Cosmic
Blobs.app/Contents/MacOS/../MacOS/../Frameworks/Python.framework/Version
s/2.4/Python' '-output' '/tmp/tmpoYncVt' '-extract' 'ppc' '-extract'
'ppc' '-extract' 'ppc' '-extract' 'i386' '-extract' 'i386' '-extract'
'i386'

So maybe it would work better if, instead of a list of architectures, we
were to use a set instead:

***
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/py2app/macholib/MachOStandalone.py~	Thu May 18 14:26:59 2006
---
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/py2app/macholib/MachOStandalone.py	Mon May 22 15:25:19 2006
***************
*** 117,130 ****
              mm = self.mms[arch]
              for node in mm.flatten(has_filename_filter):
                  machfiles[node.filename] = node
!                 archs.setdefault(node.filename, [])
!                 archs[node.filename].append(arch)
                  dest = os.path.join(contents,
node.filename[len(skipcontents):])
                  changemap[node.filename] = dest
  
          for node in machfiles.itervalues():
              filename = node.filename
!             nodearchs = archs[node.filename]
              changed = False
              for arch in nodearchs:
                  mm = self.mms[arch]
--- 117,129 ----
              mm = self.mms[arch]
              for node in mm.flatten(has_filename_filter):
                  machfiles[node.filename] = node
!                 archs.setdefault(os.path.realpath(node.filename),
set()).add(arch)
                  dest = os.path.join(contents,
node.filename[len(skipcontents):])
                  changemap[node.filename] = dest
  
          for node in machfiles.itervalues():
              filename = node.filename
!             nodearchs = list(archs[os.path.realpath(node.filename)])
              changed = False
              for arch in nodearchs:
                  mm = self.mms[arch]

With those changes, we can now get past this problem and produce a
working app bundle. Please consider incorporating some such fix into the
distributed
version of py2app. Thank you.