[Python-bugs-list] [ python-Bugs-508779 ] Disable flat namespace on MacOS X

Thu, 07 Mar 2002 02:01:04 -0800

Bugs item #508779, was opened at 2002-01-26 03:44
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=508779&group_id=5470

Category: Extension Modules
Group: Python 2.2.1 candidate
Status: Open
Resolution: None
Priority: 7
Submitted By: Manoj Plakal (terabaap)
Assigned to: Michael Hudson (mwh)
Summary: Disable flat namespace on MacOS X

Initial Comment:

Python: v2.2
OS: MacOS X 10.1

MacOS X 10.1 introduced two forms of linking
for loadable modules: flat namespace and
two-level namespace. 

Python 2.2 is set up to use flat namespace
by default on OS X for building extension modules.

I believe that this is a problem since
it introduces spurious run-time linking
errors when loading 2 or more modules that 
happen to have common symbols. The Linux
and Windows implementations do not allow
symbols within modules to clash with
each other. This behavior also goes against
expectations of C extension module writers.

As a reproducible example, consider
two dummy modules foo (foomodule.c)
and bar (barmodule.c) both of which
are built with a common file baz.c
that contains some data variables.
With the current Python 2.2 on OS X 10.1,
only one of foo or bar can be imported,
but NOT BOTH, into the same interpreter
session.

The files can be picked up from
these URLs:
    http://yumpee.org/python/foomodule.c
    http://yumpee.org/python/barmodule.c
    http://yumpee.org/python/baz.c
    http://yumpee.org/python/setup.py

If I run "python setup.py build" with
Python 2.2 (built from the 2.2
source tarball) and then import
foo followed by bar, I get an
ImportError: "Failure linking new
module" (from Python/dynload_next.c).
If I add a call to NSLinkEditError()
to print a more detailed error message,
I see that the problem is multiple definitions
of the data variables in baz.c.

The above example works fine with Python 2.1
on Red Hat Linux 7.2 and Python 2.2a4 on Win98.

If I then edit /usr/local/lib/python2.2/Makefile
and change LDSHARED and BLDSHARED to
not use flat_namespace:
$(CC) $(LDFLAGS) -bundle -bundle_loader
/usr/local/bin/python2.2 -undefined error
then the problem is solved and I can load
both foo and bar without problems.

More info and discussion is available at
these URLs (also search groups.google.com
for "comp.lang.python OS X import bug"):

http://groups.google.com/groups?hl=en&threadm=j4sn8uu517.fsf%40informatik.hu-berlin.de&prev=/groups%3Fnum%3D25%26hl%3Den%26group%3Dcomp.lang.python%26start%3D75%26group%3Dcomp.lang.python
http://mail.python.org/pipermail/pythonmac-sig/2002-January/004977.html

It would be great to have this simple
change be applied to Python 2.2.1.

Manoj 
terabaap@yumpee.org

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2002-03-07 10:01

Message:
Logged In: YES 
user_id=6656

OK, checked in as 
setup.py revision 1.84
configure.in revision 1.293
(configure revision 1.284)
will move across to the branch shortly -- I presume you
wanted them on the trunk too?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-03-01 12:22

Message:
Logged In: YES 
user_id=45365

Here's a new version of the patch, against the current state of the tree (the whitespace normalization messed it up).

And I've now tested it against Manoj's example and it works fine, so I would say we go for it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-02-28 23:25

Message:
Logged In: YES 
user_id=45365

I turns out I was mistaken: BLDSHARED is used during build, LDSHARED for distutils when Python is installed. Attached is a patch (relative to release22-maint) that does two level namespaces.

It has no adverse effects on the core (i.e. make test still works fine).

Manoj: if you could test that this not only has on adverse effects but also fixes your problem that would be great. please checkout th release22-maint branch and apply this patch.

Michael: I'm assigning this to you, feel free to check it in immediately or wait for feedback from Manoj (or ignore it completely if you don't like it:-).

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-02-25 16:19

Message:
Logged In: YES 
user_id=45365

This solution still suffers from the problem we discussed on the Pythonmac-SIG, that BLDSHARED (or whatever replaces it) would need to have one value for -bundle_loader when building the standard extension modules and another during "normal operation"...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-02-25 11:21

Message:
Logged In: YES 
user_id=45365

I usurping this bug, but I'm not sure yet whether it's a good idea to fix this for 2.2.1, as it will break other extension modules that rely on the single flat namespace.

----------------------------------------------------------------------

Comment By: Manoj Plakal (terabaap)
Date: 2002-01-26 04:25

Message:
Logged In: YES 
user_id=150105

Another idea is to provide the option for
flat or 2-level namespace when building
extension modules on OS X, maybe as 
an extra flag passed to distutils.core.Extension
or somewhere else ...

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=508779&group_id=5470