[Pythonmac-SIG] Re: Python on Mac OS X w/shared modules [long]

Tony Lownds tony-x@lownds.com
Sun, 1 Oct 2000 17:47:56 -0700


On Saturday, September 30, 2000, at 12:57 AM, Martin v. Loewis wrote:

> It then appears that you are not linking with libgcc.

Aha! Ok, another patch for configure is below. If after working on this =
I get an automatically built, fully working python with shared modules =
I'll try to move the changes into configure.in...=20

The build process goes quite smoothly, including building shared =
modules. The resulting python.x binary runs (I'm using --with-suffix=3D.x)=
 but make test and make install have a few problems!

I've detailed the problems below, hopefully others can try this out and =
get further than me with less workarounds. I'd be willing to give an ssh =
account to a knowledgable body really eager to help (email me). A =
temporary tarball of the resulting python install is at =
http://tony.lownds.com/macosx/; un-tar using the command-line:

su
tar zxvf python2.0b2_macosx.tar.gz -C /
rehash

Ok, back to the problems:

make test bombs at test_fcntl due to a strange bug caused by building on =
a case-insensitive filesystem:

test_fcntl
dyld: ./python.x multiple definitions of symbol _initfcntl
Modules/fcntlmodule.so definition of _initfcntl
Modules/FCNTLmodule.so definition of _initfcntl


This error ends the process and no more tests are performed. Continuing =
the tests (by commenting out import fcntl) gets me through the tests =
with a few more issues:

test test_format failed -- Writing: "'%#.117x' % (1,) =3D=3D =
'0x00000000000000000000000000000000000000000000000000000000000000000000000=
00000000000000000000000000000000000000000001' !=3D =
'0x00000000000000000000000000000000000000000000000000000000000000000000000=
0000000000000000000000000000000000000000000001'", expected: ''

test_largefile on HFS+ takes many minutes and sucks tons of resources =
but eventually works. Then the tests run again! Ouch!

test test_pty skipped --  Loaded module does not contain symbol =
_initFCNTL

test_re Segfaults due to low statcksize, running "limit stacksize 8192" =
then=20
"./python.x Lib/test/regrtest.py test_re" makes the test pass.

make install has more problems:
- it skips installing the shared library, because it checks for the =
wrong extension (this is from the output of make install):

if test -f libpython2.0.so; then \
                ./install-sh -c -m 644 libpython2.0.so /usr/local/lib; \
        else    true; \
        fi

The shared library is libpython2.0.dylib=20
workaround: cp libpython2.0.dylib /usr/local/lib

- generating the Lib/plat-darwin1.2 directory fails because the =
Lib/plat-generic script references python not python$(EXE)

export PATH; PATH=3D"`pwd`:$PATH"; \
        export PYTHONPATH; PYTHONPATH=3D"`pwd`/Lib"; \
        cd ./Lib/plat-darwin1.2; ./regen
python ../../Tools/scripts/h2py.py /usr/include/fcntl.h
./regen: permission denied: python [3]
.
.
.

workaround: edit Lib/plat-generic to reference python.x=20

The rest of make install runs smoothly; but then trying python somewhere =
outside the build tree gives:

[localhost:~] root# python.x
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
'import site' failed; use -v for traceback
Python 2.0b2 (#3, Oct  1 2000, 17:07:08)=20
[GCC 2.7.2.1] on darwin1.2
Type "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/', '/usr/local/lib/plat-darwin1.2', =
'/usr/local/lib/lib-tk', '/usr/local/lib/lib-dynload']
>>> sys.prefix
'/usr/local'
>>> sys.exec_prefix
'/usr/local'
>>>=20

Wierd - the prefix is *right* but the path is not. Looking in =
Modules/getpath.c reveals that WITH_NEXT_FRAMEWORK is screwing it up:

#ifdef WITH_NEXT_FRAMEWORK
    pythonModule =3D =
NSModuleForSymbol(NSLookupAndBindSymbol("_Py_Initialize"));
    /* Use dylib functions to find out where the framework was loaded =
from */
    buf =3D NSLibraryNameForModule(pythonModule);
    if (buf !=3D NULL) {
        /* We're in a framework. */
        strcpy(progpath, buf);

        /* Frameworks have support for versioning */
        strcpy(lib_python, "lib");
    }
    else {
        /* If we're not in a framework, fall back to the old way
           (even though NSNameOfModule() probably does the same thing.) =
*/
#endif

Basically, it finds the path P where libpython2.0.dylib is loaded from, =
and assumes P/../lib is the standard library location. Maybe the =
semantics of NSLibraryNameForModule have changed between Next and Darwin =
- I think a clean fix of this problem would require changes to the code =
above.=20

workaround: [localhost:/usr/local/bin] root# cat > python
#!/bin/sh
PYTHONHOME=3D/usr/local/lib/python2.0 \
PYTHONPATH=3D/usr/local/lib/python2.0:$PYTHONPATH \
        /usr/local/bin/python.x $*
[localhost:/usr/local/bin] root# chmod +x python


-Tony Lownds

*** Python-2.0b2/configure      Tue Sep 26 09:57:37 2000
--- configure   Sun Oct  1 13:27:51 2000
***************
*** 596,601 ****
--- 596,608 ----
                        sed -e 's/://' -e 's/\./_/'` && \
                        ac_sys_system=3Dnext && ac_sys_release=3D$4
 =20
+                 if test -z "$ac_sys_release"
+                 then
+                       set X `hostinfo | egrep 'Darwin Kernel.*:' | \
+                               sed -e 's/://'` && \
+                               ac_sys_system=3Ddarwin && =
ac_sys_release=3D$5
+               fi
+=20
                MACHDEP=3D"$ac_sys_system$ac_sys_release"
        fi
  fi
***************
*** 2583,2588 ****
--- 2590,2598 ----
  # Minor variations in building a framework between NextStep versions 4 =
and 5
 =20
  case $ac_sys_system/$ac_sys_release in
+   darwin/1.2)
+     ns_undef_sym=3D'_environ'
+     LIBTOOL_CRUFT=3D"-lcc_dynamic -arch_only ppc -U $ns_undef_sym" ;;
    next/4*)
      ns_undef_sym=3D'__environ'
      LIBTOOL_CRUFT=3D"-U $ns_undef_sym" ;;
***************
*** 2666,2672 ****
        hp*|HP*) LDSHARED=3D"ld -b";;
        OSF*) LDSHARED=3D"ld -shared -expect_unresolved \"*\"";;
        DYNIX/ptx*) LDSHARED=3D"ld -G";;
!       next/*)=20
                if test "$ns_dyld"
                then LDSHARED=3D'$(CC) $(LDFLAGS) -bundle -prebind'
                else LDSHARED=3D'$(CC) $(CFLAGS) -nostdlib -r';
--- 2676,2682 ----
        hp*|HP*) LDSHARED=3D"ld -b";;
        OSF*) LDSHARED=3D"ld -shared -expect_unresolved \"*\"";;
        DYNIX/ptx*) LDSHARED=3D"ld -G";;
!       darwin/*|next/*)
                if test "$ns_dyld"
                then LDSHARED=3D'$(CC) $(LDFLAGS) -bundle -prebind'
                else LDSHARED=3D'$(CC) $(CFLAGS) -nostdlib -r';
***************
*** 2741,2746 ****
--- 2751,2758 ----
        # crt1.o) gets erroneously defined as common, which breaks =
dynamic
        # loading of any modules which reference it in System.framework
        next/4*|next/5*) LINKFORSHARED=3D"-u __dummy -framework System" =
;;
+         # darwin doesn't like __dummy
+       darwin/*) LINKFORSHARED=3D"-framework System" ;;       =20
        SCO_SV*) LINKFORSHARED=3D"-Bdynamic -dy -Wl,-Bexport";;
        ReliantUNIX*) LINKFORSHARED=3D"-W1 -Blargedynsym";;
        FreeBSD*|NetBSD*)=20
***************
*** 3882,3888 ****
        AIX*) DYNLOADFILE=3D"dynload_aix.o";;
        BeOS*) DYNLOADFILE=3D"dynload_beos.o";;
        hp*|HP*) DYNLOADFILE=3D"dynload_hpux.o";;
!       next/*) DYNLOADFILE=3D"dynload_next.o";;
        *)
        # use dynload_shlib.c and dlopen() if we have it; otherwise stub
        # out any dynamic loading
--- 3894,3900 ----
        AIX*) DYNLOADFILE=3D"dynload_aix.o";;
        BeOS*) DYNLOADFILE=3D"dynload_beos.o";;
        hp*|HP*) DYNLOADFILE=3D"dynload_hpux.o";;
!       darwin/*|next/*) DYNLOADFILE=3D"dynload_next.o";;
        *)
        # use dynload_shlib.c and dlopen() if we have it; otherwise stub
        # out any dynamic loading
***************
*** 5461,5467 ****
  # check for --with-libm=3D...
 =20
  case $ac_sys_system in
! next) ;;
  BeOS) ;;
  *) LIBM=3D-lm
  esac
--- 5473,5479 ----
  # check for --with-libm=3D...
 =20
  case $ac_sys_system in
! darwin|next) ;;
  BeOS) ;;
  *) LIBM=3D-lm
  esac