[Pythonmac-SIG] Building MySqldb for Universal Python

[Christopher Barker]

NOTE: I'm sending this all here, so it will be here for the archives


This is about trying to build a Universal MySqlDb module for the 
python.org Universal python (2.5.2 in this case). The issue is that you 
need to start with Universal MySQL libs. MySQL does indeed distribute 
Universal libs (ppc, ppc64, and i386 -- apparently not i386-64), but 
only as a tarball, not as as in installer in a .dmg.

However, there appears to be a bug in the tarball for version 5.1.30, 
and this note is about that:


I've got my curiosity up, so I've poked into this a bit more:

looking in:

/usr/local/mysql-5.1.30-osx10.4-universal/lib

things are clearly broken for example, we have:

15266468 Nov 20 23:32 libmysqlclient.16.0.0.dylib
4781040 Nov 15 14:57 libmysqlclient.16.dylib

The 16.0.0 version is Universal (ppc, ppc64, and i386), and the 16.dylib 
one is only i386 (and this in on a ppc machine!) different date, too...

But it's not just that. If I look in the ppc build I got from MySQL:

/usr/local/mysql-5.1.30-osx10.4-powerpc/lib

Nov 15 15:29 libmysqlclient.16.0.0.dylib
27 Jan  7 13:13 libmysqlclient.16.dylib -> libmysqlclient.16.0.0.dylib

so the 16.dylib version is supposed to be a symlink to the 16.0.0.dylib 
version, which is the usual *nix practice. Indeed, there are no symlinks 
in the lib dir at all.


So I think the way proper way to fix it is not to copy, but rather to 
fix the symlinks. Here's a script to do it (it needs to be run from the 
lib dir):

#!/bin/sh

# script to fix mysql lib dir in Universal dist

ln -s -f libmysqlclient.16.0.0.dylib libmysqlclient.16.dylib
ln -s -f libmysqlclient.16.0.0.dylib libmysqlclient.dylib

ln -s -f libmysqlclient_r.16.0.0.dylib libmysqlclient_r.16.dylib
ln -s -f libmysqlclient_r.16.0.0.dylib libmysqlclient_r.dylib

(also enclosed as a file)

Now to test the build of the python extension (MySQL-python-1.2.2)

first remove any old build and dist dirs (distutils is not so good at 
cleaning up):

$ rm -rf build dist

now try the build:

$ python setup.py build

no errors, and I see: "-arch ppc -arch i386" in the gcc line, so it 
looks like it's getting a universal build. Poking into:

build/lib.macosx-10.3-ppc-2.5 (distutils isn't really smart about 
Universal, so it creates this dir named for the host system)

I get:
file _mysql.so
_mysql.so: Mach-O universal binary with 2 architectures
_mysql.so (for architecture i386):      Mach-O bundle i386
_mysql.so (for architecture ppc):       Mach-O bundle ppc


which looks good. and:

$ otool -L _mysql.so _mysql.so:
         /usr/local/mysql/lib/libmysqlclient_r.16.dylib (compatibility 
version 17.0.0, current version 17.0.0)
         /usr/lib/libz.1.dylib (compatibility version 1.0.0, current 
version 1.2.3)
         /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, 
current version 88.3.9)
         /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current 
version 47.1.0)

and just to make sure:

$ file /usr/local/mysql/lib/libmysqlclient_r.16.dylib
/usr/local/mysql/lib/libmysqlclient_r.16.dylib: symbolic link to 
`libmysqlclient_r.16.0.0.dylib'

that's the link I created. And:

$ file /usr/local/mysql/lib/libmysqlclient_r.16.0.0.dylib

/usr/local/mysql/lib/libmysqlclient_r.16.0.0.dylib: Mach-O universal 
binary with 3 architectures

/usr/local/mysql/lib/libmysqlclient_r.16.0.0.dylib (for architecture 
i386):     Mach-O dynamically linked shared library i386
/usr/local/mysql/lib/libmysqlclient_r.16.0.0.dylib (for architecture 
ppc64):    Mach-O 64-bit dynamically linked shared library ppc64
/usr/local/mysql/lib/libmysqlclient_r.16.0.0.dylib (for architecture 
ppc):      Mach-O dynamically linked shared library ppc

Which is what we want.

$ python setup.py install

note: I have setuptools set to never install zip packages, 'cause they 
don't work right with py2app, and I like being able to see what's in 
them, etc. You can do this by creating a .pydistutils.cfg  file in your 
home dir, and putting:

[easy_install]
zip-ok = 0

in it.

now to test:

$ python
Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import _mysql
 >>>

At least it imports -- I don't have anything real to test with!

Further note:

It would be nice to distribute a binary of the python package. However, 
it would depend in that mysql dylib. I think the right way to do it 
would be to link the python extension against the static lib instead. To 
do that, I'm going to (temporarily) remove all the dylibs from the mysql 
lib dir, so it can only link to the static lib.

(in the mysql libs dir)

$ mkdir temp
$ mv *.dylib temp/

now build the python lib again:

$ rm -rf build dist

$ python setup.py build

which ran without errors.

check the lib:

$ otool -L _mysql.so
_mysql.so:
         /usr/lib/libz.1.dylib (compatibility version 1.0.0, current 
version 1.2.3)
         /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, 
current version 88.3.9)
         /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current 
version 47.1.0)


whoo hoo! it's not longer dependent on the mysql dylib!

$ python setup.py install

and I've got it installed and it seems to work, but still needs testing.

I've now got a binary egg file -- anyone want to test it? It's about 
3MB, which I think is too big to post here, but I'll send it to anyone 
that asks.

Also, we really need a place to post it, if it works!

-Chris







-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: FixMySqlUniveralLibs.sh
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20090108/4d78903f/attachment-0001.txt>