[Pythonmac-SIG] problems compiling Python 2.2.1 from source - pyexpat.c module failure

[John Miller]

I first want to thank Bill for submitting this detailed information, and 
second, I want to confirm that exactly the same happened with me on both 
my desktop and laptop MacOS X machines (except that I didn't 
'--enable-framework'). If anyone can tell us how to fix this, I'd be 
very appreciative. I'm glad that Just told us the workaround, but I'm 
more interested in having the installation process work 'out of the 
box', if possible.

John Miller


On Friday, June 7, 2002, at 12:00  PM, Bill Bug <bbug@speakeasy.org> 
wrote:

>
> Hi All,
>
> I've gotten stuck in my attempt to compile Python 2.2.1 from the source
> tarball on my 10.1.5 OS X machine that has the Apple Dev Suite installed
> with a functional gcc compiler.
>
> Any help/corrections y'all could provide would be greatly appreciated.
>
> Following the very clear README instructions on how to compile for Mac
> OS X, I built the Python core and most all the modules just fine using
> the recommended configure command:
>
> ./configure --enable-framework
>
> I was even able to get my installed build to pass most all the
> <tests>.py.
>
> All, that is, except for one module I know I need to use - and that the
> 'smart' compile test suite knows should be functional under Darwin & OS
> X.  The module in question is pyexpat.o - the Python EXPAT-based XML
> Parser module.  The SAX module is also not created, since it requires an
> XML parser, which it expects will be coming via pyexpat.o.
>
> I hunted through the Setup files in the /Modules directory and uncovered
> the line used by setup.py to create the compiler call designed to build
> this package.
>
> #EXPAT_DIR=/usr/local/src/expat
> #pyexpat pyexpat.c -I$(EXPAT_DIR)/xmlparse -L$(EXPAT_DIR) -lexpat
>
> Unfortunately, my version of OS X does not contain that path, though it
> does contain EXPAT at /usr/local/expat.
>
> To make a long story a bit shorter, I downloaded the EXPAT source and
> compiled it, just to make certain I had all the components required by
> pyexpat.c.  That placed the files the Python expat module requires -
> expat.h & libexpat.a - at /usr/local/include and /usr/local/lib,
> respectively.  That installation tested just fine.
>
> I then added the following lines to my <PYTHON_HOME>/Modules/Setup.local
> file:
> EXPAT_DIR=/usr/local/
> pyexpat pyexpat.c -I$(EXPAT_DIR)/include -L$(EXPAT_DIR)/lib -lexpat
>
> This info is used by the following code in setup.py to create the call
> to the gcc:
>
>          expat_defs = []
>          expat_incs = find_file('expat.h', inc_dirs, [])
>          if expat_incs is not None:
>              # expat.h was found
>              expat_defs = [('HAVE_EXPAT_H', 1)]
>          else:
>              expat_incs = find_file('xmlparse.h', inc_dirs, [])
>
>          if (expat_incs is not None and
>              self.compiler.find_library_file(lib_dirs, 'expat')):
>              exts.append( Extension('pyexpat', ['pyexpat.c'],
>                                     define_macros = expat_defs,
>                                     libraries = ['expat']) )
>
> The following lines in the 'find_file(filename, std_dirs, paths)
> function ultimately define whether the proper gcc call will get
> constructed:
>
> f = os.path.join(dir, filename)
> if os.path.exists(f): return []
>
> Based on these lines, the call expat_incs = find_file('expat.h',
> inc_dirs, []) returns an empty array when it finds the 'expat.h' file at
> one of the include paths in 'inc_dirs'.  Because of this, the line
> 'expat_defs = [('HAVE_EXPAT_H', 1)]' is never run and the 'HAVE_EXPAT_H'
> macro is not defined during compilation.
>
> This creates the following line in the makefile:
> cc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -no-cpp-precomp -fno-common
> -dynamic -I. -I./Include -DHAVE_CONFIG_H   -I/usr/local/include -c
> ./Modules/pyexpat.c -o Modules/pyexpat.o
>
> which leads to the following stderr msg during compilation:
> ./Modules/pyexpat.c:20: xmlparse.h: No such file or directory
> make: *** [Modules/pyexpat.o] Error 1
>
> Examining the pyexpat.c source file shows this line is nested amongst a
> set of pre-proc directives.  I've listed these nested booleans below
> with the offending line marked with an arrow.
>
> 	#ifdef HAVE_EXPAT_H
> 		#include "expat.h"
> 		#ifdef XML_MAJOR_VERSION
> 			#define EXPAT_VERSION (0x10000 * XML_MAJOR_VERSION \
>                         	+ 0x100 * XML_MINOR_VERSION \
>                         	+ XML_MICRO_VERSION)
> 		#else
> 			/* Assume the oldest Expat that used expat.h and did not
> have version info */
> 			#define EXPAT_VERSION 0x015f00
> 		#endif
> 	#else /* !defined(HAVE_EXPAT_H) */
> --->		#include "xmlparse.h"
> 		/* Assume Expat 1.1 unless told otherwise */
> 		#ifndef EXPAT_VERSION
> 			#define EXPAT_VERSION 0x010100
> 		#endif
> 	#endif /* !defined(HAVE_EXPAT_H) */
>
> A quick scan of the logic confirms what I believe I see in setup.py -
> the 'HAVE_EXPAT_H' macro is not getting defined, because 'find_file'()
> in setup.py DOES find 'expat.h' at one of the paths in the INCLUDED
> directories variable sent into setup.py.
>
> Have I misunderstood the compiler call in the makefile?  Have I
> misunderstood what setup.py doing?  I can't imagine why it would fail
> for this one module and work for all the others.  Does that relate to
> the fact that the this is the only module being defined via Setup.local?
>
>   Not surprisingly, the following lines then show up when running the
> installation tests, along with many other statements:
> ...
> test test_pyexpat skipped -- No module named pyexpat
> test test_sax skipped -- no XML parsers available
> ...
> 3 skips unexpected on darwin:
>      test_sax test_locale test_pyexpat
> ...
>
> Once again, thanks very much for any assistance you can offer.
>
> Cheers,
> Bill Bug
> Senior Analyst
> Computer Vision Lab for Vertebrate Brain Mapping
> MCP/Hahnemann University
> Philadelphia, PA, USA 19129