[Pythonmac-SIG] problems compiling Python 2.2.1 from source - pyexpat.c module failure

[Bill Bug]

Hi All,

I've gotten stuck in my attempt to compile Python 2.2.1 from the source 
tarball on my 10.1.5 OS X machine that has the Apple Dev Suite installed 
with a functional gcc compiler.

Any help/corrections y'all could provide would be greatly appreciated.

Following the very clear README instructions on how to compile for Mac 
OS X, I built the Python core and most all the modules just fine using 
the recommended configure command:

./configure --enable-framework

I was even able to get my installed build to pass most all the 
<tests>.py.

All, that is, except for one module I know I need to use - and that the 
'smart' compile test suite knows should be functional under Darwin & OS 
X.  The module in question is pyexpat.o - the Python EXPAT-based XML 
Parser module.  The SAX module is also not created, since it requires an 
XML parser, which it expects will be coming via pyexpat.o.

I hunted through the Setup files in the /Modules directory and uncovered 
the line used by setup.py to create the compiler call designed to build 
this package.

#EXPAT_DIR=/usr/local/src/expat
#pyexpat pyexpat.c -I$(EXPAT_DIR)/xmlparse -L$(EXPAT_DIR) -lexpat

Unfortunately, my version of OS X does not contain that path, though it 
does contain EXPAT at /usr/local/expat.

To make a long story a bit shorter, I downloaded the EXPAT source and 
compiled it, just to make certain I had all the components required by 
pyexpat.c.  That placed the files the Python expat module requires - 
expat.h & libexpat.a - at /usr/local/include and /usr/local/lib, 
respectively.  That installation tested just fine.

I then added the following lines to my <PYTHON_HOME>/Modules/Setup.local 
file:
EXPAT_DIR=/usr/local/
pyexpat pyexpat.c -I$(EXPAT_DIR)/include -L$(EXPAT_DIR)/lib -lexpat

This info is used by the following code in setup.py to create the call 
to the gcc:

         expat_defs = []
         expat_incs = find_file('expat.h', inc_dirs, [])
         if expat_incs is not None:
             # expat.h was found
             expat_defs = [('HAVE_EXPAT_H', 1)]
         else:
             expat_incs = find_file('xmlparse.h', inc_dirs, [])

         if (expat_incs is not None and
             self.compiler.find_library_file(lib_dirs, 'expat')):
             exts.append( Extension('pyexpat', ['pyexpat.c'],
                                    define_macros = expat_defs,
                                    libraries = ['expat']) )

The following lines in the 'find_file(filename, std_dirs, paths) 
function ultimately define whether the proper gcc call will get 
constructed:

f = os.path.join(dir, filename)
if os.path.exists(f): return []

Based on these lines, the call expat_incs = find_file('expat.h', 
inc_dirs, []) returns an empty array when it finds the 'expat.h' file at 
one of the include paths in 'inc_dirs'.  Because of this, the line 
'expat_defs = [('HAVE_EXPAT_H', 1)]' is never run and the 'HAVE_EXPAT_H' 
macro is not defined during compilation.

This creates the following line in the makefile:
cc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -no-cpp-precomp -fno-common 
-dynamic -I. -I./Include -DHAVE_CONFIG_H   -I/usr/local/include -c 
./Modules/pyexpat.c -o Modules/pyexpat.o

which leads to the following stderr msg during compilation:
./Modules/pyexpat.c:20: xmlparse.h: No such file or directory
make: *** [Modules/pyexpat.o] Error 1

Examining the pyexpat.c source file shows this line is nested amongst a 
set of pre-proc directives.  I've listed these nested booleans below 
with the offending line marked with an arrow.

	#ifdef HAVE_EXPAT_H
		#include "expat.h"
		#ifdef XML_MAJOR_VERSION
			#define EXPAT_VERSION (0x10000 * XML_MAJOR_VERSION \
                        	+ 0x100 * XML_MINOR_VERSION \
                        	+ XML_MICRO_VERSION)
		#else
			/* Assume the oldest Expat that used expat.h and did not 
have version info */
			#define EXPAT_VERSION 0x015f00
		#endif
	#else /* !defined(HAVE_EXPAT_H) */
--->		#include "xmlparse.h"
		/* Assume Expat 1.1 unless told otherwise */
		#ifndef EXPAT_VERSION
			#define EXPAT_VERSION 0x010100
		#endif
	#endif /* !defined(HAVE_EXPAT_H) */

A quick scan of the logic confirms what I believe I see in setup.py - 
the 'HAVE_EXPAT_H' macro is not getting defined, because 'find_file'() 
in setup.py DOES find 'expat.h' at one of the paths in the INCLUDED 
directories variable sent into setup.py.

Have I misunderstood the compiler call in the makefile?  Have I 
misunderstood what setup.py doing?  I can't imagine why it would fail 
for this one module and work for all the others.  Does that relate to 
the fact that the this is the only module being defined via Setup.local?

  Not surprisingly, the following lines then show up when running the 
installation tests, along with many other statements:
...
test test_pyexpat skipped -- No module named pyexpat
test test_sax skipped -- no XML parsers available
...
3 skips unexpected on darwin:
     test_sax test_locale test_pyexpat
...

Once again, thanks very much for any assistance you can offer.

Cheers,
Bill Bug
Senior Analyst
Computer Vision Lab for Vertebrate Brain Mapping
MCP/Hahnemann University
Philadelphia, PA, USA 19129