Re: [Distutils] compiler selection with bdist_egg of setuptools

10 Oct 2006

      ...
...
While compiling some py/C source pkgs, such as:
    . RuleDispatch-0.5a0.dev-r2115.tar.gz
    . cElementTree-1.0.5-20051216.zip
    . PyProtocols-1.0a0dev_r2082.zip
    . Cheetah-2.0rc7.tar.gz
I noticed in the logs that each (after being unpacked) has
the following command invoked:
     setup.py -q bdist_egg --dist-dir /tmp/....
Since selecting a 'compiler' is not an option for the
"bdist_egg" command, are even the settings in
~/.pydistutils.cfg file:
     [build]
     compiler = unix
     [bdist_egg]
     plat_name = ppc
picked up ?
They appear not to be picked up, and I'm suspecting that
since the [build] settings are valid only for the "build" command,
it will not have any effect on the "bdist_egg" command used
by all the above comps. Is that accurate ?
...
No, it's not accurate.  bdist_egg invokes various build commands 
(such as build_ext) which get their default --compiler setting from 
the [build] configuration.
I retried, this time with the same entries in the setup.cfg file of TG:
     [build]
     compiler = unix
     build_temp = 

     [bdist_egg]
     plat_name = ppc
     verbose = 1
     keep_temp = 1
     dist_dir = 

I can see the verbose logs intially pick up all these settings:
     ...
     option dict for 'bdist_egg' command:
     {'dist_dir': ('setup.cfg',
       ''),
       'keep_temp': ('setup.cfg', '1'),
       'plat_name': ('setup.cfg', 'ppc'),
        'verbose': ('setup.cfg', '1')}
but only "verbose=1" appears to be used. The comps are all invoked as 
part of TG's:
      easy_install -f . --prefix=my_usr -s my_usr/bin TurboGears
but their respective setup commands get invoked as:
      setup.py -q bdist_egg --dist-dir /tmp/....
So, my attempts to specify/use from setup.cfg are getting overriden by 
the above "setup.py -q --dist..." call. This is the snippet that seems to 
indicate that:
      ...
      option dict for 'bdist_egg' command:
      {'dist_dir': ('command line',
        '/tmp/easy_install-3XaB58/RuleDispatch-0.5a0.dev-r2115/egg-dist-tmp-wG1geg')}
      ...
...
...
If plat_name is not picked up, how is setuptools figuring
out that the platform is i686 (given that the build is being
done for a i686 linux box). Verbose logs don't show where/how
that decision is made...
...
Try running with $DISTUTILS_DEBUG set to "yes".  You will get a ton of 
output showing what configuration files are being parsed and how the 
options for each command are getting set.  Look at what's happening when 
the second-level setup is run (i.e., the package being asked to 
'bdist_egg'); the first-level (easy_install itself) won't make any difference.
I did set $DISTUTILS_DEBUG to "yes". Its unclear how/where the fact that
its a "i686" platform is picked up frm... Taking the RuleDispatch comp as an
example, the first inkling it is going to use "i686" is this log entry:
     ...
     installing library code to build/bdist.linux-i686/egg
     ...
Its unclear how this was decided upon... Is it the platform on whicc the python
itslef was build or is it the platform of the gcc (in the path) ?

Also, what is unclear is how its being determined that the compiler is "gcc"
and how its options are being set. Taking RuleDispatch again as an example,
I see this:
     ...
     building 'dispatch._speedups' extension
     creating build/temp.linux-i686-2.4
     creating build/temp.linux-i686-2.4/src
     creating build/temp.linux-i686-2.4/src/dispatch
     gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
          -fPIC -I/<path-to-python-headers>/usr/include/python2.4 -c 
          src/dispatch/_speedups.c -o build/temp.linux-i686-2.4/src/dispatch/_speedups.o
     gcc -pthread -shared build/temp.linux-i686-2.4/src/dispatch/_speedups.o -o 
          build/lib.linux-i686-2.4/dispatch/_speedups.so
     ...
I looked in python, setuptools & RulesDisptach sources, but couldn't find the 
answers to these questions. If I knew where gcc/and-its-options were being 
ascertained, config'd, and used, then I could theoretically intervene and have
it point to my gcc and use my settings for it...

Again, Philip, thanks a bunch for all the time you've spent helping me on this.
/venkat

Re: [Distutils] compiler selection with bdist_egg of setuptools

Venkat Bommakanti