Re: [Distutils] compiler selection with bdist_egg of setuptools
While compiling some py/C source pkgs, such as: . RuleDispatch-0.5a0.dev-r2115.tar.gz . cElementTree-1.0.5-20051216.zip . PyProtocols-1.0a0dev_r2082.zip . Cheetah-2.0rc7.tar.gz I noticed in the logs that each (after being unpacked) has the following command invoked: setup.py -q bdist_egg --dist-dir /tmp/....
Since selecting a 'compiler' is not an option for the "bdist_egg" command, are even the settings in ~/.pydistutils.cfg file: [build] compiler = unix [bdist_egg] plat_name = ppc picked up ?
They appear not to be picked up, and I'm suspecting that since the [build] settings are valid only for the "build" command, it will not have any effect on the "bdist_egg" command used by all the above comps. Is that accurate ?
No, it's not accurate. bdist_egg invokes various build commands (such as build_ext) which get their default --compiler setting from the [build] configuration.
I retried, this time with the same entries in the setup.cfg file of TG: [build] compiler = unix build_temp = <my_build_temp_dir> [bdist_egg] plat_name = ppc verbose = 1 keep_temp = 1 dist_dir = <my_dist_dir> I can see the verbose logs intially pick up all these settings: ... option dict for 'bdist_egg' command: {'dist_dir': ('setup.cfg', '<my_dist_dir>'), 'keep_temp': ('setup.cfg', '1'), 'plat_name': ('setup.cfg', 'ppc'), 'verbose': ('setup.cfg', '1')} but only "verbose=1" appears to be used. The comps are all invoked as part of TG's: easy_install -f . --prefix=my_usr -s my_usr/bin TurboGears but their respective setup commands get invoked as: setup.py -q bdist_egg --dist-dir /tmp/.... So, my attempts to specify/use from setup.cfg are getting overriden by the above "setup.py -q --dist..." call. This is the snippet that seems to indicate that: ... option dict for 'bdist_egg' command: {'dist_dir': ('command line', '/tmp/easy_install-3XaB58/RuleDispatch-0.5a0.dev-r2115/egg-dist-tmp-wG1geg')} ...
If plat_name is not picked up, how is setuptools figuring out that the platform is i686 (given that the build is being done for a i686 linux box). Verbose logs don't show where/how that decision is made...
Try running with $DISTUTILS_DEBUG set to "yes". You will get a ton of output showing what configuration files are being parsed and how the options for each command are getting set. Look at what's happening when the second-level setup is run (i.e., the package being asked to 'bdist_egg'); the first-level (easy_install itself) won't make any difference.
I did set $DISTUTILS_DEBUG to "yes". Its unclear how/where the fact that its a "i686" platform is picked up frm... Taking the RuleDispatch comp as an example, the first inkling it is going to use "i686" is this log entry: ... installing library code to build/bdist.linux-i686/egg ... Its unclear how this was decided upon... Is it the platform on whicc the python itslef was build or is it the platform of the gcc (in the path) ? Also, what is unclear is how its being determined that the compiler is "gcc" and how its options are being set. Taking RuleDispatch again as an example, I see this: ... building 'dispatch._speedups' extension creating build/temp.linux-i686-2.4 creating build/temp.linux-i686-2.4/src creating build/temp.linux-i686-2.4/src/dispatch gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/<path-to-python-headers>/usr/include/python2.4 -c src/dispatch/_speedups.c -o build/temp.linux-i686-2.4/src/dispatch/_speedups.o gcc -pthread -shared build/temp.linux-i686-2.4/src/dispatch/_speedups.o -o build/lib.linux-i686-2.4/dispatch/_speedups.so ... I looked in python, setuptools & RulesDisptach sources, but couldn't find the answers to these questions. If I knew where gcc/and-its-options were being ascertained, config'd, and used, then I could theoretically intervene and have it point to my gcc and use my settings for it... Again, Philip, thanks a bunch for all the time you've spent helping me on this. /venkat
At 11:03 AM 10/10/2006 -0700, Venkat Bommakanti wrote:
I can see the verbose logs intially pick up all these settings: ... option dict for 'bdist_egg' command: {'dist_dir': ('setup.cfg', '<my_dist_dir>'), 'keep_temp': ('setup.cfg', '1'), 'plat_name': ('setup.cfg', 'ppc'), 'verbose': ('setup.cfg', '1')} but only "verbose=1" appears to be used. The comps are all invoked as part of TG's: easy_install -f . --prefix=my_usr -s my_usr/bin TurboGears but their respective setup commands get invoked as: setup.py -q bdist_egg --dist-dir /tmp/....
Right, you need to look at the part of the output that comes *after* this, as that will be the processing for the child setup script, and that's the option processing that counts.
So, my attempts to specify/use from setup.cfg are getting overriden by the above "setup.py -q --dist..." call. This is the snippet that seems to indicate that: ... option dict for 'bdist_egg' command: {'dist_dir': ('command line',
'/tmp/easy_install-3XaB58/RuleDispatch-0.5a0.dev-r2115/egg-dist-tmp-wG1geg')} ...
You need to look for the log information that shows the reading of the configuration files; it would be above this point. You would see lines like: reading /some/dir/.pydistutils.cfg reading /some/path/to/setup.cfg etc. Check and see if the config you think you're using is even being read.
I did set $DISTUTILS_DEBUG to "yes". Its unclear how/where the fact that its a "i686" platform is picked up frm... Taking the RuleDispatch comp as an example, the first inkling it is going to use "i686" is this log entry: ... installing library code to build/bdist.linux-i686/egg ... Its unclear how this was decided upon... Is it the platform on whicc the python itslef was build or is it the platform of the gcc (in the path) ?
It defaults to the platform the distutils intends to build for. To cross-compile, you would have to explicitly override the platform. Which you're doing, but evidently the config file you think you are using is not being read.
Also, what is unclear is how its being determined that the compiler is "gcc" and how its options are being set. Taking RuleDispatch again as an example, I see this: ... I looked in python, setuptools & RulesDisptach sources, but couldn't find the answers to these questions.
I think you want to look in distutils.unixccompiler. This isn't a setuptools thing.
If I knew where gcc/and-its-options were being ascertained, config'd, and used, then I could theoretically intervene and have it point to my gcc and use my settings for it...
Again, Philip, thanks a bunch for all the time you've spent helping me on this. /venkat
No problem, just remember that neither distutils nor setuptools officially provide any support whatsoever for cross-compilation. If you actually succeed in getting something to work, I suggest writing up a HowTo or at least a description of what you ended up doing, so that others can benefit from your trials.
participants (2)
-
Phillip J. Eby -
Venkat Bommakanti