Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
Also, notice that we are in the process of adding a new python module in the stdlib, called "sysconfig", that will contains all installation paths scheme for all supported platforms.
I don't think we are talking about the same thing. If the pb was just missing API, we could have just added it to numpy.distutils. We override pretty much every command of distutils and setuptools anyway. I needed to obtain the install prefix, be it the default one, or the one set through --prefix, or the current directory for in place build/develop install. I need to know it when building C libraries, in build_clib. AFAIK, there is only one way to get the information: since the --prefix is only known once install.finalize_options() has run, you need to call the method from build_clib. Naive code could be something like class MyBuildClib(build_clib): def run(self): install_cmd = self.distribution.get_command_obj("install") if not install_cmd.finalized == 1: install_cmd.finalize_options() if self.inplace == 1: top = "." else: top = install_cmd.install_libbase # top is the top directory where libraries should be installed build_clib.run(self) It is already quite ugly, but it looks like it could work. It actually does in the simple case, like python setup.py install. But if you call python setup.py build_clib install --prefix=something, it does not. The error message is a typical example of distutils style: "error: must supply either prefix/exec- prefix/home or install-base/install-platbase -- not both". The only way I managed to make this work is to replace: install_cmd = self.distribution.get_command_obj("install")) by install_cmd = copy.copy(self.distribution.get_command_obj("install")) That's not an API problem, or a documentation problem. That's a fundamental issue because --prefix is only known from install, and when you start running commands in an order different than expected by distutils, you get weird errors (the above error is actually almost sensical if you really know distutils code). I passed over the fact that in some conditions that elude me ATM and are platform specific, install_libbase does not exist and raises an attribute error. If you have a configure / build / install like the vast majority of build systems out there, this problem disappears. I don't see how the problem can be fixed without touching how commands work. Moreover, that's a typical example where the only way to be robust is to check that every attribute you are using actually exist before. At that point, you really want to run back to m4, perl, autogenerated makefiles and shell programming :)
I am still waiting for you comment on the solution to remove the compile code from build_ext and have it at the Extension level, with an option to add new compilers in a more easier way.
I will try to document how scons does it. I think the basic idea could be reused in distutils.
If there's someone from the Numpy project that can help in isolating patches againts Distutils trunk in our issue tracker, I'd be more than happy to reduce the gap between the two projects.
If that was not clear, I am that guy. I have been the main maintainer of numpy.distutils and of numpy/scipy build infrastructure for some time now, David