Tarek Ziadé wrote:
The build and instalation scheme should be part of the API. If you look in sysconfig, you get most of the install schemes via the API. If anything is missing, tell us.
(I am reading the code there: http://svn.python.org/projects/python/branches/tarek_sysconfig/Lib/distutils...) I have not tried running the code, but here is what I see missing: - anything related to build (where are libraries put in the build directory, where are the extensions put - necessary if you are interested in inter-operating with make) - I don't see the equivalent of distutils.command.install.install_libbase and similar prefixes - How can we retrieve the full customized install path ? I have not found how I could use the code in sysconfig to solve the use case I have written about before (related to retrieving install prefix).
Now for the build I am not sure what you are trying to perform here.
This is always the same use-case: assuming you push the build to a 3rd party tool like make, how can you make it work within distutils: build directories, compiler options, etc... If you are familiar with autootols, the idea could be similar in the sense that you would have a Makefile.in, and a distutils command build_make would generate a makefile, filling up the variables in makefile.in. You could also imagine to put those variables in a separate file, which could be queried from an API (using a makefile.in-like process for scons is not very practical for example). So the list of variables needed: - everything in sysconfig (but available for any compiler, including MS ones) - any distutils command option. What is needed here is a way to query the options given to command globally, that is the possibility to access at any point in your setup.py every option of every command, without running them as first as we have to do currently (like running install_cmd within build_clib). To be even more down to earth, here is how a scons call looks like on linux for numscons: /usr/bin/python "/home/david/local/lib/python2.6/site-packages/numscons/scons-local/scons.py" -f numpy/core/SConstruct -I. scons_tool_path="" src_dir="numpy/core" pkg_path="numpy/core" pkg_name="numpy.core" log_level=50 distutils_libdir="../../../../build/lib.linux-i686-2.6" distutils_clibdir="../../../../build/temp.linux-i686-2.6" distutils_install_prefix="/usr/local/lib/python2.6/dist-packages/numpy/core" cc_opt=gcc cc_opt_path="/usr/bin" debug=0 f77_opt=gfortran f77_opt_path="/usr/bin" cxx_opt=g++ cxx_opt_path="/usr/bin" include_bootstrap=../../../../numpy/core/include bypass=0 import_env=0 silent=0 bootstrapping=1 To understand the weird relative paths, you need to be aware than scons (and waf) consider all the paths relatively to a build directory. Scons copy the sources files in the build directory (this is an oversimplification, but should be enough for this discussion). For example, here, we build the package numpy.core, so it looks like (relatively to the top setup.py file): cwd/setup.py /numpy/core/setup.py /build/lib.linux-i686-2.6/ # where distutils would put all the compiled code if build_ext were used /build/scons/ # any code generated by scons is put here numpy/core # the build directory for the current package numpy.core We pass the following informations: * distutils_libdir: where distutils put/expects python extensions (which is replaced by current directory + the src_dir in case of in-place build) * distutils_install_prefix: install prefix + subpackage path. Where the extensions would be installed * distutils_clibdir: where disstutils put/expects pure C libraries (the ones built through build_clib) * numscons also encode compilation options in .ini-like files (regularly auto-generated from the info at sysconfig). The idea is that if someone wants to modify the flags, there is no need to touch any python code. Right now, I have to reimplement the distutils code to find those, as the logic is not available through API. Generally, the build directories are only known through the corresponding build_* commands, but they become available only after running the commands, which we want to avoid.
And I made a proposal for this last one too in this thread. But, like the Extensions proposal I have made, you seem not to hear them.
Because I genuinely don't understand why it is required not to replace distutils but it is ok to change its behavior in a fundamental way. For some reasons (which may well just be my own fault), I fail to see any reason to start from existing code, publicly and widely used, if it is ok to break it and change its behavior, especially if it will be done several times. I was not at Pycon, so there may have been some other requirements I have missed.
Sorry, but that's pretty vague. Are you mentioning the changes in build_ext, or something else ?
For 2.6.3, yes. For python 2.6, the mingw support was broken because of VS manifests; our fortran tools were broken because the msvc compiler was change (to msvc9compiler). There were also some other changes related to install directories and the --local option in 2.6 which caused issues. Several setuptools have also broken various parts of our own extensions in the past. If you really need to know the details for the last ones, I can look back in our svn log. cheers, David