Merging new (optional) build system in the trunk ?

Hi, I would appreciate to get some comment on whether there is any chance to get my numpy.scons branch merge into the trunk at some near future. I feel to have reached the point where the only big thing missing is more testing. I tried to test it on many platforms, but there is a limit to what I can test just by myself. The branch has been conceived such as by default, the current numpy.distutils is used to build, and the scons-based build is used only by explicit request (another setup.py), so this does not force any use now: except the work related to numpyconfig (which can be tested separately, since it was done in a different branch, and is only a few lines of code), everything else is exactly the same than before. Thanks, David P.S: Just as a reminder, here are some of the already implemented goals currently: - all the facilities of scons are available to build extensions (parallel builds, automatic dependency, easy extension through of new builders and configuration checks, etc...) - ability to build shared libraries and Ctypes-based extension in a cross platform way. - optimization flags logic cleanly separated from the other flags: you can now easily replace, add, remove optimization flags without having to handle platform specific flags such as -fPIC, etc... - facility replacing system_info which is easier to use (one function call to support most common cases) - performance library checks are stronger, for more reliable build results. MKL, Sunperf, ATLAS, vecLib and Accelerate are already supported; new libraries can generally be added in a few lines, as they all use the same infrastructure. - the current (successfully) tested platforms are : Mac OS X Intel (gfortran + gcc), Linux (g77 + gcc, gfortran + gcc, intel compilers), MS Windows (mingw, VS 2003 + g77, VS 2005 + g77), Solaris (with sun compilers). - Generally, I tried to follow the autoconf philosophy (do not check for version, check for facilities). The fortran runtime, for example, is found automatically, as is the fortran name mangling. - easier maintenance: no more distutils monkey patching because upstream distutils does not have a clean API, the current codebase is a bit below 3000 lines of code, not counting scons (as a comparison, system_info.py is almost 2000 lines by itself), most of the code is totally cross platform, and platform specific are encapsulated; there are a few hacks, but they are implementation-only hacks. So concretely, I believe that most of the things stated in http://projects.scipy.org/scipy/numpy/wiki/DistutilsRevamp are available *now*. More concrete info here: http://projects.scipy.org/scipy/numpy/wiki/NumpyScons

David Cournapeau wrote:
Hi,
I would appreciate to get some comment on whether there is any chance to get my numpy.scons branch merge into the trunk at some near future. I feel to have reached the point where the only big thing missing is more testing. I tried to test it on many platforms, but there is a limit to what I can test just by myself. The branch has been conceived such as by default, the current numpy.distutils is used to build, and the scons-based build is used only by explicit request (another setup.py), so this does not force any use now: except the work related to numpyconfig (which can be tested separately, since it was done in a different branch, and is only a few lines of code), everything else is exactly the same than before.
I think there is a chance. I'm generally favorable to the idea. I was mainly waiting for the 1.0.4 release. I think we should be able to merge it over now if there are no serious objections. I'm also interested in moving scipy.weave into numpy (without the blitz converters which will stay in scipy). For 1.0.5, I would also like to see aligned memory and some steps towards optimization using intrinsics. -Travis O.

David Cournapeau wrote:
Hi,
I would appreciate to get some comment on whether there is any chance to get my numpy.scons branch merge into the trunk at some near future. I feel to have reached the point where the only big thing missing is more testing. I tried to test it on many platforms, but there is a limit to what I can test just by myself. The branch has been conceived such as by default, the current numpy.distutils is used to build, and the scons-based build is used only by explicit request (another setup.py), so this does not force any use now: except the work related to numpyconfig (which can be tested separately, since it was done in a different branch, and is only a few lines of code), everything else is exactly the same than before.
I think there is a chance. I'm generally favorable to the idea. I was mainly waiting for the 1.0.4 release. I think we should be able to merge it over now if there are no serious objections. Ok, great. Just tell me when you want to merge it, because I did a few
Travis E. Oliphant wrote: things to make testing on the buildbot easier: - I inverted the old and new setup.py (to be able to test my branch on the buildbot). - There is also some debugging on by default, which should be disabled in the trunk. This ia a 2 second change, but just to avoid any surprises for people regularly using the trunk Also, what is the timeline for 1.0.5 ? Ideally, once the scons work is merged into the trunk, I would like to work on a scipy branch which uses scons, so that both numpy and scipy next (1.0.5 and 0.7 ?) releases can both be built using scons.
I'm also interested in moving scipy.weave into numpy (without the blitz converters which will stay in scipy).
For 1.0.5, I would also like to see aligned memory and some steps towards optimization using intrinsics.
Are you speaking about SIMD intrisics ? cheers, David

On Nov 13, 2007 6:30 AM, Travis E. Oliphant <oliphant@enthought.com> wrote:
David Cournapeau wrote: <snip> I'm also interested in moving scipy.weave into numpy (without the blitz converters which will stay in scipy).
Hi, I'm excited to hear that weave would get "elevated" to a more exposed place ! However, I was wondering, if this fits with the idea to keep numpy minimalistic !? Does this mean that weave is "simpler" that I always thought ? [I always thought it was short of magic ;-) ] Regards, Sebastian Haase
participants (3)
-
David Cournapeau
-
Sebastian Haase
-
Travis E. Oliphant