[Numpy-discussion] Splitting multiarray and umath into smaller files: ready for review

[Charles R Harris]

On Mon, Apr 27, 2009 at 2:53 AM, David Cournapeau <
david at ar.media.kyoto-u.ac.jp> wrote:

> Hi,
>
>    Following the "linker script, smaller source files and symbols
> visibility" discussion, I have implemented something which seems to work
> reasonably well. The code is available on github for review:
>
> http://github.com/cournape/numpy/tree/split_comp
>
> This branch implements the following:
>    - Both umath and multiarray sources are split into smaller files, in
> separate directories.
>    - By default, they are still all included into one big .c file, in
> which case nothing much is changed.
>    - The more interesting stuff happens when
> ENABLE_SEPARATE_COMPILATION is set to True in setup.py/setupscons.py
> file, in which case the files are actually compiled separately and link
> together at the end.
>
> This separate compilation mode uses some toolchain specific feature to
> avoid exporting symbols; I have tested it on mac os x, linux and windows
> (windows needs nothing as by default, no symbols are exported in
> dynamically loaded libraries). At some point, it should be enabled on
> platforms where it is known to work, I guess. One big advantage of
> separate compilation is the compilation/debug cycle which is much
> shorter (compilation time after modifying one file of multiarray goes
> from half a minute to a couple of seconds on my configuration). I split
> the file in multiarray following the C reference guide, as well as some
> logical units (one file for the mapping protocol, one file for
> iterators, one file for 'array constructors', etc....). There are still
> some things which do not belong where they should, but this can be
> improved later.
>
> There is a second branch which implements symbol export maps on a few
> platforms (darwin and linux for now), to specify exactly the symbols we
> want to export:
>
> http://github.com/cournape/numpy/tree/version_script
>
> This branch is much smaller, as it just implements a couple of functions
> for distutils. With this work, multiarray and umath now only export one
> function, and this can be easily extended to other extensions as well,
>

I think some of the fixes for too big arrays should be backported to 1.3.x
before this is merged.
That's r6851 and r6853. I'll do that.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090427/6950f782/attachment.html>