On Wed, Jun 27, 2012 at 8:57 PM, Dag Sverre Seljebotn <d.s.seljebotn@astro.uio.no> wrote:
On 06/27/2012 09:53 PM, Nathaniel Smith wrote:
On Wed, Jun 27, 2012 at 8:29 PM, David Cournapeau<cournape@gmail.com> wrote:
On Wed, Jun 27, 2012 at 8:07 PM, Nathaniel Smith<njs@pobox.com> wrote:
On Wed, Jun 27, 2012 at 7:50 PM, David Cournapeau<cournape@gmail.com> wrote:
On Wed, Jun 27, 2012 at 7:17 PM, Nathaniel Smith<njs@pobox.com> wrote:
Currently the numpy build system(s) support two ways of building numpy: either by compiling a giant concatenated C file, or by the more conventional route of first compiling each .c file to a .o file, and then linking those together. I gather from comments in the source code that the former is the traditional method, and the latter is the newer "experimental" approach.
It's easy to break one of these builds without breaking the other (I just did this with the NA branch, and David had to clean up after me), and I don't see what value we really get from having both options -- it seems to just double the size of the test matrix without adding value.
There is unfortunately a big value in it: there is no standard way in C to share symbols within a library without polluting the whole process namespace, except on windows where the default is to export nothing.
Most compilers support it (I actually know of none that does not support it in some way or the others), but that's platform-specific.
IIRC this isn't too tricky to arrange for with gcc
No, which is why this is supported for gcc and windows :)
, but why is this an issue in the first place for a Python extension module? Extension modules are opened without RTLD_GLOBAL, which means that they *never* export any symbols. At least, that's how it should work on Linux and most Unix-alikes; I don't know much about OS X's linker, except that it's unusual in other ways.
The pragmatic answer is that if it were not an issue, python itself would not bother with it. Every single extension module in python itself is built from a single compilation unit. This is also why we have this awful system to export the numpy C API with array of function pointers instead of simply exporting things in a standard way.
The array-of-function-pointers is solving the opposite problem, of exporting functions *without* having global symbols.
See this: http://docs.python.org/release/2.5.3/ext/using-cobjects.html
Looking quickly at the 2.7.3 sources, the more detailed answer is that python actually does not use RTLD_LOCAL (nor RTLD_GLOBAL), and what happens when neither of them is used is implementation-dependent. It seems to be RTLD_LOCAL on linux, and RTLD_GLOBAL on mac os x. There also may be consequences on the use of RTLD_LOCAL in embedded mode (I have ancient and bad memories with matlab related to this, but I forgot the details).
See, I knew OS X was quirky :-). That's what I get for trusting dlopen(3).
But seriously, what compilers do we support that don't have -fvisibility=hidden? ...Is there even a list of compilers we support available anywhere?
You could at the very least switch the default for a couple of releases, introducing a new flag with a "please email numpy-discussion if you use this" note, and see if anybody complains?
Yes, we could. That's actually why I set up travis-CI to build both configurations in the first place :) (see https://github.com/numpy/numpy/issues/315) David