On Sun, Jul 1, 2012 at 6:36 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Wed, Jun 27, 2012 at 9:05 PM, David Cournapeau <cournape@gmail.com> wrote:
On Wed, Jun 27, 2012 at 8:53 PM, Nathaniel Smith <njs@pobox.com> wrote:
But seriously, what compilers do we support that don't have -fvisibility=hidden? ...Is there even a list of compilers we support available anywhere?
Well, I am not sure how all this is handled on the big guys (bluegen and co), for once.
There is also the issue of the consequence on statically linking numpy to python: I don't what they are (I would actually like to make statically linked numpy into python easier, not harder).
All the docs I can find in a quick google seem to say that bluegene doesn't do shared libraries at all, though those may be out of date.
Also, it looks like our current approach is not doing a great job of avoiding symbol table pollution... despite all the NPY_NO_EXPORTS all over the source, I still count ~170 exported symbols on Linux with numpy 1.6, many of them with non-namespaced names ("_n_to_n_data_copy", "_next", "npy_tan", etc.) Of course this is fixable, but it's interesting that no-one has noticed. (Current master brings this up to ~300 exported symbols.)
It sounds like as far as our "officially supported" platforms go (linux/windows/osx with gcc/msvc), then the ideal approach would be to use -fvisibility=hidden or --retain-symbols-file to convince gcc to hide symbols by default, like msvc does. That would let us remove cruft from the source code, produce a more reliable result, and let us use the more convenient separate build, with no real downsides.
What cruft would it allow us to remove ? Whatever method we use, we need a whitelist of symbols to export. On the exported list I see on mac, most of them are either from npymath (npy prefix) or npysort (no prefix, I think this should be added). Once those are ignored as they should be, there are < 30 symbols exported.
(Static linking is trickier because no-one uses it anymore so the docs aren't great, but I think on Linux at least you could accomplish the equivalent by building the static library with 'ld -r ... -o tmp-multiarray.a; objcopy --keep-global-symbol=initmultiarray tmp-multiarray.a multiarray.a'.)
I am not sure why you say that static linking is not used anymore: I have met some people who do statically link numpy into python.
Of course there are presumably other platforms that we don't support or test on, but where we have users anyway. Building on such a platform sort of intrinsically requires build system hacks, and some equivalent to the above may well be available (e.g. I know icc supports -fvisibility). So I while I'm not going to do anything about this myself in the near future, I'd argue that it would be a good idea to: - Switch the build-system to export nothing by default when using gcc, using -fvisibility=hidden - Switch the default build to "separate" - Leave in the single-file build, but not "officially supported", i.e., we're happy to get patches but it's not used on any systems that we can actually test ourselves. (I suspect it's less fragile than the separate build anyway, since name clashes are less common than forgotten include files.)
I am fine with making the separate build the default (I have a patch somewhere that does that on supported platforms), but not with using -fvisibility=hidden. When I implemented the initial support around this, fvisibility was buggy on some platforms, including mingw 3.x I don't think changing what our implementation does here is worthwhile given that it works, and fsibility=hidden has no big advantages (you would still need to mark the functions to be exported). David