[Numpy-discussion] Linker script, smaller source files and symbol visibility

Wed Apr 22 01:38:08 EDT 2009

On Tue, Apr 21, 2009 at 11:24 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
> On Mon, Apr 20, 2009 at 11:06 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Mon, Apr 20, 2009 at 10:13 PM, David Cournapeau <
>> david at ar.media.kyoto-u.ac.jp> wrote:
>>
>>> Charles R Harris wrote:
>>>
>>> >
>>> > Here is a link to the start of the old discussion
>>> > <
>>> http://article.gmane.org/gmane.comp.python.numeric.general/12974/match=exported+symbols+code+reorganization
>>> >.
>>> > You took part in it also.
>>>
>>> Thanks, I remembered we had the discussion, but could not find it. The
>>> different is that I am much more familiar with the technical details and
>>> numpy codebase now :) I know how to control exported symbols on most
>>> platform which matter (I can't test for AIX or HP-UX unfortunately - but
>>> I am perfectly fine with ignoring namespace pollution on those anyway),
>>> and I would guess that the only platforms which do not support symbol
>>> visibility in one way or the other do not support shared library anyway
>>> (some CRAY stuff, for example).
>>>
>>> Concerning the file size, I don't think anyone would disagree that they
>>> are too big, but we don't need to go the "java-way" of one
>>> file/class-function either. One first split which I personally like is
>>> API/implementation. For example, for multiarray.c, we would only keep
>>> the public PyArray_* functions, and put everything else in another file.
>>> The other very big file is arrayobject.c, and this one is already mostly
>>> organized in independent parts (buffer protocol, number protocol, etc...)
>>>
>>> Another thing I would like to do it to make the global C API array
>>> pointer a 'true' global variable instead of a static one. It took me a
>>> while when I was working on the hashing protocol for dtype to understand
>>> why it was crashing (the array pointer being static, every file has its
>>> own copy, so it was never initialized in the hashdescr.c file). I think
>>> a true global variable, hidden through a symbol map, is easier to
>>> understand and more reliable.
>>>
>>
>> I made an experiment along those lines a couple of years ago. There were
>> compilation problems because the needed include files weren't available. No
>> doubt that could be fixed in the build, but at some point I would like to
>> have real include files, not the generated variety. Generated include files
>> are kind of bogus IMHO, as they don't define an interface but rather reflect
>> whatever the function definition happens to be. So as any part of a split I
>> would also suggest writing the associated include files. That would also
>> make separate compilation possible, which would make it easier to do test
>> compilations while doing development.
>>
>
> The list of visible symbols has grown ;)
>
> ./multiarray.so
> 00039360 T PyArray_DescrHash
> 00073698 T _fini
> 0003e200 T _flat_copyinto
> 00007c58 T _init
> 00027d70 T initmultiarray
> 000728e0 T npy_acos
> 000723d0 T npy_acosf
> 00072880 T npy_acosh
> 00072370 T npy_acoshf
> 00072e50 T npy_acoshl
> 00072ee0 T npy_acosl
> 00072900 T npy_asin
> 000723f0 T npy_asinf
> 000728a0 T npy_asinh
> 00072390 T npy_asinhf
> 00072e80 T npy_asinhl
> 00072f10 T npy_asinl
> 000728c0 T npy_atan
> 000726b0 T npy_atan2
> 000721b0 T npy_atan2f
> 00072c10 T npy_atan2l
> 000723b0 T npy_atanf
> 00072860 T npy_atanh
> 00072350 T npy_atanhf
> 00072e20 T npy_atanhl
> 00072eb0 T npy_atanl
> 00071f70 T npy_ceil
> 00072040 T npy_ceilf
> 00071eb0 T npy_ceill
> 00072a90 T npy_cos
> 00072580 T npy_cosf
> 00072a30 T npy_cosh
> 00072520 T npy_coshf
> 000730b0 T npy_coshl
> 00073140 T npy_cosl
> 00071fd0 T npy_deg2rad
> 000720a0 T npy_deg2radf
> 00071f00 T npy_deg2radl
> 000726e0 T npy_exp
> 000727b0 T npy_exp2
> 00072720 T npy_exp2_1m
> 00072220 T npy_exp2_1mf
> 00072cc0 T npy_exp2_1ml
> 000722b0 T npy_exp2f
> 00072d50 T npy_exp2l
> 000721e0 T npy_expf
> 00072c60 T npy_expl
> 00072920 T npy_expm1
> 00072410 T npy_expm1f
> 00072f40 T npy_expm1l
> 00071f20 T npy_fabs
> 00071ff0 T npy_fabsf
> 00071e70 T npy_fabsl
> 00071f30 T npy_floor
> 00072000 T npy_floorf
> 00071e80 T npy_floorl
> 000725f0 T npy_fmod
> 000720f0 T npy_fmodf
> 00072b10 T npy_fmodl
> 00072680 T npy_hypot
> 00072180 T npy_hypotf
> 00072bc0 T npy_hypotl
> 00072940 T npy_log
> 00072960 T npy_log10
> 00072450 T npy_log10f
> 00072fa0 T npy_log10l
> 000727d0 T npy_log1p
> 000722d0 T npy_log1pf
> 00072d80 T npy_log1pl
> 00072700 T npy_log2
> 00072200 T npy_log2f
> 00072c90 T npy_log2l
> 000727f0 T npy_logaddexp
> 00073290 T npy_logaddexp2
> 00073390 T npy_logaddexp2f
> 000731a0 T npy_logaddexp2l
> 000722f0 T npy_logaddexpf
> 00072db0 T npy_logaddexpl
> 00072430 T npy_logf
> 00072f70 T npy_logl
> 000725c0 T npy_modf
> 000720c0 T npy_modff
> 00072ad0 T npy_modfl
> 00072650 T npy_pow
> 00072150 T npy_powf
> 00072b70 T npy_powl
> 00071fb0 T npy_rad2deg
> 00072080 T npy_rad2degf
> 00071ee0 T npy_rad2degl
> 000729f0 T npy_rint
> 000724e0 T npy_rintf
> 00073050 T npy_rintl
> 00072ab0 T npy_sin
> 000725a0 T npy_sinf
> 00072a50 T npy_sinh
> 00072540 T npy_sinhf
> 000730e0 T npy_sinhl
> 00073170 T npy_sinl
> 00072980 T npy_sqrt
> 00072470 T npy_sqrtf
> 00072fd0 T npy_sqrtl
> 00072a70 T npy_tan
> 00072560 T npy_tanf
> 00072a10 T npy_tanh
> 00072500 T npy_tanhf
> 00073080 T npy_tanhl
> 00073110 T npy_tanl
> 000729d0 T npy_trunc
> 000724c0 T npy_truncf
> 00073020 T npy_truncl
> ./umath_tests.so
> 00001a38 T _fini
> 000006b0 T _init
> 000014c0 T initumath_tests
> ./_sort.so
> 00019658 T _fini
> 000005a0 T _init
> 0000e700 T init_sort
> ./_dotblas.so
> 000049d8 T _fini
> 00000918 T _init
> 00004680 T init_dotblas
> ./umath.so
> 0002f368 T _fini
> 00004b64 T _init
> 0002ce10 T initumath
> 0002e120 T npy_acos
> 0002dc10 T npy_acosf
> 0002e0c0 T npy_acosh
> 0002dbb0 T npy_acoshf
> 0002e690 T npy_acoshl
> 0002e720 T npy_acosl
> 0002e140 T npy_asin
> 0002dc30 T npy_asinf
> 0002e0e0 T npy_asinh
> 0002dbd0 T npy_asinhf
> 0002e6c0 T npy_asinhl
> 0002e750 T npy_asinl
> 0002e100 T npy_atan
> 0002def0 T npy_atan2
> 0002d9f0 T npy_atan2f
> 0002e450 T npy_atan2l
> 0002dbf0 T npy_atanf
> 0002e0a0 T npy_atanh
> 0002db90 T npy_atanhf
> 0002e660 T npy_atanhl
> 0002e6f0 T npy_atanl
> 0002d7b0 T npy_ceil
> 0002d880 T npy_ceilf
> 0002d6f0 T npy_ceill
> 0002e2d0 T npy_cos
> 0002ddc0 T npy_cosf
> 0002e270 T npy_cosh
> 0002dd60 T npy_coshf
> 0002e8f0 T npy_coshl
> 0002e980 T npy_cosl
> 0002d810 T npy_deg2rad
> 0002d8e0 T npy_deg2radf
> 0002d740 T npy_deg2radl
> 0002df20 T npy_exp
> 0002dff0 T npy_exp2
> 0002df60 T npy_exp2_1m
> 0002da60 T npy_exp2_1mf
> 0002e500 T npy_exp2_1ml
> 0002daf0 T npy_exp2f
> 0002e590 T npy_exp2l
> 0002da20 T npy_expf
> 0002e4a0 T npy_expl
> 0002e160 T npy_expm1
> 0002dc50 T npy_expm1f
> 0002e780 T npy_expm1l
> 0002d760 T npy_fabs
> 0002d830 T npy_fabsf
> 0002d6b0 T npy_fabsl
> 0002d770 T npy_floor
> 0002d840 T npy_floorf
> 0002d6c0 T npy_floorl
> 0002de30 T npy_fmod
> 0002d930 T npy_fmodf
> 0002e350 T npy_fmodl
> 0002dec0 T npy_hypot
> 0002d9c0 T npy_hypotf
> 0002e400 T npy_hypotl
> 0002e180 T npy_log
> 0002e1a0 T npy_log10
> 0002dc90 T npy_log10f
> 0002e7e0 T npy_log10l
> 0002e010 T npy_log1p
> 0002db10 T npy_log1pf
> 0002e5c0 T npy_log1pl
> 0002df40 T npy_log2
> 0002da40 T npy_log2f
> 0002e4d0 T npy_log2l
> 0002e030 T npy_logaddexp
> 0002ead0 T npy_logaddexp2
> 0002ebd0 T npy_logaddexp2f
> 0002e9e0 T npy_logaddexp2l
> 0002db30 T npy_logaddexpf
> 0002e5f0 T npy_logaddexpl
> 0002dc70 T npy_logf
> 0002e7b0 T npy_logl
> 0002de00 T npy_modf
> 0002d900 T npy_modff
> 0002e310 T npy_modfl
> 0002de90 T npy_pow
> 0002d990 T npy_powf
> 0002e3b0 T npy_powl
> 0002d7f0 T npy_rad2deg
> 0002d8c0 T npy_rad2degf
> 0002d720 T npy_rad2degl
> 0002e230 T npy_rint
> 0002dd20 T npy_rintf
> 0002e890 T npy_rintl
> 0002e2f0 T npy_sin
> 0002dde0 T npy_sinf
> 0002e290 T npy_sinh
> 0002dd80 T npy_sinhf
> 0002e920 T npy_sinhl
> 0002e9b0 T npy_sinl
> 0002e1c0 T npy_sqrt
> 0002dcb0 T npy_sqrtf
> 0002e810 T npy_sqrtl
> 0002e2b0 T npy_tan
> 0002dda0 T npy_tanf
> 0002e250 T npy_tanh
> 0002dd40 T npy_tanhf
> 0002e8c0 T npy_tanhl
> 0002e950 T npy_tanl
> 0002e210 T npy_trunc
> 0002dd00 T npy_truncf
> 0002e860 T npy_truncl
> ./scalarmath.so
> 000217d8 T _fini
> 000013d0 T _init
> 00001b50 T initscalarmath
> $[charris at f9 core]$
>

And PyArray_API is a global symbol in multiarray.so

./multiarray.so
...
00087f60 D PyArray_API

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090421/204c6926/attachment.html>