David Cournapeau wrote:
Robert wrote:
for use in binary distribution where I need only basics and fast startup/low memory footprint, I try to isolate the minimal ndarray type and what I need..
[..]
I think you need at least umath to make this work: when doing import numpy.core.multiarray, you pull out the whole numpy (because import foo.bar induces import foo I believe), whereas import multiarray just imports the multiarray C extension.
So my suggestion would be to modify numpy such as you can do import numpy after having removed most directories inside numpy. The big ones are distutils and f2py, which should already save 2.5 Mb and are not used at all in numpy itself. IIRC, the only problematic package is numpy.lib (we import numpy.lib in numpy.core IIRC).
Did like this - keeping a /numpy/core folder structure. In attachment is a README-minimal-numpy.txt Maybe thats interesting for many users / inclusion somewhere in the docs. Result is: some 300kB compressed. And startup very fast.
Strange: most imports in the package are relative - which is good for (flat) repackaging. Just one absolutue "from numpy.core.multiarray import ..." in a py file. Yet the 2 remaining DLL's obviously contain absolute imports of each other. Just because of that it is not possible to have the "minimal numpy" in a separate package folder with other name (without recompiling), but one needs to rename/remove the original numpy from the PYTHONPATH :-(
Maybe the absolute imports could be removed out of the DLLs in future. By #ifdef or so in the C code the newer Pythons also can be forced to do precisely relative import.
Robert
HOW TO create a minimal numpy ? (for fast import or freezing) ============================================================== rk 2009-05-16
* Make a copy of the original numpy folder tree * ( Rename the original tree / remove from sys.path ) * Keep only those files/folders:
numpy/ -rw-rw-rw- 1 user group 82 May 16 10:58 __init__.py -rw-rw-rw- 1 user group 593 Apr 28 21:16 version.py
numpy/core/ -rw-rw-rw- 1 user group 961 May 16 11:15 __init__.py -rw-rw-rw- 1 user group 8938 Apr 28 21:16 _internal.py -rw-rw-rw- 1 user group 16814 May 16 11:31 arrayprint.py -rw-rw-rw- 1 user group 61754 Apr 28 21:16 fromnumeric.py (optional) -rw-rw-rw- 1 user group 4635 Apr 28 21:16 info.py -rw-rw-rw- 1 user group 505132 Apr 28 21:16 multiarray.pyd (.so) -rw-rw-rw- 1 user group 56471 May 12 22:12 numeric.py -rw-rw-rw- 1 user group 20774 May 11 23:48 numerictypes.py -rw-rw-rw- 1 user group 318670 Apr 28 21:16 umath.pyd (.so)
(about 300kB compressed on win32)
* change numpy/__init__.py to simply:
========== numpy/__init__.py ======================================== """ a minimal numpy - only essential stuff of the core """ #$1 import core from core import * ========== end of numpy/__init__.py =================================
* run the patch below in numpy/core/
* (the strip-off regarding 'fromnumeric' is optional)
* from there re-add just the stuff you need
========== patch in numpy/core/ ======================================= diff -ur -x *.pyc -x *.pyo ....\numpyxx\core__init__.py .__init__.py --- ....\numpyxx\core__init__.py Tue Apr 28 21:16:32 2009 +++ .__init__.py Sat May 16 11:15:35 2009 @@ -1,36 +1,34 @@ - +# minimal numpy! # pyXpy transposer to original version: #$1 from info import __doc__ from numpy.version import version as __version__
import multiarray +from multiarray import * #$1 import umath import _internal # for freeze programs import numerictypes as nt multiarray.set_typeDict(nt.sctypeDict) -import _sort +#$1 import _sort +import numeric #$1 from numeric import * -from fromnumeric import * -from defmatrix import * -import defchararray as char -import records as rec -from records import * -from memmap import * -from defchararray import * -import scalarmath -del nt - -from fromnumeric import amax as max, amin as min, \ - round_ as round -from numeric import absolute as abs +#import fromnumeric #$1 +#$1 from fromnumeric import * +#$1 from defmatrix import * +#$1 import defchararray as char +#$1 import records as rec +#$1 from records import * +#$1 from memmap import * +#$1 from defchararray import * +#$1 import scalarmath +#$1 del nt + +#$1 from fromnumeric import amax as max, amin as min, \ +#$1 round_ as round +#$1 from numeric import absolute as abs
-__all__ = ['char','rec','memmap'] +__all__ = [] #$1 ['char','rec','memmap'] __all__ += numeric.__all__ -__all__ += fromnumeric.__all__ -__all__ += defmatrix.__all__ -__all__ += rec.__all__ -__all__ += char.__all__ - - -from numpy.testing import Tester -test = Tester().test -bench = Tester().bench +#$1 __all__ += fromnumeric.__all__ +#$1 __all__ += defmatrix.__all__ +#$1 __all__ += rec.__all__ +#$1 __all__ += char.__all__ Only in ....\numpyxx\core: _dotblas.pyd Only in ....\numpyxx\core: _sort.pyd diff -ur -x *.pyc -x *.pyo ....\numpyxx\core\arrayprint.py .\arrayprint.py --- ....\numpyxx\core\arrayprint.py Tue Apr 28 21:16:32 2009 +++ .\arrayprint.py Sat May 16 11:31:47 2009 @@ -17,7 +17,10 @@ import numerictypes as _nt from umath import maximum, minimum, absolute, not_equal, isnan, isinf from multiarray import format_longfloat -from fromnumeric import ravel +#$1 from fromnumeric import ravel +from numeric import asarray #$1 +def ravel(a, order='C'): #$1 + return asarray(a).ravel(order) #$1
def product(x, y): return x*y
Only in ....\numpyxx\core: defchararray.py Only in ....\numpyxx\core: defmatrix.py Only in ....\numpyxx\core: generate_numpy_api.py Only in ....\numpyxx\core: include Only in ....\numpyxx\core: memmap.py diff -ur -x *.pyc -x *.pyo ....\numpyxx\core\numeric.py .\numeric.py --- ....\numpyxx\core\numeric.py Tue Apr 28 21:16:32 2009 +++ .\numeric.py Tue May 12 22:12:07 2009 @@ -2050,6 +2050,14 @@ False_ = bool_(False) True_ = bool_(True)
-import fromnumeric -from fromnumeric import * -extend_all(fromnumeric) +#$1 import fromnumeric +#$1 from fromnumeric import * +#$1 extend_all(fromnumeric) +#$1{ +def any(a,axis=None, out=None): + try: + any = a.any + except AttributeError: + return _wrapit(a, 'any', axis, out) + return any(axis, out) +#$1} \ No newline at end of file diff -ur -x *.pyc -x *.pyo ....\numpyxx\core\numerictypes.py .\numerictypes.py --- ....\numpyxx\core\numerictypes.py Tue Apr 28 21:16:32 2009 +++ .\numerictypes.py Mon May 11 23:48:04 2009 @@ -78,7 +78,7 @@ 'ScalarType', 'obj2sctype', 'cast', 'nbytes', 'sctype2char', 'maximum_sctype', 'issctype', 'typecodes', 'find_common_type']
-from numpy.core.multiarray import typeinfo, ndarray, array, empty, dtype +from multiarray import typeinfo, ndarray, array, empty, dtype import types as _types
# we don't export these for import *, but we do want them accessible Only in ....\numpyxx\core: records.py Only in ....\numpyxx\core: scalarmath.pyd Only in ....\numpyxx\core: scons_support.py Only in ....\numpyxx\core: setup.py Only in ....\numpyxx\core: setup_common.py Only in ....\numpyxx\core: setupscons.py Only in ....\numpyxx\core: tests Only in ....\numpyxx\core: umath_tests.pyd