Mailman 3 December 2008 - NumPy-Discussion

numpy on windows x64 with mingw: it (almost) works
by David Cournapeau 21 Dec '08

21 Dec '08

Hi, Just a few words to mention that I've finally managed to build numpy with the mingw-w64 project (port of mingw to AMD 64 bits MS OS), and it almost run OK. By almost, I mean that numpy.test() finishes without crash, assuming a few unit tests are skipped (some long double problems). Not all unit tests pass, but almost all of them are easy to fix problems in numpy (except for the long double problem). The drawback is that you can't do that just by using the mingw-w64 binaries, you have to build your own toolchain because of some bugs/missing features in mingw-w64. I've put the gory details there: http://scipy.org/scipy/numpy/wiki/MicrosoftToolchainSupport Hopefully, this should make it easier to add fortran support with gfortran, opening the possibility to have both numpy and scipy buildable on windows x64 with free compilers, David

3 2

lfilter
by Sturla Molden 20 Dec '08

20 Dec '08

I am wondering if not scipy.signal.lfilter ought to be a part of the core NumPy. Note that it is similar to the filter function found in Matlab, and it makes a complement to numpy.convolve. May I suggest that it is renamed or aliased to numpy.filter? Sturla Molden

2 1

Re: [Numpy-discussion] Missing numpy.i
by Nicolas ROUX 19 Dec '08

19 Dec '08

Hello, Under windows, using the latest numpy 1.20 and Swig release, I can't find anymore the numpy.i file. I used to find it in numpy/doc/ in the release 1.04. I tried to look some info the web without any success. Thanks for you help, Cheers, Nicolas.

4 6

immutable numpy arrays
by Geoffrey Irving 19 Dec '08

19 Dec '08

Currently numpy arrays are either writable or unwritable, but unwritable arrays can still be changed through other copies. This means that when a numpy array is passed into an interface that requires immutability for safety reasons, a copy always has to be made. One way around this would be to add a NPY_IMMUTABLE flag signifying that the contents of the array will never change through other copies. This flag would be in addition to the current NPY_WRITEABLE flag, so it would be fully backwards compatible. The flag would be propagated along slices and views. For example, a numpy array created from C code that guarantees immutability would have the flag set. If the array was passed back into a function that required an immutable array, the code could check the immutable flag and skip the copy. This behavior could also be used to implement safe copy-on-write semantics. Making this more generally useful would probably require additional flags to document whether a writeable copy of the array might exists (something like NPY_LEAKED) in order to avoid copies for newly created writeable arrays. Has the issue of immutability been considered before? It seems like a basic NPY_IMMUTABLE flag would be fairly easy to add without backwards compatibility issues, but the secondary features such as NPY_LEAKED would be more complicated. Thanks, Geoffrey

2 8

Recent umath changes
by David Cournapeau 19 Dec '08

19 Dec '08

Hi, There have been some changes recently in the umath code, which breaks windows 64 compilation - and I don't understand their rationale either. I have myself spent quite a good deal of time to make sure this works on many platforms/toolchains, by fixing the config distutils command and that platform specificities are contained in a very localized part of the code. It may not be very well documented (see below), but may I ask that next time someone wants to change file file, people ask for review before putting it directly in the trunk ? thanks, David How to deal with platform oddities: ----------------------------------- Basically, the code to replace missing C99 math funcs is, for an hypothetical double foo(double) function: #ifndef HAVE_FOO #udnef foo static double npy_foo(double a) { // define a npy_foo function with the same requirements as C99 foo } #define npy_foo foo #else double foo(double); #endif I think this code is wrong on several accounts: - we should not undef foo if foo is available: if foo is available at that point, it is a bug in the configuration, and should not be dealt in the code. Some cases may be complicated (IEEE754-related macro which are sometimes macro, something functions, etc...), but that should be dealt in very narrow cases. - we should not declare our own function: function declaration is not portable, and varies among OS/toolchains. Some toolchains use intrinsic, some non standard inline mechanism, etc... which can crash the resulting binary because there is a discrepency between our code calling conventions and the library convention. The reported problem with VS compiler on amd64 is caused by this exact problem. Unless there is a strong rationale otherwise, I would like that we follow how "autoconfed" projects do. They have long experience on dealing with platforms idiosyncrasies, and the above method is not the one they follow. They follow the simple: #ifnfdef HAVE_FOO //define foo #endif And deal with platform oddities in the *configuration* code instead of directly in the code. That really makes my life easier when I deal with windows compilers, which are already painful enough to deal with as it is.

3 8

ANN: PyTables 2.1 (final) released
by Francesc Alted 19 Dec '08

19 Dec '08

=========================== Announcing PyTables 2.1 =========================== PyTables is a library for managing hierarchical datasets and designed to efficiently cope with extremely large amounts of data with support for full 64-bit file addressing. PyTables runs on top of the HDF5 library and NumPy package for achieving maximum throughput and convenient use. PyTables 2.1 introduces important improvements, like much faster node opening, creation or navigation, a file-based way to fine-tune the different PyTables parameters (fully documented now in a new appendix of the manual) and support for multidimensional atoms in EArray/CArray objects. Regarding the Pro edition, four different kinds of indexes are supported so that the user can choose the best for her needs. Also, and due to the introduction of the concept of chunkmaps in OPSI, the responsiveness of complex queries with low selectivity has improved quite a lot. And last but not least, it is possible now to sort tables by a specific field with no practical limit in size (tables up to 2**48 rows). Also, a lot of work has gone in the reworking of the "Optimization tips" chapter of the manual where many benchmarks have been redone using newer software and machines and a few new sections have been added. In particular, see the new "Fine-tuning the chunksize" section where you will find an in-deep introduction to the subject of chunking and the "Indexing and Solid State Disks (SSD)" where the advantages of using low-latency SSD disks have been analysed in the context of indexation. In case you want to know more in detail what has changed in this version, have a look at ``RELEASE_NOTES.txt`` in the tarball. Find the HTML version for this document at: http://www.pytables.org/moin/ReleaseNotes/Release_2.1 You can download a source package of the version 2.1 with generated PDF and HTML docs and binaries for Windows from http://www.pytables.org/download/stable For an on-line version of the manual, visit: http://www.pytables.org/docs/manual-2.1 Finally, you can get an evaluation version for PyTables Pro in: http://www.pytables.org/download/evaluation Resources ========= Go to the PyTables web site for more details: http://www.pytables.org About the HDF5 library: http://hdfgroup.org/HDF5/ About NumPy: http://numpy.scipy.org/ Acknowledgments =============== Thanks to many users who provided feature improvements, patches, bug reports, support and suggestions. See the ``THANKS`` file in the distribution package for a (incomplete) list of contributors. Many thanks also to SourceForge who have helped to make and distribute this package! And last, but not least thanks a lot to the HDF5 and NumPy (and numarray!) makers. Without them PyTables simply would not exist. Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. ---- **Enjoy data!** -- The PyTables Team

1 0

array not appending
by Prashant Saxena 18 Dec '08

18 Dec '08

Hi, This is copied from ipython console. In [42]: import numpy as np In [43]: ST = np.empty([], dtype=np.float32) In [44]: np..append(ST, 10.0) Out[44]: array([ 3.83333603e-38, 1.00000000e+01]) In [45]: np.append(ST, 10.0) Out[45]: array([ 3.83333603e-38, 1.00000000e+01]) In [46]: print ST 3.83333602707e-038 What's wrong here? win XP 32 numpy 1.2.1 python 2.5.2 Prashant Bollywood news, movie reviews, film trailers and more! Go to http://in.movies.yahoo.com/

7 10

ANNOUNCE: EPD Py25 v4.1.30101_beta2 available for testing
by Dave Peterson 18 Dec '08

18 Dec '08

Hello, The Enthought Python Distribution's (EPD) early access program website is now hosting the beta 2 build of the upcoming EPD Py25 v4.1.301 release. We would very much appreciate your assistance in making EPD as stable and reliable as possible! Please join us in our efforts by downloading an installer for Windows, Mac OS X, or RedHat EL versions 3, 4, and 5 from the following website: http://www.enthought.com/products/epdearlyaccess.php The release notes for the beta2 build are available here: https://svn.enthought.com/epd/wiki/Py25/4.1.301/Beta2 Please provide any comments, concerns, or bug reports via the EPD Trac instance at https://svn.enthought.com/epd or via e-mail to epd-support(a)enthought.com. -- Dave About EPD --------- The Enthought Python Distribution (EPD) is a "kitchen-sink-included" distribution of the Python™ Programming Language, including over 60 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization, database adapters, and a lot of other tools right out of the box. http://www.enthought.com/products/epd.php It is currently available as a single-click installer for Windows XP (x86), Mac OS X (a universal binary for OS X 10.4 and above), and RedHat 3, 4, and 5 (x86 and amd64). EPD is free for academic use. An annual subscription and installation support are available for individual commercial use. Various workgroup, departmental, and enterprise level subscription options with support and training are also available. Contact us for more information!

1 0

error compiling umathmodule.c (numpy 1.3) on 64-bit windows xp
by Lin Shao 17 Dec '08

17 Dec '08

Hi, I found this earlier dialog about refactoring umathmodule.c (see bottom) where David mentioned it wasn't tested on 64-bit Windows. I tried compiling numpy 1.3.0.dev6118 on both a 32-bit and 64-bit Windows for Python 2.6.1 with VS 9.0, and not surprisingly, it worked on 32-bit but not on 64-bit: the compiler returned a non-specific "Internal Compiler Error" when working on umathmodule.c: ...... building 'numpy.core.umath' extension compiling C sources creating build\temp.win-amd64-2.6\Release\build creating build\temp.win-amd64-2.6\Release\build\src.win-amd64-2.6 creating build\temp.win-amd64-2.6\Release\build\src.win-amd64-2.6\numpy creating build\temp.win-amd64-2.6\Release\build\src.win-amd64-2.6\numpy\core creating build\temp.win-amd64-2.6\Release\build\src.win-amd64-2.6\numpy\core\src D:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ibuild\src.win-amd64-2.6\numpy\core\src -Inumpy\core\include -Ibuild\src.win-amd64-2.6\numpy\core\include/numpy -Inumpy\core\src -Inumpy\core\include -ID:\Python26\include -ID:\Python26\PC /Tcbuild\src.win-amd64-2.6\numpy\core\src\umathmodule.c /Fobuild\temp.win-amd64-2.6\Release\build\src.win-amd64-2.6\numpy\core\src\umathmodule.obj umathmodule.c numpy\core\src\umath_funcs_c99.inc.src(140) : warning C4273: '_hypot' : inconsistent dll linkage D:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(139) : see previous definition of '_hypot' numpy\core\src\umath_funcs_c99.inc.src(341) : warning C4273: 'sinf' : inconsistent dll linkage Internal Compiler Error in D:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe. You will be prompted to send an error report to Microsoft later. Any idea what's going on? I'd like to volunteer to test compiling numpy on 64-bit Windows system since I have a VS 2008 professional edition installed. Thanks! --lin 2008/10/5 David Cournapeau <david(a)ar.media.kyoto-u.ac.jp>: >> ...... >> >> > >> > #ifndef HAVE_FREXPF >> > static float frexpf(float x, int * i) >> > { >> > return (float)frexp((double)(x), i); >> > } >> > #endif >> > #ifndef HAVE_LDEXPF >> > static float ldexpf(float x, int i) >> > { >> > return (float)ldexp((double)(x), i); >> > } >> > #endif >> >> At the time I had tried to send further output following a checkout, >> but couldn't get it to post to the list, I think the message was too >> big or something. I will probably be having a go with 1.2.0, when I >> get some time. I'll let you know how it goes. > > I did some heavy refactoring for the above problems, and it should be > now easier to handle (in the trunk). I could build 1.2.0 with VS 2008 > express on 32 bits (wo blas/lapack), and there are some test errors - > albeit relatively minor at first sight. I have not tried on 64 bits. > > cheers, > > David > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion >

2 2

Unexpected MaskedArray behavior
by Ryan May 17 Dec '08

17 Dec '08

Hi, I just noticed the following and I was kind of surprised: >>>a = ma.MaskedArray([1,2,3,4,5], mask=[False,True,True,False,False]) >>>b = a*5 >>>b masked_array(data = [5 -- -- 20 25], mask = [False True True False False], fill_value=999999) >>>b.data array([ 5, 10, 15, 20, 25]) I was expecting that the underlying data wouldn't get modified while masked. Is this actual behavior expected? Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma

3 4

NumPy-Discussion December 2008