Mailman 3 March 2009 - NumPy-Discussion

numpy int64 arrays and ctypes
by JJ March 26, 2009

March 26, 2009

Hello: I hope someone can give me a tip on how to solve this simple problem. I use Ubuntu 8.10 64 bit and want to pass a numpy integer array to a shared library C program. For some reason, my C program is not able to read the info passed in the integer array, but can read from a passed double array. It is probably a simple fix, but I do not know what it is. I have the same problem using Python 2.5 and Python 2.6 In the python program, I have something like: myInts = array([0,2,5,7]) #(of type 64 bit integer) myDoub= random.rand(100) myLib_ = ctypes.cdll['./mylib.so'] myfun = _myLib.myfun p_myInts = ndpointer(myInts.dtype, myInts.ndim, myInts.shape, "C_CONTIGUOUS") p_myDoub = ndpointer(myDoub.dtype, myDoub.ndim, myDoub.shape, "C_CONTIGUOUS") myfun.argtypes = [p_myInts, p_myDoub] myfun.restype = ctypes.c_int result = myfun(myInts, myDoub) Then in the header file I have: int myfun(int *myInts double *myDoub); Finally, in the C program I have: int myfun(int *myInts double *myDoub){ int i; for (i=0;, i<4; i++){ printf("myInt= %d, myDoub[myInt]= %g\n", myInt[i], myDoub[myInt[i]]); } } This does not work, the integer array is not passed correctly, but the double array is. Neither %d or %Int64 work for the printf statement. Any suggestions? John

2 1

[Announce] Numpy 1.3.0b1
by David Cournapeau March 26, 2009

March 26, 2009

Hi, I am pleased to announce the release of the first beta for numpy 1.3.0. You can find source tarballs and installers for both Mac OS X and Windows on the sourceforge page: https://sourceforge.net/projects/numpy/ The release note for the 1.3.0 release are below, The Numpy developers ========================= NumPy 1.3.0 Release Notes ========================= This minor includes numerous bug fixes, official python 2.6 support, and several new features such as generalized ufuncs. Highlights ========== Python 2.6 support ~~~~~~~~~~~~~~~~~~ Python 2.6 is now supported on all previously supported platforms, including windows. http://www.python.org/dev/peps/pep-0361/ Generalized ufuncs ~~~~~~~~~~~~~~~~~~ http://projects.scipy.org/numpy/ticket/887 Experimental Windows 64 bits support ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Numpy can now be built on windows 64 bits (amd64 only, not IA64), with both MS compilers and mingw-w64 compilers: This is *highly experimental*: DO NOT USE FOR PRODUCTION USE. See INSTALL.txt, Windows 64 bits section for more information on limitations and how to build it by yourself. New features ============ Formatting issues ~~~~~~~~~~~~~~~~~ Float formatting is now handled by numpy instead of the C runtime: this enables locale independent formatting, more robust fromstring and related methods. Special values (inf and nan) are also more consistent across platforms (nan vs IND/NaN, etc...), and more consistent with recent python formatting work (in 2.6 and later). Nan handling in max/min ~~~~~~~~~~~~~~~~~~~~~~~ The maximum/minimum ufuncs now reliably propagate nans. If one of the arguments is a nan, then nan is retured. This affects np.min/np.max, amin/amax and the array methods max/min. New ufuncs fmax and fmin have been added to deal with non-propagating nans. Nan handling in sign ~~~~~~~~~~~~~~~~~~~~ The ufunc sign now returns nan for the sign of anan. New ufuncs ~~~~~~~~~~ #. fmax - same as maximum for integer types and non-nan floats. Returns the non-nan argument if one argument is nan and returns nan if both arguments are nan. #. fmin - same as minimum for integer types and non-nan floats. Returns the non-nan argument if one argument is nan and returns nan if both arguments are nan. #. deg2rad - converts degrees to radians, same as the radians ufunc. #. rad2deg - converts radians to degrees, same as the degrees ufunc. #. log2 - base 2 logarithm. #. exp2 - base 2 exponential. #. logaddexp - add numbers stored as logarithms and return the logarithm of the result. #. logaddexp2 - add numbers stored as base 2 logarithms and return the base 2 logarithm of the result result. Masked arrays ~~~~~~~~~~~~~ TODO gfortran support on windows ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Gfortran can now be used as a fortran compiler for numpy on windows, even when the C compiler is Visual Studio (VS 2005 and above; VS 2003 will NOT work). Gfortran + Visual studio does not work on windows 64 bits (but gcc + gfortran does). It is unclear whether it will be possible to use gfortran and visual studio at all on x64. Arch option for windows binary ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Automatic arch detection can now be bypassed from the command line for the superpack installed: numpy-1.3.0-superpack-win32.exe /arch=nosse will install a numpy which works on any x86, even if the running computer supports SSE set. Deprecated features =================== Histogram ~~~~~~~~~ The semantics of histogram has been modified to fix long-standing issues with outliers handling. The main changes concern #. the definition of the bin edges, now including the rightmost edge, and #. the handling of upper outliers, now ignored rather than tallied in the rightmost bin. The previous behavior is still accessible using `new=False`, but this is deprecated, and will be removed entirely in 1.4.0. Documentation changes ===================== A lot of documentation improvements. New C API ========= Multiarray API ~~~~~~~~~~~~~~ The following functions have been added to the multiarray C API: * PyArray_GetEndianness: to get runtime endianness New defines ~~~~~~~~~~~ New public C defines are available for ARCH specific code through numpy/npy_cpu.h: * NPY_CPU_X86: x86 arch (32 bits) * NPY_CPU_AMD64: amd64 arch (x86_64, NOT Itanium) * NPY_CPU_PPC: 32 bits ppc * NPY_CPU_PPC64: 64 bits ppc * NPY_CPU_SPARC: 32 bits sparc * NPY_CPU_SPARC64: 64 bits sparc * NPY_CPU_S390: S390 * NPY_CPU_PARISC: PARISC New macros for CPU endianness has been added as well (see internal changes below for details): * NPY_BYTE_ORDER: integer * NPY_LITTLE_ENDIAN/NPY_BIG_ENDIAN defines Those provide portable alternatives to glibc endian.h macros for platforms without it. Portable NAN, INFINITY, etc... ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ npy_math.h now makes available several portable macro to get NAN, INFINITY: * NPY_NAN: equivalent to NAN, which is a GNU extension * NPY_INFINITY: equivalent to C99 INFINITY * NPY_PZERO, NPY_NZERO: positive and negative zero respectively Corresponding single and extended precision macros are available as well. All references to NAN, or home-grown computation of NAN on the fly have been removed for consistency. Internal changes ================ numpy.core math configuration revamp ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This should make the porting to new platforms easier, and more robust. In particular, the configuration stage does not need to execute any code on the target platform, which is a first step toward cross-compilation. http://projects.scipy.org/numpy/browser/trunk/doc/neps/math_config_clean.txt umath refactor ~~~~~~~~~~~~~~ A lot of code cleanup for umath/ufunc code (charris). Improvements to build warnings ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Numpy can now build with -W -Wall without warnings http://projects.scipy.org/numpy/browser/trunk/doc/neps/warnfix.txt Separate core math library ~~~~~~~~~~~~~~~~~~~~~~~~~~ The core math functions (sin, cos, etc... for basic C types) have been put into a separate library; it acts as a compatibility layer, to support most C99 maths functions (real only for now). The library includes platform-specific fixes for various maths functions, such as using those versions should be more robust than using your platform functions directly. The API for existing functions is exactly the same as the C99 math functions API; the only difference is the npy prefix (npy_cos vs cos). The core library will be made available to any extension in 1.4.0. CPU arch detection ~~~~~~~~~~~~~~~~~~ npy_cpu.h defines numpy specific CPU defines, such as NPY_CPU_X86, etc... Those are portable across OS and toolchains, and set up when the header is parsed, so that they can be safely used even in the case of cross-compilation (the values is not set when numpy is built), or for multi-arch binaries (e.g. fat binaries on Max OS X). npy_endian.h defines numpy specific endianness defines, modeled on the glibc endian.h. NPY_BYTE_ORDER is equivalent to BYTE_ORDER, and one of NPY_LITTLE_ENDIAN or NPY_BIG_ENDIAN is defined. As for CPU archs, those are set when the header is parsed by the compiler, and as such can be used for cross-compilation and multi-arch binaries.

13 58

Win32 MSI
by F. David del Campo Hill March 26, 2009

March 26, 2009

Dear Numpy Forum, I have found the Win64 (Windows x64) Numpy MSI installer in Sourceforge (numpy-1.3.0b1.win-amd64-py2.6.msi), but cannot find the Win32 (Windows i386) one. I have tried unpacking the Win32 EXE installer package (numpy-1.3.0b1-win32-superpack-python2.6.exe) to see if the MSI installer could be found inside, but without luck. Does the package I look for exist, and if so, where could someone point me to where I can download it from? Thank you for your help. Yours, David del Campo "The more corrupt the state, the more numerous the laws." -Gaius Cornelius Tacitus (ca. 56-ca. 117), Annals, Book III, 27

4 7

Summer of Code: Proposal for Implementing date/time types in NumPy
by Marty Fuhry March 26, 2009

March 26, 2009

Hello, Sorry for any overlap, as I've been referred here from the scipi-dev mailing list. I was reading through the Summer of Code ideas and I'm terribly interested in date/time proposal (http://projects.scipy.org/numpy/browser/trunk/doc/neps/datetime-proposal3.r…). I would love to work on this for a Google Summer of Code project. I'm a sophmore studying Computer Science and Mathematics at Kent State University in Ohio, so this project directly relates to my studies. Is there anyone looking into this proposal yet? Thank you. -Marty Fuhry

4 4

Fortran source
by Nils Wagner March 25, 2009

March 25, 2009

Hi all, How do I compile any legacy C and Fortran code in 64 bit using gcc/gfortran ? Any pointer would be appreciated. Thanks in advance Nils

1 0

Seg fault from numpy.rec.fromarrays
by Dan Yamins March 25, 2009

March 25, 2009

Hi all, I'm having a seg fault error from numpy.rec.fromarrays. I have a python list L = [Col1, Col2] where Col1 and Col2 are python lists of short strings (the max length of Col1 strings is 4 chars and max length of Col2 is 7 chars). The len of Col1 and Col2 is about 11500. Then I attempt >>> A = numpy.rec.fromarrays(L,names = ['Aggregates','__color__']) This should produce a numpy record array with two columns, one called 'Aggregates', the other called '__color__'. In and of it self, this runs. But then when I attempt to look at the contents of A, running the __getitem__ method, say by doing: >>> print A or >>> A.tolist() or >>> A[0] then I get a seg fault error. (Acutally, the segfault only occurs about 80% of the time I run these commands.) However, the __getitem__ method does work to produce attribute arrays from column names , e.g. >>> Ag = A['Aggregates'] or >>> col = A['__color__'] both produce (apparently) completely correct and working numpy arrays. Moreover, If I pickle the object A before looking at it, everything works fine. E.g. if I execute: >>> Hold_A = A.dumps() >>> A = numpy.loads(Hold_A) then A seems to work fine. (Also: pickling the list L = [Col1,Col2] first, before running the numpy.rec.fromarrays method, does not always fix the segfault.) Can someone explain why this might be happening, and how I can fix it (without having to use the pickling hack)? Thanks, Dan

3 5

Creating array containing empty lists
by Jesper Larsen March 25, 2009

March 25, 2009

Hi numpy people, I have a problem with array broadcasting for object arrays and list. I would like to create a numpy array containing empty lists (initially - I will append to them later): import numpy as npy a = npy.empty((2), dtype=npy.object_) # Works fine: for i in range(len(a)): a[i] = [] print a # Does not work: a[:] = [] a[:] = list() Is it possible to broadcast a list to all elements of a numpy array? Or will I have to loop through it and set the individual elements? Best regards, Jesper

3 2

trying to speed up the following....
by Brennan Williams March 25, 2009

March 25, 2009

I have an array (porvatt.yarray) of ni*nj*nk values. I want to create two further arrays. activeatt.yarray is of size ni*nj*nk and is a pointer array to an active cell number. If a cell is inactive then its activeatt.yarray value will be 0 ijkatt.yarray is of size nactive, the number of active cells (which I already know). ijkatt.yarray holds the ijk cell number for each active cell. My code looks something like... activeatt.yarray=zeros(ncells,dtype=int) ijkatt.yarray=zeros(nactivecells,dtype=int) iactive=-1 ni=currentgrid.ni nj=currentgrid.nj nk=currentgrid.nk for ijk in range(0,ni*nj*nk): if porvatt.yarray[ijk]>0: iactive+=1 activeatt.yarray[ijk]=iactive ijkatt.yarray[iactive]=ijk I may often have a million+ cells. So the code above is slow. How can I speed it up? TIA Brennan

2 5

Doc update for 1.3.0?
by Pauli Virtanen March 24, 2009

March 24, 2009

Hi all, (esp. David) Is there still time for a merge from the doc wiki for 1.3.x? Stefan already merged several reviewed docstrings a while ago. My current worry is that there's quite a bit good work still in there that would be useful to have in 1.3.0, but which, despite being an improvement over the what we have now, is not perfect. The only issue is actually cherry-picking "good" changes: - The new version should be a (possibly slight) improvement over the old one - No inappropriate content (eg. malicious doctests) Previously, I did the cherry-picking just by reading through the full patch, but now I added a feature to the doc wiki that allows to parallelize this: 1. Go to http://docs.scipy.org/numpy/patch/ 2. Click on a link in the patch list. This shows a diff vs. SVN 3. See if the change makes sense 4. Change "OK to apply" to Yes or No, accordingly. This serves a separate purpose from the review system and is separate from it: OK to Apply indicates that a docstring revision is "better than previously", whereas the review system indicates whether it is "perfect". I'm doing a first pass through them. I invite other people who have the reviewer permissions to check my judgment. Comments or suggestions? -- Pauli Virtanen

4 9

[ANN] mlabwrap 1.0.1
by Alexander Schmolck March 24, 2009

March 24, 2009

Mlabwrap allows pythonistas to interface to Matlab(tm) in a very straightforward fashion: >>> from mlabwrap import mlab >>> mlab.eig([[0,1],[1,1]]) array([[-0.61803399], [ 1.61803399]]) More at <http://mlabwrap.sourceforge.net>. Mlabwrap 1.0.1 is just a maintenance release that fixes a few bugs and simplifies installation (no more LD_LIBRARY_PATH hassles). No future (non-bugfix) releases of mlabwrap are currently planned, but if and when I find the time to finish overhauling and extending the API I will make an official release of scikits.mlabwrap, which probably won't be 100% backwards compatible. 'as -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01

1 0