Mailman 3 November 2006 - NumPy-Discussion

int32 twice in sctypes?
by Matthew Brett Nov. 28, 2006

Nov. 28, 2006

Hi, I was a bit confused by this on 32 bit linux: In [30]:sctypes['int'] Out[30]: [<type 'numpy.int8'>, <type 'numpy.int16'>, <type 'numpy.int32'>, <type 'numpy.int64'>, <type 'numpy.int32'>] Is it easy to explain the two entries for int32 here? I notice there is only one int32 entry for the same test on my 64 bit system. Thanks a lot, Matthew

3 2

numpy: azip ?
by Robert Nov. 28, 2006

Nov. 28, 2006

I often need a function to zip 2 (1D) arrays together to a 2D array - similar as python's zip() does. Found no function in numpy to do that directly without thinking a lot. Or is there one? Otherwise such thing would be helpful in future numpy. I have this in my tools: def azip( *ll ): return numpy.vstack(ll).transpose() Robert

3 2

idiom for deleting array elements
by Martin Wiechert Nov. 28, 2006

Nov. 28, 2006

Hi list, I'm looking for an idiom to delete a sparse subset S (given as integer indices into A) of elements from a 1d array A. Something like del L [I] if L is a list and I is a non-fancy index. My best idea is B = ones (A.shape, bool) B [S] = False A = A [B] Is there a better idiom (briefer, not requiring temporary boolean array)? Thanks, Martin

2 2

[Fwd: python-dev Summary for 2006-11-01 through 2006-11-15] - further discussion on buffer description
by Colin J. Williams Nov. 28, 2006

Nov. 28, 2006

-------- Original Message -------- Subject: python-dev Summary for 2006-11-01 through 2006-11-15 Date: Mon, 27 Nov 2006 04:52:10 +0000 (GMT) From: steven.bethard(a)gmail.com Newsgroups: gmane.comp.python.general python-dev Summary for 2006-11-01 through 2006-11-15 ++++++++++++++++++++++++++++++++++++++++++++++++++++ .. excerpt:: [The HTML version of this Summary is available at http://www.python.org/dev/summary/2006-11-01_2006-11-15] --------------------------------------------------------------- The buffer protocol and communicating binary format information --------------------------------------------------------------- The discussion of extending the buffer protocol to more binary formats continued this fortnight. Though the PIL_ had been used as an example of a library that could benefit from an extended buffer protocol, Fredrik Lundh indicated that future versions of the PIL_ would make the binary data model completely opaque, and instead provide a view-style API like:: view = object.acquire_view(region, supported formats) ... access data in view ... view.release() Along these lines, the discussion turned away from the particular C formats used in ``ctypes``, ``numpy``, ``array``, etc. and more towards the best way to communicate format information between these modules. Though it seemed like people were not completely happy with the proposed API of the new buffer protocol, the discussion seemed to skirt around any concrete suggestions for better APIs. In the end, the only thing that seemed certain was that a new buffer protocol could only be successful if it were implemented on all of the appropriate stdlib modules: ``ctypes``, ``array``, ``struct``, etc. .. _PIL: http://www.pythonware.com/products/pil/ Contributing threads: - `PEP: Adding data-type objects to Python <http://mail.python.org/pipermail/python-dev/2006-November/069706.html>`__ - `PEP: Extending the buffer protocol to share array information. <http://mail.python.org/pipermail/python-dev/2006-November/069709.html>`__ - `idea for data-type (data-format) PEP <http://mail.python.org/pipermail/python-dev/2006-November/069733.html>`__

1 0

[Fwd: python-dev Summary for 2006-10-16 through 2006-10-31]
by Colin J. Williams Nov. 28, 2006

Nov. 28, 2006

Numpy readers will likely be interested in this excerpt. Colin W. -------- Original Message -------- Subject: python-dev Summary for 2006-10-16 through 2006-10-31 Date: Mon, 27 Nov 2006 04:47:23 +0000 (GMT) From: steven.bethard(a)gmail.com Reply-To: python-list(a)python.org Newsgroups: gmane.comp.python.announce python-dev Summary for 2006-10-16 through 2006-10-31 ++++++++++++++++++++++++++++++++++++++++++++++++++++ .. Excerpt:: [The HTML version of this Summary is available at http://www.python.org/dev/summary/2006-10-16_2006-10-31] ========= Summaries ========= --------------------------------------------------------------- The buffer protocol and communicating binary format information --------------------------------------------------------------- Travis E. Oliphant presented a pre-PEP for adding a standard way to describe the shape and intended types of binary-formatted data. It was accompanied by a pre-PEP for extending the buffer protocol to handle such shapes and types. Under the proposal, a new ``datatype`` object would describe binary-formatted data with an API like:: datatype((float, (3,2)) # describes a 3*2*8=48 byte block of memory that should be interpreted # as 6 doubles laid out as arr[0,0], arr[0,1], ... a[2,0], a[1,2] datatype([( ([1,2],'coords'), 'f4', (3,6)), ('address', 'S30')]) # describes the structure # float coords[3*6] /* Has [1,2] associated with this field */ # char address[30] Alexander Belopolsky provided a nice example of why you might want to extend the buffer protocol along these lines. Currently, there's not much you can do with a basic buffer object. If you want to pass it to numpy_, you have to provide the type and shape information yourself:: >>> b = buffer(array('d', [1,2,3])) >>> numpy.ndarray(shape=(3,), dtype=float, buffer=b) array([ 1., 2., 3.]) By extending the buffer protocol appropriately so that the necessary information can be provided, you should be able to pass the buffer directly to numpy_ and have it understand the format itself:: >>> numpy.array(b) People were uncomfortable with the many ``datatype`` variants -- the constructor accepted types, strings, lists or dicts, each of which could specify the structure in a different way. Also, a number of people questioned why the existing ``ctypes`` mechanisms for describing binary data couldn't be used instead, particularly since ``ctypes`` could already describe things like function pointers and recursive types, which the pre-PEP could not. Travis said he was looking for a way to unify the data formats of all the ``array``, ``struct``, ``numpy`` and ``ctypes`` modules, and felt like using the ``ctypes`` approach was too verbose for use in the other modules. In particular, he felt like the ``ctypes`` use of type objects as binary-format specifiers was problematic because type objects were harder to manipulate at the C level. The discussion continued on into the next fortnight. .. _numpy: Contributing threads: - `PEP: Adding data-type objects to Python <http://mail.python.org/pipermail/python-dev/2006-October/069602.html>`__ - `PEP: Extending the buffer protocol to share array information. <http://mail.python.org/pipermail/python-dev/2006-October/069681.html>`__

1 0

var and std
by Aarre Laakso Nov. 28, 2006

Nov. 28, 2006

Hello, I was wondering if someone could explain the rationale for changing .var() and .std() in release 1.0b1 from normalizing by n-1 (unbiased estimate from sample) to normalizing by n (population)? I have found the note that this change happened in the Release Notes http://www.scipy.org/ReleaseNotes/NumPy_1.0 and the change itself in Changeset 2560 http://projects.scipy.org/scipy/numpy/changeset/2560 as well as a related documentation change in Ticket 388 http://projects.scipy.org/scipy/numpy/ticket/388 but I have not been able to find a description of why the change was made, despite searching the website, the Trac and the mailing list. I am aware of the argument that, if the difference between n and n-1 matters to you, then you are "up to no good". On the other hand, this change breaks a lot of my unit tests. It also seems to violate the principle of least surprise: every other numerical environment that I have used divides by n-1 by default. Examples include MATLAB: http://www.mathworks.com/access/helpdesk/help/techdoc/ref/index.html?/acces… http://www.mathworks.com/access/helpdesk/help/techdoc/ref/index.html?/acces… Octave: http://www.gnu.org/software/octave/doc/interpreter/Basic-Statistical-Functi… and R: http://finzi.psych.upenn.edu/R/library/stats/html/cor.html http://finzi.psych.upenn.edu/R/library/stats/html/sd.html It also seems to present an inconsistent interface: cov() still normalizes by n-1 instead of n. It also has a 'bias' parameter that allows normalizing by n, which is similar to the compromises provided in the other numerical packages listed above. As an aside, cov() also does not seem to be provided as a method, only as a function. In light of all that, I am sure there must have been a good reason for the change, and I am very curious what it was. Thanks for any insight you can offer. Regards, Aarre -- Aarre Laakso http://www.laakshmi.com/aarre/

1 0

recarray infos
by Lionel Roubeyrie Nov. 28, 2006

Nov. 28, 2006

Hi all, I'm interested by an array where I can call rows by their names and play with records, and recarray seems to be the solution. However, I can't find any complete documentation saying if recarray have the following behaviors: - stack datas : like numpy.array, is it possible to mak vstack, hstack, ...? - take several columns directly by their names, like recarray['col1','col3'] - setting extra-properties, like recarray['col1'].unit='µg.m^-3' After, numpy's doc says to use masked values for computation on arrays against nan values, but it seems that in recarray masked_values are not supported. Can I safely use nan for missing values? Thanks for your answers -- Lionel Roubeyrie - lroubeyrie(a)limair.asso.fr LIMAIR http://www.limair.asso.fr

1 0

Wanted: Numeric/NumPy compatible array creation expression
by konrad.hinsen＠laposte.net Nov. 28, 2006

Nov. 28, 2006

I am looking for a way to create an array of the same type as another given array (but of different shape) that works both with Numeric and NumPy without being unreasonably slow. In other words, I am looking for a a replacement for the expression array2 = Numeric.zeros(shape, array1.typecode()) that will *also* (not *only*) work under NumPy (where typecode() became dtype.char). An obvious idea is reshaping array1 and multiplying by 0., but that can become quite costly. Any ideas? Konrad. -- --------------------------------------------------------------------- Konrad Hinsen Centre de Biophysique Moléculaire, CNRS Orléans Synchrotron Soleil - Division Expériences Saint Aubin - BP 48 91192 Gif sur Yvette Cedex, France Tel. +33-1 69 35 97 15 E-Mail: hinsen(a)cnrs-orleans.fr ---------------------------------------------------------------------

3 4

Assumed 64bit-Incompatiblity in numpy svn test_regression
by Jan-Matthias Braun Nov. 27, 2006

Nov. 27, 2006

Hi all, after building current numpy from svn (1.0.1.dev3460 as of writing :), the call for numpy.test(10, 10) fails with following output added at the end of the mail. The Test seems seems to be made for Ticket #372, but the problem is a type mismatch between 32 and 64bit Integers as far as I can see. Should I file a ticket for this one? Cheers, Jan ====================================================================== FAIL: Ticket #372 ---------------------------------------------------------------------- Traceback (most recent call last): File "~/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 581, in check_recarray_fields assert_equal(a.dtype,dt) File "~/lib/python2.4/site-packages/numpy/testing/utils.py", line 143, in assert_equal assert desired == actual, msg AssertionError: Items are not equal: ACTUAL: dtype([('f0', '<i8'), ('f1', '<i8')]) DESIRED: dtype([('f0', '<i4'), ('f1', '<i4')]) ---------------------------------------------------------------------- Ran 524 tests in 0.294s

3 3

apply_along, apply_over w/ subarrays
by Pierre GM Nov. 26, 2006

Nov. 26, 2006

Folks, Is there any reason why `apply_along_axis`, `apply_over_axes` (and I expect `vectorize` as well, but I haven't tried) won't accept subclasses of ndarrays ? Would it be possible to use `asanyarray` instead of `asarray` in those functions ? Oh, and I ran into some problems with apply_along-axis: in the uncommon case where the output of the 1d functions is a scalar array (eg., N.array(0)) (or a masked_singleton), the current version raises a TypeError, as the output is not a scalar, but doesn't have a length either. I tried to correct that, but I'd prefer to get some feedback before submitting the patch. Thanks for all. P. ################################################# --- numpy/lib/shape_base.py.init 2006-11-26 13:22:57.000000000 -0500 +++ numpy/lib/shape_base.py 2006-11-26 17:01:22.000000000 -0500 @@ -13,7 +13,7 @@ def apply_along_axis(func1d,axis,arr,*ar and arr is an N-d array. i varies so as to apply the function along the given axis for each 1-d subarray in arr. """ - arr = asarray(arr) + arr = asanyarray(arr) nd = arr.ndim if axis < 0: axis += nd @@ -28,9 +28,16 @@ def apply_along_axis(func1d,axis,arr,*ar outshape = asarray(arr.shape).take(indlist) i.put(indlist, ind) res = func1d(arr[tuple(i.tolist())],*args) - # if res is a number, then we have a smaller output array - if isscalar(res): - outarr = zeros(outshape,asarray(res).dtype) + # if res is a number, or doesn't have a length , then we have a smaller output array + asscalar = isscalar(res) + if not asscalar: + try: + len(res) + except TypeError: + asscalar = True + # + if asscalar: + outarr = zeros(outshape,asarray(res).dtype).view(res.__class__) outarr[ind] = res Ntot = product(outshape) k = 1 @@ -52,7 +59,7 @@ def apply_along_axis(func1d,axis,arr,*ar holdshape = outshape outshape = list(arr.shape) outshape[axis] = len(res) - outarr = zeros(outshape,asarray(res).dtype) + outarr = zeros(outshape,asarray(res).dtype).view(res.__class__) outarr[tuple(i.tolist())] = res k = 1 while k < Ntot: @@ -78,7 +85,7 @@ def apply_over_axes(func, a, axes): to be either the same shape as a or have one less dimension. This call is repeated for each axis in the axes sequence. """ - val = asarray(a) + val = asanyarray(a) N = a.ndim if array(axes).ndim == 0: axes = (axes,) #################################################

1 0