Mailman 3 October 2014 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell July 22, 2021

July 22, 2021

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

C-API: multidimensional array indexing?
by Johann Bauer March 31, 2016

March 31, 2016

Dear experts, is there a C-API function for numpy which implements Python's multidimensional indexing? Say, I have a 2d-array PyArrayObject * M; and an index int i; how do I extract the i-th row or column M[i,:] respectively M[:,i]? I am looking for a function which gives again a PyArrayObject * and which is a view to M (no copied data; the result should be another PyArrayObject whose data and strides points to the correct memory portion of M). I searched the API documentation, Google and mailing lists for quite a long time but didn't find anything. Can you help me? Thanks, Johann

3 3

Generalize hstack/vstack --> stack; Block matrices like in matlab
by Stefan Otte Jan. 9, 2016

Jan. 9, 2016

Hey, quite often I work with block matrices. Matlab offers the convenient notation [ a b; c d ] to stack matrices. The numpy equivalent is kinda clumsy: vstack([hstack([a,b]), hstack([c,d])]) I wrote the little function `stack` that does exactly that: stack([[a, b], [c, d]]) In my case `stack` replaced `hstack` and `vstack` almost completely. If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback! Best, Stefan [1] https://github.com/numpy/numpy/pull/5057

12 24

PR added: frozen dimensions in gufunc signatures
by Jaime Fernández del Río June 23, 2015

June 23, 2015

Hi, I have just sent a PR (https://github.com/numpy/numpy/pull/5015), adding the possibility of having frozen dimensions in gufunc signatures. As a proof of concept, I have added a `cross1d` gufunc to `numpy.core.umath_tests`: In [1]: import numpy as np In [2]: from numpy.core.umath_tests import cross1d In [3]: cross1d.signature Out[3]: '(3),(3)->(3)' In [4]: a = np.random.rand(1000, 3) In [5]: b = np.random.rand(1000, 3) In [6]: np.allclose(np.cross(a, b), cross1d(a, b)) Out[6]: True In [7]: %timeit np.cross(a, b) 10000 loops, best of 3: 76.1 us per loop In [8]: %timeit cross1d(a, b) 100000 loops, best of 3: 13.1 us per loop In [9]: c = np.random.rand(1000, 2) In [10]: d = np.random.rand(1000, 2) In [11]: cross1d(c, d) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-11-72c66212e40c> in <module>() ----> 1 cross1d(c, d) ValueError: cross1d: Operand 0 has a mismatch in its core dimension 0, with gufunc signature (3),(3)->(3) (size 2 is different from 3) The speed up over `np.cross` is nice, and while `np.cross` is not the best of examples, as it needs to handle more sizes, in many cases this will allow producing gufuncs that work without a Python wrapper redoing checks that are best left to the iterator, such as dimension sizes. It still needs tests, but before embarking on fully developing those, I wanted to make sure that there is an interest on this. I would also like to further enhance gufuncs providing computed dimensions, e.g. making it possible to e.g. define `pairwise_cross` with signature '(n, 3)->($m, 3)', where the $ indicates that m is a computed dimension, that would have to be calculated by a function passed to the gufunc constructor and stored in the gufunc object, based on the other core dimensions. In this case it would make $m be n*(n-1), so that all pairwise cross products between 3D vectors could be computed. The syntax with '$' is kind of crappy, so any suggestions on how to better express this in the signature are more than welcome, as well as any feedback on the merits (or lack of them) of implementing this. Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

4 4

Numpy 'None' comparison FutureWarning
by Demitri Muna Jan. 31, 2015

Jan. 31, 2015

Hi, I just encountered the following in my code: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future. I'm very concerned about this. This is a very common programming pattern (lazy loading): class A(object): def __init__(self): self._some_array = None @property def some_array(self): if self._some_array == None: # perform some expensive setup of array return self._some_array It seems to me that the new behavior will break this pattern. I think that redefining the "==" operator is a little too aggressive here. It strikes me as very nonstandard and not at all obvious to someone reading the code that the comparison is a very special case for numpy objects. Unless there's some aspect I'm missing here, I think an element-wise comparator should be more explicit. Cheers, Demitri _________________________________________ Demitri Muna Department of Astronomy Le Ohio State University http://trillianverse.org http://scicoder.org

5 6

New function `count_unique` to generate contingency tables.
by Warren Weckesser Jan. 25, 2015

Jan. 25, 2015

I created a pull request (https://github.com/numpy/numpy/pull/4958) that defines the function `count_unique`. `count_unique` generates a contingency table from a collection of sequences. For example, In [7]: x = [1, 1, 1, 1, 2, 2, 2, 2, 2] In [8]: y = [3, 4, 3, 3, 3, 4, 5, 5, 5] In [9]: (xvals, yvals), counts = count_unique(x, y) In [10]: xvals Out[10]: array([1, 2]) In [11]: yvals Out[11]: array([3, 4, 5]) In [12]: counts Out[12]: array([[3, 1, 0], [1, 1, 3]]) It can be interpreted as a multi-argument generalization of `np.unique(x, return_counts=True)`. It overlaps with Pandas' `crosstab`, but I think this is a pretty fundamental counting operation that fits in numpy. Matlab's `crosstab` (http://www.mathworks.com/help/stats/crosstab.html) and R's `table` perform the same calculation (with a few more bells and whistles). For comparison, here's Pandas' `crosstab` (same `x` and `y` as above): In [28]: import pandas as pd In [29]: xs = pd.Series(x) In [30]: ys = pd.Series(y) In [31]: pd.crosstab(xs, ys) Out[31]: col_0 3 4 5 row_0 1 3 1 0 2 1 1 3 And here is R's `table`: > x <- c(1,1,1,1,2,2,2,2,2) > y <- c(3,4,3,3,3,4,5,5,5) > table(x, y) y x 3 4 5 1 3 1 0 2 1 1 3 Is there any interest in adding this (or some variation of it) to numpy? Warren

5 11

FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)
by Nathaniel Smith Dec. 11, 2014

Dec. 11, 2014

On 28 Oct 2014 04:07, "Matthew Brett" <matthew.brett(a)gmail.com> wrote: > > Hi, > > On Mon, Oct 27, 2014 at 8:07 PM, Sturla Molden <sturla.molden(a)gmail.com> wrote: > > Sturla Molden <sturla.molden(a)gmail.com> wrote: > > > >> If we really need a > >> kick-ass fast FFT we need to go to libraries like FFTW, Intel MKL or > >> Apple's Accelerate Framework, > > > > I should perhaps also mention FFTS here, which claim to be faster than FFTW > > and has a BSD licence: > > > > http://anthonix.com/ffts/index.html > > Nice. And a funny New Zealand name too. > > Is this an option for us? Aren't we a little behind the performance > curve on FFT after we lost FFTW? It's definitely attractive. Some potential issues that might need dealing with, based on a quick skim: - seems to have a hard requirement for a processor supporting SSE, AVX, or NEON. No fallback for old CPUs or other architectures. (I'm not even sure whether it has x86-32 support.) - no runtime CPU detection, e.g. SSE vs AVX appears to be a compile time decision - not sure if it can handle non-power-of-two problems at all, or at all efficiently. (FFTPACK isn't great here either but major regressions would be bad.) - not sure if it supports all the modes we care about (e.g. rfft) This stuff is all probably solveable though, so if someone has a hankering to make numpy (or scipy) fft dramatically faster then you should get in touch with the author and see what they think. -n

17 37

taking a 2D uneven surface slice
by Bryan Woods Dec. 4, 2014

Dec. 4, 2014

I'm trying to do something that at first glance I think should be simple but I can't quite figure out how to do it. The problem is as follows: I have a 3D grid Values[Nx, Ny, Nz] I want to slice Values at a 2D surface in the Z dimension specified by Z_index[Nx, Ny] and return a 2D slice[Nx, Ny]. It is not as simple as Values[:,:,Z_index]. I tried this: >>> values.shape (4, 5, 6) >>> coords.shape (4, 5) >>> slice = values[:,:,coords] >>> slice.shape (4, 5, 4, 5) >>> slice = np.take(values, coords, axis=2) >>> slice.shape (4, 5, 4, 5) >>> Obviously I could create an empty 2D slice and then fill it by using np.ndenumerate to fill it point by point by selecting values[i, j, Z_index[i, j]]. This just seems too inefficient and not very pythonic.

3 3

ANN: Bokeh 0.6.1 release
by Damian Avila Dec. 1, 2014

Dec. 1, 2014

On behalf of the Bokeh team, I am very happy to announce the release of Bokeh version 0.6.1! Bokeh is a Python library for visualizing large and realtime datasets on the web. Its goal is to provide to developers (and domain experts) with capabilities to easily create novel and powerful visualizations that extract insight from local or remote (possibly large) data sets, and to easily publish those visualization to the web for others to explore and interact with. This point release includes several bug fixes and improvements over our most recent 0.6.0 release: * Toolbar enhancements * bokeh-server fixes * Improved documentation * Button widgets * Google map support in the Python side * Code cleanup in the JS side and examples * New examples See the CHANGELOG for full details. In upcoming releases, you should expect to see more new layout capabilities (colorbar axes, better grid plots and improved annotations), additional tools, even more widgets and more charts, R language bindings, Blaze integration and cloud hosting for Bokeh apps. Don't forget to check out the full documentation, interactive gallery, and tutorial at http://bokeh.pydata.org as well as the Bokeh IPython notebook nbviewer index (including all the tutorials) at: http://nbviewer.ipython.org/github/ContinuumIO/bokeh-notebooks/blob/master/… If you are using Anaconda or miniconda, you can install with conda: conda install bokeh Alternatively, you can install with pip: pip install bokeh BokehJS is also available by CDN for use in standalone javascript applications: http://cdn.pydata.org/bokeh-0.6.1.min.js http://cdn.pydata.org/bokeh-0.6.1.min.css Issues, enhancement requests, and pull requests can be made on the Bokeh Github page: https://github.com/continuumio/bokeh Questions can be directed to the Bokeh mailing list: bokeh(a)continuum.io If you have interest in helping to develop Bokeh, please get involved! Cheers, Damián Avila damian.avila(a)continuum.io

6 7

Add `nrows` to `genfromtxt`
by Jaime Fernández del Río Nov. 3, 2014

Nov. 3, 2014

There is a PR in github that adds a new keyword to the genfromtxt function, to limit the number of rows that actually get read in: https://github.com/numpy/numpy/pull/5103 It is mostly ready to go, and several devs have looked at it without complaining. Since it is an API change, I wanted to check here: if no one has any strong opposition, I will be merging it sometime tomorrow. You have been warned... ;-) Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

5 16

Choosing between NumPy and SciPy functions
by dmmcf＠dmmcf.net Oct. 31, 2014

Oct. 31, 2014

A recent post raised a question about differences in results obtained with numpy.linalg.eigh() and scipy.linalg.eigh(), documented at http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eigh.html#… and http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigh.html#…, respectively. It is clear that these functions address different mathematical problems (among other things, the SciPy routine can solve the generalized as well as standard eigenproblems); I am not concerned here with numerical differences in the results for problems both should be able to solve (the author of the original post received useful replies in that thread). What I would like to ask about is the situation this illustrates, where both NumPy and SciPy provide similar functionality (sometimes identical, to judge by the documentation). Is there some guidance on which is to be preferred? I could argue that using only NumPy when possible avoids unnecessary dependence on SciPy in some code, or that using SciPy consistently makes for a single interface and so is less error prone. Is there a rule of thumb for cases where SciPy names shadow NumPy names? I've used Python for a long time, but have only recently returned to doing serious numerical work with it. The tools are very much improved, but sometimes, like now, I feel I'm missing the obvious. I would appreciate pointers to any relevant documentation, or just a summary of conventional wisdom on the topic. Regards, Michael

10 17

[ANN - JOB] Assistant Researcher - Berkeley Institute for Data Science
by Fernando Perez Oct. 30, 2014

Oct. 30, 2014

Hi all, the newly founded Berkeley Institute for Data Science is hiring researchers with a focus on open source tools for scientific computing, please see here for details: https://aprecruit.berkeley.edu/apply/JPF00590 Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail

1 0

help using np.einsum for stacked matrix multiplication
by Andrew Nelson Oct. 29, 2014

Oct. 29, 2014

Dear list, I have a 4D array, A, that has the shape (NX, NY, 2, 2). I wish to perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the matrix B. B would have shape (NX, 2, 2). I believe that np.einsum would be up to the task, but I'm not quite sure of the subscripts I would need to achieve this. Can anyone help, please? cheers, Andrew. -- _____________________________________ Dr. Andrew Nelson _____________________________________

5 5

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication
by Andrew Nelson Oct. 29, 2014

Oct. 29, 2014

On Wed, Oct 29, 2014 at 10:39 AM, Andrew Nelson <andyfaff(a)gmail.com> wrote: > Dear list, > I have a 4D array, A, that has the shape (NX, NY, 2, 2). I wish to > perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the > matrix B. B would have shape (NX, 2, 2). I believe that np.einsum would > be up to the task, but I'm not quite sure of the subscripts I would need to > achieve this. Ok, I'll try to explain in more detail of what I'm trying to do (I'm not skilled in matrix algebra). Say I have a series of matrices, M, which are all 2x2: M_0, M_1, ..., M_{NY - 1}. These all need to be multiplied by each other. i.e. N = M_0 x M_1 x ... x M_{NY - 1}. Note that I want to multiply M_0 by M_1, the result of that by M_2, the result of that by M_3 and so on. I can hold the (NY) matrices in a single array that has shape (NY, 2, 2). The first row in that array would be M_0, the last would be M_{NY-1}. The output of all that matrix multiplication would be a single 2x2 matrix. So I would've thought an operation would do something like this: #there are NY-1 matrix multiplications involved here. M[NY, 2, 2] -----> N[2, 2] Now let's make the next level of complication, I have NX of those M[NY, 2, 2] matrices. So I need to do the above matrix multiplication series NX times. I could hold all this in an array, P, with shape (NX, NY, 2, 2). Each of the NX rows are independent. Currently I am doing this in a nested loop, pseudocode follows: <python> output = np.zeros((NX, 2, 2)) for i in range(NX): temp = np.identity(2) for j in range(NY): temp = np.dot(temp, P[i, j]) output[i] = temp </python> My original question was posted as I would like to remove that doubly nested loop, with something more elegant, as well as a whole load faster. Is there an np.einsum that can furnish that? (Please forgive me if this still isn't clear or precise enough). -- _____________________________________ Dr. Andrew Nelson _____________________________________

1 0

Deprecate pkgload, PackageLoader
by Charles R Harris Oct. 29, 2014

Oct. 29, 2014

Hi All, It is proposed to deprecate, then remove, pkgload and PackageLoader. Complaints? Cries of Anguish? Chuck

4 4

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?
by Alexander Belopolsky Oct. 29, 2014

Oct. 29, 2014

On Mon, Oct 27, 2014 at 9:41 PM, Yuxiang Wang <yw5aj(a)virginia.edu> wrote: > In my opinion - because they don't do the same thing, especially when > you think in terms in lower-level. > > ndarray.flat returns an iterator; ndarray.flatten() returns a copy; > ndarray.ravel() only makes copies when necessary; ndarray.reshape() is > more general purpose, even though you can use it to flatten arrays. > Out of the four ways, I find x.flat the most confusing. Unfortunately, it is also the most obvious name for the operation (and "ravel" is the least, but it is the fault of the English language where "to ravel" means "to unravel."). What x.flat returns, is not really an iterator. It is some hybrid between a view and an iterator. Consider this: >>> x = numpy.arange(6).reshape((2,3)) >>> i = x.flat >>> i.next() 0 >>> i.next() 1 >>> i.next() 2 So far no surprises, but what should i[0] return now? If you think of i as a C pointer you would expect 3, but >>> i[0] 0 What is worse, the above resets the index and now >>> i.index 0 OK, so now I expect that i[5] will reset the index to 5, but no >>> i[5] 5 >>> i.index 0 When would you prefer to use x.flat over x.ravel()? Is x.reshape(-1) always equivalent to x.ravel()? What is x.flat.copy()? Is it the same as x.flatten()? Why does flatiter even have a .copy() method? Isn't i.copy() the same as i.base.flatten(), only slower? And with all these methods, I still don't have the one that would flatten any array including a nested array like this: >>> x = np.array([np.arange(2), np.arange(3), np.arange(4)]) I need yet another function here, for example >>> np.hstack(x) array([0, 1, 0, 1, 2, 0, 1, 2, 3]) and what if I want to flatten a higher dimensional nested array, say >>> y = np.array([x[:1],x[:2],x]) can I do better than >>> np.hstack(np.hstack(y)) array([0, 1, 0, 1, 0, 1, 2, 0, 1, 0, 1, 2, 0, 1, 2, 3]) ?

6 14

Memory efficient alternative for np.loadtxt and np.genfromtxt
by Saullo Castro Oct. 28, 2014

Oct. 28, 2014

I would like to start working on a memory efficient alternative for np.loadtxt and np.genfromtxt that uses arrays instead of lists to store the data while the file iterator is exhausted. The motivation came from this SO question: http://stackoverflow.com/q/26569852/832621 where for huge arrays the current NumPy ASCII readers are really slow and require ~6 times more memory. This case I tested with Pandas' read_csv() and it required 2 times more memory. I would be glad if you could share your experience on this matter. Greetings, Saullo

11 13

[ANN] Python for Scientific Computing conference in Boulder, CO; April'15
by Fernando Perez Oct. 28, 2014

Oct. 28, 2014

Hi folks, a colleague from NCAR in Boulder just sent me this link about a conference they are organizing in the spring: https://sea.ucar.edu/conference/2015 I figured this might be of interest to many on these lists. The actual call isn't up yet, so if you're interested, watch that site for an upcoming call when they post it (I'm not directly involved, just passing the message along). Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail

2 2

multi-dimensional c++ proposal
by Neal Becker Oct. 28, 2014

Oct. 28, 2014

The multi-dimensional c++ stuff is interesting (about time!) http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2014/n3851.pdf -- -- Those who don't understand recursion are doomed to repeat it

2 3

Fwd: numpy.i and std::complex
by Glen Mabey Oct. 28, 2014

Oct. 28, 2014

Hello, I was very excited to learn about numpy.i for easy numpy+swigification of C code -- it's really handy. Knowing that swig wraps C code, I wasn't too surprised that there was the issue with complex data types (as described at http://docs.scipy.org/doc/numpy/reference/swig.interface-file.html#other-co…), but still it was pretty disappointing because most of my data is complex, and I'm invoking methods written to use C++'s std::complex class. After quite a bit of puzzling and not much help from previous mailing list posts, I created this very brief but very useful file, which I call numpy_std_complex.i -- /* -*- C -*- (not really, but good for syntax highlighting) */ #ifdef SWIGPYTHON %include "numpy.i" %include <std_complex.i> %numpy_typemaps(std::complex<float>, NPY_CFLOAT , int) %numpy_typemaps(std::complex<double>, NPY_CDOUBLE, int) #endif /* SWIGPYTHON */ I'd really like for this to be included alongside numpy.i -- but maybe I overestimate the number of numpy users who use complex data (let your voice be heard!) and who also end up using std::complex in C++ land. Or if anyone wants to improve upon this usage I would be very happy to hear about what I'm missing. I'm sure there's a documented way to submit this file to the git repo, but let me simultaneously ask whether list subscribers think this is worthwhile and ask someone to add+push it for me … Thanks, Glen Mabey

4 11

Why ndarray provides four ways to flatten?
by Alexander Belopolsky Oct. 28, 2014

Oct. 28, 2014

Given an n-dim array x, I can do 1. x.flat 2. x.flatten() 3. x.ravel() 4. x.reshape(-1) Each of these expressions returns a flat version of x with some variations. Why does NumPy implement four different ways to do essentially the same thing?

2 1

Re: [Numpy-discussion] numpy.i and std::complex
by MinRK Oct. 27, 2014

Oct. 27, 2014

> > It's because that list is old and has not been updated. It has the > predecessor to Cython, Pyrex, but they are very different now. > > Both SciPy and NumPy has Cython as a build dependency, and also projects > like scikit-learn, scikit-image, statsmodels. > > If you find C++ projects which use Swig (wxPython, PyWin32) or SIP (PyQt) > it is mainly because they are older than Cython. A more recent addition, > PyZMQ, use Cython to wrap C++. > > Just to clarify that, while PyZMQ wraps a library written in C++, libzmq's public API is C, not C++. PyZMQ does not use any of Cython's C++ functionality. I have done other similar (unfortunately not public) projects that wrap actual C++ libraries with Cython, and I have been happy with Cython's C++ support[1]. -MinRK [1] At least as happy as I was with the wrapped C++ code, anyway.

1 0

ODE how to?
by Vincent Davis Oct. 27, 2014

Oct. 27, 2014

It's been too long since I have done differential equations and I am not sure the best tools to solve this problem. I am starting with a basic kinematic equation for the balance of forces. P\v - ((A*Cw*Rho*v^2)/2 + m*g*Crl + m*g*slope) = m*a P: power x: position v: velocity, x' a: acceleration x" (A*Cw*Rho*v^2)/2 : air resistance m*g*Crl : rolling resistance m*g*slope : potential energy (elevation) I am modifying the above equation so that air velocity and slope are dependant on location x. Vair = v + f(x) where f(x) is the weather component and a function of location x. Same goes for slope, slope = g(x) Power is a function I what to optimize/find to minimize time but at this time just simulate. maybe something like: P = 2500/(v+1) I will have restriction on P but not interested in that now. The "course" I what to simulate therefore defines slope and wind speed. and is of a fixed distance. I have played with some of the simple scipy.integrate.odeint examples. I get that I need to define a system of equations but am not really sure the rules for doing so. A little help would be greatly appreciated. Vincent Davis 720-301-3003

1 0