Mailman 3 May 2013 - NumPy-Discussion

np.dot and 'out' bug
by Nicolas Rougier 23 May '13

23 May '13

Hi, From the dot documentation, I tried something simple: a = np.array([[1, 2], [3, 4]]) b = np.array([[1, 2], [3, 4]]) np.dot(a, b) -> array([[ 7, 10], [15, 22]]) And I got expected result but if I use either a or b as output, results are wrong (and nothing in the dot documentation prevents me from doing this): a = np.array([[1, 2], [3, 4]]) b = np.array([[1, 2], [3, 4]]) np.dot(a,b,out=a) -> array([[ 6, 20], [15, 46]]) a = np.array([[1, 2], [3, 4]]) b = np.array([[1, 2], [3, 4]]) np.dot(a,b,out=b) -> array([[ 6, 10], [30, 46]]) Can anyone confirm this behavior ? (tested using numpy 1.7.1) Nicolas

7 10

pyhdf packaging
by Andreas Hilboll 23 May '13

23 May '13

Hi, I would like to package pyhdf for Ubuntu and make the package publicly available. Since the license is not totally clear to me (I cannot find any information in the sources, and the cheeseshop says "public", which doesn't mean anything to me), I tried to contact the maintainer, Andre Gosselin, however the email bounces, so I guess he's gone. Can anyone point me to how to proceed from here? Cheers, Andreas.

2 2

Possible conversion bug with record array
by Nicolas Rougier 22 May '13

22 May '13

Hi all, I got a weird output from the following script: import numpy as np U = np.zeros(1, dtype=[('x', np.float32, (4,4))]) U[0] = np.eye(4) print U[0] # output: ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.875], [0.0, 0.0, 0.0, 0.0]],) U[0] = np.eye(4, dtype=np.float32) print U[0] # output: ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]],) The first output is obviously wrong. Can anyone confirm ? (using numpy 1.7.1 on osx 10.8.3) Nicolas

3 4

another indexing question
by Neal Becker 20 May '13

20 May '13

I have a system that transmits signals for an alphabet of M symbols over and additive Gaussian noise channel. The receiver has a 1-d array of complex received values. I'd like to find the means of the received values according to the symbol that was transmitted. So transmit symbol indexes might be: x = [0, 1, 2, 1, 3, ...] and receive output might be: y = [(1+1j), (1-1j), ...] Suppose the alphabet was M=4. Then I'd like to get an array of means m[0...3] that correspond to the values of y for each of the corresponding values of x. I can't think of a better way than manually using loops. Any tricks here?

3 2

Equvalent function for Ceil() and Floor()
by Bakhtiyor Zokhidov 20 May '13

20 May '13

Hello, I am using ceil() and floor() function to get upper and lower value of some numbers. Let's say: import math x1 = 0.35 y1 = 4.46 >>> math.ceil(x1) 1.0 >>> math.floor(y1) 4.0 The problem is that If I want to get upper and lower values for the certain step, for example, step = 0.25, ceil() function should give: new_ceil(x1, step) => 0.5 new_floor(y1, step) => 4.25 Because, the step is 0.25 Question: How I can I achieve those results by using ceil() and floor() function, or Is there any equvalent function for that? -- Bakhti

4 7

Newbie trying to install NumPy
by Joe Piccoli 19 May '13

19 May '13

Hello, I've been trying to install NumPy to run with Eclipse on Windows Vista. After installing (I thought) NumPy I was seeing: ImportError: Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python intepreter from there. I next tried to follow the instructions from the scipy.org website and downloaded and ran: numpy-1.7.1-win32-superpack-python27.exe This started up but I immediately saw the following dialog: --------------------------- Cannot install --------------------------- Python version 2.7 required, which was not found in the registry. --------------------------- OK --------------------------- The next dialog prompted for a Python installation to use but the list box was empty and it would not allow me to enter a path. Is it absolutely necessary to build NumPy myself or is there a working installation out there? I know I'm doing something wrong but I don't know what it is. Any assistance would be greatly appreciated :). Thanks, Joseph A. Piccoli joe13676(a)comcast.net

5 4

ANN: python-blosc 1.1 RC1 available for testing
by Francesc Alted 17 May '13

17 May '13

================================ Announcing python-blosc 1.1 RC1 ================================ What is it? =========== python-blosc (http://blosc.pydata.org) is a Python wrapper for the Blosc compression library. Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Whether this is achieved or not depends of the data compressibility, the number of cores in the system, and other factors. See a series of benchmarks conducted for many different systems: http://blosc.org/trac/wiki/SyntheticBenchmarks. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. There is also a handy command line for Blosc called Bloscpack (https://github.com/esc/bloscpack) that allows you to compress large binary datafiles on-disk. Although the format for Bloscpack has not stabilized yet, it allows you to effectively use Blosc from your favorite shell. What is new? ============ - Added new `compress_ptr` and `decompress_ptr` functions that allows to compress and decompress from/to a data pointer. These are low level calls and user must make sure that the pointer data area is safe. - Since Blosc (the C library) already supports to be installed as an standalone library (via cmake), it is also possible to link python-blosc against a system Blosc library. - The Python calls to Blosc are now thread-safe (another consequence of recent Blosc library supporting this at C level). - Many checks on types and ranges of values have been added. Most of the calls will now complain when passed the wrong values. - Docstrings are much improved. Also, Sphinx-based docs are available now. Many thanks to Valentin Hänel for his impressive work for this release. For more info, you can see the release notes in: https://github.com/FrancescAlted/python-blosc/wiki/Release-notes More docs and examples are available in the documentation site: http://blosc.pydata.org Installing ========== python-blosc is in PyPI repository, so installing it is easy: $ pip install -U blosc # yes, you should omit the blosc- prefix Download sources ================ The sources are managed through github services at: http://github.com/FrancescAlted/python-blosc Documentation ============= There is Sphinx-based documentation site at: http://blosc.pydata.org/ Mailing list ============ There is an official mailing list for Blosc at: blosc(a)googlegroups.com http://groups.google.es/group/blosc Licenses ======== Both Blosc and its Python wrapper are distributed using the MIT license. See: https://github.com/FrancescAlted/python-blosc/blob/master/LICENSES for more details. -- Francesc Alted

1 0

[ANN] Multidimensional Array - MDArray (0.5.0)
by Rodrigo Botafogo 17 May '13

17 May '13

Although this is not directly connected to NumPy, I believe that it could be of interest to the NymPy community. If, by any reason it is inproper to post this type of announcement on this list, please let me know. I´m happy to announce a new version of MDArray... MDArray ======= MDArray is a multi dimensional array implemented for JRuby inspired by NumPy (www.numpy.org) and Narray (narray.rubyforge.org) by Masahiro Tanaka. MDArray stands on the shoulders of Java-NetCDF and Parallel Colt. NetCDF-Java Library is a Java interface to NetCDF files, as well as to many other types of scientific data formats. It is developed and distributed by Unidata ( http://www.unidata.ucar.edu). Parallel Colt (sites.google.com/site/piotrwendykier/software/parallelcolt) is a multithreaded version of Colt (http://acs.lbl.gov/software/colt/). Colt provides a set of Open Source Libraries for High Performance Scientific and Technical Computing in Java. Scientific and technical computing is characterized by demanding problem sizes and a need for high performance at reasonably small memory footprint. MDArray and SciRuby =================== MDArray subscribes fully to the SciRuby Manifesto (http://sciruby.com/). "Ruby has for some time had no equivalent to the beautifully constructed NumPy, SciPy, and matplotlib libraries for Python. We believe that the time for a Ruby science and visualization package has come. Sometimes when a solution of sugar and water becomes super-saturated, from it precipitates a pure, delicious, and diabetes-inducing crystal of sweetness, induced by no more than the tap of a finger. So is occurring now, we believe, with numeric and visualization libraries for Ruby." Main properties =============== + Homogeneous multidimensional array, a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers; + Easy calculation for large numerical multi dimensional arrays; + Basic types are: boolean, byte, short, int, long, float, double, string, structure; + Based on JRuby, which allows importing Java libraries; + Operator: +,-,*,/,%,**, >, >=, etc. + Functions: abs, ceil, floor, truncate, is_zero, square, cube, fourth; + Binary Operators: &, |, ^, ~ (binary_ones_complement), <<, >>; + Ruby Math functions: acos, acosh, asin, asinh, atan, atan2, atanh, cbrt, cos, erf, exp, gamma, hypot, ldexp, log, log10, log2, sin, sinh, sqrt, tan, tanh, neg; + Boolean operations on boolean arrays: and, or, not; + Fast descriptive statistics from Parallel Colt (complete list found bellow); + Easy manipulation of arrays: reshape, reduce dimension, permute, section, slice, etc. + Reading of two dimensional arrays from CSV files (mainly for debugging and simple testing purposes); + StatList: a list that can grow/shrink and that can compute Parallel Colt descriptive statistics. Descriptive statistics methods ============================== auto_correlation, correlation, covariance, durbin_watson, frequencies, geometric_mean, harmonic_mean, kurtosis, lag1, max, mean, mean_deviation, median, min, moment, moment3, moment4, pooled_mean, pooled_variance, product, quantile, quantile_inverse, rank_interpolated, rms, sample_covariance, sample_kurtosis, sample_kurtosis_standard_error, sample_skew, sample_skew_standard_error, sample_standard_deviation, sample_variance, sample_weighted_variance, skew, split, standard_deviation, standard_error, sum, sum_of_inversions, sum_of_logarithms, sum_of_powers, sum_of_power_deviations, sum_of_squares, sum_of_squared_deviations, trimmed_mean, variance, weighted_mean, weighted_rms, weighted_sums, winsorized_mean. Installation and download ========================= + Install Jruby + jruby -S gem install mdarray Contributors ============ + Contributors are welcome. Homepages ========= + http://rubygems.org/gems/mdarray + https://github.com/rbotafogo/mdarray/wiki HISTORY ======= + 16/05/2013: Version 0.5.0: All loops transfered to Java with over 50% performance improvement. Descriptive statistics from Parallel Colt. + 19/04/2013: Version 0.4.3: Fixes a simple (but fatal bug). No new features + 17/04/2013: Version 0.4.2: Adds simple statistics and boolean operators + 05/05/2013: Version 0.4.0: Initial release -- Rodrigo Botafogo

1 0

RuntimeWarning: divide by zero encountered in log
by Neal Becker 17 May '13

17 May '13

Is there a way to get a traceback instead of just printing the line that triggered the error?

3 5

experiments with SSE vectorization
by Julian Taylor 16 May '13

16 May '13

Hi, I have been experimenting a bit with how applicable SSE vectorization is to NumPy. In principle the core of NumPy mostly deals with memory bound operations, but it turns out on modern machines with large caches you can still get decent speed ups. The experiments are available on this fork: https://github.com/juliantaylor/numpy/tree/simd-experiments It includes a simple benchmark 'npbench.py' in the top level. No runtime detection is used, it is only enabled on amd64 systems(which always has SSE2). The simd-experiments branch vectorizes the sqrt, basic math operations and min/max reductions. For float32 operations you get speedups around 2 (simple ops) - 4 (sqrt). For double it is around 1.2 - 2, depending on the cpu. My Phenom(tm) II X4 955 retains a good speedup even for very large datasizes but on intel cpus (xeon and core2duo) you don't gain anything if the data is larger than the L3 cache. The vectorized version was never slower on phenom and xeon. But on a core2duo the normal addition with very large datasets got 10% slower. This can be compensated by using aligned load operations, but its not implemented yet. I'm interested in your results of npbench.py command on other cpus, so if you want to try it please send me the output (include /proc/cpuinfo) The code is a little rough, it can probably be cleaned up a bit by adapting the code generator used. Would this be something worth including in NumPy? Further vectorization targets on my todo list are things like std/var/mean, basically anything that has a high computation/memory ration, suggestions are welcome. Here the detailed results for my phenom: float32 datasize (2MB) operation: speedup np.float32 np.max(d) 3.04 np.float32 np.min(d) 3.1 np.float32 np.sum(d) 3.02 np.float32 np.prod(d) 3.04 np.float32 np.add(1, d) 1.44 np.float32 np.add(d, 1) 1.45 np.float32 np.divide(1, d) 3.41 np.float32 np.divide(d, 1) 3.41 np.float32 np.divide(d, d) 3.42 np.float32 np.add(d, d) 1.42 np.float32 np.multiply(d, d) 1.43 np.float32 np.sqrt(d) 4.26 float64 datasize (4MB) operation: speedup np.float64 np.max(d) 2 np.float64 np.min(d) 1.89 np.float64 np.sum(d) 1.62 np.float64 np.prod(d) 1.63 np.float64 np.add(1, d) 1.08 np.float64 np.add(d, 1) 0.993 np.float64 np.divide(1, d) 1.83 np.float64 np.divide(d, 1) 1.74 np.float64 np.divide(d, d) 1.8 np.float64 np.add(d, d) 1.02 np.float64 np.multiply(d, d) 1.05 np.float64 np.sqrt(d) 2.22 attached the results for intel cpus.

3 2

numpy.nanmin, numpy.nanmax, and scipy.stats.nanmean
by Phillip Feldman 16 May '13

16 May '13

It seems odd that `nanmin` and `nanmax` are in NumPy, while `nanmean` is in SciPy.stats. I'd like to propose that a `nanmean` function be added to NumPy. Phillip

2 1

Strange memory consumption in numpy?
by Martin Raspaud 16 May '13

16 May '13

Hi all, In the context of memory profiling an application (with memory_profiler module) we came up a strange behaviour in numpy, see for yourselves: Line # Mem usage Increment Line Contents ================================================ 29 @profile 30 23.832 MB 0.000 MB def main(): 31 46.730 MB 22.898 MB arr1 = np.random.rand(1000000, 3) 32 58.180 MB 11.449 MB arr1s = arr1.astype(np.float32) 33 35.289 MB -22.891 MB del arr1 34 35.289 MB 0.000 MB gc.collect() 35 58.059 MB 22.770 MB arr2 = np.random.rand(1000000, 3) 36 69.500 MB 11.441 MB arr2s = arr2.astype(np.float32) 37 69.500 MB 0.000 MB del arr2 38 69.500 MB 0.000 MB gc.collect() 39 69.500 MB 0.000 MB arr3 = np.random.rand(1000000, 3) 40 80.945 MB 11.445 MB arr3s = arr3.astype(np.float32) 41 80.945 MB 0.000 MB del arr3 42 80.945 MB 0.000 MB gc.collect() 43 80.945 MB 0.000 MB return arr1s, arr2s, arr3s The lines 31-34 are behaving as expected, but then we don't understand 35-38 (why is arr2 not garbage collected ?) and 39-42 (why doesn't the random allocate any memory ?). Can anyone give a reasonable explanation ? I attach the full script for reference. Best regards, Martin

2 3

__array_priority__ don't work for gt, lt, ... operator
by Frédéric Bastien 15 May '13

15 May '13

Hi, it popped again on the Theano mailing list that this don't work: np.arange(10) <= a_theano_vector. The reason is that __array_priority__ isn't respected for that class of operation. This page explain the problem and give a work around: http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-… The work around is to make a python function that will decide witch version of the comparator to call and do the call. Then we tell NumPy to use that function instead of its current function with: np.set_numeric_ops(...) But if we do that, when we import theano, we will slow down all normal numpy comparison for the user, as when <= is execute, first there will be numpy c code executed, that will call the python function to decide witch version to do, then if it is 2 numpy ndarray, it will call again numpy c code. That isn't a good solution. We could do the same override in C, but then theano work the same when there isn't a c++ compiler. That isn't nice. What do you think of changing them to check for __array_priority__ before doing the comparison? Frédéric

4 6

Integer overflow in test_einsum (1.7.1)
by Toder, Evgeny 14 May '13

14 May '13

Hello, One of the test cases in test_einsum causes integer overflow for i2 type. The test goes like this: >>> import numpy as np >>> dtype = 'i2' >>> n = 15 >>> a = np.arange(4*n, dtype=dtype).reshape(4,n) >>> b = np.arange(n*6, dtype=dtype).reshape(n,6) >>> c = np.arange(24, dtype=dtype).reshape(4,6) It then calculates AxB using einsum. The problem is that the values in the last row of the result do not fit into i2: >>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe') array([[ 6090., 6195., 6300., 6405., 6510., 6615.], [ 15540., 15870., 16200., 16530., 16860., 17190.], [ 24990., 25545., 26100., 26655., 27210., 27765.], [ 34440., 35220., 36000., 36780., 37560., 38340.]]) In my build this produces different results depending on whether out or .astype is used: >>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe').astype(dtype) array([[ 6090, 6195, 6300, 6405, 6510, 6615], [ 15540, 15870, 16200, 16530, 16860, 17190], [ 24990, 25545, 26100, 26655, 27210, 27765], [-31096, -30316, -29536, -28756, -27976, -27196]], dtype=int16) >>> np.einsum("ij,jk", a, b, out=c, dtype='f8', casting='unsafe') array([[ 6090, 6195, 6300, 6405, 6510, 6615], [ 15540, 15870, 16200, 16530, 16860, 17190], [ 24990, 25545, 26100, 26655, 27210, 27765], [-32768, -32768, -32768, -32768, -32768, -32768]], dtype=int16) The test wants these (actually the same using numpy.dot) to be equal, so this difference causes it to fail. Both ways to handle overflow seem reasonable to me. Does numpy in general assign a defined behavior to integer overflow (e.g. two's complement)? Is this use of integer overflow in the test intentional and is expected to work, or is my build broken? Best regards, Eugene This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email.

3 4

slight MapIter change
by Sebastian Berg 13 May '13

13 May '13

Hey, (this is only interesting if you know what MapIter and actually use it) In case anyone already uses the newly exposed mapiter (it was never released yet). There is a tiny change, which only affects indexes that start with np.newaxis but otherwise just simplifies a tiny bit. The old block for swapping axes should be changed like this: if ((mit->subspace != NULL) && (mit->consec)) { - if (mit->iteraxes[0] > 0) { - PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr, 0); - if (arr == NULL) { - return -1; - } + PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr, 0); + if (arr == NULL) { + return -1; } } Regards, Sebastian

2 3

somewhat less stupid problem with 0-d arrays
by Neal Becker 11 May '13

11 May '13

It would be convenient if in arithmetic 0-d arrays were just ignored - it would seem to me to be convenient in generic code where a degenerate array is treated as "nothing" np.zeros ((0,0)) + np.ones ((2,2)) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-17-27af0e0bbc6f> in <module>() ----> 1 np.zeros ((0,0)) + np.ones ((2,2)) ValueError: operands could not be broadcast together with shapes (0,0) (2,2)

3 4

printing array in tabular form
by Sudheer Joseph 11 May '13

11 May '13

Dear experts, I need to print few arrays in a tabular form for example below array IL has 25 elements, is there an easy way to print this as 5x5 comma separated table? in python IL=[] for i in np.arange(1,bno+1): IL.append(i) print(IL) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% in fortran I could do it as below %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% integer matrix(5,5) in=0 do, k=1,5 do, l=1,5 in=in+1 matrix(k,l)=in enddo enddo m=5 n=5 do, i=1,m write(*,"(5i5)") ( matrix(i,j), j=1,n ) enddo end *************************************************************** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.India@gmail.com;sudheer.joseph@yahoo.com Web- http://oppamthadathil.tripod.com ***************************************************************

7 19

0-dim arrays inconsistency
by Neal Becker 10 May '13

10 May '13

np.array ((0,0)) Out[10]: array([0, 0]) <<< ok, it's 2 dimensional In [11]: np.array ((0,0)).shape Out[11]: (2,) <<< except, it isn't

2 2

tests not running
by KACVINSKY Tom 10 May '13

10 May '13

Here is my set up: Mac OS 10.7.5 Xcode 4.5.1 Intel Fortran 12.1 Python 2.7.3 built from source Numpy 1.6.2 built from source, using MKL 11.0 nose 0.11.4 installed I run the numpy tests as documented (python -c 'import numpy; numpy.test()'), but get this output: tkacvins@macomsim> python -c 'import numpy; numpy.test()' Running unit tests for numpy NumPy version 1.6.2 NumPy is installed in /rd/gen/tky/do_not_delete/Python/macpython27/lib/python2.7/site-packages/numpy Python version 2.7.3 (default, Oct 10 2012, 14:47:52) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)] nose version 0.11.4 ---------------------------------------------------------------------- Ran 0 tests in 0.025s OK Any diagnostics or options I can pass to the tests to see what is going on? This is rather odd, I thought the tests would run and possibly fail. I didn't expect 0 tests to run. Thanks, Tom This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email. For other languages, go to http://www.3ds.com/terms/email-disclaimer

2 1

Name change of the ptp() function
by Eli Bressert 10 May '13

10 May '13

The peak-to-peak function, ptp(), is a simple and handy function to use, but it confuses the PEP20 suggestions a bit. See this discussion for reference: http://d.pr/i/snXG Based on the discussion, it may be a good idea to make the name more relevant? Some suggestions include span() or valuerange(). Among few others who have discussed this on Github (https://github.com/numpy/numpy/issues/3292#issuecomment-17353454) valuerange() appears to the best most favored one. -Eli

6 7

Scalar output from sub-classed Numpy array
by Thomas Robitaille 10 May '13

10 May '13

Hi everyone, I am currently trying to write a sub-class of Numpy ndarray, but am running into issues for functions that return scalar results rather than array results. For example, in the following case: import numpy as np class TestClass(np.ndarray): def __new__(cls, input_array, unit=None): return np.asarray(input_array).view(cls) def __array_finalize__(self, obj): if obj is None: return def __array_wrap__(self, out_arr, context=None): return np.ndarray.__array_wrap__(self, out_arr, context) I get: In [4]: a = TestClass([1,2,3]) In [5]: print type(np.dot(a,a)) <type 'numpy.int64'> In [6]: a = TestClass([[1,2],[1,2]]) In [7]: print type(np.dot(a,a)) <class '__main__.TestClass'> that is, in the case where the output is a scalar, it doesn't get wrapped, while in the case where the output is an array, it does. Could anyone explain this behavior to me, and most importantly, is there a way around this and have the above example return a wrapped 0-d TestClass array instead of a Numpy int64? Thanks, Tom

1 0

www.scipy.org down?
by KACVINSKY Tom 09 May '13

09 May '13

I am looking for a release of numpy for Mac OS, but I can't reach the scipy web server. IS it down for maintenance? Thanks, Tom This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email. For other languages, go to http://www.3ds.com/terms/email-disclaimer

3 6

Integer type casting and OverflowError
by Robert Kern 09 May '13

09 May '13

With master numpy (and back to 1.6.1, at least): [~] |1> np.int32(3054212286) -1240755010 It seems like at one time, this used to raise an OverflowError. We can see this in at least one place in scipy: https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L912 Is this a regression in numpy, or should the code in scipy be fixed to use a different test? Even if it is technically a regression, it might be one of those "it's been broken so long, it's a feature now" regressions. -- Robert Kern

3 5

Performance degradation when installing Numpy and Python not from packaging system?
by Bin Huang 08 May '13

08 May '13

Hi list, I have observed slowdown if Numpy and Python are installed manually (i.e. not from packaging system). The slowdown can be significant when problem size is large. I wish someone in this mailing list can give me some hints on this issue. I also post performance results and my source code here. My platform is a 12-core dual-socket SMP machine running Ubuntu (Linux kernel 3.2.0). On this platform, I have two sets of installations. One is pre-installed Python 2.7.3 and Numpy 1.6.1 installed with 'apt-get'. The other is Python 2.7.3 and Numpy 1.6.1 both compiled freshly from official tarball release by myself. In order to compare performance, I wrote a piece of code that simply performs matrix-matrix multiplication using one of Numpy's functions. The source code is listed as follows: *if __name__ == "__main__": import numpy, random, time import sys size = int(sys.argv[1]) # For simplicity, we only test square matrix matrix_a = numpy.matrix(numpy.random.randn(size,size)) matrix_b = numpy.matrix(numpy.random.randn(size,size)) start_time = time.time() result = numpy.dot(matrix_a, matrix_b) print '%0.3f ms' % ((time.time() - start_time)*1000.0)* I ran this code with both pre-installed Python and compiled Python. Here is performance numbers I got: Problem size Pre-installed Python Compiled Python/Numpy Slowdown 256x256 14.165 ms 44.538 ms ~3X 512x512 109.330 ms 347.143 ms ~3X 1024x1024 867.936 ms 8975.016 ms ~10X 2048x2048 7755.084 ms 84713.997 ms ~10X Here are commands I used to compile Python and Numpy: a) ~/python2.7.3$ ./configure --prefix=MY_LOCAL_DIR b) ~/python2.7.3$ make c) ~/python2.7.3$ make install e) ~/numpy-1.6.1$ MY_LOCAL_DIR/bin/python setup.py build f) ~/numpy-1.6.1$ MY_LOCAL_DIR/bin/python setup.py install Any ideas? Thanks in advance. Bin Huang

2 1

References to Numerical Recipes in Numpy code
by MACKEITH Andrew 08 May '13

08 May '13

The numpy libraries (numpy/core/machar.py, numpy/fft/info.py, numpy/lib/function_base.py) contain references to "Numerical Recipes: The Art of Scientific Computing". The book contains a licensing section which states "... Unlicensed transfer of Numerical Recipes programs to any other format is strictly prohibited". From this we understand that use of the book as a reference is permitted, but copying of code for distribution, whether non-commercially or otherwise is not permitted. We are wondering if anyone has done an analysis of this situation around numpy, and if so what conclusion they might have reached? Andrew MacKeith Python and Scripting Development Dassault Systemes Simulia Corp. This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email. For other languages, go to http://www.3ds.com/terms/email-disclaimer

2 1