Mailman 3 April 2014 - NumPy-Discussion

should rint return int?
by Neal Becker April 28, 2014

April 28, 2014

I notice rint returns float. Shouldn't it return int? Would be useful when float is no longer acceptable as an index. I think conversion to an index using rint is a common idiom.

5 6

Dates and times and Datetime64 (again)
by Sankarshan Mudkavi April 28, 2014

April 28, 2014

Hey all, It's been a while since the last datetime and timezones discussion thread was visited (linked below): http://thread.gmane.org/gmane.comp.python.numeric.general/53805 It looks like the best approach to follow is the UTC only approach in the linked thread with an optional flag to indicate the timezone (to avoid confusing applications where they don't expect any timezone info). Since this is slightly more useful than having just a naive datetime64 package and would be open to extension if required, it's probably the best way to start improving the datetime64 library. If we do wish to have full timezone support it would very likely lead to performance drops (as reasoned in the thread) and we would need to have a dedicated, maintained tzinfo package, at which point it would make much more sense to just incorporate the pytz library. (I also don't have the expertise to implement this, so I would be unable to help resolve the current logjam) I would like to start writing a NEP for this followed by implementation, however I'm not sure what the format etc. is, could someone direct me to a page where this information is provided? Please let me know if there are any ideas, comments etc. Cheers, Sankarshan

10 49

Slightly off-topic - accuracy of C exp function?
by Matthew Brett April 27, 2014

April 27, 2014

Hi, I'm exploring Mingw-w64 for numpy building, and I've found it gives a slightly different answer for 'exp' than - say - gcc on OSX. The difference is of the order of the eps value for the output number (2 * eps for a result of ~2.0). Is accuracy somewhere specified for C functions like exp? Or is accuracy left as an implementation detail for the C library author? Cheers, Matthew

5 8

Proposed new function for joining arrays: np.interleave
by Björn Dahlgren April 26, 2014

April 26, 2014

Hello, Interleaving arrays is something I need to do every now and then, and by the looks of stackoverflow so do others: http://stackoverflow.com/questions/12861314/interleave-rows-of-two-numpy-ar… http://stackoverflow.com/questions/5347065/interweaving-two-numpy-arrays I think the code needed for the general n dimensional case with m number of arrays is non-trivial enough for it to be useful to provide such a function in numpy, so I took the liberty to make a Pull Request <https://github.com/numpy/numpy/pull/4596> with my implementation. This would be my first contribution to NumPy so I apologize if something is not adhering to your practices. Jaime has already pointed out that I should email this list (I hope I managed to email the list in the right way - never used a mailing list before) for you to notice the pull request. He also pointed out some improvements of my proposed implementation (not forcing consistent dtype), but before I go on and make changes I might need to ask you guys if this is really something you are interested in including? Best regards, /Björn Dahlgren

3 2

1.9.x branch
by Charles R Harris April 26, 2014

April 26, 2014

Hi All, I'd like to branch 1.9.x at the end of the month. There are a couple of reasons for the timing. First, we have a lot of new stuff in the development branch. Second, there is work ongoing in masked arrays that I'd like to keep out of the release so that it has more time to settle. Third, it's past time ;) There are currently a number of 1.9.0 blockers, which can be seen here<https://github.com/numpy/numpy/issues?direction=desc&milestone=29&page=1&so…>. *Datetime timezone handling broken in 1.7.x*<https://github.com/numpy/numpy/issues/3388> I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. *Return multiple field selection as ro view <https://github.com/numpy/numpy/issues/3804>* I have a branch for this, but because the returned type differs from a copy by alignment spacing there was a test failure. Merging that branch might cause some incompatibilities. *Object array creation new conversion to int*<https://github.com/numpy/numpy/issues/3804> This one needs a decision. Julian, Sebastian, thoughts? *Median of np.matrix is broken* <https://github.com/numpy/numpy/issues/4301> Not sure what the status of this one is. *1.8 deprecations: Follow-up ticket <https://github.com/numpy/numpy/issues/4131>* Things that might should be removed. *ERROR: test_big_arrays (test_io.TestSavezLoad) on OS X + Python 3.3 <https://github.com/numpy/numpy/issues/3858>* I believe this one was fixed. For general problems reading/writing big files on OS X, I believe they were fixed in Maverick and I'm inclined to recommend an OS upgrade rather than work to chunk all the io. Deprecate NPY_CHAR <https://github.com/numpy/numpy/issues/2801>This one is waiting on a fix from Pearu to make f2py use numpy strings. I think we will need to do this ourselves if we want to carry through the deprecation. In any case it probably needs to be pushed off to 1.10. 1.7 deprecations: Follow-up ticket<https://github.com/numpy/numpy/issues/456>Some of these changes have been made, ro diagonal view for instance, some still remain. Additions, updates, and thoughts welcome. Chuck

4 5

Test error with ATLAS, Windows 64 bit
by Matthew Brett April 24, 2014

April 24, 2014

Hi, With Carl Kleffner, I am trying to build a numpy 1.8.1 wheel for Windows 64-bit, and latest stable ATLAS. It works fine, apart from the following test failure: ====================================================================== FAIL: test_special (test_umath.TestExpm1) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python27\lib\site-packages\numpy\core\tests\test_umath.py", line 329, in test_special assert_equal(ncu.expm1(-0.), -0.) File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 311, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 0.0 DESIRED: -0.0 Has anyone seen this? Is it in fact necessary that expm1(-0.) return -0 instead of 0? Thanks a lot, Matthew

7 17

ANN: DistArray 0.2 -- first public development release
by Kurt Smith April 23, 2014

April 23, 2014

GitHub repo: https://github.com/enthought/distarray Documentation: http://distarray.readthedocs.org License: Three-clause BSD Python versions: 2.7 and 3.3 OS support: *nix and Mac OS X DistArray aims to bring the strengths of NumPy to data-parallel high-performance computing. It provides distributed multi-dimensional NumPy-like arrays and distributed ufuncs, distributed IO capabilities, and can integrate with external distributed libraries, like Trilinos. DistArray works with NumPy and builds on top of it in a flexible and natural way. Brian Granger started DistArray as a NASA-funded SBIR project in 2008. Enthought picked it up as part of a DOE Phase II SBIR [0] to provide a generally useful distributed array package. It builds on IPython, IPython.parallel, NumPy, MPI, and interfaces with the Trilinos suite of distributed HPC solvers (via PyTrilinos) [1]. Distarray: - has a client-engine (or master-worker) process design -- data resides on the worker processes, commands are initiated from master; - allows full control over what is executed on the worker processes and integrates transparently with the master process; - allows direct communication between workers bypassing the master process for scalability; - integrates with IPython.parallel for interactive creation and exploration of distributed data; - supports distributed ufuncs (currently without broadcasting); - builds on and leverages MPI via MPI4Py in a transparent and user-friendly way; - supports NumPy-like structured multidimensional arrays; - has basic support for unstructured arrays; - supports user-controllable array distributions across workers (block, cyclic, block-cyclic, and unstructured) on a per-axis basis; - has a straightforward API to control how an array is distributed; - has basic plotting support for visualization of array distributions; - separates the array’s distribution from the array’s data -- useful for slicing, reductions, redistribution, broadcasting, all of which will be implemented in coming releases; - implements distributed random arrays; - supports `.npy`-like flat-file IO and hdf5 parallel IO (via h5py); leverages MPI-based IO parallelism in an easy-to-use and transparent way; and - supports the distributed array protocol<http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/>[2], which allows independently developed parallel libraries to share distributed arrays without copying, analogous to the PEP-3118 new buffer protocol. This is the first public development release. DistArray is not ready for real-world use, but we want to get input from the larger scientific-Python community to help drive its development. The API is changing rapidly and we are adding many new features on a fast timescale. For that reason, DistArray is currently implemented in pure Python for maximal flexibility. Performance improvements are coming. The 0.2 release's goals are to provide the components necessary to support upcoming features that are non-trivial to implement in a distributed environment. Planned features for upcoming releases: - Distributed reductions - Distributed slicing - Distributed broadcasting - Distributed fancy indexing - Re-distribution methods - Integration with Trilinos [1] and other packages [3] that are compatible with the distributed array protocol [2] - Lazy evaluation and deferred computation for latency hiding - Out-of-core computations - Extensive examples, tutorials, documentation - Support for distributed sorting and other non-trivial distributed algorithms - MPI-only communication for non-interactive deployment on clusters and supercomputers - End-user control over communication, temporary array creation, and other performance aspects of distributed computations [0] http://www.sbir.gov/sbirsearch/detail/410257 [1] http://trilinos.org/ [2] http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/ [3] http://www.mcs.anl.gov/petsc/ -- Kurt W. Smith, Ph.D. <ksmith(a)enthought.com> Enthought, Inc. <http://www.enthought.com> | 512.536.1057

1 0

numerical gradient, Jacobian, and Hessian
by Alan G Isaac April 23, 2014

April 23, 2014

Awhile back there were good signs that SciPy would end up with a `diff` module: https://github.com/scipy/scipy/issues/2035 Is this still moving forward? It would certainly be nice for SciPy to have intuitive numerical gradients, Jacobians, and Hessians. The last two are I think missing altogether. The first exists as scipy.optimize.approx_fprime. `approx_fprime` seems to work fine, but I suggest it has the following drawbacks: - it is hard to find (e.g., try doing a Google search on "scipy gradient" or "scipy numerical gradient" - related, it is in the wrong location (scipy.optimize) - the signature is odd: (x,f,dx) instead of (f,x,dx) (This matters for ease of recall and for teaching.) In any case, as I understand it, the author's of numdifftools http://code.google.com/p/numdifftools/ expressed willingness to have their code moved into SciPy. This seems like an excellent way forward. There was talk of making this a summer of code project, but that seems to have sputtered. Alan Isaac

4 3

f2py links extensions to incorrect python installation on OSX / Anaconda
by Alex Goodman April 21, 2014

April 21, 2014

Hi all, I have used f2py in the past on a Linux machine with virtually no issues. However on my Mac, I get the following error when importing an f2py generated extension: Fatal Python error: PyThreadState_Get: no current thread Abort trap: 6 After doing some research I found out that the extension is linked to the wrong python installation: otool -L add.so add.so: ./add.so (compatibility version 0.0.0, current version 0.0.0) /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.2) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0) This seems odd because I am using the f2py executable included in Anaconda 1.9.1. I can easily fix this problem by manually using install_name_tool -change on the extension to link the correct library location, but this is really cumbersome. Is there an alternative solution, such as an additional command-line argument when invoking f2py? For what it is worth, I am also using Version 14.0.2 of the Intel Fortran Compiler. Thanks, Alex -- Alex Goodman Graduate Research Assistant Department of Atmospheric Science Colorado State University

4 4

string replace
by Siegfried Gonzi April 21, 2014

April 21, 2014

Hi all I know this is not numpy related but a colleague insists the following is supposed to work. But it doesn't: == line_left = './erfo/restart.ST010.EN0001-EN0090.YYYYMMDDhh' enafix = 'ST000.EN0001-EN0092' line_left = line_left.replace('STYYY.ENXXXX-ENXXXX', enafix) print 'line_left',line_left == [right answer would be: ./erfo/restart.ST000.EN0001-EN0092.YYYYMMDDhh' ] I think it works in Fortran but not in Python. What would be the easiest way to replacing this kind of pattern in a variable lenght string? Thanks, Siegfried -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

3 2