Mailman 3 June 2017 - NumPy-Discussion

[SciPy-Dev] PyRSB: Python interface to librsb sparse matrices library
by Michele Martone 21 Jun '17

21 Jun '17

Hi. I'm the author of the high performance multithreaded sparse matrix library `librsb' (mostly C, LGPLv3): http://librsb.sourceforge.net/ I'm *not* a user of SciPy/NumPy/Python, but using Cython I have written a proof-of-concept interface to librsb, named `PyRSB': https://github.com/michelemartone/pyrsb PyRSB is in a prototypal state; e.g. still lacks good error handling. Its interface is trivial, as it mimicks that of SciPy's 'csr_matrix'. Advantages over csr_matrix are in fast multithreaded multiplication of huge sparse matrices. Intended application area is iterative solution of linear systems; particularly fast if with symmetric matrices and many rhs. With this email I am looking for prospective: - users/testers - developers (any interest to collaborate/adopt/include the project?) Looking forward for your feedback, Michele

1 0

Controlling NumPy __mul__ method or forcing it to use __rmul__ of the "other"
by Ilhan Polat 19 Jun '17

19 Jun '17

I will assume some simple linear systems knowledge but the question can be generalized to any operator that implements __mul__ and __rmul__ methods. Motivation: I am trying to implement a gain matrix, say 3x3 identity matrix, for time being with a single input single output (SISO) system that I have implemented as a class modeling a Transfer or a state space representation. In the typical usecase, suppose you would like to create an n-many parallel connections with the same LTI system sitting at each branch. MATLAB implements this as an elementwise multiplication and returning a multi input multi output(MIMO) system. G = tf(1,[1,1]); eye(3)*G produces (manually compactified) ans = From input 1 to output... [ 1 ] [ ------ , 0 , 0 ] [ s + 1 ] [ 1 ] [ 0 , ------ , 0 ] [ s + 1 ] [ 1 ] [ 0 , 0 , ------ ] [ s + 1 ] Notice that the result type is of LTI system but, in our context, not a NumPy array with "object" dtype. In order to achieve a similar behavior, I would like to let the __rmul__ of G take care of the multiplication. In fact, when I do G.__rmul__(np.eye(3)) I can control what the behavior should be and I receive the exception/result I've put in. However the array never looks for this method and carries out the default array __mul__ behavior. The situation is similar if we go about it as left multiplication G*eye(3) has no problems since this uses directly the __mul__ of G. Therefore we get a different result depending on the direction of multiplication. Is there anything I can do about this without forcing users subclassing or just letting them know about this particular quirk in the documentation? What I have in mind is to force the users to create static LTI objects and then multiply and reject this possibility. But then I still need to stop NumPy returning "object" dtyped array to be able to let the user know about this. Relevant links just in case the library : https://github.com/ilayn/harold/ the issue discussion (monologue actually) : https://github.com/ilayn/harold/issues/7 The question I've asked on SO (but with a rather offtopic answer): https://stackoverflow.com/q/40694380/4950339 ilhan

3 5

np.diff on boolean arrays now raises
by Jaime Fernández del Río 15 Jun '17

15 Jun '17

There is an ongoing discussion on github: https://github.com/numpy/numpy/issues/9251 In 1.13 np.diff has started raising on boolean arrays, since subtraction of boolean arrays is now deprecated. A decision has to be made whether: - raising an error is the correct thing to do, and only the docstring needs updating, or - backwards compatibility is more important and diff should still work on boolean arrays. If you have opinions on the subject, or enjoy arguing for the sake of it, please share your thoughts either here or on github. Thanks! -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

4 4

Taking NumPy in Stride workshop at PyData Barcelona
by Jaime Fernández del Río 15 Jun '17

15 Jun '17

I ended up going to the PyData event in Barcelona last month and running a workshop there. The notebook I used can be found here: https://github.com/jaimefrio/pydatabcn2017/tree/master/taking_numpy_in_stri… I was afraid that the content would be too technical, but all the feedback I got from attendees was that they very much enjoyed it and want more of this deep technical dives into the belly of the beast. Of course IRL, contrary to the internet, it is easier to approach someone to tell them you like what they presented, than to tell them they suck. So maybe this is just a biased illusion, but until proven wrong I am encouraging anyone with the knowledge and the opportunity to go ahead and organize a similar workshop in any conference they go to. Jaime P.S. I also got a chance to catch up with Francesc Alted and Travis Oliphant, which is always great! -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

4 3

EuroSciPy 2017 call for contributions
by Pierre de Buyl 12 Jun '17

12 Jun '17

(Apologies if you receive multiple copies of this message) 10th European Conference on Python in Science August 28 - September 1, 2017 in Erlangen, Germany Description: The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research. This event strives to bring together both users and developers of scientific tools, as well as academic research and state of the art industry. Erlangen is one of Germany's major science hubs and located north of Munich (90 minutes by train). Topics of Interest Presentations of scientific tools and libraries using the Python language, including but not limited to: - Vector and array manipulation - Parallel computing - Scientific visualization - Scientific data flow and persistence - Algorithms implemented or exposed in Python - Web applications and portals for science and engineering- - Reports on the use of Python in scientific achievements or ongoing projects. - General-purpose Python tools that can be of special interest to the scientific community. Submission Details: You can send your proposal by submitting your application here: https://www.papercall.io/euroscipy-2017 The Call for Papers closes on June 25, 2017 00:00 CEST Regards, The EuroSciPy team https://www.euroscipy.org/2017/

1 0

Add changelogs to repo
by Charles R Harris 07 Jun '17

07 Jun '17

Hi All, I've made a PR <https://github.com/numpy/numpy/pull/9226> adding the NumPy `1.13.0-changelog.rst` to a new `doc/changelog` directory. The idea is to keep the lengthy changelogs separate from the release notes, but still easily accessible and permanently part to the NumPy history. If you hit the `rich diff` tag at the PR, you can see the rendered rst version. Thoughts? Chuck

1 0

NumPy 1.13.0 release
by Charles R Harris 07 Jun '17

07 Jun '17

Hi All, On behalf of the NumPy team, I amn pleased to announce the NumPy 1.13.0 release. This release supports Python 2.7 and 3.4-3.6 and contains many new features. It is one of the most ambitious releases in the last several years. Some of the highlights and new functions are *Highlights* - Operations like ``a + b + c`` will reuse temporaries on some platforms, resulting in less memory use and faster execution. - Inplace operations check if inputs overlap outputs and create temporaries to avoid problems. - New __array_ufunc__ attribute provides improved ability for classes to override default ufunc behavior. - New np.block function for creating blocked arrays. *New functions* - New ``np.positive`` ufunc. - New ``np.divmod`` ufunc provides more efficient divmod. - New ``np.isnat`` ufunc tests for NaT special values. - New ``np.heaviside`` ufunc computes the Heaviside function. - New ``np.isin`` function, improves on ``in1d``. - New ``np.block`` function for creating blocked arrays. - New ``PyArray_MapIterArrayCopyIfOverlap`` added to NumPy C-API. Wheels for the pre-release are available on PyPI. Source tarballs, zipfiles, release notes, and the changelog are available on github <https://github.com/numpy/numpy/releases/tag/v1.13.0>. A total of 102 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - A. Jesse Jiryu Davis + - Alessandro Pietro Bardelli + - Alex Rothberg + - Alexander Shadchin - Allan Haldane - Andres Guzman-Ballen + - Antoine Pitrou - Antony Lee - B R S Recht + - Baurzhan Muftakhidinov + - Ben Rowland - Benda Xu + - Blake Griffith - Bradley Wogsland + - Brandon Carter + - CJ Carey - Charles Harris - Christoph Gohlke - Danny Hermes + - David Hagen + - David Nicholson + - Duke Vijitbenjaronk + - Egor Klenin + - Elliott Forney + - Elliott M Forney + - Endolith - Eric Wieser - Erik M. Bray - Eugene + - Evan Limanto + - Felix Berkenkamp + - François Bissey + - Frederic Bastien - Greg Young - Gregory R. Lee - Importance of Being Ernest + - Jaime Fernandez - Jakub Wilk + - James Cowgill + - James Sanders - Jean Utke + - Jesse Thoren + - Jim Crist + - Joerg Behrmann + - John Kirkham - Jonathan Helmus - Jonathan L Long - Jonathan Tammo Siebert + - Joseph Fox-Rabinovitz - Joshua Loyal + - Juan Nunez-Iglesias + - Julian Taylor - Kirill Balunov + - Likhith Chitneni + - Loïc Estève - Mads Ohm Larsen - Marein Könings + - Marten van Kerkwijk - Martin Thoma - Martino Sorbaro + - Marvin Schmidt + - Matthew Brett - Matthias Bussonnier + - Matthias C. M. Troffaes + - Matti Picus - Michael Seifert - Mikhail Pak + - Mortada Mehyar - Nathaniel J. Smith - Nick Papior - Oscar Villellas + - Pauli Virtanen - Pavel Potocek - Pete Peeradej Tanruangporn + - Philipp A + - Ralf Gommers - Robert Kern - Roland Kaufmann + - Ronan Lamy - Sami Salonen + - Sanchez Gonzalez Alvaro - Sebastian Berg - Shota Kawabuchi - Simon Gibbons - Stefan Otte - Stefan Peterson + - Stephan Hoyer - Søren Fuglede Jørgensen + - Takuya Akiba - Tom Boyd + - Ville Skyttä + - Warren Weckesser - Wendell Smith - Yu Feng - Zixu Zhao + - Zè Vinícius + - aha66 + - drabach + - drlvk + - jsh9 + - solarjoe + - zengi + Cheers, Chuck

1 0

Variance without temporary array
by marc 07 Jun '17

07 Jun '17

Dear Numpy developers, I'm trying to write a version of the variance function that does not use temporary array. I got something that start to be functional (the branch is located here https://github.com/mbarbry/numpy/tree/sum_square_shift) but when running the tests I get the following memory error, Running unit tests for numpy NumPy version 1.14.0.dev0+f9db20b NumPy relaxed strides checking option: True NumPy is installed in /home/marc/programs/python/my_numpy/local/lib/python2.7/site-packages/numpy-1.14.0.dev0+f9db20b-py2.7-linux-x86_64.egg/numpy Python version 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] nose version 1.3.7 Program received signal SIGSEGV, Segmentation fault. 0x00007ffff1ef1f6d in inittest_rational () at numpy/core/src/umath/test_rational.c.src:1262 1262 REGISTER_UFUNC_BINARY_RATIONAL(add) This is weird because I didn't touch the umath module, I'm working in the multiarray one. Do you have a idea how I can debug this? Thanks for your help, Marc Barbry

1 0

Array and string interoperability
by Mikhail V 06 Jun '17

06 Jun '17

Array and string interoperability Just sharing my thoughts and few ideas about simplification of casting strings to arrays. In examples assume Numpy is in the namespace (from numpy import *) Initialize array from a string currently looks like: s= "012 abc" A= fromstring(s,"u1") print A -> [48 49 50 32 97 98 99] Perfect. Now when writing values it will not work as IMO it should, namley consider this example: B= zeros(7,"u1") B[0]=s[1] print B -> [1 0 0 0 0 0 0] Ugh? It tries to parse the s[1] character "1" as integer and writes 1 to B[0]. First thing I would expect is a value error and I'd never expect it does that high-level manipulations with parsing. IMO ideally it would do the following instead: B[0]=s[1] print B -> [49 0 0 0 0 0 0] So it should just write ord(s[1]) to B. Sounds logical? For me very much. Further, one could write like this: B[:] = s print B-> [48 49 50 32 97 98 99] Namely cast the string into byte array. IMO this would be the logical expected behavior. Currently it just throws the value error if met non-digits in a string, so IMO current casting hardly can be of practical use. Furthermore, I think this code: A= array(s,"u1") Could act exactly same as: A= fromstring(s,"u1") But this is just a side-idea for spelling simplicty/generality. Not really necessary. Further thoughts: If trying to create "u1" array from a Pyhton 3 string, question is, whether it should throw an error, I think yes, and in this case "u4" type should be explicitly specified by initialisation, I suppose. And e.g. translation from unicode to extended ascii (Latin1) or whatever should be done on Python side or with explicit translation. Python3 assumes 4-byte strings but in reality most of the time we deal with 1-byte strings, so there is huge waste of resources when dealing with 4-bytes. For many serious projects it is just not needed. Furthermore I think some of the methods from "chararray" submodule should be possible to use directly on normal integer arrays without conversions to other array types. So I personally don't realy get why the need of additional chararray type, Its all numbers anyway and it's up to the programmer to decide what size of translation tables/value ranges he wants to use. There can be some convinience methods for ascii operations, like eg char.toupper(), but currently they don't seem to work with integer arrays so why not make those potentially useful methots usable and make them work on normal integer arrays? Or even migrate them to the root namespace to e.g. introduce names with prefixes: A=ascii_toupper(A) A=ascii_tolower(A) Many things can be be achieved with general numeric methods, e.g. translate/reduce the array. Here obviosly I mean not dynamical arrays, just fixed-sized arrays. How to deal with dynamically changing array sizes is another problematic, and it depends on how the software is designed in the first place and what it does with the data. For my own text-editing software project I consider fixed allocated 1D and 2D "uint8" arrays only. And specifically I experiment with own encodings, so just as a side-note, I don't think that encoding should be assumed much for creating new array types, it is up to the programmer to decide what 'meanings' the bytes have. Kind regards, Mikhail

3 9

Evaluate function of pairs from two meshgrids
by Florian Lindner 02 Jun '17

02 Jun '17

Hello, I have 2*N arrays (let's say 2 arrays of length 4) that I combine using np.meshgrid xxA, yyA = np.meshgrid(xA, yA) xxB, yyB = np.meshgrid(xB, yB) which gives me two meshes xx.shape = yy.shape = (4,4) wish represent N-dimensional mesh with 16 elements. Now I want to evaluate a function f on every possible pair of N-dimensional points in the grid, resulting in a 16 x 16 matrix: in a flattened notation, pA = (xxA, yyA) f(pA[1]-pB[1]) f(pA[1]-pB[2]) f(pA[1]-pB[3]) ... f(pA[2]-pB[1]) f(pA[2]-pB[2]) f(pA[2]-pB[3]) ... f(pA[3]-pB[1]) f(pA[3]-pB[2]) f(pA[3]-pB[3]) ... . . . Let's say xA = yA = [1,2,3] and xB = yB = [10,20,30] that gives me a mesh A: (1,3) (2,3) (3,3) (1,2) (2,2) (3,2) (1,1) (2,1) (3,1) and a mesh B alike. My result matrix now should be of size 9 x 9: f( (1,3), (10,30) ) f( (2,3), (20,30) ) f( (3,3), (30, 30) ) f( (1,2), (10,20) ) f( (2,2), (20,20) ) f( (3,2), (30, 20) ) ... f always takes two N-dimensional vectors and returns a scalar. I hope I was able to explain what I want to achieve. What is the best way to do that in numpy? Thanks, Florian

1 0

NumPy-Discussion June 2017