Mailman 3 June 2012 - NumPy-Discussion

boolean indexing change
by Ralf Gommers 14 Jun '12

14 Jun '12

Something that we just ran into trying to merge a scipy PR: With 1.5.1: >>> np.arange(10)[np.array([0,1,0,1,2,3]) > 0] array([1, 3, 4, 5]) With current master: In [1]: np.arange(10)[np.array([0,1,0,1,2,3]) > 0] --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /Users/rgommers/<ipython-input-1-49ea14823e3d> in <module>() ----> 1 np.arange(10)[np.array([0,1,0,1,2,3]) > 0] ValueError: operands could not be broadcast together with shapes (10) (6) The reason for this is noted in the 2.0.0 release notes: "Full-array boolean indexing used to allow boolean arrays with a size non-broadcastable to the array size. Now it forces this to be broadcastable. Since this affects some legacy code, this change will require discussion during alpha or early beta testing, and a decision to either keep the stricter behavior, or add in a hack to allow the previous behavior to work. " I'm not opposed to the change in principle, but just wanted to note it can lead to code breaking and puzzled users. Ralf

3 2

[ANN] Bento 0.1.0
by David Cournapeau 14 Jun '12

14 Jun '12

Hi, I am pleased to announce a new release of bento, a packaging solution for python which aims at reproducibility, extensibility and simplicity. The main features of this 0.1.0 release are: - new commands register_pypi and upload_pypi to register a package to pypi and upload tarballs to it. - add sphinx command to build a package documentation if it uses sphinx. - add tweak_library/tweak_extension functions to build contexts to simplify simple builder customization (e.g. include_dirs, defines, etc...) - waf backend: cython tool automatically loaded if cython files are detected in sources - UseBackends feature: allows to declare which build backend to use when building C extensions in the bento.info file directly - add --use-distutils-flags configure option to force using flags from distutils (disabled by default). - add --disable-autoconfigure build option to bypass configure for fast partial rebuilds. This is not reliable depending on how the environment is changed, so one should only use this during development. - add register_metadata API to register new metadata to be filled in MetaTemplateFile Bento source code can be found on github: https://github.com/cournape/Bento Bento documentation is there as well: https://cournape.github.com/Bento regards, David

1 0

consecutive node sequence and pathlength of graph
by bob tnur 13 Jun '12

13 Jun '12

Let say,I have a conjugated cyclic polygon and its nodes are given by the list: list_p=[a,b,c,d,e,f,g,a,a,b,d,d,d,d,d,c,c,e,e,a,d,d,g]. If X & Y are any elements in a list_p except d, and Z is also an element of list_p but has value only d, i.e, Z=d. Now,I want to compute the number of paths with sequence X-Z- Y consecutive nodes, example: a-d-d-e,a-d-d-a, a-d-d-d-g, etc. Consecutive path length of Z can be 1,2,3,4 or 5. To generalize: Find total number of paths with consecutive X-Z-Y sequence for path length of z within range(1,6): or (i) X-Z-Y for len(Z)=1 (ii) X-Z-Y for len(Z)=2 (iii)X-Z-Y for len(Z)=3 (iv) X-Z-Y for len(Z)=4 (v) X-Z-Y for len(Z)=5 Is there any easy way to program this using python networkx or igraph? , can numpy group have some module to simplify this problem?

1 0

multiarray.so: undefined symbol: _Py_ascii_whitespace
by Nicholas Carl 13 Jun '12

13 Jun '12

Hello, I installed Numpy 1.5.1 with Python 2.7.2 and when I try to test my QIIME install it gives me the following error: /apps/qiime-1.5/dependencies/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ascii_whitespace ldd for that file gives: libm.so.6 => /lib64/libm.so.6 (0x00002b664c3a4000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b664c627000) libc.so.6 => /lib64/libc.so.6 (0x00002b664c842000) /lib64/ld-linux-x86-64.so.2 (0x0000003bae400000) I saw something similar here with no reply: http://article.gmane.org/gmane.comp.python.matplotlib.general/26025/match=m… Any ideas? Thanks! Nick

1 0

Neighborhood iterator: way to easily check which elements have already been visited in parent iterator?
by Thouis (Ray) Jones 13 Jun '12

13 Jun '12

Hello, I'm rewriting scipy.ndimage.label() using numpy's iterator API, and would like to add the ability for it to operate in-place. However, to do so, I need to limit the neighbors consulted to those that have already been processed in the parent iterator over the input and output arrays. Is there a general way to do this? If the parent iterator is constructed with the default flags (order='K' and readonly), can I assume that it will always move forward in memory? Thanks for any advice, Ray Jones

4 4

numpy array in networkx graph?
by bob tnur 13 Jun '12

13 Jun '12

can anyone give me a hint on the following code? import network as nx import pylab as plt G=nx.Graph(M) # M is numpy matrix ,i.e:type(M)=numpy.ndarray for i in xrange(len(M)): tt=P[i,:].sum() if tt==1: G.add_node(i,color='blue') elif tt==2: G.add_node(i,color='red') elif tt==3: G.add_node(i,color='white') else: tt==4 G.add_node(i,color='green') G.nodes(data=True) T=nx.draw(G) plt.axis('off') plt.savefig("test.png") I didn't get color change, still the defualt color is used.Did I miss something? my aim is to obtain: something like: find total number of w-red-red-z path number of w-red-red-red-z path number of w-red-red-red-red-z path where w (left side of some cyclic polygon(can also be conjugated ring)) and z(right-side of it)are any of the colors except red. any comment is appreciated? Thanks Bob

2 2

SciPy2012 conference: Last week for early birds, poster submissions
by Stéfan van der Walt 12 Jun '12

12 Jun '12

Hi everyone We're rapidly approaching SciPy2012 <http://conference.scipy.org/scipy2012>, which takes place in Austin, Texas from July 16th to 21st. This is a reminder that the *discounted early bird registration* closes on the 18th of this month. Also, we decided to keep the queue for *poster submissions* open until all slots are filled. So, whether you have a neat side project, a lightning talk gone rogue, or simply want to get the community talking about your latest and greatest idea--send in a poster abstract to 2012submissions(a)scipy.org. See you in Austin! Stéfan

1 0

Migrating numpy Trac to github issues
by Thouis (Ray) Jones 12 Jun '12

12 Jun '12

I've volunteered to help manage the migration of numpy tickets from Trac to github issues. The first part of this process is to decide which tickets to migrate, and how to map Trac ticket data to github issue data. Question 1: Which tickets should be migrated? Open? Open and recently closed? All tickets? Question 2: Which parts of the tickets should be included, and in what way? For reference, these are the parts of a ticket (from http://trac.edgewall.org/wiki/TracTickets) Reporter Type (e.g., defect, enhancementt). Component Version Keywords Priority Milestone Assigned to/Owner Cc Resolution (fixed, invalid, wontfix, duplicate, worksforme, others?) Status (e.g., new, needs_review, etc.) Summary (brief description) Description (full description) Comments Attachments (not listed on the link above, but enabled for numpy Trac) ??? (Other) — I don't have access to the Trac configuration, so it's not clear if there are other fields I may be missing. As an initial suggestion, I propose: Reporter - yes, mapping Trac users to github users where possible. Type - yes, using github issues labels Component - yes, using labels Version - probably yes, using labels Keywords - Yes, as a line in the github issue comments Priority - yes, using labels Milestone - yes, using github milestones Assigned to/Owner - yes, mapping to github users Cc - Yes, for github users we can map to, using @username Resolution - Only if tickets other than "open" are migrated, and as part of the issue comments Status - Yes, using labels Summary - Yes, as the github issue title Description - Yes, as the github issue body Comments - Yes, as github issue comments Attachments - Yes, linked in an issue comment Note that the Trac database will be available for some time, and every migrated ticket will include a link to the original. If the Trac database is ever disabled, these links could be migrated to a static copy of the site. At some point, there may be a new location for ticket attachments, requiring changing links in some github issues. That's a separate issue from this discussion. Trac allows WikiFormatting, while github has a more limited markup language. As much translation as is possible will be performed within comments (code blocks, at an absolute minimum). Tickets will be migrated to a test repository before being moved to the numpy github repo, and there will be ample time to review the result before proceeding beyond that step. Ray Jones

2 1

not expected output of fill_diagonal
by Frédéric Bastien 11 Jun '12

11 Jun '12

Hi, While reviewing the Theano op that wrap numpy.fill_diagonal, we found an unexpected behavior of it: # as expected for square matrix >>> a=numpy.zeros((5,5)) >>> numpy.fill_diagonal(a, 10) >>> print a # as expected long rectangular matrix >>> a=numpy.zeros((3,5)) >>> numpy.fill_diagonal(a, 10) >>> print a [[ 10. 0. 0. 0. 0.] [ 0. 10. 0. 0. 0.] [ 0. 0. 10. 0. 0.]] # Not as expected >>> a=numpy.zeros((5,3)) >>> numpy.fill_diagonal(a, 10) >>> print a [[ 10. 0. 0.] [ 0. 10. 0.] [ 0. 0. 10.] [ 0. 0. 0.] [ 10. 0. 0.]] I can make a PR that will add a parameter wrap that allow to control if it return the old behavior or what I would expect in the last case: [[ 10. 0. 0.] [ 0. 10. 0.] [ 0. 0. 10.] [ 0. 0. 0.] [ 0. 0. 0.]] My questions is, do someone else expect the current behavior? Should we change the default to be what I expect? Do you want that we warn if the user didn't specify witch behavior and in the future we change it? Anything else I didn't think? thanks Fred

2 2

lazy evaluation
by mark florisson 11 Jun '12

11 Jun '12

Hey, Another discussion on lazy evaluation, given the recent activity here: https://github.com/ContinuumIO/numba/pull/6#issuecomment-6117091 A somewhat recent previous thread can be found here: http://mail.scipy.org/pipermail/numpy-discussion/2012-February/060862.html , and a NEP here: https://github.com/numpy/numpy/blob/master/doc/neps/deferred-ufunc-evaluati… I think trying to parse bytecode and build an expression graph for array expressions from that has disadvantages and is harder in general. For instance it won't be able to deal with branching at execution time, and things like inter-procedural analysis will be harder (not to mention you'd have to parse dtype creation). Instead, what you really want to do is hook into a lazy evaluating version of numpy, and generate your own code from the operations it records. It would be great if we implement the NEP listed above, but with a few extensions. I think Numpy should handle the lazy evaluation part, and determine when expressions should be evaluated, etc. However, for each user operation, Numpy will call back a user-installed hook implementing some interface, to allow various packages to provide their own hooks to evaluate vector operations however they want. This will include packages such as Theano, which could run things on the GPU, Numexpr, and in the future https://github.com/markflorisson88/minivect (which will likely have an LLVM backend in the future, and possibly integrated with Numba to allow inlining of numba ufuncs). The project above tries to bring together all the different array expression compilers together in a single framework, to provide efficient array expressions specialized for any data layout (nditer on steroids if you will, with SIMD, threaded and inlining capabilities). We could allow each hook to specify which dtypes it supports, and a minimal data size needed before it should be invoked (to avoid overhead for small arrays, like the openmp 'if' clause). If an operation is not supported, it will simply raise NotImplementedError, which means Numpy will evaluate the expression built so far and run its own implementation, resulting in a non-lazy array. E.g. if a library supports adding things together, but doesn't support the 'sin' function, np.sin(a + b) will result in the library executing a + b, and numpy evaluating sin on the result. So the idea is that the numpy lazy array will wrap an expression graph, which is built when the user performs operations and evaluated when needed (when a result is required or when someone tells numpy to evaluate all lazy arrays). Numpy will simply use the first hook willing to operate on data of the specified size and dtype, and will keep using that hook to build the expression until evaluated. Anyway, this is somewhat of a high-level overview. If there is any interest, we can flesh out the details and extend the NEP. Mark

5 15