Mailman 3 May 2008 - NumPy-Discussion

SQUEEK, SQUEEK, trac ticket mailing list still busted.
by Charles R Harris May 23, 2008

May 23, 2008

I've unsubscribed, resubscribed, changed addresses, and nothing works. Wassup with that? Chuck

1 1

Multiple Boolean Operations
by Andrea Gavana May 23, 2008

May 23, 2008

Hi All, I am building some 3D grids for visualization starting from a much bigger grid. I build these grids by satisfying certain conditions on x, y, z coordinates of their cells: up to now I was using VTK to perform this operation, but VTK is slow as a turtle, so I thought to use numpy to get the cells I am interested in. Basically, for every cell I have the coordinates of its center point (centroids), named xCent, yCent and zCent. These values are stored in numpy arrays (i.e., if I have 10,000 cells, I have 3 vectors xCent, yCent and zCent with 10,000 values in them). What I'd like to do is: # Filter cells which do not satisfy Z requirements: zReq = zMin <= zCent <= zMax # After that, filter cells which do not satisfy Y requirements, # but apply this filter only on cells who satisfy the above condition: yReq = yMin <= yCent <= yMax # After that, filter cells which do not satisfy X requirements, # but apply this filter only on cells who satisfy the 2 above conditions: xReq = xMin <= xCent <= xMax I'd like to end up with a vector of indices which tells me which are the cells in the original grid that satisfy all 3 conditions. I know that something like this: zReq = zMin <= zCent <= zMax Can not be done directly in numpy, as the first statement executed returns a vector of boolean. Also, if I do something like: zReq1 = numpy.nonzero(zCent <= zMax) zReq2 = numpy.nonzero(zCent[zReq1] >= zMin) I lose the original indices of the grid, as in the second statement zCent[zReq1] has no more the size of the original grid but it has already been filtered out. Is there anything I could try in numpy to get what I am looking for? Sorry if the description is not very clear :-D Thank you very much for your suggestions. Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://xoomer.alice.it/infinity77/

9 18

C API
by Charles R Harris May 23, 2008

May 23, 2008

All, I added a function to array_api_order.txt and apparently this changed the order of the pointers in the API, which caused ctypes to segfault until I removed the build directory and did a complete rebuild. It seems to me that if we really want to make adding these API functions safe, then we should only have one list instead of the current two. This looks to require some mods to the build system. What do folks think? Chuck

2 5

Different attributes for NumPy types
by Bruce Southey May 23, 2008

May 23, 2008

Hi, Is it bug if different NumPy types have different attributes? Based on prior discussion, 'complex', 'float' and 'int' are Python types and others are NumPy types. Consequently 'complex', 'float' and 'int' do not inherit from NumPy. However, an element from array created using dtype=numpy.float has the numpy.float64 type. So this is really a documentation issue than an implementation issue. Also different NumPy types have different attributes, for example, 'float64' contains attributes (eg __coerce__) that are not present in 'float32' and 'float128' (these two have the same attributes). This can cause attribute errors in somewhat contrived examples that probably are unlikely to appear in practice because of the casting involved in array creation. The 'uint' types all seem to have the same attributes so do not have these issues. import numpy len(dir(float)) # 47 len(dir(numpy.float)) # 47 len(dir(numpy.float32)) # 131 len(dir(numpy.float64)) # 135 len(dir(numpy.float128)) # 131 len(dir(int)) # 54 len(dir(numpy.int)) # 54 len(dir(numpy.int0)) # 135 len(dir(numpy.int16)) # 132 len(dir(numpy.int32)) # 132 len(dir(numpy.int64)) # 135 len(dir(numpy.int8)) # 132 print (numpy.float64(1234).size) # 1 print (numpy.float(1234).size) ''' prints error: Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'float' object has no attribute 'size' ''' Regards Bruce

4 9

workaround for searchsorted with strings?
by Lewis Hyatt May 22, 2008

May 22, 2008

Hello- I see from this thread: http://article.gmane.org/gmane.comp.python.numeric.general/18746/ that searchsorted does not work correctly with strings. Is there a workaround, though, that I can use with 1.0.4 until there is a new official numpy release that includes the fix mentioned in the reference above? Using the latest SVN version is not an option for me. My understanding was that searchsorted works OK if the strings are all the same data type, but that does not appear to be the case: p >>> x=array(['0', '1', '2', '12']) p >>> y=array(['0', '0', '2', '3', '123']) p >>> x.searchsorted(y) array([0, 0, 0, 2, 0]) p >>> x.astype(y.dtype).searchsorted(y) array([0, 0, 2, 4, 2]) I understand that the first call to searchsorted fails because y has type S3 and x has type S2. But it seems that changing the type of x produces still incorrect (albeit) different results. Is there something similar I can do to make this work for now? Thanks very much. -Lewis

2 3

new numpy docs, missing function and parse error - dirichlet distribution
by joep May 22, 2008

May 22, 2008

Hi, I was just looking around at the new numpy documentation and got a xhtml parsing error on the page (with Firefox): http://mentat.za.net/numpy/refguide/random.xhtml#index-29351 The offending line contains $X pprox prod_{i=1}^{k}{x^{lpha_i-1}_i}$< in the docstring of the dirichlet distribution the corresponding line in the source at http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/random/mtrand/mtr… is .. math:: X \\approx \\prod_{i=1}^{k}{x^{\\alpha_i-1}_i} (I have no idea, why it seems not to parse \\a correctly). When looking for this, I found that the Dirichlet distribution is missing from the new Docstring Wiki, http://sd-2116.dedibox.fr/doc/Docstrings/numpy/random Then I saw that Dirichlet is also missing in __all__ in http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/random/info.py As a consequence numpy.lookfor does not find Dirichlet >>> numpy.lookfor('dirichlet') Search results for 'dirichlet' ------------------------------ >>> import numpy.random >>> dir(numpy.random) contains dirichlet >>> numpy.random.__all__ does not contain dirichlet. To me this seems to be a documentation bug. Josef

2 4

buglet: Dirichlet missing in numpy.random.__all__ as defined in numpy/random/info.py
by joep May 22, 2008

May 22, 2008

The Dirichlet distribution is missing in __all__ in http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/random/info.py As a consequence numpy.lookfor does not find Dirichlet >>> numpy.lookfor('dirichlet') Search results for 'dirichlet' ------------------------------ >>> import numpy.random >>> dir(numpy.random) contains dirichlet >>> numpy.random.__all__ does not contain dirichlet. looks like a tiny bug. Josef (this is kind of a duplicate email, but I didn't want it to get lost in the DocWiki discussion)

1 0

osX leopard linker setting
by Thomas Hrabe May 22, 2008

May 22, 2008

Hi, does anybody know the linker settings for python c modules on osX ? i have the original xcode3 tools installed -> gcc 4.0.1 I use '-bundle -flat_namespace' for linking now, but I get ld: can't insert lazy pointers, __dyld section not found for inferred architecture ppc Does anybody know of flags working for the linker? Thank you, Thomas

2 2

Fancier indexing
by Kevin Jacobs <jacobs＠bioinformed.com> May 22, 2008

May 22, 2008

After poking around for a bit, I was wondering if there was a faster method for the following: # Array of index values 0..n items = numpy.array([0,3,2,1,4,2],dtype=int) # Count the number of occurrences of each index counts = numpy.zeros(5, dtype=int) for i in items: counts[i] += 1 In my real code, 'items' contain up to a million values and this loop will be in a performance critical area of code. If there is no simple solution, I can trivially code this using the C-API. Thanks, -Kevin

4 7

ANN: NumPy/SciPy Documentation Marathon 2008
by Joe Harrington May 22, 2008

May 22, 2008

NUMPY/SCIPY DOCUMENTATION MARATHON 2008 As we all know, the state of the numpy and scipy reference documentation (aka the docstrings) is best described as "incomplete". Most functions have docstrings shorter than 5 lines, whereas our competitors IDL and Matlab usually have a concise and well-written page or two per function. The (wonderful) categorized list of functions is very new and isn't included in the package yet. There isn't even a "Getting Started"-type of document you can hand a new user so they can dive right in. Documentation tools are limited to plain-text paginators, while our competition enjoys HTML-based documents with formulae, images, search capability, and cross linking. Tales of woe abound. A university class switched to Numpy and got hopelessly bogged down because students couldn't find out how to call the functions. A developer looked something up while giving a presentation and the words "Blah, Blah, Blah" stared down at the audience in response. To head off another pedagogical meltdown, the University of Central Florida has hired Stefan van der Walt full time to coordinate a community documentation effort to write reference documentation and tools. The project starts now and continues through the summer. The goals: 1. Produce complete docstrings for all numpy functions and as much of scipy as possible, 2. Produce an 8-15 page Getting Started tutorial that is not discipline-specific, 3. Write reference sections on topics in numpy, such as slicing and the use principles of the modules, 4. Complete a first edition, in both PDF and HTML, of a NumPy Reference Manual, and 5. Check everything into the sources by 1 August 2008 so that the Packaging Team can cut a release and have it available in time for Fall 2008 classes. Even Stefan could not document the hundreds of functions that need it by himself, and in any case such a large contribution requires community review. To make it easy for everyone to contribute, Pauli Virtanen and Emmanuelle Guillart have provided a wiki system for editing reference documentation. The idea was developed by Fernando Perez, Stefan, and Gael Varoquaux. We encourage community members to write, review, and proofread reference pages on this wiki. Stefan will check updates into the sources roughly weekly. Near the end of the project, we will put these wiki pages through a vetting process and then check them into the sources a final time for a release hopefully to occur in early August. Meanwhile, Perry Greenfield has taken the lead on on task 3, writing reference docs for things that currently don't have docstrings, such as basic concepts like slicing. We have proposed two small extensions to the current docstring format, for images (to be used sparingly) and indexing. These appear in updated versions of the doc standard, which are linked from the wiki frontpage. Please take a look and comment on these if you like. All docstrings will remain readable in plain text, but we are now generating a full reference guide in PDF and HTML (you guessed it, linked from the wiki). These are searchable formats. There are several ways you can help: 1. Write some docstrings on the wiki! Many people can do this, many more than can write code for the package itself. However, you must know numpy, the function group, and the function you are writing well. You should be familiar with the concept of a reference page and write in that concise style. We'll do tutorial docs in another project at a later date. See the instructions on the wiki for guidelines and format. 2. Review others' docstrings and leave comments on their wiki pages. 3. Proofread docstrings. Make sure they are correct, complete, and concise. Fix grammar. 4. Write examples ("doctests"). Even if you are not a top-notch English writer, you can help by producing a code snippet of a few lines that demonstrates a function. It is fine for them to go into the docstring templates before the actual text. 5. Write a new help function that optionally produces ASCII or points the user's PDF or HTML reader to the right page (either local or global). 6. If you are in a position to hire someone, such as a knowledgeable student or short-term consultant, hire them to work on the tasks above for the summer. We can provide supervision to them or guidance to you if you like. The home for this project is here: http://scipy.org/Developer_Zone/DocMarathon2008 This is not a sprint. It is a marathon, and this time we are going to finish. We hope you will join us! --jh-- and Stefan and Perry and Pauli and Emmanuelle...and you! Joe Harrington Stefan van der Walt Perry Greenfield Pauli Virtanen Emmanuelle Guillart ...and you!

12 34