Mailman 3 July 2008 - NumPy-Discussion

Documentation: glossary and refactoring
by Stéfan van der Walt 17 Jul '08

17 Jul '08

Hi all, We recenty added a framework for topical documentation to NumPy, which is visible as `numpy.doc`. The content is here: http://sd-2116.dedibox.fr/pydocweb/doc/numpy.doc/ I think new users will find the glossary especially useful: http://sd-2116.dedibox.fr/pydocweb/doc/numpy.doc.reference.glossary/ For technical terms and explanation, we also have `numpy.doc.jargon` (which is still empty; but feel free to break the ice!). To those participating: if you find yourself explaining a common concept or term over and over, consider adding it to the glossary or jargon files instead. Which brings me to refactoring: as with code, we try to keep functionality documented in one place. Please *refer* to other docstrings, instead of copying their content. For example, `argsort` refers to `sort`, instead of re-explaining the different sorting methods. Contributors are now listed on the stats page: http://sd-2116.dedibox.fr/pydocweb/stats/ Again, thanks for all your help; you make a mammoth task look easy! Your docstrings will be included in the 1.2 NumPy release (the first release candidate which should appear around 4 August). Regards Stéfan

1 0

proposal: add a header and footer function to numpy.savetxt
by Tim Michelsen 17 Jul '08

17 Jul '08

Hello, sometime scripts and programs create a lot of data output. For the programmer and also others not involved in the scripting but in the evaluation of the output it would be very nice the output files could be prepended with a file header describing what is written in the columns below and to append a footer. A good example has been developed by the scipy.scikits.timeseries developers: http://scipy.org/scipy/scikits/wiki/TimeSeries#Parameters These formatting flags are a convenient way to save additional meta information. E. g. of it is important to state the physical units of the data saved. I would be happy if such a thing could be added to np.savetxt(). What is the current common way to save a header above the saved ascii array? Kind regaed, Timmie

1 0

Testing -heads up with #random
by Fernando Perez 17 Jul '08

17 Jul '08

Hi Alan, I was trying to reuse your #random checker for ipython but kept running into problems. Is it working for you in numpy in actual code? Because in the entire SVN tree I only see it mentioned here: maqroll[numpy]> grin #random ./numpy/testing/nosetester.py: 43 : if "#random" in want: 67 : # "#random" directive to allow executing a command while ignoring its 375 : # try the #random directive on the output line 379 : <BadExample object at 0x084D05AC> #random: may vary on your system maqroll[numpy]> I'm asking because I suspect it is NOT working for numpy. The reason is some really nasty, silent exception trapping being done by nose. In nose's loadTestsFromModule, which you've overridden to include: yield NumpyDocTestCase(test, optionflags=optionflags, checker=NumpyDoctestOutputChecker()) it's likely that this line can cause an exception (at least it was doing it for me in ipython, because this class inherits from npd but tries to directly call __init__ from doctest.DocTestCase). Unfortunately, nose will silently swallow *any* exception there, simply ignoring your tests and not even telling you what happened. Very, very annoying. You can see if you have an exception by doing something like try: dt = DocTestCase(test, optionflags=optionflags, checker=checker) except: from IPython import ultraTB ultraTB.AutoFormattedTB()() yield dt to force a traceback printing. Anyway, I mention this because I just wasted a good chunk of time fighting this one for ipython, where I need the #random functionality. It seems it's not used in numpy yet, but I imagine it will soon, and I figured I'd save you some time. Cheers, f

2 1

Buildbot failures since r5443
by Pauli Virtanen 17 Jul '08

17 Jul '08

Hi, Since r5443 the Sparc buildbots show a "Bus error" in the test phase: http://buildbot.scipy.org/builders/Linux_SPARC_64_Debian/ builds/102/steps/shell_2/logs/stdio while the one on FreeBSD-64 passes. -- Pauli Virtanen

4 4

Ticket #837
by Pauli Virtanen 17 Jul '08

17 Jul '08

http://scipy.org/scipy/numpy/ticket/837 Infinite loop in fromfile and fromstring with sep=' ' and malformed input. I committed a fix to trunk. Does this need a 1.1.1 backport? -- Pauli Virtanen

2 3

Numpy Advanced Indexing Question
by Jack.Cook＠shell.com 17 Jul '08

17 Jul '08

Greetings, I have an I,J,K 3D volume of amplitude values at regularly sampled time intervals. I have an I,J 2D slice which contains a time (K) value at each I, J location. What I would like to do is extract a subvolume at a constant +/- K window around the slice. Is there an easy way to do this using advanced indexing or some other method? Thanks in advanced for your help. - Jack Kind Regards, Jack Cook Jack Cook

4 8

NumPy date/time types and the resolution concept
by Francesc Alted 17 Jul '08

17 Jul '08

Hi, Before giving more thought to the new proposal of the date/time types for NumPy based in the concept of 'resolution', I'd like to gather more feedback on your opinions about this. After pondering about the opinions about the first proposal, the idea we are incubating is to complement the ``datetime64`` with a 'resolution' metainfo. The ``datetime64`` will still be based on a int64 type, but the meaning of the 'ticks' would depend on a 'resolution' property. This is best seen with an example: In [21]: numpy.arange(3, dtype=numpy.dtype('datetime64', 'sec')) Out [21]: [1970-01-01T00:00:00, 1970-01-01T00:00:01, 1970-01-01T00:00:02] In [22]: numpy.arange(3, dtype=numpy.dtype('datetime64', 'hour')) Out [22]: [1970-01-01T00, 1970-01-01T01, 1970-01-01T02] i.e. the 'resolution' gives the actual meaning to the 'int64' counter. The advantage of this abstraction is that the user can easily choose the scale of resolution that better fits his need. I'm thinking in providing the next resolutions: ["femtosec", "picosec", "nanosec", "microsec", "millisec", "sec", "min", "hour", "month", "year"] Also, together with the absolute ``datetime64`` one can have a relative counterpart, say, ``timedelta64`` that also supports the notion of 'resolution'. Between both one would cover the needs for most uses, while providing the user with a lot of flexibility, IMO. We very much prefer this new approach than the stated in our first proposal. Now, it comes the tricky part: how to integrate the notion of 'resolution' with the 'dtype' data type factory of NumPy? Well, we have thought a couple of possibilities. 1) Using the NumPy 'dtype' factory: nanoabs = numpy.dtype('datetime64', resolution="nanosec") nanorel = numpy.dtype('timedelta64', resolution="nanosec") 2) Extending the string notation by using the '[]' square brackets: nanoabs = numpy.dtype('datetime64[nanosec]') # long notation nanoabs = numpy.dtype('T[nanosec]') # short notation nanorel = numpy.dtype('timedelta64[nanosec]') # long notation nanorel = numpy.dtype('t[nanosec]') # short notation With these building blocks, one may obtain more complex dtype structures easily. Now, the question is: would that proposal enter in conflict with the spirit of the current 'dtype' factory? And another important one, would that complicate the implementation too much? If the answer to the both previous questions is 'no', then we will study this more and provide another proposal based on this. BTW, I suppose that the best candidate to answer these would be Travis O., but if anybody feels brave enough ;-) please go ahead and give your advice. Cheers, -- Francesc Alted

5 16

Python version support for NumPy 1.2
by Alan McIntyre 16 Jul '08

16 Jul '08

Which versions of Python are to be officially supported by NumPy 1.2? I've been working against 2.5 and testing against 2.4 occasionally, but 2.3 still has some issues I need to address (or at least that was the case the last time I checked).

2 1

Monkeypatching vs nose plugin?
by Fernando Perez 16 Jul '08

16 Jul '08

Howdy, In working on the ipython testing machinery, I looked at the numpy nosetester.py file and found that it works by monkeypatching nose itself. I'm curious as to why this approach was taken rather than constructing a plugin object. In general, monkeypatching should be done as a last-resort trick, because it tends to be brittle and can cause bizarre problems to users who after running numpy.test() find that their normal nose-using code starts doing funny things. Any thoughts/insights? Cheers, f

3 7

Ticket review: #843
by Michael Abbott 16 Jul '08

16 Jul '08

I'm reviewing my tickets (seems a good thing to do with a release imminent), and I'll post up each ticket that merits comment as a separate message. Ticket #843 has gone into trunk (commit 5361, oliphant) ... but your editor appears to be introducing hard tabs! Hard tab characters are fortunately relatively rare in numpy source, but my patch has gone in with tabs I didn't use. /me wanders off muttering darkly about the evils of tab characters...

6 11