Mailman 3 November 2008 - NumPy-Discussion

by axis iterator
by Gabriel Gellner 12 Nov '08

12 Nov '08

Something I use a lot is a little generator that iterates over a ndarray by a given axis. I was wondering if this is already built-in to numpy (and not using the apply_along_axis which I find ugly) and if not would there be interest in adding it? the function is just: def by_axis(nobj, axis=0): index_set = [slice(None)]*len(ndobj.shape) for i in xrange(ndobj.shape[axis]): index_set[axis] = i yield ndobj[index_set] and can be just like >>> [sum(x) for x in by_axis(a, 1)] >>> for col in by_axis(a, 1): ... print col I use it when porting R code that uses a lot of apply like logic. I know most numpy functions have the axis argument built in, but when writing my own functions I find this a real time saver. Anyway, if someone can show be a better way I would be overjoyed, or if people like this I can make a ticket on Trac. Gabriel

1 0

Matlib docstring typos
by Ryan May 12 Nov '08

12 Nov '08

Hi, Here's a quick diff to fix some typos in the docstrings for matlib.zeros and matlib.ones. They're causing 2 (of many) failures in the doctests for me on SVN HEAD. Filed in trac as #953 (http://www.scipy.org/scipy/numpy/ticket/953) (Unless someone wants to give me SVN rights for fixing/adding small things like this.) Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma

2 3

numpy, swig and TNT-Arrays
by Rolf Wester 12 Nov '08

12 Nov '08

Hi all, I would like to wrap some C++ classes that use TNT-Arrays. Is it possible to pass numpy arrays to C++ functions that expect TNT-Arrays as function parameter? Does anybody know how the wrappers could be generated using swig? I would be very appreciative for any help. With kind regards Rolf

5 7

f2py call-back and newbie questions
by Dave Lang 12 Nov '08

12 Nov '08

Hello, I apologize for this long listing, but I have a lot of pent-up f2py queries occupying my mind [?] I realize that f2py is a non-commercial service to the community, and am highly impressed that it exists at all. I can only offer in return my own assistance to others (say via this list), after I have crossed some of these bridges and can in turn assist others who venture in. I am new to this list, as well as Python and am trying to use the f2py call-back feature . I am embarking upon a rather large Fortran->Python (80,000 line code) conversion for a dynamics simulation system ( http://home.comcast.net/~GTOSS/), and will be willing to share all knowledge of how this goes. I am converting code by evolving from 100% F90 -> xxx % Python starting at the "results-data-base/post-processing/interpretive-manipulation" side of the existing code. So far, I can successfully create a wrap that allows me to call a Fortran program from Python, but can't successfully do the reverse call-back scheme. I think I have located just about every documentation available, including numerous versions of some, but I have questions that I can't seem to resolve through this documentation. At the end of this request is shown the actual code (it is very simple). My system...... Ubuntu 8.0.4; GNU C++/g95 Compiler (current release) Python 2.5.2; Numpy 1.0.4; f2py 2.4422 The Problem..... The symptom is: Depending upon how I invoke f2py to create the wrap, one of 2 failures occur: 1. At Python execution, Python insists that my arguments to the Fortran subroutine (CBSET) are incorrect (I use 2, it insists on 1), OR, 2. Python accepts my 2 arguments to the Fortran subroutine, and when I execute Python, I end up getting a segmentation fault when I attempt to make the call-back from Fortran. This would normally tell me that likely I have some kind of argument issue (although the call-back routine defn itself has NO arguments...although the "external" attribute routine name does appear as an argument in the subroutine at the INSISTENCE of the g95 compiler). General questions (maybe related to the problem, maybe not).... 1. I never get a "makefile" created by running f2py. Why? 2. I never get a module document (ie. "__xxx__") created by running f2py (although I do get the ".pyf" sig-file). Why? 3. Is the Python "call-back" program code supposed to reside in the Python program that originally makes the Python -> Fortran wrap-connection (I assumed it is)? 4. Does the "XXXX.so" file need to be linked into the Fortran code via a "makefile" operation to successfully "call-back"? 5. Where can I find some simple, but illustrative examples of using f2py to "wrap a Fortran 90 data Class-definition"? I need to understand how the f2py calls would look, and what is manifest with in the Python code to actually lay hands on the elements of the Fortran "derived data types". In general, it would be nice to find a detailed example of: a python program, a corresponding consistent Fortran program that combines to demonstrate the call-back scheme (and that one can type in, and will actually function correctly), and the preferred (ie. correct?) f2py statements that need to be executed to accomplish this (I would be glad to contribute same to the body of knowledge, if I can ever get it to work). I will be most grateful for comments on any or all of this. thanks Dave Lang The Fortran Code: ! PROGAM TO DEMONSTRATE GTOSS FORTRAN -> PYTHON CONNECTION SUBROUTINE CBSET( N, call_back ) !f2py intent(in) N INTEGER N !f2py intent(callback) call_back EXTERNAL call_back ! IDENTIFY THAT FORTRAN IS EXECUTING & IDENTIFY WHY CBSET IS CALLED PRINT *,'Enter CBSET.F, N=', N IF(N .EQ. 1) THEN PRINT *,' in CBSET: this Call originated to test call-back' PRINT *,' in CBSET: Make callback to PYTHON...EXPECT reply' CALL call_back() PRINT *,' in CBSET: Returned from call-back to PYTHON' END IF IF(N .EQ. 2) THEN PRINT *,' in CBSET: Call came from Python to test Py->For' END IF RETURN END *The Python Program:* import os import foo def main(): print 'Hello, pymain is about to call CBSET to test f2py operation' #satisfy arg list for cbset ????? dummy=call_back #test "Python --> Fortran" (nominal wrap) from Python n = 2 foo.cbset(n, dummy) #test "Fortran --> Python" (call-back) scheme as seen from Python n = 1 foo.cbset(n, dummy) # define call-back routine def call_back(): print '...Confirming...."call_back"is entered from CBSET call' # end of call-back defn if __name__ =='__main__': main() NOTE: depending upon how f2py is invoked to create the wrap, a disagreement can be experienced between Python and Fortran as to "how many args the F90 program 'CBSET' really exhibits"; in one case, Python aborts at execution, insisting that "CBSET has exactly 1 argument", The other type event is that Python is happy with the code as shown, but Fortran gets a runtime "Segmentation fault" when the "call_back" is executed. BTW, at compile, Fortran is NEVER happy UNLESS the EXTERNAL routine named "call_back" actually appears as an arg in the CBSET subroutine definition.

2 1

numpy array serialization with JSON
by Simon Palmer 12 Nov '08

12 Nov '08

Does anyone have a recommendation of a library/method for serialization of numpy arrays to and from text (specifically for the purposes of embedding in XML)? I don't want to use pickle or tostring() because my XML has to be consumable across a variety of programming environments. I'm currently using simplejson for other array types (list) but it does not handle numpy arrays. I could go in and out via a list, but that feels a bit resource intensive and wasteful just for serializaton.

6 10

Numpy.test() hangs
by James Philbin 11 Nov '08

11 Nov '08

Python 2.5.2 (r252:60911, Oct 5 2008, 19:29:17) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.3.0.dev6005' >>> numpy.test(verbosity=2) ... test_umath.TestLogAddExp.test_logaddexp_values ... The test hangs at the last line and never progresses further. James

2 12

Bugs in histogram and matplotlib-hist
by Mike Ressler 11 Nov '08

11 Nov '08

I did an update to a Fedora 9 workstation yesterday that included updating numpy to 1.2.0 and matplotlib 0.98.3 (python version is 2.5.1). This seems to have played havoc with some of the histogram plotting we do. I was aware of the histogram changes in 1.2.0, but something doesn't seem to have worked out right. First issue is that the histogram function doesn't like giving "normed" any sort of a value. We get a "'NoneType' object is not iterable" error whenever we tried. I could work around this by editing function_base.py and given the normed keyword in the histogram function a default value of False, then changing the last few lines of histogram to read if not normed: return n, bins else : db = array(np.diff(bins), float) return n/(n*db).sum(), bins (i.e. using normed as a true boolean, rather than a testable variable). That seems to have fixed up that particular issue. I know this isn't quite the right place to report matplotlib errors, but the hist function expects the output of the histogram function, the left edges of the bins, and the frequency values, to be the same length. Histogram now returns all edges (e.g. 100), while the frequencies correspond to the number of bins (e.g. 99). Are these known issues? I didn't see anything in the numpy-1.2.1 release notes to indicate that this was addressed. I guess I should ask whether other people have even seen this? I can submit patches after I think a bit harder about proper fixes, but I'm not a programmer and anything I write should be considered highly suspect. Please advise. Thanks. Mike -- mike.ressler(a)alum.mit.edu

1 0

License review, cont.
by Jarrod Millman 11 Nov '08

11 Nov '08

Now that I have removed all GPL/LGPL code from scipy, I wanted to double check on the licenses of some NumPy code. In particular, 1. FreeBSD license: http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/core/include/nump… http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/core/include/nump… 2. Python license: SafeEval class in http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/lib/utils.py Is there any need to look into getting the authors to re-license their code? The license are pretty liberal (and both look like they are compatible with the revised BSD license), but I thought I'd ask anyway. Should we note the additional licenses in: http://projects.scipy.org/scipy/numpy/browser/trunk/LICENSE.txt I was imagining something like: http://projects.scipy.org/scipy/scipy/browser/trunk/scipy/weave/LICENSE.txt Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

2 1

New ufuncs
by Charles R Harris 11 Nov '08

11 Nov '08

Hi All, I'm thinking of adding some new ufuncs. Some possibilities are - expadd(a,b) = exp(a) + exp(b) -- For numbers stored as logs: - absdiff(a,b) = abs(a - b) -- Useful for forming norms - absmax(a,b) = max(abs(a), abs(b)) - absadd(a,b) = abs(a) + abs(b) -- Useful for L_1 norm and inequalities? I would really like a powadd = abs(a)**p + abs(b)**p, but I can't think of an easy way to pass a variable p that is compatible with the way ufuncs work without going to three variables, something that might not work with the reduce functions. Along these lines I also think it is time to review generalized ufuncs to see what we can do with them. Thoughts? Chuck

8 26

Generalised ufuncs
by Stéfan van der Walt 11 Nov '08

11 Nov '08

Hi everyone, I finally merged the generalised ufunc patches that awaited comment as the gen_ufuncs branch. Please review the changes on trunk, and let me know if problems occur. One of the tests fail on FreeBSD 64: ERROR: Failure: ImportError (/tmp/numpy-buildbot/b12/numpy-install/lib/python2.4/site-packages/numpy/core/umath_tests.so: Undefined symbol "cblas_dgemm") Is there any reason why blasdot should work fine, but that this should be broken? Regards Stéfan

3 3