Mailman 3 March 2010 - SciPy-User

scipy.signal not installing (mac)
by Brian Larsen 25 Mar '10

25 Mar '10

Hello all, I didn't see an answer on this in my google searching. I installed scipy from the latest svn and for some reason scipy.signal does not exist (or isnt installed). Files like scipy/scipy/signal/waveforms.py do seem to exist but aren't being compiled. Is there a way to tell if I am missing prerequisites or something? Sorry I am not sure what more info to pass along, aside from that I followed http://blog.hyperjeff.net/?p=160 to do my install so I have numpy, fftw, etc installed. Thanks much, Brian -- Brian A. Larsen Space Science and Applications Group ISR-1, MS D466 Los Alamos National Laboratory Los Alamos, NM 87545 USA (For overnight add: SM-30, Bikini Atoll Road) Phone: TBD FAX: 505-665-7395 email: balarsen(a)lanl.gov Correspondence / Technical data or Software Publicly Available

2 1

2d convolution
by Nico Schlömer 24 Mar '10

24 Mar '10

Hi, I'm trying to compute the the convolution if s 2D array, and I see that there are several ways in SciPy to do that. As the original data C and the kernel R are about the same size in my case, I'd profit from an FFT-based implementation, which I see right now is given by scipy.signal.fftconvolve( C, R, mode='same' ) and also scipy.stsci.convolve.convolve2d( data=C, kernel=R, mode='constant', cval=0.0, fft=1 ) The second method is much faster than the first, and as far as I can see it would spit out the the same results. Now, when the kernel is actually larger than the data, the resulting array would have the shape of the kernel. Is there any way to restrict the computations to the size of the data? At first, I thought that was what "mode='same'"" is for. I tried cutting the extra data off of the resulting array, but I'm not quite sure which is the part that I would like to get rid of. Any hints here? Cheers, Nico

6 13

distributed optimisation package
by Dan Goodman 24 Mar '10

24 Mar '10

Hi all, Disclaimer: This isn't strictly about Scipy, but I think the people here are probably the most likely to have something interesting to say about this, as it's about scientific computing with Python. Myself and a student in our lab have written a package that we're using internally to distribute optimisation problems over several CPUs (or GPUs using PyCUDA) over the small cluster of computers in our lab using multiprocessing. It has two parts: some code for distributing work over several computers and over several CPUs/GPUs on those computers. There's no load balancing, we assume that each piece of work is roughly the same, however it does handle using shared memory arrays with numpy automatically (recently discussed on this list and the numpy one). It can use named pipes or IP for communications (we had to use named pipes because of the firewall setup in our lab). The other part of the code is a framework for optimisation routines that can be used with this. So far we've implemented a particle swarm optimisation algorithm which is ideal for this sort of distributed processing because it's very local and so splitting amongst several machines is quite nice. We're also thinking about implementing a genetic algorithm in the same framework. Anyway, getting to the point. We've written this largely for our own use, but it struck us that this might be of interest more widely. The code is fairly simple (a few hundred lines), but there are maybe some non-obvious things in there (at least for some people). We're thinking about releasing it as an independent package, but there's some work involved in that, so we were just looking to see if there was enough interest to make it worth the effort really. Also, it's very likely that our code is less than perfect, so if anyone would be interested in working on it that could be good too. Thanks for any comments! Dan

3 3

PSF GSoC 2010 (Py3K focus)
by Jarrod Millman 23 Mar '10

23 Mar '10

Hello, Given the interest in participating in the GSoC this summer, I am forwarding a very interesting email from Titus Brown. If you are interested in doing a GSoC or mentoring, please read his email carefully. Basically, the PSF will be focuing on Py3K-related projects. Given Pauli's work on Py3K support for NumPy, I think we might be in a good position to move forward on porting the rest of our stack to Py3K. So we should focus on projects to: 1. finish porting and testing NumPy with Py3K 2. port and test SciPy with Py3K 3. port and test matplotlib with Py3K 4. port and test ipython with Py3K 5. etc. Given the PSF's stated emphasis this year, it probably doesn't make sense to pursue any non-Py3K projects. Jarrod ---------- Forwarded message ---------- From: C. Titus Brown <ctb(a)msu.edu> Date: Tue, Mar 2, 2010 at 6:12 AM Subject: [SoC2009-mentors] [ctb(a)msu.edu: GSoC 2010 - it's on!] To: soc2009-mentors(a)python.org ----- Forwarded message from "C. Titus Brown" <ctb(a)msu.edu> ----- Date: Wed, 24 Feb 2010 12:54:52 -0800 From: "C. Titus Brown" <ctb(a)msu.edu> To: psf-members(a)python.org Cc: gsoc2010-mentors(a)python.org Subject: GSoC 2010 - it's on! Hi all, it's that time of year again, and Google has decided to run the Google Summer of Code again! http://groups.google.com/group/google-summer-of-code-discuss/browse_thread/… http://socghop.appspot.com/ Arc Riley has stepped up to run it for the PSF again this year, and I'm backstopping him. If you are interested in mentoring or kibbitzing on those who are, please sign up for the soc2010-mentors mailing list here, http://mail.python.org/mailman/listinfo/soc2010-mentors This year we're proposing to solicit and prioritize applications for Python 3.x -- 3K tools, porting old projects, etc. Python 2.x projects will be a distinct second. There will be no "core" category this year, although obviously if someone on one of the core teams wants to push a project it'll help! If you have an idea for a project, please send it to the -mentors list and add it to the wiki at http://wiki.python.org/moin/SummerOfCode/2010 We're also going to change a few things up to make it more useful to the PSF. Specifically, - the foundation is going to *require* 1 blog post/wk from each student. - we're going to hire an administrative assistant to monitor the students. - the student application process will be a bit more rigorous and job-app like; the Django SF has been doing this for at least one round and they claim that it results in much better and more serious students. - we'll be focusing on student quality more than on project egalitarianism. If project X can recruit three fantastic students to one fantastic and one mediocre student for project Y, then project X gets three and project Y gets one. The hope is that this will make the GSoC much more useful for Python than it has been in the past. Arc will be posting something to the www.python.org site and python-announce soon, too. Followups to soc2010-mentors. cheers, --titus -- C. Titus Brown, ctb(a)msu.edu ----- End forwarded message -----

5 6

Choosing sub array based on values in a column
by Vincent Davis 23 Mar '10

23 Mar '10

I feel kinda stupid as I think this must be easier than I am making it. Below my question you will see an answer I got to a question that I thought I would be able to complete the last steps my self I was wrong :) So if I have an array Y = np.rec.array([(1.0, 0.0, 3.0, 3.5), (0.0, 0.0, 6.0, 6.5), (1.0, 1.0, 9.0, 9.5)], dtype=[('var1', '<f8'), ('var2', '<f8'), ('var3', '<f8'), ('var4', '<f8')]) do this works like I would expect Y[['var3','var4']][Y['var1']==1] >>>array([(3.0, 3.5), (9.0, 9.5)], dtype=[('var3', '<f8'), ('var4', '<f8')]) But I would like to do this, >>> Y[['var3','var4']][Y['var1']==0 and Y['var2']==0] Traceback (most recent call last): File "<string>", line 1, in <fragment> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() I tried some any() and all() combination but nothing worked. What s the right way to go about this? Thanks Vincent This answer was received on the mail list from Skipper Seabold If you have a rec array Y = np.rec.array([(1.0, 2.0, 3.0), (4.0, 5.0, 6.0), (7.0, 8.0, 9.0)], dtype=[('var1', ''var2','var3']) You can access the rows like, Y[['var1','var2','var3']] Note the list within []. If you want a "normal" array, I like this way that Pierre recently pointed out. 3 is the number of columns, and it fills in the number of rows. Y[['var1','var2','var3']].view((float,3)) note the tuple for the view, if they're all floats. Taking a view might not work if var# have different types, like ints and floats. If you want the mean of the rows (mean over the columns axis = 1) Y[['var1','var2','var3']].view((float,3)).mean(1) Some shortcuts. Y[list(Y.dtype.names)].view((float,len(Y.dtype))).mean(1) Also, for now, the columns will given back to you in the order they're in in the array no matter which way you ask for them. A patch has been submitted for returning the order you ask that I hope gets picked up... *Vincent Davis 720-301-3003 * vincent(a)vincentdavis.net my blog <http://vincentdavis.net> | LinkedIn<http://www.linkedin.com/in/vincentdavis>

2 1

Remove the need for the loop
by Martin De Kauwe 23 Mar '10

23 Mar '10

Hi, I would like to transfer the contents of a 1D array to a 2D array based on two arrays listing the relevant locations (the arrays rows and cols). Ideally I would like to solve my problem without the need for the current for loop that it quite slow due to the array size. Any help doing this more elegantly would be much appreciated. What I currently have... for fname in glob.glob(os.path.join(path, 'Snowf_P3_*.nc')): year = int(fname.split('/')[9].split('_')[2].split('.')[0][0:4]) if year >= year_jules_run_starts and year <= year_jules_run_ends: month = int(fname.split('/')[9].split('_')[2].split('.')[0][4:6]) f = Dataset(fname, 'r') snow = f.variables['Snowf'] landpts = len(f.dimensions['land']) # sum monthly vals gswp_snow = (snow[:] * 1000.0 * (3.0 * 60.0 * 60.0) / 1000.0).sum(axis=0) f.close() # fill up the holding array, this is the bit where I would like to avoid the for loop for point in xrange(numpts): snowfall[month-1,(180-(rows[point]-1)),cols[point]-1] += gswp_snow[point] Many Thanks Martin

2 3

Author SciPy Books- Packt Publishing.
by Kshipra Singh 23 Mar '10

23 Mar '10

Hi All, I am writing to you for Packt Publishing, the publishers of computer related books. We are planning to extend our catalogue of books based on Scientific Computing Tools and are currently inviting authors interested in writing for Packt. This doesn't need any previous writing experience. Just an expert knowledge of your subject and a passion to share it with others is all that we require. So, if you love SciPy and are interested in authoring a book, here's an opportunity knocking your door. Write to us with your SciPy book ideas at author(a)packtpub.com. Even if you don't have a book idea and are simply interested in authoring a book, we are still keen to hear from you. More details about the opportunity are available at: http://authors.packtpub.com/content/scientific-compting-tools-write-Packt Thanks Kshipra Singh Author Relationship Manager Packt Publishing www.PacktPub.com Skype: kshiprasingh15 Twitter: http://twitter.com/kshipras Interested in becoming an author? Visit http://authors.packtpub.com for all the information you need about writing for Packt.

2 1

Opportunity to "plug" scipy
by David Goldsmith 23 Mar '10

23 Mar '10

Hi, folks. In trying to drum up funding for the doc project, one of the orgs I contacted - The Society for Imaging Informatics in Medicine - invited us to present at the SIIM 2010 open source plugfest at their Annual Meeting in June. If people are interested, they should probably coordinate efforts, i.e., assemble one team and one presentation/booth. Anyway, here's the URL: http://www.siimweb.org/open and here's the contact info: Anna Marie Mason, MS,CAE Executive Director Society for Imaging Informatics in Medicine (SIIM) 19440 Golf Vista Plaza, Suite 330 Leesburg, VA 20176 703-723-0432 x 306 amason(a)siimweb.org "Tell her David Goldsmith sent you," as they say. (I'll email her separately to let her know that someones may be emailing her separately about this.) DG

3 5

using sunperf for compiling scipy and numpy
by Wolfgang Kerzendorf 22 Mar '10

22 Mar '10

Dear all, I would like to use the sunperf libraries when compiling scipy and numpy. I tried using setupscons.py which seems to check from SUNPERF libraries, but it didnt recognize where mine are: here is a listing of /pkg/linux/SS12/sunstudio12.1 (thats where the sunperf library lives): wkerzend@mosura:/home/wkerzend>ls /pkg/linux/SS12/sunstudio12.1/lib/ CCios/ libdbx_agent.so@ libsunperf.so.3@ amd64/ libfcollector.so@ libtha.so@ collector.jar@ libfsu.so@ libtha.so.1@ dbxrc@ libfsu.so.1@ locale/ debugging.so@ libfui.so@ make.rules@ er.rc@ libfui.so.1@ rw7/ libblacs_openmpi.so@ librtc.so@ sse2/ libblacs_openmpi.so.1@ libscalapack.so@ stlport4/ libcollectorAPI.so@ libscalapack.so.1@ svr4.make.rules@ libcollectorAPI.so.1@ libsunperf.so@ tools_svc_mgr@ ------------------------------- I tried to specify this directory in sites.cfg, but I still get the following errors: Checking if g77 needs dummy main - MAIN__. Checking g77 name mangling - '_', '', lower-case. Checking g77 C compatibility runtime ...-L/usr/lib/gcc/x86_64-redhat-linux/3.4.6 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.6 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../.. -L/lib/../lib64 -L/usr/lib/../lib64 -lfrtbegin -lg2c -lm Checking MKL ... Failed (could not check header(s) : check config.log in build/scons/scipy/integrate for more details) Checking ATLAS ... Failed (could not check header(s) : check config.log in build/scons/scipy/integrate for more details) Checking SUNPERF ... Failed (could not check symbol cblas_sgemm : check config.log in build/scons/scipy/integrate for more details)) Checking Generic BLAS ... yes Checking for BLAS (Generic BLAS) ... Failed: BLAS (Generic BLAS) test could not be linked and run Exception: Could not find F77 BLAS, needed for integrate package: File "/priv/manana1/wkerzend/install_dir/scipy-0.7.1/scipy/integrate/SConstruct", line 2: GetInitEnvironment(ARGUMENTS).DistutilsSConscript('SConscript') File "/home/wkerzend/python_coala/numscons-0.10.1-py2.6.egg/numscons/core/numpyenv.py", line 108: build_dir = '$build_dir', src_dir = '$src_dir') File "/priv/manana1/wkerzend/python_coala/numscons-0.10.1-py2.6.egg/numscons/scons-local/scons-local-1.2.0/SCons/Script/SConscript.py", line 549: return apply(_SConscript, [self.fs,] + files, subst_kw) File "/priv/manana1/wkerzend/python_coala/numscons-0.10.1-py2.6.egg/numscons/scons-local/scons-local-1.2.0/SCons/Script/SConscript.py", line 259: exec _file_ in call_stack[-1].globals File "/priv/manana1/wkerzend/install_dir/scipy-0.7.1/build/scons/scipy/integrate/SConscript", line 15: raise Exception("Could not find F77 BLAS, needed for integrate package") error: Error while executing scons command. See above for more information. If you think it is a problem in numscons, you can also try executing the scons command with --log-level option for more detailed output of what numscons is doing, for example --log-level=0; the lowest the level is, the more detailed the output it.----- -------------------------- any help is appreciated Wolfgang

1 0

accessing a set of columns from a recarray
by Vincent Davis 22 Mar '10

22 Mar '10

I have a few things I am trying to understand. I have a record array and and I list of columns for which I would like to get the row means. My current solution is to iterate though my list of column names and make a new "normal" array. I would like to do something like np.mean(A['x','y','z']) It would be great to append the results as an additional column to the array, but I think I have seen how to do this. Next, to make it a little more interesting I have a column that has values 0-14. I would like to get the mean of a row conditional on the value of this column. Ideally I get a new array with columns 0-14 with missing values where needed. I hope this is clear, please ask for clarification if not. *Vincent Davis 720-301-3003 * vincent(a)vincentdavis.net my blog <http://vincentdavis.net> | LinkedIn<http://www.linkedin.com/in/vincentdavis>

2 5