Mailman 3 June 2011 - SciPy-User

Spline interpolation using splrep,splev
by Jaidev Deshpande June 27, 2011

June 27, 2011

Hi I'm using scipy.interpolate.splrep and splev to construct cubic splines. These functions are part of the 'fitpack' module, and they raise an error when the number of nodes to be interpolated are less than the order of the spline. For instance, these functions can't produce a cubic spline for less than four nodes. Why does this happen? Surely, there can be two cubical functions between three points that follow the necessary end-point restraints!

2 2

tool for running simulations
by Dan Goodman June 26, 2011

June 26, 2011

Hi all, I have an idea for a tool that might be useful for people writing and running scientific simulations. It might be that something like this already exists, in which case if anyone has any good suggestions please let me know! Otherwise, I might have a go at writing something, in which case any feature requests or useful advice would be great. Basically, the situation I keep finding myself in, and I assume many others do so too, is that I have some rather complicated code to set up and run a simulation (in my case, computational neuroscience simulations). I typically want to run each simulation many times, possibly with different parameters, and then do some averaging or more complicated analysis at the end. Usually these simulations take around 1h to 1 week to run, depending on what I'm doing and assuming I'm using multiple computers/CPUs to do it. The issue is that I want to be able to run my code on several computers at once, and have the results available on all the computers. I've been coming up with all sorts of annoying ways to do this, for example having each computer generate one file with a unique name, and then merging them afterwards - but this is quite tedious. What I imagine is a tool that does something like this: * Run a server process on each of several computers, that controls file access (this avoids any issues with contention). One computer is the master and if the other ones want to read or write a file then it is transferred to the master. Some files might want to be cached/mirrored on each computer for faster access (typically for read only files in my case). * Use a nice file format like HDF5 that allows fast access, store metadata along with your data, and for which there are good tools to browse the data. This is important because as you change your simulation code, you might want to weed out some old data based on the metadata, but not have to recompute everything, etc. * Allows you to store multiple data entries (something like tables in HDF5 I guess) and then select out specific ones for analysis. * Allows you to use function cacheing. For example, I often have the situation that I have a computation that takes about 10m for each set of parameter values that is then used in several simulations. I'd like these to be automatically cached (maybe based on a hash of the arguments to the function). As far as I can tell, there are tools to do each of the things above, but nothing to combine them all together simply. For example, there are lots of tools for distributed filesystems, for HDF5 and for function value cacheing, but is there something that when you call a function with some particular values, creates a hash, checks in a distributed version of HDF5 for that hash value and then either returns the value or stores it in the HDF5 file with the relevant metadata (maybe the values of the arguments and not just the hash). Since all the tools are basically already there, I don't think this should take too long to write (maybe just a few days), but could be useful for lots of people because at the moment it requires mastering quite a few different tools and writing code to glue them together. The key thing is to choose the best tools for the job and take the right approach, so any ideas for that? Or maybe it's already been done? Dan

7 13

ANN: Visvis version 1.5 - The object oriented approach to visualization
by Almar Klein June 26, 2011

June 26, 2011

Hi all, On behalf of the visvis development team, I'm pleased to announce the latest release of visvis! We have a new backend, we've done improvements to the Mesh class, we've done a lot of work on the cameras, and we've got a fun flight-sim style camera <http://code.google.com/p/visvis/wiki/example_meshes>. And much more... website: http://code.google.com/p/visvis/ Discussion group: http://groups.google.com/group/visvis/ Documentation: http://code.google.com/p/visvis/wiki/Visvis_basics Release notes: http://code.google.com/p/visvis/wiki/releaseNotes What is visvis? --------------- Visvis is a pure Python library for visualization of 1D to 4D data in an object oriented way. Essentially, visvis is an object oriented layer of Python on top of OpenGl, thereby combining the power of OpenGl with the usability of Python. A Matlab-like interface in the form of a set of functions allows easy creation of objects (e.g. plot(), imshow(), volshow(), surf()). Visvis with Reinteract ---------------------- Robert Schroll has been working to enable using visvis in interact:http://www.reinteract.org/trac/<http://www.google.com/url?sa=D&q=http://www.reinteract.org/trac/>. See this discussion: http://groups.google.com/group/visvis/browse_thread/thread/bfe129a265453140 Most notable changes -------------------- * Visvis now also has a GTK backend. * The cameras are now more explicitly exposed to the user, making it easier for the user to set a new camera, for example to use a single camera for multiple axes. * Reimplemented the FlyCamera so it is much easier to control. Some gaming experience will still help though :) see the meshes example<http://code.google.com/p/visvis/wiki/example_meshes>for a movie. * The 3D camera now also has a perspective view. Use shift+RMB to interactively change the field of view. * A mesh() convenience funcion was added. The signature of the Mesh class was changed to make it more intuitive. The old signature if still supported but may be removed in future versions. * Visvis now has a settings object, which can be used to change user-specific defaults, such as the preferred backend and the size of new figures. * 3D color data can now be rendered. * Implemented volshow2(), which displays a volume using three 2D slices, which can be moved interactively through the volume. Visvis automatically falls back to this way of visualization if 3D volume rendering is not possible on the client hardware. (see release notes <http://code.google.com/p/visvis/wiki/releaseNotes> for a more detailed list) Regards, Almar

1 1

Re: [SciPy-User] iterative matrix methods seem slow
by Richard Sharp June 25, 2011

June 25, 2011

Thanks Pauli, > I don't see you passing in a preconditioner here -- it goes in via > the M= parameter of gmres. Right, I had one in there to begin with, but removed it later since it seemed to slow down the convergence. > On preconditioners: If you want "automatic" preconditioners, you can > try the following: > > http://docs.scipy.org/doc/scipy/reference/sparse.linalg.html#scipy.sparse.l… Thanks for this. Using the incomplete as follows LU I was able to cut the iterative runtimes from 1700s to 30s P = scipy.sparse.linalg.spilu(matrix, drop_tol=1e-5) M_x = lambda x: P.solve(x) M = scipy.sparse.linalg.LinearOperator((n * m, n * m), M_x) result = scipy.sparse.linalg.lgmres(matrix, b, tol=1e-4, M=M)[0] But the spilu factors in about 3s, solves in 0.1s and seems to give a result that's very close to my continuous solution, so I'm using that now with no memory or runtime problems: P = scipy.sparse.linalg.spilu(matrix, drop_tol=1e-5) result = P.solve(b) Thanks for the guidance and help! Rich

3 5

How to fit parameters of beta distribution?
by John Reid June 25, 2011

June 25, 2011

Hi, I can see a instancemethod scipy.stats.beta.fit. I can't work out from the docs how to use it. From trial & error I got the following: In [12]: scipy.stats.beta.fit([.5]) Out[12]: array([ 1.87795851e+00, 1.81444871e-01, 2.39026963e-04, 4.99760973e-01]) What are the 4 values output by the method? Thanks, John.

3 7

[ANN] Euroscipy 2011 - registration now open
by Emmanuelle Gouillart June 25, 2011

June 25, 2011

Dear all, After some delay due to technical problems, registration for Euroscipy 2011 is now open! Please go to http://www.euroscipy.org/conference/euroscipy2011, login to your account if you have one, or create a new account (right side of the upper banner of the Euroscipy webpage), then click on the registration panel on the sidebar on the left. Choose the items (tutorials and/or conference) that you wish to attend to, and proceed to pay for your items by credit card. Early-bird registration fees are as follows: * for academia, and self-employed persons: 50 euros for the tutorials (two days), and 50 euros for the conference (two days). * for corporate participants: 100 euros for tutorials (two days), and 100 euros for the conference (two days). For all days, lunch will be catered for and is included in the fee. Early-bird fees apply until July 24th; prices will be doubled for late registration. Book early to take advantage of the early-bird prices! As there is a limited number of seats in the lecture halls, registrations shall be accepted on a first-come first-serve basis. All questions regarding registration should be addressed exclusively to org-team(a)lists.euroscipy.org Looking forward to seeing you at Euroscipy! The organizers

1 0

[ANN] Numerical integration with guaranteed precision by interalg
by Dmitrey June 24, 2011

June 24, 2011

Hi all, some ideas implemented in the solver interalg (INTERval ALGorithm) that already turn out to be more effective than its competitors in numerical optimization (benchmark) appears to be extremely effective in numerical integration with guaranteed precision. Here are some examples where interalg works perfectly while scipy.integrate solvers fail to solve the problems and lie about obtained residual: * 1-D (vs scipy.integrate quad) > * 2-D (vs scipy.integrate dblquad) > * 3-D (vs scipy.integrate tplquad) > see http://openopt.org/IP for more details. Regards, D.

1 0

scipy.stats.mstats.linregress bug?
by Andreas June 24, 2011

June 24, 2011

Hi there, I found a peculiarity in scipy.stats.mstats.linregress. It doesn't give the same results as scipy.stats.linregress on a numpy.array (see below). Is this a bug or a feature? Cheers, Andreas. In [1]: import numpy In [2]: import scipy In [3]: numpy.__version__ Out[3]: '1.5.1' In [4]: scipy.__version__ Out[4]: '0.9.0' In [5]: import scipy.stats In [6]: import scipy.stats.mstats In [7]: In [8]: data = numpy.array([5.28511864e+15, 4.87487615e+15, 5.56279671e+15, ...: 4.72866954e+15, 5.08328669e+15, 4.06702155e+15, ...: 4.99224913e+15, 6.29268616e+15, 9.16149273e+15, ...: 5.47843819e+15, 5.86477063e+15, 6.96145031e+15, ...: 7.25725121e+15, 6.52453707e+15, 6.01766151e+15]) In [9]: In [10]: x = numpy.arange(data.shape[0]) In [11]: In [12]: scipy.stats.linregress(x,data) Out[12]: (149163178178571.41, 4832678167416667.0, 0.53093100793359638, 0.041709303490156953, 66031024254034.961) In [13]: scipy.stats.mstats.linregress(x,data) Out[13]: (149163178178571.44, 4832678167416667.0, 0.53093100793359627, masked_array(data = 0.0417093034902, mask = False, fill_value = 1e+20) , 1028615575651548.9)

3 8

Simulate hybrid dynamical system
by Gus Ishere June 24, 2011

June 24, 2011

Hi all, I am looking to simulate a hybrid dynamical system, containing both continuous-time and discrete-time models. I'm familiar with the ODE integrators in SciPy and have seen a discrete event simulator for Python. Is there a framework or example of simulating a hybrid model? Ideally there would be some tool which would compile the model so that the simulation isn't scripted, similar to PyDST. Thanks! Gustavo

2 1

Differentiate raw data
by Andy June 24, 2011

June 24, 2011

Hi People Is there an easy way to use scipy to numerically differentiate a raw data series? I'm trying to do some Thermogravimetric analysis (http://en.wikipedia.org/wiki/Thermogravimetric_analysis), have data in the form of x,y and and would appreciate some wisdom to help differentiate the data so I could plot(using matplotlib) x, dy/dx. There isn't any mathematical fit to the data. Andy

1 0