Mailman 3 August 2012 - SciPy-User

scipy.stats.gaussian_kde ppf?
by Serge Rogov Aug. 30, 2012

Aug. 30, 2012

Hi all, I need to compute confidence intervals from gaussian_kde, but I found that the ppf function is missing. I have implemented my own naïve version that precomputes the cdf for the kde and then performs a search for the desired probability, but it's extremely slow and shows a bit of error when compared to norm.ppf even at large sample sizes. Has anyone had to do something like this before? Kind regards, Serge

3 3

Matlab figure
by Nils Wagner Aug. 30, 2012

Aug. 30, 2012

Hi all, I have received a Matlab figure stored in a matlab file. How can I extract the curves from the file ? Cheers, Nils

4 3

Tensioned Spline interpolation
by gary ruben Aug. 30, 2012

Aug. 30, 2012

Hi, I'm rewriting some IDL code in Python. The original code uses IDL's spline function to interpolate a 1D ungridded data series onto a regular grid. Unfortunately, I can't find a close enough equivalent in scipy. I'm guessing the IDL spline function implements a "tensioned spline." The signature of the IDL spline function is Result = SPLINE(X, Y, T [, Sigma]) where Sigma is 'The amount of "tension" that is applied to the curve. If sigma is close to 0, (e.g., .01), then effectively there is a cubic spline fit. If sigma is large, (e.g., greater than 10), then the fit will be like a polynomial interpolation.' In the code I'm trying to reproduce, the author has set sigma=15 so the result is quite different to a simple cubic spline. Scipy's pchip algorithm gives a similar result, but is unfortunately too slow for my application. Can anyone confirm that IDL's spline is a tensioned spline routine? Does anyone know of a BSD-licensed Python module that implements a tensioned spline. Failing that any suggestions for a good routine I should look at wrapping (maybe Alan Cline's fitpack on netlib?) thanks, Gary

1 0

A sad day for our community. John Hunter: 1968-2012.
by Fernando Perez Aug. 30, 2012

Aug. 30, 2012

Dear friends and colleagues, [please excuse a possible double-post of this message, in-flight internet glitches] I am terribly saddened to report that yesterday, August 28 2012 at 10am, John D. Hunter died from complications arising from cancer treatment at the University of Chicago hospital, after a brief but intense battle with this terrible illness. John is survived by his wife Miriam, his three daughters Rahel, Ava and Clara, his sisters Layne and Mary, and his mother Sarah. Note: If you decide not to read any further (I know this is a long message), please go to this page for some important information about how you can thank John for everything he gave in a decade of generous contributions to the Python and scientific communities: http://numfocus.org/johnhunter. Just a few weeks ago, John delivered his keynote address at the SciPy 2012 conference in Austin centered around the evolution of matplotlib: http://www.youtube.com/watch?v=e3lTby5RI54 but tragically, shortly after his return home he was diagnosed with advanced colon cancer. This diagnosis was a terrible discovery to us all, but John took it with his usual combination of calm and resolve, and initiated treatment procedures. Unfortunately, the first round of chemotherapy treatments led to severe complications that sent him to the intensive care unit, and despite the best efforts of the University of Chicago medical center staff, he never fully recovered from these. Yesterday morning, he died peacefully at the hospital with his loved ones at his bedside. John fought with grace and courage, enduring every necessary procedure with a smile on his face and a kind word for all of his caretakers and becoming a loved patient of the many teams that ended up involved with his case. This was no surprise for those of us who knew him, but he clearly left a deep and lasting mark even amongst staff hardened by the rigors of oncology floors and intensive care units. I don't need to explain to this community the impact of John's work, but allow me to briefly recap, in case this is read by some who don't know the whole story. In 2002, John was a postdoc at the University of Chicago hospital working on the analysis of epilepsy seizure data in children. Frustrated with the state of the existing proprietary solutions for this class of problems, he started using Python for his work, back when the scientific Python ecosystem was much, much smaller than it is today and this could have been seen as a crazy risk. Furthermore, he found that there were many half-baked solutions for data visualization in Python at the time, but none that truly met his needs. Undeterred, he went on to create matplotlib (http://matplotlib.org) and thus overcome one of the key obstacles for Python to become the best solution for open source scientific and technical computing. Matplotlib is both an amazing technical achievement and a shining example of open source community building, as John not only created its backbone but also fostered the development of a very strong development team, ensuring that the talent of many others could also contribute to this project. The value and importance of this are now painfully clear: despite having lost John, matplotlib continues to thrive thanks to the leadership of Michael Droetboom, the support of Perry Greenfield at the Hubble Telescope Science Institute, and the daily work of the rest of the team. I want to thank Perry and Michael for putting their resources and talent once more behind matplotlib, securing the future of the project. It is difficult to overstate the value and importance of matplotlib, and therefore of John's contributions (which do not end in matplotlib, by the way; but a biography will have to wait for another day...). Python has become a major force in the technical and scientific computing world, leading the open source offers and challenging expensive proprietary platforms with large teams and millions of dollars of resources behind them. But this would be impossible without a solid data visualization tool that would allow both ad-hoc data exploration and the production of complex, fine-tuned figures for papers, reports or websites. John had the vision to make matplotlib easy to use, but powerful and flexible enough to work in graphical user interfaces and as a server-side library, enabling a myriad use cases beyond his personal needs. This means that now, matplotlib powers everything from plots in dissertations and journal articles to custom data analysis projects and websites. And despite having left his academic career a few years ago for a job in industry, he remained engaged enough that as of today, he is still the top committer to matplotlib; this is the git shortlog of those with more than 1000 commits to the project: 2145 John Hunter <jdh2358(a)gmail.com> 2130 Michael Droettboom <mdboom(a)gmail.com> 1060 Eric Firing <efiring(a)hawaii.edu> All of this was done by a man who had three children to raise and who still always found the time to help those on the mailing lists, solve difficult technical problems in matplotlib, teach courses and seminars about scientific Python, and more recently help create the NumFOCUS foundation project. Despite the challenges that raising three children in an expensive city like Chicago presented, he never once wavered from his commitment to open source. But unfortunately now he is not here anymore to continue providing for their well-being, and I hope that all those who have so far benefited from his generosity, will thank this wonderful man who always gave far more than he received. Thanks to the rapid action of Travis Oliphant, the NumFOCUS foundation is now acting as an escrow agent to accept donations that will go into a fund to support the education and care of his wonderful girls Rahel, Ava and Clara. If you have benefited from John's many contributions, please say thanks in the way that would matter most to him, by helping Miriam continue the task of caring for and educating Rahel, Ava and Clara. You will find all the information necessary to make a donation here: http://numfocus.org/johnhunter Remember that even a small donation helps! If all those who ever use matplotlib give just a little bit, in the long run I am sure that we can make a difference. If you are a company that benefits in a serious way from matplotlib, remember that John was a staunch advocate of keeping all scientific Python projects under the BSD license so that commercial users could benefit from them without worry. Please say thanks to John in a way commensurate with your resources (and check how much a yearly matlab license would cost you in case you have any doubts about the value you are getting...). John's family is planning a private burial in Tennessee, but (most likely in September) there will also be a memorial service in Chicago that friends and members of the community can attend. We don't have the final scheduling details at this point, but I will post them once we know. I would like to again express my gratitude to Travis Oliphant for moving quickly with the setup of the donation support, and to Eric Jones (the founder of Enthought and another one of the central figures in our community) who immediately upon learning of John's plight contributed resources to support the family with everyday logistics while John was facing treatment as well as my travel to Chicago to assist. This kind of immediate urge to come to the help of others that Eric and Travis displayed is a hallmark of our community. Before closing, I want to take a moment to publicly thank the incredible staff of the University of Chicago medical center. The last two weeks were an intense and brutal ordeal for John and his loved ones, but the hospital staff offered a sometimes hard to believe, unending supply of generosity, care and humanity in addition to their technical competence. The latter is something we expect from a first-rate hospital at a top university, where the attending physicians can be world-renowned specialists in their field. But the former is often forgotten in a world often ruled by a combination of science and concerns about regulations and liability. Instead, we found generous and tireless staff who did everything in their power to ease the pain, always putting our well being ahead of any mindless adherence to protocol, patiently tending to every need we had and working far beyond their stated responsibilities to support us. To name only one person (and many others are equally deserving), I want to thank Dr. Carla Moreira, chief surgical resident, who spent the last few hours of John's life with us despite having just completed a solid night shift of surgical work. Instead of resting she came to the ICU and worked to ensure that those last hours were as comfortable as possible for John; her generous actions helped us through a very difficult moment. It is now time to close this already too long message... John, thanks for everything you gave all of us, and for the privilege of knowing you. Fernando. ps - I have sent this with my 'mailing lists' email. If you need to contact me directly for anything regarding the above, please write to my regular address at Fernando.Perez(a)berkeley.edu, where I do my best to reply more promptly.

1 0

A sad day for our community. John Hunter: 1968-2012.
by Fernando Perez Aug. 30, 2012

Aug. 30, 2012

Dear friends and colleagues, I am terribly saddened to report that yesterday, August 28 2012 at 10am, John D. Hunter died from complications arising from cancer treatment at the University of Chicago hospital, after a brief but intense battle with this terrible illness. John is survived by his wife Miriam, his three daughters Rahel, Ava and Clara, his sisters Layne and Mary, and his mother Sarah. Note: If you decide not to read any further (I know this is a long message), please go to this page for some important information about how you can thank John for everything he gave in a decade of generous contributions to the Python and scientific communities: http://numfocus.org/johnhunter. Just a few weeks ago, John delivered his keynote address at the SciPy 2012 conference in Austin centered around the evolution of matplotlib: http://www.youtube.com/watch?v=e3lTby5RI54 but tragically, shortly after his return home he was diagnosed with advanced colon cancer. This diagnosis was a terrible discovery to us all, but John took it with his usual combination of calm and resolve, and initiated treatment procedures. Unfortunately, the first round of chemotherapy treatments led to severe complications that sent him to the intensive care unit, and despite the best efforts of the University of Chicago medical center staff, he never fully recovered from these. Yesterday morning, he died peacefully at the hospital with his loved ones at his bedside. John fought with grace and courage, enduring every necessary procedure with a smile on his face and a kind word for all of his caretakers and becoming a loved patient of the many teams that ended up involved with his case. This was no surprise for those of us who knew him, but he clearly left a deep and lasting mark even amongst staff hardened by the rigors of oncology floors and intensive care units. I don't need to explain to this community the impact of John's work, but allow me to briefly recap, in case this is read by some who don't know the whole story. In 2002, John was a postdoc at the University of Chicago hospital working on the analysis of epilepsy seizure data in children. Frustrated with the state of the existing proprietary solutions for this class of problems, he started using Python for his work, back when the scientific Python ecosystem was much, much smaller than it is today and this could have been seen as a crazy risk. Furthermore, he found that there were many half-baked solutions for data visualization in Python at the time, but none that truly met his needs. Undeterred, he went on to create matplotlib (http://matplotlib.org) and thus overcome one of the key obstacles for Python to become the best solution for open source scientific and technical computing. Matplotlib is both an amazing technical achievement and a shining example of open source community building, as John not only created its backbone but also fostered the development of a very strong development team, ensuring that the talent of many others could also contribute to this project. The value and importance of this are now painfully clear: despite having lost John, matplotlib continues to thrive thanks to the leadership of Michael Droetboom, the support of Perry Greenfield at the Hubble Telescope Science Institute, and the daily work of the rest of the team. I want to thank Perry and Michael for putting their resources and talent once more behind matplotlib, securing the future of the project. It is difficult to overstate the value and importance of matplotlib, and therefore of John's contributions (which do not end in matplotlib, by the way; but a biography will have to wait for another day...). Python has become a major force in the technical and scientific computing world, leading the open source offers and challenging expensive proprietary platforms with large teams and millions of dollars of resources behind them. But this would be impossible without a solid data visualization tool that would allow both ad-hoc data exploration and the production of complex, fine-tuned figures for papers, reports or websites. John had the vision to make matplotlib easy to use, but powerful and flexible enough to work in graphical user interfaces and as a server-side library, enabling a myriad use cases beyond his personal needs. This means that now, matplotlib powers everything from plots in dissertations and journal articles to custom data analysis projects and websites. And despite having left his academic career a few years ago for a job in industry, he remained engaged enough that as of today, he is still the top committer to matplotlib; this is the git shortlog of those with more than 1000 commits to the project: 2145 John Hunter <jdh2358(a)gmail.com> 2130 Michael Droettboom <mdboom(a)gmail.com> 1060 Eric Firing <efiring(a)hawaii.edu> All of this was done by a man who had three children to raise and who still always found the time to help those on the mailing lists, solve difficult technical problems in matplotlib, teach courses and seminars about scientific Python, and more recently help create the NumFOCUS foundation project. Despite the challenges that raising three children in an expensive city like Chicago presented, he never once wavered from his commitment to open source. But unfortunately now he is not here anymore to continue providing for their well-being, and I hope that all those who have so far benefited from his generosity, will thank this wonderful man who always gave far more than he received. Thanks to the rapid action of Travis Oliphant, the NumFOCUS foundation is now acting as an escrow agent to accept donations that will go into a fund to support the education and care of his wonderful girls Rahel, Ava and Clara. If you have benefited from John's many contributions, please say thanks in the way that would matter most to him, by helping Miriam continue the task of caring for and educating Rahel, Ava and Clara. You will find all the information necessary to make a donation here: http://numfocus.org/johnhunter Remember that even a small donation helps! If all those who ever use matplotlib give just a little bit, in the long run I am sure that we can make a difference. If you are a company that benefits in a serious way from matplotlib, remember that John was a staunch advocate of keeping all scientific Python projects under the BSD license so that commercial users could benefit from them without worry. Please say thanks to John in a way commensurate with your resources (and check how much a yearly matlab license would cost you in case you have any doubts about the value you are getting...). John's family is planning a private burial in Tennessee, but (most likely in September) there will also be a memorial service in Chicago that friends and members of the community can attend. We don't have the final scheduling details at this point, but I will post them once we know. I would like to again express my gratitude to Travis Oliphant for moving quickly with the setup of the donation support, and to Eric Jones (the founder of Enthought and another one of the central figures in our community) who immediately upon learning of John's plight contributed resources to support the family with everyday logistics while John was facing treatment as well as my travel to Chicago to assist. This kind of immediate urge to come to the help of others that Eric and Travis displayed is a hallmark of our community. Before closing, I want to take a moment to publicly thank the incredible staff of the University of Chicago medical center. The last two weeks were an intense and brutal ordeal for John and his loved ones, but the hospital staff offered a sometimes hard to believe, unending supply of generosity, care and humanity in addition to their technical competence. The latter is something we expect from a first-rate hospital at a top university, where the attending physicians can be world-renowned specialists in their field. But the former is often forgotten in a world often ruled by a combination of science and concerns about regulations and liability. Instead, we found generous and tireless staff who did everything in their power to ease the pain, always putting our well being ahead of any mindless adherence to protocol, patiently tending to every need we had and working far beyond their stated responsibilities to support us. To name only one person (and many others are equally deserving), I want to thank Dr. Carla Moreira, chief surgical resident, who spent the last few hours of John's life with us despite having just completed a solid night shift of surgical work. Instead of resting she came to the ICU and worked to ensure that those last hours were as comfortable as possible for John; her generous actions helped us through a very difficult moment. It is now time to close this already too long message... John, thanks for everything you gave all of us, and for the privilege of knowing you. Fernando. ps - I have sent this with my 'mailing lists' email. If you need to contact me directly for anything regarding the above, please write to my regular address at Fernando.Perez(a)berkeley.edu, where I do my best to reply more promptly.

1 0

Interpolation problem with interp1d(.)
by eat Aug. 29, 2012

Aug. 29, 2012

Hi, Apparently I'm somehow misusing the functionality of interp1d(.) or does following behavior imply a bug in scipy. A minimum snippet (with plots) to demonstrate the problem: In []: from scipy.interpolate import interp1d In []: n= 1000 In []: x, y= randn(n), linspace(0, 1, n) In []: x.sort() In []: plot(x, y, lw= 2) Out[]: [<matplotlib.lines.Line2D object at 0x12E190D0>] In []: f= interp1d(x, y, 'cubic') In []: xi= linspace(x.min(), x.max(), n) In []: plot(xi, f(xi)) Out[]: [<matplotlib.lines.Line2D object at 0x12E20830>] Regards, -eat

2 3

2D integration of irregularly gridded data
by Andrew York Aug. 29, 2012

Aug. 29, 2012

I have 2D data sampled at irregular points. I want to estimate the integral of this data over a finite x-y region. Is there a standard way to do this in python/scipy, or should I roll my own integration? The algorithm I imagine is to use Delaunay triangulation to construct a surface f(x, y), and then sum the volume under this approximate surface. The form my data currently takes: x = [1.1, 1.3, 1.31, 1.33, 2, 2.05] #x-coordinates at which data is known y = [0.15, 0.7, 0.01, 0.01, 0.9, 0] #y-coordinates at which data is known f = [1.1, 1.3, 1.15, 1.2, 1.18, 1.3] # Data at these x-y coordinates (This isn't real data, just meant to concretely illustrate the type of data I have) I have to process roughly a million of these datasets to produce a single image, so a vectorized solution would be nice. A version of my question is described here: http://math.stackexchange.com/questions/187730/efficiently-estimate-a-2d-in…

1 0

Bug in freqz returning angular frequency and frequency response in backward order
by Jeffrey Spencer Aug. 29, 2012

Aug. 29, 2012

The freqz function appears to return the frequency values and the frequency response backwards. Like it is supposed to return in the order (h,w) but it actually returns in the order (w,h). This can even be seen in the documentation here: http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.freqz.html

2 3

scipy.ndimage.measurements.label
by Sarah Mount Aug. 26, 2012

Aug. 26, 2012

Hi all, Can someone please tell me what algorithm scipy.ndimage.measurements.label implements? Many thanks, Sarah -- Sarah Mount, Senior Lecturer, University of Wolverhampton website: http://www.snim2.org/ twitter: @snim2

2 2

Re: [SciPy-User] SciPy crashes Idle
by Chris Stehlik Aug. 25, 2012

Aug. 25, 2012

I ran as scipy.test(verbose=2) and it crashed at test_random_tall (test_decomp.TestQR) ... I opened a ticket. It's #1723. The whole file is long, so I won't post it here unless people specifically ask. > From: scipy-user-request(a)scipy.org > Subject: SciPy-User Digest, Vol 108, Issue 42 > To: scipy-user(a)scipy.org > Date: Sat, 25 Aug 2012 12:00:14 -0500 > > Send SciPy-User mailing list submissions to > scipy-user(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request(a)scipy.org > > You can reach the person managing the list at > scipy-user-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Scipy.test() crashes Idle (Chris Stehlik) > 2. Re: Scipy.test() crashes Idle (Scott Sinclair) > 3. Re: Scipy.test() crashes Idle (josef.pktd(a)gmail.com) > 4. Re: Scipy.test() crashes Idle (Ralf Gommers) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 24 Aug 2012 13:52:39 -0700 > From: Chris Stehlik <chrisfs59(a)hotmail.com> > Subject: [SciPy-User] Scipy.test() crashes Idle > To: <scipy-user(a)scipy.org> > Message-ID: <BAY168-W16FE180719C0967A93C1BB5BD0(a)phx.gbl> > Content-Type: text/plain; charset="iso-8859-1" > > > Hi, > > I am new to scipy and python itself. I was told to run scipy.test() to check it. I have a pc running Windows XP with the Enthought Free Python distribution 2.7.3 with Numpy 1.6.1 and Scipy 0.10.1. When I run scipy.test() in Idle, it crashes part way through. Numpy.test() goes through with no problems, but scipy.test() crashes each time. > How does one go about reporting a bug one this ? > > Chris > >

1 0