import scipy.stats hangs in mod_python application
Hi all, This may be really tricky to diagnose, given all the rather weird parameters, but here goes: I have a mod_python application that takes a bunch of mortality rates via a form, and processes them with scipy, numpy, and matplotlib. It is running on a Gentoo box. (Gentoo + mod_python seemed like a good idea at the time, but so did the Edsel... anyway, I still have to maintain this beast...) In the multitude of files that get loaded when a mod_python based page is called, there is a line "import scipy.stats as ST". That line works just great if I am running python at the command line, but it hangs forever when in the modpython application; I know this because I have run it both with an exception before it (barfs on the exception) and after it (just sits there, no errors to the apache log, etc). scipy.__version__ 0.9.0 python is 2.7.1 Anybody have any ideas how to fix this or even diagnose it?
On Sun, Aug 28, 2011 at 18:01, fork <forkandwait@gmail.com> wrote:
Hi all,
This may be really tricky to diagnose, given all the rather weird parameters, but here goes:
I have a mod_python application that takes a bunch of mortality rates via a form, and processes them with scipy, numpy, and matplotlib. It is running on a Gentoo box. (Gentoo + mod_python seemed like a good idea at the time, but so did the Edsel... anyway, I still have to maintain this beast...)
In the multitude of files that get loaded when a mod_python based page is called, there is a line "import scipy.stats as ST". That line works just great if I am running python at the command line, but it hangs forever when in the modpython application; I know this because I have run it both with an exception before it (barfs on the exception) and after it (just sits there, no errors to the apache log, etc).
scipy.__version__ 0.9.0
python is 2.7.1
Anybody have any ideas how to fix this or even diagnose it?
So, has your program ever worked under mod_python? Is this an intermittent problem, or is it reliably repeatable? In your program, do you import numpy before the scipy.stats import? IIRC, under certain configurations mod_python tries to instantiate multiple interpreters in the same process. For pure Python code, this is usually not too atrocious, even though it is officially unsupported. However, numpy maintains some global pointer tables, and people have run into problems trying to run multiple Python interpreters in the same process when the programs use numpy. There may be mod_python configuration options that control this behavior. First, I would convert the entry point into your application into a WSGI application. Since it looks like there is only one entry point, doing basic form processing before passing the inputs to the computational code, this should be trivial. It's really just a matter of converting the mod_python idioms into WSGI idioms. http://wsgi.org/wsgi/ Now that you have a WSGI application, there are a *lot* of different ways to deploy it. For Apache, mod_wsgi is much preferred over mod_python these days, although since it was written by the same author under very similar constraints as mod_python, he may have used the same multiple interpreters trick. You can use mod_fcgid or mod_scgi to communicate with a separate process running your scipy code fronted by flup: http://trac.saddi.com/flup Using mod_proxy, you can have Apache sit on the front end and communicate with any number of WSGI-specific HTTP servers. gunicorn is a nice one: http://gunicorn.org/ You can look at other options here: http://wsgi.org/wsgi/Servers -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern <robert.kern <at> gmail.com> writes:
So, has your program ever worked under mod_python? Is this an intermittent problem, or is it reliably repeatable? In your program, do you import numpy before the scipy.stats import?
Yeah, worked great until last week.
IIRC, under certain configurations mod_python tries to instantiate multiple interpreters in the same process. For pure Python code, this is usually not too atrocious, even though it is officially unsupported. However, numpy maintains some global pointer tables, and people have run into problems trying to run multiple Python interpreters in the same process when the programs use numpy. There may be mod_python configuration options that control this behavior.
Huh. Interesting.
First, I would convert the entry point into your application into a WSGI application. Since it looks like there is only one entry point, doing basic form processing before passing the inputs to the computational code, this should be trivial. It's really just a matter of converting the mod_python idioms into WSGI idioms.
The problem is that I use a lot of the convenience dispatching functions in mod_python, and I really, really don't want to rewrite them. Speed isn't the real issue, actually. If I could do it over again, I would have stuck with simple cgi, since the real computational expense is in a bunch of linear algebra not in input/ output.
On Sun, Aug 28, 2011 at 20:38, fork <forkandwait@gmail.com> wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
So, has your program ever worked under mod_python? Is this an intermittent problem, or is it reliably repeatable? In your program, do you import numpy before the scipy.stats import?
Yeah, worked great until last week.
Would you mind answering all of the questions rather than just the first? :-) Can you identify what changed last week? Did you change your program? Did you upgrade Apache/mod_python? That's going to be the only way to figure out what is wrong.
First, I would convert the entry point into your application into a WSGI application. Since it looks like there is only one entry point, doing basic form processing before passing the inputs to the computational code, this should be trivial. It's really just a matter of converting the mod_python idioms into WSGI idioms.
The problem is that I use a lot of the convenience dispatching functions in mod_python, and I really, really don't want to rewrite them.
You may want to take a quick look at a couple of the WSGI micro-frameworks around to see if they might be able to quickly implement the dispatching you need. http://bottlepy.org/docs/dev/ http://flask.pocoo.org/ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern <robert.kern <at> gmail.com> writes:
On Sun, Aug 28, 2011 at 20:38, fork <forkandwait <at> gmail.com> wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
So, has your program ever worked under mod_python? Is this an intermittent problem, or is it reliably repeatable? In your program, do you import numpy before the scipy.stats import?
Yeah, worked great until last week.
Would you mind answering all of the questions rather than just the first?
Sorry (hehe) ... worked great for years, reliable problem, import numpy (and scipy) before scipy.stats.
Can you identify what changed last week? Did you change your program? Did you upgrade Apache/mod_python? That's going to be the only way to figure out what is wrong.
We upgraded the default python on gentoo from 2.6 to 2.7, so, like, everything changed and it is almost impossible to say what exactly. My partner helped upgrade a bunch of math libraries, but I wasn't part of that so don't know exactly (yet...). I don't *believe* that apache got upgraded, and mod_python is definitely not upgraded, but it was reinstalled.
You may want to take a quick look at a couple of the WSGI micro-frameworks around to see if they might be able to quickly implement the dispatching you need.
That is great advice, but (1) I am still smarting from depending on a framework that got dropped from current development (three years ago cherrypy would have been in the list above), and (2) I don't have the 100 or so hours to refactor everything to fix this one (new) problem on code that probably won't see a next generation anyway. So, ... probably not gonna happen. On a fresher project, ... well, I would avoid frameworks like the plague and make it all cgi. (I hope I am not coming off obnoxious, but we probably don't need to discuss this any more.)
fork <forkandwait <at> gmail.com> writes:
Sorry (hehe) ... worked great for years, reliable problem, import numpy (and scipy) before scipy.stats.
And I just put scipy.stats ahead of everything, same problem.
On Mon, Aug 29, 2011 at 00:32, fork <forkandwait@gmail.com> wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
On Sun, Aug 28, 2011 at 20:38, fork <forkandwait <at> gmail.com> wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
So, has your program ever worked under mod_python? Is this an intermittent problem, or is it reliably repeatable? In your program, do you import numpy before the scipy.stats import?
Yeah, worked great until last week.
Would you mind answering all of the questions rather than just the first?
Sorry (hehe) ... worked great for years, reliable problem, import numpy (and scipy) before scipy.stats.
Hmm. Check what shared libraries are being linked to all of the scipy extension modules. There might be environment variables affecting how the shared libraries get resolved.
Can you identify what changed last week? Did you change your program? Did you upgrade Apache/mod_python? That's going to be the only way to figure out what is wrong.
We upgraded the default python on gentoo from 2.6 to 2.7, so, like, everything changed and it is almost impossible to say what exactly. My partner helped upgrade a bunch of math libraries, but I wasn't part of that so don't know exactly (yet...). I don't *believe* that apache got upgraded, and mod_python is definitely not upgraded, but it was reinstalled.
You may want to take a quick look at a couple of the WSGI micro-frameworks around to see if they might be able to quickly implement the dispatching you need.
That is great advice, but (1) I am still smarting from depending on a framework that got dropped from current development (three years ago cherrypy would have been in the list above), and
FWIW, CherryPy is still being developed: http://www.cherrypy.org/timeline -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Sun, Aug 28, 2011 at 7:01 PM, fork <forkandwait@gmail.com> wrote:
Hi all,
This may be really tricky to diagnose, given all the rather weird parameters, but here goes:
I have a mod_python application that takes a bunch of mortality rates via a form, and processes them with scipy, numpy, and matplotlib. It is running on a Gentoo box. (Gentoo + mod_python seemed like a good idea at the time, but so did the Edsel... anyway, I still have to maintain this beast...)
In the multitude of files that get loaded when a mod_python based page is called, there is a line "import scipy.stats as ST". That line works just great if I am running python at the command line, but it hangs forever when in the modpython application; I know this because I have run it both with an exception before it (barfs on the exception) and after it (just sits there, no errors to the apache log, etc).
scipy.__version__ 0.9.0
python is 2.7.1
Anybody have any ideas how to fix this or even diagnose it?
I have no idea about the actual problem, but scipy.stats requires or loads many of the other scipy subpackages. Since stats is relatively light in compiled extension modules, maybe trying to load first the other subpackages might help to narrow it down. import scipy import scipy.special sparse, optimize, ... Josef
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
<josef.pktd <at> gmail.com> writes:
I have no idea about the actual problem, but scipy.stats requires or loads many of the other scipy subpackages. Since stats is relatively light in compiled extension modules, maybe trying to load first the other subpackages might help to narrow it down. import scipy import scipy.special sparse, optimize, ...
import scipy works ok, but hangs in import scipy.stats.distributions. I have not started putting in "raise" statements to see where the problem might be. It only happens in my web-application, so it probably isn't a problem in scipy, unless there is something in the handling of imports that cycles forever which stops when called at command line...
On Tue, Sep 6, 2011 at 6:52 PM, fork <forkandwait@gmail.com> wrote:
<josef.pktd <at> gmail.com> writes:
I have no idea about the actual problem, but scipy.stats requires or loads many of the other scipy subpackages. Since stats is relatively light in compiled extension modules, maybe trying to load first the other subpackages might help to narrow it down. import scipy import scipy.special sparse, optimize, ...
import scipy works ok, but hangs in import scipy.stats.distributions.
"import scipy" is pretty empty and doesn't load any subpackages. If the other subpackages haven't been loaded explicitly before, then "import scipy.stats" imports also the other packages for the first time. I still cannot think of anything that would be specific to scipy.stats or scipy.stats.distributions, and I think Pauli and David have moved some functions during the python 3 porting to avoid circular imports. Josef
I have not started putting in "raise" statements to see where the problem might be. It only happens in my web-application, so it probably isn't a problem in scipy, unless there is something in the handling of imports that cycles forever which stops when called at command line...
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Tue, 06 Sep 2011 22:52:12 +0000, fork wrote:
<josef.pktd <at> gmail.com> writes:
I have no idea about the actual problem, but scipy.stats requires or loads many of the other scipy subpackages. Since stats is relatively light in compiled extension modules, maybe trying to load first the other subpackages might help to narrow it down. import scipy import scipy.special sparse, optimize, ...
import scipy works ok, but hangs in import scipy.stats.distributions.
How about import scipy.constants # import scipy.cluster # import scipy.fftpack ... I.e., try to import each subpackage, one at a time, to see which ones hang. Also try import scipy.stats.vonmises_cython -- Pauli Virtanen
participants (4)
-
fork -
josef.pktd＠gmail.com -
Pauli Virtanen -
Robert Kern