[Python-Dev] Re: [PyWX] RE: PyWX (Python AOLserver plugin)

[Guido van Rossum]

> > This is largely true, but we run across trouble with the way
> > the individual threads handle 'argv' variables and current
> > working directory.
> > 
> > CGI scripts typically pass data as variables to the script
> > (as argv).  These (unfortunately) are changed globally across
> > all Python interpreter threads, which can cause problems....
> > 
> > In addition, the current working directory is not unique
> > among independent Python interpreters.  So if a script changes
> > its directory to something, all other running scripts (in
> > unique python interpreter threads) now have their cwd set to
> > this directory.

There's no easy way to fix the current directory problem.  Just tell
your CGI programmers that os.chdir() is off-limits; you may remove it
from the os module (and from the posix module) during initialization
of your interpreter to enforce this.

I don't understand how you would be sharing sys.argv between multiple
interpreters.  Sure, the initial sys.argv is the same (they all
inherit that from the C main()) but after that you can set it to
whatever you want and they should not be shared.

Are you *sure* you are using PyInterpreterState_New() and not just
creating new threads?

> -> So we are basically stuck.  We can't link against Python multiple
> -> times, so our only avenue to provide multiple interpreter instances
> -> is to use the "Py_InterpreterNew" call and hope for the best.
> -> 
> -> Any hope for better interpreter isolation in 2.0? (2.1?)
> 
> Perhaps a better question is: is there any way to get around these problems
> without moving from a threaded model (which we like) to a process model?
> 
> Many of the problems we're running into because of this lack of interpreter
> isolation are due to the UNIX threading model, as I see it.  For example,
> the low-level file operation interference, cwd problems, and environment
> variable problems are all caused by UNIX's determination to share this stuff
> across threads.  I don't see any way of changing this without causing far
> more problems than we fix.

That's the whole point of using threads -- they share as much state as
they can.  I don't see how you can do better without going to
processes.  You could perhaps maintain the illusion of a per-thread
current directory, but you'd have to modify every function that uses
pathnames to take the simulated pwd into account...

--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)