[Python-Dev] Re: [PyWX] RE: PyWX (Python AOLserver plugin)

[Brent Fulgham]

> There's no easy way to fix the current directory problem.  Just tell
> your CGI programmers that os.chdir() is off-limits; you may remove it
> from the os module (and from the posix module) during initialization
> of your interpreter to enforce this.
>

This is probably a good idea.
 
[ ... snip ... ]

> Are you *sure* you are using PyInterpreterState_New() and not just
> creating new threads?
>
Yes.
 
[ ... snip ... ]

> > Many of the problems we're running into because of this 
> > lack of interpreter isolation are due to the UNIX threading 
> > model, as I see it. 

Titus -- any chance s/UNIX/pthreads/ ?  I.e., would using something
like AOLserver's threading libraries help by providing more
thread-local storage in which to squirrel away various environment
data, dictionaries, etc.?

> > For example, the low-level file operation interference, 
> > cwd problems, and environment variable problems are all caused 
> > by UNIX's determination to share this stuff across threads.  
> > I don't see any way of changing this without causing far
> > more problems than we fix.
> 
> That's the whole point of using threads -- they share as much state as
> they can.  I don't see how you can do better without going to
> processes.  You could perhaps maintain the illusion of a per-thread
> current directory, but you'd have to modify every function that uses
> pathnames to take the simulated pwd into account...
> 

I think we just can't be all things to all people, which is a point
Michael has patiently been making this whole time.  I propose:

1.  We disable os.chdir in PyWX initialization.
2.  We assume "standard" CGI behavior of CGIDIR being a single 
directory that all CGI's share.
3.  We address sys.argv (is this just a bug on our part maybe?)
4.  Can we address the os.environ leak similarly?  I'm trying to 
think of cases where a CGI really should be allowed to add to
the environment.  Maybe someone needs to set an environment variable
used by some other program that will be run in a subshell.  If
so, maybe we can somehow serialize activities that modify os.environ
in this way?

Idea:  If Python forks a subshell, it inherits the parent
process's environment.  That's probably the only time we really want
to let someone modify the os.environ -- so it can be passed to
a child.  What if we serialized through the fork somehow like so:

1.  Python script wants to set environment, makes call to os.environ
1a. We serialize here, so now we are single-threaded
2.  Script forks a subshell.
2b. We remove the entry we just added and release mutex.
3.  Execution continues.

This probably still won't work because the script might now expect
these variables to be in the environment dictionary.

Perhaps we can dummy up a fake os.environ dictionary per interpreter
thread that doesn't actually change the true UNIX environment?

What do you guys think...

Thanks,

-Brent