[Python-Dev] Issue 13524: subprocess on Windows

[Brad Allen]

On Thu, Mar 22, 2012 at 2:35 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:

> Also, in order to execute in any installation environment where libraries are found in non-default locations, you will need to set LD_LIBRARY_PATH.  Oh, and you will also need to set $PATH on UNIX so that libraries can find their helper programs and %PATH% on Windows so that any compiled dynamically-loadable modules and/or DLLs can be loaded.  And by the way you will also need to relay DYLD_LIBRARY_PATH if you did a UNIX-style build on OS X, not LD_LIBRARY_PATH.  Don't forget that you probably also need PYTHONPATH to make sure any subprocess environments can import the same modules as their parent.  Not to mention SSH_AUTH_SOCK if your application requires access to _remote_ process spawning, rather than just local.  Oh and DISPLAY in case your subprocesses need GUI support from an X11 program (which sometimes you need just to initialize certain libraries which don't actually do anything with a GUI).  Oh and __CF_USER_TEXT_ENCODING is important sometimes too, don't forget that.  And if your subprocess is in Perl or Ruby or Java you may need a couple dozen other variables which your deployment environment has set for you too.  Did I mention CFLAGS or LC_ALL yet?  Let me tell you a story about this one HP/UX machine...
>
> Ahem.
>
> Bottom line: it seems like screwing with the process spawning environment to make it minimal is a good idea for simplicity, for security, and for modularity.  But take it from me, it isn't.  I guarantee you that you don't actually know what is in your operating system's environment, and initializing it is a complicated many-step dance which some vendor or sysadmin or product integrator figured out how to do much better than your hapless Python program can.
>
> %SystemRoot% is just the tip of a very big, very nasty iceberg.  Better not to keep refining why exactly it's required, or someone will eventually be adding a new variable (starting with %APPDATA% and %HOMEPATH%) that can magically cause your subprocess not to spawn properly to this page every six months for eternity.  If you're spawning processes as a regular user, you should just take the environment you're given, perhaps with a few specific light additions whose meaning you understand.  If you're spawning a process as an administrator or root, you should probably initialize the environment for the user you want to spawn that process as using an OS-specific mechanism like login(1).  (Sorry that I don't know the Windows equivalent.)


Thanks, Glyph. In that case maybe the Python subprocess docs need not
single out SystemRoot, but instead plaster a big warning around the
use of the 'env' parameter.:

Here is what the docs currently state for the Popen constructor 'env' parameter:

>If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of inheriting the current process’ environment, which is the default behavior.

> Note: If specified, env must provide any variables required for the program to execute. On Windows, in order to run a side-by-side assembly the specified env must include a valid SystemRoot.

The "Note" section could instead state something like: "In most cases,
the child process will need many of the same environment variables as
the current process. Usually the safest course of action is to build
the env dict to contain all the same keys and values from os.environ.
For example... <insert Glyph's examples here>"