python, shell, environment variable

[James T. Dennis]

oliver <fwang2 at yahoo.com> wrote:

> hi, folks,

> I am trying to get a set of python and bash scripts to work, and I
> constantly run into trouble with environment variables:

> First, my python script need to modify environment variable, say,
> PATH, and want to keep the modification even after the script is done.
> os.environ["PATH"]= ... doesn't seem to work, any idea?

 Yes.  You seem to have a fundamental misunderstanding about how
 environment variables work (under UNIX and UNIX-like OSes, at least).

 The environment is a region of your process' memory that is not
 overwritten by an exec*() system call.  Thus it is an efficient way
 for a parent process to pass some (textual or textually encoded) data
 to its children.  The environment is *NOT* a share memory mechanism.
 
 To execute Python your shell performed a fork() and then an exec*()
 (some form of exec; it's not important which one).  fork() creates a
 new private address space for your new process, and address space that
 is initially an (almost) exact copy of the parent's memory (except for
 the portion of the stack or heap that's holding the return value to 
 the fork() call itself).  [Please understand that this is a new
 "virtual" address space; the actual physical memory pages are usually
 shared through page table aliasing on modern processors/architectures
 running modern kernels --- using a technique called CoW: copy on write].

 So your assignments to os.environ["PATH"]=something are just happening
 to your address space.  They can't affect your parent's memory (that
 would be a SEGV, segmentation violation; or it would require some form
 of shared memory).  The assignments would be visible in any processes
 you created *under* your python process (os.popen('printenv') and search
 for the PATH entry for one way to demonstrate that), and they should
 remain set in that process (that was python) if you finish your script
 with one of the os.exec*() functions (except for the exec*e() which
 require you to explicitly prepare and pass a new environment to them).

> Second, it would be better if my python script can call bash shell and
> still keep the environment variable modification done by bash. I tried
> os.popen("source some_shell"), it doesn't work. ?

 os.popen is creating yet another process; and the source command
 therein is modifying popen's memory.

> Thanks for help.

> -oliver

 This is a fundamental consequence of UNIX' fork()/exec*() and 
 memory management/protection model.  It is a basic principle that
 leads to several FAQs.  It is something I have to emphasize at 
 considerable length in my shell scripting classes.  It is also one
 of the best examples of how important it is to understand the big
 picture, the "model" of an architecture, an OS, a language, or any
 other technology that one must control and use in any but the most
 trivial ways.  Once you understand how fork(), exec*() and *NIX
 memory management works then you'd never have this question --- because
 the answer would be so obvious.  (When my son tattoos his arm, why
 can't my grandpa read it off *his* arm?  Because it's a different skin
 that's being modified!).