[ python-Bugs-996748 ] os.environ documentation should indicate unreliability

[SourceForge.net]

Bugs item #996748, was opened at 2004-07-23 14:41
Message generated for change (Comment added) made by tim_one
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=996748&group_id=5470

>Category: Documentation
>Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Richard Tibbetts (tibbetts)
>Assigned to: Tim Peters (tim_one)
Summary: os.environ documentation should indicate unreliability

Initial Comment:
Looking at the documenation for the os module, nothing
in the documentation of os.environ would indicate that
it is unreliable. But then looking at os.putenv, we see
that os.environ will not reflect changes made to the
environment through other means.

To me, this makes os.environ much less useful. In fact,
it makes it somewhat harmful. I would argue for
deprecating it, or at least documenting this problem.

Thanks,
Richard


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2004-07-25 20:50

Message:
Logged In: YES 
user_id=31435

Closed with a doc patch, Doc/lib/libos.tex revision 1.140.

tibbetts, I don't know what use case you have in mind, and 
you didn't respond to Martin's request to clarify.  I can tell 
you that in more than a decade of using Python, I've never 
heard of anyone getting into any trouble using os.environ, 
except for the irrelevant (wrt what you're saying) problem 
than C putenv() leaks memory on some platforms.

We're certainly not going to deprecate this useful feature, 
and you've made no case here for complicating and slowing it 
by crawling over the OS environment structures each time a 
new query is made (getenv() is slo*, and the relatively 
greater speed of an os.environ lookup (typically constant-
time regardless of the number of environment keys) is 
important to some Python programs).

----------------------------------------------------------------------

Comment By: Richard Tibbetts (tibbetts)
Date: 2004-07-25 20:27

Message:
Logged In: YES 
user_id=18317

It seems to me that one would reasonably expect the
os.environ map to reflect the current state of the
environment, that it would be identical to using putenv/setenv.

I think it is very important that it either be
removed/deprecated in favor of people using interfaces they
are likely to understand (putenv, getenv), or that at
minimum it be very well documented.

Just a note from the lowly user.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2004-07-25 16:53

Message:
Logged In: YES 
user_id=21627

That's easy to answer: posixinit/ntinit is called when the
module posix/nt is first imported.

Now, one may wonder when that happens for the first time.
That depends very much on the application, but typically
posix is imported first through os, and os is first imported
through site.py, right after sys is imported.

Tim, with that information, can you add text to the docs?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2004-07-25 14:57

Message:
Logged In: YES 
user_id=31435

Martin, to judge from what Raymond's recipe does, by "its 
own environment" he means the environment of the parent 
process:  his recipe uses the Windows-specific quirk that 
a .bat script *can* change its invoker's (parent process's) 
environment (under Unixy shells you use a "source" command 
to get this effect).

Sjoerd, your example works the same way under Windows:

>>> import os
>>> os.system('SET X')
Environment variable X not defined
1
>>> os.environ['X']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\python23\lib\os.py", line 417, in __getitem__
    return self.data[key.upper()]
KeyError: 'X'
>>> os.putenv('X', 'Y')
>>> os.system('SET X')
X=Y
0
>>> os.environ['X']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\python23\lib\os.py", line 417, in __getitem__
    return self.data[key.upper()]
KeyError: 'X'
>>> os.environ['X'] = 'Z'
>>> os.system('SET X')
X=Z
0
>>> os.environ['X']
'Z'
>>>

All, what's missing in the docs is a statement that os.environ 
captures the state of the environment at a particular point in 
time, so that os.environ[key] doesn't *necessarily* reflect 
the current binding of key in the OS's notion of environment.  
I'd add that info, except I'm not exactly sure at which 
specific point in time we capture it (it's when we initialize 
posixmodule.c, but I'm not sure when that happens).

----------------------------------------------------------------------

Comment By: Sjoerd Mullender (sjoerd)
Date: 2004-07-25 14:06

Message:
Logged In: YES 
user_id=43607

On Linux:

>>> import os
>>> os.system('printenv x')
256
>>> os.environ['x']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File
"/home/sjoerd/src/python/build.linux/../Lib/UserDict.py",
line 17, in __getitem__
    def __getitem__(self, key): return self.data[key]
KeyError: 'x'
>>> os.putenv('x', 'y')
>>> os.system('printenv x')
y
0
>>> os.environ['x']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File
"/home/sjoerd/src/python/build.linux/../Lib/UserDict.py",
line 17, in __getitem__
    def __getitem__(self, key): return self.data[key]
KeyError: 'x'
>>> os.environ['x'] = 'z'
>>> os.system('printenv x')
z
0
>>> os.environ['x']
'z'
>>>

This seems clear enough: os.putenv modifies the environment
(look at the printenv output), but does not modify
os.environ.  Modifying os.environ also modifies the
environment for child processes.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2004-07-25 13:52

Message:
Logged In: YES 
user_id=21627

rhettinger: why are you saying that putenv does not modify
the current environment? According to

http://www.opengroup.org/onlinepubs/009695399/functions/putenv.html

it does modify the current environment; more specifically,
it modifies environ.

tibbetts, can you please explain what other means of
modifying the environment may occur? In a Python program,
the only way of modifying the environment is through
modifying os.environ, which calls putenv, or through
directly using putenv. If anything should be changed, we
should advise people not to call putenv directly.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-07-24 16:25

Message:
Logged In: YES 
user_id=80475

FWIW, os.environ is reliable; it is putenv that is not doing
what you expect.

putenv() is there for updating the environment of
sub-processes. There doesn't appear to be a reliable cross
platform approach having any program update its own environment.

Look on ASPN for my recipe that shows an o.s. trick for
solving this problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=996748&group_id=5470