Excluding the current path from module search path?
Hi All, I'm being bitten by this issue: http://bugs.python.org/issue1734860 I'm not sure I agree with Daniel's closing of it so thought I'd ask here... Am I right in thinking that the general idea is that "the current working directory at the time of invoking a script or interpreter ends up on the python path" or should I be thinking "the directory that a script exists in should end up on the python path"? If the latter, then what happens in the case of just starting up an interpreter? If neither, then how come when I have two .py files in a directory, I can import one as a module from the other? In any case, as a parting comment, http://bugs.python.org/issue1232023 seems to have been committed with no tests and the only documentation being a one liner in the NEWS.txt file. Was there other discussion of this? (Incidentally, export PYTHONPATH= or its Windows equivalent circumvents whatever the patch was trying to achieve, so the change doesn't seem to make sense anyway...) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, 25 Aug 2009 at 16:59, Chris Withers wrote:
It probably should have gone into What's New as well, but it was too late for that at the time the bug was filed.
The change was fixing a clear bug: blank path elements were being introduced into the path _unintentionally_ and unexpectedly. Setting PYTHONPATH would be a way to do it intentionally. --David
Chris Withers wrote:
The details of the sys.path manipulation at program startup are documented here: http://docs.python.org/using/cmdline.html#command-line The directory prepended to sys.path is based on the code executed by the command line. stdin, -c, -m or nothing specified: current directory Filesystem path pointing to script (source or compiled): directory containing script Filesystem path pointing to directory or zipfile: the named directory or zipfile Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan wrote:
It's more subtle than that though... The OP in http://bugs.python.org/issue1734860 is being bitten by the same expectation that I am: sitecustomize.py should be found somewhere on the sys.path present at the start of the script/module/command/etc being executed. (The bug referenced in that report makes things worse, because this used to work, at least on Windows ;-) ) The problem is that site.py (and therefore sitecustomize.py) is imported early in main.c on line 516 as part of Py_Initialize(), but the path of the current script only gets added later on in RunMainFromImporter called on line 569. Strictly speaking, the docs at http://docs.python.org/library/site.html aren't lying, but it takes an understanding of when site.py is imported that isn't available to anyone who doesn't read C to know why a path that is present on sys.path when the user's script starts isn't being searched for sitecustomize.py What do people feel about this? At the very least, I'd like to add a warning box in site.html to explain why sitecustomize might not be found where people expect. I'd *like* to have the paths be the same for site.py as they are for the subsequent code that's executed, but would that make too much of a mess of main.c and runpython.c? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
Ah, OK - I see the problem now. However, I think the current behaviour is correct, it just needs to be documented better (probably noted in both the command line doco regarding sys.path manipulation and in the doco for site.py). The reason I think the current behaviour is correct is that site.py and sitecustomize.py are meant to be about customising the *site* (i.e. the installation of Python that is being executed) rather than about customizing a particular application. Importing them before the script specific directories are prepended to sys.path goes a long way towards achieving that. Also, as was pointed out on the tracker item, having a script that can automatically be executed when running an arbitrary Python script without any request from or notification to the user is not a good idea from a security standpoint. When it comes to adding additional paths for specific applications, you can either bundle the relevant packages into a single directory and use 2.6's directory execution feature or else look into the assorted application environment customisation tools that are out there like virtualenv. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan wrote:
Not sure what you mean by this?
regarding sys.path manipulation and in the doco for site.py).
Agreed :-)
Unless you use virtualenv as Guido suggested in the other thread ;-)
If sitecustomize.py had more uses that the setdefaultencoding hack, I'd argue more about this... If it does have other uses, my argument would be that "site" wide is a very subjective term tht many people, myself included, would like to be able to mean "per project, I don't *ever* want to screw with my actual Python install, it should stay pristine"...
Agreed, but I think that's only an issue when you're starting up an interpreter. If you're running a script from a file or module, I'd say it's more akin to what's specified in PYTHONSTARTUP being executed that than a random script being silently executed without your permission. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, 25 Aug 2009 at 16:59, Chris Withers wrote:
It probably should have gone into What's New as well, but it was too late for that at the time the bug was filed.
The change was fixing a clear bug: blank path elements were being introduced into the path _unintentionally_ and unexpectedly. Setting PYTHONPATH would be a way to do it intentionally. --David
Chris Withers wrote:
The details of the sys.path manipulation at program startup are documented here: http://docs.python.org/using/cmdline.html#command-line The directory prepended to sys.path is based on the code executed by the command line. stdin, -c, -m or nothing specified: current directory Filesystem path pointing to script (source or compiled): directory containing script Filesystem path pointing to directory or zipfile: the named directory or zipfile Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan wrote:
It's more subtle than that though... The OP in http://bugs.python.org/issue1734860 is being bitten by the same expectation that I am: sitecustomize.py should be found somewhere on the sys.path present at the start of the script/module/command/etc being executed. (The bug referenced in that report makes things worse, because this used to work, at least on Windows ;-) ) The problem is that site.py (and therefore sitecustomize.py) is imported early in main.c on line 516 as part of Py_Initialize(), but the path of the current script only gets added later on in RunMainFromImporter called on line 569. Strictly speaking, the docs at http://docs.python.org/library/site.html aren't lying, but it takes an understanding of when site.py is imported that isn't available to anyone who doesn't read C to know why a path that is present on sys.path when the user's script starts isn't being searched for sitecustomize.py What do people feel about this? At the very least, I'd like to add a warning box in site.html to explain why sitecustomize might not be found where people expect. I'd *like* to have the paths be the same for site.py as they are for the subsequent code that's executed, but would that make too much of a mess of main.c and runpython.c? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
Ah, OK - I see the problem now. However, I think the current behaviour is correct, it just needs to be documented better (probably noted in both the command line doco regarding sys.path manipulation and in the doco for site.py). The reason I think the current behaviour is correct is that site.py and sitecustomize.py are meant to be about customising the *site* (i.e. the installation of Python that is being executed) rather than about customizing a particular application. Importing them before the script specific directories are prepended to sys.path goes a long way towards achieving that. Also, as was pointed out on the tracker item, having a script that can automatically be executed when running an arbitrary Python script without any request from or notification to the user is not a good idea from a security standpoint. When it comes to adding additional paths for specific applications, you can either bundle the relevant packages into a single directory and use 2.6's directory execution feature or else look into the assorted application environment customisation tools that are out there like virtualenv. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan wrote:
Not sure what you mean by this?
regarding sys.path manipulation and in the doco for site.py).
Agreed :-)
Unless you use virtualenv as Guido suggested in the other thread ;-)
If sitecustomize.py had more uses that the setdefaultencoding hack, I'd argue more about this... If it does have other uses, my argument would be that "site" wide is a very subjective term tht many people, myself included, would like to be able to mean "per project, I don't *ever* want to screw with my actual Python install, it should stay pristine"...
Agreed, but I think that's only an issue when you're starting up an interpreter. If you're running a script from a file or module, I'd say it's more akin to what's specified in PYTHONSTARTUP being executed that than a random script being silently executed without your permission. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
participants (5)
-
Benjamin Peterson -
Chris Withers -
Nick Coghlan -
P.J. Eby -
R. David Murray