Improving distutils' script and GUI app handling
Every so often, there's a perennial debate on the distutils-sig about script filenames. Should they have .py extensions or not? What about .pyw apps on Windows? What about Mac OS? and so on.
It occurred to me this morning that we now have a new tool for resolving this issue in setuptools' "entry points" feature. For various reasons, it's common practice to write scripts as Python modules with a "main" function of some kind. These modules are then run directly with '-m', or use "if __name__=='__main__'", or have a short script that imports the main function and runs it.
So, if these "main" functions were simply declared as entry points in the project's setup script, then EasyInstall could automatically generate stub scripts for them, in a platform-appropriate fashion, with no need for '-m', "__name__=='__main__'", or fiddling with file extensions. For example, if PyUnit were distributed as an egg, with the following entry points:
[distutils.console_apps] unittest = unittest:main
Then EasyInstall could create a 'unittest' script with a #! line on Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on Windows. In each case, the generated program would simply load and run the entry point with no arguments. Similarly, there could be a "distutils.gui_apps" entry point group, which could be handled differently according to the target platform. (For example, by creating desktop or menu shortcuts on Windows.) And, tools that create standalone applications or installers could use this information to create other kinds of wrappers around the same entry points.
In order for this to work well, there are obviously some details to be worked out. Unix-like OSes are pretty obvious: #! line and no extension should work fine for all app types' executables. But adding desktop icons or menu items for GUI apps is very platform-specific (e.g. Gnome vs. KDE). So, whatever mechanism is used needs to be extensible and configurable. Ideally, it should be possible for the platform's Python installation to provide the necessary hooks to do this, because I personally don't want to have to write and maintain all that code. :)
As for Mac OS, I have almost no experience with it, so I'm not sure what GUI applications there need. Does everything need py2app? If you have a wx-based app, would you just make a #! script? Bob Ippolito previously mentioned that you don't "install" applications there, that people just drag applications wherever they want them rather than using shortcuts, so at least that part isn't a problem. :)
On Windows, I'd say that applications are pretty much always better as .exe's, whether console or GUI. The .py/.pyw form is dependent on a single global consistent version of Python, but it's possible and reasonable to have multiple Python versions installed. An .exe also has a lot more control over how Python is initialized, and that can be particularly important for applications. On the other hand, in the short run I can also see using .bat or .cmd files for console apps, and .pyw for GUI apps, just to have something that works and wait for the path management use cases for various .exe options to work themselves out.
Anyway, my idea here is that when using setuptools, you would define entry points instead of creating scripts and listing them in setup(). Then, using either EasyInstall or "setup.py install" would automatically create platform-appropriate scripts.
Some of the open questions:
* What groups (if any) should exist besides "console_apps" and "gui_apps"?
* How can we allow easy control of installation options on the target system? (e.g., I only want Windows Programs Menu items, you only want desktop shortcuts for KDE, etc., but we can't have six billion command line options, and all of this stuff can't go into setuptools anyway)
* Should launchers hardcode their sys.path? This would prevent breakage caused by installing new and incompatible versions of their dependencies, but require an explicit upgrade action to update them, and make the installation process more complex.
Thoughts, anyone?
Phillip J. Eby wrote:
Every so often, there's a perennial debate on the distutils-sig about script filenames. Should they have .py extensions or not? What about .pyw apps on Windows? What about Mac OS? and so on.
It occurred to me this morning that we now have a new tool for resolving this issue in setuptools' "entry points" feature. For various reasons, it's common practice to write scripts as Python modules with a "main" function of some kind. These modules are then run directly with '-m', or use "if __name__=='__main__'", or have a short script that imports the main function and runs it.
I've already noted my distaste for __name__ == '__main__', so I'd be happy to see it go (or, at least replaced with something better). An entry point seems easy.
One issue that has come up for me is the way setuptools currently generates scripts causes problems for me. For instance, if I do tag_svn_revision, then the version of my package can change in place (whenever I do setup.py egg_info). The script has a version hardcoded in it, and so it fails.
I'm not sure what the best way to handle this is, but the current behavior can look very ugly from a user's perspective when it fails. At least if I'm using tag_svn_revision, if I could put a requirement spec in setup.cfg to also control what the script requires, that would help. Also, if we support multiple versions of packages, each with a script tied to that version, then there needs to be an easier way to control the script names, e.g., --script-suffix=0.4 or something.
The other issue is which Python version is used to load the script; it's not uncommon to want to switch versions. It would be nice if there was something -- an option, perhaps, or an environmental variable -- which could override the Python version. E.g. script --python-version=python2.4, or PYTHON_VERSION=python2.4 script (I don't think the later isn't reasonable to do in Windows, though)
- What groups (if any) should exist besides "console_apps" and "gui_apps"?
I can't think of any, except of course for ad hoc subcommands like distutils.command. Well, rc scripts and system services perhaps, but that's not something I'd want to tackle right now; it's not even clear that a package should map to an rc script.
- Should launchers hardcode their sys.path? This would prevent breakage
caused by installing new and incompatible versions of their dependencies, but require an explicit upgrade action to update them, and make the installation process more complex.
I'm wary of hardcoding paths. I don't think we should completely embrace encapsulation -- it might be better for long-term stability (maybe), but it makes it harder to fix a system that is broken (even when "brkoen" just means one missing requirement).
At 10:41 AM 9/15/2005 -0500, Ian Bicking wrote:
One issue that has come up for me is the way setuptools currently generates scripts causes problems for me. For instance, if I do tag_svn_revision, then the version of my package can change in place (whenever I do setup.py egg_info). The script has a version hardcoded in it, and so it fails.
Run "develop" instead of "egg_info", and your active scripts will be upgraded. (The "develop" command also runs "egg_info" as a subcommand.)
Also, if we support multiple versions of packages, each with a script tied to that version, then there needs to be an easier way to control the script names, e.g., --script-suffix=0.4 or something.
That would indeed be handy.
The other issue is which Python version is used to load the script; it's not uncommon to want to switch versions. It would be nice if there was something -- an option, perhaps, or an environmental variable -- which could override the Python version. E.g. script --python-version=python2.4, or PYTHON_VERSION=python2.4 script (I don't think the later isn't reasonable to do in Windows, though)
I don't think this can be made to work in any meaningful fashion, since eggs are specific to a particular Python version (because of bytecode format differences), and may contain different modules or entry points or dependencies for different Python versions. For example, if somebody packages a new module for use with older Pythons, users of that package will likely have their setup() include a dependency for the separately-packaged module if running on the older Python. Another example: setuptools doesn't publish an entry point for the 'build_py' command when running on Python 2.4, but it does on 2.3 because the 2.3 build_py command doesn't support installing package data.
- Should launchers hardcode their sys.path? This would prevent breakage
caused by installing new and incompatible versions of their dependencies, but require an explicit upgrade action to update them, and make the installation process more complex.
I'm wary of hardcoding paths. I don't think we should completely embrace encapsulation -- it might be better for long-term stability (maybe), but it makes it harder to fix a system that is broken (even when "brkoen" just means one missing requirement).
Well, with a sufficiently complex launcher, you could have it try to fix the problem (e.g. by running EasyInstall to find the missing bits). But I guess we can just evolve that way over time. :)
I haven't yet played with setuptools/eggs/etc. so my comments might be ignorant.
[Phillip J. Eby wrote]
Then EasyInstall could create a 'unittest' script with a #! line on Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on Windows. In each case, the generated program would simply load and run the entry point with no arguments.
Effbot has some good things to say about/use for this: http://effbot.org/zone/exemaker.htm
On Windows, I'd say that applications are pretty much always better as .exe's, whether console or GUI. The .py/.pyw form is dependent on a single global consistent version of Python, but it's possible and reasonable to have multiple Python versions installed. An .exe also has a lot more control over how Python is initialized, and that can be particularly important for applications. On the other hand, in the short run I can also see using .bat or .cmd files for console apps, and .pyw for GUI apps, just to have something that works and wait for the path management use cases for various .exe options to work themselves out.
There is a bug in the current cmd.exe (or maybe it is only up to Win2k? can't remember) where shell redirection doesn't work if your script is launched indirectly via a .bat of .cmd file. As well, Ctrl+C signal handling for kill a script is quite annoying if launched via a .bat file: you get a secondary question from the shell
Terminate batch file (Y/N)?
or something like that. A .exe launcher/stub is definitely preferred.
Trent
[Ian Bicking wrote]
I've already noted my distaste for __name__ == '__main__', so I'd be happy to see it go (or, at least replaced with something better).
That facility is about one of the most useful in Python. ...but I haven't read your previous comments on that. :)
Trent
Overall, I like the idea of using an entry point for scripts and getting some of the benefits you're talking about...
On 9/15/05, Phillip J. Eby pje@telecommunity.com wrote:
As for Mac OS, I have almost no experience with it, so I'm not sure what GUI applications there need. Does everything need py2app? If you have a wx-based app, would you just make a #! script? Bob Ippolito previously mentioned that you don't "install" applications there, that people just drag applications wherever they want them rather than using shortcuts, so at least that part isn't a problem. :)
GUI apps on the Mac need py2app.
Kevin
Trent Mick trentm@ActiveState.com writes:
I haven't yet played with setuptools/eggs/etc. so my comments might be ignorant.
[Phillip J. Eby wrote]
Then EasyInstall could create a 'unittest' script with a #! line on Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on Windows. In each case, the generated program would simply load and run the entry point with no arguments.
Effbot has some good things to say about/use for this: http://effbot.org/zone/exemaker.htm
On Windows, I'd say that applications are pretty much always better as .exe's, whether console or GUI. The .py/.pyw form is dependent on a single global consistent version of Python, but it's possible and reasonable to have multiple Python versions installed. An .exe also has a lot more control over how Python is initialized, and that can be particularly important for applications. On the other hand, in the short run I can also see using .bat or .cmd files for console apps, and .pyw for GUI apps, just to have something that works and wait for the path management use cases for various .exe options to work themselves out.
Which path management? Do you mean the default distutils build directory (which we discussed some time ago)?
There is a bug in the current cmd.exe (or maybe it is only up to Win2k? can't remember) where shell redirection doesn't work if your script is launched indirectly via a .bat of .cmd file. As well, Ctrl+C signal handling for kill a script is quite annoying if launched via a .bat file: you get a secondary question from the shell
Terminate batch file (Y/N)?or something like that. A .exe launcher/stub is definitely preferred.
Isn't the shell redirection problem also present if the .py script is run via the shell association?
The Ctrl+C behaviour annoys me also because I usually run my scripts via batch files (py23.bat, py24.bat, py_d.bat and so on). Maybe this problem can be avoided by copying the python.exe's somewhere to my path, with names p23.exe, p24.exe and so on. They only contain a pointer to the actual PythonXY.dll name, afaik. So I don't have to update p24.exe when I install 2.4.2 over my 2.4.1.
Somewhat similar to the exemaker approach, but using the name of the exe to specify the Python version instead of a '-i ...' command line option, or the #! line in the script.
The only missing piece so far, for me at least, is that it's not possible to use dotted module names with the -m option:
p24 -m ctypes.wrap.h2xml windows.h
Thomas
Kevin Dangoor wrote:
On 9/15/05, Phillip J. Eby pje@telecommunity.com wrote:
As for Mac OS, I have almost no experience with it, so I'm not sure what GUI applications there need. Does everything need py2app? If you have a wx-based app, would you just make a #! script? Bob Ippolito previously mentioned that you don't "install" applications there, that people just drag applications wherever they want them rather than using shortcuts, so at least that part isn't a problem. :)
GUI apps on the Mac need py2app.
It depends. wx-based programs don't have a technical need to be in a .app bundle. They will run fine from the command line. Now, the *user* might want it packaged in a .app bundle, and we as developers might need to respond to that desire.
I think that PyObjC apps might actually need to be in a .app bundle to work reliably, though.
I have a feeling that there isn't a pressing need to be able to install .app bundles from easy_install. If the package maintainer is going to go to the trouble of writing a setup.py that can be used with py2app, he'll almost certainly actually build the .app binary and distribute it. Since they're standalone, they don't really interact with the other installed packages.
At 11:10 AM 9/15/2005 -0700, Robert Kern wrote:
I have a feeling that there isn't a pressing need to be able to install .app bundles from easy_install. If the package maintainer is going to go to the trouble of writing a setup.py that can be used with py2app, he'll almost certainly actually build the .app binary and distribute it. Since they're standalone, they don't really interact with the other installed packages.
If you can run a wx or tkinter (or other cross-platform GUI) app on Mac OS without needing anything special, but users would prefer to have the .app, it would probably be a good idea to have it, even if the original developer hasn't targeted that platform specifically. For example, packages like idle or the pydoc tkinter GUI would seem to merit something like this.
Phillip J. Eby wrote:
At 11:10 AM 9/15/2005 -0700, Robert Kern wrote:
I have a feeling that there isn't a pressing need to be able to install .app bundles from easy_install. If the package maintainer is going to go to the trouble of writing a setup.py that can be used with py2app, he'll almost certainly actually build the .app binary and distribute it. Since they're standalone, they don't really interact with the other installed packages.
If you can run a wx or tkinter (or other cross-platform GUI) app on Mac OS without needing anything special, but users would prefer to have the .app, it would probably be a good idea to have it, even if the original developer hasn't targeted that platform specifically. For example, packages like idle or the pydoc tkinter GUI would seem to merit something like this.
Oh, certainly it would be nice. But AFAICT, you really do have to write a setup.py specifically for py2app. The information you need isn't something you can simply divine from a generic setup.py .
[Thomas Heller wrote]
There is a bug in the current cmd.exe (or maybe it is only up to Win2k? can't remember) where shell redirection doesn't work if your script is launched indirectly via a .bat of .cmd file. As well, Ctrl+C signal handling for kill a script is quite annoying if launched via a .bat file: you get a secondary question from the shell
Terminate batch file (Y/N)?or something like that. A .exe launcher/stub is definitely preferred.
Isn't the shell redirection problem also present if the .py script is run via the shell association?
Oh shoot, I think I was wrong and you are right: the shell redirection problem is for the case of using the PATHEXT stuff. I.e. I think shell redirection works fine with a .bat or .cmd stub. Sorry about causing confusion.
The Ctrl+C behaviour annoys me also because I usually run my scripts via batch files (py23.bat, py24.bat, py_d.bat and so on). Maybe this problem can be avoided by copying the python.exe's somewhere to my path, with names p23.exe, p24.exe and so on. They only contain a pointer to the actual PythonXY.dll name, afaik. So I don't have to update p24.exe when I install 2.4.2 over my 2.4.1.
I tend to copy python.exe to pythonXY.exe (as appropriate) after I install a new Python. I'm considering doing this by default in the ActivePython installers I make. (Similar to the pythonX.Y executable copies that get made on Linux/Un*x.) What do you think?
Somewhat similar to the exemaker approach, but using the name of the exe to specify the Python version instead of a '-i ...' command line option, or the #! line in the script.
Agreed.
The only missing piece so far, for me at least, is that it's not possible to use dotted module names with the -m option:
p24 -m ctypes.wrap.h2xml windows.h
That's too bad. I didn't know "-m" didn't support that... but then I haven't really used it that much yet.
Trent
On 9/16/05, Trent Mick trentm@activestate.com wrote:
The only missing piece so far, for me at least, is that it's not possible to use dotted module names with the -m option:
p24 -m ctypes.wrap.h2xml windows.h
That's too bad. I didn't know "-m" didn't support that... but then I haven't really used it that much yet.
There's a patch on SF to implement that (1043356 ) and a corresponding PEP (338), but I don't know what's required to get it added.
Another annoyance with -m is that it doesn't support modules in zipfiles (e.g., zipped eggs). I submitted a bug for that one (1250389) but I don't have a C build environment that I can use to try out any form of fix - so I'm relying on someone else's goodwill to look into it...
Paul.
At 11:19 AM 9/15/2005 -0400, Phillip J. Eby wrote:
Every so often, there's a perennial debate on the distutils-sig about script filenames. Should they have .py extensions or not? What about .pyw apps on Windows? What about Mac OS? and so on.
...
So, if these "main" functions were simply declared as entry points in the project's setup script, then EasyInstall could automatically generate stub scripts for them, in a platform-appropriate fashion, with no need for '-m', "__name__=='__main__'", or fiddling with file extensions. For example, if PyUnit were distributed as an egg, with the following entry points:
[distutils.console_apps] unittest = unittest:mainThen EasyInstall could create a 'unittest' script with a #! line on Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on Windows. In each case, the generated program would simply load and run the entry point with no arguments.
FYI, I've now implemented console script generation in the CVS version of setuptools, except that I decided to call the entry point group "console_scripts". There is a custom .exe launcher for Windows, too. The whole thing was surprisingly easy to implement, and the docs are updated in CVS as well. I'll probably implement the find_package_data() thing this weekend as well, and put it out as an 0.6a2 release.
Note that generated scripts can't be run with -m, but then again neither can a lot of things. ;) Using -m scripts means your eggs can't be installed compressed anyway, so these "new-style" scripts are superior in almost every way. You don't need an if __name__=='__main__', you don't need separate scripts, just write your "main" function in a module somewhere and list the entry point in your setup().
Anyway, for backward-compatibility I'm still allowing "python -m easy_install" in the next release or two, but a warning is printed to the console. Later, this feature will be removed so that it's possible to install setuptools in compressed form.
At 10:00 PM 9/16/2005 +0100, Paul Moore wrote:
On 9/16/05, Trent Mick trentm@activestate.com wrote:
The only missing piece so far, for me at least, is that it's not possible to use dotted module names with the -m option:
p24 -m ctypes.wrap.h2xml windows.h
That's too bad. I didn't know "-m" didn't support that... but then I haven't really used it that much yet.
There's a patch on SF to implement that (1043356 ) and a corresponding PEP (338), but I don't know what's required to get it added.
Another annoyance with -m is that it doesn't support modules in zipfiles (e.g., zipped eggs).
Really, the only reasonable way to solve both problems is something like 'python -m run foo.bar ....'. That is, have a bootstrap module to do the rest. (Or alternately, have -m fallback to using the bootstrap module if it's unsuccessful.)
However, after reflection, I think now that -m probably only really makes sense for stdlib modules, since projects using setuptools can now get all the benefits of -m without any of the drawbacks, without even writing any __name__=='__main__' code. For example defining an entry point thus:
[console_scripts] h2xml = ctypes.wrap.h2xml:main
automatically creates 'h2xml.exe' on Windows and a #!-prefixed 'h2xml' script on any other platform. (Using the current CVS version of setuptools, that is.)
On 15-sep-2005, at 20:10, Robert Kern wrote:
Kevin Dangoor wrote:
On 9/15/05, Phillip J. Eby pje@telecommunity.com wrote:
As for Mac OS, I have almost no experience with it, so I'm not sure what GUI applications there need. Does everything need py2app? If you have a wx-based app, would you just make a #! script? Bob Ippolito previously mentioned that you don't "install" applications there, that people just drag applications wherever they want them rather than using shortcuts, so at least that part isn't a problem. :)
GUI apps on the Mac need py2app.
It depends. wx-based programs don't have a technical need to be in a .app bundle. They will run fine from the command line. Now, the *user* might want it packaged in a .app bundle, and we as developers might need to respond to that desire.
IIRC you need to use bundles if you want full control over the menu contents. But I haven't used wx on the mac for quite a while.
I think that PyObjC apps might actually need to be in a .app bundle to work reliably, though.
PyObjC apps do need to be in a .app bundle, that's needed to find resources, such as NIB files, using the Cocoa API's.
I have a feeling that there isn't a pressing need to be able to install .app bundles from easy_install. If the package maintainer is going to go to the trouble of writing a setup.py that can be used with py2app, he'll almost certainly actually build the .app binary and distribute it. Since they're standalone, they don't really interact with the other installed packages.
Think big :-). It would be nice if we'd end up with a generic method for describing how to build standalone apps that py2app and py2exe could hook into. You currently have to write slightly different code on windows and osx to get a standalone app.
Ronald
On 9/17/05, Phillip J. Eby pje@telecommunity.com wrote:
However, after reflection, I think now that -m probably only really makes sense for stdlib modules, since projects using setuptools can now get all the benefits of -m without any of the drawbacks, without even writing any __name__=='__main__' code.
One thing you get with -m is that the module doesn't have to be on PATH. Where does setuptools install its wrapper executables? (The usual \Python24\Scripts directory isn't added to PATH by the installer).
I know, I'm nitpicking. Sorry :-)
Paul.
Paul Moore p.f.moore@gmail.com writes:
On 9/17/05, Phillip J. Eby pje@telecommunity.com wrote:
However, after reflection, I think now that -m probably only really makes sense for stdlib modules, since projects using setuptools can now get all the benefits of -m without any of the drawbacks, without even writing any __name__=='__main__' code.
One thing you get with -m is that the module doesn't have to be on PATH. Where does setuptools install its wrapper executables? (The usual \Python24\Scripts directory isn't added to PATH by the installer).
I know, I'm nitpicking. Sorry :-)
No, you're not. IMO. setuptools installs them in PythonXY\Scripts.
BTW: Shouldn't 'setup.py develop --uninstall' remove them again? It doesn't.
And yet another question: How should my setup-script start? Is this the correct way:
""" from ez_setup import use_setuptools use_setuptools()
from distutils.core import setup """
I'm considering to finally use setuptools in the comtypes setup script.
Thomas
At 03:07 PM 9/23/2005 +0200, Thomas Heller wrote:
Paul Moore p.f.moore@gmail.com writes:
On 9/17/05, Phillip J. Eby pje@telecommunity.com wrote:
However, after reflection, I think now that -m probably only really makes sense for stdlib modules, since projects using setuptools can now get all the benefits of -m without any of the drawbacks, without even writing any __name__=='__main__' code.
One thing you get with -m is that the module doesn't have to be on PATH. Where does setuptools install its wrapper executables? (The usual \Python24\Scripts directory isn't added to PATH by the installer).
I know, I'm nitpicking. Sorry :-)
No, you're not. IMO. setuptools installs them in PythonXY\Scripts.
Unless you set --install-scripts somewhere else, which you can do in a configuration file. To the greatest extent possible, I'm trying to have setuptools minimize surprises with respect to installation locations, by conforming to the active distutils configuration.
BTW: Shouldn't 'setup.py develop --uninstall' remove them again? It doesn't.
setuptools has hardly any uninstallation capabilities as yet.
And yet another question: How should my setup-script start? Is this the correct way:
""" from ez_setup import use_setuptools use_setuptools()
from distutils.core import setup
It's a way that works. I personally use 'from setuptools import setup' and whatever else I need to import, but whatever you prefer is fine. setuptools patches itself into the distutils core. It used to not do that, but then I realized that the way py2exe and other distutils extensions patch themselves in, you would lose setuptools functionality unless I patched myself in also. Hopefully in future everybody will use setuptools' extension functionality so that patching will be unnecessary.
Where does setuptools install its wrapper executables? (The usual \Python24\Scripts directory isn't added to PATH by the installer).
setuptools installs them in PythonXY\Scripts.
Unless you set --install-scripts somewhere else, which you can do in a configuration file. To the greatest extent possible, I'm trying to have setuptools minimize surprises with respect to installation locations, by conforming to the active distutils configuration.
Does anybody else think that installing to PythonXY\Scripts (instead of to PythonXY) is broken? With Perl and Ruby, for example, scripts from a 3rd party package will be installed next to the main interpreter binary (i.e. on the PATH) on all platforms. Would having setuptools (and changing distutils) to install scripts next to python.exe wreak unwarranted havoc?
Cheers, Trent
At 03:45 PM 9/23/2005 -0700, Trent Mick wrote:
Where does setuptools install its wrapper executables? (The usual \Python24\Scripts directory isn't added to PATH by the installer).
setuptools installs them in PythonXY\Scripts.
Unless you set --install-scripts somewhere else, which you can do in a configuration file. To the greatest extent possible, I'm trying to have setuptools minimize surprises with respect to installation locations, by conforming to the active distutils configuration.
Does anybody else think that installing to PythonXY\Scripts (instead of to PythonXY) is broken?
Windows doesn't really have any obvious standard place to put this sort of thing, unless you consider the Windows\Command directory.
With Perl and Ruby, for example, scripts from a 3rd party package will be installed next to the main interpreter binary (i.e. on the PATH) on all platforms. Would having setuptools (and changing distutils) to install scripts next to python.exe wreak unwarranted havoc?
Sadly, yes. First of all, python.exe isn't *on* the PATH on Windows unless you put it there yourself. Second, the python.exe directory is on sys.path, so it would turn your scripts into modules, conflicting with any same-named modules.
So, yes, I'm afraid "havoc" is the correct word, though it would certainly be nice if someone would step up and prove me wrong here.
[Phillip J. Eby wrote]
With Perl and Ruby, for example, scripts from a 3rd party package will be installed next to the main interpreter binary (i.e. on the PATH) on all platforms. Would having setuptools (and changing distutils) to install scripts next to python.exe wreak unwarranted havoc?
Sadly, yes. First of all, python.exe isn't *on* the PATH on Windows unless you put it there yourself.
It gets on the PATH with ActivePython <wink>. I'd recommend that the python.org Python installer do the same, but I seem to remember MvL (and others?) stating their preference not to do so (don't recall why).
In any case, I'd say it is extremely *common* for Windows users to put the Python install dir on their PATH -- just as it is common for Windows Perl and Ruby users to put the interpreter dir on their PATH. I'd then say that the obvious place to put user scripts installed via distutils is that same directory.
Second, the python.exe directory is on sys.path, so it would turn your scripts into modules, conflicting with any same-named modules.
That never occured to me that that could be a problem. I have a few scripts that I've been distributing for a while and haven't had a particular issue... because I tend to write so that my scripts are (somewhat) useful as modules as well.
But you are right, some people could get surprised by this. Hrm. Either (1) this behaviour (Python putting the interpreter dir on sys.path) is just broken or (2) it was an intentional design and script-writers should be expected to ensure their scripts either work reasonably or at least don't cause damage when imported. Since the directory with the interpreter is NOT on sys.path on other platforms (Linux, Un*x) then I'd say this is just broken behaviour.
Anyone know, off hand, *why* the Python interpreter dir is on sys.path on Windows?
Trent
Trent Mick trentm@ActiveState.com writes:
[Phillip J. Eby wrote]
With Perl and Ruby, for example, scripts from a 3rd party package will be installed next to the main interpreter binary (i.e. on the PATH) on all platforms. Would having setuptools (and changing distutils) to install scripts next to python.exe wreak unwarranted havoc?
Sadly, yes. First of all, python.exe isn't *on* the PATH on Windows unless you put it there yourself.
It gets on the PATH with ActivePython <wink>. I'd recommend that the python.org Python installer do the same, but I seem to remember MvL (and others?) stating their preference not to do so (don't recall why).
In any case, I'd say it is extremely *common* for Windows users to put the Python install dir on their PATH -- just as it is common for Windows Perl and Ruby users to put the interpreter dir on their PATH. I'd then say that the obvious place to put user scripts installed via distutils is that same directory.
Well, I don't think so. It *may* be common for windows users who use the command line ;-), but I would guess that's a minority anyway.
Even though I *use* the command line, I've never put python.exe on my PATH. It's easier to type 'myscript' or 'myscript.py' than 'python myscript.py' anyway. Then, I have multiple versions installed on my machine, and to select between 2.3.5, 2.4.1, and the CVS version I can always use 'py23 myscript.py', 'py24 myscript.py', 'py myscript.py' and 'py_d myscript.py' - the latter to run it with the debug compiled CVS version. I had created batchfiles py23.bat and so on which specify the directory of the python.exe or python_d.exe files.
And this also explains why I'm still waiting for the extented version of the '-m' command line option and did not bother to put the PythonXY\Scripts directory on the PATH - I would loose the ability to easily switch versions.
Thomas
[Trent]
In any case, I'd say it is extremely *common* for Windows users to put the Python install dir on their PATH -- just as it is common for Windows Perl and Ruby users to put the interpreter dir on their PATH. I'd then say that the obvious place to put user scripts installed via distutils is that same directory.
[Thomas Heller wrote]
Well, I don't think so. It *may* be common for windows users who use the command line ;-), but I would guess that's a minority anyway.
A minority of Windows users of Python use the command line? Really? Anyone coming from Unix will immediately go there. Last I remember "Learning Python" and the Python tutorial and other intro texts encourage the user to open the interactive shell at the command line to play around. You may be right, though. Even if it is "a minority" I suspect that is still a big number.
Even for that set of Windows users that *do* use the command line, having Python get on their PATH automatically just makes sense:
It allows other systems that depend on Python to be able to just work after an installation. For example, the Mozilla build system on Windows requires an installation of Perl. The build instructions can just say "install ActivePerl" (which puts Perl on the user's PATH) and thereafter the build system can easily find the Perl is needs. I bet that systems out there requiring Python have more difficulty.
It allows tutorials/introductory-texts to have a lowest common denominator that just works for newbies on all platforms: 1. install Python 2. type "python script.py" Then they can get on to all the subtle platform-specific bells and whistles, viz:
Even though I *use* the command line, I've never put python.exe on my PATH. It's easier to type 'myscript' or 'myscript.py' than 'python myscript.py' anyway.
That is presuming that the Windows installer set up a .py file association and put .py;.pyc;... in the PATHEXT env. var.
Does putting an installed Python on the PATH cause damage in any way?
Then, I have multiple versions installed on my machine, and to select between 2.3.5, 2.4.1, and the CVS version I can always use 'py23 myscript.py', 'py24 myscript.py', 'py myscript.py' and 'py_d myscript.py' - the latter to run it with the debug compiled CVS version. I had created batchfiles py23.bat and so on which specify the directory of the python.exe or python_d.exe files.
A week or so ago I asked you what you thought of the idea of having the Windows build (and installers) have a "pythonXY.exe" or "pythonX.Y.exe" along with the usual "python.exe". This would match what is done on other platforms to allow multiple versions on Python on the PATH and still be able pick a specific one. (I'm not sure if you replied.) I do something similar to your "py23", "py24", ... thing. A common mechanism for this would be nice.
And this also explains why I'm still waiting for the extented version of the '-m' command line option and did not bother to put the PythonXY\Scripts directory on the PATH - I would loose the ability to easily switch versions.
And gain a subtle python-specific way to launch a script.
I realize I am ranting above, but I just think it is crazy to have the situation these days where, for example, I download and install pychecker and then try to run it:
C:\trentm\tmp>pychecker 'pychecker' is not recognized as an internal or external command, operable program or batch file.
I understand your wanting a mechanism to easily be able to run any Python scripts/modules you have with any Python installation/build you have, but the common user case is just: (1) I have my latest Python installation, (2) I want to run something installed with "python setup.py install"
Trent
participants (8)
-
Ian Bicking -
Kevin Dangoor -
Paul Moore -
Phillip J. Eby -
Robert Kern -
Ronald Oussoren -
Thomas Heller -
Trent Mick