Re: [Distutils] Adding entry points into Distutils ?
At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
1. Different versions of Python conflict with previous versions of console scripts. Take paste for example.
I don't understand what you mean.
2. The entry point mechanism IIRC recursively scans the site-packages directory and loads up the system path with eggs. This is too expensive of an operation for the current environment I work in.
The scan is not recursive; only files that are actually *on* sys.path are scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info in a directory that is directly on sys.path. Now, it's possible that some application you are using does such a scan explicitly; I'm just noting that merely querying or loading entry points doesn't cause any recursive scans, and it most definitely does not add anything new to sys.path, unless the entry point to be loaded has declared an additional dependency that's *not* on sys.path yet.
3. There doesn't seem to be a clean way to inject user specific environment details to the console script. I often need the ability to alter the sys.path in a user specific way for the entry point without needing to mess up the global sys.path permanently.
I don't understand what you mean here, either.
resending, as I accidently only sent to PJE On Fri, May 8, 2009 at 11:48 AM, Noah Gift <noah.gift@gmail.com> wrote:
On Fri, May 8, 2009 at 11:24 AM, P.J. Eby <pje@telecommunity.com> wrote:
At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
1. Different versions of Python conflict with previous versions of console scripts. Take paste for example.
I don't understand what you mean.
Sorry that was a bit curt.
One of the problems with Pylons installation and virtualenv is that you might have previously installed paste script, and that could have been triggered to a specific version of paste like this:
#!/vol/apps/python-2.5.1_64/bin/python # EASY-INSTALL-ENTRY-SCRIPT: 'PasteScript==1.7.3','console_scripts','paster' __requires__ = 'PasteScript==1.7.3' import sys from pkg_resources import load_entry_point
sys.exit( load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
if you do a which paste, you will see this:
CSH#ngift@naseberry# which paste /usr/bin/paste
What this means is the you could really want some theoretical new version of the paste script in your virtualenv, or in a standard python site-packages directory, and or a different version of python, say 2.6, but the name of the script hides which actual version you want and it specifically chooses one version of Python. There isn't that good of a solution to solve this, but easy_install itself, seems to "do the right thing", but appending the actual python version to the script name.
I think the idea way is that one bootstrap could work across all version of python so that python versions wouldn't need to be appended to the script name. Additionally, it could be useful, but I have no idea how this would work, to have the bootstrap dynamically pick the right version of itself that it needs to run in a given context.
2. The entry point mechanism IIRC recursively scans the site-packages directory and loads up the system path with eggs. This is too expensive of an operation for the current environment I work in.
The scan is not recursive; only files that are actually *on* sys.path are scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info in a directory that is directly on sys.path.
I could be wrong, as this is from memory, but I believe that each egg inside of site-packages gets seperately injected in sys.path. I suppose this is the only way to deal with package versions cleanly. I have noticed that whole process is reasonably expensive.
Now, it's possible that some application you are using does such a scan explicitly; I'm just noting that merely querying or loading entry points doesn't cause any recursive scans, and it most definitely does not add anything new to sys.path, unless the entry point to be loaded has declared an additional dependency that's *not* on sys.path yet.
3. There doesn't seem to be a clean way to inject user specific environment details to the console script. I often need the ability to alter the sys.path in a user specific way for the entry point without needing to mess up the global sys.path permanently.
I don't understand what you mean here, either.
In film enviroments the whole way we work is by toggling between different enviroments. A developer, artists, etc, will need to work on a specific shot in a movie, and they also need to toggle between films, etc. Each of these enviroments is different because different code is being used. Keeping all possible combinations of python environments in your global environment can make the python interpeter grind to halt because of the way Python itself recursives over paths looking for files. The idea scenario, at least for my current environment, is for sys.path to "inject" a custom path or N paths at the actual run time of the command line tool, because it limits the scanning Python needs to do. Here is an example of my quick and dirty hack to make IPython run quicker:
if __name__ == "__main__": import sys sys.path = [ '/vol/apps/python-2.5.1_64/lib/python2.5', '/vol/apps/python-2.5.1_64/lib/python2.5/site-packages', '/vol/apps/python-2.5.1_64/lib/python2.5/lib-dynload/', '/vol/filmstudio/linux64/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg', '/vol/filmstudio/linux64/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-x86_64.egg', '/vol/filmstudio/linux64/lib/python2.5/site-packages/lxml-2.1.2-py2.5-linux-x86_64.egg', '/vol/filmstudio/linux64/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg', ]
import IPython.Shell IPython.Shell.start().mainloop()
On one hand, IPython is now very snappy and runs in a useable fashion. The problem now, is that all flexibility is lost to dynamically add more things based on the users environment. Perhaps by having the boostrap script look at some custom environmental variable to "inject" one more path, but just for that boostrap, in that given context, as we don't want to keep extra stuff around in the global PYTHONPATH and it is different for each user.
-- Cheers,
Noah
-- Cheers, Noah On Fri, May 8, 2009 at 11:24 AM, P.J. Eby <pje@telecommunity.com> wrote:
At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
1. Different versions of Python conflict with previous versions of console scripts. Take paste for example.
I don't understand what you mean.
2. The entry point mechanism IIRC recursively scans the site-packages directory and loads up the system path with eggs. This is too expensive of an operation for the current environment I work in.
The scan is not recursive; only files that are actually *on* sys.path are scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info in a directory that is directly on sys.path.
Now, it's possible that some application you are using does such a scan explicitly; I'm just noting that merely querying or loading entry points doesn't cause any recursive scans, and it most definitely does not add anything new to sys.path, unless the entry point to be loaded has declared an additional dependency that's *not* on sys.path yet.
3. There doesn't seem to be a clean way to inject user specific environment details to the console script. I often need the ability to alter the sys.path in a user specific way for the entry point without needing to mess up the global sys.path permanently.
I don't understand what you mean here, either.
-- Cheers, Noah
Noah Gift wrote:
I don't understand what you mean here, either. In film enviroments the whole way we work is by toggling between different enviroments. A developer, artists, etc, will need to work on a specific shot in a movie, and they also need to toggle between films, etc.
Have you guys looked at buildout? I know the docs suck, but it strikes me that it might solve a lot of the problems you're complaining about... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote:
Noah Gift wrote:
I don't understand what you mean here, either. In film enviroments the whole way we work is by toggling between different enviroments. A developer, artists, etc, will need to work on a specific shot in a movie, and they also need to toggle between films, etc.
Have you guys looked at buildout? I know the docs suck, but it strikes me that it might solve a lot of the problems you're complaining about...
The docs have gotten a lot better lately, thanks largely to the efforts of Baiju M, who has done a lot of the gardening to get the project site up and useful: http://www.buildout.org/ Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKBcT5+gerLs4ltQ4RAljeAJ9dnfZRiKLtt+9YgRHdpA0V0FxHlwCfWO7V 5EQmUxX+t2DBGnG7aguXvnQ= =dib9 -----END PGP SIGNATURE-----
participants (4)
-
Chris Withers -
Noah Gift -
P.J. Eby -
Tres Seaver