[Distutils] Adding entry points into Distutils ?

[Noah Gift]

resending, as I accidently only sent to PJE

On Fri, May 8, 2009 at 11:48 AM, Noah Gift <noah.gift at gmail.com> wrote:
> On Fri, May 8, 2009 at 11:24 AM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
>>>
>>> 1.  Different versions of Python conflict with previous versions of
>>> console scripts.  Take paste for example.
>>
>> I don't understand what you mean.
>
> Sorry that was a bit curt.
>
> One of the problems with Pylons installation and virtualenv is that
> you might have previously installed paste script, and that could have
> been triggered to a specific version of paste like this:
>
>
> #!/vol/apps/python-2.5.1_64/bin/python
> # EASY-INSTALL-ENTRY-SCRIPT: 'PasteScript==1.7.3','console_scripts','paster'
> __requires__ = 'PasteScript==1.7.3'
> import sys
> from pkg_resources import load_entry_point
>
> sys.exit(
>   load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
>
> if you do a which paste, you will see this:
>
> CSH#ngift at naseberry# which paste
> /usr/bin/paste
>
>
> What this means is the you could really want some theoretical new
> version of the paste script in your virtualenv, or in a standard
> python site-packages directory, and or a different version of python,
> say 2.6, but the name of the script hides which actual version you
> want and it specifically chooses one version of Python.  There isn't
> that good of a solution to solve this, but easy_install itself, seems
> to "do the right thing", but appending the actual python version to
> the script name.
>
> I think the idea way is that one bootstrap could work across all
> version of python so that python versions wouldn't need to be appended
> to the script name.  Additionally, it could be useful, but I have no
> idea how this would work, to have the bootstrap dynamically pick the
> right version of itself that it needs to run in a given context.
>
>
>
>>
>>> 2.  The entry point mechanism IIRC recursively scans the site-packages
>>> directory and loads up the system path with eggs.  This is too
>>> expensive of an operation for the current environment I work in.
>>
>> The scan is not recursive; only files that are actually *on* sys.path are
>> scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info
>> in a directory that is directly on sys.path.
>
> I could be wrong, as this is from memory, but I believe that each egg
> inside of site-packages gets seperately injected in sys.path.  I
> suppose this is the only way to deal with package versions cleanly.  I
> have noticed that whole process is reasonably expensive.
>>
>> Now, it's possible that some application you are using does such a scan
>> explicitly; I'm just noting that merely querying or loading entry points
>> doesn't cause any recursive scans, and it most definitely does not add
>> anything new to sys.path, unless the entry point to be loaded has declared
>> an additional dependency that's *not* on sys.path yet.
>>
>>
>>> 3.  There doesn't seem to be a clean way to inject user specific
>>> environment details to the console script.
>>> I often need the ability to alter the sys.path in a user specific way
>>> for the entry point without needing to mess up the global sys.path
>>> permanently.
>>
>> I don't understand what you mean here, either.
>
> In film enviroments the whole way we work is by toggling between
> different enviroments.  A developer, artists, etc, will need to work
> on a specific shot in a movie, and they also need to toggle between
> films, etc.  Each of these enviroments is different because different
> code is being used. Keeping all possible combinations of python
> environments in your global environment can make the python interpeter
> grind to halt because of the way Python itself recursives over paths
> looking for files.  The idea scenario, at least for my current
> environment, is for sys.path to "inject" a custom path or N paths at
> the actual run time of the command line tool, because it limits the
> scanning Python needs to do.  Here is an example of my quick and dirty
> hack to make IPython run quicker:
>
> if __name__ == "__main__":
>    import sys
>    sys.path = [
>     '/vol/apps/python-2.5.1_64/lib/python2.5',
>     '/vol/apps/python-2.5.1_64/lib/python2.5/site-packages',
>     '/vol/apps/python-2.5.1_64/lib/python2.5/lib-dynload/',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-x86_64.egg',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/lxml-2.1.2-py2.5-linux-x86_64.egg',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg',
>     ]
>
>    import IPython.Shell
>    IPython.Shell.start().mainloop()
>
>
> On one hand, IPython is now very snappy and runs in a useable fashion.
> The problem now, is that all flexibility is lost to dynamically add
> more things based on the users environment.  Perhaps by having the
> boostrap script look at some custom environmental variable to "inject"
> one more path, but just for that boostrap, in that given context, as
> we don't want to keep extra stuff around in the global PYTHONPATH and
> it is different for each user.
>
>>
>>
>
>
>
> --
> Cheers,
>
> Noah
>



-- 
Cheers,

Noah



On Fri, May 8, 2009 at 11:24 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
>>
>> 1.  Different versions of Python conflict with previous versions of
>> console scripts.  Take paste for example.
>
> I don't understand what you mean.
>
>> 2.  The entry point mechanism IIRC recursively scans the site-packages
>> directory and loads up the system path with eggs.  This is too
>> expensive of an operation for the current environment I work in.
>
> The scan is not recursive; only files that are actually *on* sys.path are
> scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info
> in a directory that is directly on sys.path.
>
> Now, it's possible that some application you are using does such a scan
> explicitly; I'm just noting that merely querying or loading entry points
> doesn't cause any recursive scans, and it most definitely does not add
> anything new to sys.path, unless the entry point to be loaded has declared
> an additional dependency that's *not* on sys.path yet.
>
>
>> 3.  There doesn't seem to be a clean way to inject user specific
>> environment details to the console script.
>> I often need the ability to alter the sys.path in a user specific way
>> for the entry point without needing to mess up the global sys.path
>> permanently.
>
> I don't understand what you mean here, either.
>
>



-- 
Cheers,

Noah