Re: [Distutils] RFC : PEP 376 - egg.info
At 12:03 PM 5/3/2009 +0200, Tarek Ziadé wrote:
The name of each file will have to be normalized: all upper case with no extensions.
Any opinions ?
I don't see any point to the normalization. However, being able to install arbitrary files in .egg-info is currently supported by setuptools, and that capability is used by e.g. the EggTranslations project. I would therefore suggest that perhaps the ability to specify file trees for egg-info inclusion would be better, if the goal is to eventually replace setuptools.
2009/5/3 P.J. Eby <pje@telecommunity.com>:
At 12:03 PM 5/3/2009 +0200, Tarek Ziadé wrote:
The name of each file will have to be normalized: all upper case with no extensions.
Any opinions ?
I don't see any point to the normalization.
To avoid different naming conventions like: PKG-INFO, requires.txt, SOURCES.txt
However, being able to install arbitrary files in .egg-info is currently supported by setuptools, and that capability is used by e.g. the EggTranslations project.
I would therefore suggest that perhaps the ability to specify file trees for egg-info inclusion would be better,
Are you talking about the directory that is built by egg_info in setuptools ? or something else ? I mean, do we want to have subdirectories in .egg-info ?
if the goal is to eventually replace setuptools.
The goal is to provide an install_egg_info command that can be configured without the need to replace/override it. I guess, this can be done by taking part of the code done in setuptools install_egg_info and provide a way for setuptools or another project to add more files through configuration. Now, if entry_points were added in Distutils, things would be more simple to extend every command I guess. (like you have pointed) That's some other work we started here : http://wiki.python.org/moin/Distutils/PluginSystem But my mail to python-ideas about adding a plugin system into Python didn't get a lot of interest. Maybe this straight-forward sentence would work better: "let's add entry points in Distutils" ? Tarek -- Tarek Ziadé | http://ziade.org
2009/5/3 P.J. Eby <pje@telecommunity.com>:
At 12:03 PM 5/3/2009 +0200, Tarek Ziadé wrote:
The name of each file will have to be normalized: all upper case with no extensions.
Any opinions ?
I don't see any point to the normalization. However, being able to install arbitrary files in .egg-info is currently supported by setuptools, and that capability is used by e.g. the EggTranslations project.
I am discovering a new callback mechanism here, I am confused. I was looking at another package last week called "eggtestinfo", that uses the 'egg_info.writers' entry point to add package in the egg-info dir created by the egg_info command So the callbacks are used at installation time and the entry points only at packaging time, but I don't understand how EggTranslations is used since it registers callback only when the code is called Furtermore, if we provide the ability to fill egg-info with third party packages registered through a plugin system, it make sense to prepare it at packaging time to avoid having to install this third party package on the target system. So 'egg_info.writers' + the egg_info command is what makes sense right now to me. Tarek -- Tarek Ziadé | http://ziade.org
Hi Tarek, On Tue, 5 May 2009 01:37:34 +0200, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Furtermore, if we provide the ability to fill egg-info with third party packages registered through a plugin system, it make sense to prepare it at packaging time to avoid having to install this third party package on the target system.
I'm trying to understand why there is a need for a "new" plug-in system. Isn't the site-packages directory already a very simple and effective plug-in system? Does not the .PTH file provision... with it's "import .." prefix capability, allow any code to be run through the .PTH? I'm not suggesting don't change anything. I'm just wondering about the implications on my own project and trying to support both old and new installations. A lot of windows packages just come with their own windows installer... how would your proposed system work with these? Regards David
On Tue, May 5, 2009 at 1:47 AM, David Lyon <david.lyon@preisshare.net> wrote:
Hi Tarek,
On Tue, 5 May 2009 01:37:34 +0200, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Furtermore, if we provide the ability to fill egg-info with third party packages registered through a plugin system, it make sense to prepare it at packaging time to avoid having to install this third party package on the target system.
I'm trying to understand why there is a need for a "new" plug-in system.
Isn't the site-packages directory already a very simple and effective plug-in system?
Does not the .PTH file provision... with it's "import .." prefix capability, allow any code to be run through the .PTH?
This is not a registery that indexes code under specific markers : you are not able for instance to ask "give me all classes that are filling the .egg-info dir so I can use them on this 'foo' package" for this you need a system that let you register your classes under the 'write-into-egg-info' group, with an unique name for each, no matter where the class is located in your package. That is what entry points are providing : the ability to mark a code locate anywhere in your installation and to load it when needed in your execution context.
I'm not suggesting don't change anything. I'm just wondering about the implications on my own project and trying to support both old and new installations.
A lot of windows packages just come with their own windows installer... how would your proposed system work with these?
I don't think it affects any of these topics. Tarek -- Tarek Ziadé | http://ziade.org
On Tue, 5 May 2009 01:58:57 +0200, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
That is what entry points are providing : the ability to mark a code locate anywhere in your installation and to load it when needed in your execution context.
From my reading, it already does just that every time the
ok - but don't we already have this in site.py ? inside the interpreter. python interpreter starts up. David
On Tue, May 5, 2009 at 2:00 AM, David Lyon <david.lyon@preisshare.net> wrote:
On Tue, 5 May 2009 01:58:57 +0200, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
That is what entry points are providing : the ability to mark a code locate anywhere in your installation and to load it when needed in your execution context.
ok - but don't we already have this in site.py ? inside the interpreter.
From my reading, it already does just that every time the python interpreter starts up.
I am not sure to understand what you are explaining - when the python interpreter starts up, it doesn't load every installed package in memory. the loading happens when you do "import foo" But entry points provides a way to tell you for example that the "bar" function located in the "foo" module is a plugin, *without having to load this module in your interpreter*, because this info is writtent in a static text file located in the .egg-info directory. After, you will eventually load it (by using a load() function in the entry point, that is basically an import statement) Tarek
David
-- Tarek Ziadé | http://ziade.org
On Tue, 5 May 2009 02:11:36 +0200, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I am not sure to understand what you are explaining - when the python interpreter starts up, it doesn't load every installed package in memory. the loading happens when you do "import foo"
Correct. I didn't say it loads them all. The code however seems to load up all the names of the packages from packages along the python-path. It then executes any line starting with 'import'. --- http://svn.python.org/projects/python/trunk/Lib/site.py ----- def addpackage(sitedir, name, known_paths): """Process a .pth file within the site-packages directory: For each line in the file, either combine it with sitedir to a path and add that to known_paths, or execute it if it starts with 'import '. """ if known_paths is None: _init_pathinfo() reset = 1 else: reset = 0 fullname = os.path.join(sitedir, name) try: f = open(fullname, "rU") except IOError: return with f: for line in f: if line.startswith("#"): continue if line.startswith(("import ", "import\t")): exec line continue line = line.rstrip() dir, dircase = makepath(sitedir, line) if not dircase in known_paths and os.path.exists(dir): sys.path.append(dir) known_paths.add(dircase) if reset: known_paths = None return known_paths -------------------------------------------------------------------------
*without having to load this module in your interpreter*, because this info is writtent in a static text file located in the .egg-info directory. After, you will eventually load it (by using a load() function in the entry point, that is basically an import statement)
Yes .. that perfectly describes existing functionality.
But entry points provides a way to tell you for example that the "bar" function located in the "foo" module is a plugin,
I see now what you are getting at... But packages along the site-packages directory are already a form of a plug-in system. It just doesn't have the modern name. The code is already implemented as shown above.. it all works "today".. What really is missing is the sophisticated API to properly manage it all. There's no way to find out what's installed... remove anything.. The whole system is very brittle... I think it's possible to come up with something "new" by just addressing many of the major deficiencies that already exist in distutils. For me personally - I'm a big admirer of the existing code.. problem to me seems to be there's just not enough of it.. David
participants (3)
-
David Lyon -
P.J. Eby -
Tarek Ziadé