Distribute sprint #1 wrapup
As promised, here's a quick wrapup of the work started during the first sprint, on Distribute 0.7 Thanks to all the people that have participated (we were 6 or 7 IIRC) If you have sprinted and see a mistake or something missing is the wrapup, please complete it; = The fate of 0.6 code = First of all, we've discussed what is happening to the 0.6 code and we've decided that it was better to keep it under the pillow as a reference code, and code everything from scratch. The only code that is going to be recycled is some parts of pkg_resources. = 0.7 philosophy = Distribute 0.7 will be a good Distutils citizen and wants to be its incubator. It will implement Distutils commands, without changing Distutils behavior as it is doing now. Another important point is that the Distribute development will be driven by the PEP work that is going on and drop most other concepts of Setuptools: - PEP 376 : a *single* installation standard - PEP 345 : Requires, Obsoletes, etc, fields - PEP 390 : platform-dependant markers for the metadata - PEP 382 : namespace packages - PEP 386 : a new version scheme Distribute will implement those PEPs by gathering all the prototypes that have been created in the past months and when possible, will try to push them in Distutils. The biggest drops we are making are: - the egg formats are removed in favor of a unique PEP 376 standard. - easy_install is removed, and Distribute will cooperate with the pip project What's unclear right now is what will be pushed in Distutils by the time Python 2.7 and 3.3 are out. = what was started during the sprint = We started to write Distutils commands: - develop command: * Created a skeleton command for develop * Waiting for work to complete on the install and build_egg_info commands before work can continue here. - build_egg_info command (in setuptools, it was called "egg_info", we've renamed it): * Rewrote this command using setuptools code as a model * Removed all SVN support code * Removed obsolete egg-info writers * Waiting on the manifest generation feature to finish this one * Will implement PEP 376 egg-info writers when possible - generate_manifest command. Not started yet, but will be used to generate the MANIFEST file, and provide a plugin system for VCS support. (not an implicit system like in Setuptools) - install command - building a PEP 376 compatible install command (by rewriting pkg_resources, using the PEP 376 prototype we have written some time ago) - distutils.extensions : lightweight entry points, based on the PEP 376 standard taken back from the "extensions" project - Currently working on testing strategy = next sprint = The next sprint will be next week-end, and will be announced here; Anyone can join ! Thanks Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与
On Sat, Oct 24, 2009 at 12:50 AM, Eric Smith <eric@trueblade.com> wrote:
Tarek Ziadé wrote:
<exciting stuff deleted>
What's unclear right now is what will be pushed in Distutils by the time Python 2.7 and 3.3 are out.
Why 3.3 and not 3.2?
Oops: that was 3.2 of course ! All releases that we should have next summer, Tarek
Note about pep376 :: .egg-info becomes a directory As explained earlier, the EggFormats standard from setuptools proposes two formats to install the metadata information of a distribution: * A self-contained directory that can be zipped or left unzipped and contains the distribution files and an .egg-info directory containing the metadata. * A distinct .egg-info directory located in the site-packages directory, with the metadata inside. This PEP proposes to keep just one format and make it the standard way to install the metadata of a distribution : a distinct .egg-info directory located in the site-packages directory, containing the metadata. There is something i don't understand there: Does it imply that having no access on site-packages prevent you from installing third party libraries as a limited user? For example, on shared hosting where you have limited privileges. Before, i could upload/build some libs and start from there using, for exemple, buildout to scan that directory. I make assumption that also the entry points are written along the metadata dir. That will say that even if i don't have access to the site-packages but i have played with sys.path to include my distribution, i won't have its entry points (as the egg-info is not there) ? Tarek Ziadé a écrit :
On Sat, Oct 24, 2009 at 12:50 AM, Eric Smith <eric@trueblade.com> wrote:
Tarek Ziadé wrote:
<exciting stuff deleted>
What's unclear right now is what will be pushed in Distutils by the time Python 2.7 and 3.3 are out. Why 3.3 and not 3.2?
Oops: that was 3.2 of course ! All releases that we should have next summer,
Tarek _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF
On Sat, Oct 24, 2009 at 7:57 AM, kiorky <kiorky@cryptelium.net> wrote:
This PEP proposes to keep just one format and make it the standard way to install the metadata of a distribution : a distinct .egg-info directory located in the site-packages directory, containing the metadata.
There is something i don't understand there: Does it imply that having no access on site-packages prevent you from installing third party libraries as a limited user?
As it s today, site-packages is just the default place, that is added and scan when Python starts. And you need root privileges to install packages there. There's another place where you can use when you are not root: per-user site packages. But those two place are just the places for "python-the-package-manager".
For example, on shared hosting where you have limited privileges. Before, i could upload/build some libs and start from there using, for exemple, buildout to scan that directory. I make assumption that also the entry points are written along the metadata dir. That will say that even if i don't have access to the site-packages but i have played with sys.path to include my distribution, i won't have its entry points (as the egg-info is not there) ?
PEP 376 comes with a set of tools that will allow you to install/uninstall distributions in an arbitrary site-packages folder, and play with them. So it basically makes almost no differences for tools like zc.buildout-the-package-manager that is tweaking sys.path in the generated scripts. I guess the simplest way will be to make the "eggs" directory a regular site-package like folder. It will even simplify the scripts because you will not have to add one entry per eggs there as it is today. What could be awesome is to see a branch of zc.buildout built against distribute 0.7 when it starts to be usable, to experiment this. zc.buildout is a package manager, so it makes it one of the target use case for PEP 376 and other changes we will provide in distribute. Tarek
Tarek Ziadé a écrit :
As it s today, site-packages is just the default place, that is added and scan when Python starts. And you need root privileges to install packages there.
There's another place where you can use when you are not root: per-user site packages.
But those two place are just the places for "python-the-package-manager".
Correct me if i am wrong but what i have understood is that the egg-info is not writed along with the "python code" but in another well known place. From then, how can i do the following as i don't want to be defaulted. 1. register directoy(ies) where to find distributions. 2. Register where to install the "python code" (new egg format). 3. Register where to install "egg-infos" 4. Be sure that both egg-infos from default places (system, home) AND my thirdparty place(?) are gathered together if possible. Or maybe just one place is possible ? In this case, we may need to provide means to assemble the whole installation to reproduce how things behave today. Maybe, we can make some additional writing on the PEP to make things clear on that point, i think that just things need to be clear. On the same subject, I have another reguard, It's for eggs actually released as only eggs on pypi, see [1]. How can i install them now ? Can we make wrappers to install them? I have at least two hosting services that give me access to /var/foo, but the user has no right either on python installation or its $HOME. [1] - http://pypi.python.org/pypi/Plomino/1.4
Tarek
-- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF
Whow sorry, Ignore my precedent mail, i missed the end of that email., i do not know why. Blame my eyes or thunderbird :/
PEP 376 comes with a set of tools that will allow you to install/uninstall distributions in an arbitrary site-packages folder, and play with them. So it basically makes almost no differences for tools like zc.buildout-the-package-manager that is tweaking sys.path in the generated scripts. Great (really) :) Entry points are working as well on those non standard places also, aren't they?
I guess the simplest way will be to make the "eggs" directory a regular site-package like folder. It will even simplify the scripts because you will not have to add one entry per eggs there as it is today. I don't think it is a really good idea because you will not have isolation at project level. If you have A, B, C in this alternative site-packages, it would mean that A, B, and C are importable. But when i install something with "eggs=A C", i want that my pythonpath contains A and C but no B.I think the buildout way to do that actually is not that bad :)
What could be awesome is to see a branch of zc.buildout built against distribute 0.7 Indeed. I ll rework the minitage recipes also to use pip and prepare work for distribute on a branch as soon a possible, i hope next we.
when it starts to be usable, to experiment this.
zc.buildout is a package manager, so it makes it one of the target use case for PEP 376 and other changes we will provide in distribute.
How about trying to make a mirror of pypi packages rebuilt in the distribute format. That will prevent the overhead for users to contact the maintainer in case or breakage, to the maintainer to have to repackage its already packaged things, and also for the user to have problems installing 'foo' even it is a very simple package that we could just provide a rebuilt version to him. I think about that mainly for distributions packaged only as eggs (in the actual format). We could maybe extract this list of packages with no sdist (not that much i think and the only one i know is plomino) and repackage them. That's very low priority, but it can be useful and one more other step which will make the transition as transparent as possible. So mainly because i did not find time yet to test, and i will, my only concern WHICH MAY BE UNFOUNDED is on namespaces conflicts between setuptools and distribute. For me, the first one loaded module wins. And, it's low priority any way i think as a patch may by easy to do. Just one thing to borrow from the precedent mail would be to make one sentence on the PEP to write in rock that we can make installation elsewhere that in standard place for both python code & metadata. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF
participants (3)
-
Eric Smith -
kiorky -
Tarek Ziadé