Mailman 3 November 2005 - Distutils-SIG

[DistUtils] - Problem with versions...
by Sharky On PTNet Nov. 29, 2005

Nov. 29, 2005

Hi, Trying to upgrade cherrypy pkg with ease_install, i notice that easy_install don't detect the newer version. You can see that 2.1.0 is the final and not 2.1.0-rc2 in the sourceforge page: http://sourceforge.net/project/showfiles.php?group_id=56099 Thanks for all Sharkypt @ Helio Pereira # easy_install -U cherrypy Searching for cherrypy Reading http://www.python.org/pypi/cherrypy/ Couldn't find index page for 'cherrypy' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://www.python.org/pypi/ Reading http://www.python.org/pypi/CherryPy/2.1.0 Reading http://www.cherrypy.org Reading http://sourceforge.net/project/showfiles.php?group_id=56099 Best match: CherryPy 2.1.0-rc2 Processing CherryPy-2.1.0_rc2-py2.4.egg CherryPy 2.1.0-rc2 is already the active version in easy-install.pth Using /usr/lib/python2.4/site-packages/CherryPy-2.1.0_rc2-py2.4.egg Processing dependencies for cherrypy

2 2

Re: [Distutils] formencode as .egg in Debian ??
by Bob Tanner Nov. 29, 2005

Nov. 29, 2005

On Tuesday 22 November 2005 12:15 am, Martin v. Löwis wrote: > I don't think Debian should use the egg structure. It apparently relies > on building a long sys.path (even though through only a single .pth > file); I'm not sure of how .eggs are implemented, but I'm going to cross-post this info to the python-distutils mailing list. See below for additional comments. > this adds additional costs to all import statements on startup. > It gets worse if these are zipfiles, because then each import statement > will have to look into each zipfile (until the import is resolved). The is the opposite of what I was told by upstream development over on distutils, snippet from Phillip J. Eby <pje(a)telecommunity.com>: <snip> Note also that in many cases, the package will be a single .egg file, (analagous to a Java .jar file) rather than a directory, and files are preferable to directories in most cases as they make Python import processing faster. <snip> ( Rest of the comments debian-python specific) -- Bob Tanner <tanner(a)real-time.com> | Phone : (952)943-8700 http://www.real-time.com, Minnesota, Linux | Fax : (952)943-8500 Key fingerprint = AB15 0BDF BCDE 4369 5B42 1973 7CF1 A709 2CC1 B288

19 141

Problems instaling Kid template
by Sharky On PTNet Nov. 28, 2005

Nov. 28, 2005

Hi again :P I'm having trobles trying to install Kid template with easy_install. Don't know if is a problem related with kid package or something else. Here is the output of the error: # easy_install kid Searching for kid Reading http://www.python.org/pypi/kid/ Reading http://kid.lesscode.org/ Best match: kid 0.7.1 Downloading http://lesscode.org/dist/kid/0.7.1/kid-0.7.1.tar.gz Processing kid-0.7.1.tar.gz Running kid-0.7.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-gJouzW/kid-0.7.1/egg-dist-tmp-noS3cf error: SandboxViolation: mkdir('/home/sharky/.python-eggs', 511) {} The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted. This package cannot be safely installed by EasyInstall, and may not support alternate installation locations even if you run its setup script by hand. Please inform the package's author and the EasyInstall maintainers to find out if a fix or workaround is available.

2 1

problems installing elementTree
by Sharky On PTNet Nov. 26, 2005

Nov. 26, 2005

Hi, I'm getting problems instaling Turbogears when easy_install try to install cElementTree. I have ubuntu breezy (5.10) with python 2.4.2. Any one can helpme with this? Thanks, Helio Pereira Sharky @ PTNet [ERROR] Using /usr/lib/python2.4/site-packages/TurboGears-0.8a4-py2.4.egg Processing dependencies for turbogears Searching for cElementTree>=1.0.2 Reading http://www.python.org/pypi/cElementTree/ Reading http://www.effbot.org/zone/celementtree.htm Reading http://effbot.org/downloads#celementtree Best match: cElementTree 1.0.2-20050302 Downloading http://effbot.org/downloads/index.cgi/cElementTree-1.0.2-20050302.zip?index error: Unexpected HTML page found at http://effbot.org/downloads/index.cgi/cElementTree-1.0.2-20050302.zip?index

2 1

Re: [Distutils] [Bulk] Re: formencode as .egg in Debian ??
by Phillip J. Eby Nov. 25, 2005

Nov. 25, 2005

[Debian trimmed since this is Windows-specific] At 04:33 PM 11/25/2005 +0000, Paul Moore wrote: >On 11/25/05, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > At 10:28 AM 11/25/2005 -0500, Phillip J. Eby wrote: > > >Actually, I'm increasingly tempted to PEP the idea of a > > >name-version.pkg-info file being installed alongside the code as a > standard > > >Python distutils feature. This discussion has made it clearer (even to > > >me!) how useful such a thing would be, and if it became a standard part of > > >the distutils, it would simply become part of Debian and every other > system > > >packager unless explicitly removed. > > > > Also, it just occurred to me that this would ultimately fix one of Paul > > Moore's complaints about eggs as well. If bdist_wininst included this > > .pkg-info, then even Windows "managed" packages would be supported in this > > scheme, > >That would be a major bonus for me, certainly. > > > and setuptools could include a bdist_wininst that made similar > > .exe's to wrap eggs. > >That would be the other half of the equation. I had avoided suggesting >precisely that, because it's daft of me to expect you to do this work >just to stop me moaning (and installer-writing is enough of a black >art to me that I can't assume it's an easy change to the existing >bdist_wininst code). Actually, the only thing weighing against it in my priorities is the other half of the equation. Until regular win32.exe's include pkg-info, it doesn't make sense to only half-solve this problem; it's better to focus on improving the egg-centric tools (like easy_install), since these can provide a complete solution, and you're the only person who has said they would rather download and install individual .exe packages than use easy_install to take care of the whole business.

1 0

Re: [Distutils] formencode as .egg in Debian ??
by Phillip J. Eby Nov. 25, 2005

Nov. 25, 2005

[debian folks trimmed from the cc: list since this is mainly Windows-related] At 04:19 PM 11/25/2005 +0000, Paul Moore wrote: >On 11/25/05, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > At 11:34 AM 11/25/2005 +0000, Paul Moore wrote: > > >- what would happen if I downloaded the TurboGears egg and just put it > > >on sys.path - no easy_install or whatever?) > > > > You could import stuff from it, but it might or might not work. Where it > > explicitly require()'s things or tries to load a plugin, you'll get > > DependencyNotFound errors. > >OK, I made an assumption which I wasn't clear about here. Assume I've >done what I currently do, and installed all of its dependencies. > >Will it work now? If not, why not? See what I said above; It answers both questions. >Actually, I think I know the answer (although I've thought that before >as well :-)) - it will work, as long as I didn't screw up with my >manual dependency management. A screw-up equates to me getting a >DependencyNotFound error. If that's correct, then I'm more or less >fine with it. If the system-packaged dependencies don't have some way for the egg runtime to identify them, you'll also get DependencyNotFound errors for those. >But... > >To install a non-egg Python package is just a matter of: > 1. Running a bdist_wininst exe (or apt-get on Debian, etc), or > 2. Dumping the package directory in site-packages (Debian folks, >please don't scream :-)) > >For an egg, like TurboGears, my options (assuming I don't want to use >easy_install) are: > 1. Wait for someone to wrap the TG egg in a system package, or > 2. Put the egg somewhere (likely site-packages), then create or >modify a .pth file by hand to add the egg to sys.path. > >In both cases, (1) is the managed option, and (2) is the unmanaged >one. And (2) is *harder* for eggs. ... >I find this jarring, because eggs are presented as a "zero installation" >format. Eggs were really only intended to be "zero installation" for application plugins. The package management stuff came later; the purpose of dependencies was originally there to allow pre-import plugin integrity checking to better preserve application stability. However, at some point I realized that the way eggs were designed meant that you could build some simple package management on top, and easy_install was born. Anyway, I would say that "Wait for someone to wrap the TG egg in a system package" is the correct approach for people who want system packages, and don't want to use easy_install. >And I'd be curious to know what sort of reaction I'd get from the TG >community if I posted a message saying I'd dumped various eggs into >site-packages, manually fiddled with .pth files, and TG wasn't >working. :-) Well, as far as I know, TurboGears does a require("TurboGears") when you import it, which will promptly fail with DependencyNotFound or VersionConflict errors if you haven't done the job right, describing the first missing or screwed-up thing. > > The drop-in-and-go aspect of eggs is mainly used for application plugins in > > systems like Trac, although in truth they actually use the discovery > > machinery to scan their plugins directory and identify which plugins are > > loadable - that is, which plugins can have all their dependencies met. > >Yes. Plugins are a clear benefit. I don't think *anyone* is arguing >that this is an issue. > > > This is one reason that runtime integrity checks are important; an app can > > detect that it doesn't have what's needed for a given plugin, and skip > > loading it. This is also why dependency resolution needs to be able to > > roll back if it fails, even for deeply-nested dependencies. > >OK. And I see that a plugin can need a system library (a Trac >"progress chart" plugin needing PIL installed, for example). And >non-egg installs can't provide that discovery mechanism directly. OK. >You've got that covered (as best you can) with the single egg-info >file option. That's the idea. >As a *user* of other people's packages, I want to use system managed >packages. As a *Windows* user, I'm a pain, because I expect the >developer to supply that system managed installer, rather than the >suppliers of my distribution. Yeah, expecting Microsoft to ship .msi's of Free software is a bit much. :) >(and round full circle) As a *developer* again, I don't have a way of >satisfying my Windows user's requirement for a wrapped version of my >egg. This could be solved by making setuptools' bdist_wininst build an egg in an .exe. The other half of the issue would be resolved by making a .pkg-info inclusion be a standard distutils feature. With these two things done, any egg can be had as an .exe, and any .exe can be had as an egg. In the long run, .msi might be a better format if it supports dependencies, but at least with .exe-wrapped eggs you have the integrity checks to catch when essential stuff got removed. >[An only vaguely related question - is the feature of generating >wrappers for scripts based on entry points an easy_install feature, or >an egg feature? If so, what happens to it with a system package >wrapping the egg? I ask because I see that Kid has swiched from >bdist_wininst to egg as its distribution format, as far as I can tell, >just for the script-wrapper feature - so can I use the manual option >(2) above to install Kid???] Script wrappers are implemented by easy_install, but the entry points themselves are an egg API feature. You can invoke the entry points whether or not there's a script wrapper, you just can't do it from the command line without typing some code. :)

1 0

Re: [Distutils] formencode as .egg in Debian ??
by Phillip J. Eby Nov. 25, 2005

Nov. 25, 2005

At 11:34 AM 11/25/2005 +0000, Paul Moore wrote: >On 11/25/05, Donovan Baarda <abo(a)minkirri.apana.org.au> wrote: > > On thing about this worried me; > > > > If TurboGears depends on an "egg'ed" ElementTree, what happens if a > > system has ElementTree installed as a non-egg package? Does installing > > TurboGears as an egg inside a Debian package require also installing > > another ElementTree as an egg inside a Debian package? Or worse, will it > > automatically download and install an egg'ed ElementTree not from a > > Debian package? Will you end up with two ElementTree's installed, one > > egg'ed and one not? > >Yes. That's the issue I'm not keen on, and I believe your >interpretation is right - TurboGears depends on ElementTree, which is >fine. But *because* TurboGears is an egg [1] it *needs* an egg-ified >version of ElementTree. If I have ElementTree installed already, TG >will *ignore* that, and grab an extra version from the internet (at >install time, presumably, assuning I'm using EasyInstall - but what if >I'm not?) If you're not then you'll just get an error at runtime telling you the integrity checking failed; i.e. a DistributionNotFound error. >just to get it in egg format. More precisely, just to get something it knows for sure is an ElementTree whose version is in the desired range. >[1] It's not exactly because TG is an egg per se, as easy_install and >dependency checking is involved as well, which not all eggs have to >use. I'm slowly coming to understand that eggs (and setuptools, and >easy_install, which are related but subtly different items) provide a >host of largely unrelated functionality, Yep; this is why they have separate web pages. ;-) (And also why some people call it the "eggs/easy_install/setuptools trifecta".) >and discussions are getting >confused because these are not being separated out sufficiently. +1 :) >- what would happen if I downloaded the TurboGears egg and just put it >on sys.path - no easy_install or whatever?) You could import stuff from it, but it might or might not work. Where it explicitly require()'s things or tries to load a plugin, you'll get DependencyNotFound errors. The drop-in-and-go aspect of eggs is mainly used for application plugins in systems like Trac, although in truth they actually use the discovery machinery to scan their plugins directory and identify which plugins are loadable - that is, which plugins can have all their dependencies met. This is one reason that runtime integrity checks are important; an app can detect that it doesn't have what's needed for a given plugin, and skip loading it. This is also why dependency resolution needs to be able to roll back if it fails, even for deeply-nested dependencies.

2 1

Re: [Distutils] formencode as .egg in Debian ??
by Phillip J. Eby Nov. 25, 2005

Nov. 25, 2005

At 01:39 PM 11/25/2005 +0000, Paul Moore wrote: >On 11/25/05, Donovan Baarda <abo(a)minkirri.apana.org.au> wrote: > > Sorry... that's what I meant; don't deb an egg... install it as an egg, > > outside of the Debian package management system. As an egg, it is under > > development and not ready for release as a deb. When the software is > > ready for release, strip the egg package management layer and create a > > proper deb out of it. > >Again, this is something I picked up from previous discussions of this >nature, but you can't "strip the egg package management layer" without >also stripping out the other features of eggs, such as plugin support, >metadata introspection, etc, which are useful whether or not you care >about package management. > >So, as a package developer, if I want to use egg metadata or plugin >support, I have to buy into the whole egg experience, including the >installation stuff. Actually, I thought you wanted eggs to be wrapped in system packages, so that you don't have to use the "installation stuff". That is, after all, what this extended thread is about: providing system packages of egg projects, and allowing egg dependencies to be met by system packages. >I'd argue that the different features should be separated, so that >people can choose the ones they want to use (and as a consequence, the >"egg package management layer" could be stripped out independently). >But that doesn't seem to be the direction the development is going. > >Apologies if this is incorrect. I'm still researching... The issue is integrity checking. Python imports as an integrity check are not safe in the general case, *especially* in Python 2.3, where a failed nested import results in an importable but broken module. Integrity checks are also essential for plugin support, since many plugins will probably *not* be system-managed, so the app has to be able to tell if it can load the plugin or not. A plugin, however, may have dependencies that can be satisfied by system packages, like if the plugin uses ReportLab or something to add PDF support to an application's export menu. That's why you can't really just "peel off" the egg shell here; the whole stack was designed to support quality plugin systems for Python apps, similar to the Eclipse or OSGi systems in the Java world.

1 0

Including package_data in a source distribution
by Noel O'Boyle Nov. 25, 2005

Nov. 25, 2005

Dear all, I have been trying to use either distutils or setuptools to include some package data in a source distribution ("python setup.py sdist") using the package_data option. It *is* included if I use "python setup.py bdist_egg". Can someone explain to me whether this is a feature or a bug? The documentation led me to expect that this would work, but it does not. I am using the example on http://www.python.org/dev/doc/devel/dist/node12.html. Regards, Noel O'Boyle.

3 3

Re: [Distutils] formencode as .egg in Debian ??
by Phillip J. Eby Nov. 24, 2005

Nov. 24, 2005

At 08:10 AM 11/24/2005 -0800, Kurt Schwehr wrote: >As a person who works as a packager with a debian based distribution (fink >on Mac OSX) I am glad to see this discussion. I have no personal opinion >about python eggs, but would really like to find a way to get some nice >packages into the system and have been totally unable to figure out how to >deal with eggs. At the moment, easy_deb appears to be the most immediately available option for creating debs that support all egg features, from both egg-based and non-egg-based Python packages. If I understand Vincenzo correctly, easy_deb includes tools to allow switching the active package versions, although I don't know how or if it relates to dpkg-divert and the "alternatives" system, as I'm not a Debian user myself. >I see the point of allowing developers to have multiple versions of >packages when testing, but we definitely want to avoid that in the actual >fink tree at all costs. It would be great to have a build system that can >switch between what we need as stable distribution packagers and what >people want/need in their local trees. My hope is that eggs in the long >run can help the packagers track dependencies, but still let the packagers >handle those dependencies. On Mac OS X, we recommend that users install to ~/Library/Python2.x/site-packages, as this is private to the user and already supports .pth files. That way, they can switch between versions at will. See also: http://peak.telecommunity.com/DevCenter/EasyInstall#non-root-installation I would be happy to answer any questions or hear any concerns you may have about how any of this works or how you'd like it to. Thanks.

1 0