[Distutils] formencode as .egg in Debian ??

[Phillip J. Eby]

[debian folks trimmed from the cc: list since this is mainly Windows-related]

At 04:19 PM 11/25/2005 +0000, Paul Moore wrote:
>On 11/25/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 11:34 AM 11/25/2005 +0000, Paul Moore wrote:
> > >- what would happen if I downloaded the TurboGears egg and just put it
> > >on sys.path - no easy_install or whatever?)
> >
> > You could import stuff from it, but it might or might not work.  Where it
> > explicitly require()'s things or tries to load a plugin, you'll get
> > DependencyNotFound errors.
>
>OK, I made an assumption which I wasn't clear about here. Assume I've
>done what I currently do, and installed all of its dependencies.
>
>Will it work now? If not, why not?

See what I said above; It answers both questions.


>Actually, I think I know the answer (although I've thought that before
>as well :-)) - it will work, as long as I didn't screw up with my
>manual dependency management. A screw-up equates to me getting a
>DependencyNotFound error. If that's correct, then I'm more or less
>fine with it.

If the system-packaged dependencies don't have some way for the egg runtime 
to identify them, you'll also get DependencyNotFound errors for those.


>But...
>
>To install a non-egg Python package is just a matter of:
>     1. Running a bdist_wininst exe (or apt-get on Debian, etc), or
>     2. Dumping the package directory in site-packages (Debian folks,
>please don't scream :-))
>
>For an egg, like TurboGears, my options (assuming I don't want to use
>easy_install) are:
>     1. Wait for someone to wrap the TG egg in a system package, or
>     2. Put the egg somewhere (likely site-packages), then create or
>modify a .pth file by hand to add the egg to sys.path.
>
>In both cases, (1) is the managed option, and (2) is the unmanaged
>one. And (2) is *harder* for eggs.
...
>I find this jarring, because eggs are presented as a "zero installation" 
>format.

Eggs were really only intended to be "zero installation" for application 
plugins.  The package management stuff came later; the purpose of 
dependencies was originally there to allow pre-import plugin integrity 
checking to better preserve application stability.

However, at some point I realized that the way eggs were designed meant 
that you could build some simple package management on top, and 
easy_install was born.

Anyway, I would say that "Wait for someone to wrap the TG egg in a system 
package" is the correct approach for people who want system packages, and 
don't want to use easy_install.


>And I'd be curious to know what sort of reaction I'd get from the TG
>community if I posted a message saying I'd dumped various eggs into
>site-packages, manually fiddled with .pth files, and TG wasn't
>working. :-)

Well, as far as I know, TurboGears does a require("TurboGears") when you 
import it, which will promptly fail with DependencyNotFound or 
VersionConflict errors if you haven't done the job right, describing the 
first missing or screwed-up thing.


> > The drop-in-and-go aspect of eggs is mainly used for application plugins in
> > systems like Trac, although in truth they actually use the discovery
> > machinery to scan their plugins directory and identify which plugins are
> > loadable - that is, which plugins can have all their dependencies met.
>
>Yes. Plugins are a clear benefit. I don't think *anyone* is arguing
>that this is an issue.
>
> > This is one reason that runtime integrity checks are important; an app can
> > detect that it doesn't have what's needed for a given plugin, and skip
> > loading it.  This is also why dependency resolution needs to be able to
> > roll back if it fails, even for deeply-nested dependencies.
>
>OK. And I see that a plugin can need a system library (a Trac
>"progress chart" plugin needing PIL installed, for example). And
>non-egg installs can't provide that discovery mechanism directly. OK.
>You've got that covered (as best you can) with the single egg-info
>file option.

That's the idea.


>As a *user* of other people's packages, I want to use system managed
>packages. As a *Windows* user, I'm a pain, because I expect the
>developer to supply that system managed installer, rather than the
>suppliers of my distribution.

Yeah, expecting Microsoft to ship .msi's of Free software is a bit much.  :)


>(and round full circle) As a *developer* again, I don't have a way of
>satisfying my Windows user's requirement for a wrapped version of my
>egg.

This could be solved by making setuptools' bdist_wininst build an egg in an 
.exe.  The other half of the issue would be resolved by making a .pkg-info 
inclusion be a standard distutils feature.  With these two things done, any 
egg can be had as an .exe, and any .exe can be had as an egg.

In the long run, .msi might be a better format if it supports dependencies, 
but at least with .exe-wrapped eggs you have the integrity checks to catch 
when essential stuff got removed.


>[An only vaguely related question - is the feature of generating
>wrappers for scripts based on entry points an easy_install feature, or
>an egg feature? If so, what happens to it with a system package
>wrapping the egg? I ask because I see that Kid has swiched from
>bdist_wininst to egg as its distribution format, as far as I can tell,
>just for the script-wrapper feature - so can I use the manual option
>(2) above to install Kid???]

Script wrappers are implemented by easy_install, but the entry points 
themselves are an egg API feature.  You can invoke the entry points whether 
or not there's a script wrapper, you just can't do it from the command line 
without typing some code.  :)