[Distutils] Adding egg support (Resource API) to existing packages

[Paul Moore]

On 7/13/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:20 AM 7/13/2005 +0100, Paul Moore wrote:
> >I can just say that users must install setuptools, but this seems ugly, as
> >there is no obvious reason why a date utility package should depend on a setup
> >utility package.
> 
> I think you're missing something.  It makes more sense to bundle
> ez_setup.py than pkg_resources.  See this link:

I had read that link. I think I failed to express my problem well
enough. I am not the author of the python-dateutils package, so I am
acting as an interested user, who is considering submitting a patch to
the author.

My patch will help *me* build an egg from the package, but unless
(until) the author becomes interested in easy_setup, setuptools and
eggs for his own reasons, I don't expect him to be building eggs
himself.

What I want to do is to send a patch to the author, saying "Hey - I
like your package, but for my needs the following patch would help.
Others may also benefit, so would it be possible to include it in the
standard distribution?" Now, at this level, setuptools/pkg_utils isn't
relevant. I'd have the same issues if I was advocating a dependency on
*any* other package - why should the author impose an extra
installation burden on all of his users just because I'd like an
additional feature? That's precisely the issue that eggs are designed
to address, but we aren't yet in a position where we can assume
everyone is using eggs, and the problem is solved. So I have to
minimise my dependency requirements, and carefully judge the costs and
benefits - particularly for other users who may not get any benefit at
all!

In addition, bundling ez_setup imposes a burden on the package
*author*, who will potentially need to make changes to his own build
process as a result - even if it's only to make sure that he has an
internet connection when he builds, so that ez_setup can do its stuff.
And the same burden applies to anyone using the package who still
wants to use "python setup.py install".

Don't get me wrong - ez_setup is a great solution to the transitional
issue of getting from where we are now to a position where everyone
can be assumed to have (some version of) setuptools installed. But
that's good for setup/distribution issues, not resource lookup.

What I'm saying, is that I see a need for *another* transitional
process - enabling users and/or package authors to make their code
"resource safe", *without* having to buy into the egg/setuptools
transition. I see the resource API as a major benefit in isolation -
it's a general tool which can be used with zipimport, or other import
hooks, and is not egg-specific. There's very little downside (IMHO) to
using the resource API. On the other hand, I see eggs as a less
clear-cut benefit - many people will still prefer installation
processes tied into the OS package management facilities (even
something as basic as bdist_wininst).

Alongside this, I see setuptools as basically an egg-enabling change -
and as such tied to the "less clear-cut" side of the equation. I'd
like to see pkg_resources separated from this, as I see far less
reason why anyone might object to it.

[This is why I feel that pkg_resource is pretty much a no-brainer for
inclusion into the stdlib, whereas setuptools may be more
controversial].

> You're also missing the other half of the problem, where pkg_resources
> needs setuptools so it can be installed in a way that's safe from being
> overwritten by random packages reinstalling it!  This is also why other
> packages can't bundle it, without effectively bundling setuptools too.  So,
> the simplest solution is to Just Drink the Kool-Aid and use setuptools
> already.  It's not like there aren't numerous other benefits to using it
> for your packaging, anyway.

Hmm, I've read this a number of times and it still seems to me to be
saying that all of setuptools' package management goodies break
horribly if an unmanaged copy of a package is installed - making it a
very "all or nothing" solution. While I accept that something like
require("setuptools=0.5a7,=0.5a11,=0.5a12") is probably not going to
work well if there's an unmanaged copy of setuptools on sys.path,
can't you make some attempt in simpler cases (maybe introspect for
__version__, and if not present assume the unmanaged version is
"wrong", and put a good version at the *front* of sys.path - or
something). Some form of graceful degradation should be available,
surely? What if I install a new package, which I know depends on a
number of major packages I already have installed - and the new one is
an egg, but the old ones have been there for ages and are used by a
number of existing scripts. Surely I don't have to switch everything
to managed eggs in one "big bang" just to get the one new package
installed? I can't see people doing that in a production environment.

If I misread your paragraph above, my apologies.

Cheers,
Paul.