[Distutils] formencode as .egg in Debian ??

["Martin v. Löwis"]

As for terminology, you seem to suggest to use "distribution" where
Debian uses "package". So "Debian package" would become "Debian
distribution". This does not sound right, because "Debian distribution"
is the entire collection of packages that is released e.g. on a DVD-ROM.
I'll try to use "project" in your sense and "package" in the
Python sense whenever I can.

Phillip J. Eby wrote:
> An "egg" is a "distribution" of a "project" that is importable and can 
> carry both standardized and individualized metadata that can be read by 
> the pkg_resources module.  There are various distribution *formats* in 
> which an "egg" may be physically manifested, but the "egg" itself is a 
> logical concept, not a physical one.  It is therefore, as I said, "not 
> merely a distribution format".  Is that any clearer?

Yes. When I said "an egg", I meant "a zipfile with a .egg extension,
or a directory with a .egg extension". In response to

# [...] who will quite simply need eggs for many packages.
# If Debian doesn't provide them, the users will be forced to obtain
# them elsewhere.

I meant

"Debian should provide the distributions, but not as .egg files";
it should provide the distribution as a deb file. So users are provided
with the project, but in a form that is not one of the three forms
an egg could have.

> The "contradiction in terms" was that I took your meaning of "package" 
> to be the same as my term "project" - i.e., a functional collection of 
> Python resources.  Projects that *are* eggs, can't be provided "but not 
> as eggs".  They *are* eggs, so not providing them as eggs means not 
> providing them at all.

I would expect that you can "unegg" a project. You can distribute the
project as a collection of Python modules, not as a collection of
Python resources. The Debian developer could (and I was suggesting
he should) just ignore the entire egg structure, and distribute
the code of the library only.

>>  If so, Debian should not distribute them.
> 
> 
> This is what I don't understand, as it has nothing to do whether or not 
> is a distribution format, at least not that I can see.  My statement was 
> that eggs are not merely a distribution format; they are a logical 
> concept that can be physically packaged in various ways, and if it's 
> necessary to invent yet another physical layout, well, we can do that too.

Yes, but this logical concept is in the way of Debian 
packages/distributions (atleast if done naively by the Debian
developer). This is what started the entire discussion: Matthias
Urlichs complained that Bob Tanner included the egg structure
in the formencode Debian package/distribution.

The specific initial complaints where:
- you can't use it with a simple "import formencode",
- pydoc does not work on "eggs".

I would add the complaint:
- it increases sys.path for no good reason.

> Which would be the same as saying you wouldn't distribute, say, 
> setuptools itself.  Setuptools is an egg, and can't function except as 
> an egg, because it is more than a Python package.  Again, an "egg" is 
> some specific release of a project and its introspectable metadata.

I could rewrite setuptools to function as a regular Python package.
After a shallow inspection, there aren't many places where it really
needs the pkg_resources functionalities for itself - I could only
identify the part that locates cli.exe. As this is used on Windows
only, a Debian port of setuptools could simply ignore this code.

>> It is not a distutils setup because it does not invoke
>> distutils.core.setup.
> 
> 
> Now I really don't understand you.  Line 43 of setuptools/__init__.py 
> reads:
> 
>     setup = distutils.core.setup
> 
> So, how is it not invoking distutils.core.setup?

Ah, I didn't look so far. I noticed that when I replace

from setuptools import setup

with

from distutils.core import setup

I get warnings about package_data and extras_require, and assumed this
means setup was a different function; instead, it really is the import
that plays tricks here.

>> Extending distutils is fine. An extension is a feature that, if not
>> invoked, has no effect. easy_setup changes install in a way that
>> has an effect.
> 
> So do all the packages that rework install_data to be more to their 
> liking - and there are quite a lot of them, as I discovered when I began 
> testing easy_install.

Right. It really isn't that much about what is and is not conforming;
it more matters what the practical effects on the Debian developer
are. If "setup.py install" just puts some files into some locations,
and the files don't conflict with files in other Debian
packages/distributions, the developer can easily package the entire
thing. If "setup.py install" does other things, like editing an
existing file, it is not so easy anymore.

>> That is not true. Usability also suffers if sys.path becomes long.
> 
> 
> How?  I don't understand this.

People will often inspect sys.path to understand where Python
is looking for their code. They can do so manually if sys.path
fits on one or two lines of terminal output. On my system, it
is now four lines, primarily thanks to .pth files. This is getting
unusable.

>> That is certainly possible - but currently, each maintainer would have
>> to come up with his own solution. This is more tedious to do than it
>> could be.
> 
> 
> easy_deb implements this, so it seems to me it would be a simple matter 
> of running easy_deb to produce the .deb from the .egg.  (Caveat: I have 
> not used easy_deb, but its author assures me that it is able to handle 
> the .pth manipulation in a sane way.)

I can't comment on this. Somebody probably should examine whether
easy_deb complies with all policies, and this is what the developer
should use (I'm worried though that I don't see a mentioning
of dpkg-buildpackage in http://www.python.org/pypi/easydeb)

> What I would suggest here is having a namespace (e.g. pyegg2.4-whatever) 
> for naming packages based on their PyPI names, so that there can be an 
> automated relationship between setuptools dependencies and Debian ones.

That would be a policy change (I think). Whether it would be agreeable,
I have no idea.

> Anyway, I don't see any obvious reasons why this can't be an automated 
> process, even for the system library dependencies.  easy_deb even has a 
> simple configuration file that can augment the setuptools-style 
> dependencies with explicit Debian dependencies.

Debian policy currently seems to require that the dependencies are
provided as plain text in a patch to the upstream sources(*). So the
idea certainly is that dependencies are managed by the developer,
not automatically.

Regards,
Martin

(*) More precisely, it requires that after unpacking the source of
the source package/distribution, the dependencies are listed
in debian/control.