[Distutils] Twisted packaging in Debian (was Re: Pre-pre-PEP: Requirements for the Python BUILDS Specification)

[Josselin Mouette]

Le mardi 07 octobre 2008 à 11:44 -0400, Jean-Paul Calderone a écrit :
> They're modules.  They get imported.  That makes them modules, right?

Fine. I hadn’t understood that they are actually meant to be imported as
is (which makes you right in your assertions that the explanations
weren’t completely clear to me yet). If they are modules, meant to be
imported, they actually have their place in the modules directory, so
please disregard what I’ve said about private directories.

So, if they are modules, they sometimes need namespace packages to be
imported properly. However you intentionally do not import modules that
have associated namespace packages, which is the exact opposite of what
we expect for other modules.

Let me comment on your previous email with that in mind.

        In previous releases of Twisted, this help was given by twisted/plugins/
        __init__.py.  It included an extremely primitive implementation of the
        concept of namespace packages (_extremely_ primitive).  It would find all
        directories named "twisted/plugins" which were in the module search path
        and add them to its own __path__.  The purpose of this feature wasn't to
        support installations, however.  The purpose of this feature was as a
        developer convenience.  It allows developers to edit plugins in a non-
        installed location (for example, their home directory) and then use them
        without copying them to the installed Twisted's twisted/plugins/ directory.

OK, I get that now. You need to do that because importing
plugins.something in the twisted/ directory will only look for
subdirectories of the same installation path. 

        In a more recent release of Twisted, this feature was tweaked slightly.
        The reason for the change is that there was a significant bug in the old
        implementation of namespace packages.  If a developer was using their own
        copy of Twisted (for example, an SVN checkout in their home directory)
        instead of the installed version, the namespace package implementation
        would still load plugins from the *installed* version of Twisted.  In the
        best possible case, this gave you two copies of every plugin.  In the
        worst case, the installed version of Twisted wasn't compatible with the
        other version of Twisted and the plugins would raise exceptions.

About this issue, I still think you need to version your plugins
directory if the plugins are not compatible across versions. Otherwise -
and this is something that will hit distributions hard - you may install
a an incompatible version of, say, Nevow, and get your installation
fucked up.

However, in the end, I think you’ve been hit by an issue that was only
sporadically met until now: managing files belonging to the same module
hierarchy with two different tools doesn’t always work. And this is
something that we need to manage inside Debian (which is more a
political problem than a technical one).

> Do we have the same definition of "module"?

I think so.

> I chose the phrase "The ideal fix, from my perspective as a Twisted
> developer" very intentionally.  If the file ends up somewhere else
> but it's still an importable module, fine - if, from a Debian packager's
> perspective that's better, fine.  I don't believe that the current
> situation is better from a Debian packager's perspective, though, since
> it involves carrying a patch that reverts an upstream bugfix.

In the end, I don’t think this patch was added as there is another
workaround. Either way, it may break again if you change the way you
look for plugins, so you are right to expect that they land in the same
directory.

Cheers,
-- 
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20081007/5a830277/attachment.pgp>