[Distutils] Outdated packages on pypi

[Nick Coghlan]

On 15 July 2016 at 23:59, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> On Fri, Jul 15, 2016, at 01:25 AM, Donald Stufft wrote:
>
> Off the top of my head I can only really think of PIL, and *maybe* suds.
> Unless there’s a lot of these maybe all we really need is a policy for when
> administrators can/will edit the page to direct people towards a different
> project or a way to add an admin message directing people to another
> project.
>
>
> Proposal: let's put some such manual intervention policy in place for now.
> Apply it for PIL to point to Pillow, and query the active suds forks to see
> if there's a generally agreed successor.
>
> If this works well, great! If the admins are flooded with 'successor
> requests', then we can come back to question of an automated mechanism. If
> there are too many abandoned packages with competing successors, that's a
> trickier problem to solve, but at least we'd be considering it with more
> information.

+1, although I'd propose decoupling the policy aspect ("Project X has
been declared unmaintained, with Project Y as its official successor")
from the implementation aspect of how that policy is applied in PyPI
and pip. That way we wouldn't be adding to the existing workload of
the PyPI admins - their involvement would just be implementing the
collective policy decisions of the PyPA, rather than being directly
tasked with making those policy decisions themselves.

For example, suppose we had a "Replacement Packages" page on
packaging.python.org, that documented cases like PIL -> Pillow, where:

- a de facto community standard package has become unmaintained
- attempts to reach the existing maintainers to transfer ownership have failed
- a de facto replacement package has emerged
- the overall newcomer experience for the Python ecosystem is being
harmed by legacy documentation that still recommends the old de facto
standard

Adding new entries to that page would then require filing an issue at
https://github.com/pypa/python-packaging-user-guide/issues/ to
establish:

- the package being replaced is a de facto community standard
- the package being replaced is important to the user experience of
newcomers to the Python ecosystem
- the package being replaced has become unmaintained
- a newer community fork has gained sufficient traction to be deemed a
de facto successor
- the existing maintainer has been contacted, and is unresponsive to
requests to accept help with maintenance

If *all* of those points are credibly established, *then* the package
replacement would be added to the "Replacement Packages" list on
packaging.python.org.

How that list was utilised in PyPI and pip, as well as in other
package introspection tools (e.g. IDEs, VersionEye), would then be the
decision of the designers of those tools.

> As further examples: pydot, pexpect and python-modernize have all been
> unmaintained, leading to forks springing up. In all three cases, some of the
> forkers eventually coordinated to contact the original maintainer, get
> upload rights, and make new releases with the original name. It would
> certainly have been nice if that could have happened sooner in each case,
> but I doubt that any technical fix would have made a big difference.

The PyCA folks obtaining maintenance access to PyOpenSSL would be
another example of this being navigated successfully without a long
term split.

One of the longest running eventually resolved examples to date would
be the multi-year setuptools/distribute split, and I'd actually
consider that the ideal outcome of this process in general: while we
understand entirely that folks may need to step away from open source
software maintenance for a wide range of reasons, we strongly prefer
to see projects providing critical functionality handed over to a new
set of maintainers that have earned the trust of either the original
maintainer or the wider community rather than letting them languish
indefinitely.

We can't mandate that any given project invest time in succession
planning though, so having a system in place to designate successor
projects at the ecosystem level when maintainers aren't able to
resolve it at a project level makes sense.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia