[Catalog-sig] PEP 345 Update

[P.J. Eby]

At 02:47 PM 8/26/2010 +0200, Alexis Métaireau wrote:
>Right, the installation *will not* cause problems, but they will 
>apear after that. If so, we have two choices:
>- First is to tell nothing to the user about that, and I think 
>that's a good option, as it could result in a end user bad 
>experience (if that's not handled at the installation level, what 
>will handle this?)
>- Second is to prompt the user about what he want to do: Do we have 
>to install the release/distribution, even if it can cause problems 
>while running them? Are you sure you want to install both, even if 
>it could lead to such issues ?, etc.

Informing the user about the conflict isn't a bad idea...  but then 
again, it sounds like something that would normally be covered in a 
project's README.

Notice here the difference between the Python environment and a Linux 
distribution: the distribution is not merely *installing* software, 
it is also *deploying* it in some sense -- such software is 
preconfigured as well as installed, and integrated with the system as a whole.

However, that same software as released by its author is not so 
integrated and preconfigured, and thus, these conflict issues cannot 
necessarily even be *known* in advance by the author.

IMO, then, the very ideas of Obsoletes and Conflicts are broken in 
the software *distribution* context, vs. the software "integration" 
or "deployment" context.  System packagers can make these kind of 
evaluations in the context of their supported integration.

However, with the exception of informing people about an obsoleted 
package of theirs, these are not judgments that can be made so 
unequivocally by the individual author of a piece of software, as 
they have no overview of the integrated whole (as a system 
integrator/distributor does).



>>In other words, none of the above is actually a use-case for having 
>>a "Conflicts" field.
>I have pain to imagine use cases where conflicts could raise at the 
>*installation* level too, but the question seems to be: "do we need 
>something to describe installation-related conflicts, or run-related 
>conflicts ? In debian apt system, conflicts are described for 
>"on-run" issues, IIRC.

...and they're described relative to a specific runtime environment, 
which won't really be the case for the individual Python package author.


>>Here's the thing: so far I'm the ONLY person in the discussion who 
>>has even offered an example of when Obsoletes would be used...  and 
>>for the two examples I gave, I would totally be willing to make a 
>>new release for them to include that field, if there was some 
>>benefit (e.g. user notification) to having the field.
>And that's the use case described in the PEP. So, now, we have to 
>choose if it's acceptable to make a new release for updating this 
>field, or not.

I was just pointing out that so far, 100% of the people who have 
given use cases for Obsoletes are fine with having to issue a new 
release to make it Obsoleted-By.  ;-)

(However, PyPI already allows metadata field editing of an 
already-released package, so as long as the new field were added to 
the existing editable fields, that'd be fine too.)



>>(Note that forked packages using the same API only conflict because 
>>of *installing conflicting files* -- so if you're thinking of a 
>>fork as your example here, it is leading you to confuse the ideas 
>>of Obsoletes and Conflicts.)
>I was not thinking about a fork, but about a new rename. That said, 
>I must admit that the difference between the two fields is sometimes 
>hard to see.

At this point, the only reasonable use of either is to inform the 
user about something, and in the case of Obsoletes, it's not 
necessarily even the user who's doing the installing!  If somebody 
installs Foo that depends on RuleDispatch, they are not necessarily 
in a position to port Foo to using PEAK-Rules.  So, only in the case 
where you are declaring a direct dependency from your package to the 
obsoleted package, or where you are directly installing the obsoleted 
package, do you even need to know anything about it.

In the case of conflicts, conflicts presuppose a specific runtime 
environment if the conflict is runtime environment related, and 
again, at best all that can be done is to inform the user.

If the conflict is runtime-process related (e.g. two libraries 
registering the same Python hook), then only a project that declares 
dependencies to both is affected, and only that project's authors 
need to be informed.

Given these conditions, it isn't clear that any of these three cases 
(obsoletes, environment conflict, in-process conflict) constitute 
something that an installation tool should be informing an end-user 
about, vs. the developer or integrator who's declaring the 
dependencies in the first place.

For example, it might make sense for developer-side tools like 
setuptools and buildout to check that one's declared dependencies 
don't have conflicts before pushing out an sdist or bdist or doing an 
install, and to warn for any obsoleted dependencies.  In contrast, an 
end-user tool like pip or easy_install would have little reason to 
check any of this, as their user isn't really affected by anything 
but environment-level conflicts that are unlikely to arise as an 
immediate result of installing the software.



>+1. This series of mails point out that we definitely need to talk a 
>bit more about the usability of the fields.
>Maybe are we missing something here, and maybe arent we talking 
>about the same type of conflicts. In any case, we need to clarify 
>all this *before* doing the implementation stuff.

I think that the main thing being missed is that these fields are 
being copied from a completely different kind of use case: the 
concept of a "package" as a bundle of preconfigured software that is 
also pre-integrated with a uniform operating environment.  In that 
context, it is the curators of that overall environment who are in a 
position to make judgments regarding what is a "conflict", or what 
packages will "obsolete" each other in the context of that environment.

Lacking such an overall curator organization in the PyPI environment 
(and not desiring one, anyway, since PyPI is inherently a software 
distribution *catalog* rather than a platform of its own), these 
fields make very little sense.  They are perhaps most useful as a 
centralized way to pass along notifications to developers (or to 
curator/integrators of system packages), but have little or no impact 
on the software installation process itself.

I also suspect that if you ask those curator/integrators of system 
packages, that they will say that the information in these fields 
would be of little value to them in any case, as it's unlikely that 
the author will *know* about the kinds of conflicts they'd be 
concerned about in their packaging process.  But it would certainly 
be worth asking them.

More to the point though...  I wonder whose idea it was to have these 
fields in the first place?  Neither PEP 314 nor PEP 345 don't really 
say, "Oh, such and such people asked for these fields for thus-and-so 
use cases", and in the absence of such requestors and use cases, it 
would probably be best just to drop them entirely.  (Unless of course 
Fred offers a description of what his actual use case for Conflicts 
is/was, and it doesn't fall into one of the categories previously discussed.)