[Distutils] PyPI is a sick sick hoarder

[Donald Stufft]

> On May 16, 2015, at 1:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 17 May 2015 at 00:36, Justin Cappos <jcappos at nyu.edu> wrote:
>> This only considers computation cost though.  Other factors can become more
>> expensive than computation.  For example, SAT solvers need all the rules to
>> consider.  So a SAT solution needs to effectively download the full
>> dependency graph before starting.  A backtracking dependency resolver can
>> just download packages or dependency information as it considers them.
> 
> This is the defining consideration for pip at this point: a SAT solver
> requires publication of static dependency metadata on PyPI, which is
> dependent on both the Warehouse migration *and* the completion and
> acceptance of PEP 426. Propagation out to PyPI caching proxies and
> mirrors like devpi and the pulp-python plugin will then take even
> longer.
> 
> A backtracking resolver doesn't have those gating dependencies, as it
> can tolerate the current dynamic metadata model.
> 


Even when we have Warehouse and PEP 426, that only gives us that data going
forward, the 400k files that currently exist on PyPI still won’t have static
metadata. We could parse it out for Wheels but not for anything else. For the
foreseeable future any solution will need to be able to handle iteratively
finding constraints. Though I think a SAT solver can do it if it can handle
incremental solving or just by re-doing the SAT problem each time we discover
a new constraint.


---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150516/3cb63eb6/attachment.sig>