[Distutils] GSoC 2017 - Working on pip

[Sebastien Awwad]

While there may be some clever way of delivering changes to dependency
metadata that consumes less bandwidth (The first delivery for a client will
be somewhat large, but versioned metadata information plus compressed
deltas could move clients from one version of the full metadata set to
another, for example?), the larger problem, I think, is the lack of fixed
dependencies. Even with a moderately small percentage of distributions
having variable immediate dependencies, this expands out substantially when
you consider all distributions that depend on those and so on, meaning that
the full set of installed distributions when you run `pip install
xyz==a.b.c` is surprisingly variable. In a series of install attempts run
over about 400,000 of the package versions on PyPI last year, I found that
simply changing the version of Python employed in an otherwise identical
virtual environment results in pip installing different packages or package
versions, for 16% of the distributions.

If dependencies were knowable in static metadata, there would be a decent
case for SAT solving. I'll try to get back to a write-up after the current
rush on my main project subsides.

On Fri, Feb 10, 2017 at 2:34 PM Justin Cappos <jcappos at nyu.edu> wrote:

> Yes, don't use a SAT solver.  It requires all metadata from all packages
> (~30MB uncompressed) and gives hard to predict results in some cases.
> Also the lack of fixed dependencies is a substantial problem for a SAT
> solver.  Overall, we think it makes more sense to use a simple backtracking
> dependency resolution algorithm.
>
> Sebastien Awwad (CCed) has been looking at a bunch of data around the
> speed and other tradeoffs of the different algos.  Sebastien:  Sometime
> next week, can you write it up in a way that is suitable for sharing?
>
>
> Justin
>
> On Fri, Feb 10, 2017 at 1:59 PM, Wes Turner <wes.turner at gmail.com> wrote:
>
> From the discussion on
> https://github.com/pypa/pip/issues/988#issuecomment-279033079:
>
>
>    - https://github.com/ContinuumIO/pycosat (picosat)
>       - https://github.com/ContinuumIO/pycosat/blob/master/pycosat.c (C)
>       - https://github.com/ContinuumIO/pycosat/blob/master/picosat.c
>       - https://github.com/ContinuumIO/pycosat/tree/master/examples
>    - https://github.com/enthought/sat-solver (MiniSat)
>       -
>       https://github.com/enthought/sat-solver/tree/master/simplesat/tests
>       -
>       https://github.com/enthought/sat-solver/blob/master/requirements.txt (PyYAML,
>       enum34)
>
>
> Is there a better way than SAT?
>
> On Fri, Feb 10, 2017 at 12:20 PM, Pradyun Gedam <pradyunsg at gmail.com>
> wrote:
>
> Yay! Thank you so much for a prompt and positive response! I'm pretty
> excited and looking forward to this.
>
> On Thu, Feb 9, 2017, 20:23 Donald Stufft <donald at stufft.io> wrote:
>
> I’ve never done it before, but I’m happy to provide mentoring on this.
>
> On Feb 8, 2017, at 9:15 PM, Pradyun Gedam <pradyunsg at gmail.com> wrote:
>
> Hello Everyone!
>
> Ralf Gommers suggested that I put this proposal here on this list, for
> feedback and for seeing if anyone would be willing to mentor me. So, here
> it is.
>
> -----
>
> My name is Pradyun Gedam. I'm currently a first year student VIT
> University in India.
>
> I would like to apply for GSoC 2017 under PSF.
>
> I currently have a project in mind - the "pip needs a dependency resolver"
> issue [1]. I would like to take on this specific project but am willing to
> do some other project as well.
>
> For some background, around mid 2016, I started contributing to pip. The
> first issue I tackled was #59 [2] - a request for upgrade command and an
> upgrade-all command that has been open for over 5.5 years. Over the months
> following that, I've have had the opportunity to work with and understand
> multiple parts of pip's codebase while working on this issue and a few
> others. This search on GitHub issues [3] also provides a good summary of
> what work I've done on pip.
>
> [2]: https://github.com/pypa/pip/issues/988
> [2]: https://github.com/pypa/pip/issues/59
> [3]: https://github.com/pypa/pip/issues?q=author%3Apradyunsg
>
> Eagerly-waiting-for-a-response-ly,
> Pradyun Gedam
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
> —
>
> Donald Stufft
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170210/9b620e28/attachment-0001.html>