<div dir="ltr">While there may be some clever way of delivering changes to dependency metadata that consumes less bandwidth (The first delivery for a client will be somewhat large, but versioned metadata information plus compressed deltas could move clients from one version of the full metadata set to another, for example?), the larger problem, I think, is the lack of fixed dependencies. Even with a moderately small percentage of distributions having variable immediate dependencies, this expands out substantially when you consider all distributions that depend on those and so on, meaning that the full set of installed distributions when you run `pip install xyz==a.b.c` is surprisingly variable. In a series of install attempts run over about 400,000 of the package versions on PyPI last year, I found that simply changing the version of Python employed in an otherwise identical virtual environment results in pip installing different packages or package versions, for 16% of the distributions.<div><br></div><div>If dependencies were knowable in static metadata, there would be a decent case for SAT solving. I'll try to get back to a write-up after the current rush on my main project subsides.</div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Feb 10, 2017 at 2:34 PM Justin Cappos <<a href="mailto:jcappos@nyu.edu">jcappos@nyu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg">Yes, don't use a SAT solver.  It requires all metadata from all packages (~30MB uncompressed) and gives hard to predict results in some cases.   Also the lack of fixed dependencies is a substantial problem for a SAT solver.  Overall, we think it makes more sense to use a simple backtracking dependency resolution algorithm.  <div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Sebastien Awwad (CCed) has been looking at a bunch of data around the speed and other tradeoffs of the different algos.  Sebastien:  Sometime next week, can you write it up in a way that is suitable for sharing?    </div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg"><br class="gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Justin</div></div></div><div class="gmail_extra gmail_msg"><br class="gmail_msg"><div class="gmail_quote gmail_msg">On Fri, Feb 10, 2017 at 1:59 PM, Wes Turner <span dir="ltr" class="gmail_msg"><<a href="mailto:wes.turner@gmail.com" class="gmail_msg" target="_blank">wes.turner@gmail.com</a>></span> wrote:<br class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg">From the discussion on <a href="https://github.com/pypa/pip/issues/988#issuecomment-279033079" class="gmail_msg" target="_blank">https://github.com/pypa/pip/issues/988#issuecomment-279033079</a>:<div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><ul style="box-sizing:border-box;padding-left:2em;color:rgb(51,51,51);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:14px;margin-top:0px;margin-bottom:0px" class="gmail_msg"><li style="box-sizing:border-box;margin-left:0px" class="gmail_msg"><a href="https://github.com/ContinuumIO/pycosat" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/ContinuumIO/pycosat</a> (picosat)<ul style="box-sizing:border-box;padding-left:2em;margin-top:0px;margin-bottom:0px" class="gmail_msg"><li style="box-sizing:border-box;margin-left:0px" class="gmail_msg"><a href="https://github.com/ContinuumIO/pycosat/blob/master/pycosat.c" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/ContinuumIO/pycosat/blob/master/pycosat.c</a> (C)</li><li style="box-sizing:border-box;margin-top:0.25em;margin-left:0px" class="gmail_msg"><a href="https://github.com/ContinuumIO/pycosat/blob/master/picosat.c" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/ContinuumIO/pycosat/blob/master/picosat.c</a></li><li style="box-sizing:border-box;margin-top:0.25em;margin-left:0px" class="gmail_msg"><a href="https://github.com/ContinuumIO/pycosat/tree/master/examples" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/ContinuumIO/pycosat/tree/master/examples</a></li></ul></li><li style="box-sizing:border-box;margin-top:0.25em;margin-left:0px" class="gmail_msg"><a href="https://github.com/enthought/sat-solver" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/enthought/sat-solver</a> (MiniSat)</li><ul style="box-sizing:border-box;padding-left:2em;margin-top:0px;margin-bottom:0px" class="gmail_msg"><li style="box-sizing:border-box;margin-left:0px" class="gmail_msg"><a href="https://github.com/enthought/sat-solver/tree/master/simplesat/tests" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/enthought/sat-solver/tree/master/simplesat/tests</a></li><li style="box-sizing:border-box;margin-top:0.25em;margin-left:0px" class="gmail_msg"><a href="https://github.com/enthought/sat-solver/blob/master/requirements.txt" style="box-sizing:border-box;background-color:transparent;color:rgb(64,120,192);text-decoration:none" class="gmail_msg" target="_blank">https://github.com/enthought/sat-solver/blob/master/requirements.txt</a> (PyYAML, enum34)</li></ul></ul><div class="gmail_msg"><font color="#333333" face="-apple-system, blinkmacsystemfont, segoe ui, helvetica, arial, sans-serif, apple color emoji, segoe ui emoji, segoe ui symbol" class="gmail_msg"><span style="font-size:14px" class="gmail_msg"><br class="gmail_msg"></span></font></div></div><div class="gmail_msg"><font color="#333333" face="-apple-system, blinkmacsystemfont, segoe ui, helvetica, arial, sans-serif, apple color emoji, segoe ui emoji, segoe ui symbol" class="gmail_msg"><span style="font-size:14px" class="gmail_msg">Is there a better way than SAT?</span></font></div></div><div class="m_-4036516303427846524HOEnZb gmail_msg"><div class="m_-4036516303427846524h5 gmail_msg"><div class="gmail_extra gmail_msg"><br class="gmail_msg"><div class="gmail_quote gmail_msg">On Fri, Feb 10, 2017 at 12:20 PM, Pradyun Gedam <span dir="ltr" class="gmail_msg"><<a href="mailto:pradyunsg@gmail.com" class="gmail_msg" target="_blank">pradyunsg@gmail.com</a>></span> wrote:<br class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Yay! Thank you so much for a prompt and positive response! I'm pretty excited and looking forward to this.</div><div class="m_-4036516303427846524m_6186889061556752972HOEnZb gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972h5 gmail_msg"><span class="gmail_msg">
</span><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="gmail_quote m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div dir="ltr" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">On Thu, Feb 9, 2017, 20:23 Donald Stufft <<a href="mailto:donald@stufft.io" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">donald@stufft.io</a>> wrote:<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"></div><blockquote class="gmail_quote m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">I’ve never done it before, but I’m happy to provide mentoring on this.</div><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><blockquote type="cite" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"></blockquote></div></div><div style="word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><blockquote type="cite" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">On Feb 8, 2017, at 9:15 PM, Pradyun Gedam <<a href="mailto:pradyunsg@gmail.com" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">pradyunsg@gmail.com</a>> wrote:</div><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546m_3275078904521864441m_6600394735712870104Apple-interchange-newline m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"></blockquote></div></div><div style="word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><blockquote type="cite" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div dir="ltr" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div dir="ltr" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div dir="ltr" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div dir="ltr" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Hello Everyone!<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Ralf Gommers suggested that I put this proposal here on this list, for 
feedback and for seeing if anyone would be willing to mentor me. So, here it is.<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">-----<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">My name is Pradyun Gedam. I'm currently a first year student VIT University in India.<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">I would like to apply for GSoC 2017 under PSF.<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">I currently have a project in mind - the "pip needs a dependency resolver" issue [1]. I would like to take on this specific project but am willing to do some other project as well.<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">For some background, around mid 2016, I started contributing to pip. The first issue I tackled was #59 [2] - a request for upgrade command and an upgrade-all command that has been open for over 5.5 years. Over the months following that, I've have had the opportunity to work with and understand multiple parts of pip's codebase while working on this issue and a few others. This search on GitHub issues [3] also provides a good summary of what work I've done on pip.<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">[2]: <a href="https://github.com/pypa/pip/issues/988" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">https://github.com/pypa/pip/issues/988</a><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">[2]: <a href="https://github.com/pypa/pip/issues/59" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">https://github.com/pypa/pip/issues/59</a><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">[3]: <a href="https://github.com/pypa/pip/issues?q=author%3Apradyunsg" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">https://github.com/pypa/pip/issues?q=author%3Apradyunsg</a><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Eagerly-waiting-for-a-response-ly,<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Pradyun Gedam</div></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><blockquote type="cite" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">
_______________________________________________<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Distutils-SIG maillist  -  <a href="mailto:Distutils-SIG@python.org" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">Distutils-SIG@python.org</a><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><a href="https://mail.python.org/mailman/listinfo/distutils-sig" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"></div></blockquote></div></div><div style="word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><blockquote type="cite" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"></div></blockquote></div><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">
<div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">—</div></div></div><div style="word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"><br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg">Donald Stufft<br class="m_-4036516303427846524m_6186889061556752972m_-3193125452219004546gmail_msg gmail_msg"></div></div></div></blockquote></div>
</div></div><br class="gmail_msg">_______________________________________________<br class="gmail_msg">
Distutils-SIG maillist  -  <a href="mailto:Distutils-SIG@python.org" class="gmail_msg" target="_blank">Distutils-SIG@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/distutils-sig" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class="gmail_msg">
<br class="gmail_msg"></blockquote></div><br class="gmail_msg"></div>
</div></div><br class="gmail_msg">_______________________________________________<br class="gmail_msg">
Distutils-SIG maillist  -  <a href="mailto:Distutils-SIG@python.org" class="gmail_msg" target="_blank">Distutils-SIG@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/distutils-sig" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class="gmail_msg">
<br class="gmail_msg"></blockquote></div><br class="gmail_msg"></div>
</blockquote></div>