[Distutils] Please do not remove dependency_links

[Hannes Schmidt]

Thank you for your suggestions, Donald. They are both feasible but they
feel like workarounds to me.

Manually listing transitive dependencies seems like a step backwards to me.
Isn't the whole point of setuptools/distutils that each component manages
its own list of dependencies? How will this scale to dozens of transitive
dependencies and busy development teams where dependencies are revved
frequently. I really liked what Maven did for Java and therefore liked what
pip is trying to do for Python.

I haven't used --find-links yet so I may be wrong but tarring up packages
and serving them by a web server is additional work that I'd rather avoid.
It just creates yet another copy of the package and requires maintaining an
additional server component.

I loved the fact that I could point pip directly at my source repos for
both my top-level projects as well as their dependencies. That seemed like
a perfect fit for an interpreted language like Python where packages are
distributed in source form. Removing dependency_links makes that feature
useless to me because it doesn't extend to dependencies, only top level
components.


On Fri, Jan 17, 2014 at 4:44 AM, Donald Stufft <donald at stufft.io> wrote:

>
> On Jan 16, 2014, at 9:52 PM, Hannes Schmidt <hannes at ucsc.edu> wrote:
>
> I read through the "Removing dependency_links" thread [1] and I beg you
> not follow through with the deprecation and removal of dependency_links and
> to rethink your approach.
>
> The mentioned thread indicates that research was done to gauge the
> popularity of the dependency_links in publicly hosted Python projects. That
> approach is fundamentally flawed: Publicly hosted projects are much more
> likely to also be available on PyPI than private, closed-source projects.
> Consequently, their dependencies are also more likely to be hosted on PyPI
> as well. Because of that, they are much less likely to rely on the
> dependency_links feature.
>
> Another misconception seem to be that dependency_links is predominantly
> used for installing patched or customized versions of dependencies hosted
> on PyPI. I'm pretty sure the predominant use case for dependency_links is
> with projects that are hosted privately, e.g. for an organization's
> internal use. I represent such an organization and removing
> dependency_links would impact us negatively. We host a set of internal
> projects and their dependencies on Bitbucket and we rely on
> dependency_links to install them directly from there.
>
> I understand the motivation for this change – security – but there must be
> smarter way to handle it. Could we fallback to dependency_links if a PyPI
> lookup isn't successful? Could we restrict dependency_links to links that
> share a prefix with the link from which the package is currently being
> installed? A combination of the two?
>
> [1]:
> https://mail.python.org/pipermail/distutils-sig/2013-October/022937.html
>
>
>
> Pip 1.5 has already been released which turns dependency links off by
> default and adds a flag —process-dependency-links and they are
> scheduled to be removed in 1.6. It’s possible we can “undeprecate” them
> and leave them off by default however my question to you would be why
> you can’t use a requirements.txt file to handle installing things from
> bitbucket?
>
> Fundamentally a setup.py should not contain “concrete” locations but
> instead
> should contain abstract names/version specifiers whereas a requirements.txt
> (or the pip invocation itself) takes those abstract names and makes them
> concrete by pairing them with an index.
>
> So to be more explicit, is there any reason why one of:
>
> 1) Create a requirements.txt and point to bitbucket in that and install
> your
>     environment using ``pip install -r requirements.txt``
> 2) Create tar balls of your packages and host them in a directory somewhere
>     behind nginx and use —find-links https://example.com/that-directory/as
>     your install location.
>
> Won’t work just as fine for you?
>
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>


-- 
Hannes Schmidt
Software Application Developer
Data Migration Engineer
Cancer Genomics Hub
University of California, Santa Cruz

(206) 696-2316 (cell)
hannes at ucsc.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140117/a6775b42/attachment.html>