[Distutils] Parsing requirements, pip has no API ...

Donald Stufft donald at stufft.io
Sat Feb 14 00:11:17 CET 2015

> On Feb 13, 2015, at 4:44 PM, Ian Cordasco <graffatcolmingov at gmail.com> wrote:
> pip is a command-line tool. The fact that you can import it doesn't
> make it a library. If it documents no public API then it has none and
> you shouldn't be relying on things that you can import from pip. You
> can import requests from pip but you shouldn't do that either.
> There seems to be a need for a separate library for this use case but
> there isn't at the moment. In general, requirements.txt seems to be an
> anti-pattern. You either have to use likely to break tooling or you'll
> have to reinvent that from scratch. You're better off putting it
> directly in setup.py and using setup.py to install dependencies in a
> virtualenv instead of requirements.txt
> On Fri, Feb 13, 2015 at 3:27 PM, Thomas Güttler
> <guettliml at thomas-guettler.de> wrote:
>> I was told:
>> {{{
>> Pip does not have a public API and because of that there is no backwards compatibility contract. It's impossible to fully parse every type of requirements.txt without a session so either parse_requirements needs to create one if it doesn't (which means if we forget to pass in a session somewhere it'll use the wrong one) or it needs one passed in.
>> }}}
>> From https://github.com/pypa/pip/issues/2422#issuecomment-74271718
>> Up to now we used parse_requirements() of pip, but in new versions you need to pass in a
>> session.
>> If I see changes like this:
>> setup.py
>> - install_requires=[str(req.req) for req in parse_requirements("requirements.txt")],
>> + install_requires=[str(req.req) for req in parse_requirements("requirements.txt", session=uuid.uuid1())],
>> ... I think something is wrong.
>> I am not an expert in python packaging details. I just want it to work.
>> What is wrong here?
>>  - You should not use parse_requirements() in setup.py
>>  - pip should not change its API.
>>  - you should not use pip at all, you should use ...?
>> Regards,
>>  Thomas
>> --
>> http://www.thomas-guettler.de/
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

requirements.txt and setup.py serve different purposes, requirements.txt is for
an environment, setup.py is for a package. It doesn't make sense for a setup.py
to read from a requirement.txt just like it wouldn't make sense for a deb
package to read from a Chef cookbook.

Often the reason people do this is they want to support people installing their
thing with ``pip install -r requirements.txt`` from within a check out without
needing to list their dependencies twice. That's a reasonable thing to want
which is why the requirements file format has a construct that enables it,
simply make a requirements.txt file that contains "." or "-e ." and pip will
automatically install the project and all of it's dependencies.

For more information, see https://caremad.io/2013/07/setup-vs-requirement/.

Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150213/cfc40ee7/attachment.sig>

More information about the Distutils-SIG mailing list