[Distutils] Accessing tests_require and setup_requires from the setup.py

Sebastien Awwad sebastien.awwad at nyu.edu
Fri Jun 3 12:28:20 EDT 2016


This ties into what I've been working on to fix the package dependency conflict resolution problem for pip <https://github.com/pypa/pip/issues/988>, actually:

You may be able to use a tool I wrote to automatically extract requirements from setup.py, without installing (knowing that setup.py is arbitrary code and that dependencies are not strictly static). I opted to go with an admittedly drastic method of patching pip 8 to extract dependency data from each source distribution it touches in download mode when called by my dependency scraper. I decided that in the absence of static requirements for source distributions, the best I could really do in practice was to parse requirements exactly the way pip does. If you want, you can run the scraper from my project, which is here (project itself still a WIP). In particular, if you install it and run 'python depresolve/scrape_deps_and_detect_conflicts.py "some-package-name(1.0.0)"', it'll spit out the dependencies to a json file for the sdist for version 1.0.0 of some-package-name (more instructions here <https://github.com/awwad/depresolve#instructions-for-use-scraper> - it can also operate with local sdists or indexes).

In my case, for pypa/pip:issue988, I needed to harvest mass dependency info to test a few different dependency conflict resolvers on. I'm working on writing up some of what I've learned and will probably end up recommending a basic integrated backtracking resolver within pip - probably an updated version of rbtcollins' backtracking resolver pip patches <https://github.com/pypa/pip/pull/2716> (which I'd be happy to rework and send a PR to pip on, if Robert doesn't have the bandwidth for it).

Sebastien

> On Jun 3, 2016, at 09:58, Daniel Holth <dholth at gmail.com> wrote:
> 
> Here is how you can write setup_requires and test_requires to a file, by adding a plugin to egg_info.writers in setuptools.
> 
> https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3 <https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3>
> 
> On Fri, Jun 3, 2016 at 9:29 AM Paul Moore <p.f.moore at gmail.com <mailto:p.f.moore at gmail.com>> wrote:
> On 3 June 2016 at 14:24, Christopher Baines <mail at cbaines.net <mailto:mail at cbaines.net>> wrote:
> > On 03/06/16 14:19, Paul Moore wrote:
> >> On 3 June 2016 at 13:20, Christopher Baines <mail at cbaines.net <mailto:mail at cbaines.net>> wrote:
> >>> I'm trying to write a script to get information about a source
> >>> distributions requirements (from the source distribution), but I'm not
> >>> sure how to access the tests_require and setup_requires that can
> >>> sometimes be found in the setup.py?
> >>>
> >>> Apologies if this is really simple, and I've just missed the answer, but
> >>> I've searched for it a few times now, and not come up with anything.
> >>
> >> If I understand what you're trying to achieve, the only way of getting
> >> the "final" information (i.e, what will actually get used to install)
> >> is by running the setup.py script. That's basically the key issue with
> >> the executable setup.py format - there's no way to know the
> >> information without running the script.
> >>
> >> You may be able to get the information without doing a full install by
> >> using the "setup.py egg_info" subcommand provided by setuptools.
> >> That's what pip uses, for example (but pip doesn't look at
> >> tests_require or setup_requires, so you'd have to check if that
> >> information was available by that route).
> >
> > As far as I can see (I checked setuptools and flake8), neither
> > tests_require or setup_requires are present in the egg_info metadata
> > directory.
> >
> > Is there no way of getting setuptools to write the data out to a file?
> 
> Maybe you could write your own command class? Or monkeypatch
> setuptools.setup() to write its arguments to a file?
> 
> I don't know of any non-ugly way, though, sorry...
> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org <mailto:Distutils-SIG at python.org>
> https://mail.python.org/mailman/listinfo/distutils-sig <https://mail.python.org/mailman/listinfo/distutils-sig>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160603/e276ee6f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160603/e276ee6f/attachment-0001.sig>


More information about the Distutils-SIG mailing list