Accessing tests_require and setup_requires from the setup.py
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py? Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything. Thanks, Chris
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script. You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route). Paul
On 03/06/16 14:19, Paul Moore wrote:
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script.
You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route).
As far as I can see (I checked setuptools and flake8), neither tests_require or setup_requires are present in the egg_info metadata directory. Is there no way of getting setuptools to write the data out to a file?
On 3 June 2016 at 14:24, Christopher Baines <mail@cbaines.net> wrote:
On 03/06/16 14:19, Paul Moore wrote:
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script.
You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route).
As far as I can see (I checked setuptools and flake8), neither tests_require or setup_requires are present in the egg_info metadata directory.
Is there no way of getting setuptools to write the data out to a file?
Maybe you could write your own command class? Or monkeypatch setuptools.setup() to write its arguments to a file? I don't know of any non-ugly way, though, sorry... Paul
Here is how you can write setup_requires and test_requires to a file, by adding a plugin to egg_info.writers in setuptools. https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3 On Fri, Jun 3, 2016 at 9:29 AM Paul Moore <p.f.moore@gmail.com> wrote:
On 3 June 2016 at 14:24, Christopher Baines <mail@cbaines.net> wrote:
On 03/06/16 14:19, Paul Moore wrote:
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script.
You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route).
As far as I can see (I checked setuptools and flake8), neither tests_require or setup_requires are present in the egg_info metadata directory.
Is there no way of getting setuptools to write the data out to a file?
Maybe you could write your own command class? Or monkeypatch setuptools.setup() to write its arguments to a file?
I don't know of any non-ugly way, though, sorry... Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
This ties into what I've been working on to fix the package dependency conflict resolution problem for pip <https://github.com/pypa/pip/issues/988>, actually: You may be able to use a tool I wrote to automatically extract requirements from setup.py, without installing (knowing that setup.py is arbitrary code and that dependencies are not strictly static). I opted to go with an admittedly drastic method of patching pip 8 to extract dependency data from each source distribution it touches in download mode when called by my dependency scraper. I decided that in the absence of static requirements for source distributions, the best I could really do in practice was to parse requirements exactly the way pip does. If you want, you can run the scraper from my project, which is here (project itself still a WIP). In particular, if you install it and run 'python depresolve/scrape_deps_and_detect_conflicts.py "some-package-name(1.0.0)"', it'll spit out the dependencies to a json file for the sdist for version 1.0.0 of some-package-name (more instructions here <https://github.com/awwad/depresolve#instructions-for-use-scraper> - it can also operate with local sdists or indexes). In my case, for pypa/pip:issue988, I needed to harvest mass dependency info to test a few different dependency conflict resolvers on. I'm working on writing up some of what I've learned and will probably end up recommending a basic integrated backtracking resolver within pip - probably an updated version of rbtcollins' backtracking resolver pip patches <https://github.com/pypa/pip/pull/2716> (which I'd be happy to rework and send a PR to pip on, if Robert doesn't have the bandwidth for it). Sebastien
On Jun 3, 2016, at 09:58, Daniel Holth <dholth@gmail.com> wrote:
Here is how you can write setup_requires and test_requires to a file, by adding a plugin to egg_info.writers in setuptools.
https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3 <https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3>
On Fri, Jun 3, 2016 at 9:29 AM Paul Moore <p.f.moore@gmail.com <mailto:p.f.moore@gmail.com>> wrote: On 3 June 2016 at 14:24, Christopher Baines <mail@cbaines.net <mailto:mail@cbaines.net>> wrote:
On 03/06/16 14:19, Paul Moore wrote:
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net <mailto:mail@cbaines.net>> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script.
You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route).
As far as I can see (I checked setuptools and flake8), neither tests_require or setup_requires are present in the egg_info metadata directory.
Is there no way of getting setuptools to write the data out to a file?
Maybe you could write your own command class? Or monkeypatch setuptools.setup() to write its arguments to a file?
I don't know of any non-ugly way, though, sorry... Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org <mailto:Distutils-SIG@python.org> https://mail.python.org/mailman/listinfo/distutils-sig <https://mail.python.org/mailman/listinfo/distutils-sig> _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Tell me what you know about SAT solvers, dnf and composer. On Fri, Jun 3, 2016, 12:28 Sebastien Awwad <sebastien.awwad@nyu.edu> wrote:
This ties into what I've been working on to fix the package dependency conflict resolution problem for pip <https://github.com/pypa/pip/issues/988>, actually:
You may be able to use a tool I wrote to automatically extract requirements from setup.py, without installing (knowing that setup.py is arbitrary code and that dependencies are not strictly static). I opted to go with an admittedly drastic method of patching pip 8 to extract dependency data from each source distribution it touches in download mode when called by my dependency scraper. I decided that in the absence of static requirements for source distributions, the best I could really do in practice was to parse requirements exactly the way pip does. If you want, you can run the scraper from my project, which is here (project itself still a WIP). In particular, if you install it and run 'python depresolve/scrape_deps_and_detect_conflicts.py "some-package-name(1.0.0)"', it'll spit out the dependencies to a json file for the sdist for version 1.0.0 of some-package-name (more instructions here <https://github.com/awwad/depresolve#instructions-for-use-scraper> - it can also operate with local sdists or indexes).
In my case, for pypa/pip:issue988, I needed to harvest mass dependency info to test a few different dependency conflict resolvers on. I'm working on writing up some of what I've learned and will probably end up recommending a basic integrated backtracking resolver within pip - probably an updated version of rbtcollins' backtracking resolver pip patches <https://github.com/pypa/pip/pull/2716> (which I'd be happy to rework and send a PR to pip on, if Robert doesn't have the bandwidth for it).
Sebastien
On Jun 3, 2016, at 09:58, Daniel Holth <dholth@gmail.com> wrote:
Here is how you can write setup_requires and test_requires to a file, by adding a plugin to egg_info.writers in setuptools.
https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3
On Fri, Jun 3, 2016 at 9:29 AM Paul Moore <p.f.moore@gmail.com> wrote:
On 3 June 2016 at 14:24, Christopher Baines <mail@cbaines.net> wrote:
On 03/06/16 14:19, Paul Moore wrote:
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script.
You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route).
As far as I can see (I checked setuptools and flake8), neither tests_require or setup_requires are present in the egg_info metadata directory.
Is there no way of getting setuptools to write the data out to a file?
Maybe you could write your own command class? Or monkeypatch setuptools.setup() to write its arguments to a file?
I don't know of any non-ugly way, though, sorry... Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
- I'm not at all familiar with composer. - For DNF - I assume you mean CNF, logical expression format generally used in SAT solvers (not DNF as in I did-not-finish the blog post I'm working on on this yet). If the question is why backtracking rather than SAT solving, I start to get into that in background here <https://github.com/awwad/depresolve/blob/master/docs/background.md#sat-solvi...>. I'm hesitant to link to this scribbling. I'll have a broader summary with data later - I've run some comparisons using enthought/depsolver's SAT solver, along with some backtrackers (the pip patch previously linked to and a currently-flawed one of my own). On Fri, Jun 3, 2016 at 12:46 PM Daniel Holth <dholth@gmail.com> wrote:
Tell me what you know about SAT solvers, dnf and composer.
On Fri, Jun 3, 2016, 12:28 Sebastien Awwad <sebastien.awwad@nyu.edu> wrote:
This ties into what I've been working on to fix the package dependency conflict resolution problem for pip <https://github.com/pypa/pip/issues/988>, actually:
You may be able to use a tool I wrote to automatically extract requirements from setup.py, without installing (knowing that setup.py is arbitrary code and that dependencies are not strictly static). I opted to go with an admittedly drastic method of patching pip 8 to extract dependency data from each source distribution it touches in download mode when called by my dependency scraper. I decided that in the absence of static requirements for source distributions, the best I could really do in practice was to parse requirements exactly the way pip does. If you want, you can run the scraper from my project, which is here (project itself still a WIP). In particular, if you install it and run 'python depresolve/scrape_deps_and_detect_conflicts.py "some-package-name(1.0.0)"', it'll spit out the dependencies to a json file for the sdist for version 1.0.0 of some-package-name (more instructions here <https://github.com/awwad/depresolve#instructions-for-use-scraper> - it can also operate with local sdists or indexes).
In my case, for pypa/pip:issue988, I needed to harvest mass dependency info to test a few different dependency conflict resolvers on. I'm working on writing up some of what I've learned and will probably end up recommending a basic integrated backtracking resolver within pip - probably an updated version of rbtcollins' backtracking resolver pip patches <https://github.com/pypa/pip/pull/2716> (which I'd be happy to rework and send a PR to pip on, if Robert doesn't have the bandwidth for it).
Sebastien
On Jun 3, 2016, at 09:58, Daniel Holth <dholth@gmail.com> wrote:
Here is how you can write setup_requires and test_requires to a file, by adding a plugin to egg_info.writers in setuptools.
https://gist.github.com/dholth/59e4c8a0c0d963b019d81e18bf0a89e3
On Fri, Jun 3, 2016 at 9:29 AM Paul Moore <p.f.moore@gmail.com> wrote:
On 3 June 2016 at 14:24, Christopher Baines <mail@cbaines.net> wrote:
On 03/06/16 14:19, Paul Moore wrote:
On 3 June 2016 at 13:20, Christopher Baines <mail@cbaines.net> wrote:
I'm trying to write a script to get information about a source distributions requirements (from the source distribution), but I'm not sure how to access the tests_require and setup_requires that can sometimes be found in the setup.py?
Apologies if this is really simple, and I've just missed the answer, but I've searched for it a few times now, and not come up with anything.
If I understand what you're trying to achieve, the only way of getting the "final" information (i.e, what will actually get used to install) is by running the setup.py script. That's basically the key issue with the executable setup.py format - there's no way to know the information without running the script.
You may be able to get the information without doing a full install by using the "setup.py egg_info" subcommand provided by setuptools. That's what pip uses, for example (but pip doesn't look at tests_require or setup_requires, so you'd have to check if that information was available by that route).
As far as I can see (I checked setuptools and flake8), neither tests_require or setup_requires are present in the egg_info metadata directory.
Is there no way of getting setuptools to write the data out to a file?
Maybe you could write your own command class? Or monkeypatch setuptools.setup() to write its arguments to a file?
I don't know of any non-ugly way, though, sorry... Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/03/2016 09:58 AM, Daniel Holth wrote:
Here is how you can write setup_requires and test_requires to a file, by adding a plugin to egg_info.writers in setuptools.
FWIW, eggtestinfo is an 'egg_info.writers' plugin which dumps test-related metadata to a text file: https://pypi.python.org/pypi/eggtestinfo Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXUdRpAAoJEPKpaDSJE9HYQhsP/jrMq4q1BgCCMCaazvo9B4vX uS9uzHj+T2noHwMcluSQB9zdSvCufNbKSoXQ62MUkbPEKwa1TarKzm+XZci/tBba eOxwEPilHHRBF7no44ZTgo6kq27oOC9Z0em7F9awADxWmJ0mcgOksQTXjzczuP7h BPoj1v/vxSLc87Q61qXYWMFTpiY4SjHn7+0xu7txQ13GJSaNjqNK/geQI+18Tpw2 76G0I1TDbXel+2JlgYEmwHkFPpFuhVaJXyJi8ZjvjdH/jN47oT6wsFwyxRzlVOXx oFp//hdb05K9i2R9rdLznKzkrQCrlYUTEBXEIpMCacfnoHgKrn+MKHCdI4FXKXGz Rm8LUh+oS0VU5IA8Rt4od08Qd68HE9CQERvx03JOQ7mKJIjxEODeC80JduzVGZxj 7YpA5sRxbsDTbnHD+8uodpFDDU0h9NMWLbxdDJ42Q1j2iNNr7ZZDR2tJZ8Y76EMl 55Mf5WZK5W+kzkUVodpK1yTxLR8+swadfjno2QnVhjA9h3vW1ZV11AREPTLauHbE S+eoyN92ga4eyhZSwaQdhz6KsZgFpUcmhwvlrsl0dIlkkHs1mFjPL/J4PvR2SVKl ug1Wmm0EvwPUQ0cDcRBeJtPF4git5SeMYLvGcMbsIpHdZloK+xaFVyMjgrTGA1o5 NTZokg2p47XvwuCA4XlH =d2oJ -----END PGP SIGNATURE-----
participants (6)
-
Christopher Baines -
Daniel Holth -
Paul Moore -
Sebastien Awwad -
Sebastien Awwad -
Tres Seaver