New to tox; seeing strange behavior creating venvs
So for years I worked in a buildout shop, and now I'm in a Debian-packages-are-good-enough shop, and I'm looking to improve that situation. For some of my libraries at work, I want to get cross-version testing straightened out a bit, and tox seems to be the state of the art. (Yes, Python 2.7 is part of the mix, but getting past that means getting library tests running on Python 3 before we tackle applications.) I'm using a fairly simple tox.ini file, defining three environments built on top of "clean" Python installations: ------------------------ [tox] envlist = py27,py35,py36 [testenv] commands = python -m unittest discover -p *tests.py -p *test.py install_command = pip install -r tox-requirements.txt {opts} {packages} [testenv:py27] basepython = /opt/kt-python27/bin/python2 [testenv:py35] basepython = /opt/kt-python35/bin/python3 [testenv:py36] basepython = /opt/kt-python36/bin/python3 ------------------------ The tox-requirements.txt is pretty minimal, and doesn't do anything particularly interesting, but does include options to use a local DevPI server instead of PyPI. My project has a venv based on my clean python 3.6 installation. tox is installed into the venv. When running bin/tox, tox creates an sdist for my library package, then creates the venvs and installs my package into them. When nearly complete, the installation fails because of an attempt to write a script into the system /usr/bin, which shouldn't happen in the venv: ------------------------ GLOB sdist-make: /home/fdrake/p/ks3.mediaflux/setup.py py27 create: /home/fdrake/p/ks3.mediaflux/.tox/py27 py27 inst: /home/fdrake/p/ks3.mediaflux/.tox/dist/ks3.mediaflux-3.5.0.zip ERROR: invocation failed (exit code 1), logfile: /home/fdrake/p/ks3.mediaflux/.tox/py27/log/py27-1.log ERROR: actionid: py27 msg: installpkg cmdargs: ['/home/fdrake/p/ks3.mediaflux/.tox/py27/bin/pip', 'install', '-r', 'tox-requirements.txt', '/home/fdrake/p/ks3.mediaflux/.tox/dist/ks3.mediaflux-3.5.0.zip'] Looking in indexes: http://devpi/root/pypi/+simple/, http://devpi/keeper/keepertech/+simple/ Processing ./.tox/dist/ks3.mediaflux-3.5.0.zip ... Successfully built ks3.mediaflux Installing collected packages: six, funcsigs, pbr, mock, idna, chardet, certifi, urllib3, requests, kt.testing, zc.lockfile, kt.common, pytz, itsdangerous, Werkzeug, click, MarkupSafe, Jinja2, Flask, aniso8601, flask-RESTful, netaddr, kt.restful, ks3.mediaflux Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/bin/pbr' Consider using the `--user` option or check the permissions. ------------------------ If I manually cd into .tox/py27 and running bin/pip install -r ../../tox-requirements.txt succeeds, and there's no attempt to write into /usr/bin/. Doing that for each of the environments allows tox to run the tests and produce the expected summary. At this point, I presume I've done some ridiculous in my configuration somewhere. Is there something obvious to others that I'm just missing because I'm not familiar with the tools? Thanks! -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
Hello Fred, you are entering this on the deep end and there is a lot of things that might yield interesting results, so you are definitely not doinng something ridiculous, but a lot of unusual things and some things that give me the feeling that the tox docs need some serious TLC :) Just shooting a few things from the hip here without having time to look in more detail, but I will try to give you some pointers: The main red flag for me is pbr. Never having used it myself - I have no idea what it is doing exactly and how that might interfer with tox and "normal" virtualenvs, but I can remember dimly reading a discussion where someone struggled with getting this to work together with tox, so you should maybe first try to get some simpler project working without using pbr and then work your way up. The pbr folks use tox themselves, so if you are still running into problems them, mabye they might be better suited to help you. Now for some more obvious things:
install_command = pip install -r tox-requirements.txt {opts} {packages}
should be: deps = -rtox-requirements.txt or even simper: listing the packages directly in tox deps instead of a special tox requirements file. Or you csan have the test deps as extras or tests_require as part of your setup.py. Read up about it here: * https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras... * specifiying extras in tox: http://tox.readthedocs.io/en/latest/config.html#confval-extras=MULTI-LINE-LI... simplest way for now would be deps = pytest flexmock # or whatever stuff you only need to run tests with in tox envs Then:
basepython = /op/...all kind of interesting interpreters
basepython chooses the python that the venv should be built with, so you might have a different expectation here and I am not even quite sure what is supposed to happen here, as I never have played around with interesting basepython settings yet. So. Have a look at http://tox.readthedocs.io/en/latest/config.html#confval-basepython=NAME-OR-P... and decide for yourself and observe with tox -vvv what exactly is happening. Having the indexes in the requirements file is another interesting setting that I personally have never used. You can set the index also in tox, so maybe you want to experiment with that: http://tox.readthedocs.io/en/latest/config.html#confval-indexserver Cheers, Oliver On Wed, 2 May 2018 at 19:51 Fred Drake <fred@fdrake.net> wrote:
So for years I worked in a buildout shop, and now I'm in a Debian-packages-are-good-enough shop, and I'm looking to improve that situation.
For some of my libraries at work, I want to get cross-version testing straightened out a bit, and tox seems to be the state of the art. (Yes, Python 2.7 is part of the mix, but getting past that means getting library tests running on Python 3 before we tackle applications.)
I'm using a fairly simple tox.ini file, defining three environments built on top of "clean" Python installations:
------------------------ [tox] envlist = py27,py35,py36
[testenv] commands = python -m unittest discover -p *tests.py -p *test.py
install_command = pip install -r tox-requirements.txt {opts} {packages}
[testenv:py27] basepython = /opt/kt-python27/bin/python2
[testenv:py35] basepython = /opt/kt-python35/bin/python3
[testenv:py36] basepython = /opt/kt-python36/bin/python3 ------------------------
The tox-requirements.txt is pretty minimal, and doesn't do anything particularly interesting, but does include options to use a local DevPI server instead of PyPI.
My project has a venv based on my clean python 3.6 installation. tox is installed into the venv.
When running bin/tox, tox creates an sdist for my library package, then creates the venvs and installs my package into them. When nearly complete, the installation fails because of an attempt to write a script into the system /usr/bin, which shouldn't happen in the venv:
------------------------ GLOB sdist-make: /home/fdrake/p/ks3.mediaflux/setup.py py27 create: /home/fdrake/p/ks3.mediaflux/.tox/py27 py27 inst: /home/fdrake/p/ks3.mediaflux/.tox/dist/ks3.mediaflux-3.5.0.zip ERROR: invocation failed (exit code 1), logfile: /home/fdrake/p/ks3.mediaflux/.tox/py27/log/py27-1.log ERROR: actionid: py27 msg: installpkg cmdargs: ['/home/fdrake/p/ks3.mediaflux/.tox/py27/bin/pip', 'install', '-r', 'tox-requirements.txt', '/home/fdrake/p/ks3.mediaflux/.tox/dist/ks3.mediaflux-3.5.0.zip']
Looking in indexes: http://devpi/root/pypi/+simple/, http://devpi/keeper/keepertech/+simple/ Processing ./.tox/dist/ks3.mediaflux-3.5.0.zip ... Successfully built ks3.mediaflux Installing collected packages: six, funcsigs, pbr, mock, idna, chardet, certifi, urllib3, requests, kt.testing, zc.lockfile, kt.common, pytz, itsdangerous, Werkzeug, click, MarkupSafe, Jinja2, Flask, aniso8601, flask-RESTful, netaddr, kt.restful, ks3.mediaflux Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/bin/pbr' Consider using the `--user` option or check the permissions. ------------------------
If I manually cd into .tox/py27 and running bin/pip install -r ../../tox-requirements.txt succeeds, and there's no attempt to write into /usr/bin/. Doing that for each of the environments allows tox to run the tests and produce the expected summary.
At this point, I presume I've done some ridiculous in my configuration somewhere. Is there something obvious to others that I'm just missing because I'm not familiar with the tools?
Thanks!
-Fred
-- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein _______________________________________________ tox-dev mailing list tox-dev@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
OpenStack uses pbr and tox together all the time, but they wrote pbr. Further, I wouldn't recommend it but it does have some nice features so I wouldn't strongly advise against it Sent from my phone with my typo-happy thumbs. Please excuse my brevity On Wed, May 2, 2018, 15:58 Oliver Bestwalter <oliver@bestwalter.de> wrote:
Hello Fred,
you are entering this on the deep end and there is a lot of things that might yield interesting results, so you are definitely not doinng something ridiculous, but a lot of unusual things and some things that give me the feeling that the tox docs need some serious TLC :)
Just shooting a few things from the hip here without having time to look in more detail, but I will try to give you some pointers:
The main red flag for me is pbr. Never having used it myself - I have no idea what it is doing exactly and how that might interfer with tox and "normal" virtualenvs, but I can remember dimly reading a discussion where someone struggled with getting this to work together with tox, so you should maybe first try to get some simpler project working without using pbr and then work your way up. The pbr folks use tox themselves, so if you are still running into problems them, mabye they might be better suited to help you.
Now for some more obvious things:
install_command = pip install -r tox-requirements.txt {opts} {packages}
should be:
deps = -rtox-requirements.txt
or even simper: listing the packages directly in tox deps instead of a special tox requirements file. Or you csan have the test deps as extras or tests_require as part of your setup.py. Read up about it here:
* https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras... * specifiying extras in tox: http://tox.readthedocs.io/en/latest/config.html#confval-extras=MULTI-LINE-LI...
simplest way for now would be
deps = pytest flexmock # or whatever stuff you only need to run tests with in tox envs
Then:
basepython = /op/...all kind of interesting interpreters
basepython chooses the python that the venv should be built with, so you might have a different expectation here and I am not even quite sure what is supposed to happen here, as I never have played around with interesting basepython settings yet. So. Have a look at http://tox.readthedocs.io/en/latest/config.html#confval-basepython=NAME-OR-P... and decide for yourself and observe with tox -vvv what exactly is happening.
Having the indexes in the requirements file is another interesting setting that I personally have never used. You can set the index also in tox, so maybe you want to experiment with that: http://tox.readthedocs.io/en/latest/config.html#confval-indexserver
Cheers, Oliver
On Wed, 2 May 2018 at 19:51 Fred Drake <fred@fdrake.net> wrote:
So for years I worked in a buildout shop, and now I'm in a Debian-packages-are-good-enough shop, and I'm looking to improve that situation.
For some of my libraries at work, I want to get cross-version testing straightened out a bit, and tox seems to be the state of the art. (Yes, Python 2.7 is part of the mix, but getting past that means getting library tests running on Python 3 before we tackle applications.)
I'm using a fairly simple tox.ini file, defining three environments built on top of "clean" Python installations:
------------------------ [tox] envlist = py27,py35,py36
[testenv] commands = python -m unittest discover -p *tests.py -p *test.py
install_command = pip install -r tox-requirements.txt {opts} {packages}
[testenv:py27] basepython = /opt/kt-python27/bin/python2
[testenv:py35] basepython = /opt/kt-python35/bin/python3
[testenv:py36] basepython = /opt/kt-python36/bin/python3 ------------------------
The tox-requirements.txt is pretty minimal, and doesn't do anything particularly interesting, but does include options to use a local DevPI server instead of PyPI.
My project has a venv based on my clean python 3.6 installation. tox is installed into the venv.
When running bin/tox, tox creates an sdist for my library package, then creates the venvs and installs my package into them. When nearly complete, the installation fails because of an attempt to write a script into the system /usr/bin, which shouldn't happen in the venv:
------------------------ GLOB sdist-make: /home/fdrake/p/ks3.mediaflux/setup.py py27 create: /home/fdrake/p/ks3.mediaflux/.tox/py27 py27 inst: /home/fdrake/p/ks3.mediaflux/.tox/dist/ks3.mediaflux-3.5.0.zip ERROR: invocation failed (exit code 1), logfile: /home/fdrake/p/ks3.mediaflux/.tox/py27/log/py27-1.log ERROR: actionid: py27 msg: installpkg cmdargs: ['/home/fdrake/p/ks3.mediaflux/.tox/py27/bin/pip', 'install', '-r', 'tox-requirements.txt', '/home/fdrake/p/ks3.mediaflux/.tox/dist/ks3.mediaflux-3.5.0.zip']
Looking in indexes: http://devpi/root/pypi/+simple/, http://devpi/keeper/keepertech/+simple/ Processing ./.tox/dist/ks3.mediaflux-3.5.0.zip ... Successfully built ks3.mediaflux Installing collected packages: six, funcsigs, pbr, mock, idna, chardet, certifi, urllib3, requests, kt.testing, zc.lockfile, kt.common, pytz, itsdangerous, Werkzeug, click, MarkupSafe, Jinja2, Flask, aniso8601, flask-RESTful, netaddr, kt.restful, ks3.mediaflux Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/bin/pbr' Consider using the `--user` option or check the permissions. ------------------------
If I manually cd into .tox/py27 and running bin/pip install -r ../../tox-requirements.txt succeeds, and there's no attempt to write into /usr/bin/. Doing that for each of the environments allows tox to run the tests and produce the expected summary.
At this point, I presume I've done some ridiculous in my configuration somewhere. Is there something obvious to others that I'm just missing because I'm not familiar with the tools?
Thanks!
-Fred
-- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein _______________________________________________ tox-dev mailing list tox-dev@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
_______________________________________________ tox-dev mailing list tox-dev@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
On Wed, May 2, 2018 at 4:58 PM, Oliver Bestwalter <oliver@bestwalter.de> wrote:
Hello Fred,
you are entering this on the deep end and there is a lot of things that might yield interesting results, so you are definitely not doinng something ridiculous, but a lot of unusual things and some things that give me the feeling that the tox docs need some serious TLC :)
Yeah, well, tox and every other testing and packaging tool for Python. :-) Thanks for taking the time to respond! I'm not (yet) sure what's requiring pbr; that's indirect.
Just shooting a few things from the hip here without having time to look in more detail, but I will try to give you some pointers:
The main red flag for me is pbr. Never having used it myself - I have no idea what it is doing exactly and how that might interfer with tox and "normal" virtualenvs, but I can remember dimly reading a discussion where someone struggled with getting this to work together with tox, so you should maybe first try to get some simpler project working without using pbr and then work your way up. The pbr folks use tox themselves, so if you are still running into problems them, mabye they might be better suited to help you.
Now for some more obvious things:
install_command = pip install -r tox-requirements.txt {opts} {packages}
should be:
deps = -rtox-requirements.txt
or even simper: listing the packages directly in tox deps instead of a special tox requirements file. Or you csan have the test deps as extras or tests_require as part of your setup.py. Read up about it here:
I've reduced this to: deps = kt.testing
Then:
basepython = /op/...all kind of interesting interpreters
basepython chooses the python that the venv should be built with, so you might have a different expectation here and I am not even quite sure what is supposed to happen here, as I never have played around with interesting basepython settings yet. So. Have a look at http://tox.readthedocs.io/en/latest/config.html#confval-basepython=NAME-OR-P... and decide for yourself and observe with tox -vvv what exactly is happening.
This seems to all be handled as expected; there are no visible references to the system Python (this is an Ubuntu 16.04 VM). I didn't see how else to tell tox where the Python interpreter was, and they're not on my PATH. Hence, [testenv:*] basepython.
Having the indexes in the requirements file is another interesting setting that I personally have never used. You can set the index also in tox, so maybe you want to experiment with that: http://tox.readthedocs.io/en/latest/config.html#confval-indexserver
I was avoiding that because of the deprecation warning, even though the doc doesn't point to an alternative. Also, currently using both --index-url and --extra-index-url. I should be able to create another index in my DevPI that combines my local packages and the root/pypi index. Something to try tomorrow. -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
On Wed, May 2, 2018 at 4:58 PM, Oliver Bestwalter <oliver@bestwalter.de> wrote:
Having the indexes in the requirements file is another interesting setting that I personally have never used. You can set the index also in tox, so maybe you want to experiment with that: http://tox.readthedocs.io/en/latest/config.html#confval-indexserver
On Wed, May 2, 2018 at 7:53 PM, Fred Drake <fred@fdrake.net> wrote:
I was avoiding that because of the deprecation warning, even though the doc doesn't point to an alternative. Also, currently using both --index-url and --extra-index-url. I should be able to create another index in my DevPI that combines my local packages and the root/pypi index. Something to try tomorrow.
OK, I have my DevPI providing a suitable index that advertises all the packages in a single index. My tox.ini now directly references that index: indexserver = default = http://...@devpi/keeper/prod/+simple/ But it's ignored, because the index is currently not behind SSL. (I'm working on getting something suitable set up, with SSL and LDAP authorization set up with our A/D server, but that's not ready yet.) -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
On Fri, May 4, 2018 at 10:45 AM, Fred Drake <fred@fdrake.net> wrote:
But it's ignored, because the index is currently not behind SSL. (I'm working on getting something suitable set up, with SSL and LDAP authorization set up with our A/D server, but that's not ready yet.)
This much is solved by bringing back install_command: ---------------- [tox] envlist = py27,py35,py36 indexserver = default = http://keeper:Keeper1SS@devpi/keeper/prod/+simple/ [testenv] commands = python -m unittest discover -p *test.py -p *tests.py deps = kt.testing install_command = pip install --trusted-host devpi {opts} {packages} ---------------- Now back to the earlier problem of understanding why any package install wants to touch /usr/bin/. -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
On Fri, May 4, 2018 at 11:31 AM, Fred Drake <fred@fdrake.net> wrote:
Now back to the earlier problem of understanding why any package install wants to touch /usr/bin/.
/home/fdrake/p/ks3.mediaflux/.tox/log/tox-0.log GLOB finish: packaging after 0.22 seconds copying new sdistfile to '/home/fdrake/.tox/distshare/ks3.mediaflux-3.5.0.zip'
/home/fdrake/p/ks3.mediaflux/.tox/py36/log/py36-0.log
Using the attached tox.ini, I'm seeing pip attempt to write scripts into /usr/bin/, even though it's not using a Python that's installed there: ------------------------ $ bin/tox -vvv -e py36 using tox.ini: /home/fdrake/p/ks3.mediaflux/tox.ini using tox-3.0.0 from /home/fdrake/p/ks3.mediaflux/lib/python3.6/site-packages/tox/__init__.py GLOB start: packaging GLOB sdist-make: /home/fdrake/p/ks3.mediaflux/setup.py /home/fdrake/p/ks3.mediaflux$ /home/fdrake/p/ks3.mediaflux/bin/python3 /home/fdrake/p/ks3.mediaflux/setup.py sdist --formats=zip --dist-dir /home/fdrake/p/ks3.mediaflux/.tox/dist py36 start: getenv /home/fdrake/p/ks3.mediaflux/.tox/py36 py36 create: /home/fdrake/p/ks3.mediaflux/.tox/py36 setting PATH=/home/fdrake/p/ks3.mediaflux/.tox/py36/bin:/home/fdrake/bin:/home/fdrake/.local/bin:/home/fdrake/.nvm/versions/node/v6.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin /home/fdrake/p/ks3.mediaflux/.tox$ /home/fdrake/p/ks3.mediaflux/bin/python3 -m virtualenv --python /opt/kt-python36/bin/python3 py36 py36 installdeps: kt.testing setting PATH=/home/fdrake/p/ks3.mediaflux/.tox/py36/bin:/home/fdrake/bin:/home/fdrake/.local/bin:/home/fdrake/.nvm/versions/node/v6.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin /home/fdrake/p/ks3.mediaflux$ /home/fdrake/p/ks3.mediaflux/.tox/py36/bin/pip install --trusted-host devpi -i http://...@devpi/keeper/prod/+simple/ kt.testing Looking in indexes: http://...@devpi/keeper/prod/+simple/ Collecting kt.testing Collecting requests (from kt.testing) Downloading http://devpi/root/pypi/+f/6a1/b267aa90cac58/requests-2.18.4-py2.py3-none-any... (88kB) 100% |████████████████████████████████| 92kB 25.2MB/s Collecting six (from kt.testing) Downloading http://devpi/root/pypi/+f/832/dc0e10feb1aa2/six-1.11.0-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/9fa/520c1bacfb634/certifi-2018.4.16-py2.py3-none-a... (150kB) 100% |████████████████████████████████| 153kB 35.3MB/s Collecting urllib3<1.23,>=1.21.1 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/063/30f386d6e4b19/urllib3-1.22-py2.py3-none-any.wh... (132kB) 100% |████████████████████████████████| 133kB 35.5MB/s Collecting chardet<3.1.0,>=3.0.2 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/fc3/23ffcaeaed0e0/chardet-3.0.4-py2.py3-none-any.w... (133kB) 100% |████████████████████████████████| 143kB 34.4MB/s Collecting idna<2.7,>=2.5 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/8c7/309c718f94b3a/idna-2.6-py2.py3-none-any.whl (56kB) 100% |████████████████████████████████| 61kB 28.4MB/s Installing collected packages: certifi, urllib3, chardet, idna, requests, six, kt.testing Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/bin/chardetect' Consider using the `--user` option or check the permissions. ERROR: could not install deps [kt.testing]; v = InvocationError("'/home/fdrake/p/ks3.mediaflux/.tox/py36/bin/pip install --trusted-host devpi -i http://...@devpi/keeper/prod/+simple/ kt.testing'", 1) py36 finish: getenv after 3.04 seconds _____________________________________________________ summary ______________________________________________________ ERROR: py36: could not install deps [kt.testing]; v = InvocationError("'/home/fdrake/p/ks3.mediaflux/.tox/py36/bin/pip install --trusted-host devpi -i http://...@devpi/keeper/prod/+simple/ kt.testing'", 1) ------------------------ Everything looks good right up until the reference to /usr/bin/. Some things are installed into the environment: ------------------------ $ ls -lF .tox/py36/lib/python3.6/site-packages/ total 44 drwxrwxr-x 2 fdrake fdrake 4096 May 9 11:53 __pycache__/ drwxrwxr-x 4 fdrake fdrake 4096 May 9 11:53 chardet/ drwxrwxr-x 2 fdrake fdrake 4096 May 9 11:53 chardet-3.0.4.dist-info/ -rw-rw-r-- 1 fdrake fdrake 126 May 9 11:53 easy_install.py drwxrwxr-x 5 fdrake fdrake 4096 May 9 11:53 pip/ drwxrwxr-x 2 fdrake fdrake 4096 May 9 11:53 pip-10.0.1.dist-info/ drwxrwxr-x 5 fdrake fdrake 4096 May 9 11:53 pkg_resources/ drwxrwxr-x 6 fdrake fdrake 4096 May 9 11:53 setuptools/ drwxrwxr-x 2 fdrake fdrake 4096 May 9 11:53 setuptools-39.1.0.dist-info/ drwxrwxr-x 5 fdrake fdrake 4096 May 9 11:53 wheel/ drwxrwxr-x 2 fdrake fdrake 4096 May 9 11:53 wheel-0.31.0.dist-info/ ------------------------ If I run the pip install command directly, it doesn't try referring to /usr/bin/, and reports success: ------------------------ $ PATH=/home/fdrake/p/ks3.mediaflux/.tox/py36/bin:/home/fdrake/bin:/home/fdrake/.local/bin:/home/fdrake/.nvm/versions/node/v6.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin /home/fdrake/p/ks3.mediaflux/.tox/py36/bin/pip install --trusted-host devpi -i http://...@devpi/keeper/prod/+simple/ kt.testing Looking in indexes: http://keeper:Keeper1SS@devpi/keeper/prod/+simple/ Collecting kt.testing Collecting requests (from kt.testing) Downloading http://devpi/root/pypi/+f/6a1/b267aa90cac58/requests-2.18.4-py2.py3-none-any... (88kB) 100% |████████████████████████████████| 92kB 41.1MB/s Collecting six (from kt.testing) Downloading http://devpi/root/pypi/+f/832/dc0e10feb1aa2/six-1.11.0-py2.py3-none-any.whl Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.tox/py36/lib/python3.6/site-packages (from requests->kt.testing) (3.0.4) Collecting idna<2.7,>=2.5 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/8c7/309c718f94b3a/idna-2.6-py2.py3-none-any.whl (56kB) 100% |████████████████████████████████| 61kB 42.8MB/s Collecting certifi>=2017.4.17 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/9fa/520c1bacfb634/certifi-2018.4.16-py2.py3-none-a... (150kB) 100% |████████████████████████████████| 153kB 36.3MB/s Collecting urllib3<1.23,>=1.21.1 (from requests->kt.testing) Downloading http://devpi/root/pypi/+f/063/30f386d6e4b19/urllib3-1.22-py2.py3-none-any.wh... (132kB) 100% |████████████████████████████████| 133kB 41.7MB/s Installing collected packages: idna, certifi, urllib3, requests, six, kt.testing Successfully installed certifi-2018.4.16 idna-2.6 kt.testing-3.0.0 requests-2.18.4 six-1.11.0 urllib3-1.22 ------------------------ Note that it doesn't attempt to re-install chardet, presumably since the installation appears successful as far the Python package goes (only the cli tool didn't install). The chardetect script is packaged as a console_scripts entry point, not an old-school distutils script. If I remove my application dependencies from .tox/py36/lib/python3.6/site-packages/, leaving only easy_install, pip, pkg_resources, setuptools, and wheel, the manually-run pip install command produces the same error as in the initial, tox-driven run. The py36 environment includes pip 10.0.1, so not particularly out of date. I'd really appreciate any ideas on how to track this down. Even a work-around that would cause pip not to generate scripts for the console_scripts entry points would be very helpful (since I don't want those anyway!). -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
On Wed, May 9, 2018 at 12:15 PM, Fred Drake <fred@fdrake.net> wrote:
I'd really appreciate any ideas on how to track this down. Even a work-around that would cause pip not to generate scripts for the console_scripts entry points would be very helpful (since I don't want those anyway!).
The more I think about this, it feels like a pip issue. Looking through the pip docs again, I found --install-option, and was able to adjust tox.ini to toss aside the junk scripts: ---------------- [testenv] install_command = pip install --trusted-host devpi --install-option=--install-scripts=/tmp/bin-junk {opts} {packages} ---------------- tells pip to install scripts in a junk directory where they can be ignored. Not as good as just not generating the scripts, but tox was able to continue with running my tests (the whole point). Unfortunately, this causes pip to avoid wheels, which isn't going to fly for long (some of my extensions require build environments that won't be available to most builds). -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
participants (3)
-
Fred Drake -
Ian Stapleton Cordasco -
Oliver Bestwalter