Hello,
I recently discovered tox, due to a testing issue that i had. It is working great locally on my machine. I am using it to set up different environment that contain different solvers for my optimization code. However the solvers are not part of the requirements for this project as the user might want to choose the solver for a specific purpose. The main solvers used are cbc, gplk, and cplex. At the moment I have set up the solvers on my system and invoke them using environment variables. However, this approach causes Tox to look for the solver outside of the created environment, which is not the behavior i intended or expected. I would like to be able to install the solver inside the environment as a system dependency, so that i can move my test setup to a github action or really any computer i like, without having to install a solver there.
The solvers are installed on my system with the following commands, and that is basically what i would like to move to the virtual environment..
apt-get install coinor-cbc
apt-get install glpk-utils libglpk-dev glpk-doc
and whatever 300 hoops you have to jump to install cplex.
Please find also attached my tox.ini file.
Could you advice what the best approach would be to achieve the installation with virtenv?
Thank you for your help
Meike Moeckel
Hello,
tox is a cross-platform tool aimed to work on all platforms the same. By default, virtualenv is able to handle Python dependencies via pip and virtualenv. The dependencies you've enumerated there live outside of the Python world. They are binary dependencies. The install commands you've enlisted there hint to me that you're using some Debian derivative. Note those will not work on some other distribution such as Fedora, macOS, or Windows. Furthermore, executing those commands might need a system administrator account/password and as such in general cannot be automated. Dependencies that are not install-able from the Python world into virtual environments are generally out of scope for tox to handle. Your project might still use them if you enlist them under whitelist_external_commands. You should otherwise encourage the user to install those dependencies though the best they can on their platform, with their own tool and authentication.
Thanks.
On Fri, Feb 19, 2021 at 4:59 AM Meike Moeckel meike.moeckel@anu.edu.au wrote:
Hello,
I recently discovered tox, due to a testing issue that i had. It is working great locally on my machine. I am using it to set up different environment that contain different solvers for my optimization code. However the solvers are not part of the requirements for this project as the user might want to choose the solver for a specific purpose. The main solvers used are cbc, gplk, and cplex. At the moment I have set up the solvers on my system and invoke them using environment variables. However, this approach causes Tox to look for the solver outside of the created environment, which is not the behavior i intended or expected. I would like to be able to install the solver inside the environment as a system dependency, so that i can move my test setup to a github action or really any computer i like, without having to install a solver there.
The solvers are installed on my system with the following commands, and that is basically what i would like to move to the virtual environment..
apt-get install coinor-cbc
apt-get install glpk-utils libglpk-dev glpk-doc
and whatever 300 hoops you have to jump to install cplex.
Please find also attached my tox.ini file.
Could you advice what the best approach would be to achieve the installation with virtenv?
Thank you for your help
Meike Moeckel _______________________________________________ tox-dev mailing list -- tox-dev@python.org To unsubscribe send an email to tox-dev-leave@python.org https://mail.python.org/mailman3/lists/tox-dev.python.org/ Member address: jokerjokerer@gmail.com
Yep, tox will never do system level changes, is outside its scope. From some point of view it can even be considered against it scope (isolation).
While it does not install the missing system dependencies itself, you may still be interested about https://github.com/tox-dev/tox-bindep plugin.
That one identifies missing dependencies and fails the run if some are missing, making execution less confusing for the user.
Some CI/CD systems are able to install bindep dependencies automatically, like Zuul CI. You could also write some ansible playbooks (or shell scripts) and run them if needed, that being another common approach.
Keep in mind that installing missing system dependencies is the kind of feature that keeps giving... bug reports. You can never have full coverage of all possible environments on which someone will try to run your code. Just be warned, saying this because I got hit by it myself.
Cheers Sorin
Thank you Sorin and Bernat, for the clarification. I will see if i can find a workaround, using the git actions setup for the installation or even docker..
Yes, i am using Ubuntu, some of my colleagues are on mac os using some quite old systems.. Most solvers are available on Mac as well, that is why i would like to use tox to run the testso
Since working with optimisation, i am quite aware that testing all options isn't possible.. But we do the best we can and I think Tox makes it very easy.
Thanks again for your quick help and for providing tox to the community!
Cheers Meike
________________________________ From: Sorin Sbarnea ssbarnea@redhat.com Sent: Friday, 19 February 2021 8:46 PM To: Bernat Gabor jokerjokerer@gmail.com Cc: tox-dev@python.org tox-dev@python.org; Meike Moeckel Meike.Moeckel@anu.edu.au Subject: Re: [tox-dev] Re: install solvers via tox
Yep, tox will never do system level changes, is outside its scope. From some point of view it can even be considered against it scope (isolation).
While it does not install the missing system dependencies itself, you may still be interested about https://github.com/tox-dev/tox-bindep plugin.
That one identifies missing dependencies and fails the run if some are missing, making execution less confusing for the user.
Some CI/CD systems are able to install bindep dependencies automatically, like Zuul CI. You could also write some ansible playbooks (or shell scripts) and run them if needed, that being another common approach.
Keep in mind that installing missing system dependencies is the kind of feature that keeps giving... bug reports. You can never have full coverage of all possible environments on which someone will try to run your code. Just be warned, saying this because I got hit by it myself.
Cheers Sorin
participants (4)
-
Bernat Gabor -
Meike Moeckel -
Meike Moeckel
-
Sorin Sbarnea