Requirements.txt inside virtual environment
Consider the following scenario: 1) You have to format your pc. 2) You copy your python projects to your hard disk along with the virtual environment(I keep the virtual environment in the project folder only). 3)When you copy your projects back after successfully formatting your pc, the useless virtualenv lies there and you can't get the requirements.txt now and have to install all packages one by one. It could be great if pip or python or anyone would by default store a requirements.txt file inside the virtual env and on `pip install xyz` would append `xyz` to that file.
On Fri, 18 Feb 2022 at 13:12, Vishesh Mangla <manglavishesh64@gmail.com> wrote:
Consider the following scenario: 1) You have to format your pc. 2) You copy your python projects to your hard disk along with the virtual environment(I keep the virtual environment in the project folder only). 3)When you copy your projects back after successfully formatting your pc, the useless virtualenv lies there and you can't get the requirements.txt now and have to install all packages one by one.
It could be great if pip or python or anyone would by default store a requirements.txt file inside the virtual env and on `pip install xyz` would append `xyz` to that file.
OR! You maintain your own requirements.txt manually, and always use 'pip install -r requirements.txt' when you change it. That works much better with source control. ChrisA
`pip freeze` can be useful for this kind of thing On Thu, 17 Feb 2022 at 18:19, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, 18 Feb 2022 at 13:12, Vishesh Mangla <manglavishesh64@gmail.com> wrote:
Consider the following scenario: 1) You have to format your pc. 2) You copy your python projects to your hard disk along with the
3)When you copy your projects back after successfully formatting your
virtual environment(I keep the virtual environment in the project folder only). pc, the useless virtualenv lies there and you can't get the requirements.txt now and have to install all packages one by one.
It could be great if pip or python or anyone would by default store a
requirements.txt file inside the virtual env and on `pip install xyz` would append `xyz` to that file.
OR! You maintain your own requirements.txt manually, and always use 'pip install -r requirements.txt' when you change it. That works much better with source control.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YNU3D2... Code of Conduct: http://python.org/psf/codeofconduct/
I realize it's still very new to a lot of people, but doesn't pyproject.toml solve this problem? Just include a dependencies entry in there and use a tool like flit or poetry. On Fri, Feb 18, 2022, 8:24 AM Vishesh Mangla <manglavishesh64@gmail.com> wrote:
Yes, I too think they meant the same. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2YNSI5... Code of Conduct: http://python.org/psf/codeofconduct/
Ok, then I mean that if this process could be added to the standard. Package install means that individually users will have to know this thing and they may or may not be able to find this package. I don't know what words to google for to find a package that serves the objective.
On Fri, Feb 18, 2022 at 8:46 AM Vishesh Mangla <manglavishesh64@gmail.com> wrote:
Ok, then I mean that if this process could be added to the standard. Package install means that individually users will have to know this thing and they may or may not be able to find this package. I don't know what words to google for to find a package that serves the objective.
pyproject.toml is a great way to do it. it is directly supported by pip. check out PEP 631 <https://www.python.org/dev/peps/pep-0631/>. the users don't need to know about anything else, they just pip install the package or app like normal. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
We create the __init__.py too. Maybe python can use that file to prepare a default setup.py if it would be difficult to automate it using pip. I am just asking if it is easier to automate the task.
On Fri, Feb 18, 2022 at 01:18:55PM +1100, Chris Angelico wrote:
OR! You maintain your own requirements.txt manually, and always use 'pip install -r requirements.txt' when you change it.
That only works the first two times you use pip to install something. On the third time, you write a shell script to automate the process, and by the time you have finished debugging the script, the reason for creating the venv in the first place is no longer relevent. *wink* Isn't the issue here that pip is not a full package manager, but just an installer? -- Steve
On Sat, 19 Feb 2022 at 01:30, Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Feb 18, 2022 at 01:18:55PM +1100, Chris Angelico wrote:
OR! You maintain your own requirements.txt manually, and always use 'pip install -r requirements.txt' when you change it.
That only works the first two times you use pip to install something. On the third time, you write a shell script to automate the process, and by the time you have finished debugging the script, the reason for creating the venv in the first place is no longer relevent.
*wink*
Not sure. I can type 'pip install -r re<tab>" fairly quickly, so there's nothing to script. Not every one-line operation needs to be a shell script.
Isn't the issue here that pip is not a full package manager, but just an installer?
Hmm, what do you expect it to be? It installs, it removes, it resolves dependencies, it upgrades. ChrisA
On Sat, Feb 19, 2022 at 01:35:25AM +1100, Chris Angelico <rosuav@gmail.com> wrote:
On Sat, 19 Feb 2022 at 01:30, Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Feb 18, 2022 at 01:18:55PM +1100, Chris Angelico wrote:
OR! You maintain your own requirements.txt manually, and always use 'pip install -r requirements.txt' when you change it.
That only works the first two times you use pip to install something. On the third time, you write a shell script to automate the process, and by the time you have finished debugging the script, the reason for creating the venv in the first place is no longer relevent.
*wink*
Not sure. I can type 'pip install -r re<tab>" fairly quickly, so there's nothing to script. Not every one-line operation needs to be a shell script.
pip i<tab> -r re<tab> # ;-)
Isn't the issue here that pip is not a full package manager, but just an installer?
Hmm, what do you expect it to be? It installs, it removes, it resolves dependencies, it upgrades.
It doesn't replace. I cannot declare package "X" to be replacement for package "y". I cannot declare "y" is outdated by "X". It doesn't have post-install and pre-uninstall scripts. Post-install can be implemented for sdists but not for wheels.
ChrisA
Oleg. -- Oleg Broytman https://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Sat, Feb 19, 2022 at 01:35:25AM +1100, Chris Angelico wrote:
Isn't the issue here that pip is not a full package manager, but just an installer?
Hmm, what do you expect it to be? It installs, it removes, it resolves dependencies, it upgrades.
I don't have any expectations either way. My requirements for pip are so low I barely even know what a requirements file is :-) But pip stands for "pip installs packages", not "pip manages packages". For example: - there are no rollbacks and it doesn't handle revisions; - pip uninstall is completely unaware of dependencies; https://github.com/pypa/pip/issues/3867 When pip resolves dependencies, it does it in the simplest possible way that *usually* works. It has no satisfiability solver. "No effort is made to ensure that the dependencies of all packages are fulfilled simultaneously. This can lead to environments that are broken in subtle ways, if packages installed earlier in the order have incompatible dependency versions relative to packages installed later in the order." https://www.anaconda.com/blog/understanding-conda-and-pip SageMath: "Pip is NOT a package manager. Pip is just a Python module installer, it does very little to help install non-Python dependencies, and is not very smart about version handling." https://wiki.sagemath.org/days77/packaging#Pip.2FPyPI OpenSuse: "The way pip and wheels interact with each other seems to demonstrate that pip is not a package manager but more a python module manager." https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/message/... -- Steve
I’m a big conda fan, but most of those quotes are simply saying the obvious — pip is not for anything other than Python packages — which is well known. I would argue that pip is a *Python* package manager— maybe not as full featured as it could be, but very useful none the less. -CHB On Fri, Feb 18, 2022 at 3:53 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Sat, Feb 19, 2022 at 01:35:25AM +1100, Chris Angelico wrote:
Isn't the issue here that pip is not a full package manager, but just an installer?
Hmm, what do you expect it to be? It installs, it removes, it resolves dependencies, it upgrades.
I don't have any expectations either way. My requirements for pip are so low I barely even know what a requirements file is :-)
But pip stands for "pip installs packages", not "pip manages packages". For example:
- there are no rollbacks and it doesn't handle revisions;
- pip uninstall is completely unaware of dependencies;
https://github.com/pypa/pip/issues/3867
When pip resolves dependencies, it does it in the simplest possible way that *usually* works. It has no satisfiability solver.
"No effort is made to ensure that the dependencies of all packages are fulfilled simultaneously. This can lead to environments that are broken in subtle ways, if packages installed earlier in the order have incompatible dependency versions relative to packages installed later in the order."
https://www.anaconda.com/blog/understanding-conda-and-pip
SageMath: "Pip is NOT a package manager. Pip is just a Python module installer, it does very little to help install non-Python dependencies, and is not very smart about version handling."
https://wiki.sagemath.org/days77/packaging#Pip.2FPyPI
OpenSuse: "The way pip and wheels interact with each other seems to demonstrate that pip is not a package manager but more a python module manager."
https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/message/...
-- Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OUNIZG... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Fri, 18 Feb 2022 at 23:53, Steven D'Aprano <steve@pearwood.info> wrote:
When pip resolves dependencies, it does it in the simplest possible way that *usually* works. It has no satisfiability solver.
"No effort is made to ensure that the dependencies of all packages are fulfilled simultaneously. This can lead to environments that are broken in subtle ways, if packages installed earlier in the order have incompatible dependency versions relative to packages installed later in the order."
This is no longer true - pip incorporated a proper dependency resolver in 2020. Paul
On Sat, Feb 19, 2022 at 10:17:58AM +0000, Paul Moore wrote:
On Fri, 18 Feb 2022 at 23:53, Steven D'Aprano <steve@pearwood.info> wrote:
When pip resolves dependencies, it does it in the simplest possible way that *usually* works. It has no satisfiability solver. [...]
This is no longer true - pip incorporated a proper dependency resolver in 2020.
Good to know! Thanks. -- Steve
Steven D'Aprano writes:
by the time you have finished debugging the script, the reason for creating the venv in the first place is no longer relevent.
Eh? Would you be willing to unpack that reference to 'venv' specifically, or was that just your patent-pending snark? I can't recall a venv being obviated by later developments, but I can recall a lot of cases where I later wished I had started with a venv on general principles. Steve
On Sat, Feb 19, 2022 at 04:14:57PM +0900, Stephen J. Turnbull wrote:
Steven D'Aprano writes:
by the time you have finished debugging the script, the reason for creating the venv in the first place is no longer relevent.
Eh? Would you be willing to unpack that reference to 'venv' specifically, or was that just your patent-pending snark? I can't recall a venv being obviated by later developments,
You have never created a venv and then later decided that it wasn't needed because the project was cancelled, the job fell through, somebody solved the problem in another way, your client changed the project specifications and you deleted the old venv and created a new one, you decided to work on something else instead, the company went bust, etc? Lucky you. For a mailing list community that loves to be hyper-skeptical about the usefulness of 90% of ideas proposed here, we sure are touchy about any suggestion that pip and venvs aren't the greatest tools ever made. Here's a hint for the future. When I follow a comment with a smiley emoticon or emoji, or *wink*, it means that my comment isn't intended to be taken too seriously. It's a bit of light-hearted banter. And there's at least a 50% chance that it is intended to be self-mocking, or at least affectionate teasing of the tropes and memes that are common in programming circles, such as the "Three Strikes Rule". http://wiki.c2.com/?ThreeStrikesAndYouAutomate especially the tendency to spend more time automating a task than will ever be saved by the automation. Even XKCD has joked about that. https://xkcd.com/1319/ -- Steve
Steven D'Aprano writes:
On Sat, Feb 19, 2022 at 04:14:57PM +0900, Stephen J. Turnbull wrote:
Steven D'Aprano writes:
by the time you have finished debugging the script, the reason for creating the venv in the first place is no longer relevent.
Eh? Would you be willing to unpack that reference to 'venv' specifically, or was that just your patent-pending snark? I can't recall a venv being obviated by later developments,
You have never created a venv and then later decided that it wasn't needed because the project was cancelled, the job fell through, somebody solved the problem in another way, your client changed the project specifications and you deleted the old venv and created a new one, you decided to work on something else instead, the company went bust, etc?
Oh, sure, that happens. If most of a code base goes away, of course so does the venv. That's actually a feature! But the *reason* for a venv in the first place is to ensure that neither the system nor the project pollutes the other with the "wrong" version of some dependency. I thought *that* was the "reason" you referred to as becoming irrelevant.
we sure are touchy about any suggestion that pip and venvs aren't the greatest tools ever made.
Eh, no, I really wanted to know if the need for isolation that venvs provide somehow goes away and under what circumstances. I have nothing invested in pip or venvs that would prevent me from adopting (or even helping to develop) something better (at least for future projects).
Here's a hint for the future. When I follow a comment with a smiley emoticon or emoji, or *wink*, it means that my comment isn't intended to be taken too seriously.
Well, yes, that's why I asked whether it was your patent-pending snark. Thank you for confirming that it was.
especially the tendency to spend more time automating a task than will ever be saved by the automation.
LOL. All of the automation using pip and venvs that I work with has more than paid back my investment (especially those that were developed by somebody else! :-) Yet-another-Steve-who-knows-how-when-and-why-to-use-smilies
participants (9)
-
Chris Angelico -
Christopher Barker -
Oleg Broytman -
Paul Moore -
Ricky Teachey -
Shantanu Jain -
Stephen J. Turnbull -
Steven D'Aprano -
Vishesh Mangla