Hi All,
Couldn't find an obviously better discussion forum than this for pipenv stuff, but please point me to where the rest of the discussions are happening...
So, with pipenv, what files do we version control for a project? both Pipfile and Pipfile.lock? Hopefully one I missed from the docs: with the correct files source controlled, how do I reproduce the environment on another machine?
Continuing on the deployment theme: I'm used to being able to put /path/to/some/ve/bin/some_script in crontabs and the like, where some_script comes form a setuptools entry point. What's the canonical way of doing this with pipenv?
Last up, how should pipenv be used for library development? (ie: where dependencies and minimal constraints are expressed in setup.py and where you want to test against as many python versions and library combinations as you support).
Other than sadness around capitalisation of the config file names, it looks really exciting :-)
cheers,
Chris
On Mon, 15 Jan 2018 at 00:33 Chris Withers chris@withers.org wrote:
Hi All,
Couldn't find an obviously better discussion forum than this for pipenv stuff, but please point me to where the rest of the discussions are happening...
Not here from what I can tell. :) Probably your best bet is to ask on the pipenv GitHub project and file an issue so the pipenv docs can give you guidance. I know for me personally it's getting a bit hazy between this list and pypa-dev where stuff should go (I'm personally starting to shift towards pypa-dev and considering this actually for distutils proper).
But I happen to know the answer to your pipenv questions, Chris, so I'll answer them here.
>
So, with pipenv, what files do we version control for a project? both Pipfile and Pipfile.lock?
Yes.
Hopefully one I missed from the docs: with the correct files source controlled, how do I reproduce the environment on another machine?
pipenv install --ignore-pipfile
>
Continuing on the deployment theme: I'm used to being able to put /path/to/some/ve/bin/some_script in crontabs and the like, where some_script comes form a setuptools entry point. What's the canonical way of doing this with pipenv?
Beats me. I think there's a command to get back to the venv that pipenv created on your behalf.
>
Last up, how should pipenv be used for library development? (ie: where dependencies and minimal constraints are expressed in setup.py and where you want to test against as many python versions and library combinations as you support).
That doesn't fall under pipenv's purview. Think of pipenv as trying to make venv + pip easier to work with. Since you don't use pip to express dependencies for your library then you shouldn't with pipenv either. Or put another way, think of your pipfile as just a different format for a requirements.txt file, not a replacement for flit or setuptools.
On 16 January 2018 at 05:46, Brett Cannon brett@python.org wrote:
On Mon, 15 Jan 2018 at 00:33 Chris Withers chris@withers.org wrote: >
Hi All,
Couldn't find an obviously better discussion forum than this for pipenv stuff, but please point me to where the rest of the discussions are happening...
Not here from what I can tell. :) Probably your best bet is to ask on the pipenv GitHub project and file an issue so the pipenv docs can give you guidance. I know for me personally it's getting a bit hazy between this list and pypa-dev where stuff should go (I'm personally starting to shift towards pypa-dev and considering this actually for distutils proper).
Personally, I don't think mailing lists in general are a great medium for asking usage and support questions.
Anyway, we genuinely don't have a clear answer to this question, so I've posted a meta-issue on the pipenv tracker to decide how we want to handle it: https://github.com/pypa/pipenv/issues/1305
Continuing on the deployment theme: I'm used to being able to put /path/to/some/ve/bin/some_script in crontabs and the like, where some_script comes form a setuptools entry point. What's the canonical way of doing this with pipenv?
Beats me. I think there's a command to get back to the venv that pipenv created on your behalf.
"pipenv --venv" gives the path for the current project, so what I tend to do is run "ls -s $(pipenv --venv) .venv" from the directory containing Pipfile and Pipfile.lock in order to create a deterministic path name.
Last up, how should pipenv be used for library development? (ie: where dependencies and minimal constraints are expressed in setup.py and where you want to test against as many python versions and library combinations as you support).
That doesn't fall under pipenv's purview. Think of pipenv as trying to make venv + pip easier to work with. Since you don't use pip to express dependencies for your library then you shouldn't with pipenv either. Or put another way, think of your pipfile as just a different format for a requirements.txt file, not a replacement for flit or setuptools.
pipenv does cover this case to some degree - the trick is that it involves thinking of your library's test suite as the application you want to deploy. (Hence the reference to "development and testing environments" at the end of the intro to https://packaging.python.org/tutorials/managing-dependencies/)
For that case, you don't put any of your dependencies from
setup.py into Pipfile, you just run "pipenv install -e src",
which
will give you a Pipfile entry like:
"<commit-hash>" = {editable = true, path = "src"}
I personally then edit that entry to replace the commit hash with the actual project name
All the testing and linting dependencies then go under [dev-packages].
The part you can't fully rely on yet is the lock file, since that aims to lock down the version of Python as well, not just the versions of the Python level dependencies. So if you're testing across multiple versions (e.g. with tox), the best current approach is to use "pipenv lock --requirements" to export just the library dependencies from Pipfile.lock. That's an area that could definitely use some UX improvement, but it isn't clear to me at this point what that would actually look like (short of adding a notion of named "install profiles" to the lock file format, which could then be aligned with tox test environments).
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 16 January 2018 at 14:24, Nick Coghlan ncoghlan@gmail.com wrote:
On 16 January 2018 at 05:46, Brett Cannon brett@python.org wrote:
On Mon, 15 Jan 2018 at 00:33 Chris Withers chris@withers.org wrote:
Continuing on the deployment theme: I'm used to being able to put /path/to/some/ve/bin/some_script in crontabs and the like, where some_script comes form a setuptools entry point. What's the canonical way of doing this with pipenv?
Beats me. I think there's a command to get back to the venv that pipenv created on your behalf.
"pipenv --venv" gives the path for the current project, so what I tend to do is run "ls -s $(pipenv --venv) .venv" from the directory containing Pipfile and Pipfile.lock in order to create a deterministic path name.
Oops, that should have been "ln -s $(pipenv --venv) .venv".
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 16/01/2018 04:24, Nick Coghlan wrote:
Anyway, we genuinely don't have a clear answer to this question, so I've posted a meta-issue on the pipenv tracker to decide how we want to handle it: https://github.com/pypa/pipenv/issues/1305
Thanks!
Continuing on the deployment theme: I'm used to being able to put /path/to/some/ve/bin/some_script in crontabs and the like, where some_script comes form a setuptools entry point. What's the canonical way of doing this with pipenv?
"pipenv --venv" gives the path for the current project, so what I tend to do is run "ln -s $(pipenv --venv) .venv" from the directory containing Pipfile and Pipfile.lock in order to create a deterministic path name.
Hmm, that feels hacky. It doesn't feel like there's consensus on this issue, and there's enough dupes coming in that it feels a shame that https://github.com/pypa/pipenv/issues/1049 has been closed.
I actually want both use cases in different scenarios. For dev, I typically want one venv per python version (how do I do that?) somewhere central. For deployment, I want a ./.venv or ./ve right next to the Pipfile.
Last up, how should pipenv be used for library development? (ie: where dependencies and minimal constraints are expressed in setup.py and where you want to test against as many python versions and library combinations as you support).
pipenv does cover this case to some degree - the trick is that it involves thinking of your library's test suite as the application you want to deploy. (Hence the reference to "development and testing environments" at the end of the intro to https://packaging.python.org/tutorials/managing-dependencies/)
As an aside: I find the doc split difficult. Even as a relatively experience python dev, knowing that I wanted to figure out what pipenv does and how to use it, I started at http://pipenv.readthedocs.io/en/latest/ which has no links to the tutorial(s).
For that case, you don't put any of your
dependencies from
setup.py into Pipfile, you just run "pipenv install -e src",
which
will give you a Pipfile entry like:
"<commit-hash>" = {editable = true, path = "src"}
I personally then edit that entry to replace the commit hash with the actual project name
I'm afraid I don't know enough about pipenv internals to understand the implications of the above, or why you might want to change the commit has to the project name, and indeed, what the implications of that might be.
All the testing and linting dependencies then go under [dev-packages].
Where can I find out more about [dev-packages]?
The part you can't fully rely on yet is the lock file, since that aims to lock down the version of Python as well, not just the versions of the Python level dependencies. So if you're testing across multiple versions (e.g. with tox),
...or a .travis.yml axis for CI, or just separate venvs for local dev, in my case...
the best current approach is to use "pipenv lock --requirements" to export just the library dependencies from Pipfile.lock.
...but what do I do with Pipfile.lock then? .gitigore?
cheers,
Chris
On 15/01/2018 19:46, Brett Cannon wrote:
But I happen to know the answer to your pipenv questions, Chris, so I'll answer them here.
Continuing here then :-)
So, with pipenv, what files do we version
control for a project? both
Pipfile and Pipfile.lock?
Yes.
great, thanks! How does Pipfile.lock work in the context of a project which may be installed on multiple operating systems with different final package requirements?
Hopefully one I missed from the docs: with
the correct files source
controlled, how do I reproduce the environment on another machine?
pipenv install --ignore-pipfile
That's a surprising spelling. Why does Pipfile need to be ignored? Surely Pipfile.lock will be consulted if present and used for the explicit requirements?
Last up, how should pipenv be used for
library development? (ie: where
dependencies and minimal constraints are expressed in setup.py and where
you want to test against as many python versions and library
combinations as you support).
That doesn't fall under pipenv's purview. Think of pipenv as trying to make venv + pip easier to work with. Since you don't use pip to express dependencies for your library then you shouldn't with pipenv either. Or put another way, think of your pipfile as just a different format for a requirements.txt file, not a replacement for flit or setuptools.
Well, kinda, pipenv is shaping up to be the "replacement" for pip+virtualenv, where the latter becomes just an implementation detail. Would be great if it had a good story for that use case :-)
cheers,
Chris
On 16 January 2018 at 08:08, Chris Withers chris@withers.org wrote:
That doesn't fall under pipenv's purview. Think of pipenv as trying to make venv + pip easier to work with. Since you don't use pip to express dependencies for your library then you shouldn't with pipenv either. Or put another way, think of your pipfile as just a different format for a requirements.txt file, not a replacement for flit or setuptools.
Well, kinda, pipenv is shaping up to be the "replacement" for pip+virtualenv, where the latter becomes just an implementation detail. Would be great if it had a good story for that use case :-)
I think the big issue is that it's not clear what use cases pipenv is trying to address. Your description of it being a "replacement" for pip+virtualenv is more or less what I originally assumed it was intended to be. But I tripped over this a while back, when I found it very difficult to use pipenv on a project where I needed to:
- Reference an unreleased version of a project from github
- Have one of its dependencies be satisfied from a locally (within the project directory) stored copy of a wheel, as there was no suitable wheel on PyPI.
I can't recall the details - they are in various issues on the pipenv tracker - but it was clear that I was trying to do something that wasn't considered "expected use" of pipenv, and I was getting very frustrated by the workarounds I needed to use to support my requirements. The guys on the pipenv tracker were very helpful, but when you're hitting design limitations, you can't expect quick fixes, so I ended up going back to pew + requrements.txt.
I think that if the pipenv docs had some better guidance on what use cases it was intended to cover (and what it wasn't, in relation to the broader range of pip+virtualenv use cases) that would help people better understand its place in the ecosystem.
Paul
On 16 January 2018 at 19:47, Paul Moore p.f.moore@gmail.com wrote:
I think that if the pipenv docs had some better guidance on what use cases it was intended to cover (and what it wasn't, in relation to the broader range of pip+virtualenv use cases) that would help people better understand its place in the ecosystem.
That's fair, and making the PyPUG tutorial specifically about application dependency management was an initial step in that direction - for the library development use case, you're generally going to step out of pipenv's world as soon as you try to run your tests across multiple versions.
The basic usage constraint is that pipenv expects you to control your
target Python version, and it expects you to have exactly one - to go
beyond that (as is needed for multi-version library support), you'll
still need to drop down to the lower level tooling, either skipping
the use of pipenv entirely, or else by using pipfile lock
--requirements to shift levels.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-
Brett Cannon -
Chris Withers -
Nick Coghlan -
Paul Moore