It is very difficult to guarantee that your library will never work with any newer version of some other library, but if you can (e.g. python-dateutil, versions >= 2 being exclusively Python 3 compatible), a < or <= qualifier may be appropriate. != is a good one if you know an exact version of some other library is incompatible. == just causes problems because it gets in the way of the integrator who has a lot more information than the library-author-from-some-time-ago. On Mon, May 13, 2013 at 9:57 AM, Ian Cordasco wrote:

On Mon, May 13, 2013 at 7:53 AM, Jim Fulton wrote:

...

Gaaa. Libraries shouldn't pin versions (although min and max versions with reasonable ranges is fine).

As a library author, I'm intrigued. Why shouldn't I or others pin versions? Let's provide a situation where it may be necessary (in my opinion):

If I release a library dependent upon a particular API in one version of a dependency and before I release my next version I notice plans to break the existing API, why shouldn't I pin the version to protect users (or at least provide a maximum version) from getting horrible exceptions? I have no guarantee I'll be able to update my library based on the new API quickly enough to get a version out concurrent with the new API on that dependency. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

On 13-05-13 15:57, Ian Cordasco wrote:

If I release a library dependent upon a particular API in one version of a dependency and before I release my next version I notice plans to break the existing API, why shouldn't I pin the version to protect users (or at least provide a maximum version) from getting horrible exceptions?

Best example: if you pin "somelibrary=1.2" in your library, none of your users can use the critical 1.2.1 bugfix release. Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"

On Mon, May 13, 2013 at 10:16 AM, Ian Cordasco wrote:

On Mon, May 13, 2013 at 10:12 AM, Reinout van Rees wrote:

...

On 13-05-13 15:57, Ian Cordasco wrote:

...

If I release a library dependent upon a particular API in one version of a dependency and before I release my next version I notice plans to break the existing API, why shouldn't I pin the version to protect users (or at least provide a maximum version) from getting horrible exceptions?

Best example: if you pin "somelibrary=1.2" in your library, none of your users can use the critical 1.2.1 bugfix release.

Thanks to you and Daniel. (I accidentally replied to Daniel off-list.) I had just been advised by someone formerly a part of the distutils (or distribute, I forget which) team that you either pin or don't. I try not to, but there have been occasions where I found it necessary. I'll certainly move forward a better developer for your explanations.

<soapbox>Unfortunately, when people discuss solutions, they often forget to state the problem they're thinking of.</soapbox> As Daniel pointed out, when deploying an *application*, you should generally pin all of your dependencies, so you can reproduce the deployment. This can be done via the apps setup.cfg, pip requirements or buildout versions. When releasing libraries, you should restrict versions as little as possible. The more you restrict dependency versions, the harder your library is to use. A common recommendation is to set a minimum version to the lowest version of the dependency known to work and to set the maximum version to less then the next major release:

=2, <3dev

In practice, most people just set a lower bound, if any. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton

On Tue, May 14, 2013 at 10:36 PM, Daniel Holth wrote:

Why don't we simply provide an option to ignore == if it is not a "root" dependency?

I don't understand what you mean. Jim

On Mon, May 13, 2013 at 2:02 PM, Jim Fulton wrote:

...

On Mon, May 13, 2013 at 10:16 AM, Ian Cordasco wrote:

...

On Mon, May 13, 2013 at 10:12 AM, Reinout van Rees wrote:

...

On 13-05-13 15:57, Ian Cordasco wrote:

...

If I release a library dependent upon a particular API in one version of a dependency and before I release my next version I notice plans to break the existing API, why shouldn't I pin the version to protect users (or at least provide a maximum version) from getting horrible exceptions?

Best example: if you pin "somelibrary=1.2" in your library, none of your users can use the critical 1.2.1 bugfix release.

Thanks to you and Daniel. (I accidentally replied to Daniel off-list.) I had just been advised by someone formerly a part of the distutils (or distribute, I forget which) team that you either pin or don't. I try not to, but there have been occasions where I found it necessary. I'll certainly move forward a better developer for your explanations.

<soapbox>Unfortunately, when people discuss solutions, they often forget to state the problem they're thinking of.</soapbox>

As Daniel pointed out, when deploying an *application*, you should generally pin all of your dependencies, so you can reproduce the deployment. This can be done via the apps setup.cfg, pip requirements or buildout versions.

When releasing libraries, you should restrict versions as little as possible. The more you restrict dependency versions, the harder your library is to use. A common recommendation is to set a minimum version to the lowest version of the dependency known to work and to set the maximum version to less then the next major release:

...

=2, <3dev

In practice, most people just set a lower bound, if any.

Jim

-- Jim Fulton http://www.linkedin.com/in/jimfulton _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

-- Jim Fulton http://www.linkedin.com/in/jimfulton

Pip respects == but requirements.txt can override. On May 16, 2013, at 9:51 AM, Daniel Holth wrote:

If you were to say:

install gerbil==3 wheel==0.16

and gerbil version 3's requirements were:

water_bottle == 4 shavings < 7 wheel >= 0.16 # of course

and shavings's requirements were:

cedar == 0.9

The root of the dependency graph is "gerbil==3 wheel==0.16.0". These are the only == constraints that will be honored.

The proposed option would keep "gerbil==3 wheel==0.16.0", convert water_bottle==4 and cedar==0.9 to just water_bottle and cedar, and respect the >= and < constraints.

On Thu, May 16, 2013 at 9:31 AM, Jim Fulton wrote:

...

On Tue, May 14, 2013 at 10:36 PM, Daniel Holth wrote:

...

Why don't we simply provide an option to ignore == if it is not a "root" dependency?

I don't understand what you mean.

Jim

...

On Mon, May 13, 2013 at 2:02 PM, Jim Fulton wrote:

...

On Mon, May 13, 2013 at 10:16 AM, Ian Cordasco wrote:

...

On Mon, May 13, 2013 at 10:12 AM, Reinout van Rees wrote:

...

On 13-05-13 15:57, Ian Cordasco wrote: > > If I release a library dependent upon a particular API in one version > of a dependency and before I release my next version I notice plans to > break the existing API, why shouldn't I pin the version to protect > users (or at least provide a maximum version) from getting horrible > exceptions?

Best example: if you pin "somelibrary=1.2" in your library, none of your users can use the critical 1.2.1 bugfix release.

Thanks to you and Daniel. (I accidentally replied to Daniel off-list.) I had just been advised by someone formerly a part of the distutils (or distribute, I forget which) team that you either pin or don't. I try not to, but there have been occasions where I found it necessary. I'll certainly move forward a better developer for your explanations.

<soapbox>Unfortunately, when people discuss solutions, they often forget to state the problem they're thinking of.</soapbox>

As Daniel pointed out, when deploying an *application*, you should generally pin all of your dependencies, so you can reproduce the deployment. This can be done via the apps setup.cfg, pip requirements or buildout versions.

When releasing libraries, you should restrict versions as little as possible. The more you restrict dependency versions, the harder your library is to use. A common recommendation is to set a minimum version to the lowest version of the dependency known to work and to set the maximum version to less then the next major release:

...

=2, <3dev

In practice, most people just set a lower bound, if any.

Jim

-- Jim Fulton http://www.linkedin.com/in/jimfulton _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

-- Jim Fulton http://www.linkedin.com/in/jimfulton

---

Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

On Thu, May 16, 2013 at 10:08 AM, Jim Fulton wrote:

On Thu, May 16, 2013 at 9:51 AM, Daniel Holth wrote:

...

If you were to say:

install gerbil==3 wheel==0.16

and gerbil version 3's requirements were:

water_bottle == 4 shavings < 7 wheel >= 0.16 # of course

and shavings's requirements were:

cedar == 0.9

The root of the dependency graph is "gerbil==3 wheel==0.16.0". These are the only == constraints that will be honored.

The proposed option would keep "gerbil==3 wheel==0.16.0", convert water_bottle==4 and cedar==0.9 to just water_bottle and cedar, and respect the >= and < constraints.

Ignoring requirements, especially in the absence of conflict, as in the case above, seems like a bad idea to me. I could see high-level == requirements overriding lower-level requirements, possibly with a warning.

(I'm not sure what the scope of this discussion is; whether it's just pip, or whether the original question was meant to be general.)

It is a general idea which would of course have to be implemented in pip or whatever. Generally, what do do when your dependencies declare incorrect dependencies of any kind; specifically the idea of making it easy to ignore == because it is almost always wrong.

I don't think it should be up to the tool to decide that a dependency is wrong.

IMO, the tool should satisfy the declared dependencies as well as possible, report conflicts, and give the user a way to decide a conflict.

In this case we don't expect a conflict, but are unfortunate enough to be using a dependency that pins some other project. The tool would ideally warn us that there was a newer version of some pinned dependency on an interior node of our dependency graph and let us do something about it.

On Mon, May 13, 2013 at 6:08 AM, Marcus Smith wrote:

If I have a shared dependency between 2 packages but they have pinned

...

different versions of a 3rd package, is there a way to override this?

Most of the time when people pin a version they are doing it because they have tested up to that version, but it doesn't always mean they don't work with a later version.

For example:

P1: sqlalchemy==0.7.6

P2: sqlalchemy

P3: sqlalchemy==0.8

If I want to to override this for all packages and tell them its fine to just use 0.8.1 and ignore whatever they were pinned at, can I?

pip requirement files do this. items in the requirement file override found dependency versions. http://www.pip-installer.org/en/latest/cookbook.html#requirements-files

Marcus

We use easy_install/setup.py instead of pip requirements files because there is no way to configure pip to look at a different index via an application configuration file (or at least we couldn't find one). With easy_install we are able to do: [easy_install] index_url = <corp index> and everything works. With pip it seems to only support a system wide/user wide configuration but not for a specific application/repo. Thanks for the info though!

see here in the docs http://www.pip-installer.org/en/latest/logic.html#requirements-file-format On Mon, May 13, 2013 at 9:57 AM, Daniel Holth wrote:

You can include many of pip's command line arguments in its requirements files; try putting --index-url right there, seems to work on the tip:

--index-url=someindex ... other requirements

On Mon, May 13, 2013 at 12:48 PM, John Anderson wrote:

...

On Mon, May 13, 2013 at 6:08 AM, Marcus Smith wrote:

...

...

If I have a shared dependency between 2 packages but they have pinned different versions of a 3rd package, is there a way to override this?

Most of the time when people pin a version they are doing it because

...

...

...

have tested up to that version, but it doesn't always mean they don't work with a later version.

For example:

P1: sqlalchemy==0.7.6

P2: sqlalchemy

P3: sqlalchemy==0.8

If I want to to override this for all packages and tell them its fine to just use 0.8.1 and ignore whatever they were pinned at, can I?

pip requirement files do this. items in the requirement file override found dependency versions. http://www.pip-installer.org/en/latest/cookbook.html#requirements-files

Marcus

We use easy_install/setup.py instead of pip requirements files because

they there

...

is no way to configure pip to look at a different index via an application configuration file (or at least we couldn't find one).

With easy_install we are able to do:

[easy_install] index_url = <corp index>

and everything works. With pip it seems to only support a system wide/user wide configuration but not for a specific application/repo.

Thanks for the info though!

_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig