On Wed, 20 May 2009 10:17:31 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

Jean-Paul Calderone <exarkun@divmod.com> writes:

...

For a long time, the idiom I've used to specify version information in my distutils-based packages has been this simple one: put version information into the Python package somewhere, structure the source layout so that the package can be imported directly from an unpacked source tarball (or vcs wc), and import the version into setup.py. This worked great. And I only had to define my version once.

Increasingly, it seems that new distutils-related tools don't support this idiom and break in various ways. (eg

http://divmod.org/trac/ticket/2629 http://divmod.org/trac/ticket/2831 )

Those breakages seem to be caused in part because the module containing version information is doing too much.

The only thing the version module does is define the version. For example, $ cat _version.py # This is an auto-generated file. Use Epsilon/bin/release-divmod to update. from twisted.python import versions version = versions.Version(__name__[:__name__.rfind('.')], 0, 9, 33) $ Of course, the "too much" here is the use of a structured version object. Doing anything simpler would be much like the approach Phillip's pointed out, to have a simple text file and parse its contents without getting Python involved at all.

...

What is the recommendation for specifying version information in a way which is compatible with all these new tools?

I would recommend having a module in your package whose *only* responsibility is version information for the package, so that importing it won't break in different environments.

That module can the be imported both by the package itself when it needs to know its version string, and by the ‘setup.py’ to get that same version string for package building.

This has issues. For examples, sometimes a system-wide installation of the package will get used instead (generally not when running setup.py directly, but it seems to happen when other tools are used). Otherwise, the existing _version.py from above works fine. Jean-Paul

No, I would say “too much” here is importing a module not guaranteed to be installed, for creating something as simple as a version. I'd prefer the above to be::

$ cat _version.py # This is an auto-generated file. Use magicbean to update. version = "0.9.33" $

Or my preferred way: $cat foobar/__init__.py __version__ = "0.9.33" and import foobar should not trigger code execution anyway IMHO so $ python -c 'import foobar; print foobar.__version__' 0.9.33

I still like the approach of having it in a module that can be imported to automatically have the name bound to an appropriate value.

Yes, agree 100% the KISS approach is always the best. Regards, Antonio

Jeff Rush wrote:

A. Cavallo wrote:

...

...

No, I would say “too much” here is importing a module not guaranteed to be installed, for creating something as simple as a version. I'd prefer the above to be::

$ cat _version.py # This is an auto-generated file. Use magicbean to update. version = "0.9.33" $

Or my preferred way: $cat foobar/__init__.py __version__ = "0.9.33"

and import foobar should not trigger code execution anyway IMHO so $ python -c 'import foobar; print foobar.__version__' 0.9.33

That doesn't work in all cases. Your example is of an external query of the version of an installed module. You also need to query the version -before- it is installed (during the packaging phase) on sys.path, and also from -within- the foobar module itself. Your code does not handle those cases. An attempt to 'import foobar; print foobar.__version__' from a setup.py inside foobar won't find foobar.

That's my experience as well. I am a bit delighted to see that other people struggled on this, I felt very stupid the first time I tried to setup this consistently without success. My current approach is to set up the version in setup.py, but to always generated a trivial version.py within the package (for the installed version). The setup.py version is used by paver, the installed version is used by the sphinx build inside a bootstrapped environment. I think the situation where you want to build documentation at the same time as the software happens relatively often, I would be glad to hear of a better alternative. cheers, David

On Wed, 20 May 2009 12:37:19 +0100, "A. Cavallo" <a.cavallo@cavallinux.eu> wrote:

...

...

Or my preferred way: $cat foobar/__init__.py __version__ = "0.9.33"

and import foobar should not trigger code execution anyway IMHO so $ python -c 'import foobar; print foobar.__version__' 0.9.33

That doesn't work in all cases.

No it is not that the case and I'm going to bore you probably (sorry).

[snip - explanation of how you can import from the package you're about to install]

This is true, but it is not the complete story. You explained the case where setup.py is being run directly to install the package. My first post in the thread explained that this case works fine. It's other cases, such as easy_install and buildout which do not work with this strategy. (Someone who has experience with these tools might be able to explain why this is; I haven't investigated the details yet, I only know that people are reporting bugs against my packages.)

[snip]

Then we need to agree on what do we mean for install and packaging.... "python setup.py install", "python setup.py bdist_rpm/wininst" or during a complete deb/rpm package build? In the latter the version the "version" is the one available in the spec/deb files and it cannot be reflected from the sources anyway, no matter how hard one try to do it.

The only version I'm talking about here is the one I assign to my package. If someone else is assigning a different version, then it becomes their problem to make things work.

[snip]

Finding modules can always be forced using PYTHONPATH (like in case of foobar-1.0/src/foobar layout) or from setup.py inserting into sys.path the subdir: that's the easiest and in my opinion the best way to do it during build/install stages.

Maybe so, but it can be made compatible with all (or most, or any?) of the tools people are using to install distutils-based packages these days? Jean-Paul

Wheat <wheatix@gmail.com> writes:

The version information for a Python distribution should be specified in setup.py (or setup.py can read this information from somewhere in the project's files).

Yes, agreed completely.

It should *not* be specified inside a Python package or module that is part of the distribution itself.

Why not? It certainly makes sense for the programs to be able to discover their own version. I hope you don't advocate duplicating the version in another place for this purpose?

One area where embedding the version inside the module is problematic is a distribution which provides multiple modules or pacakges. Remember, one package (or module) does not have to equal one distribution! A distribution can provide many modules and packages.

In this case, I'm of the opinion that each Python package should know its own version (via a simple module that declares or discovers it, with no dependency on anything but standard Python), and the package distribution tools should choose one of those Python packages to be the one which determines the distribution version.

Let's say you have the Python project 'foobar'. A 'foobar' distribution provides the 'foo' package and the 'bar' package. If you are putting the distribution version in the package, should it be in foo.__version__ or bar.__version__?

I'd say the distribution version should be identical to one of the packages, as chosen by whoever's maintaining the distribution configuration.

Also it's worth mentioning that PEP 376 (http://www.python.org/dev/ peps/pep-0376/) proposes to solve some of this problem, by providing an API for reading the version of an installed distribution - and does so without needing to import any part of that installed distribution. For the 'foobar' distribution this would be:

import pkgutil foobar_metadata = pkgutil.get_metadata('foobar') print foobar_metadata.version

Sure. This fits my constraint that the version should be discoverable by importing a Python module without any dependencies on anything except standard Python. It just means that “standard Python” would need to include that ‘pkgutil’ module. -- \ “I don't care to belong to a club that accepts people like me | `\ as members.” —Groucho Marx | _o__) | Ben Finney