Re: [Distutils] automating __version__' ing issues
At 11:45 AM 5/13/2007 -0400, Alexander Michael wrote:
SVN does support keywords, but you need to turn-on keyword interpolation (see the SVN documentation describing properties for how to do this), but I don't think you'll need this functionality. Development release suffixes can be added with setuptools (http://peak.telecommunity.com/DevCenter/setuptools#managing-continuous-relea...). When using this technique, the setup.py version string must be the *next* version (the one under development that is typically your SVN Trunk).
It only has to be the next version if you're using a prerelease suffix like "dev". For example "1.2-r5499" is SVN revision 5499 of an already-released version 1.2. If you *haven't* released 1.2, you want to call it '1.2.dev-r5499' instead, which compares less than '1.2', e.g.:
from pkg_resources import parse_version as pv pv('1.2') > pv('1.2dev') True pv('1.2-r5499') > pv('1.2') True pv('1.2-r5499') > pv('1.2dev-r5499') True
Here's the rub-- it must be in setup.py and you probably want to give your package access to it so that it can be reported. It appears that the current idiom for solving this dilemma is to but a release.py or version.py file in your package, which is sucked up into the setup.py file with execfile (see http://kid-templating.org/trac/browser/trunk/setup.py for an example). You then manually maintain the version number in one place (the release.py file).
Yep, this is what most people do. I actually use a PEAK tool called "version" (peak.tools.version) which maintains a data file with the version, and does targeted search-and-replace operations on the files listed in a configuration file. This is a lot easier when you have documentation files that also need version information updated, and it also avoids the execfile pain.
"Phillip J. Eby"
Here's the rub-- it must be in setup.py and you probably want to give your package access to it so that it can be reported. It appears that the current idiom for solving this dilemma is to but a release.py or version.py file in your package, which is sucked up into the setup.py file with execfile (see http://kid-templating.org/trac/browser/trunk/setup.py for an example). You then manually maintain the version number in one place (the release.py file).
Yep, this is what most people do.
Is there a standard trick to get the desired svn-revision? Running `'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building command is issued and then importing that in a hand-maintained ``version.py``/``release.py`` file should work, but is there a way to check whether a setup.py command falls into a certain category?
I actually use a PEAK tool called "version" (peak.tools.version) which maintains a data file with the version, and does targeted search-and-replace operations on the files listed in a configuration file. This is a lot easier when you have documentation files that also need version information updated, and it also avoids the execfile pain.
This sounds more like it, but I don't assume this package is going to be ready for an official release anytime soon? cheers, 'as
At 01:09 AM 5/14/2007 +0100, Alexander Schmolck wrote:
"Phillip J. Eby"
writes: Here's the rub-- it must be in setup.py and you probably want to give your package access to it so that it can be reported. It appears that the current idiom for solving this dilemma is to but a release.py or version.py file in your package, which is sucked up into the setup.py file with execfile (see http://kid-templating.org/trac/browser/trunk/setup.py for an example). You then manually maintain the version number in one place (the release.py file).
Yep, this is what most people do.
Is there a standard trick to get the desired svn-revision? Running `'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building command is issued and then importing that in a hand-maintained ``version.py``/``release.py`` file should work, but is there a way to check whether a setup.py command falls into a certain category?
I don't understand what you're trying to do. Why do you need the SVN revision in a .py file?
I actually use a PEAK tool called "version" (peak.tools.version) which maintains a data file with the version, and does targeted search-and-replace operations on the files listed in a configuration file. This is a lot easier when you have documentation files that also need version information updated, and it also avoids the execfile pain.
This sounds more like it, but I don't assume this package is going to be ready for an official release anytime soon?
Nope, and it's not documented either. I stole the idea, however, from another open source program that does something quite similar, but unfortunately I don't remember the name of that other program.
"Phillip J. Eby"
At 01:09 AM 5/14/2007 +0100, Alexander Schmolck wrote:
"Phillip J. Eby"
writes: Here's the rub-- it must be in setup.py and you probably want to give your package access to it so that it can be reported. It appears that the current idiom for solving this dilemma is to but a release.py or version.py file in your package, which is sucked up into the setup.py file with execfile (see http://kid-templating.org/trac/browser/trunk/setup.py for an example). You then manually maintain the version number in one place (the release.py file).
Yep, this is what most people do.
Is there a standard trick to get the desired svn-revision? Running `'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building command is issued and then importing that in a hand-maintained ``version.py``/``release.py`` file should work, but is there a way to check whether a setup.py command falls into a certain category?
I don't understand what you're trying to do.
I'd like to have a mechanism that has me specify two things in *one* central location: 1. A self-chosen version number, e.g. "1.1" 2. A release flag (e.g. official_release=False, meaning it's an official and not a development release) Now in all other locations I'd like to get a version string that either is "1.1" if release is True "1.1.dev43" (or whatever the svn revision) if release is False. All other locations means at least setup.py and a __init__.py (via import or execfile if needs be).
Why do you need the SVN revision in a .py file?
I need it somewhere where it can be accessed by setup.py and the installed package at a negligible cost. Using some convoluted invocation to create a __svn_version__.py file in setup.py is the way numpy.distutils based project do it, but I'd really like to avoid that (and numpy.distutils in general).
This sounds more like it, but I don't assume this package is going to be ready for an official release anytime soon?
Nope, and it's not documented either. I stole the idea, however, from another open source program that does something quite similar, but unfortunately I don't remember the name of that other program.
vertoo? Unfortunately that's alpha, too (and no releases since 2003). cheers, 'as p.s. I notice that a cc: or to: to your email address bounces -- is that intended? (For what it's worth, I'd prefer if docutils-sig would just set the reply to header to the list anyway -- it's what the scipy, numpy and many other lists already do anyway and clearly the desired default behavior for the vast majority of replies to messages on this or most technical lists; I guess it might some unfortunate standard for the *-sig lists, though).
At 01:58 PM 5/14/2007 +0100, Alexander Schmolck wrote:
"Phillip J. Eby"
writes: At 01:09 AM 5/14/2007 +0100, Alexander Schmolck wrote:
"Phillip J. Eby"
writes: Here's the rub-- it must be in setup.py and you probably want to give your package access to it so that it can be reported. It appears that the current idiom for solving this dilemma is to but a release.py or version.py file in your package, which is sucked up into the setup.py file with execfile (see http://kid-templating.org/trac/browser/trunk/setup.py for an example). You then manually maintain the version number in one place (the release.py file).
Yep, this is what most people do.
Is there a standard trick to get the desired svn-revision? Running `'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building command is issued and then importing that in a hand-maintained ``version.py``/``release.py`` file should work, but is there a way to check whether a setup.py command falls into a certain category?
I don't understand what you're trying to do.
I'd like to have a mechanism that has me specify two things in *one* central location:
1. A self-chosen version number, e.g. "1.1"
Put this in setup.py.
2. A release flag (e.g. official_release=False, meaning it's an official and not a development release)
Put this in setup.cfg for a development release (or leave blank for official): [egg_info] tag_svn_revision = 1 tag_build = dev
Now in all other locations I'd like to get a version string that either is "1.1" if release is True "1.1.dev43" (or whatever the svn revision) if release is False.
All other locations means at least setup.py and a __init__.py (via import or execfile if needs be).
At runtime, you can use this code to get the version of your project (or any other project): from pkg_resources import require __version__ = require('MyProjectname')[0].version Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of your distribution.) You can also do this at the interpreter prompt, e.g.: Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
from pkg_resources import require require('PEAK') [peak 0.5a4.dev-r2287 (c:\python23\lib\site-packages\peak-0.5a4.dev_r2287-py2.3-win32.egg), DecoratorTools 1.4 (c:\cygwin\home\pje\projects\decoratortools), zconfig 2.3 (c:\python23\lib\site-packages\zconfig-2.3-py2.3.egg), wsgiref 0.1.2 (c:\cygwin\home\pje\projects\wsgiref), SymbolType 1.0 (c:\cygwin\home\pje\projects\symboltype), Importing 1.9.3 (c:\cygwin\home\pje\projects\importing), PyProtocols 1.0a0dev-r2302 (c:\cygwin\home\pje\projects\pyprotocols\src), ruledispatch 0.5a0.dev-r2287 (c:\python23\lib\site-packages\ruledispatch-0.5a0.dev_r2287-py2.3-win32.egg)]
As you can see, require() returns a list of objects representing all the packages needed to meet the given request, and the default repr() inlcudes both version numbers and file paths. This is really the best way to find out what version(s) of something somebody is using, as it includes your dependencies as well as the base package.
p.s. I notice that a cc: or to: to your email address bounces -- is that intended?
No, and AFAIK it's not happening to anybody else.
"Phillip J. Eby"
I'd like to have a mechanism that has me specify two things in *one* central location:
1. A self-chosen version number, e.g. "1.1"
Put this in setup.py.
2. A release flag (e.g. official_release=False, meaning it's an official and not a development release)
Put this in setup.cfg for a development release (or leave blank for official):
[egg_info] tag_svn_revision = 1 tag_build = dev
Will do. But wouldn't it make sense to be able to specify the version name in this place to?
Now in all other locations I'd like to get a version string that either is "1.1" if release is True "1.1.dev43" (or whatever the svn revision) if release is False.
All other locations means at least setup.py and a __init__.py (via import or execfile if needs be).
At runtime, you can use this code to get the version of your project (or any other project):
from pkg_resources import require __version__ = require('MyProjectname')[0].version
Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of your distribution.)
Just to make sure because I'm not 100% clear on this and I think I have have come across conflicting information from different sources -- if my package is named scikits.mlabwrap and scikits is a namespace package, setup.py will have to look like so, right? setup(name='sckits.mlabwrap', # rather than just 'mlabwrap' namespace_packages=['scikits'] ... ) and then ``require(scikits.mlabwrap)[0].version`` will get me the version number. As a suggestion: I think that the ``namespace_packages`` example for the ZoperInterface project at http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages would profit from adding the ``name=`` parameter, because it is not immediately obvious whether it should include the namespace_package or not.
[...]
Thanks for all the info! I've tried to document the most important points about setuptools for other scikits developers, if you or someone else who's more knowledgable about setuptools wants to give some feedback (although it's obviously still work in progress), here's the url: http://projects.scipy.org/scipy/scikits/wiki/WikiStart#Scikitsuseofsetuptool... Thanks again for your help, cheers 'as
p.s. I notice that a cc: or to: to your email address bounces -- is that intended?
No, and AFAIK it's not happening to anybody else.
FWIW, here's the gist of the error message:
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
pje@telecommunity.com
SMTP error from remote mail server after RCPT TO:
At 09:14 PM 5/15/2007 +0100, Alexander Schmolck wrote:
"Phillip J. Eby"
writes: Will do. But wouldn't it make sense to be able to specify the version name in this place to?
Well, the version is a distribution level option; it can't be specified in setup.cfg. That's a distutils thing, I'm afraid. Btw, I erred in one point in my last email, your setup.cfg should actually read: [egg_info] tag_svn_revision = 1 tag_build = .dev Note the extra '.' - this ensures that if you have something like '1.1a' as your version, you won't end up with '1.1adev' as your version.
Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of your distribution.)
Just to make sure because I'm not 100% clear on this and I think I have have come across conflicting information from different sources -- if my package is named scikits.mlabwrap and scikits is a namespace package, setup.py will have to look like so, right?
setup(name='sckits.mlabwrap', # rather than just 'mlabwrap' namespace_packages=['scikits'] ... )
Uh, no. Your distribution name could be "Matlab Wrapper for SciKits" or "FooBarBaz", for all the distutils or setuptools care. :) Project names don't have any defined relationship to package names. For example the "Importing" project on the Cheeseshop provides a module called "peak.util.imports".
and then ``require(scikits.mlabwrap)[0].version`` will get me the version number.
You need quotes around the scikits.mlabwrap... or 'FooBarBaz', if that's your project name. :)
As a suggestion: I think that the ``namespace_packages`` example for the ZoperInterface project at http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages would profit from adding the ``name=`` parameter, because it is not immediately obvious whether it should include the namespace_package or not.
It's entirely unrelated - note that a project can contain multiple packages, or even no packages at all.
Thanks for all the info! I've tried to document the most important points about setuptools for other scikits developers, if you or someone else who's more knowledgable about setuptools wants to give some feedback (although it's obviously still work in progress), here's the url:
http://projects.scipy.org/scipy/scikits/wiki/WikiStart#Scikitsuseofsetuptool...
Reading that page, I'm not sure whether you want a scikits namespace package in the first place. A namespace package is used to group several projects under a single "owner", ala the various zope.* and peak.* and plone.* packages. However, nobody but Zope Corp. should be releasing a zope.* package, and nobody but me releasing a peak.* one. The whole point is to "claim" a namespace for packages being released by a single owner - either an organization or umbrella project. This corresponds to things like the 'org.apache' and 'com.sun' package namespaces in Java - it's just a namespace to allow authors free rein to put whatever modules and packages they like under that namespace. However, it sounds from that page like scikits are independently created - which means there's no need for a namespace package there, and in fact it's counter-indicated for that situation! Also, may I suggest that rather than having a __version__ variable, why not just have a get_version() function? That way, you needn't import pkg_resources or do a require() as part of a simple import operation. Finally, it's not the case that you mustn't write a MANIFEST.in; setuptools will use MANIFEST.in just fine. What setuptools *won't* do is rely on a generated MANIFEST for anything; it always uses a freshly-generated internal manifest, using MANIFEST.in + revision control info. Thus, it's impossible to mess up setuptools with a stale MANIFEST file, the way you can with distutils. (And if you're using CVS or SVN, you mostly won't need to bother with MANIFEST.in.)
Phillip J. Eby wrote:
Reading that page, I'm not sure whether you want a scikits namespace package in the first place. A namespace package is used to group several projects under a single "owner", ala the various zope.* and peak.* and plone.* packages. However, nobody but Zope Corp. should be releasing a zope.* package, and nobody but me releasing a peak.* one. The whole point is to "claim" a namespace for packages being released by a single owner - either an organization or umbrella project. This corresponds to things like the 'org.apache' and 'com.sun' package namespaces in Java - it's just a namespace to allow authors free rein to put whatever modules and packages they like under that namespace.
This is exactly what we are aiming for. While different people will be responsbile for each subpackage, they are all operating under the aegis of the scikits infrastructure. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
At 04:47 PM 5/15/2007 -0500, Robert Kern wrote:
Phillip J. Eby wrote:
Reading that page, I'm not sure whether you want a scikits namespace package in the first place. A namespace package is used to group several projects under a single "owner", ala the various zope.* and peak.* and plone.* packages. However, nobody but Zope Corp. should be releasing a zope.* package, and nobody but me releasing a peak.* one. The whole point is to "claim" a namespace for packages being released by a single owner - either an organization or umbrella project. This corresponds to things like the 'org.apache' and 'com.sun' package namespaces in Java - it's just a namespace to allow authors free rein to put whatever modules and packages they like under that namespace.
This is exactly what we are aiming for. While different people will be responsbile for each subpackage, they are all operating under the aegis of the scikits infrastructure.
But who adds things to the namespace? Is there a single organization that decides what packages or modules will exist in scikits?
Phillip J. Eby wrote:
At 04:47 PM 5/15/2007 -0500, Robert Kern wrote:
Phillip J. Eby wrote:
Reading that page, I'm not sure whether you want a scikits namespace package in the first place. A namespace package is used to group several projects under a single "owner", ala the various zope.* and peak.* and plone.* packages. However, nobody but Zope Corp. should be releasing a zope.* package, and nobody but me releasing a peak.* one. The whole point is to "claim" a namespace for packages being released by a single owner - either an organization or umbrella project. This corresponds to things like the 'org.apache' and 'com.sun' package namespaces in Java - it's just a namespace to allow authors free rein to put whatever modules and packages they like under that namespace. This is exactly what we are aiming for. While different people will be responsbile for each subpackage, they are all operating under the aegis of the scikits infrastructure.
But who adds things to the namespace? Is there a single organization that decides what packages or modules will exist in scikits?
Yes, the scipy developer team. mlabwrap is just the first scikits project. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
"Phillip J. Eby"
At 09:14 PM 5/15/2007 +0100, Alexander Schmolck wrote:
"Phillip J. Eby"
writes: Will do. But wouldn't it make sense to be able to specify the version name in this place to? Well, the version is a distribution level option; it can't be specified in setup.cfg. That's a distutils thing, I'm afraid.
Why can't [egg_info] just grow a 'version' field? Due to the way the code is organized? I don't really understand why, logically, the .dev suffix is not distribution level -- it forms part of the .egg version (and what can be required/queried by other packages) after all...
Btw, I erred in one point in my last email, your setup.cfg should actually read:
[egg_info] tag_svn_revision = 1 tag_build = .dev
Note the extra '.' - this ensures that if you have something like '1.1a' as your version, you won't end up with '1.1adev' as your version.
Thanks, corrected
Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of your distribution.)
Just to make sure because I'm not 100% clear on this and I think I have have come across conflicting information from different sources -- if my package is named scikits.mlabwrap and scikits is a namespace package, setup.py will have to look like so, right?
setup(name='sckits.mlabwrap', # rather than just 'mlabwrap' namespace_packages=['scikits'] ... )
Uh, no. Your distribution name could be "Matlab Wrapper for SciKits" or "FooBarBaz", for all the distutils or setuptools care. :) Project names don't have any defined relationship to package names. For example the "Importing" project on the Cheeseshop provides a module called "peak.util.imports".
OK, I see (I guess I must have been aware of that once). Maybe it would be good to state this somewhere were it's easy to find -- try searching for "name" on the setuptools page to see what I mean -- 'name' occurs in all sort of combinations and the first body text occurence is in http://peak.telecommunity.com/DevCenter/setuptools#basic-use, but I it's meaning is not further explained there. Maybe something like: "Note that the ``name = "HelloWorld"`` option has no other effect than specifiying what the distribtuion name of your package will be [[i.e. .egg name and cheeseshop?]]; in particular it is completely independent of the package name itself (i.e. the package import statement could in principle be ``import goodbye.cruel.world`` rather than ``import HelloWorld``)." or just name='HelloWorld' # what the project will be listed as in Cheeseshop
and then ``require(scikits.mlabwrap)[0].version`` will get me the version number.
You need quotes around the scikits.mlabwrap... or 'FooBarBaz', if that's your project name. :)
Yup, typo.
As a suggestion: I think that the ``namespace_packages`` example for the ZoperInterface project at http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages would profit from adding the ``name=`` parameter, because it is not immediately obvious whether it should include the namespace_package or not.
It's entirely unrelated - note that a project can contain multiple packages, or even no packages at all.
Actually mlabwrap itself once had several toplevel modules -- in distutils/pre-scikits days -- so as I said, I likely was once aware of this :)
Thanks for all the info! I've tried to document the most important points about setuptools for other scikits developers, if you or someone else who's more knowledgable about setuptools wants to give some feedback (although it's obviously still work in progress), here's the url:
http://projects.scipy.org/scipy/scikits/wiki/WikiStart#Scikitsuseofsetuptool...
Reading that page, I'm not sure whether you want a scikits namespace package in the first place. A namespace package is used to group several projects under a single "owner", ala the various zope.* and peak.* and plone.* packages. However, nobody but Zope Corp. should be releasing a zope.* package, and nobody but me releasing a peak.* one. The whole point is to "claim" a namespace for packages being released by a single owner - either an organization or umbrella project.
I think this holds true here (the umbrella community being scipy). Although the barriers for entry are lower than for scipy core, not anyone who feels like it can just create a scikits package -- for one thing someone needs to give you svn etc. access (somone = Jeff Strunk from enthought).
This corresponds to things like the org.apache' and 'com.sun' package namespaces in Java - it's just a namespace to allow authors free rein to put whatever modules and packages they like under that namespace. However, it sounds from that page like scikits are independently created - which means there's no need for a namespace package there, and in fact it's counter-indicated for that situation!
Well, independent as in not (necessarily) having mutual dependencies or requiring joint install (although pretty much all packages will need scipy or at least numpy) -- not as in having absolutely nothing to do with each other or being free for all. I think[1] an publication in a specilized journal run by a dedicated scientific community (say the ACM's APLers) might be vaguely analogous -- it provides a way for people who are interested in a particular niche to find pool and find relevant stuff that passed at least some weak form of peer-review to get to be associated with the ACM brand.
Also, may I suggest that rather than having a __version__ variable, why not just have a get_version() function?
The great advantage of __version__ is that it's standard. I personally frequently look up the version of an installed package by typing <MODULE>.__v<tab> and I'm seldomly disappointed; it wouldn't occur to me to look for a get_version function.
That way, you needn't import pkg_resources or do a require() as part of a simple import operation.
I hope it doesn't potentially take ages? For mlabwrap it wouldn't matter much (matlab takes ages to start up on import), but for other scikits this might be a problem.
Finally, it's not the case that you mustn't write a MANIFEST.in; setuptools will use MANIFEST.in just fine. What setuptools *won't* do is rely on a generated MANIFEST for anything; it always uses a freshly-generated internal manifest, using MANIFEST.in + revision control info. Thus, it's impossible to mess up setuptools with a stale MANIFEST file, the way you can with distutils. (And if you're using CVS or SVN, you mostly won't need to bother with MANIFEST.in.)
OK, thanks for the clarification -- I've edited the webpage. cheers, 'as Footnotes: [1] The whole scikits stuff isn't my idea and its not fully fleshed out yet, so I'm just guessing wildly :) Robert Kern is more qualified to say something about the scope and aims of scikits.
Btw, I forgot to address this part earlier... At 09:14 PM 5/15/2007 +0100, Alexander Schmolck wrote:
p.s. I notice that a cc: or to: to your email address bounces -- is that intended?
No, and AFAIK it's not happening to anybody else.
FWIW, here's the gist of the error message:
A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed:
pje@telecommunity.com SMTP error from remote mail server after RCPT TO:
: host mail.telecommunity.com [209.190.5.234]: 550 : Recipient address rejected: SPF Reports: SPF fail - not authorized ------ This is a copy of the message, including all the headers. ------
Return-path:
Received: from oc.ex.ac.uk ([144.173.207.34]) by dot.ex.ac.uk with esmtp (Exim 4.62/mail) id 1HnLB1-0003Br-Sq; Sun, 13 May 2007 22:00:23 +0100
You're sending your message with a return address @gmx.net, but gmx.net's SPF records read: gmx.net. 300 IN TXT "v=spf1 ip4:213.165.64.0/23 -all" Which means that your ISP says that anybody using a return address @gmx.net who isn't sending their mail via the 213.165.64.0/23 IP block is a lying spammer pretending to have a gmx.net address. :) Thus, any SPF-supporting mail server is going to think you're a spammer and block your mail, unless you send it through a gmx.net-supported SMTP server.
participants (3)
-
Alexander Schmolck
-
Phillip J. Eby
-
Robert Kern