PEP 376 -- support for release versions?
Hi, While reading the PEP I noticed that there seems to be no straightforward to specify the package release versions.
egginfo_dirname('python-ldap', '2.3.10') 'python_ldap-2.3.10.egg-info'
Consider the following PyPM package file name: python-ldap-2.3.10_linux-x86_2.6_2.pypm Here 2.3.10 is python-ldap's version, 2.6 is the Python version .. and 2 (the last component) is the binary release number (for ActiveState repository). This is to upgrade previous releases of the same python-ldap (say, python-ldap-2.3.10_linux-x86_2.6_1.pypm). If PyPM is to rely on PEP 376 style .egg-info directories (and the RECORD files within them), how is it to store/retrieve this release number? Currently we store everything (including RECORD) in a sqlite database under site-packages. I can continue using the db for release numbers alone, but I really prefer that this is covered by a standard that evidently provides support for installer tools. -srid
On Thu, Nov 5, 2009 at 9:31 PM, Sridhar Ratnakumar <sridharr@activestate.com> wrote:
Hi,
While reading the PEP I noticed that there seems to be no straightforward to specify the package release versions.
>>> egginfo_dirname('python-ldap', '2.3.10') 'python_ldap-2.3.10.egg-info'
Consider the following PyPM package file name:
python-ldap-2.3.10_linux-x86_2.6_2.pypm
Here 2.3.10 is python-ldap's version, 2.6 is the Python version .. and 2 (the last component) is the binary release number (for ActiveState repository). This is to upgrade previous releases of the same python-ldap (say, python-ldap-2.3.10_linux-x86_2.6_1.pypm).
If I understand correctly, ActiveState has its own releases of python-ldap repackaged in pypm files, and has several releases per python-ldap versions. So, why it doesn't have its own versions ? like: 2.3.10.2 in that case. (following a PEP 386-compatible scheme) So this would be:
egginfo_dirname('python-ldap', '2.3.10.2') 'python_ldap-2.3.10.2.egg-info'
Tarek
On 2009-11-05 15:03 PM, Tarek Ziadé wrote:
On Thu, Nov 5, 2009 at 9:31 PM, Sridhar Ratnakumar <sridharr@activestate.com> wrote:
Hi,
While reading the PEP I noticed that there seems to be no straightforward to specify the package release versions.
egginfo_dirname('python-ldap', '2.3.10') 'python_ldap-2.3.10.egg-info'
Consider the following PyPM package file name:
python-ldap-2.3.10_linux-x86_2.6_2.pypm
Here 2.3.10 is python-ldap's version, 2.6 is the Python version .. and 2 (the last component) is the binary release number (for ActiveState repository). This is to upgrade previous releases of the same python-ldap (say, python-ldap-2.3.10_linux-x86_2.6_1.pypm).
If I understand correctly, ActiveState has its own releases of python-ldap repackaged in pypm files, and has several releases per python-ldap versions.
So, why it doesn't have its own versions ? like: 2.3.10.2 in that case. (following a PEP 386-compatible scheme)
So this would be:
egginfo_dirname('python-ldap', '2.3.10.2') 'python_ldap-2.3.10.2.egg-info'
Generally speaking, it is desirable to distinguish between the upstream package's version and the build number rather than munging them together. You never know when the upstream package will decide that they need to make a really quick bugfix release and just append ".1". The main package version is and should remain entirely under the control of the upstream developers, not the binary packagers. However, the binary packagers need a place to mark revisions of the binary packages distinct from the upstream sources. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Thu, Nov 5, 2009 at 10:31 PM, Robert Kern <robert.kern@gmail.com> wrote: [..]
Generally speaking, it is desirable to distinguish between the upstream package's version and the build number rather than munging them together. You never know when the upstream package will decide that they need to make a really quick bugfix release and just append ".1". The main package version is and should remain entirely under the control of the upstream developers, not the binary packagers. However, the binary packagers need a place to mark revisions of the binary packages distinct from the upstream sources.
I see. The question is: do we want to handle this at Distutils level (e.g. by adding a new metadata field as I said in my earlier mail), or is this to be done by the package manager that is on the top of Python on the target system (pypm, debian dpkg, etc..) and can drive installations with PEP 376 's APIs we have described earlier. I would say the latter is the best pick. Tarek
On Thu, 05 Nov 2009 13:03:45 -0800, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
If I understand correctly, ActiveState has its own releases of python-ldap repackaged in pypm files, and has several releases per python-ldap versions.
Correct. Much like what Linux distributions (like debian) do.
So, why it doesn't have its own versions ? like: 2.3.10.2 in that case. (following a PEP 386-compatible scheme) So this would be:
egginfo_dirname('python-ldap', '2.3.10.2') 'python_ldap-2.3.10.2.egg-info'
Because python-ldap may release a similar version in future. We don't have to cause collisions. This is why the release version is not part of package version in the *.pypm filename. I actually now think that storing the release number inside the .egg-info/ directory may be a better idea. Like: $ cat python_ldap-2.3.10.egg-info/pypm.txt release: 2 $ What do you think? -srid
On Thu, Nov 5, 2009 at 10:31 PM, Sridhar Ratnakumar <sridharr@activestate.com> wrote:
Because python-ldap may release a similar version in future. We don't have to cause collisions. This is why the release version is not part of package version in the *.pypm filename.
Ok
I actually now think that storing the release number inside the .egg-info/ directory may be a better idea. Like:
$ cat python_ldap-2.3.10.egg-info/pypm.txt release: 2 $
What do you think?
In that case, an arbitrary file that gets installed in the .egg-info/ directory, that you can read back when needed through the PEP 376 pkgutil new API, seem to be a good idea. This implies that Python+distutils itself will not be able to distinguish two different releases that have the same version, but a different build version. e.g. that just Active State's package manager is able to manage this extra number. The other option, if this extra binary version number is a generic need, would be to add a new metadata field in PEP 345. (like in RPMs IIRC) and build an egg info folder name that contains this extra info when present. Regards Tarek
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziadé wrote:
In that case, an arbitrary file that gets installed in the .egg-info/ directory, that you can read back when needed through the PEP 376 pkgutil new API, seem to be a good idea.
PEP 376 doesn't include a pkgutil API for reading arbitrary metadata files from .egg-info. But IMO it probably should. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK80+CFRcxmeyPUXIRAkojAJ41bHVhe0Omasz5fOR1sxaRWURvMQCfYxSq JteyNimZuPYNdo9b9Pz+dHU= =1G0s -----END PGP SIGNATURE-----
On Thu, 05 Nov 2009 14:14:19 -0800, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I actually now think that storing the release number inside the .egg-info/ directory may be a better idea. Like:
$ cat python_ldap-2.3.10.egg-info/pypm.txt release: 2 $
What do you think? In that case, an arbitrary file that gets installed in the .egg-info/ directory, that you can read back when needed through the PEP 376 pkgutil new API, seem to be a good idea. This implies that Python+distutils itself will not be able to distinguish two different releases that have the same version, but a different build version.
I assume that the user is not supposed to have two copies of python-ldap-2.3.10 (but with different release versions) anyway. The purpose of multiple release version is to provide upgrades to a package within a single package version. The *value* of the release version itself is specific to PyPM, of course. Other binary installers will not have a need for it (they will use their own format -- see below).
e.g. that just Active State's package manager is able to manage this extra number.
Yes. And package manager FooBar may use different release number formats, say, timestamps. $ foobar install python-ldap Getting python-ldap_2.3.10_linux_20091023.fbar FooBar should not read PyPM's release number.
The other option, if this extra binary version number is a generic need, would be to add a new metadata field in PEP 345. (like in RPMs IIRC) and build an egg info folder name that contains this extra info when present.
Since release numbers are specific to the installer (and repository), how about having it in the .egg-info/INSTALLER file? $ cat .egg-info/INSTALLER name: PyPM [package] release: 2 url: http://pypm.activestate.com/..../python-ldap....pypm pinned: True ... This will also enable adding extra metadata (specific to installer) like version pinning. -srid
On Thu, Nov 5, 2009 at 11:39 PM, Sridhar Ratnakumar <sridharr@activestate.com> wrote: [..]
Since release numbers are specific to the installer (and repository), how about having it in the .egg-info/INSTALLER file?
$ cat .egg-info/INSTALLER name: PyPM
[package] release: 2 url: http://pypm.activestate.com/..../python-ldap....pypm pinned: True ...
We specified an INSTALLER file already, see: http://www.python.org/dev/peps/pep-0376/#id33 But just with the name of the installer. The use case was to be able to avoid removing a distribution that was installed by another installer. I don't think the extra bit ([package]) is necessary in the INSTALLER file, because it's specific to the installer tool. And, as long as you can read/write arbitrary files in /.egg-info, you can put it in a specific file. Regards Tarek
On Thu, Nov 05, 2009 at 11:45:47PM +0100, Tarek Ziadé wrote:
On Thu, Nov 5, 2009 at 11:39 PM, Sridhar Ratnakumar <sridharr@activestate.com> wrote: [..]
Since release numbers are specific to the installer (and repository), how about having it in the .egg-info/INSTALLER file?
$ cat .egg-info/INSTALLER name: PyPM
[package] release: 2 url: http://pypm.activestate.com/..../python-ldap....pypm pinned: True ...
We specified an INSTALLER file already, see: http://www.python.org/dev/peps/pep-0376/#id33
But just with the name of the installer. The use case was to be able to avoid removing a distribution that was installed by another installer.
I don't think the extra bit ([package]) is necessary in the INSTALLER file, because it's specific to the installer tool. And, as long as you can read/write arbitrary files in /.egg-info, you can put it in a specific file.
It seems a bad idea to put this extra info in a distutils standard, the binary release version have been managed outside of distutils for a long time and it works perfect. Allowing an arbitrary file to to be read via the egg-metadata API seems a much better solution, then any packager using that facility can completely decide what format to use and freely encode more information without hiting new distutils limitations (like your pinning). Of course there's nothing wrong with pypm keeping it's sqlite db for it's own data either, other binary package managers keep that info outside too. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
[Sridhar Ratnakumar, 2009-11-05]
On Thu, 05 Nov 2009 13:03:45 -0800, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
If I understand correctly, ActiveState has its own releases of python-ldap repackaged in pypm files, and has several releases per python-ldap versions.
Correct. Much like what Linux distributions (like debian) do.
Debian revision is used only in .deb, .egg-info or __version__ is not changed. -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
On Mon, 09 Nov 2009 07:48:15 -0800, Piotr Ozarowski <ozarow@gmail.com> wrote:
[Sridhar Ratnakumar, 2009-11-05]
On Thu, 05 Nov 2009 13:03:45 -0800, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
If I understand correctly, ActiveState has its own releases of python-ldap repackaged in pypm files, and has several releases per python-ldap versions.
Correct. Much like what Linux distributions (like debian) do. Debian revision is used only in .deb, .egg-info or __version__ is not changed.
Same with PyPM. Which is why release numbers need to be stored elsewhere. Given the current design/proposals - what makes sense to me now is to store it in .egg-info/PyPM.txt. -srid
On Mon, 09 Nov 2009 13:45:29 -0800, "Sridhar Ratnakumar" <sridharr@activestate.com> wrote:
Same with PyPM. Which is why release numbers need to be stored elsewhere.
Given the current design/proposals - what makes sense to me now is to store it in .egg-info/PyPM.txt.
But release numbers are already stored in the PKG_INFO metadata file. Why store it somewhere else? David
On 2009-11-09 16:28 PM, David Lyon wrote:
On Mon, 09 Nov 2009 13:45:29 -0800, "Sridhar Ratnakumar" <sridharr@activestate.com> wrote:
Same with PyPM. Which is why release numbers need to be stored elsewhere.
Given the current design/proposals - what makes sense to me now is to store it in .egg-info/PyPM.txt.
But release numbers are already stored in the PKG_INFO metadata file. Why store it somewhere else?
No, they're not. http://www.python.org/dev/peps/pep-0241/ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Mon, 09 Nov 2009 16:39:40 -0600, Robert Kern <robert.kern@gmail.com> wrote:
But release numbers are already stored in the PKG_INFO metadata file. Why store it somewhere else?
No, they're not.
Ok, well they're part of http://www.python.org/dev/peps/pep-0345/ I'll revise my statement back to saying that doing it according to PEP-345 makes the most sense imho. PEP-376 specifies the simplest way of storing a version number in a package that I can see. David
On 2009-11-09 16:53 PM, David Lyon wrote:
On Mon, 09 Nov 2009 16:39:40 -0600, Robert Kern<robert.kern@gmail.com> wrote:
But release numbers are already stored in the PKG_INFO metadata file. Why store it somewhere else?
No, they're not.
Ok, well they're part of http://www.python.org/dev/peps/pep-0345/
I'll revise my statement back to saying that doing it according to PEP-345 makes the most sense imho.
Where? I don't see it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Mon, 09 Nov 2009 17:01:53 -0600, Robert Kern <robert.kern@gmail.com> wrote:
Ok, well they're part of http://www.python.org/dev/peps/pep-0345/
I'll revise my statement back to saying that doing it according to PEP-345 makes the most sense imho.
Where? I don't see it.
Right at the top, 3rd field, after Metadata-Version and Name: Version A string containing the package's version number. This field must be in the format specified in PEP 386 [3]. Example: Version: 1.0a2
On 2009-11-09 17:17 PM, David Lyon wrote:
On Mon, 09 Nov 2009 17:01:53 -0600, Robert Kern<robert.kern@gmail.com> wrote:
Ok, well they're part of http://www.python.org/dev/peps/pep-0345/
I'll revise my statement back to saying that doing it according to PEP-345 makes the most sense imho.
Where? I don't see it.
Right at the top, 3rd field, after Metadata-Version and Name:
Version
A string containing the package's version number. This field must be in the format specified in PEP 386 [3].
Example:
Version: 1.0a2
That's not what we're talking about. That is the version of the upstream code itself. We're talking about release numbers for the binary package made by third parties like ActiveState or Enthought. As we fix bugs in our build process for each package, we may need to release multiple binary packages for each main version of the upstream code. Please read the rest of the thread for the reasons why we need to do this and why we should not simply munge the main package version to do this. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Mon, 09 Nov 2009 17:25:45 -0600, Robert Kern <robert.kern@gmail.com> wrote:
We're talking about release numbers for the binary package made by third parties like ActiveState or Enthought. As we fix bugs in our build process for each package, we may need to release multiple binary packages for each main
Thanks for clarifying. Ok, well if it hidden way inside the package where nobody really bothers to look I guess that it won't make much difference. I have to say that my opinion is that we should be cleaning up the crap that is inside packages and not adding to it. Presently we have all sorts of duplication of information in there, much of which isn't used. Lets look: 1) dependency_links : imo obsolete, usually left empty 2) requires : imo made obsolete soon by PEP-345 3) sources : Works fine 4) top_level : wt.. is this? What program uses this? 5) not-zip-safe : couldn't this be a setting in setup.cfg? 6) PKG-INFO : The metadata file itself So from the above list, 4 out 6 files serve no real purpose. I'm actually ok with new files and directories created after installation. But that should be made a little clearer in the PEP. It's ambiguous as to whether the files are carried in the package or created at installation time. David
participants (7)
-
Carl Meyer -
David Lyon -
Floris Bruynooghe -
Piotr Ozarowski -
Robert Kern -
Sridhar Ratnakumar -
Tarek Ziadé