I was wondering if anyone was aware of anything that relies on the current structure of package urls, namely: /packages/source|any|etc/D/Django-1.0.tar.gz? I would like to change this but before I work up a concrete plan for people to comment on and discuss I'm trying to figure out what, if anything, depends on that current structure. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
I was wondering if anyone was aware of anything that relies on the current structure of package urls, namely:
/packages/source|any|etc/D/Django-1.0.tar.gz?
I would like to change this but before I work up a concrete plan for people to comment on and discuss I'm trying to figure out what, if anything, depends on that current structure.
I've told Launchpad that it can look for new releases at URLs like http://pypi.python.org/packages/source/g/gtimelog/gtimelog-*.tar.gz (for a few projects, not just GTimeLog). I'm not sure how Launchpad crawls those pages looking for releases; all the documentation I could find in 5 minutes was this blog post: http://blog.launchpad.net/cool-new-stuff/automatically-import-files-to-launc... Some Debian packages have debian/watch files using similar URL patterns to watch for new upstream releases showing up on PyPI. This mechanism is documented at https://wiki.debian.org/debian/watch/. Here's python-requests: $ apt-get source python-requests $ cat requests-1.1.0/debian/watch version=3 http://pypi.python.org/packages/source/r/requests/requests-(.*)\.tar\.gz HTH, Marius Gedminas -- Some people have told me they don't think a fat penguin really embodies the grace of Linux, which just tells me they have never seen a angry penguin charging at them in excess of 100mph. They'd be a lot more careful about what they say if they had. -- Linus Torvalds, announcing Linux v2.0
On Tue, Oct 08, 2013 at 05:05:54PM +0300, Marius Gedminas wrote:
On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
I was wondering if anyone was aware of anything that relies on the current structure of package urls, namely:
/packages/source|any|etc/D/Django-1.0.tar.gz?
I would like to change this but before I work up a concrete plan for people to comment on and discuss I'm trying to figure out what, if anything, depends on that current structure.
Some Debian packages have debian/watch files using similar URL patterns
Perhaps "Some" is an understatement ;) http://qa.debian.org/developer.php?login=python-modules-team@lists.alioth.de... lists over 500 packages, almost all of which have watch files (last 2 columns). There are probably quite a few more Debian packages with watch files not maintained by the Python Modules Team. -- Brian Sutherland
Hrm, I'm assuming these require a file listing at /packages/source/D/ too. Of course these files should probably be using the simple API and not the packages url directly :/ On Oct 8, 2013, at 10:05 AM, Marius Gedminas <marius@pov.lt> wrote:
On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
I was wondering if anyone was aware of anything that relies on the current structure of package urls, namely:
/packages/source|any|etc/D/Django-1.0.tar.gz?
I would like to change this but before I work up a concrete plan for people to comment on and discuss I'm trying to figure out what, if anything, depends on that current structure.
I've told Launchpad that it can look for new releases at URLs like http://pypi.python.org/packages/source/g/gtimelog/gtimelog-*.tar.gz (for a few projects, not just GTimeLog).
I'm not sure how Launchpad crawls those pages looking for releases; all the documentation I could find in 5 minutes was this blog post: http://blog.launchpad.net/cool-new-stuff/automatically-import-files-to-launc...
Some Debian packages have debian/watch files using similar URL patterns to watch for new upstream releases showing up on PyPI. This mechanism is documented at https://wiki.debian.org/debian/watch/. Here's python-requests:
$ apt-get source python-requests $ cat requests-1.1.0/debian/watch version=3 http://pypi.python.org/packages/source/r/requests/requests-(.*)\.tar\.gz
HTH, Marius Gedminas -- Some people have told me they don't think a fat penguin really embodies the grace of Linux, which just tells me they have never seen a angry penguin charging at them in excess of 100mph. They'd be a lot more careful about what they say if they had. -- Linus Torvalds, announcing Linux v2.0 _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Tue, Oct 08, 2013 at 10:44 -0400, Donald Stufft wrote:
Hrm, I'm assuming these require a file listing at
/packages/source/D/ too.
Of course these files should probably be using the simple API and not the packages url directly :/
Indeed. Why not watch a project's simple index instead? It might give false updates if a project is in non-pypi hosting mode and a new description is registered (this could change links in the simple page). But those should be rare i'd think. Watching such distribution-type specific URLs is unreliable, for example when a pypi maintainer change their distribution format (e.g. to wheel which is kind of realistic in the mid-term). So a change will need to happen in suchs cases, anyway. cheers, holger
On Oct 8, 2013, at 10:05 AM, Marius Gedminas <marius@pov.lt> wrote:
On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
I was wondering if anyone was aware of anything that relies on the current structure of package urls, namely:
/packages/source|any|etc/D/Django-1.0.tar.gz?
I would like to change this but before I work up a concrete plan for people to comment on and discuss I'm trying to figure out what, if anything, depends on that current structure.
I've told Launchpad that it can look for new releases at URLs like http://pypi.python.org/packages/source/g/gtimelog/gtimelog-*.tar.gz (for a few projects, not just GTimeLog).
I'm not sure how Launchpad crawls those pages looking for releases; all the documentation I could find in 5 minutes was this blog post: http://blog.launchpad.net/cool-new-stuff/automatically-import-files-to-launc...
Some Debian packages have debian/watch files using similar URL patterns to watch for new upstream releases showing up on PyPI. This mechanism is documented at https://wiki.debian.org/debian/watch/. Here's python-requests:
$ apt-get source python-requests $ cat requests-1.1.0/debian/watch version=3 http://pypi.python.org/packages/source/r/requests/requests-(.*)\.tar\.gz
HTH, Marius Gedminas -- Some people have told me they don't think a fat penguin really embodies the grace of Linux, which just tells me they have never seen a angry penguin charging at them in excess of 100mph. They'd be a lot more careful about what they say if they had. -- Linus Torvalds, announcing Linux v2.0 _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Oct 08, 2013, at 10:44 AM, Donald Stufft wrote:
Hrm, I'm assuming these require a file listing at
/packages/source/D/ too.
Of course these files should probably be using the simple API and not the packages url directly :/
Maybe that URL should be given on the PyPI page for the package? The current watch patterns are used overwhelmingly probably because they are easily discovered. E.g. Copy the URL from the PyPI page, and "patternize" it. -Barry
On 9 Oct 2013 21:03, "Barry Warsaw" <barry@python.org> wrote:
On Oct 08, 2013, at 10:44 AM, Donald Stufft wrote:
Hrm, I'm assuming these require a file listing at
/packages/source/D/ too.
Of course these files should probably be using the simple API and not the packages url directly :/
Maybe that URL should be given on the PyPI page for the package?
The current watch patterns are used overwhelmingly probably because they
are
easily discovered.
E.g. Copy the URL from the PyPI page, and "patternize" it.
Sounds like this could be addressed with a lintian warning and a long transition time (~2 years) so leaving proper redirects around would be advisable. Regards, Floris
On Oct 12, 2013, at 08:50 PM, Floris Bruynooghe wrote:
Sounds like this could be addressed with a lintian warning and a long transition time (~2 years) so leaving proper redirects around would be advisable.
Sure, but my point was that people will use the most easily discoverable pattern available, which currently is the download link from the package's PyPI page. If we want to make some other link more commonly used, it should be more easily discoverable. -Barry
On Oct 12, 2013, at 5:42 PM, Barry Warsaw <barry@python.org> wrote:
On Oct 12, 2013, at 08:50 PM, Floris Bruynooghe wrote:
Sounds like this could be addressed with a lintian warning and a long transition time (~2 years) so leaving proper redirects around would be advisable.
Sure, but my point was that people will use the most easily discoverable pattern available, which currently is the download link from the package's PyPI page. If we want to make some other link more commonly used, it should be more easily discoverable.
-Barry
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Just to be clear, I don't fault folks for using the /packages/ urls. I was just trying to get some sort of idea if anyone actually used that url structure or not. I also don't plan on breaking anything for people who do. That being said, do you know if the Debian case requires something to be served at /packages/source/D/Django/ in order to locate a file at /packages/source/D/Django/Django-*.tar.gz? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Oct 12, 2013, at 06:15 PM, Donald Stufft wrote:
Just to be clear, I don't fault folks for using the /packages/ urls. I was just trying to get some sort of idea if anyone actually used that url structure or not. I also don't plan on breaking anything for people who do.
That being said, do you know if the Debian case requires something to be served at /packages/source/D/Django/ in order to locate a file at /packages/source/D/Django/Django-*.tar.gz?
It does. uscan(1) is the tool that parses and acts on debian/watch files. My Perl-fu is pretty rusty so I'm not sure I'm reading it correctly, but I think it's evident what the tool does (modulo tons of corner cases that mostly aren't relevant for PyPI urls). Here for example is tox's debian/watch file: -----snip snip----- version=3 http://pypi.python.org/packages/source/t/tox/tox-(.*).tar.gz -----snip snip----- And verbose output: % uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: http://pypi.python.org/packages/source/t/tox/tox-(.*).tar.gz -- Found the following matching hrefs: tox-0.5.tar.gz (0.5) tox-0.5.tar.gz (0.5) tox-0.6.tar.gz (0.6) tox-0.6.tar.gz (0.6) tox-0.7.tar.gz (0.7) tox-0.7.tar.gz (0.7) tox-0.8.tar.gz (0.8) tox-0.8.tar.gz (0.8) tox-1.4.3.tar.gz (1.4.3) tox-1.4.3.tar.gz (1.4.3) tox-1.5.0.tar.gz (1.5.0) tox-1.5.0.tar.gz (1.5.0) tox-1.6.0.tar.gz (1.6.0) tox-1.6.0.tar.gz (1.6.0) tox-1.6.1.tar.gz (1.6.1) tox-1.6.1.tar.gz (1.6.1) Newest version on remote site is 1.6.1, local version is 1.6.0 => Newer version available from http://pypi.python.org/packages/source/t/tox/tox-1.6.1.tar.gz -- Downloading updated package tox-1.6.1.tar.gz -- Successfully downloaded updated package tox-1.6.1.tar.gz and symlinked tox_1.6.1.orig.tar.gz to it -- Scan finished And a directory listing of the URL leading up to the pattern (edited for readability): Index of /raw-packages/source/t/tox/ ../ tox-0.5.tar.gz 12-Jul-2010 11:30 40853 tox-0.6.tar.gz 12-Jul-2010 18:21 41424 tox-0.7.tar.gz 14-Jul-2010 20:48 45934 tox-0.8.tar.gz 30-Jul-2010 23:41 41828 tox-0.9.zip 25-Nov-2010 20:05 57001 tox-1.0.zip 28-May-2011 14:59 61367 tox-1.1.zip 09-Jul-2011 10:09 99297 tox-1.2.zip 10-Nov-2011 18:40 66284 tox-1.3.zip 21-Dec-2011 08:40 67862 tox-1.4.1.zip 03-Jul-2012 09:34 76920 tox-1.4.2.zip 20-Jul-2012 11:05 77565 tox-1.4.3.tar.gz 01-Mar-2013 09:44 71661 tox-1.4.zip 13-Jun-2012 15:01 76147 tox-1.5.0.tar.gz 22-Jun-2013 13:05 73866 tox-1.6.0.tar.gz 15-Aug-2013 13:39 130734 tox-1.6.1.tar.gz 04-Sep-2013 14:17 131654 So yes there needs to be some url that uscan can get a list of to determine the highest available version matching a wildcarded pattern. Cheers, -Barry
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/08/2013 09:19 AM, Donald Stufft wrote:
I was wondering if anyone was aware of anything that relies on the current structure of package urls, namely:
/packages/source|any|etc/D/Django-1.0.tar.gz?
I would like to change this but before I work up a concrete plan for people to comment on and discuss I'm trying to figure out what, if anything, depends on that current structure.
Any such change should leave behind "rewrite rules" which 301 to the new, "blest" URLs. #dontbreaktheweb and all that. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJVpfAACgkQ+gerLs4ltQ6rhwCffZISAc+VDiqX5NQ/3ET9inJp daAAmgNsHb/0vIVjSTMLrxEGT9S4yodT =2TOU -----END PGP SIGNATURE-----
participants (7)
-
Barry Warsaw -
Brian Sutherland -
Donald Stufft -
Floris Bruynooghe -
holger krekel -
Marius Gedminas -
Tres Seaver