[Twisted-Python] Twisted 9.0.0?
Hi everyone! I am a bit puzzled about the way new version of Twisted is released. Or better: the way the old version seems to be taken out of service. Last weeks I've been developing agains version Twisted 9.0.0. Yesterday I wanted to deploy my code to my production server, but to my surprise I could not easy_install that version anymore: $ easy_install Twisted==9.0.0 Searching for Twisted==9.0.0 Reading http://pypi.python.org/simple/Twisted/ Reading http://www.twistedmatrix.com Reading http://twistedmatrix.com/products/download Reading http://twistedmatrix.com/projects/core/ Reading http://twistedmatrix.com/ No local packages or download links found for Twisted==9.0.0 Best match: None (Okay, actually I'm using buildout to setup my environment, but the above demonstrates my problem more clearly.) I'm sure the new version of Twisted is better and improved but I still wonder why version 9.0.0 has effectively been decommissioned. Or am I missing something and is it still possible to easy_install version 9.0.0? Thanks, Mark -- Mark van Lent m.van.lent@zestsoftware.nl
On Thu, Mar 4, 2010 at 10:45 AM, Mark van Lent <m.van.lent@zestsoftware.nl> wrote:
It's an oversight. I don't know how the PyPI announcement was done (although I'd love to!), but I think you can announce new versions without decommissioning old ones. As for the twistedmatrix.com website, I can fix that so that 9.0.0 is easy-installable, but I don't know how. Let me know & I'll get it done. jml
On Mar 4, 2010, at 7:59 AM, Jonathan Lange wrote:
I think it's actually a bug in easy_install, or at least a mis-design. To make something easy_installable, you don't provide structured information in PyPI; there may be a way to do that, but most packages I've seen (Twisted included) just provide a link to a page which then provides a link to a file with a filename of vaguely the appropriate shape. easy_install works by screen-scraping that link. In our case, the "home page", http://twistedmatrix.com, links directly to a Twisted-<version>.tar.bz2, which is then used. If someone (perhaps Mark van Lent!) could point us at how to provide the correctly structured information to easy_install via PyPI so it will _stop_ screen-scraping our website, and just point at the correct files, this problem could be addressed. However, as exarkun says: screen-scraping a plain-text wiki and then executing code out of whatever comes out of it is not the most reliable or secure way to deploy code. You may want to specify the URL directly - you can still use easy_install for that if you want, like this: $ easy_install http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.tar.bz2
I don't know how the PyPI announcement was done (although I'd love to!), but I think you can announce new versions without decommissioning old ones.
I assume you'll write it down on a web page, when you figure it out :). The old version, from PyPI's perspective, wasn't decommissioned. It's here: <http://pypi.python.org/pypi/Twisted/9.0.0>. The problem is that it also just points at twistedmatrix.com, not a version-specific download page, and there is no Twisted-<version>.tar.bz2 which matches "9.0.0" any more. (I believe that another way to fix this, although I would be significantly less happy about it, would be to add some hidden links to twistedmatrix.com that had the right filenames. If this is a real problem for many users though, perhaps we should just live with an ugly fix.)
exarkun@twistedmatrix.com wrote:
What issues? I've never had problems with either the register or upload commands... Having the distributions on PyPI makes the OP's problem go away, and reduces the maintenance burned for you guys who release twisted... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Mar 4, 2010, at 1:44 PM, Chris Withers wrote:
This has been discussed before, on distutils-sig - there's a _giant_ thread, but I think the most relevant message is <http://mail.python.org/pipermail/distutils-sig/2009-December/015064.html>. Now that our release process is better documented (THANK YOU JML), you might be able to have a look at <http://twistedmatrix.com/trac/wiki/ReleaseProcess> and figure out where 'setup.py upload' could fit in without uploading a tarball that is _not_ actually our official release.
Tim Allen wrote:
No, it doesn't... see the release process I use for my own packages here: http://packages.python.org/errorhandler/development.html#making-a-release cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Fri, Mar 05, 2010 at 11:11:35AM +0000, Chris Withers wrote:
Hmm... at the bottom of that page, I see you run (among other things): setup.py sdist upload ...so presumably sdist *is* a prerequisite for upload. Or are you saying that you can store any old file as dist/$PACKAGE-$VERSION.tar.gz and "setup.py upload" will ship it to PyPI?
Tim Allen wrote:
I think that's true, but I admit, I haven't checked myself ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Glyph Lefkowitz wrote:
I think that easy_install and related tools can be directed to the proper download location by providing a Download-URL on the PyPI page. E.g. the page for Twisted 9.0.0: http://pypi.python.org/pypi/Twisted/9.0.0 should have a Download-URL link to: http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/ Regards, Ziga
exarkun@twistedmatrix.com wrote:
It looks like it works now: I:\> easy_install Twisted==9.0.0 Searching for Twisted==9.0.0 Reading http://pypi.python.org/simple/Twisted/ Reading http://www.twistedmatrix.com Reading http://twistedmatrix.com/products/download Reading http://twistedmatrix.com/projects/core/ Reading http://twistedmatrix.com/ Reading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/ Best match: Twisted 9.0.0 Downloading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.win32-py2.5.exe Now, how to get rid of the egg yolk... Regards, Ziga
On 3/4/10 21:21 , exarkun@twistedmatrix.com wrote:
Thanks Jean-Paul and Ziga! It indeed works like a charm! I have to admit that, for someone who uses PyPi daily, I know too little about it. I probably would have started tinkering with my zc.buildout configuration to get it working, but this is a much better solution. Thanks again, Mark
Jonathan Lange wrote:
PyPI already does the right thing w.r.t. "hide previous versions" (either way works, it doesn't affect tools like easy_install) but ONLY if you put your distros on PyPI... ...obviously PyPI has not knowledge of your website and so can't do anything about it. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On 10:45 am, m.van.lent@zestsoftware.nl wrote:
Decommissioned? Not really. There is a link to older releases at the top of <http://twistedmatrix.com/trac/wiki/Downloads>. You can find releases going all the way back to 2.0 there. Perhaps easy_install isn't the best way to manage deployments to your production server? Jean-Paul
exarkun@twistedmatrix.com wrote:
Perhaps easy_install isn't the best way to manage deployments to your production server?
No, but buildout is... However, this process of discovering packages (hit pypi, scrape web pags if the distro isn't on pypi) is the only solution the python community (be it pip, easy_install, virtualenv or buildout) is using to download packages... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Thu, Mar 4, 2010 at 10:45 AM, Mark van Lent <m.van.lent@zestsoftware.nl> wrote:
It's an oversight. I don't know how the PyPI announcement was done (although I'd love to!), but I think you can announce new versions without decommissioning old ones. As for the twistedmatrix.com website, I can fix that so that 9.0.0 is easy-installable, but I don't know how. Let me know & I'll get it done. jml
On Mar 4, 2010, at 7:59 AM, Jonathan Lange wrote:
I think it's actually a bug in easy_install, or at least a mis-design. To make something easy_installable, you don't provide structured information in PyPI; there may be a way to do that, but most packages I've seen (Twisted included) just provide a link to a page which then provides a link to a file with a filename of vaguely the appropriate shape. easy_install works by screen-scraping that link. In our case, the "home page", http://twistedmatrix.com, links directly to a Twisted-<version>.tar.bz2, which is then used. If someone (perhaps Mark van Lent!) could point us at how to provide the correctly structured information to easy_install via PyPI so it will _stop_ screen-scraping our website, and just point at the correct files, this problem could be addressed. However, as exarkun says: screen-scraping a plain-text wiki and then executing code out of whatever comes out of it is not the most reliable or secure way to deploy code. You may want to specify the URL directly - you can still use easy_install for that if you want, like this: $ easy_install http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.tar.bz2
I don't know how the PyPI announcement was done (although I'd love to!), but I think you can announce new versions without decommissioning old ones.
I assume you'll write it down on a web page, when you figure it out :). The old version, from PyPI's perspective, wasn't decommissioned. It's here: <http://pypi.python.org/pypi/Twisted/9.0.0>. The problem is that it also just points at twistedmatrix.com, not a version-specific download page, and there is no Twisted-<version>.tar.bz2 which matches "9.0.0" any more. (I believe that another way to fix this, although I would be significantly less happy about it, would be to add some hidden links to twistedmatrix.com that had the right filenames. If this is a real problem for many users though, perhaps we should just live with an ugly fix.)
exarkun@twistedmatrix.com wrote:
What issues? I've never had problems with either the register or upload commands... Having the distributions on PyPI makes the OP's problem go away, and reduces the maintenance burned for you guys who release twisted... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Mar 4, 2010, at 1:44 PM, Chris Withers wrote:
This has been discussed before, on distutils-sig - there's a _giant_ thread, but I think the most relevant message is <http://mail.python.org/pipermail/distutils-sig/2009-December/015064.html>. Now that our release process is better documented (THANK YOU JML), you might be able to have a look at <http://twistedmatrix.com/trac/wiki/ReleaseProcess> and figure out where 'setup.py upload' could fit in without uploading a tarball that is _not_ actually our official release.
Tim Allen wrote:
No, it doesn't... see the release process I use for my own packages here: http://packages.python.org/errorhandler/development.html#making-a-release cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Fri, Mar 05, 2010 at 11:11:35AM +0000, Chris Withers wrote:
Hmm... at the bottom of that page, I see you run (among other things): setup.py sdist upload ...so presumably sdist *is* a prerequisite for upload. Or are you saying that you can store any old file as dist/$PACKAGE-$VERSION.tar.gz and "setup.py upload" will ship it to PyPI?
Tim Allen wrote:
I think that's true, but I admit, I haven't checked myself ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Glyph Lefkowitz wrote:
I think that easy_install and related tools can be directed to the proper download location by providing a Download-URL on the PyPI page. E.g. the page for Twisted 9.0.0: http://pypi.python.org/pypi/Twisted/9.0.0 should have a Download-URL link to: http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/ Regards, Ziga
exarkun@twistedmatrix.com wrote:
It looks like it works now: I:\> easy_install Twisted==9.0.0 Searching for Twisted==9.0.0 Reading http://pypi.python.org/simple/Twisted/ Reading http://www.twistedmatrix.com Reading http://twistedmatrix.com/products/download Reading http://twistedmatrix.com/projects/core/ Reading http://twistedmatrix.com/ Reading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/ Best match: Twisted 9.0.0 Downloading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.win32-py2.5.exe Now, how to get rid of the egg yolk... Regards, Ziga
On 3/4/10 21:21 , exarkun@twistedmatrix.com wrote:
Thanks Jean-Paul and Ziga! It indeed works like a charm! I have to admit that, for someone who uses PyPi daily, I know too little about it. I probably would have started tinkering with my zc.buildout configuration to get it working, but this is a much better solution. Thanks again, Mark
Jonathan Lange wrote:
PyPI already does the right thing w.r.t. "hide previous versions" (either way works, it doesn't affect tools like easy_install) but ONLY if you put your distros on PyPI... ...obviously PyPI has not knowledge of your website and so can't do anything about it. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On 10:45 am, m.van.lent@zestsoftware.nl wrote:
Decommissioned? Not really. There is a link to older releases at the top of <http://twistedmatrix.com/trac/wiki/Downloads>. You can find releases going all the way back to 2.0 there. Perhaps easy_install isn't the best way to manage deployments to your production server? Jean-Paul
exarkun@twistedmatrix.com wrote:
Perhaps easy_install isn't the best way to manage deployments to your production server?
No, but buildout is... However, this process of discovering packages (hit pypi, scrape web pags if the distro isn't on pypi) is the only solution the python community (be it pip, easy_install, virtualenv or buildout) is using to download packages... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
participants (8)
-
Chris Withers -
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
Jonathan Lange -
Kevin Horn -
Mark van Lent -
Tim Allen -
Žiga Seilnacht