<div><span style="color: rgb(160, 160, 168); ">On Friday, March 1, 2013 at 2:31 PM, M.-A. Lemburg wrote:</span></div>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;">
<span><div><div><div>On 01.03.2013 12:17, holger krekel wrote:</div><blockquote type="cite"><div><div>On Fri, Mar 01, 2013 at 06:09 -0500, Donald Stufft wrote:</div><blockquote type="cite"><div><div>On Friday, March 1, 2013 at 6:04 AM, M.-A. Lemburg wrote:</div><blockquote type="cite"><div><div>On 01.03.2013 11:19, holger krekel wrote:</div><blockquote type="cite"><div><div>Hi Richard, all,</div><div><br></div><div>somewhere deep in the threads i mentioned i wrote a little "cleanpypi.py"</div><div>script which takes a project name as an argument and then goes to </div><div>pypi.python.org (<a href="http://pypi.python.org">http://pypi.python.org</a>) and removes all homepage/download metadata entries for </div><div>this project. This sanitizes/speeds up installation because</div><div>pip/easy_install don't need to crawl them anymore. I just did this for</div><div>three of my projects, (pytest, tox and py) and it seems to work fine.</div></div></blockquote><div><br></div><div><br></div><div>Does it also cleanup the links that PyPI adds to the /simple/ by</div><div>parsing the project description for links ?</div><div><br></div><div>I think those are far nastier than the homepage and download links,</div><div>which can be put to some good use to limit the external lookups</div><div>(see <a href="http://wiki.python.org/moin/PyPI/DownloadMetaDataProposal">http://wiki.python.org/moin/PyPI/DownloadMetaDataProposal</a>)</div><div><br></div><div>See e.g. <a href="https://pypi.python.org/simple/zc.buildout/">https://pypi.python.org/simple/zc.buildout/</a></div><div>for a good example of the mess this generates... even mailto links</div><div>get listed and "file:///" links open up the installers for all</div><div>kinds of nasty things (unless they explicitly protect against</div><div>following these).</div></div></blockquote><div><br></div><div>pip at least, and I assume the other tools don't spider those links, but</div><div>they do consider them for download (e.g. if the link looks installable</div><div>it will be a candidate for installing, but it won't fetch it, and look for </div><div>more links like it will donwnload_url/home_page).</div><div><br></div><div>I believe that's the way it's structured atm.</div></div></blockquote><div><br></div><div>That's right. Even though the long-description extracted links </div><div>look ugly on a simple/PKGNAME page, neither pip nor easy_install do anything</div><div>with them except if the "href" ends in "#egg=PKGNAME-" in which case they are</div><div>taken as pointing to a development tarball (e.g. at github or bitbucket).</div><div>ASFAIK a link like "PKGNAME-VER.tar.gz" will not be treated as</div><div>an installation candidate, just the "#egg=PKGNAME" one.</div></div></blockquote><div><br></div><div>Hmm, then why not remove links that don't match the above from</div><div>the /simple/ index pages ?</div><div><br></div><div>Note that it's easily possible to make e.g. file:/// links</div><div>have a fragment that matches what you described, so I guess the</div><div>filters would have to be more careful about what to allow</div><div>(e.g. only http/ftp schemes, perhaps even only https schemes)</div><div>and what not.</div><div><br></div><div>BTW: Are those links also shown as-is on the description page ?</div><div>People could do nasty stuff by adding "javascript:" links which look</div><div>like normal links to the descriptions.</div></div></div></span></blockquote><div>The descriptions don't allow javascript: urls anymore (I reported that</div><div>ages ago and Richard fixed it). home_page and probably download_url</div><div>do though.</div><blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;"><span><div><div><div><br></div><div>-- </div><div>Marc-Andre Lemburg</div><div><a href="http://eGenix.com">eGenix.com</a></div><div><br></div><div>Professional Python Services directly from the Source (#1, Mar 01 2013)</div><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><div><div>Python Projects, Consulting and Support ... <a href="http://www.egenix.com">http://www.egenix.com</a>/</div><div>mxODBC.Zope/Plone.Database.Adapter ... <a href="http://zope.egenix.com">http://zope.egenix.com</a>/</div><div>mxODBC, mxDateTime, mxTextTools ... <a href="http://python.egenix.com">http://python.egenix.com</a>/</div></div></blockquote></blockquote></blockquote><div>________________________________________________________________________</div><div><br></div><div>::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::</div><div><br></div><div> <a href="http://eGenix.com">eGenix.com</a> Software, Skills and Services GmbH Pastor-Loeh-Str.48</div><div> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg</div><div> Registered at Amtsgericht Duesseldorf: HRB 46611</div><div> <a href="http://www.egenix.com/company/contact/">http://www.egenix.com/company/contact/</a></div></div></div></span>
</blockquote>
<div>
<br>
</div>