[Catalog-sig] parsing setuptools style requires.txt

Dylan Jay djay at pretaweb.com
Mon Sep 12 10:43:11 CEST 2011


Hi,

A recent request[1] (which I didn't submit) was rejected with the  
following comment:

"Assuming you are talking about setuptools-style dependencies: this  
has been
rejected by the community, as hurting PEP 345. PEP-345-style  
dependencies
are already parsed and provided. If you want to discuss further, post to
catalog-sig."

I had a search of the archives and couldn't locate the discussion,  
unless it's this one [2] which seemed to indicate that there was a  
suitable way to publish both pep345 compatible requirements as well as  
PIP and setuptools requirements via PYPI.

It strikes me that
1. if someone is prepared to write a patch to pypi to handle  
setuptools style requirements
2. if there is a lot of packages out there and will continue to be for  
a long time, that have setuptools style
   dependency specification
3. if PEP345 isn't implemented in any tools yet (excuse my ignorance.  
I'm assuming PEP345 tool support is in distutils2 and that isn't  
finished yet)

then why not allow a change to parsing of setuptools style dependencies?
or perhaps point me to the discussion that explains what I'm missing.

Its also been mentioned that in order to make this parsing work you  
need to run setup.py to get the requires.txt for setuptools packages  
and this is a security concern. However many packages already have the  
egg-info commend run before upload so there is no need to run  
setup.py. For those packages where there is a need I think security  
concerns could be overcome with the use of the restrictedpython  
package. Anything trying to import anything but the bare minimum is  
skipped.

My interest in this is the idea that we could get distutils2 and/or  
zc.buildout to be able to download regular updates of metadata  
including dependencies, then perhaps those tools could avoid certain  
kinds of conflict errors which are a pain to debug without that  
information. For instance, the current design of zc.buildout means that:

Download Bob. Bob 1.0 requires Fred >= 2.0.
Download Fred 3.0
Download Marry. Marry 1.0 requires Fred < 2.5
Conflict error. Marry 1.0 requires Fred < 2.5 but we already have Fred  
3.0.

If instead we knew in advance of this conflict we could have chosen to  
download Fred 2.4 or at least warned the user there was a potential  
conflict and they should pick a compatible version. In the case of  
preinstalled packages, it could offer to downgrade Fred from 3.0 to 2.4.


[1] http://sourceforge.net/tracker/?func=detail&aid=3407539&group_id=66150&atid=513503
[2] http://mail.python.org/pipermail/catalog-sig/2011-January/003452.html



---
Dylan Jay
Technical Solutions Manager
PretaWeb: reducing duplication in the government web.
P: +612 80819071 | M: +61421477460 | twitter.com/djay75 | linkedin.com/ 
in/djay75



More information about the Catalog-SIG mailing list