[Catalog-sig] parsing setuptools style requires.txt

[Dylan Jay]

On 12/09/2011, at 11:31 PM, Jim Fulton wrote:

> On Mon, Sep 12, 2011 at 4:43 AM, Dylan Jay <djay at pretaweb.com> wrote:
> ...
>> My interest in this is the idea that we could get distutils2 and/or
>> zc.buildout to be able to download regular updates of metadata  
>> including
>> dependencies, then perhaps those tools could avoid certain kinds of  
>> conflict
>> errors which are a pain to debug without that information. For  
>> instance, the
>> current design of zc.buildout means that:
>>
>> Download Bob. Bob 1.0 requires Fred >= 2.0.
>> Download Fred 3.0
>> Download Marry. Marry 1.0 requires Fred < 2.5
>> Conflict error. Marry 1.0 requires Fred < 2.5 but we already have  
>> Fred 3.0.
>
>> If instead we knew in advance of this conflict we could have chosen  
>> to
>> download Fred 2.4 or at least warned the user there was a potential  
>> conflict
>> and they should pick a compatible version. In the case of  
>> preinstalled
>> packages, it could offer to downgrade Fred from 3.0 to 2.4.
>
> The issue here isn't that buildout has to download packages
> to get meta data. The problem is that it doesn't backtrack.  This is
> compounded by fact that when computing and fetching
> requirements, it uses a depth-first strategy and doesn't use
> information about requirements of other packages in the working
> set until too late.  There's definitely room for improvement in the
> strategy it uses, even short of backtracking.

true but to do backtracking properly could result in downloading many  
different available versions of potentially large packages which isn't  
so nice.


>
> In any case, allowing buildout to get requirements info before
> downloading packages would be a useful optimization, but wouldn't
> solve the conflict problem.

yes, backtracking would :) or at least it would solve some of them. Do  
you think it's doable even if there's no pypi api to get requirements?
I think this would be a great feature as conflict errors are very  
frustrating for new users. They also tend come up in buildout more  
than any other tool due to the way to it's normally used and new users  
tend to blame buildout for this.

>
> Jim
>
> -- 
> Jim Fulton
> http://www.linkedin.com/in/jimfulton