[Catalog-sig] parsing setuptools style requires.txt
djay at pretaweb.com
Tue Sep 13 03:53:37 CEST 2011
On 12/09/2011, at 11:01 PM, Tarek Ziadé wrote:
> On Mon, Sep 12, 2011 at 10:43 AM, Dylan Jay <djay at pretaweb.com> wrote:
>> then why not allow a change to parsing of setuptools style
>> or perhaps point me to the discussion that explains what I'm missing.
>> Its also been mentioned that in order to make this parsing work you
>> need to
>> run setup.py to get the requires.txt for setuptools packages and
>> this is a
>> security concern. However many packages already have the egg-info
>> run before upload so there is no need to run setup.py. For those
>> where there is a need I think security concerns could be overcome
>> with the
>> use of the restrictedpython package. Anything trying to import
>> anything but
>> the bare minimum is skipped.
> the egg-info produced by setuptools is dependent on the platform it's
> run on, so running on the target client is what you need to get the
> right values for sure.
true. but I'd guess that 99% of eggs have the same set of requirements
regardless of platform.
And if we had someway to label these requirements on pypi as "dodgy-
setuptools" and should be used only as a hint rather the exact set of
requirements for your particular platform?
> with PEP 345-style metadata, you have ways to define static
> per-platform metadata.
> In fact, one of the goal is to have setup.cfg published at pypi
>> My interest in this is the idea that we could get distutils2 and/or
>> zc.buildout to be able to download regular updates of metadata
>> dependencies, then perhaps those tools could avoid certain kinds of
>> errors which are a pain to debug without that information. For
>> instance, the
>> current design of zc.buildout means that:
>> Download Bob. Bob 1.0 requires Fred >= 2.0.
>> Download Fred 3.0
>> Download Marry. Marry 1.0 requires Fred < 2.5
>> Conflict error. Marry 1.0 requires Fred < 2.5 but we already have
>> Fred 3.0.
>> If instead we knew in advance of this conflict we could have chosen
>> download Fred 2.4 or at least warned the user there was a potential
>> and they should pick a compatible version. In the case of
>> packages, it could offer to downgrade Fred from 3.0 to 2.4.
> I think Pip solves this by downloading all dependencies beforehand, no
> ? and it works for all platforms.
yes it can be solved by downloading all dependencies but you'd have to
download all versions of all dependencies potentially to avoid a
conflict in the worst case.
> The bottom line is that spending efforts on adding on server side
> platform-specific metadata that are not guaranteed to be accurate for
> your target platform, is not worth it since you can already do
> something accurate on client side at this time.
> PEP 345 will provide an improvement in the long term since you will be
> able to read dependencies, but also the file list, without having to
> download the package.
> IOW, installers will speed up because they will be able to build
> dependency graphs on the fly -- as long as all packages are PEP 345,
> which won't happen before a while.
can't wait, but in the meantime I can't see why installers can't use
these hints to speed things up as long as they use it with the caveat
that requirements could potentially differ once actually installed.
> Tarek Ziadé | http://ziade.org
More information about the Catalog-SIG