2009/9/11 :

On 02:33 pm, ziade.tarek@gmail.com wrote:

...

Hello

here's a detailed proposal for the setup.cfg format. I've included all remarks, and tried to make the syntax as simple as possible. I've dropped the long_description stuff and the template language stuff. (people can do it as a post-release process)

A/ Config file structure:

""" [setup] foo: bar conditional-sections: one, two

[one] condition: test

[two] condition: test """

The idea here is that the [setup] section includes everything in section [one] if all of the "condition" clauses in [one] are satisfied, and includes everything in section [two] if all of the "condition" clauses in [two] are satisfied?

yes, here's a concrete example : """ [setup] name: distutils version: 1.2 conditional-sections: py26, py3 [py26] condition: python_version <= 2.6 install_requires = lxml [py3] condition: python_version >= 3 install_requires = foo """

...

2/ Execution environment:

 python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])  os_name = os.name  platform = sys.platform  uname = os.uname()  python_version_info = sys.version_info

3/ Grammar:

comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not' comparison: expr (comp_op expr)* expr: 'python_version'|'python_version_info'|'os_name'|'uname'|'platform'|STRING test: or_test or_test: and_test ('or' and_test)* and_test: not_test ('and' not_test)* not_test: 'not' not_test | comparison

What's "STRING"?  What's the precedence of and, or, and not?  What does this mean?

STRING is any string value that will be compared to one of the values provided in the environment, Also, I've just realized that uname() is a tuple, so I need to flatten it in strings to restrict the grammar: - os_sysname - os_nodename - os_release - os_version - os_machine

  os_name > uname > 7

What does "is" mean?

That can be dropped since we have '==' and it's all strings. so: comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'

Is this all really Python, with lots of features thrown out?

I copy-pasted part of Python grammar and restricted to what we need yes, but the idea is just to have string comparisons with ands and ors

...

4/ Process

distutils will interpret [setup] and act upon, to fill the Distribution object and its metadata attribute object. It will look only for Metadata fields and ignores the others.

In case an option is passed in setup.py, it overrides the one in setup.cfg.

Let me know if it's good enough. I won't write a PEP for this change, and I will just push it for information at python-dev as soon as I get enough +1 :)

I'm a little skeptical about creating a new mini language (particularly one with branching) for setup.cfg, but I haven't really been paying close attention to this discussion.  Still, it might be good to give a brief summary of the justification for this someplace.  Maybe you were already planning to do that.

The idea is to be able to get with setup.cfg all the metadata without running any code from the distribution, and without installing it. Which is possible for most cases. (if not possible, setup.py can still be used) the mini-language is to be able to adapt these metadata depending on the execution context. I'll add a doc somewhere. Tarek

2009/9/11 :

On 02:33 pm, ziade.tarek@gmail.com wrote:

...

Hello

here's a detailed proposal for the setup.cfg format. I've included all remarks, and tried to make the syntax as simple as possible. I've dropped the long_description stuff and the template language stuff. (people can do it as a post-release process)

A/ Config file structure:

""" [setup] foo: bar conditional-sections: one, two

[one] condition: test

[two] condition: test """

The idea here is that the [setup] section includes everything in section [one] if all of the "condition" clauses in [one] are satisfied, and includes everything in section [two] if all of the "condition" clauses in [two] are satisfied?

yes, here's a concrete example : """ [setup] name: distutils version: 1.2 conditional-sections: py26, py3 [py26] condition: python_version <= 2.6 install_requires = lxml [py3] condition: python_version >= 3 install_requires = foo """

...

2/ Execution environment:

 python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])  os_name = os.name  platform = sys.platform  uname = os.uname()  python_version_info = sys.version_info

3/ Grammar:

comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not' comparison: expr (comp_op expr)* expr: 'python_version'|'python_version_info'|'os_name'|'uname'|'platform'|STRING test: or_test or_test: and_test ('or' and_test)* and_test: not_test ('and' not_test)* not_test: 'not' not_test | comparison

What's "STRING"?  What's the precedence of and, or, and not?  What does this mean?

STRING is any string value that will be compared to one of the values provided in the environment, Also, I've just realized that uname() is a tuple, so I need to flatten it in strings to restrict the grammar: - os_sysname - os_nodename - os_release - os_version - os_machine

  os_name > uname > 7

What does "is" mean?

That can be dropped since we have '==' and it's all strings. so: comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'

Is this all really Python, with lots of features thrown out?

I copy-pasted part of Python grammar and restricted to what we need yes, but the idea is just to have string comparisons with ands and ors

...

4/ Process

distutils will interpret [setup] and act upon, to fill the Distribution object and its metadata attribute object. It will look only for Metadata fields and ignores the others.

In case an option is passed in setup.py, it overrides the one in setup.cfg.

Let me know if it's good enough. I won't write a PEP for this change, and I will just push it for information at python-dev as soon as I get enough +1 :)

I'm a little skeptical about creating a new mini language (particularly one with branching) for setup.cfg, but I haven't really been paying close attention to this discussion.  Still, it might be good to give a brief summary of the justification for this someplace.  Maybe you were already planning to do that.

The idea is to be able to get with setup.cfg all the metadata without running any code from the distribution, and without installing it. Which is possible for most cases. (if not possible, setup.py can still be used) the mini-language is to be able to adapt these metadata depending on the execution context. I'll add a doc somewhere. Tarek

On Fri, Sep 11, 2009 at 5:36 PM, Jim Fulton wrote:

On Fri, Sep 11, 2009 at 11:14 AM, Tarek Ziadé wrote:

...

2009/9/11  :

...

On 02:33 pm, ziade.tarek@gmail.com wrote: I'm a little skeptical about creating a new mini language (particularly one with branching) for setup.cfg,

Me too.

...

...

but I haven't really been paying close attention to this discussion.

Me neither. :)

...

...

 Still, it might be good to give a brief summary of the justification for this someplace.  Maybe you were already planning to do that.

The idea is to be able to get with setup.cfg all the metadata without running any code from the distribution, and without installing it.

Which is possible for most cases. (if not possible, setup.py  can still be used)

the mini-language is to be able to adapt these metadata depending on the execution context. I'll add a doc somewhere.

Is that really necessary?  I mean, if you have to adapt the meta data to the execution context, maybe you should fall back to writing a setup.py.

The main background motivation is the future adding of "install_requires" in PEP 345. So you can for example define a list of dependencies that changes wether you are under windows or linux, etc.. The second motivation is : why do I need to run some code to get the metadata fields ? do I need to install a package to gets its metadata ? In 99% of the cases that would'nt be necessary. try to run this with lxml source tarball: $ python setup.py --name I don't know what it is doing, but a whole lot more that what it is suppose to do just to let me know the 'name' metadata. So if name was in setup.cfg, it would be better.

I agree that more background and motivational information would be very helpful, as would some real world examples. I know that >90% (maybe 99% :) of the packages I distribute don't adapt to their execution context (beyond what distutils does internally) or need anything but meta data.

That's why you could push your fields in the static setup.cfg. (and probably have an 'empty' setup.py file) I will prepare a document summarizing. Maybe a PEP would be required then if it's not an obvious change. Tarek

On Fri, Sep 11, 2009 at 7:40 PM, Jim Fulton wrote:

...

The second motivation is : why do I need to run some code to get the metadata fields ?

Turning the config file into code means you'd be running code. :)

Well that's very different. You will be running a function that is provided in the stdlib, in a vanilla Python, and that will be restricted to string comparisons basically. In other words, it is more likely to be a translation of setup.cfg into a "localized" setup.cfg.

...

do I need to install a package to gets its metadata ?

This is an important use case.  It is orthogonal to defining packages with static meta data.

Sure. but both can exist. although It would require a flag in setup.cfg indicating that it is not standalone. e.g. doesn't provides all metadata without requiring to execute some code in the distribution. If the flag is not present, you know you can work with setup.cfg alone.

Somewhat. I think the main concern though is to get the meta data without installing.

Well, without running the setup.py code at the minimum, to extract the metadata if they are platform-dependant. Which can start to happen more and more the day we add the famous "requires" in the metadata. For instance, how can I say that I need simplejson only if it's Python 2.5 or above ? People will have to download the package and run something to know it, where they could just read the setup.cfg file. The simplejson dependency will appear at PyPI *only* if the person that ran the register command used python 2.5 when he did... that's a problem indeed. Security-wise, if the micro-language is restricted to string comparisons, with a restricted list of variables, people and tool will be able to query setup.cfg with no risk.

In any case, I think a sufficiently large percentage of cases can be handled without logic in the setup.cfg.

yes

...

...

I agree that more background and motivational information would be very helpful, as would some real world examples. I know that >90% (maybe 99% :) of the packages I distribute don't adapt to their execution context (beyond what distutils does internally) or need anything but meta data.

That's why you could push your fields in the static setup.cfg.

(and probably have an 'empty' setup.py file)

Yup. Don't get me wrong. I'm in favor of being able to define packages with static meta data. I just don't consider configuration files with logic in them to be static. :)

Let's call it differently then, because if we want static metadata, we will have to provide that logic for the 1% that needs it. In any case, is such a feature is a problem for the 99% left ? You could still add your metadata in the [setup] section and never use that feature.

On Mon, Sep 14, 2009 at 4:13 PM, Glyph Lefkowitz wrote:

I may be missing some context, but I did about an hour of googling around before responding here, and for the life of me I can't find a simple answer to the question: how do I currently programatically get the metadata out of a distribution?

You can't do it with Distutils to get metadata of installed distributions. If you have the source, you can just do : $ python setup.py --field (where field is the metadata - PEP 314) This need you are describing is exactly the purpose of PEP 376 in fact.

The answer seems to be a bizarre combination of running setup.py, maybe reading PKG-INFO, maybe something to do with egg-info...?  Or maybe making some direct calls with xmlrpclib?  Or setuptools?  Setuptools must be able to do it internally, but I can't find an API in the documentation. Before trying to make the metadata static, let's try to have the metadata, for real, at all.  Here's an example of something I want to be able to do with metadata:

from distutils.something import (     load_metadata, pypi_metadata, installed_metadata) meta = installed_metadata("Twisted") meta2 = load_metadata("/home/glyph/Downloads/Twisted-x.y") meta3 = pypi_metadata("Twisted") if meta2.version > meta.version:     print 'More recent version of Twisted already downloaded!' if meta3.version > meta2.version:     print 'More recent version of Twisted is available online!'

Important points here:

I can get the same metadata, in the same format, from PyPI, from a directory containing the distribution, and from my installed system. There are supported functions that abstract this stuff away for me.  Working with the packages, I don't have to know how the metadata is actually declared. There is a single data structure, hopefully with some API documentation I can look at.

Now, maybe the data structure that I want already exists and I just don't know about it (is it a distutils Distribution, maybe?) but it's certainly hard to get a handle on one.

Basically, what you are describing is PKG-INFO, the file that contains all metadata, and that's what PyPI sends you back through XLM RPC. And the static setup.cfg [setup] section ressembles a lot at the final PKG-INGO the Distribution class creates. For installed packages, setuptools does provides a similar feature through it's pkg_resources module, which knows how to look for the PKG-INFO file of a given distribution in your system (several format exists, it can be an egg-info file, or a PKG-INFO file inside an egg info directory. PEP 376 tries to put a standard on that) So yes, your example would be what we would want, and if we add the static setup.cfg thing in distutils, an API would be able to send you back the metadata as they wil be in the PKG-INFO

In order to be useful, the code that loads this metadata is going to need to execute setup.py sometimes anyway.

Not sure to understand, when installing you mean ?

Tarek, as you said in your blog post, "there will always be edge cases".  So it doesn't seem too useful to be focusing on getting rid of that requirement right away.  And when you download from PyPI, you probably eventually have to execute a bunch of untrusted code that somebody uploaded to a wiki, as root.  Again it would be nice to fix that, but we have a long way to go :). For what it's worth, on the implementation side of things, I think that the library code on the client side should receive enough metadata that it can do the job of applying the execution environment locally.  I don't necessarily want to send pypi a bunch of information about my machine just to figure out what distutils would tell me if I tried to install something.

Right, that's a good point the execution could be on client-side, that wouldn't matter. I kinda like the zero download principle though, to query PyPI on distributions that fits your target system.

If there were such an API then setup.cfg would simply be a small implementation detail.  The problem with lxml will still need to be addressed, by promoting awareness of the fact that your setup.py may be executed to do things other than install or submit your package.  Before adding conditional support to setup.cfg I would want to have a separate 'metadata.py' whose responsibilities were clearer: just produce the metadata, don't do any pre-build steps.  Ideally that would be easy to load correctly from setup.py as well.

That wouldn't be different from setup.py I guess, because as long as you let people write a Python module they can do whatever they want. Now if we say "metadata.py" is: - a module that returns an instance of the Distribution class, that we use in setup.py - shall not have any dependency except vanilla python - executed whith a restricted list of paths (basically, just Python's lib) That would be better than the current situation. But in any case, this is not opposed to what we've proposed for setup.cfg, it could be combined imho. For clarity, maybe the [setup] section should be replaced by a plain PKG-INFO file alongside setup.py and setup.cfg, that is loaded by the Distribution class with the micro-language still present. So we're "PKG-INFO"-ing at every stage (pypi, installed package, source directory) Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

On Mon, 14 Sep 2009 07:13:55 -0700, Glyph Lefkowitz wrote:

...

So you can for example define a list of dependencies that changes wether

...

you are under windows or linux, etc..

OK. I've never had that situation. I'm sure someone has, but it doesn't see common.

Twisted actually has this. On Windows, we need pywin32 to implement many of the features which can be implemented with just the stdlib on Posix. But this is the only thing I can imagine computing based on the execution environment.

I believe virtualenv requires pywin32. Maybe several other Python packages too.

I may be missing some context, but I did about an hour of googling around before responding here, and for the life of me I can't find a simple answer to the question: how do I currently programatically get the metadata out of a distribution?

http://pypi.python.org/pypi/pkginfo/

[...] Before adding conditional support to setup.cfg I would want to have a separate 'metadata.py' whose responsibilities were clearer: just produce the metadata, don't do any pre-build steps. Ideally that would be easy to load correctly from setup.py as well.

I double users would follow this proposed convention. Once in a while, I stumble upon Python packages that do not even succeed at the "python setup.py" stage. Even if they did, what prevents them from making metadata.py a little more sophisticated so as to introduce inadvertent bugs on other platforms? (Just today, argparse's new release broke on linux and mac, but not on Windows .. due to regex parsing behavior) I vote for static metadata anyday.

Inspecting metadata could also be made easier by having an API more like this for setup.py:

d = Distribution(... metadata ....) if d.installing: my_project.custom_prepare_for_build() d.setup()

since you can't really tell what setup() is going to do until you've already called it.

I doubt this would happen in practice even if widely suggested by distutils/setuptools.

So finally, if there's still no standard programmatic way to get at the distribution metadata after setup.cfg is modified to have conditional expressions in it, what exactly is the point?

... Conditional metadata for Python versions does not have to be very complicated in my opinion. Simply have the following in setup.cfg, and thus in PKG-INFO: install_requires = ['lxml', 'multiprocessing[pyver<(2,6)]', 'argparse', 'pywin32[platform.name=windows]']

...

...

metadata.get_install_requires(for_pyver=(2,6), platform=Platform(name='windows', arch='x86')) ['lxml', 'argparse', 'pywin32']

I don't imagine fields other than `install_requires` to vary based on the target environment. Simple, no? -srid

Sridhar Ratnakumar wrote:

...

I may be missing some context, but I did about an hour of googling around before responding here, and for the life of me I can't find a simple answer to the question: how do I currently programatically get the metadata out of a distribution?

http://pypi.python.org/pypi/pkginfo/

Yeah, but you have to remember to sacrifice the right goats to end up with the right things in whatever files pkginfo uses :-( Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

Tres Seaver wrote:

You bothering the goats again, Chris? ;)

pkginfo uses the standard information generated by the 'sdist', 'develop' and 'bdist_egg' commenads, and can even introspect installed packages on Python 2.6.x. It is documented here:

http://packages.python.org/pkginfo/

Yep, I know all this, I experienced some weirdness when working on dev packages with the right metadata not being used when I was generating sphinx docs using pkginfo to get version information. I'll post again when/if I can reproduce this easily... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On Sat, Sep 12, 2009 at 02:52:17AM +0200, Tarek Ziadé wrote:

On Fri, Sep 11, 2009 at 5:49 PM, Tarek Ziadé wrote:

...

...

I agree that more background and motivational information would be very helpful, as would some real world examples. I know that >90% (maybe 99% :) of the packages I distribute don't adapt to their execution context (beyond what distutils does internally) or need anything but meta data.

That's a first draft:

http://tarekziade.wordpress.com/2009/09/12/static-metadata-for-distutils/

Your example shows an empty setup() call but does not say which package/module is supposed to be installed. I'm confused now to how much you would like in setup.cfg. Is the idea to leave the package/module/extension module/scripts etc in setup.py? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org

On Fri, Sep 11, 2009 at 05:14:58PM +0200, Tarek Ziadé wrote: [...]

That can be dropped since we have '==' and it's all strings.

'2.6' < '2.10'? Although not close you must be wary of this. Bring on PEP 386 ;-) Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org

On Fri, Sep 11, 2009 at 07:02:53PM +0200, Tarek Ziadé wrote:

On Fri, Sep 11, 2009 at 6:32 PM, Floris Bruynooghe wrote:

...

On Fri, Sep 11, 2009 at 05:14:58PM +0200, Tarek Ziadé wrote: [...]

...

That can be dropped since we have '==' and it's all strings.

'2.6' < '2.10'?

Although not close you must be wary of this.  Bring on PEP 386 ;-)

Right. Maybe we can drop < >, >= and >=, completely.

In real world use cases, we can use "or" to make a section work for 2.6, 3.1, etc.. for python versions.

I can't think of a use case where "in" and "==" is not enough.

Version comparison would be nice though. Taking your example from your blog post: if I where to try to install that on Python 2.4 I'd have to edit the condition of the [py25] section to "in ('2.4', '2.5')". Maybe this is what you want though, in case you explicitly don't support for 2.4. OTOH it's nice if things would just work in case of developers simply not having tried earlier versions. This also shows how "in" implies a tuple or a list as data type, your mini-language was trying to avoid that. Or did I miss something? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org

Floris Bruynooghe wrote:

On Fri, Sep 11, 2009 at 07:02:53PM +0200, Tarek Ziadé wrote:

...

On Fri, Sep 11, 2009 at 6:32 PM, Floris Bruynooghe wrote:

...

On Fri, Sep 11, 2009 at 05:14:58PM +0200, Tarek Ziadé wrote: [...]

...

That can be dropped since we have '==' and it's all strings. '2.6' < '2.10'?

Although not close you must be wary of this. Bring on PEP 386 ;-) Right. Maybe we can drop < >, >= and >=, completely.

In real world use cases, we can use "or" to make a section work for 2.6, 3.1, etc.. for python versions.

I can't think of a use case where "in" and "==" is not enough.

Version comparison would be nice though. Taking your example from your blog post: if I where to try to install that on Python 2.4 I'd have to edit the condition of the [py25] section to "in ('2.4', '2.5')". Maybe this is what you want though, in case you explicitly don't support for 2.4. OTOH it's nice if things would just work in case of developers simply not having tried earlier versions.

This also shows how "in" implies a tuple or a list as data type, your mini-language was trying to avoid that. Or did I miss something?

A lot of these problems would go away if we could just treat "Python" as a package like every other package when it comes to dependencies. Why is that so hard to do? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

David Lyon wrote:

Don't worry Chris, if your idea is good enough, it is going to get forked...

Sorry, I misheard that last word, in which case I agree ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On Fri, Sep 11, 2009 at 05:14:58PM +0200, Tarek Ziadé wrote:

Also, I've just realized that uname() is a tuple, so I need to flatten it in strings to restrict the grammar:

- os_sysname - os_nodename - os_release - os_version - os_machine

When I asked for this I was imagening (wrongly) way more to go in setup.cfg then just the meta-data. To be honest in that case I don't think I've got a use for os.uname() to be available. While nothing is wrong with having it, if no one else thinks they need it maybe it's unnecessary complication? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org

On Tue, 15 Sep 2009 16:46:59 +0100, Floris Bruynooghe wrote:

On Fri, Sep 11, 2009 at 05:14:58PM +0200, Tarek Ziadé wrote:

...

Also, I've just realized that uname() is a tuple, so I need to flatten it in strings to restrict the grammar:

- os_sysname - os_nodename - os_release - os_version - os_machine

When I asked for this I was imagening (wrongly) way more to go in setup.cfg then just the meta-data. To be honest in that case I don't think I've got a use for os.uname() to be available. While nothing is wrong with having it, if no one else thinks they need it maybe it's unnecessary complication?

I think it is an unneccessary complication. And imo the discussion has gone way off track.. The use case isn't abstract. All this discussion is about trying to rewrite two lines of code. -- setup.py -- """ if sys.platform == 'win32': setup.dependencies.add('win32com','162') setup() """ That would be the simplest way to do it in code. It appears as if "if sys.platform == 'win32':" is an evil line of code - that certain people want to go to great lengths to stomp out. It's one line of code for crying out loud... We don't need a mini-language just because we don't like writing it the shortest way... talk about platform bias gone utterly crazy... David

On 12:09 am, ssteinerx@gmail.com wrote:

On Sep 15, 2009, at 7:52 PM, David Lyon wrote:

...

And imo the discussion has gone way off track..

The use case isn't abstract. All this discussion is about trying to rewrite two lines of code.

-- setup.py -- """ if sys.platform == 'win32': setup.dependencies.add('win32com','162')

setup()

"""

That would be the simplest way to do it in code.

It appears as if "if sys.platform == 'win32':" is an evil line of code - that certain people want to go to great lengths to stomp out.

It's one line of code for crying out loud...

We don't need a mini-language just because we don't like writing it the shortest way...

talk about platform bias gone utterly crazy...

Assuming that it really is that simple...

+1 on everything above.

A 1 line change is much better than a 30 message debate, BNF diagram, and DSL for a simple case. If it doesn't work out for some reason, some _real_ reason, debate it then in context, if it ever comes up again.

I don't think it's really this simple. The point is to have static definition of the package information. Having a Python program define this information at runtime does not satisfy this requirement. You can say that the simple sys.platform test is not evil, but how does that lead to a solution allowing PyPI to tell users what the dependencies for a package are on their platform? Still, it may be that if the only use case here is conditionally declaring a dependency if the install target is Windows, a simpler solution might suffice. I won't try to describe such a solution. Instead, I'll ask if that premise seems valid. What use cases do we have? There's the one described above, which lots of people have been talking about. I think there's another one related to target Python version - eg, on Python 2.3, depend on simplejson, but on Python 2.6, don't. What else? Jean-Paul

David Lyon kirjoitti:

On Wed, 16 Sep 2009 00:42:24 -0000, exarkun@twistedmatrix.com wrote:

...

I don't think it's really this simple.

If it isn't, then it should be made to be that simple...

...

The point is to have static definition of the package information.

It would be a huge win for everybody if we could get over this hurdle.

...

Having a Python program define this information at runtime does not satisfy this requirement.

I agree with that. That's why I suggested it in the first place.

...

You can say that the simple sys.platform test is not evil, but how does that lead to a solution allowing PyPI to tell users what the dependencies for a package are on their platform?

I've been ardently campaigning for the static metadata to include platform specific package definition for a number of months. I'll just assume you missed my posts.

Here I'll just redescribe a typical win32/linux situation. On win32 the odbc is built in, but you need win32com to access it. On Linux, you don't have odbc so you need pyodbc.

""" [setup] name = myframework version = 1.1

[dependencies] packages = cherrypy, 1.1 objectmapper, 1.5

[dependencies win32] packages = win32com

[dependencies linux2] packages = pyodbc

"""

It is really very simple.

...

Still, it may be that if the only use case here is conditionally declaring a dependency if the install target is Windows, a simpler solution might suffice. I won't try to describe such a solution. Instead, I'll ask if that premise seems valid.

Yes. The premise is true for both windows and linux. They're similar but different.

...

What use cases do we have? There's the one described above, which lots of people have been talking about. I think there's another one related to target Python version - eg, on Python 2.3, depend on simplejson, but on Python 2.6, don't. What else?

Ok. Not what else, here's the complete list:

1) Additional packages on old python versions . eg need lxml if xml support isn't available. simplejson.

2) Additional packages due to o/s ommission. eg win32com on on windows and pyodbc on linux.

They are the typical use cases.

It's either one or two lines in a config file or one or two lines in the setup.py script..

So it is either

1) """ [dependencies linux2] packages = pyodbc """

or..

2) """ if setup.platform == "linux2": setup.dependencies.append('pyodbc') """

Anyway, the real issue here is philosophy. Eliminating platform specifics vs making it simple for real world use cases.

I'm also pretending that I didn't look in distutils code and see how many "darwin" references there actually are. But it isn't about that.. just being able to conditionally install extra packages as required based on whichever of the three major o/s the user happened to choose.

Make sure you don't forget about Java (which is a "platform" of its own) and .NET.

David

_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

On Wed, Sep 16, 2009 at 2:42 AM, wrote:

What use cases do we have?  There's the one described above, which lots of people have been talking about.  I think there's another one related to target Python version - eg, on Python 2.3, depend on simplejson, but on Python 2.6, don't.  What else?

So if I resume, so far the uses cases are: - the OS given by os.name and sys.platform (linux/mac/windows/riscos/ce/ etc..) - the architecture, given by os.uname() (32/64 bits) - the python version, given sys.version_info Tarek

On Fri, Sep 18, 2009 at 8:13 PM, Sridhar Ratnakumar wrote:

On Tue, 15 Sep 2009 17:42:24 -0700, wrote:

...

What use cases do we have?  There's the one described above, which lots of people have been talking about.  I think there's another one related to target Python version - eg, on Python 2.3, depend on simplejson, but on Python 2.6, don't.  What else?

Today I stumbled on a project called PyGUI that depends on packages based on the platform.

 http://www.cosc.canterbury.ac.nz/greg.ewing/python_gui/

""" MacOSX - requires PyObjC (tested with PyObjC-1.2) Linux - requires pygtk (tested with pygtk-2.14.0 and Gtk+-2.14.7) Windows - requires pywin32 (tested with pywin32-213) and ctypes (included with Python 2.5 and later) """

If we use setuptools' extras like syntax, this would be:

 extras_require={'platform=macosx': ['pyobjc'],                  'platform=win32': ['pywin32', 'ctypes']                  'platform=unix': ['pygtk']}

I think you miss the "included with Python 2.5 and later" part Anyway, I find it rather verbose compared to what we have proposed so far in setup.cfg I think the latest proposal, where the condition is located in the section name is good. So I'll just BDFL a bit here to move things forward. I am adding it in Distutils, with a notification mail in python-dev, and a blog entry describing the result. Thanks all for the feedback ! -- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

On Sun, Sep 13, 2009 at 2:56 AM, Greg Ewing wrote:

Tarek Ziadé wrote:

...

[setup] foo: bar conditional-sections: one, two

Is conditional-sections: really necessary? Isn't the presence of a conditions: line in a section enough to mark it as conditional?

I find having a conditional section explicitely defined better that doing a convention on names because if someone implements a command that uses that name (it's likely to be improbable but...) it will break his ability to use setup.cfg to give options to his commands Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

On Tue, Sep 15, 2009 at 10:47 AM, Greg Ewing wrote:

Tarek Ziadé wrote:

...

I find having a conditional section explicitely defined better that doing a convention on names because if someone implements a command that uses that name (it's likely to be improbable but...) it will break his ability to use setup.cfg to give options to his commands

I don't follow that. If a "conditions:" line is being used to express the condition, then you can't use the name "conditions" for anything else anyway.

setupt.cfg is also used by commands that is. Imagine a command with the name "foo", that has an option called "condition". The setup.cfg file might look like this: """ [global] verbose: 1 [setup] name: MyDistribution version: 1.0 [only_windows] condition: sys_platform == 'win32' requires: pywin32 [foo] verbose: 1 condition: badababdalolo """" Here, the foo section is not supposed to be used by the setup section, it's just a section for the command value, which is allowed (see http://docs.python.org/distutils/configfile.html) Now Distutils, if told to grab all sections that have a "condition" variable in them, will fail at reading "badababdalolo". With an explicit list of the conditional sections, the probelm can't occur, because it's explicit: """ [global] verbose: 1 [setup] name: MyDistribution version: 1.0 conditional-sections: only_windows [only_windows] condition: sys_platform == 'win32' requires: pywin32 [foo] verbose: 1 condition: badababdalolo """" Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

On Tue, Sep 15, 2009 at 1:34 PM, Jeff Rush wrote:

Tarek Ziadé wrote:

...

Imagine a command with the name "foo", that has an option called "condition".

The setup.cfg file might look like this:

""" [global] verbose: 1

[setup] name: MyDistribution version: 1.0

[only_windows] condition: sys_platform == 'win32' requires: pywin32

[foo] verbose: 1 condition: badababdalolo """"

Here, the foo section is not supposed to be used by the setup section, it's just a section for the command value, which is allowed (see http://docs.python.org/distutils/configfile.html)

Ooh, nasty.  I would -want- this case to error out because it could introduce a hard to find problem.  Perhaps that condition: is intended for use by [setup] but a tired person forgot to list it in [setup].  He keeps looking at his setup.cfg and cannot see why it doesn't use his condition.  I'd rather a big error popped up than a silent ambiguity.

Another options is to prefix those sections that are to be used with [setup]: """ [global] verbose: 1 [setup] name: MyDistribution version: 1.0 [setup:only_windows] condition: sys_platform == 'win32' requires: pywin32 [setup:only_mac] condition: sys_platform == 'darwin' requires: pywin32 [foo] verbose: 1 condition: badababdalolo """"

-Jeff

-- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

On Tue, Sep 15, 2009 at 5:44 PM, Floris Bruynooghe wrote:

That's quite nice, +1 And is it worth to consider erroring out when you have a [setup:.*] section without a "condition: ..." line?

Sure,

On Wed, Sep 16, 2009 at 2:30 AM, Greg Ewing wrote:

...

Tarek Ziadé wrote:

...

Imagine a command with the name "foo", that has an option called "condition".

If you really want to keep the entire keyword namespace available for use as a section sees fit, it would be better to put the conditions inside the section headers, as I suggested earlier.

I think that would also make it easier to see which sections are conditional when looking through the file.

yes, that's another option. It just feel weird to my eyes but I guess that's just me.. So this would be: """ [global] verbose: 1 [setup] name: MyDistribution version: 1.0 [setup:sys_platform == 'win32'] requires: pywin32 [setup:sys_platform == 'darwin'] requires: foo [setup:'linux' in sys_platform] requires: bar [foo] verbose: 1 condition: badababdalolo """" I am not including 'requires' in the section like you have previously shown, since it's a metadata field and since we will probably have other fields in these section (see PEP 341 proposals like 'obsoles') Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

2009/9/15 Sridhar Ratnakumar :

On Tue, 15 Sep 2009 02:00:41 -0700, Tarek Ziadé wrote:

...

[only_windows] condition: sys_platform == 'win32' requires: pywin32

I am ambivalent about using sys.platform. How do I specify Linux?

  >>> sys.platform   'linux2'

"linux2"? What are the possible values for `sys.platform`? Is it even a well-defined list?

Yes of course, http://docs.python.org/library/sys.html os.uname() also give you more details on the platform/

What if Linux 3.x comes out .. and the package would stop using the conditional dependent package (that is dependent on 'linux2')?

The same thing that what would happen when Python 4 is out I guess, all distributions will have to change their dependencies list (thus the setup.cfg file) accordingly in their next release.

cf. http://code.google.com/p/platinfo/issues/detail?id=1

-srid

-- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

On Wed, Sep 16, 2009 at 10:21 AM, Greg Ewing wrote:

Sridhar Ratnakumar wrote:

...

  >>> sys.platform   'linux2'

"linux2"? What are the possible values for `sys.platform`?

This is why I usually write Python code that checks sys.platform like

 if sys.platform.startswith("linux"):    ...

So the condition language for the metadata needs a "startswith" operator, I think.

What about 'in' ? 'linux' in sys.platform should cover all cases

-- Greg _______________________________________________ Distutils-SIG maillist  -  Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

-- Tarek Ziadé | http://ziade.org | オープンソースはすごい!