On Mon, Oct 12, 2009 at 4:45 AM, Tarek Ziadé wrote:

On Mon, Oct 12, 2009 at 2:29 AM, Ian Bicking wrote:

...

The grammar in "Context-dependant sections" indicates possible EXPR values.  Because the "in" operator is supported, I would assume that tuples should also be allowed.

"in" here is restricted to string. It was added so we could write things like:

'linux' in sys_platform  (where sys_platform can be linux2)

If you don't have tuples or <, >, etc, it seems like something like "Python version 2.6 or higher" is hard to express. You'd have to enumerate 2.6, 2.7, and speculate on 2.8 and 2.9.

I'll add a note on that.

...

One aspect of the current setup.cfg is that it supports multiple sections, for different setup.py commands or components.  This gives a kind of namespacing.  I assume, but it isn't that specified, that any section (not just "metadata") will be parsed the same way?

I guess yes, even if I don't see a use case yet for that.

One use case for the current setup.cfg is for extensions, like generating Sphynx documentation. Those extensions can sometimes take quite a lot of options, so they are best when partitioned into their own section. I'm also not sure whether [metadata] is intended to have extensible variables, or a fixed set of variables.

...

Presumably setup.py will just contain an empty call to setup()?  (I assume at least setup.py will be allowed for, for backward compatibility, even if it is not required.)

No because we might need to define extra options in setup() that are not part of the metadata, like what is required the for the sdist command (package_data, scripts, package, etc)

OK, so setup.cfg is for generating PKG-INFO, but installing a package still involves running setup.py and some maybe-declarative code in there.

...

I believe this does not include the concept of extra requirements. Possibly it could as a side-effect of some variable available to the language.  Like:

 [metadata:'xmlrpc' in features]  requires = lxml

Sets and the & operator could be useful for this.

How would you define/provide "features" here ?

I'm not sure. With Setuptools the equivalent is "extras", like: setup(name='foo', extras_require={'xmlrpc': ['lxml']}) Then if you do "easy_install foo[xmlrpc]" (or somehow require foo[xmlrpc]) it will also install/require lxml. If I was to say why this is a problem, I think it's because the default is that there are no features for a package. So someone naively wants your package and installs it, but they don't get all the features they thought the package provided (and which the package actually *does* provide, but just doesn't have the prerequisite libraries for -- since the package ships with the xmlrpc code, it's just not working xmlrpc code). There's also not a good way of seeing what extras are provided, or what their purpose is. So library authors avoid the issue entirely and don't factor out requirements into specific extras/features. If there was a set of "default" extras maybe it would be more workable. I.e., "easy_install foo" installs foo with all its default extras, and "easy_install foo[]" installs foo without any extras (or you put the specific extras you want in []). Anyway, the way extra requirements are serialized to disk is requires.txt, with: default_req1 default_req2 [extra] extra_req1 ... So, since the result involves multiple sections it wouldn't naturally map to what you are proposing.

...

The way variables are handled is unclear.  Presumably all variables are cumulative.  But given:

 requires = Foo  requires = Bar

What is the value of "requires"?  A list, a value with newlines? Presumably you could also do:

 requires = Foo      Bar

Anyway, we're diverging from INI semantics at this point, so it needs to be specified how it works.

Right, this needs to be defined. I would be in favor of the latter, to stay ConfigParser compatible.

You still have to define how options are combined from multiple sections, and what the resulting value is from the API. That is, if you have: [metadata] requires = Foo Bar [metadata:python_version == '2.4' or python_version== '2.3' or python_version=='2.2'] requires = BackwardCompat Then what is the ultimate value of "requires"? Is it "Foo\nBar\nBackwardCompat" or ["Foo", "Bar", "BackwardCompat"], or [Requirement.parse("Foo"), Requirement.parse('Bar"), Requirement.parse("BackwardCompat")]. And given other variables (ones that perhaps distribute doesn't even know about) how are they combined?

...

Is there a way to eliminate values?  Right now, for instance, if you have "[build] home = /foo" in a distutils.cfg, there's no way to unset that.  I'd like to see this functionality included.  Perhaps to delete a specific value, but the simpler case of deleting a variable is really all I want.

Do you have a syntax in mind for this ?

Well, the way I added more meta-operations in Paste Deploy (which uses ConfigParser) was to mess with the variable names. So maybe: [metadata:python_version=="2.5"] del requires = Foo It might have a value to delete a specific value (or line) from that "requires" variable, or if empty it would clear that variable. Since ConfigParser doesn't order the variables, it would delete them from "earlier" sections, not the current section. This requires defining what makes a section "earlier". Or maybe it would delete "Foo" from whatever the value of requires ends up being, or if you do "del requires =" would make that section provide the only value for requires. Another thought: one thing distutils does but sometimes confuses me is that it has boolean options (options which take no values, sometimes with an option and an anti-option, e.g., --feature-foo and --no-feature-foo) which turn into a single configuration variable with a boolean value. How that is supposed to work seems apropos (it's not strictly syntax, but I don't feel like I can judge the syntax without considering the full semantics of the configuration file, because the semantics drive the actual use cases which should in turn be used to judge the syntax). -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker

On Mon, Oct 12, 2009 at 11:34 AM, Tarek Ziadé wrote:

On Mon, Oct 12, 2009 at 6:10 PM, Ian Bicking wrote:

...

If you don't have tuples or <, >, etc, it seems like something like "Python version 2.6 or higher" is hard to express.  You'd have to enumerate 2.6, 2.7, and speculate on 2.8 and 2.9.

python_version not in '2.3,2.4,2.5'

(it's not optimal, but enough I guess, until PEP 386 is accepted)

Given the small number of values available, string comparison works fine in this case. Is there a situation when the data would be dependent on a package version other than Python's?

...

One use case for the current setup.cfg is for extensions, like generating Sphynx documentation.  Those extensions can sometimes take quite a lot of options, so they are best when partitioned into their own section.  I'm also not sure whether [metadata] is intended to have extensible variables, or a fixed set of variables.

The variables are fixed so it works with a vanilla python and doesn't require anything else to run. Having extensible variables would break that idea and would just make people move code they use to have in setup.py in another place imho.

It should at least be allowed to have other values with a warning, similar to keyword arguments to setup() that are unknown. This lets the format be forward compatible with new variables. E.g., if in Python 2.8 there's a new variable, people can just dump it in.

...

...

How would you define/provide "features" here ?

I'm not sure.  With Setuptools the equivalent is "extras", like:

setup(name='foo',    extras_require={'xmlrpc': ['lxml']})

Then if you do "easy_install foo[xmlrpc]" (or somehow require foo[xmlrpc]) it will also install/require lxml.

[...]

...

 ...

So, since the result involves multiple sections it wouldn't naturally map to what you are proposing.

That could be done I think as long as the extras are not calculated by third-party code.

Let's say, we define an 'extras' variable, which value can be provided when interpreting setup.cfg.

If could be described like this :

[metadata:'xmlrpc' in extras] requires = lxml

The part that is unclear to me is how to list the extras a setup.cfg file has.

There needs to be some enumeration of extras, yes. And there really needs to be more metadata than Setuptools allows for -- extras should have at least a readable description. So wherever they are defined, there should be room for that documentation as well.

...

And given other variables (ones that perhaps distribute doesn't even know about) how are they combined?

Right, this needs clarification. In any case, I think using multiline is a bad idea because it'll break RCF 232 compatibility for the long_description field.

Oh... long_description -- I forgot about that. How will that be dealt with? I now typically do something like this in setup.py: long_desc = open(os.path.join(os.path.dirname(__file__), 'docs', 'index.txt')).read() long_desc = long_desc.split('..contents::', 1)[1].strip() setup(..., long_description=long_desc) Keeping the long description in setup.py (or in setup.cfg) is not something I want to do. It would be even worse in setup.cfg than it is in setup.py. Also ConfigParser eats leading whitespace (I believe that's also a problem with the PKG-INFO format), which breaks ReST. (I assume the "register" command does something other than uploading PKG-INFO to get long_description in place?) (Come to think of it, I'm not actually sure what we're accomplishing with this declarative metadata; "setup.py --name" is too minimal, but some new "setup.py metadata" command that dumps everything isn't hard to imagine; what are the *real* advantages of this new proposal? If I have to use a build process to build my setup.cfg, then absolutely nothing will be accomplished. And is the complexity all just because sometimes people need to use version-specific requirements?) -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker

On Tue, 20 Oct 2009 13:28:56 +0100, Chris Withers wrote:

Yes, and yet everytime I ask for the ability specify long description as a file path relative to the pacakge, I get shouted at ;-)

[setup] name = artistflair version = 1.2 description_file = flairdescript.rst A soft voice is calling you from the other (alternate) side... David

Tarek Ziadé wrote:

Then, the day PEP 386 is accepted, we turn "python_version" into a Version() object and we introduce '>', '<' and al.

What's stopping PEP 386 being accepted? Seems like it'd be a good idea to get it out of the way first... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

Tarek Ziadé wrote:

On Tue, Oct 20, 2009 at 2:30 PM, Chris Withers wrote:

...

Tarek Ziadé wrote:

...

Then, the day PEP 386 is accepted, we turn "python_version" into a Version() object and we introduce '>', '<' and al. What's stopping PEP 386 being accepted? Seems like it'd be a good idea to get it out of the way first...

The last round (last summer) was not in favor of having post/dev markers in the version scheme (these are required by some developers), so PEP 345 and PEP 386 where sleeping a bit.

I'm sure I can't be the only person suffering from PEP overload when it comes to packaging. Any chance we could at least get dev/post markers in PEP386 and get it done and out of the way? I have a feeling that PEP345 and PEP390 along with David's alternative proposal are all related in such a way that the best thing ot do is bottom out the latter two first, but they all seem to depend (whether or not they want to admit it ;-) ) on PEP 386... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

Tarek Ziadé wrote:

Agreed. And every piece of puzzle is starting to emerge.

As Marc-André said, PEP 390 is less important and could be done in distutils even without PEP, as long as we add the markers in PEP 345, (meaning we accept 386).

What markers are you referring to? A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of any of the contentious bits of PEP390. What am I missing? (other than that 386 should be accepted and implemented asap...) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On Oct 20, 2009, at 9:53 AM, Tarek Ziadé wrote:

...

(other than that 386 should be accepted and implemented asap...)

Its implemented, we just need a consensus

+1, ship it! S

On Tue, Oct 20, 2009 at 5:17 PM, Chris Withers wrote:

Tarek Ziadé wrote:

...

...

A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of any of the contentious bits of PEP390. What am I missing?

That's the part I have not added yet. Markers are the "sys_platform == 'win32" things discussed in PEP 390.

But PEP345 is about the metadata that appears in the installed package, no?

What use is it having conditionals in there at all?

Because we want to add the PEP 390 context dependent markers in there, as explained in PEP 390 (I've updated it yesterday, have a look) It provides the markers to any third party app that wants to read the metadata, and make it avaiable directly at PyPI.

...

...

(other than that 386 should be accepted and implemented asap...)

What are the current sticking points? I do remember some discussions surrounding this but didn't see any active threads when I went through distutils-sig today...

That's from this summer: some folks found the scheme overcomplex and dont want to see it as the standard. I am working in finding back the details. Tarek

On Oct 20, 2009, at 8:56 AM, Chris Withers wrote:

I'm sure I can't be the only person suffering from PEP overload when it comes to packaging.

+1 on PEP overload

Any chance we could at least get dev/post markers in PEP386 and get it done and out of the way?

+1 on getting the versioning number scheme nailed down. The reference implementation looks pretty good from a quick review and the tests, while not insanely exhaustive, seem to cover everything that would come up in practice. If anyone disagrees with that last assessment, how about providing a test case?

I have a feeling that PEP345 and PEP390 along with David's alternative proposal are all related in such a way that the best thing ot do is bottom out the latter two first

I'm not sure anymore what "bottom out the latter two first" means, but getting 386 out of the way would certainly make everything else easier. S

On Tue, Oct 13, 2009 at 8:18 AM, David Lyon wrote:

On Mon, 12 Oct 2009 00:17:10 +0200, Tarek Ziadé wrote:

...

Hey

this is the PEP for setup.cfg, as requested : http://www.python.org/dev/peps/pep-0390

Please comment,

Tarek

Can I have assistance submitting an alternate PEP?

Generally speaking you never answer any email from me but in this case I ask that you assist in respect to having a fair and due process.

That is not true. If you look at the distutils-archive I've answered to most of your questions.

I have subscribed to peps@python.org but havent received an email back. I have posted it to peps@python.org.

Here is my PEP...

Why are you doing a similar PEP, what's the point ? You've already helped a bit. I suggest that you help us finishing in the existing PEP 390 instead, by proposing changes. Tarek

On Tue, Oct 13, 2009 at 11:16 AM, David Lyon wrote:

On Tue, 13 Oct 2009 10:36:12 +0200, Tarek Ziadé wrote:

...

Why are you doing a similar PEP, what's the point ? You've already helped a bit.

Tarek,

I championed the use of static metadata on the distutils mailing list before you picked it up. Don't you remember?

Frankly, I don't. The first time we've mentioned static metadata was at Pycon last year. The first one who mentioned the idea was Eric IIRC, then Matthias and Tres worked on it. But who cares ? I am not claiming any ownership on ideas. I am not tracking either who threw which idea. But if someone cares, speak up : I'll detail who participated in the Acknowledgment section. I just happen to be the one maintaining PEP 390 and I am trying to collect ideas here because I am maintaining Distutils so I'll be the one who will apply the changes there.

The reason that I am doing an alternate PEP is because it contains improvements that would be very useful on windows and linux.

A fair and open development process should mean that people are able to freely contribute their ideas and have them evaluated on the merits of the idea.

That's exactly how PEP 390 was built. The process is far from being perfect though, maybe some ideas didn't make it through. But that's how it goes.

...

I suggest that you help us finishing in the existing PEP 390 instead, by proposing changes.

If I see things I can help with, then I'm happy to do that...

Great then, I am looking forward for its improvements. Tarek

David Lyon wrote:

On Tue, 13 Oct 2009 11:33:26 +0200, Tarek Ziadé wrote:

...

The first one who mentioned the idea was Eric IIRC, then Matthias and Tres worked on it.

Fine. They've been awfully quiet on distutils-sig lately :-)

Not so!

...

I just happen to be the one maintaining PEP 390 and I am trying to

collect > ideas here because I am maintaining Distutils so I'll be the one > will apply the changes there.

Sure. Well "collect" is a word that means different things to different people. I'd suggest some caution with using that word internationally because in some places it really can mean just outright theft. I'm very sure that that isn't how you want others to interpret your working process. Please.. don't use that word like that..

I'm sure no offense was intended. I don't see how we can all know all possible words that might be construed as offensive. But back to the matter at hand: I don't see what David is proposing as being radical or even different from what we've been discussing: a single static file that contains enough metadata to describe what's in a distribution. This file needs to be extensible. The Distutils-SIG approach has been incremental, adding small parts. It seems he wants to go "big bang". I think that's fine, but it will be more difficult to migrate to, I fear. He's proposing shipping an application in each distribution (his new setup.py) that processes that file to do installations. Sounds like a bootstrapper of sorts, and not so radical of an idea. This setup.py would use parts of existing distutils to do its work, where it can. I think a name other than setup.py would reduce confusion, but the idea seems practical. I'd be interested to hear how this would interact with PyPI and buildout. Eric.

Hi Eric.. On Wed, 14 Oct 2009 08:46:26 -0400, Eric Smith wrote:

...

...

The first one who mentioned the idea was Eric IIRC, then Matthias and Tres worked on it.

Fine. They've been awfully quiet on distutils-sig lately :-)

Not so!

:-)

...

Sure. Well "collect" is a word ..

I'm sure no offense was intended. I don't see how we can all know all possible words that might be construed as offensive.

Collect or collection isn't the slightest bit offensive as Python itself is merely a collection of a lot of peoples good work.

I don't see what David is proposing as being radical or even different from what we've been discussing: a single static file that contains enough metadata to describe what's in a distribution. This file needs to be extensible. The Distutils-SIG approach has been incremental, adding small parts. It seems he wants to go "big bang". I think that's fine, but it will be more difficult to migrate to, I fear.

Downloading a new setup.py from a website and writing a setup file is easier than somebody new than coding a tradional setup.py file. A tool should be provided to read an existing setup.py and create the corresponding static file.

He's proposing shipping an application in each distribution (his new setup.py) that processes that file to do installations. Sounds like a bootstrapper of sorts, and not so radical of an idea.

It isn't so different technically from what exists in setuptools.

This setup.py would use parts of existing distutils to do its work, where it can. I think a name other than setup.py would reduce confusion, but the idea seems practical.

Speeking as an end user (system administrator), all the documentation says type 'setup.py install'. It's a habit. I think it's the least confusing thing for an end user.. it might confuse developers.. not so much I think.

I'd be interested to hear how this would interact with PyPI and buildout.

There is no impact on pypi or buildout. The setup.py has all the same behavior as a traditional setup.py except when it is run at the end users system. What I mean is that all existing command line options get passed through and work the same way as before. David

On Tue, 13 Oct 2009 17:55:36 +0900, David Cournapeau

...

[datafiles] files = artisticdb.dbf artisticdb.mdx authors.dbf authors.mdx directory = db

How would you handle the case where you have several data directories ?

[datafiles] directories = data/us_data data/de_data data/uk_data data/fr_data or: [datafiles] directories = us_data de_data uk_data fr_data

Concerning data files, the notion is too vague IMHO - I know that's the notion used by distutils

Vague as in what? Datafiles is an installation term term meaning that's mostly related to databases, but could also be any other type of resource down to icons and bitmaps. David

On Tue, Oct 13, 2009 at 02:18:11AM -0400, David Lyon wrote:

This PEP proposes a change to the way that applications and libraries are installed in python, by using a new file called setup.info rather than setup.py. To hold installation information.

This seems to propose a whole build system, while PEP390 only aims for static metadata. Not restricting it to static metadata makes the domain a lot wider and complicated as shown by countless previous discussions on this list (several of these things have been outlined before). Since the problem domain is so huge it will take a very long time to come up with a stable version that addresses all issues adequately, something which this PEP sidesteps solving the problem it targets in a sensible and contained way. It certainly is a laudable goal to come up with such a build system but I don't think it can be seen as *instead of* the metatdata PEP. IMHO such a system should be using the static metadata from whatever PEP 390 will become. Regards Floris

On Tue, 13 Oct 2009 19:01:30 +0200, Tarek Ziadé wrote:

If wanted, I can go ahead I create a new PEP for David's proposal. And David can work it out using PEP 390 references maybe

let me know

Thank you Tarek, I appreciate it. Good management decision :-) David

On Tue, 13 Oct 2009 23:26:57 +0200, Tarek Ziadé wrote:

...

...

If wanted, I can go ahead I create a new PEP for David's proposal. And David can work it out using PEP 390 references maybe

let me know

Thank you Tarek,

I appreciate it.

So is that a yes ?

I will answer offlist.. Thanks Tarek

On Wed, 14 Oct 2009 00:36:54 +0100, Floris Bruynooghe wrote:

To me it would seem a little early to start a PEP like this, there's been virtually no discussion about this particular proposal nor any proof of concept code. And given the scope of wanting to change the entire way build-steps are declared some code that can give experience and show the good and bad points seems important to me.

There's been much discussion on the distutils sig about static metadata this year. So I have to disagree with you there on that. However, you are correct about proof of concept code, and in some respects that's the easy part. I will start work on that immediately. Central to this proposal is providing a boilerplate setup.py that is able to do an install of a package/application entirely from the information contained in the setup.info file. What it means is that a developer won't need to write a setup.py at all. Just use the one provided and edit the data in the setup.info file. Thank you for your suggestion. David

On Wed, 14 Oct 2009 11:21:40 +0200, Tarek Ziadé wrote:

My suggestion is that once you have a revised PEP that includes some proof of concept code with less overlapping with PEP 390 (they are different proposals obviously), you post it here for a round of feedback, then I'll push it in the PEPs for further processing

Thank you kindly Tarek. I'll look into seeing how I can try to reduce overlap if you think that it's an issue. The problem for me is that under windows it's considered not good programming practice to build logic implicitly into section headers within a configuration file. So things like: [metadata:python_version == '2.4' or python_version == '2.5'] requires = bar Doesn't look at all good to me. It looks like code in a configuration file. I'm really against it. But hey - I'm biased. So for me, the overlap is not strong. Because I see my alternative as a more technically correct way to approach using configuration files. The sample code you provided on the other hand is very nice. Perphaps when you see my code you'll be as horrified at my code as I am at your configuration files. :-) David

Ian Bicking wrote:

On Tue, Oct 13, 2009 at 6:36 PM, Floris Bruynooghe wrote:

...

To me it would seem a little early to start a PEP like this, there's been virtually no discussion about this particular proposal nor any proof of concept code. And given the scope of wanting to change the entire way build-steps are declared some code that can give experience and show the good and bad points seems important to me.

Could an example API be encapsulated in something like this in setup.py?

from test_this_pep import setup_cfg setup(other args, **setup_cfg())

Then packages could be converted to test it out, without breaking the package.

This would work for the parts which are concerned using static metadata for what exists in distutils, but it would not work for pre_install, etc... The only way I can think to make this work is to actually generate a setup.py (or in the other direction, to generate a setup.info from setup.py). I think it is much better long-term solution for a 'distutils v2' than trying to fix the existing code, but it is certainly a lot of work, David

David Lyon wrote:

What's a few thousand lines of code between friends..

I think that's some quite tough thousand lines, though. You have to know a lot about the different platforms, and in my experience at least, designing a good tool and API around this kind of problems is hard. The good news is that there are existing tools in other environments which do this in a pretty good way already, so we could steal their idea and just reimplement the thing for python. This problem space has seen a lot of new ideas since distutils was implemented ten years ago (and we can learn from distutils mistakes). I still like the idea of reusing the same format as .cabal files (see here: -http://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/authors.ht...). It does everything you want to do with .cfg files, with the advantage of being proven for real softwares and more complete. The use of ini-like files with conditional in sections really feels like an ugly and inflexible hack for a new system, I am not sure I understand why you want to use them for a brand-new system where backward compatibility with setup.cfg isn't a concern ? David

On Wed, Oct 14, 2009 at 5:59 PM, David Lyon wrote:

ConfigParser is in every python version that I know of.

ConfigParser was first documented in Python 1.5.2, released 30 April 1999. Some of us remember the releases that came before, but (thankfully) we're outnumbered. :-) -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

On Wed, 14 Oct 2009 18:52:57 -0700, Kevin Teague wrote:

...

ConfigParser was first documented in Python 1.5.2, released 30 April 1999.

Some of us remember the releases that came before, but (thankfully) we're outnumbered. :-)

1.5.1 was the first Python that I used, seems like I managed to squeak onto the outnumbered side.

Show's how behind the 8-ball I am.. I only noticed it in 2007.. :-) David

On Thu, 15 Oct 2009 10:59:48 +0900, David Cournapeau wrote:

Saying that a packaging tool is about copying files is not very useful - you could say that programming is just moving bytes around and you would be right as well :)

Well it is about moving bytes around.. That *is* just how I see it..

I am quite agnostic on the syntax as well - as PJE mentioned a couple of months ago when talking about a new distutils from scratch, even using XML for a first prototype would be acceptable. The .cabal file has a nice syntax, though, and quite pythonic in some ways (indentation instead of brace for example).

It's just too hard to do and adds another layer of complexity to deal with. I'm on limited time like everybody else. ConfigParser for all it's annoying faults is still quire powerful. I don't like nesting in configuration files. I tend to side with the configparser approach of not allowing nesting. More importantly than that, I don't believe that this particular use case gets complicated enough to require nesting. You can achieve a type nesting with configparser by just making the section names ever long. It achieves the same thing imho. Some people disapproved strongly of xml. Most were lukewarm about configparser - and we can't find an implementation of haskel. So for me, I'll go with the lukewarmers.. Primary concern is using something that exists or will work on 'most' python versions. I'm not sure how many python 1.5 thrillseekers still use 1.5 solely anymore. But if there are some, configparser shouldn't offend them. David

On Wed, Oct 14, 2009 at 11:16 PM, David Cournapeau wrote:

I don't think python 1.5 should be considered - I happen to contribute to scons, which is a build tool in python, and supports all python versions from 1.5.2: the 1.5 requirement is a real downside.

I hope 1.x isn't a target for any of these projects. At this point, I'd consider support for Python 2.4 to be quite generous on the part of the developers. And yes, I'm still using Python 2.4. Thankfully I'm not developing or maintaining anything on Python 2.3. I would be surprised if there was widespread interest in adopting new tools in projects using such old versions; keep in mind that even if something stable is available in 6 months, it'll be a year or more before adoption becomes widespread anyway. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

On Thu, 15 Oct 2009 12:16:48 +0900, David Cournapeau wrote:

I think nested conditions for configurations is a must-have. Once you have more than 2-3 variables, doing it like PEP 390 is not really manageable. Particularly for C extensions, having many configurations is quite common in my experience.

Like I said, one can achieve 'and' and 'or' operations with configparser headings. [dependencies] packages = pyopengl [dependencies python<=2.4] packages = lxml >= 2.5 [dependencies mac python<=2.4] packages = shinyxml Translates roughly into the following python code: dependencies = ["pyopengl"] if sys.version <= '2.4': dependencies.append('lxml') if sys.platform == 'darwin': dependencies.append('shinyxml') So to me, configparser does provide a way to achieve a sort of nested capability. It just needs careful choices of headings. Some people also use '-' to provide nesting. For example: [top] value = 1 [top - nested level 1] value = 2 [top - nested level 2] value = 3 These are accessed with 'top' + " - nested level %d" % levelno To me this is all nesting. I'm extremely interested to know of a practical example of something related to packaging that we can't script for. If you think of something, please post it. 'build' and 'test' were too other keywords that were suggested to work in also. I don't really want to do a build platform, only something for the install platform. But here's how they could work. [dependencies windows build] packages = py2exe [dependencies mac build] packages = py2app [dependencies test] packages = unittestplus David

On Thu, 15 Oct 2009 13:39:32 +0900, David Cournapeau wrote:

I understand this, but when you a lot of conditioning, "flattening" the conditions is not really readable. If you have say 10 variables instead of 2 to condition on, the sections would be several lines.

I'm only allowing up to four. These are: - specifier ['dependencies','preinstall','postinstall','scripts', 'application', 'package', 'documentation', 'uninstall', 'datafiles'] - operating system ['windows','mac','linux' etc...] plus some twiddle-bits like 32, 64, gnome, kde and os-version. - python version - operation ['test','build' ..]. So never above 4. Typically two or three.

More significantly though, how can you define new conditions variables with .ini files ? Would you have a special section for this ?

I don't have a way of doing that right now - and am not aware of practical examples that might require it. Anything that can't be handled with the above will have to go into a pre-install or post-install script and be coded with traditional python coding. If you can find any common use cases that aren't handled, please advise. In the real world, there is a point where things might need to go to code. This capability is provided for. [preinstall] script = checksystem.py [preinstall linux-kde] script = checkmylinux.sh and: [postinstall windows-64-xp python-2.4] script = regedit.exe /s fixpy24win.txt [postinstall windows-64-xp python-2.5] script = regedit.exe /s fixpy25win.txt What that would do would be to make a system registry correction after loading on 64 bit windows only for python versions 2.4 or 2.5. Similarly: [postinstall linux-debian-64 python-2.5] script = fixpy25deb.py [postinstall linux-suse-kde python-2.5] script = fixpy25susekde.py [postinstall mac] script = popupkoolmachinechoice.py I'm really not trying to handle every possible case, just the most common ones. That is, trying to provide a hook for specific unique situations that exist on different platforms and operating systems. David

David

On Thu, 15 Oct 2009 14:48:00 +0900, David Cournapeau wrote:

Let's take a common case: using cython/pyrex to accelerate some code, but you still want people to be able to use your package without a C compiler (bzr, cython are examples of such packages).

I believe that's possible through the [postinstall] section..

If we had a way to define variables as in .cabal files, this could be used to control the build from 'outside'.

Any key can be variable: [postinstall] variable_value = 10 Outside programs can 'read' and 'write' and pass values via this setup.info file during build time. That is the whole point. Simply change the setup.info file.

For me, the goal of having static metadata should be easier interoperation between different tools, and as such the format should be rich enough to describe entirely common cases.

I totally agree.

You could then have a distutils2-debianize tool which would read this file and produce a .deb, another tool for eggs, windows installers, etc... or you could use the parsed file description to control the build from a real build tool system like waf or scons. That's the kind of things I want to be able to do and are so painful right now with distutils.

I'm not really addressing the developer build side at the moment because my experience with it is less. What I feel that I have more knowledge with is the install side and any build/configuration that might take place there. Thanks for the discussion - very informative. David

On Fri, 16 Oct 2009 12:22:20 +0900, David Cournapeau wrote:

The problem is a bit more complicated, because it cannot be handled in one single section.

I really appreciate you sharing this challenge with me. I'm really convinced that I can "tweak" something to make it work. And now is a really good time for me to examine it because I'm coding now. Let me post how the code is looking: def main(): # -- Return Code rc = 0 # -- Check for our config file if not os.path.exists(setupconfigname): logger.error("No %s file found. Installation is unable to proceed" % setupconfigname) sys.exit(10) try: # -- See what our platform is build_platform_bit_map() # -- Work out where to copy files to build_local_directory_targets() # -- Load basic install information load_basic_install_requirements() # -- Process python package dependencies process_dependencies() # -- Build a list of all the files that need to be copied build_file_copy_list() # -- Check that all the files in the copy list exist and are ready for copying validate_file_list() # -- Run preinstall scripts run_preinstall_scripts() # -- Copy files to the local system copy_files() # -- Adjust the python configuration finalise_python() # -- Update platform links/shortcuts finalise_local_links() # -- Run postinstall scripts run_postinstall_scripts() except: rollback(): rc = 1 return rc So let's just take what you wrote step by step..

You need to detect cython (configuration stage)

cython is a type of python interpretor? like jython or ironpython Assuming yes,

you need to modify the extensions sources depending on this, and you need to install different files depending on this:

[Application] scripts = artisticflairgui.py [Application cython] scripts = artisticflairguicython.py That's a different file being installed if we detect we are using cython and not python. Since we are copying files "out", like we used to do, we can include all the files that we need in our package. Only the files needed for the local platform end up getting copied.

it touches at least config, build_ext and install commands to use the distutils terminology.

[postinstall] scripts = buildit.py [postinstall cython] scripts = builtitspecialforcython.py [postinstall jython] scripts = builtitspecialforjython.py [postinstall ironpython] scripts = killvistaupgradetowin7.py

This is similar to my main complain about distutils: you can't easily share data between commands. AFAIK, the only way to do it is to use the Distribute class instance, which is problematic as that's what most tools on top of distutils need to monkey-patch.

It seems to be a really unusual (for me) problem. But I don't dispute that its a real problem. I don't follow all the python spin offs but I agree that they need to be. Hence, they need bits (constants). I just hope I'm understanding the problem the right way. Maybe next week I might even have something that can be tried to see if it will work. I'm busy coding now, so you asked at a good time. David

On Tue, 13 Oct 2009 22:05:25 -0500, Ian Bicking wrote:

Could an example API be encapsulated in something like this in setup.py?

from test_this_pep import setup_cfg setup(other args, **setup_cfg())

Then packages could be converted to test it out, without breaking the package.

Actually, my proposal doesn't break any package. Distutils included. There's one fatal flaw with patching distutils. That is that any patches done to distutils won't automatically find their way onto all the old systems that are out there. What we actually need, and I am building, is a setup.py script that is going to be a 'trimmed' version of distutils that includes all neccessary package/application installation code to do the install from the setup.info file. In a radical departure from the old way, you *don't* edit the setup.py. Just edit the setup.info. Then attach the setup.py unmodified. I'm very sure that Tarek is going to have many good suggestions on the best way to write this *new* boiler-plate setup.py and it is likely that it will turn out a joint effort, because he writes nicer code than me. Maybe I will beg him to help me.. Still, I need to do a proof of concept. Thats what I'm starting now. ---setup.py--------------------------------------------- #---------------------------------------------------- # -- DO NOT EDIT THIS FILE # # This file is the standard setup.py for your # application and you don't need to edit it. # # It was provided from the new distutils setup system # and can be downloaded from: # # www.python.org/distutils/standardsetup/setup.py # # Instead, modify your setup.info file and # provide a pre-install or post-install script # to do your custom processing if that is required. #---------------------------------------------------- # .. .. -------------------------------------------------------- ---setup.info---- [setup] name = artistflair version = 1.2 description_file = artisticflair.rst .. .. ----------------- David yes - it's setup.py upside down...

On Tue, 20 Oct 2009 13:53:31 +0100, Chris Withers wrote:

Be very aware of the problems of proliferating boilerplate code (buildout has already suffered this):

- old versions will have to work forever

- bug fixes are *very* hard to do

Well, you're probably right. So I will build an option in to check on the net for an updated version before starting.

I would like my setup.py to look like:

from xxx import setup setup()

...and have all the code be in xxx. If xxx is a separately available package for python<2.7, then fine.

I would prefer to see "hard logic" in setup.py where there is a good programming language to support it any everything else in setup.cfg

Of course. Code in one place, data in another.

(don't like .info, sorry ;-) ).

I'm softening.

My hunch is that for 95% of packages, setup.py will be nothing more than the above...

Hope so David

On Wed, 21 Oct 2009 08:29:02 +0100, Chris Withers wrote:

David Lyon wrote:

...

So I will build an option in to check on the net for an updated version before starting.

That won't fly.

but it will most definitely work... and it will be backwards compatable with older python installations.

A significant number of people will be installing in situations where setup.py will have no access to the internet.

Of course. So an update check won't make it through.

Why is it that you basically want to put an entire module's worth of code inside every single package's setup.py?

Because it is easy. Let me put it another way - how does a 'normal-user' update distutils in the stdlib?

Take something like errorhandler:

http://pypi.python.org/pypi/errorhandler

The actual code of the package is only about 30 lines long.

Having to include a 500 line setup.py just so I can include metadata in the packages seem pretty insane ;-)

Well, 500 lines is just this week. Maybe by next week it will be even more. If the file is 100kb long, I don't think a user would care. I don't think users in the real world will look at the file.. and think "wow - that's 100k.. we can't possibly run that program on our new quad core 3ghz machine with 4GB ram and 1TB disk space. To me, I think it will be the other way around.. I think they'll think "here's a crunchy setup.py.. lets run that". Same as always. Business as usual. David

On Wed, 21 Oct 2009 09:02:40 +0100, Chris Withers wrote:

...

If the file is 100kb long, I don't think a user would care.

I'm "a user" in this case, and I care... The maintenance burden a file like the setup.py you're proposing places on me as a package author in making sure the packages I maintain all include the right setup.py, and needing to bump my package's versions when code that has nothing to do with my package has bugs fixed isn't acceptable.

ok. But nobody I know in software can stop new patches and versions of software from being made. Versions advance.. For some it's even a day job.

...

I don't think users in the real world will look at the file..

What happens for the author of one of the small number of packages that actually needs to have logic in setup.py?

That's another issue. I'm providing external pre-install and post-install script hooks. To answer the question - It's too early to tell. I'm flexible. We'll see. A lot of code to write still. David

Hi list, I'm a Debian developer who is interested in changes you want to introduce so I'll add my 2 cents from time to time if you don't mind. [Chris Withers, 2009-10-21]

David Lyon wrote:

...

...

Why is it that you basically want to put an entire module's worth of code inside every single package's setup.py?

Because it is easy.

and when there's a bug in this module you want all packages to be reuploaded instead of uploading the one with a bug? We already have to patch setup.py files f.e. to remove ez_setup which hardcodes setuptools versions (so we cannot use newer one) or tries to download packages from internet (that are not really needed at build time).

...

Well, 500 lines is just this week. Maybe by next week it will be even more.

If the file is 100kb long, I don't think a user would care.

oh, reviewing 100kb setup.py files to see why they don't work with Python 2.X (and worked fine with Python 2.Y) sounds like fun ;-P -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[David Lyon, 2009-10-21]

On Wed, 21 Oct 2009 11:24:12 +0200, Piotr Ozarowski wrote:

...

and when there's a bug in this module you want all packages to be reuploaded instead of uploading the one with a bug?

We already have to patch setup.py files f.e. to remove ez_setup which hardcodes setuptools versions (so we cannot use newer one) or tries to download packages from internet (that are not really needed at build time).

I'm not arguing with you. It's no more final a solution than any other piece of source code..

But what happens to code that isn't updated? it is considered as dead code. Then everybody hates that too.

Last MySQLdb stable release was over 2 years ago (there's a release candidate available since March 2009) and to make it work with Python >= 2.6.3 you just have to fix/update setuptools. Now imagine every package would ship its own (old) version of setuptools (just like packages ship their own version of ez_setup now). Do you see a problem already? Sure, one could just: | try: | import setuptools | except ImportError: | import ez_setup | ez_setup.use_setuptools() and thus use installed version if it's available (replace ez_setup/setuptools with what you propose to ship in setup.py) but that's not what module authors do unfortunately. My point is: it's easier to fix a bug in one place and 100kb setup.py would contain a lot of potential bugs or incompatibilities with new Python versions. -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

Hi Chris, Well I put my code up here: https://bitbucket.org/djlyon/original-setup.py-with-metadata/ Plenty on the todo list.. David On Tue, 20 Oct 2009 13:53:31 +0100, Chris Withers wrote:

David Lyon wrote:

...

What we actually need, and I am building, is a setup.py script that is going to be a 'trimmed' version of distutils that includes all neccessary package/application installation code to do the install from the setup.info file.

Be very aware of the problems of proliferating boilerplate code (buildout has already suffered this):

- old versions will have to work forever

- bug fixes are *very* hard to do

I would like my setup.py to look like:

from xxx import setup setup()

...and have all the code be in xxx. If xxx is a separately available package for python<2.7, then fine.

I would prefer to see "hard logic" in setup.py where there is a good programming language to support it any everything else in setup.cfg (don't like .info, sorry ;-) ). My hunch is that for 95% of packages, setup.py will be nothing more than the above...

Chris

On Wed, 21 Oct 2009 08:24:06 +0100, Chris Withers wrote:

David Lyon wrote:

...

Well I put my code up here:

https://bitbucket.org/djlyon/original-setup.py-with-metadata/

The thing is, there's just no need for all this code to be in setup.py.

For me, anything more than:

from xxx import setup setup()

...in the case where all the information is in setup.cfg is totally unacceptable.

so you are a minimalist... me too... but I want my proposed features implemented.. and code is the only way to do it. David :-) "what's a few thousand lines of code between friends" :-)

Chris Withers wrote:

The thing is, there's just no need for all this code to be in setup.py.

For me, anything more than:

from xxx import setup setup()

...in the case where all the information is in setup.cfg is totally unacceptable.

FWIW, I have played a bit with something in that direction - this is not a suggestion for anything at that point, just a toy prototype to play with some ideas I have concerning a new distribution tool in python: http://github.com/cournape/toydist The basis is a static file format, but richer that the proposed PEP 390: - the goal is to be able to describe fully a package from the static data: meta-data, packages, doc, extensions, etc... - converting from existing setup.py to a static file is possible through an API (one could have a distutils command to automatically generate such a file) - a two lines toysetup.py can build basic packages from an existing toysetup.info static file - for the majority of packages out there (the ones which do not extend distutils, basically), the passage setup.py -> toysetup.info -> toysetup.py should be "lossless". This is huge IMHO, because it provides a transition to a system which does not depend on distutils anymore. An example of the static file is: """ Name: hello Version: 1.0 Library: BuildRequires: paver, sphinx, numpy if os(windows) BuildRequires: pywin32 Packages: hello Extension: hello._bar sources: src/hellomodule.c if os(linux) Extension: hello._linux_backend sources: src/linbackend.c """ The corresponding setup.py is simply: """ from distutils.core import setup from toydist import parse_static info_dict = parse_static('toysetup.info').to_dict() setup(**info_dict) """ The format is basically a rip-of from cabal (haskell distribution format) at this point, cheers, David

On Tue, 27 Oct 2009 16:52:36 +0000, Chris Withers wrote:

from distutils.core import setup,parse_static setup(**parse_static('setup.cfg').to_dict())

...and of course, because setup.cfg is the default name, we just end up with:

from distutils.core import setup setup()

Nice.. But one still ends up with a user based setup.py After OPs comments about security I'm totally convinced that we need to ban setup.py, or at a very minimum move a standard setup.py into distutils itself. So one would say.. setup.py --install mypackage.egg or setup.py --install mypackage.tgz (from any directory) In simple cases, setup.py programs don't add any value. All the information required to do a package install is pretty much all to be found in the PKG_INFO file anyway. For typical packages, that is. So we could even wave the magic wand on:

from distutils.core import setup setup()

making it just '' (disappear) if we wanted.. We'd just have to add a script to distutils. I'm not sure how hard that is. David

On Wed, 28 Oct 2009 12:26:19 +0900, David Cournapeau

You can't ban setup.py: although ...

haha - of course. ban setup.py as in ban alcohol or ban using a mobile phone when driving.

With toydist, there would be a new build system not based on distutils, and which would indeed works like this. Assuming you have a static file toydist.info, I would have something like:

toydist configure --prefix=/home/david/local toydist build toydist install

Or even, for a default build:

toydist install (which would do configure and build automatically).

I just want .EGG built into my OS as the registered file extension for a python package. Right-click install..

and would enable people like me who are convinced that distutils design is fundamentally flawed for heavy-weight build needs.

What design? It's a collection of nice pieces of programming along with less nice pieces of programming along with nice pieces of programming that were nice once and now are only nice if viewed with a heavily artistic coloured glass and crystal operating system.. Nobody can bag the design of distutils. It's just a collection of 'stuff'. To me it needs a cleanout.. make things that nearly work.. work. Add some django interfaces or cherrypy interfaces or something so that we can build things 'in safety'. We only have to look at the postings by new users to see how hazardous it is. Once they've broken their legs... tore off their shoulders and blinded themselves in one eye, only then are they ready to get some work for themselves done.. Not saying it distutils doesn't work - only that going along with what you're saying in that it isn't a particularly good working environment. On top of that, we know we have all these dudes here which use these arty farty shiny glossy computers. So there shouldn't be any excuse to keep the old stuff going on forever without a revamp. There are things here that don't make sense.. David

Fred Drake wrote:

For the many packages that don't need that level of flexibility, using a tool that consumes static metadata is a major win, so long as we keep in mind that tools need to be able to determine when that static metadata is sufficient. That allows the development of tools that can usefully get things done *and* refuse to run untrusted code themselves.

I like the idea (I think I saw it in 390 or one of the other PEPs in this aread) that: setup() ...would get everyone from the static metadata file, while any keyword parameters to setup would override the appropriate setting from the static metadata file... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

Fred Drake wrote:

On Wed, Oct 28, 2009 at 10:23 AM, Chris Withers wrote:

...

setup()

...would get everyone from the static metadata file, while any keyword parameters to setup would override the appropriate setting from the static metadata file...

I'd rather see the static data in setup.cfg and no setup.py if there's no need for custom Python code.

I think "python setup.py install" is so idiomatic that it seems silly to break it for the sake of two lines of python. There's also an element of "explicit is better than implicit" in the feel of actually running setup.py...

There are too many ways to write

from distutils.core import setup setup()

to make it useful as a way to say the static metadata is sufficient.

I genuinely don't know what you mean :-S Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On Fri, Oct 30, 2009 at 6:24 PM, Chris Withers wrote:

I think "python setup.py install" is so idiomatic that it seems silly to break it for the sake of two lines of python.

Idiomatic, yes. But it's an idiom that developed as a side effect of the implementation. This may well have been a reasonable choice at the time it was made over a decade ago (and yes, I was there, discussing distutils with Greg Ward during the birthing, so you can probably even lay some of the blame at my feet). That would reflect the youth of Python packaging more than anything else.

There's also an element of "explicit is better than implicit" in the feel of actually running setup.py...

Running a known implementation isn't explicit? I worry more about having to review someone else's ad-hoc setup.py, usually under deadline pressure, especially given how many variations there are. Both are explicit; one is less risky. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

Chris Withers wrote:

I think "python setup.py install" is so idiomatic that it seems silly to break it for the sake of two lines of python. There's also an element of "explicit is better than implicit" in the feel of actually running setup.py...

Besides the already given arguments, getting away of the current distutils commands/subcommands and their imperative nature is a big advantage by itself. David

On Sun, 01 Nov 2009 09:31:46 +0100, Georg Brandl wrote:

...

I think "python setup.py install" is so idiomatic that it seems silly to

...

break it for the sake of two lines of python. There's also an element of

...

"explicit is better than implicit" in the feel of actually running setup.py...

Has anyone already suggested something like "python -m setup install"? It would be rather similar and explicit, too.

Not as far as I know. David

Floris Bruynooghe schrieb:

On Sun, Nov 01, 2009 at 09:31:46AM +0100, Georg Brandl wrote:

...

Chris Withers schrieb:

...

Fred Drake wrote:

...

On Wed, Oct 28, 2009 at 10:23 AM, Chris Withers wrote:

...

setup()

...would get everyone from the static metadata file, while any keyword parameters to setup would override the appropriate setting from the static metadata file...

I'd rather see the static data in setup.cfg and no setup.py if there's no need for custom Python code.

I think "python setup.py install" is so idiomatic that it seems silly to break it for the sake of two lines of python. There's also an element of "explicit is better than implicit" in the feel of actually running setup.py...

Has anyone already suggested something like "python -m setup install"? It would be rather similar and explicit, too.

I'm sure that approach was mentioned in one of Tarek's very early proposals. IIRC it went away because people didn't like the idea of a built-in installer/uninstaller in distutils, but I'm not sure what the arguments where.

It's not like it would contain an installer or uninstaller, just the equivalent of "import setup; setup()" or distributions that don't need a customized setup.py.

Personally I have no objection against it, if it would means you can omit setup.py if it's only going to call 'setup()' then why not.

Exactly. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

David Lyon schrieb:

On Sun, 1 Nov 2009 11:26:43 +0000, Floris Bruynooghe

...

...

Has anyone already suggested something like "python -m setup install"? It would be rather similar and explicit, too.

Actually, I never came across that command line option and had no idea that it even existed. But this is normal, because I don't try to find out every command line possibility or even try to remember many past half a dozen.

Of course you wouldn't have to guess that, it would supersede "python setup.py" in the respective manuals.

When I run it, I even get something (which is interesting):

Not very. You had a setup.py in the current directory. The beauty of the "python -m setup" approach is that it will execute the setup.py in the current directory if present, or a stdlib-supplied one if not. Georg

On Wed, 28 Oct 2009 14:23:58 +0000, Chris Withers wrote:

...would get everyone from the static metadata file, while any keyword parameters to setup would override the appropriate setting from the static metadata file...

Hi Chris, David, Fred, Floris, Tarek, PJE, ... Everybody has been so helpful and I feel that I've now run out of things to discuss. I would like to invite everybody to make some sort of contribution to this, and in return we'll just keep your name as authors of the various bits/files. What I am trying to accomplish is a shell based install system for .EGG/Python packages, along with metadata driven installation. TSo basically getting all the good ideas that *everybody* has had into a cohesive (and real) collection. I'm calling for volounteers/co-owners to: - contribute utilities (say, convert existing setup.py files) - specify/design/code test cases (different platforms etc) - write sphinx documentation (rst) - create examples - critique The url is: https://bitbucket.org/djlyon/original-setup.py-with-metadata/ If we can complete it and have a working system by December, which I believe we can, we can find a new name and have a democratic vote on what that might be. Thank you everybody David

On Tue, 13 Oct 2009 11:30:21 +0100, Floris Bruynooghe wrote:

This seems to propose a whole build system, while PEP390 only aims for static metadata.

It's not a whole new build system, but potentially a whole new install system. There's a big difference. However, the whole new install system is likely to be contained within the boilerplate setup.py, where those facilities don't already exist within distutils now. For the developer, that's really easy.. Just take a copy of the boilerplate setup.py...

Not restricting it to static metadata makes the domain a lot wider and complicated as shown by countless previous discussions on this list (several of these things have been outlined before).

Yes, it is much further reaching and broader. It's actually less complicated than you might think. What I'm proposing is less complicated than say.... setuptools..

Since the problem domain is so huge it will take a very long time to come up with a stable version that addresses all issues adequately,

Define long time.. I believe only a few months..

something which this PEP sidesteps solving the problem it targets in a sensible and contained way.

I disagree. I believe that it's a small programming project amounting to no more than a few thousand lines of code. To me, the email correspondence and mailing list activity will be more voluminous.

It certainly is a laudable goal to come up with such a build system but I don't think it can be seen as *instead of* the metatdata PEP. IMHO such a system should be using the static metadata from whatever PEP 390 will become.

The metadata is the build/install system. How metadata is specified in setup.py now is dated. It's hard to understand for developers and hard to cajole into something that works well on all platforms. My proposal is actually about taking the shortest and easiest route to cleaning up python package/application installation issues. The end result is a setup.py --install and a setup.info file. That doesn't seem so overly complicated to me. David