Refactoring installation schemes
Hello,
Since the addition of PEP 370, (per-user site packages), site.py and distutils/command/install.py are *both* providing the various installation directories for Python, depending on the system and the Python version.
We have also started to discuss lately in various Mailing Lists the addition of new schemes for IronPython and Jython, meaning that we might add some more in both places.
I would like to suggest a simplification by adding a dedicated module to manage these installation schemes in one single place in the stdlib.
This new independant module would be used by site.py and distutils and would also make it easier for third party code to work with these schemes. Of course this new module would be rather simple and not add any new import statement to avoid any overhead when Python starts and loads site.py
Regards Tarek
Tarek Ziadé wrote:
Hello,
Since the addition of PEP 370, (per-user site packages), site.py and distutils/command/install.py are *both* providing the various installation directories for Python, depending on the system and the Python version.
We have also started to discuss lately in various Mailing Lists the addition of new schemes for IronPython and Jython, meaning that we might add some more in both places.
I would like to suggest a simplification by adding a dedicated module to manage these installation schemes in one single place in the stdlib.
This new independant module would be used by site.py and distutils and would also make it easier for third party code to work with these schemes. Of course this new module would be rather simple and not add any new import statement to avoid any overhead when Python starts and loads site.py
+1
M.-A. Lemburg wrote:
Tarek Ziadé wrote:
Hello,
Since the addition of PEP 370, (per-user site packages), site.py and distutils/command/install.py are *both* providing the various installation directories for Python, depending on the system and the Python version.
We have also started to discuss lately in various Mailing Lists the addition of new schemes for IronPython and Jython, meaning that we might add some more in both places.
I would like to suggest a simplification by adding a dedicated module to manage these installation schemes in one single place in the stdlib.
This new independant module would be used by site.py and distutils and would also make it easier for third party code to work with these schemes. Of course this new module would be rather simple and not add any new import statement to avoid any overhead when Python starts and loads site.py
+1
Also +1. It seems like this would make things easier for the alternative implementations.
Michael
On Oct 28, 2009, at 7:02 AM, Michael Foord wrote:
M.-A. Lemburg wrote:
Tarek Ziadé wrote:
Hello,
Since the addition of PEP 370, (per-user site packages), site.py and distutils/command/install.py are *both* providing the various installation directories for Python, depending on the system and the Python version.
We have also started to discuss lately in various Mailing Lists the addition of new schemes for IronPython and Jython, meaning that we might add some more in both places.
I would like to suggest a simplification by adding a dedicated module to manage these installation schemes in one single place in the stdlib.
This new independant module would be used by site.py and distutils and would also make it easier for third party code to work with these schemes. Of course this new module would be rather simple and not add any new import statement to avoid any overhead when Python starts and loads site.py
+1
This would help unpollute ~/ which is getting quite full of 'stuff.'
S
M.-A. Lemburg wrote:
+1
On Oct 28, 2009, at 7:02 AM, Michael Foord wrote:
Also +1. It seems like this would make things easier for the alternative implementations.
On Wed, Oct 28, 2009 at 3:30 PM, ssteinerX@gmail.com ssteinerx@gmail.com wrote:
+1
Ok then I'll work on a patch for that change and start an issue about it soon.
This would help unpollute ~/ which is getting quite full of 'stuff.'
Notice that there's a discussion going on about that at http://bugs.python.org/issue7175
Regards Tarek
On Wed, Oct 28, 2009 at 7:05 PM, Tarek Ziadé ziade.tarek@gmail.com wrote:
Ok then I'll work on a patch for that change and start an issue about it soon.
As I expected, being able to provide all those paths pulls a lot of other stuffs out of distutils.
In fact, almost all the APIs that are located in distutils/sysconfig can be taken out of distutils, and cleaned up for stdlib's benefit. I've started to refactor the code in a module I have called "sysconfig", reusing distutils/sysconfig, distutils/command/install and site code.
This "sysconfig" module should provide at the end very useful APIs to query the current Python environment.
That's a work in progress but: I've started a branch at /python/branches/tarek_sysconfig so it's easier to get some feedbacks if anyone want to help on this.
Regards Tarek
On Wed, 4 Nov 2009 13:29:35 +0100, Tarek Ziadé ziade.tarek@gmail.com wrote:
I've started to refactor the code in a module I have called "sysconfig", reusing distutils/sysconfig, distutils/command/install and site code.
This "sysconfig" module should provide at the end very useful APIs to query the current Python environment.
That's a work in progress but: I've started a branch at /python/branches/tarek_sysconfig so it's easier to get some feedbacks if anyone want to help on this.
Good job so far. Keep going.
imho a refactoring of the installation and build schemes of distutils is due. It seems like you easily have the skills to do it.
My advice would be to do it gradually, as you are. And focus on simplification, and filling in the functional gaps. Ask people what the functional gaps are on their platforms and try to mould it together in an unplatform specific way.
Myself and others can assist with this, but best to do it on distutils-sig.
I would even go so far as to use the python 3 as a carrot for the new work.
imho The biggest and best thing that you could do for python packaging is to do a refactor of the .EGG format.
What I mean here, is *take* the egg, and run with it. Modernise it and revamp it into a platform independent thing. People 'get' the egg idea, they just hate the current implementation.
What we all *need* is for the egg to be what the source distribution now is. Not for it to be some proprietory out out of python object.
Call it the new Tarek egg...
Then refactor distutils build to focus on good egg creation..
Many people will help you.. including myself..
David
On Wed, Nov 4, 2009 at 5:16 PM, David Lyon david.lyon@preisshare.net wrote:
I would even go so far as to use the python 3 as a carrot for the new work.
The packaging story is in such bad shape that it needs the work regardless, and keeping it to Python 3 would significantly reduce the set of potential volunteers.
imho The biggest and best thing that you could do for python packaging is to do a refactor of the .EGG format.
What I mean here, is *take* the egg, and run with it. Modernise it and revamp it into a platform independent thing. People 'get' the egg idea, they just hate the current implementation.
There's certainly work on that, but... is it that people hate the format? Or working with setuptools?
The fact that there's more than one egg format doesn't help, so you may be right.
Call it the new Tarek egg...
The tegg? ;-)
-Fred
On Wed, 4 Nov 2009 17:40:35 -0500, Fred Drake fdrake@acm.org wrote:
The packaging story is in such bad shape that it needs the work regardless, and keeping it to Python 3 would significantly reduce the set of potential volunteers.
Well I guess that is a 'marketing decision'. Not a coding issue.
Actually, I don't honestly think that there is a lack of volounteers or any lack of resources.
The main challenge imho is to get peoples bikesheds (including my own bikeshed) to line up in such a way that people can ride together sharing some sort of parts.
Tarek has been doing just fine at facilitating this.
There's certainly work on that, but... is it that people hate the format? Or working with setuptools?
In my experience, working with the setuptools implementation is the difficult part. There's a barrage of impossible to remember command line options and the documentation is imho more convoluted than it needs to be.
Let me put the distutils documentation forward as some sort of reference as to where the setuptools documentation should be.
Then, there are some relatively minor issues that just annoy users to no end. The simple one is that they live in your site-packages directory unpacked. People wouldn't have so many issues if they lived like every other 'normal' package.
As for the format itself, there's nothing to hate that I can see.
The fact that there's more than one egg format doesn't help, so you may be right.
Yes. The implementation can be confusing. That's the only problem imo.
Call it the new Tarek egg...
The tegg? ;-)
l'oeuf ?
In summmary, it doesn't matter so much what an 'egg' is.
All that is important is that it works on lots of python platforms, distutils can easily pop them out, they are well documented and relatively trouble free.
Most importantly, credit for the original idea goes back to PJE and everything gets refactored to make it 'nice'.
I don't think it's a massive refactoring operation myself, and I'm very sure well within Tareks skillset to take on.
David
2009/11/5 David Lyon david.lyon@preisshare.net:
Call it the new Tarek egg...
The tegg? ;-)
l'oeuf ?
While I like 'tegg' a lot ;) , I would like to make it clear that "eggs" are not part of Distutils. That's the Setuptools format.
Although there's a distinction to make between the *installation* format, and the *distribution* format (sdist, bdist, etc)
Right now we have several installation formats which leads to problems I won't mention here.
PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme.
But there's no plan to include a new *distribution* format in Distutils.
In any case those PEPs are not finished yet, so everyone can help at distutils-SIG
Regards Tarek
On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé ziade.tarek@gmail.com wrote:
PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme.
Perphaps..
But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code...
Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010.
But there's no plan to include a new *distribution* format in Distutils.
I wasn't suggesting that - at all.
And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years.
No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic.
Be fair...
I'm saying finish what is already there.. or take out the crap ..
It isn't fair to suggest that I am somehow asking for some big change when I am simply pointing out all the junk that's in there that's already half built.
In any case those PEPs are not finished yet, so everyone can help at distutils-SIG
True - and False.
But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet.
Seriously, if you won't allocate work out to people then how can it get done?
Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway?
I'm a fairly proficient develper, but I can't get assigned a single work item..
And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items...
Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010..
David
On 2009-11-05 16:29 PM, David Lyon wrote:
But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet.
Seriously, if you won't allocate work out to people then how can it get done?
With rare exceptions, open source projects don't really "assign" work. If you want to help, help. Don't wait for someone to tell you exactly what to do. No one will. Go to the tracker, find something interesting, and do it.
On Thu, Nov 5, 2009 at 11:29 PM, David Lyon david.lyon@preisshare.net wrote: [...]
But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code...
Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010.
The work that was done during the last year was brainstorming on PEPs. You can't count that work in SLOCs, but in trying to find a consensus on packaging problems.
And I can fairly say that we are very close to something that can get accepted by the community, then turned into code in Distutils/pkgutil.
But there's no plan to include a new *distribution* format in Distutils.
I wasn't suggesting that - at all.
And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years.
They are new to Distutils. They are not new to the Python packaging eco-system of course, and PEP 376 rely on many ideas created in Setuptools.
No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic.
The consistency will come through the standard we are building in PEP 376.
[..]
But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet.
Seriously, if you won't allocate work out to people then how can it get done?
I am sorry that you feel that way. We don't allocate coding work to people in this process. That's not the way it works.
Rather, people help in building the PEPs by providing their own feedback/experience.
At the end, we are trying to have PEPs that adresses the wider range of cases.
Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway?
I'm a fairly proficient develper, but I can't get assigned a single work item..
We have already prototypes for each PEP so people can try them out, enhance them while the PEPs are being built. If you want to help in their coding, you are more than welcome. They are on a DVCS at bitbucket.
And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items...
Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010..
Again, the big challenge is not on the coding part. If it was, Distutils would have them already. The challenge is on the PEPs, and on making sure we collect all PoVs and feedbacks, before we change Distutils in Python 2.7/3.2
Regards Tarek
David, you have an attitude problem. Your contributions (the post below is just an example) don't sound healthy to me -- you just complain and whine and denigrate Tarek's work. In a previous post you claimed to have had a particular idea first (it doesn't matter which idea) and you managed to make it sound bad that your idea was eventually accepted. This is not a productive attitude.
Surely the problem isn't writing 300 lines of code. The problem is getting everyone to agree on which 300 lines of code should be written. That is the problem at hand, and claiming that nothing happened because no code was written and all that was agreed on amounts to 300 lines of code is outright demeaning.
Stop it. You are wearing out your welcome.
--Guido
PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG.
On Thu, Nov 5, 2009 at 2:29 PM, David Lyon david.lyon@preisshare.net wrote:
On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé ziade.tarek@gmail.com wrote:
PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme.
Perphaps..
But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code...
Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010.
But there's no plan to include a new *distribution* format in Distutils.
I wasn't suggesting that - at all.
And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years.
No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic.
Be fair...
I'm saying finish what is already there.. or take out the crap ..
It isn't fair to suggest that I am somehow asking for some big change when I am simply pointing out all the junk that's in there that's already half built.
In any case those PEPs are not finished yet, so everyone can help at distutils-SIG
True - and False.
But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet.
Seriously, if you won't allocate work out to people then how can it get done?
Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway?
I'm a fairly proficient develper, but I can't get assigned a single work item..
And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items...
Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010..
David
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
Tarek, Guido,
Forgive my grumpy tone..
Looking positive, if we now have a rough consensus that the PEPS might represent some 300+ lines of code... then good - lets get started, that's all that I meant. I'm glad above all, that you noticed that part foremost.
If it's a simple case that alternative's to the solutions cannot be considered - because I am grumpy - whatever, then that's something I can live with. I didn't know that proposing a different solution wasn't a good idea.
PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG.
Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal.
The proposal is in the final stages of being wrapped up.
Here are the discussion References that have occurred over the year:
http://mail.python.org/pipermail/distutils-sig/2009-November/thread.html
November 2009 Archives by thread
* Messages sorted by: [ subject ] [ author ] [ date ] * More info on this list...
Starting: Sun Nov 1 09:31:46 CET 2009 Ending: Fri Nov 6 00:09:22 CET 2009 Messages: 65
* [Distutils] Alternate static metadata PEP submission... Georg Brandl o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe + [Distutils] Alternate static metadata PEP submission... Georg Brandl + [Distutils] Alternate static metadata PEP submission... David Lyon # [Distutils] Alternate static metadata PEP submission... Georg Brandl # [Distutils] Alternate static metadata PEP submission... David Lyon
http://mail.python.org/pipermail/distutils-sig/2009-October/thread.html
# [Distutils] Alternate static metadata PEP submission... David Lyon
* [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Eric Smith o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon * [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau * [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Ian Bicking o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Kevin Teague o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... Piotr Ozarowski o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Piotr Ozarowski o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... Eric Smith o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon
http://mail.python.org/pipermail/distutils-sig/2009-August/thread.html [Distutils] Static metadata using setup.cfg Tarek Ziadé
* [Distutils] Static metadata using setup.cfg Hanno Schlichting * [Distutils] Static metadata using setup.cfg P.J. Eby o [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg Tarek Ziadé * [Distutils] Static metadata using setup.cfg David Lyon o [Distutils] Static metadata using setup.cfg Jeff Rush + [Distutils] Static metadata using setup.cfg Ben Finney + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg David Lyon + [Distutils] Static metadata using setup.cfg Eric Smith + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg Chris Withers + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg Chris Withers * [Distutils] Static metadata using setup.cfg David Lyon o [Distutils] Static metadata using setup.cfg Sridhar Ratnakumar + [Distutils] Static metadata using setup.cfg David Lyon o [Distutils] Static metadata using setup.cfg Alex Grönholm + [Distutils] Static metadata using setup.cfg David Lyon + [Distutils] Static metadata using setup.cfg Floris Bruynooghe + [Distutils] Static metadata using setup.cfg Ronald Oussoren + [Distutils] Static metadata using setup.cfg P.J. Eby + [Distutils] Static metadata using setup.cfg Chris Withers o [Distutils] Static metadata using setup.cfg Chris Withers + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg Chris Withers + [Distutils] Static metadata using setup.cfg Tarek Ziadé + [Distutils] Static metadata using setup.cfg David Lyon + [Distutils] Static metadata using setup.cfg Tarek Ziadé
http://mail.python.org/pipermail/distutils-sig/2009-July/thread.html
# [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon # [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) P.J. Eby
* [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Eric Smith * [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Tres Seaver o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] Questions about virtual-python on ubuntu David Lyon o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Paul Moore o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure Jeff Rush # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure P.J. Eby # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure Jean-Paul Calderone # [Distutils] Setup.py as a Class, not a Procedure David Lyon # [Distutils] Setup.py as a Class, not a Procedure Jean-Paul Calderone # [Distutils] Setup.py as a Class, not a Procedure David Lyon [Distutils] introspecting package data Chris Withers
http://mail.python.org/pipermail/distutils-sig/2009-June/thread.html
* [Distutils] introspecting package data Tarek Ziadé o [Distutils] introspecting package data Chris Withers + [Distutils] introspecting package data Jim Fulton + [Distutils] introspecting package data Chris Withers + [Distutils] introspecting package data Tarek Ziadé + [Distutils] introspecting package data David Lyon
On Thu, 5 Nov 2009 15:18:12 -0800, Guido van Rossum guido@python.org wrote:
David, you have an attitude problem. Your contributions (the post below is just an example) don't sound healthy to me -- you just complain and whine and denigrate Tarek's work. In a previous post you claimed to have had a particular idea first (it doesn't matter which idea) and you managed to make it sound bad that your idea was eventually accepted. This is not a productive attitude.
Surely the problem isn't writing 300 lines of code. The problem is getting everyone to agree on which 300 lines of code should be written. That is the problem at hand, and claiming that nothing happened because no code was written and all that was agreed on amounts to 300 lines of code is outright demeaning.
Stop it. You are wearing out your welcome.
--Guido
PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG.
On Thu, Nov 5, 2009 at 2:29 PM, David Lyon david.lyon@preisshare.net wrote:
On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé ziade.tarek@gmail.com wrote:
PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme.
Perphaps..
But if you put all these PEPs together, implementing all the new
features
can't come to more than 300 lines of code...
Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010.
But there's no plan to include a new *distribution* format in
Distutils.
I wasn't suggesting that - at all.
And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years.
No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic.
Be fair...
I'm saying finish what is already there.. or take out the crap ..
It isn't fair to suggest that I am somehow asking for some big change when I am simply pointing out all the junk that's in there that's already half built.
In any case those PEPs are not finished yet, so everyone can help at distutils-SIG
True - and False.
But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet.
Seriously, if you won't allocate work out to people then how can it get done?
Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway?
I'm a fairly proficient develper, but I can't get assigned a single work item..
And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items...
Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010..
David
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
2009/11/6 David Lyon david.lyon@preisshare.net: [..]
Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal.
The proposal is in the final stages of being wrapped up.
There are three different topics :
1. a standard for the installation format: PEP 376 2. changes in the metadata format : PEP 345 (-- work in progress here, some parts are still missing) 3. ways to built the metadata that are shipped into a distribution statically, using Distutils : PEP 390 was started for that
And as a matter of fact, PEP 390 is not so important, because at the end, the changes that are made in PEP 345 are the most important ones. They allow describing metadata fields that can vary depending on the target platform.
PEP 390 is just the simplest way to make it possible to describe static metadata using a setup.cfg ConfigParser section, rather than some code in setup.py. And when we built it, it helped us understand what we really wanted on PEP 345 side (with a big insights from people like Marc-André on this)
But PEP 390 limits its scope to the Metadata fields and now waits for PEP 345 changes.
Your proposal is a partial alternative to PEP 390, and is a quite interesting work to dig in, and the SIG is discussing it, but a very important point about it is that it does a lot more. It's a new *building* system on its own, and does not limit itself to describing metadata fields.
That's fine, and that's quite interesting, but I doubt we will be able to push it in Distutils itself any sooner, because it a whole different system ala scons (which is a great tool don't get me wrong). I think (as I told you before IIRC) that you need to create a project to demonstrate it and I think it can be great to continue getting some help from Distutils-SIG for this.
Furthermore, building such a tool on the top of Distutils can improve Distutils itself, because it would be yet another project that would provide valuable feedback on the APIs and standards we provide. That's for example what we are getting from projects like PyPM or Enthought Enstaller.
As a matter of fact, there's already great feedback from Active State on PEP 376 because they want to be early adopters of the upcoming standard and make sure we do the right choices in there. Enthought also started to use the prototype we have written for PEP 386 (version comparisons) IIRC.
But in the existing PEPs discussions, we need to focus on their respective scopes. And they are all targeted to improve the existing stdlib tool : Distutils. (and a wanted side effect is to make Distutils smaller if we can, and a robust basis for all third party tools)
That's why you can't just drop a counter-PEP to any existing PEP we have started for Distutils. Imho help us build those PEPs, and / or create your own build system, wether its based or not on Distutils.
Regards Tarek
Tarek,
Your proposal is a partial alternative to PEP 390, and is a quite interesting work to dig in, and the SIG is discussing it, but a very important point about it is that it does a lot more. It's a new *building* system on its own, and does not limit itself to describing metadata fields.
Actually no, the proposal is only for a metadata installation of packages that addresses the security problem of requiring users to run untrusted and
unverified setup.py's.
That was where this discussion originated and that is what I am addressing.
PEP-390 doesn't go there at all...
My proposal uses the static metadata contained within existing package formats (PKG_INFO + sources.txt) and therefore doesn't require any changes to the build system.
There isn't a new build system. I never proposed that.
However PEP-390 does imply changing the build system.
That's why you can't just drop a counter-PEP to any existing PEP we have started for Distutils. Imho help us build those PEPs, and / or create your own build system, wether its based or not on Distutils.
Once again, I can't see why all the fuss.
All I did was have a discussion on distutils-ml about a different way of specifying package dependencies that might work across different python platforms.
It was suggested to me that I do a PEP..
So I took the documentation on face value that developers could submit a PEP-proposal.
Imho help us build those PEPs,
This could best apply to PEP-345:
(PKG_INFO) Requires: sys Requires(python <= 2.4): lxml Requires(windows): win32com Requires(linux): pyodbc Requires(linux ubuntu gnome python <= 3.4): gix Requires(windows xp python < 2.2): win32prn
Thank's for reading my alternate-metadata installation proposal and I accept your feedback. What I can do is clarify that my proposal is not for a build system but for a metadata installation scheme based on a static setup.py, existing metadata and use of "python -m setup install" for invocation.
I'm just trying to make this stuff no more complicated than it needs to be...
David
On Thu, Nov 5, 2009 at 15:55, David Lyon david.lyon@preisshare.net wrote:
Tarek, Guido,
Forgive my grumpy tone..
Looking positive, if we now have a rough consensus that the PEPS might represent some 300+ lines of code... then good - lets get started, that's all that I meant. I'm glad above all, that you noticed that part foremost.
If it's a simple case that alternative's to the solutions cannot be considered - because I am grumpy - whatever, then that's something I can live with. I didn't know that proposing a different solution wasn't a good idea.
That is passive-aggressive and uncalled for. Proposing an opposing view is fine when done respectfully which you have not done here on python-dev.
PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG.
Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal.
That's fine, but it's Tarek's call and as a group we reached the conclusion that Tarek is doing a fine job and is being more than fair. It has been stated as such and yet you keep pushing us like we have not heard you. We did hear you the first time, but you just seem to not get the message. Adding to the fact that your tone has been an issue and that we don't want to listen to it anymore has pushed Guido to the brink and me over the edge. I honestly don't know how Tarek keeps his patience over on the distutils SIG if he has to deal with kind of crap constantly.
Personally, I am now sending your emails to the trash since you can't seem to even reply to Guido's email w/o some nasty undertone. Hopefully someday you can learn to communicate in a better, friendlier manner.
-Brett
participants (9)
-
Brett Cannon -
David Lyon -
Fred Drake -
Guido van Rossum -
M.-A. Lemburg -
Michael Foord -
Robert Kern -
ssteinerX＠gmail.com -
Tarek Ziadé