Mailman 3 Packaging situation + mailing list rules - Distutils-SIG

newer
Re: [Distutils] How to force...

Packaging situation + mailing list rules

older
Re: [Distutils] Packaging...

Tarek Ziadé

2 Jul 2010 2 Jul '10

10:08 a.m.

Hello,

...

From time to time this mailing list is getting very unpleasant to work in because some old disagreements, and because some people are starting to get really nasty.

Here's a reminder of the current packaging situation, and a few rules I suggest, for the benefit of all 1. Python <= 2.7 contains Distutils 2. Distutils is frozen, and won't evolve anymore 3. Setuptools is built on Distutils 4. Distribute forks Setuptools for various reasons. If you disagree with this choice, there's no need to send a mail here, because this is done. You can send me a private mail, but don't send a mail here because things are always getting nasty afterwards. Make sure to read the archives first, to understand why it was done. 5. Debian, Ubuntu and Fedora are using Distribute rather than Setuptools. If you find differences in the behavior, you can send a mail in this list, so we can fix Distribute. If you disagree with the distros choice, there's no need to send a mail here, because this is done. You can send *them* a mail, or tell them you disagree with this choice. Of course, the most effective behavior would be to work on having Distribute fixed if you find an issue, but that's up to you. 6. Distutils2 is being built, and has its own mailing list now. Distutils2 will replace Distutils in Python 3.2 or 3.3. It' s built on the various PEPs that were accepted lately, and any help is welcome.. The list is here: http://groups.google.fr/group/the-fellowship-of-the-packaging, and is moderated. 7. New members of the lists will not know the situation, so they won't be able to follow these rules of course. But people that are hanging around here for years can apply them. Of course this is just a series of suggestion. You can disagree with all these rules if you want, but I think they need to be applied in this mailing-list for the benefit of all. Thanks, Tarek -- Tarek Ziadé | http://ziade.org

Show replies by date

anatoly techtonik

2 Jul 2 Jul

12:09 p.m.

Great post, Tarek. Following good old newsgroups/FIDOnet tradition it could be nice to see this transformed to Rules/FAQ document that will be reposted automatically here by a robot about once a month. Without such documents your proposal will be weakly supported, because people will still have questions, and you will need to answer them reasonably to eliminate the source of conflict for making collaboration moving into the right direction (which you also need to define). 1. Why the rules?

...

From time to time this mailing list is getting very unpleasant to work in because some old disagreements, and because some people are starting to get really nasty.

2. What are those 'disagreements' people can agree upon? 3. "Python <= 2.7 contains Distutils" - is there anything wrong with that? 4. "Distutils is frozen, and won't evolve anymore" - mmm, ahem.. well.. ok. 5. But you know - I am somewhat familiar with distutils code - it is much easier for me to continue patching it than start using something that will only appear in some distant 3k Python I have no desire to play with now. I need to solve real world problems and these are all with Python 2.x 6. Why Distribute forks Setuptools? It is said `for various reasons`, but I can't get it.

...

There's no need to send a mail here, because this is done. If you send a mail here expect things become nasty.

7. But I want to know!

...

No you don't.

8. Nooooo. I want!

...

See. It is already starting..

9. But I want to know it. :-(

...

Make sure to read the archives first, to understand why it was done.

10. Maan, that's too long, pydotorg archives suxx - may I have a summary?

...

Well, does anybody want to post recent Shakespeare script here?

11. It is said that people may disagree that Debian, Ubuntu and Fedora are using Distribute? But why?? 12. BTW, is Setuptools still alive? 13. What are those Distribute, Setuptools and Distutils2 anyway? After this FAQ it would be really nice to see editable draft of Roadmap/Status document that people can get for an overview of the current state and choose the tasks they feel most important for further collaboration. P.S. http://groups.google.com/group/the-fellowship-of-the-packaging is more international than http://groups.google.fr/group/the-fellowship-of-the-packaging -- anatoly t.

David Cournapeau

1:04 p.m.

On Fri, Jul 2, 2010 at 9:09 PM, anatoly techtonik wrote:

...

Great post, Tarek. Following good old newsgroups/FIDOnet tradition it could be nice to see this transformed to Rules/FAQ document that will be reposted automatically here by a robot about once a month.

Without such documents your proposal will be weakly supported, because people will still have questions, and you will need to answer them reasonably to eliminate the source of conflict for making collaboration moving into the right direction (which you also need to define).

1. Why the rules?

...
From time to time this mailing list is getting very unpleasant to work in because some old disagreements, and because some people are starting to get really nasty.

2. What are those 'disagreements' people can agree upon?

I think the following in uncontroversial: distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

...

11. It is said that people may disagree that Debian, Ubuntu and Fedora are using Distribute? But why??

Nobody disagrees about the use of distribute. That's open source, and one of the point of open source is to let people do what they want, and nobody should be prevented to use distribute because someone does not like it. But that works both ways - if I do not want to use distribute, I should not be forced to use it because someone else decided it was good for me. David

Tarek Ziadé

1:31 p.m.

2010/7/2 David Cournapeau :

...

On Fri, Jul 2, 2010 at 9:09 PM, anatoly techtonik wrote:

...
Great post, Tarek. Following good old newsgroups/FIDOnet tradition it could be nice to see this transformed to Rules/FAQ document that will be reposted automatically here by a robot about once a month.

Without such documents your proposal will be weakly supported, because people will still have questions, and you will need to answer them reasonably to eliminate the source of conflict for making collaboration moving into the right direction (which you also need to define).

1. Why the rules?

...
From time to time this mailing list is getting very unpleasant to work in because some old disagreements, and because some people are starting to get really nasty.

2. What are those 'disagreements' people can agree upon?

I think the following in uncontroversial:

distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

You keep saying that for years, but in the meantime, the code was cleaned. And it's perfectly manageable. We have 5 GSOC students working on it right now and they don't seem to have so much problems in changing/reading the code. And now, it's being rewritten in distutils2 in a backward incompatible way. So if you don't like some things in the design, you can say it out loud and make some constructive proposals if you want.

...

...
11. It is said that people may disagree that Debian, Ubuntu and Fedora are using Distribute? But why??

Nobody disagrees about the use of distribute. That's open source, and one of the point of open source is to let people do what they want, and nobody should be prevented to use distribute because someone does not like it. But that works both ways - if I do not want to use distribute, I should not be forced to use it because someone else decided it was good for me.

If you use Ubuntu, they make some choices for you. For instance Distutils is patched in Ubuntu, and you don't get the same version that one you would get in a plain Python. Package are installed in a specific location etc. So if you want to have the very same version for all packages in your Linux, and have it behaving exactly like the initial author of each package wanted, you should install a low-level distro like Linux From Scratch, and build the environment you like. So yes, it's open source: you can install a distro and adopt its choices, or change it, or miltate to have other choices. Or you can use another distro.

David Cournapeau

1:42 p.m.

On Fri, Jul 2, 2010 at 10:31 PM, Tarek Ziadé wrote:

...

2010/7/2 David Cournapeau :

...
On Fri, Jul 2, 2010 at 9:09 PM, anatoly techtonik wrote:

...
Great post, Tarek. Following good old newsgroups/FIDOnet tradition it could be nice to see this transformed to Rules/FAQ document that will be reposted automatically here by a robot about once a month.

Without such documents your proposal will be weakly supported, because people will still have questions, and you will need to answer them reasonably to eliminate the source of conflict for making collaboration moving into the right direction (which you also need to define).

1. Why the rules?

...
From time to time this mailing list is getting very unpleasant to work in because some old disagreements, and because some people are starting to get really nasty.

2. What are those 'disagreements' people can agree upon?

I think the following in uncontroversial:

distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

You keep saying that for years, but in the meantime, the code was cleaned.

I was just summarizing the situation to answer the original question from the OP. There was absolutely no judgement in the text I have written. David

Tarek Ziadé

2 p.m.

On Fri, Jul 2, 2010 at 3:42 PM, David Cournapeau wrote: [..]

...

...
...
I think the following in uncontroversial:

distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

You keep saying that for years, but in the meantime, the code was cleaned.

I was just summarizing the situation to answer the original question from the OP. There was absolutely no judgement in the text I have written.

You are judging that distutils code is unmanageable and too complicated, and stating that this is an uncontroversial statement about the current situation. This would have perfectly answered to the question 3 years ago, but this is not the reality anymore. So I am just describing the situation as it is in reality, because I know exactly the status of Distutils.

David Cournapeau

2:05 p.m.

On Fri, Jul 2, 2010 at 11:00 PM, Tarek Ziadé wrote:

...

On Fri, Jul 2, 2010 at 3:42 PM, David Cournapeau wrote: [..]

...
...
...
I think the following in uncontroversial:

distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

You keep saying that for years, but in the meantime, the code was cleaned.

I was just summarizing the situation to answer the original question from the OP. There was absolutely no judgement in the text I have written.

You are judging that distutils code is unmanageable and too complicated, and stating that this is an uncontroversial statement about the current situation.

This is not what I said. The judgement you mention was clearly stated as my own opinion, not as an uncontroversial point. David

Tarek Ziadé

2:16 p.m.

On Fri, Jul 2, 2010 at 4:05 PM, David Cournapeau wrote:

...

On Fri, Jul 2, 2010 at 11:00 PM, Tarek Ziadé wrote:

...
On Fri, Jul 2, 2010 at 3:42 PM, David Cournapeau wrote: [..]

...
...
...
I think the following in uncontroversial:

distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

You keep saying that for years, but in the meantime, the code was cleaned.

I was just summarizing the situation to answer the original question from the OP. There was absolutely no judgement in the text I have written.

You are judging that distutils code is unmanageable and too complicated, and stating that this is an uncontroversial statement about the current situation.

This is not what I said. The judgement you mention was clearly stated as my own opinion, not as an uncontroversial point.

maybe so, but we need an answer with facts that are not mixing opinions. e.g. : "distutils2 is built with distutils code in a backward incompatible way"

David Cournapeau

2:18 p.m.

On Fri, Jul 2, 2010 at 11:16 PM, Tarek Ziadé wrote:

...

On Fri, Jul 2, 2010 at 4:05 PM, David Cournapeau wrote:

...
On Fri, Jul 2, 2010 at 11:00 PM, Tarek Ziadé wrote:

...
On Fri, Jul 2, 2010 at 3:42 PM, David Cournapeau wrote: [..]

...
...
...
I think the following in uncontroversial:

distutils and setuptools are useful packaging solutions which have significant shortcoming, both design and implementation-wise. Some people believe the distutils/setuptools/distribute issues can be solved by gradually deprecating code and adding new features, other people (me, but I am not alone) believe it would be better and faster to rewrite something from scratch because the distutils code is unmanageable and too complicated.

You keep saying that for years, but in the meantime, the code was cleaned.

I was just summarizing the situation to answer the original question from the OP. There was absolutely no judgement in the text I have written.

You are judging that distutils code is unmanageable and too complicated, and stating that this is an uncontroversial statement about the current situation.

This is not what I said. The judgement you mention was clearly stated as my own opinion, not as an uncontroversial point.

maybe so, but we need an answer with facts that are not mixing opinions. e.g. :

"distutils2 is built with distutils code in a backward incompatible way"

And how does this answer the question "what are the disagreements" ? Short of saying what those are, I fail to see how to give a good answer, David

Tarek Ziadé

2:29 p.m.

On Fri, Jul 2, 2010 at 4:18 PM, David Cournapeau wrote:

...

And how does this answer the question "what are the disagreements" ? Short of saying what those are, I fail to see how to give a good answer,

I am not sure to understand your point here. You stated two years ago, IIRC, that distutils code was unmanageable and too complicated, and that a write from scracth was better. Since then, Guido has decided that the new distutils would be built in a new namespace, with no backward compatibility in the APIs. I think that partially gives you what you wanted: we can break everything in it for the new version. Last, the code was cleaned up, pep8-fied, etc and 5 GSOC students are working in it. So what is your disagreement on the distutils devenir precisely ? Either you work on distutils2, and propose big changes etc. (we welcome you, as I said manby times) Either you work on your side, and propose an alternative solution, but there's no point to criticize what the distutils team is doing but let's not mix things, the disagreements between setuptools and distribute is a different issue than what we are doing in distutils.

David Cournapeau

3:05 p.m.

On Fri, Jul 2, 2010 at 11:29 PM, Tarek Ziadé wrote:

...

On Fri, Jul 2, 2010 at 4:18 PM, David Cournapeau wrote:

...
And how does this answer the question "what are the disagreements" ? Short of saying what those are, I fail to see how to give a good answer,

I am not sure to understand your point here. You stated two years ago, IIRC, that distutils code was unmanageable and too complicated, and that a write from scracth was better.

I am sorry if I gave the impression I wanted to start again this discussion. I really do not want to, and I don't think anyone else does either. Eric stated very cleary what was my intention. As for where distutils2 is going, we (me and other scipy people) have stated why we *believed* (and still do) it was a wrong way when Guido asked about it in the context of scipy, and I don't remember having mentioned my disagreement over this ever since. It is pretty obvious you won't change your mind, and I won't change mine either. You will notice that I have almost never mentioned my own project on this mailing list to avoid giving the impression I wanted to circumvent what you are doing, I explicitly requested not to send email about it on this ML for that exact reason. And I still give what I believe are useful advices from time to time for people using distutils/setuptools on this ML (without telling them: use bento instead). David

Tarek Ziadé

3:54 p.m.

On Fri, Jul 2, 2010 at 5:05 PM, David Cournapeau wrote:

...

On Fri, Jul 2, 2010 at 11:29 PM, Tarek Ziadé wrote:

...
On Fri, Jul 2, 2010 at 4:18 PM, David Cournapeau wrote:

...
And how does this answer the question "what are the disagreements" ? Short of saying what those are, I fail to see how to give a good answer,

I am not sure to understand your point here. You stated two years ago, IIRC, that distutils code was unmanageable and too complicated, and that a write from scracth was better.

I am sorry if I gave the impression I wanted to start again this discussion. I really do not want to, and I don't think anyone else does either. Eric stated very cleary what was my intention.

Ok, I am also sorry if I understood it the wrong way.

...

As for where distutils2 is going, we (me and other scipy people) have stated why we *believed* (and still do) it was a wrong way when Guido asked about it in the context of scipy, and I don't remember having mentioned my disagreement over this ever since. It is pretty obvious you won't change your mind, and I won't change mine either.

This is precisely where I don't understand. When this happened, I've asked you what should be done to improve, and you gave a few ideas on the build system. We are building it *now*, and *you* can make distutils2 goes where you want it to go.. As a matter of fact there's a student (Eric) working on this, and it follows your idea of separating the build configuration from other commands. It would have been great to have your guidance in this. Now if you want to work on a new project, that's perfectly fine, and even more fun. And that's good for distutils because at some point we could work on the same standards. But don't say it's because we disagree, because you could have what you want in distutils2 today..

David Cournapeau

3 Jul 3 Jul

12:13 a.m.

On Sat, Jul 3, 2010 at 12:54 AM, Tarek Ziadé wrote:

...

On Fri, Jul 2, 2010 at 5:05 PM, David Cournapeau wrote:

...
On Fri, Jul 2, 2010 at 11:29 PM, Tarek Ziadé wrote:

...
On Fri, Jul 2, 2010 at 4:18 PM, David Cournapeau wrote:

...
And how does this answer the question "what are the disagreements" ? Short of saying what those are, I fail to see how to give a good answer,

I am not sure to understand your point here. You stated two years ago, IIRC, that distutils code was unmanageable and too complicated, and that a write from scracth was better.

I am sorry if I gave the impression I wanted to start again this discussion. I really do not want to, and I don't think anyone else does either. Eric stated very cleary what was my intention.

Ok, I am also sorry if I understood it the wrong way.

...
As for where distutils2 is going, we (me and other scipy people) have stated why we *believed* (and still do) it was a wrong way when Guido asked about it in the context of scipy, and I don't remember having mentioned my disagreement over this ever since. It is pretty obvious you won't change your mind, and I won't change mine either.

This is precisely where I don't understand.

Here is my understanding of what happened: as we (we being at least a couple of major maintainers in the numpy community) understood it 6 months ago in the "we want CPAN" thread started by Guido, the idea was to gradually evolve from the existing code to whatever distutils2 aims up to be. Because several attempts have been made to do so by proeminent numpy developers in the last decade, we have the *experience* that this could not work, at least for us. Certainly, there was a failure from us to communicate what's wrong with current distutils design, but the situation is still the same today: absolutely none of the thing that distribute2 is working on is an improvement for us. None of the recent PEP addresses any significant issue for us. We don't care about pypi as is, we don't care about complex requirement/versioning, we don't care about uninstall, etc... We simply do not agree in any significant way on the distutils issues. If you look at bento goals, they do not even attempt to solve any issue raised in the recent PEP. And again, I am not saying this to criticize your work (or anyone's else). Surely, I cannot complain that you or someone's else is not working on my issues. But I think it is a good reason to go on our separate ways to implement a solution, at least for now. David

Tarek Ziadé

12:53 a.m.

On Sat, Jul 3, 2010 at 2:13 AM, David Cournapeau wrote: [..]

...

...
This is precisely where I don't understand.

Here is my understanding of what happened: as we (we being at least a couple of major maintainers in the numpy community) understood it 6 months ago in the "we want CPAN" thread started by Guido, the idea was to gradually evolve from the existing code to whatever distutils2 aims up to be. Because several attempts have been made to do so by proeminent numpy developers in the last decade, we have the *experience* that this could not work, at least for us.

All these attempts were doomed to fail, because of the backward compatibility issues. I was hit by that too. But this barrier is gone, because we are working on a new namespace. Which means that we can build whatever we want now in distutils2. And the build process is the part that can be entirely redone. But you didn't answer on that. This was your main critic against distutils. ISTM that this is precisely what we could improve wrt the usage of distutils in sci

...

Certainly, there was a failure from us to communicate what's wrong with current distutils design, but the situation is still the same today: absolutely none of the thing that distribute2 is working on is an improvement for us. None of the recent PEP addresses any significant issue for us. We don't care about pypi as is, we don't care about complex requirement/versioning, we don't care about uninstall, etc... We simply do not agree in any significant way on the distutils issues. If you look at bento goals, they do not even attempt to solve any issue raised in the recent PEP.

This is not entirely true, as far as I understand your work. For example, we are currently removing setup.py in favor of 100% static definitions using setup.cfg, thanks to PEP 345 / 390. (Bento seems to have similar static files now) Overall, I am curious to know what are your issues, if it not about the building process and the definition of metadata.

...

And again, I am not saying this to criticize your work (or anyone's else). Surely, I cannot complain that you or someone's else is not working on my issues. But I think it is a good reason to go on our separate ways to implement a solution, at least for now.

I know you want to work on your side, and I am not criticizing that. I am just very skeptical that your issues are so different from the issues we are working in distutils2.

David Cournapeau

1:33 a.m.

On Sat, Jul 3, 2010 at 9:53 AM, Tarek Ziadé wrote:

...

Overall, I am curious to know what are your issues, if it not about the building process and the definition of metadata.

Without going into the technical details, it really boils down to separate the concerns of the different parts of a packaging solution, and having clear interface between them. For example, it is *possible* to integrate a new build system within distutils. I know this for a fact because I have done so (numpy and scipy can be entirely built with scons instead of distutils for the C/C++/Fortran extensions). But it was an awful experience.

...

I am just very skeptical that your issues are so different from the issues we are working in distutils2.

Yes, other people have stated this as well. On the other hand, what I am doing with bento "feels" obvious within the numpy/scipy community as far as I can tell. I think it mostly boils down to different backgrounds and different concerns. If you were building/developing complex C extensions daily, you would understand our issues much better from experience, and I guess I would understand better what distutils2 is about if I were doing web development. There is also a strong disagreement on how to do deployment - it has been stated that pypi current philosophy of not enforcing metadata policy will stay in place, whereas I am absolutely convinced it is the root of most pypi issues. On this, we will only ever be able to agree on disagreeing I am afraid. David

Éric Araujo

2 Jul 2 Jul

2:32 p.m.

*puts linguist apprentice hat on* David’s message was saying it was uncontroversial that there was a controversy. So his judgment *was* stated as his own, but there was a parsing ambiguity (is the part after “because” a fact or still the opinion?) that tripped Tarek. *puts hat off* Regards

Greg Ewing

3 Jul 3 Jul

1:53 a.m.

Tarek Ziadé wrote:

...

4. Distribute forks Setuptools for various reasons. If you disagree with this choice, there's no need to send a mail here

David isn't disagreeing with that choice, as far as I can see. The *only* thing he's saying is that the fork should have a different name, because it's a different thing. I think he has a valid point. -- Greg

5045

Age (days ago)

5046

Last active (days ago)

List overview

Download

16 comments

5 participants

participants (5)

anatoly techtonik
David Cournapeau
Greg Ewing
Tarek Ziadé
Éric Araujo

Packaging situation + mailing list rules

tags

participants (5)