Re: [stdlib-sig] Breaking out the stdlib
In a message of Tue, 15 Sep 2009 14:38:20 -0000, Collin Winter writes: <stuff that indicates that my users shouldn't be changing versions of Python without going through some sort of migration process> This is what happens every time /usr/bin/env python changes. <lots of stuff indicating that optparse really should be removed on the grounds that it is poorly written, still buggy, and unmaintained> Ok, so it's type B-2, the maintainer's nightmare. It probably fits in the category of 'things that never should be in the library in the first place'. It's an embarassment. I notice that you aren't talking about getopt, though. And there may be things that are so badly written that we actually want to throw them out of the standard library before people start using them. A note 'this module only exists for backwards compatibility, we recommend that you use XXX instead' will not be good enough for the purpose, we really want to stamp out the use of this mistake as soon and as well as possible. This could happen. But, as much as I dislike optparse, I don't think that this is the module where that happened. I will be happy when people stop using it, but I don't think that using it is so hazardous that we should force all the optparse users to rewrite all their code. final quote:
To speak more personally, and specifically to the issue of getopt/optparse vs argparse: at Google, I'm part of the Python readability team, which helps train the large numbers of Python developers that the company produces. Part of this job involves conducting detailed code reviews for new Python programmers, explaining both Google style and idiomatic Python code generally, suggesting library A over hand-written solution B. I am, frankly, embarrassed whenever I have to explain the difference between getopt and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess, string.* vs str.*, etc. I cannot imagine how embarrassed I will be when I have to explain why the standard library includes getopt, optparse and argparse.
You need to practice saying 'argparse is the preferred way to do things; getopt and optparse are maintained for compatibility reasons' until you can do this without embarassment. It also sounds to me as if you would be far less embarassed explaining to Google employees why they have to go off and rewrite all their old code which uses getopt than explaining to them why we have getopt, optparse, and argparse in the standard library. I find this very odd. Laura
Collin Winter
On Tue, Sep 15, 2009 at 1:02 PM, Laura Creighton <lac@openend.se> wrote:
In a message of Tue, 15 Sep 2009 14:38:20 -0000, Collin Winter writes:
<stuff that indicates that my users shouldn't be changing versions of Python without going through some sort of migration process>
This is what happens every time /usr/bin/env python changes.
I do not understand. Why is "/usr/bin/env python" changing so frequently on these systems? Why is it being changed without consideration of the requirements of the software running on top of it?
final quote:
To speak more personally, and specifically to the issue of getopt/optparse vs argparse: at Google, I'm part of the Python readability team, which helps train the large numbers of Python developers that the company produces. Part of this job involves conducting detailed code reviews for new Python programmers, explaining both Google style and idiomatic Python code generally, suggesting library A over hand-written solution B. I am, frankly, embarrassed whenever I have to explain the difference between getopt and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess, string.* vs str.*, etc. I cannot imagine how embarrassed I will be when I have to explain why the standard library includes getopt, optparse and argparse.
You need to practice saying 'argparse is the preferred way to do things; getopt and optparse are maintained for compatibility reasons' until you can do this without embarassment.
I am embarrassed explaining why the core Python team, of which I am a member, would consent to indefinitely maintain multiple, separate, mutually-incompatible systems that all have the same purpose without deprecating any of them. I do not understand why getopt was not deprecated when optparse was introduced seven years ago; perhaps someone can shed historical perspective on that decision. Pending that elaboration, I do not understand why we would repeat that mistake again with argparse now, or with some other module in the future.
It also sounds to me as if you would be far less embarassed explaining to Google employees why they have to go off and rewrite all their old code which uses getopt than explaining to them why we have getopt, optparse, and argparse in the standard library. I find this very odd.
I don't find "we found a better way" embarrassing. I don't find having a single, unified, best-of-breed solution embarrassing. I don't find having multiple years of advance notice to update your code embarrassing (which is vastly longer than the deprecation windows we have internally). I don't find consolidating developer time, energy and attention embarrassing. I don't find progress embarrassing. The commonly-expressed idea behind the stdlib is that it represents best-of-breed code: it should have an independent userbase first, it should have proven itself in the wild before it is allowed into the stdlib. "Best" comes with baggage: it means that other solutions are worse, and it admits the possibility that other code will someday overtake the current solution to *become* best-of-breed. That was the motivation then for including optparse: it was a more Pythonic way of doing argument parsing, as opposed to getopt's faithful recreation of C's limitations and restrictions. That is the motivation now for including argparse: optparse is inflexible and doesn't support useful functionality. We have to be open to the idea that we may have found a new "best". Collin Winter
Le mardi 15 septembre 2009 à 13:48 -0400, Collin Winter a écrit :
The commonly-expressed idea behind the stdlib is that it represents best-of-breed code: it should have an independent userbase first, it should have proven itself in the wild before it is allowed into the stdlib.
Asking libraries to be proven in the wild sounds like a good idea, but it promotes disruptive changes (replacing a module with another one) rather than evolutionary changes. It also means that we get potentially bloated or bizarrely idiomatic packages, because they weren't subject to the same standards of simplicity / austerity that we value in the stdlib (multiprocessing comes to mind, doesn't it? how much time did we lose because of its byzantine API and implementation? how robust and maintainable is it, even now, although the original PyPI package was acclaimed?). Therefore, I think it shouldn't be imposed as a general rule, rather a guideline. For example, Michael's work on unittest (a work of evolutionary changes) is much better news, IMO, that someone proposing to include nose in the stdlib. And I say that as a nose user. I don't want to maintain nose in the stdlib.
"Best" comes with baggage: it means that other solutions are worse, and it admits the possibility that other code will someday overtake the current solution to *become* best-of-breed.
Of course. But it is also a double-sided argument because, if each new package remains the best-of-breed during only 2 years after it is integrated into the stdlib, we will spend a lot of time considering new packages for inclusion, deprecating other ones, and ultimately confusing the hell of our users. Regards Antoine.
Antoine Pitrou wrote:
[snip...] For example, Michael's work on unittest (a work of evolutionary changes) is much better news, IMO, that someone proposing to include nose in the stdlib. And I say that as a nose user. I don't want to maintain nose in the stdlib.
Evolving an existing library, as a rule, is definitely better than replacing it with a new one. There is a cost involved in removing a library. It isn't always possible though to meet requirements with an existing API - as is the case with optparse / argparse.
"Best" comes with baggage: it means that other solutions are worse, and it admits the possibility that other code will someday overtake the current solution to *become* best-of-breed.
Of course. But it is also a double-sided argument because, if each new package remains the best-of-breed during only 2 years after it is integrated into the stdlib, we will spend a lot of time considering new packages for inclusion, deprecating other ones, and ultimately confusing the hell of our users.
The bar for including a module in the standard library is high (which is where the best of breed requirement comes from) because *we* do commit to maintain it. That doesn't mean we commit to maintaining things forever though (at least I can't find that promise in the documentation anywhere...). Flipping modules every two years would of course be ridiculous - but we still don't guarantee that any module will remain forever. I don't think I would go as far as Collin hinted (without necessarily meaning anything so radical) in maintaining the standard library as a constantly changing "current best of breed", but I agree with his other sentiments. Michael
Regards
Antoine.
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
2009/9/15 Michael Foord <michael@voidspace.org.uk>:
Evolving an existing library, as a rule, is definitely better than replacing it with a new one. There is a cost involved in removing a library. It isn't always possible though to meet requirements with an existing API - as is the case with optparse / argparse.
MAL pointed out http://code.activestate.com/recipes/573441/ - extended optparse to allow definition of required options. Given that one of the requirements that argparse is claimed to meet where optparse doesn't is supporting required arguments, how come this simple recipe hasn't been incorporated into optparse? The optparse/argparse case seems to rest on the argument that optparse cannot be extended to do what argparse does. It seems like this isn't true for all requirements. (And maybe some others could be addressed by judiciously deprecating support for specific internal details that maybe should not have been documented in the first place...) Paul.
On Sep 15, 2009, at 2:41 PM, Paul Moore wrote:
MAL pointed out http://code.activestate.com/recipes/573441/ - extended optparse to allow definition of required options. Given that one of the requirements that argparse is claimed to meet where optparse doesn't is supporting required arguments, how come this simple recipe hasn't been incorporated into optparse?
That's an excellent question which kind of says something about people's enthusiasm for maintaining optparse, eh? -Barry
Barry Warsaw wrote:
On Sep 15, 2009, at 2:41 PM, Paul Moore wrote:
MAL pointed out http://code.activestate.com/recipes/573441/ - extended optparse to allow definition of required options. Given that one of the requirements that argparse is claimed to meet where optparse doesn't is supporting required arguments, how come this simple recipe hasn't been incorporated into optparse?
That's an excellent question which kind of says something about people's enthusiasm for maintaining optparse, eh?
It says something about the apparent importance of this particular feature in an argument parsing module ;-) Seriously, I wouldn't expect an argument parsing tool to do any complicated option dependency checking for me. That's something you do after having parsed the command line and it's likely different for every single script that needs such checks. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 15 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Tue, 15 Sep 2009 at 21:43, M.-A. Lemburg wrote:
Barry Warsaw wrote:
On Sep 15, 2009, at 2:41 PM, Paul Moore wrote:
MAL pointed out http://code.activestate.com/recipes/573441/ - extended optparse to allow definition of required options. Given that one of the requirements that argparse is claimed to meet where optparse doesn't is supporting required arguments, how come this simple recipe hasn't been incorporated into optparse?
That's an excellent question which kind of says something about people's enthusiasm for maintaining optparse, eh?
It says something about the apparent importance of this particular feature in an argument parsing module ;-)
Actually I believe I heard from someone other than Laura that required options were explicitly rejected. And then there's this from the documentation for optparse: required option an option that must be supplied on the command-line; note that the phrase “required option” is self-contradictory in English. optparse doesn’t prevent you from implementing required options, but doesn’t give you much help at it either. See examples/required_1.py and examples/required_2.py in the optparse source distribution for two ways to implement required options with optparse. --David
Le mardi 15 septembre 2009 à 17:14 -0400, R. David Murray a écrit :
Actually I believe I heard from someone other than Laura that required options were explicitly rejected.
This is one of the reasons why I'm against exclusive module ownership. If a reasonable number of people think a feature would benefit the community, the module owner shouldn't be able to veto it on ideological (or whatever other personal) grounds. In any case, this is not directly an argument against optparse itself, if someone (Armin?) decides to maintain it with a more open attitude.
Antoine Pitrou wrote:
Le mardi 15 septembre 2009 à 17:14 -0400, R. David Murray a écrit :
Actually I believe I heard from someone other than Laura that required options were explicitly rejected.
This is one of the reasons why I'm against exclusive module ownership. If a reasonable number of people think a feature would benefit the community, the module owner shouldn't be able to veto it on ideological (or whatever other personal) grounds.
I dislike exclusive module ownership too. We end up in situations where modules (like ElementTree) are 'owned' by someone who is absent and no-one else is able to (or dares to) touch the code. Michael
In any case, this is not directly an argument against optparse itself, if someone (Armin?) decides to maintain it with a more open attitude.
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
Michael Foord schrieb:
Antoine Pitrou wrote:
Le mardi 15 septembre 2009 à 17:14 -0400, R. David Murray a écrit :
Actually I believe I heard from someone other than Laura that required options were explicitly rejected.
This is one of the reasons why I'm against exclusive module ownership. If a reasonable number of people think a feature would benefit the community, the module owner shouldn't be able to veto it on ideological (or whatever other personal) grounds.
I dislike exclusive module ownership too. We end up in situations where modules (like ElementTree) are 'owned' by someone who is absent and no-one else is able to (or dares to) touch the code.
Well, one part of module ownership would be to complain about and/or revert unwanted changes. Which one obviously cannot do when absent or inactive :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Le mardi 15 septembre 2009 à 23:49 +0200, Georg Brandl a écrit :
Well, one part of module ownership would be to complain about and/or revert unwanted changes.
We don't need ownership for that. Anyone can complain about a change and ask for it to be reverted.
On Tue, Sep 15, 2009 at 6:00 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 23:49 +0200, Georg Brandl a écrit :
Well, one part of module ownership would be to complain about and/or revert unwanted changes.
We don't need ownership for that. Anyone can complain about a change and ask for it to be reverted.
And anyone can complain about an owners decision. I've backed off on things I've not wanted in multiprocessing because of user feedback and compelling use cases. I would expect the same of everyone with +commit. jesse
On Tue, Sep 15, 2009 at 5:22 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Antoine Pitrou wrote:
Le mardi 15 septembre 2009 à 17:14 -0400, R. David Murray a écrit :
Actually I believe I heard from someone other than Laura that required options were explicitly rejected.
This is one of the reasons why I'm against exclusive module ownership. If a reasonable number of people think a feature would benefit the community, the module owner shouldn't be able to veto it on ideological (or whatever other personal) grounds.
I dislike exclusive module ownership too. We end up in situations where modules (like ElementTree) are 'owned' by someone who is absent and no-one else is able to (or dares to) touch the code.
Michael
There is no such thing as "exclusive" ownership, and there can not be. I am advocating for owners in as much as I'd like (like Georg) someone to assign bugs, patches and other things to for a given module. If an owner, such as ElementTree's chooses to be absent forever, or no longer be involved - then they are replaced. We've all(?) worked in business settings and most of us probably understand the "drawbacks" to "exclusive" ownership. Ergo, that's not what I am advocating. However, having someone be the "thought leader, patch reviewer and guy to send angry emails to when something is so broken it causes convulsions" would be nice. jesse
On Tue, Sep 15, 2009 at 5:22 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 17:14 -0400, R. David Murray a écrit :
Actually I believe I heard from someone other than Laura that required options were explicitly rejected.
This is one of the reasons why I'm against exclusive module ownership. If a reasonable number of people think a feature would benefit the community, the module owner shouldn't be able to veto it on ideological (or whatever other personal) grounds.
In any case, this is not directly an argument against optparse itself, if someone (Armin?) decides to maintain it with a more open attitude.
And I can find at least 176 reasons why owners are a good idea: http://bugs.python.org/issue?%40search_text=&title=&%40columns=title&id=&%40columns=id&stage=4&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search There will always be some owners who insist on idealistic purity over serving external users - just look at every PEP that crosses python-dev. However; this is not a case against owners - this is a case against bad owners. The fact is, we need people who feel responsibility for every one of these modules to review patches, and have some amount of mental design integrity to ensure modules don't just wander off into the sunset and die. jesse
Le mardi 15 septembre 2009 à 18:22 -0400, Jesse Noller a écrit :
And I can find at least 176 reasons why owners are a good idea:
Sorry, what's that URL supposed to prove? What does it even represent? It is a populist argument at best, because I won't skim through those 176 bugs to try and make sense of your argument. If you want to make a point, please don't try to leave the burden of proof on me. Actually, I'll just take one of them, because I know it quite well: http://bugs.python.org/issue4967 This is a bug in _ssl for which I had to write a patch, although I knew nothing about _ssl, because the owner wouldn't react. He wouldn't react for review either. The bug *had* to be fixed because it blocked the whole process of including the C IO library. Finally, Benjamin committed the patch. The owner didn't give a sign of life *during the whole process*. He isn't dead, he still sometimes contributes to python-dev, but he was totally unavailable when his presence was needed. So much for owners being a good thing.
The fact is, we need people who feel responsibility for every one of these modules to review patches, and have some amount of mental design integrity to ensure modules don't just wander off into the sunset and die.
But this is not the same as having an owner. Perhaps it wasn't clear, but I draw a clear separation between exclusive owners and maintainers. I'm all for non-exclusive maintainership, with people having reasonable authority over code they understand thoroughly. You taking care of multiprocessing falls into this category (as long as you don't demand to approve of all changes before they are committed). I'm against ownership, however. I'm against mandatory signoffs (imprimaturs), and I'm against the possessive sentiment that some might develop towards a piece of code they contributed. Any core developer should be allowed to commit a patch if he thinks the patch is reasonable enough and serves an useful purpose. Ownership prevents proper maintenance when the owner is absent (which *will* happen, since we are all unpaid volunteers). It discourages other people from getting acquainted with the code, which gradually makes the problem worse. Furthermore, it is often correlated with strange (personal) idioms, coding styles and design principles. Regards Antoine.
On Tue, Sep 15, 2009 at 6:52 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 18:22 -0400, Jesse Noller a écrit :
Sorry, what's that URL supposed to prove? What does it even represent? It is a populist argument at best, because I won't skim through those 176 bugs to try and make sense of your argument. If you want to make a point, please don't try to leave the burden of proof on me.
Actually, I'll just take one of them, because I know it quite well: http://bugs.python.org/issue4967
This is a bug in _ssl for which I had to write a patch, although I knew nothing about _ssl, because the owner wouldn't react. He wouldn't react for review either. The bug *had* to be fixed because it blocked the whole process of including the C IO library. Finally, Benjamin committed the patch. The owner didn't give a sign of life *during the whole process*. He isn't dead, he still sometimes contributes to python-dev, but he was totally unavailable when his presence was needed.
And there are over 176 bugs with patches, and more needing patches which could use patches, docs, or tests. I guess that makes all of us negligent and bad maintainers.
So much for owners being a good thing.
So much for bad owners. Owners, by the very definition, must be active, and responsive (even if it's not real-time). People who are domain experts, but who are largely MIA are not a benefit.
The fact is, we need people who feel responsibility for every one of these modules to review patches, and have some amount of mental design integrity to ensure modules don't just wander off into the sunset and die.
But this is not the same as having an owner.
yes it is, but I think we're arguing the semantics of the word "owner" - how about "person on the hook"
Perhaps it wasn't clear, but I draw a clear separation between exclusive owners and maintainers.
Maybe we can agree on "maintainer" than "owner" - I did not mean exclusive ownership, and I apologize if I gave that impression.
I'm all for non-exclusive maintainership, with people having reasonable authority over code they understand thoroughly. You taking care of multiprocessing falls into this category (as long as you don't demand to approve of all changes before they are committed).
If I did that, I'd go insane.
I'm against ownership, however. I'm against mandatory signoffs (imprimaturs), and I'm against the possessive sentiment that some might develop towards a piece of code they contributed. Any core developer should be allowed to commit a patch if he thinks the patch is reasonable enough and serves an useful purpose.
Agreed.
Ownership prevents proper maintenance when the owner is absent (which *will* happen, since we are all unpaid volunteers). It discourages other people from getting acquainted with the code, which gradually makes the problem worse. Furthermore, it is often correlated with strange (personal) idioms, coding styles and design principles.
Agreed; but the maintainer should at least have a chance to say something, or be +noisy on issues at very least. I completely agree code dictatorship is bad, and I've seen it harm open source, and business code bases *a lot*. jesse
Perhaps it wasn't clear, but I draw a clear separation between exclusive owners and maintainers.
Maybe we can agree on "maintainer" than "owner" - I did not mean exclusive ownership, and I apologize if I gave that impression.
[...]
Agreed; but the maintainer should at least have a chance to say something, or be +noisy on issues at very least. I completely agree code dictatorship is bad, and I've seen it harm open source, and business code bases *a lot*.
Ok, so we do agree after all :) To elaborate a bit, I think the word "owner" is too strong and would easily give those "owners" the impression that they have full power over their respective modules. It is important to stress that code inside the CPython repository (or the stdlib one, once it is split) can be modified by any core developer with sufficiently good reasons. It is even more important to stress it when including a new module in the stdlib (and to refuse external maintainership). Regards Antoine.
Antoine Pitrou wrote:
I'm all for non-exclusive maintainership, with people having reasonable authority over code they understand thoroughly. You taking care of multiprocessing falls into this category (as long as you don't demand to approve of all changes before they are committed).
I'm against ownership, however. I'm against mandatory signoffs (imprimaturs), and I'm against the possessive sentiment that some might develop towards a piece of code they contributed. Any core developer should be allowed to commit a patch if he thinks the patch is reasonable enough and serves an useful purpose.
Ownership prevents proper maintenance when the owner is absent (which *will* happen, since we are all unpaid volunteers). It discourages other people from getting acquainted with the code, which gradually makes the problem worse. Furthermore, it is often correlated with strange (personal) idioms, coding styles and design principles.
I don't see things as negative as you apparently do. Owning a piece usually means that you wrote it and have good reasons for the design and code being what it is. It's a matter of respect towards the author to ask for review and discussion of a patch or new idea. If maintenance is passed on to someone else, the maintainer becomes the authority to discuss a patch with. If a maintainer or owner doesn't respond within 2-3 weeks (yes, people do go on vacation sometimes ;-), then I think it's ok to get review of the patch by some other core developer and then check it in - the maintainer or owner can always go back and revert the patch, if it doesn't fit for some reason. If this happens for longer periods of time, the maintainer or owner should be asked to pass on maintenance to someone else. Now, what to do in case of conflicts ? I.e. a core developer wants to add some new feature, change the design, etc. and the maintainer or owner disagrees. Well, we do what we've always done in the past: discuss the proposed changes on python-dev. If things go well, a compromise is found, otherwise the BDFL decides as tie-breaker. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 16 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Tue, Sep 15, 2009 at 11:15, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 13:48 -0400, Collin Winter a écrit :
The commonly-expressed idea behind the stdlib is that it represents best-of-breed code: it should have an independent userbase first, it should have proven itself in the wild before it is allowed into the stdlib.
Asking libraries to be proven in the wild sounds like a good idea, but it promotes disruptive changes (replacing a module with another one) rather than evolutionary changes. It also means that we get potentially bloated or bizarrely idiomatic packages, because they weren't subject to the same standards of simplicity / austerity that we value in the stdlib (multiprocessing comes to mind, doesn't it? how much time did we lose because of its byzantine API and implementation? how robust and maintainable is it, even now, although the original PyPI package was acclaimed?).
Therefore, I think it shouldn't be imposed as a general rule, rather a guideline.
For example, Michael's work on unittest (a work of evolutionary changes) is much better news, IMO, that someone proposing to include nose in the stdlib. And I say that as a nose user. I don't want to maintain nose in the stdlib.
I don't think anyone is suggesting that we always reach outside of the standard library when evolution is desired. It happens that for argparse that occurred, and that's partially because optparse did not lend itself to extension. Michael's work was great that it fit right in, but that also occurred because he was given commit privileges to do it. Without it he would either have needed to wait for someone to check his stuff in or start a fork to get things done. Another reason we need to do what we can to get through patches on the issue tracker.
"Best" comes with baggage: it means that other solutions are worse, and it admits the possibility that other code will someday overtake the current solution to *become* best-of-breed.
Of course. But it is also a double-sided argument because, if each new package remains the best-of-breed during only 2 years after it is integrated into the stdlib, we will spend a lot of time considering new packages for inclusion, deprecating other ones, and ultimately confusing the hell of our users.
We are not about to change modules every release. -Brett
2009/9/15 Antoine Pitrou <solipsis@pitrou.net>:
Le mardi 15 septembre 2009 à 13:48 -0400, Collin Winter a écrit :
The commonly-expressed idea behind the stdlib is that it represents best-of-breed code: it should have an independent userbase first, it should have proven itself in the wild before it is allowed into the stdlib.
Asking libraries to be proven in the wild sounds like a good idea, but it promotes disruptive changes (replacing a module with another one) rather than evolutionary changes.
Two points here: 1. In response to Collin - the idea that the stdlib is best of breed is common, and commonly stated, but it is actually not always the case. For example, wsgiref is not intended as a best of breed WSGI implementation, but as a reference implementation. Similarly with SimpleHTTPServer - it's there for "batteries included" rather than because it is better than Apache :-) 2. Antoine's point is a very good one - the "proven in the wild" principle really only applies to libraries providing entirely new functionality (and maybe not even there). Where there's an overlap with existing stdlib functionality, having the stdlib steal ideas from competing 3rd party solutions is probably better. (Of course, if the stdlib module has no maintainer, there's a problem...) Paul.
On Tue, Sep 15, 2009 at 2:15 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 13:48 -0400, Collin Winter a écrit :
The commonly-expressed idea behind the stdlib is that it represents best-of-breed code: it should have an independent userbase first, it should have proven itself in the wild before it is allowed into the stdlib.
Asking libraries to be proven in the wild sounds like a good idea, but it promotes disruptive changes (replacing a module with another one) rather than evolutionary changes. It also means that we get potentially bloated or bizarrely idiomatic packages, because they weren't subject to the same standards of simplicity / austerity that we value in the stdlib (multiprocessing comes to mind, doesn't it? how much time did we lose because of its byzantine API and implementation? how robust and maintainable is it, even now, although the original PyPI package was acclaimed?).
Yup, multiprocessing is a perfect example of something popular in the wild, but was rushed to inclusion because I proposed it late in the cycle. If I had to do it again - as the guy who is still on the hook for bug fixes, evolving it, and general maintenance, I'd not have gotten it in there in the state it went in. I should have proposed it earlier in the 2.6/3.0 process, and spent more time working on it. Thanks for your vote though. Today, I still fix bugs, work on improvements, etc. In my mind, multiprocessing is a poor example because it was pulled in *so quickly* - not because it was pulled in, and not because it has bugs. But rather a bar that should have been met, was not due to time. On the other hand - at least it has a maintainer (me) unlike some 50% of the rest of the libraries. So it has some small thing going for it. It also has tests. And documentation. And I continue to add to those when I can. How much of the rest of the standard libs can claim that? jesse
Jesse Noller schrieb:
Yup, multiprocessing is a perfect example of something popular in the wild, but was rushed to inclusion because I proposed it late in the cycle. If I had to do it again - as the guy who is still on the hook for bug fixes, evolving it, and general maintenance, I'd not have gotten it in there in the state it went in. I should have proposed it earlier in the 2.6/3.0 process, and spent more time working on it.
Thanks for your vote though.
Today, I still fix bugs, work on improvements, etc. In my mind, multiprocessing is a poor example because it was pulled in *so quickly* - not because it was pulled in, and not because it has bugs. But rather a bar that should have been met, was not due to time.
On the other hand - at least it has a maintainer (me) unlike some 50% of the rest of the libraries. So it has some small thing going for it. It also has tests. And documentation. And I continue to add to those when I can.
How much of the rest of the standard libs can claim that?
Not much, and this is a problem. How often have I wished for an official maintainer for some module, who I could defer a tracker issue to and say "please decide NOW whether this is a valid bug/request". There are literally hundreds of issues that could either be closed immediately as rejected/ wontfix or fixed with a small effort as soon as somebody makes the decision. Of course, for modules without a specific maintainer, we're all supposed to share the job, but it simply doesn't work out. I do fix bugs that are obviously bugs, but in more involved situations I often simply don't feel "qualified" enough to do so. Often a maintainer wouldn't even need special skills, but simply be there to take the blame and make decisions, complete with fixing the problems that arise from them later. I realize we can't have a maintainer for every module, but we should be glad about every one that does. So, bottom line: We love you, Jesse, and we need more of you! cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On Tue, Sep 15, 2009 at 4:21 PM, Georg Brandl <g.brandl@gmx.net> wrote:
How much of the rest of the standard libs can claim that?
Not much, and this is a problem. How often have I wished for an official maintainer for some module, who I could defer a tracker issue to and say "please decide NOW whether this is a valid bug/request". There are literally hundreds of issues that could either be closed immediately as rejected/ wontfix or fixed with a small effort as soon as somebody makes the decision.
Bingo. Yes. Correct.
Of course, for modules without a specific maintainer, we're all supposed to share the job, but it simply doesn't work out. I do fix bugs that are obviously bugs, but in more involved situations I often simply don't feel "qualified" enough to do so. Often a maintainer wouldn't even need special skills, but simply be there to take the blame and make decisions, complete with fixing the problems that arise from them later.
Also correct - this and the latter issue is exactly why I am advocating owners, and ultimately a smaller, cleaner standard library (but that's a different PEP)
I realize we can't have a maintainer for every module, but we should be glad about every one that does.
Except those owners who are perma-afk. That's bad. Just as bad as having none at all.
So, bottom line: We love you, Jesse, and we need more of you!
Yeah, well - I don't know about that ;) jesse
On Tue, 15 Sep 2009 at 19:02, Laura Creighton wrote:
And there may be things that are so badly written that we actually want to throw them out of the standard library before people start using them. A note 'this module only exists for backwards compatibility, we recommend that you use XXX instead' will not be good enough for the purpose, we really want to stamp out the use of this mistake as soon and as well as possible. This could happen.
On a smaller scale, it did a while ago. Consider ConfigParser and SafeConfigParser. Although in that case the docs about the deprecation are not even as good as they could be (see http://bugs.python.org/issue6517). --David
R. David Murray wrote:
On Tue, 15 Sep 2009 at 19:02, Laura Creighton wrote:
And there may be things that are so badly written that we actually want to throw them out of the standard library before people start using them. A note 'this module only exists for backwards compatibility, we recommend that you use XXX instead' will not be good enough for the purpose, we really want to stamp out the use of this mistake as soon and as well as possible. This could happen.
On a smaller scale, it did a while ago. Consider ConfigParser and SafeConfigParser. Although in that case the docs about the deprecation are not even as good as they could be (see http://bugs.python.org/issue6517). Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html The deprecation notice could be stronger / more prominent. Michael
--David _______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :) They could also be moved to a dedicated section. For example, md5 would be inside "XX - deprecated modules" rather than "15 - Cryptographic services".
If Twitter is any indication Georg is now ignoring this mailing list, so I am adding him directly so he can answer Antoine's questions. On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
They could also be moved to a dedicated section. For example, md5 would be inside "XX - deprecated modules" rather than "15 - Cryptographic services".
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
Brett Cannon schrieb:
If Twitter is any indication Georg is now ignoring this mailing list, so I am adding him directly so he can answer Antoine's questions.
Doesn't help because your mail's still sorted into the same folder with all the other mails having [stdlib-sig] in the title :) FWIW, I asked gmane today to add the stdlib-sig; but I fear that this period of high volume is over before they get to it.
On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see. But still possible, if you want it :)
They could also be moved to a dedicated section. For example, md5 would be inside "XX - deprecated modules" rather than "15 - Cryptographic services".
It would still have an ordinary number, but that's of course possible. Georg
2009/9/15 Georg Brandl <georg@python.org> schrieb:
On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see.
Wouldn't a flashing red background with a scrolling marquee in a gothic font be more effective? -- Regards, Benjamin
Benjamin Peterson wrote:
2009/9/15 Georg Brandl <georg@python.org> schrieb:
On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see.
Wouldn't a flashing red background with a scrolling marquee in a gothic font be more effective?
+1 for <blink><marquee> on the whole document... Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
Michael Foord wrote:
Benjamin Peterson wrote:
2009/9/15 Georg Brandl <georg@python.org> schrieb:
On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see.
Wouldn't a flashing red background with a scrolling marquee in a gothic font be more effective?
+1 for <blink><marquee> on the whole document...
Those were deprecated a long long time ago by the W3C: http://www.w3.org/TR/WCAG10/ Today, you have to use a different concept for this called "CSS" and the new syntax goes like this: <body style="text-decoration:blink;">...</body> That's just as annoying, but hey, it's standards conform ;-) Hmm, now why didn't they just remove it altogether... (sorry, couldn't resist) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 15 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Michael Foord schrieb:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
Two examples of deprecated modules in the Python documentation:
http://docs.python.org/library/md5.html http://docs.python.org/library/mimewriter.html
The deprecation notice could be stronger / more prominent.
Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see.
Wouldn't a flashing red background with a scrolling marquee in a gothic font be more effective?
+1 for <blink><marquee> on the whole document...
And now for something completely reasonable: same markup as for warnings? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
2009/9/15 Georg Brandl <g.brandl@gmx.net>:
Michael Foord schrieb:
Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit :
> Two examples of deprecated modules in the Python documentation: > > http://docs.python.org/library/md5.html > http://docs.python.org/library/mimewriter.html > > The deprecation notice could be stronger / more prominent. > Perhaps deprecated modules can get some kind of greyish style (background and/or font). Georg, is it possible? :)
The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see.
Wouldn't a flashing red background with a scrolling marquee in a gothic font be more effective?
+1 for <blink><marquee> on the whole document...
And now for something completely reasonable: same markup as for warnings?
Isn't there a "deprecated" directive? -- Regards, Benjamin
Benjamin Peterson schrieb:
2009/9/15 Georg Brandl <g.brandl@gmx.net>:
Michael Foord schrieb:
> Le mardi 15 septembre 2009 à 19:27 +0100, Michael Foord a écrit : > >> Two examples of deprecated modules in the Python documentation: >> >> http://docs.python.org/library/md5.html >> http://docs.python.org/library/mimewriter.html >> >> The deprecation notice could be stronger / more prominent. >> > Perhaps deprecated modules can get some kind of greyish style > (background and/or font). Georg, is it possible? :) > The whole document? Seems a bit too intrusive to me, and there is no obvious connection to the deprecation if the deprecation notice is still hard to see.
Wouldn't a flashing red background with a scrolling marquee in a gothic font be more effective?
+1 for <blink><marquee> on the whole document...
And now for something completely reasonable: same markup as for warnings?
Isn't there a "deprecated" directive?
Of course, I meant "styling", not "markup". Shame on you for not reading my mind :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
participants (13)
-
Antoine Pitrou -
Barry Warsaw -
Benjamin Peterson -
Brett Cannon -
Collin Winter -
Georg Brandl -
Georg Brandl -
Jesse Noller -
Laura Creighton -
M.-A. Lemburg -
Michael Foord -
Paul Moore -
R. David Murray