From ubershmekel at gmail.com  Thu Sep 10 02:38:01 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Thu, 10 Sep 2009 03:38:01 +0300
Subject: [stdlib-sig] Pyopt - command-line options that are alot easier than
	optparse
Message-ID: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>

Hi stdlib-sig,
I wanted to ease exposing functions to the command-line using python 3
features and the result is http://code.google.com/p/pyopt/

<http://code.google.com/p/pyopt/>I think this could be good stuff for the
standard library and I'm not sure how to go about it, should I work on
integrating with optparse/optik (is that project still active?) or should
this be a separate module?

This is an example usage of pyopt.0.71:
---
import pyopt
expose = pyopt.Exposer()

@expose.args
def possy(archer:str, boulder:float, magic:int=42):
    """Shows an example positional command-line function.
        archer - is a str
        boulder - should be a float
        magic - a number that is magical"""
    print(repr(archer), repr(boulder), repr(magic))

if __name__ == "__main__":
    expose.run()

---
And this is what you get:

C:\>example.py -h
Usage: example.py archer boulder [magic]
        Shows an example positional command-line function.
        archer - is a str
        boulder - should be a float
        magic - a number that is magical

C:\>example.py 1 2 3
'1' 2.0 3

C:\>example.py 1 2
'1' 2.0 42

C:\>example.py 5
2 arguments required, got only 1. Run with ? or -h for more help.
---

For more examples and functionality (like multiple functions) look at:
http://code.google.com/p/pyopt/wiki/Examples

Notes:
1. I know the names aren't perfect (module pyopt, class Exposer and the
decorator methods args/kwargs/mixed). Please reply if you have better names
in mind.
2. I realize this isn't as flexible as optparse and passing options around
multiple different sub-functions is harder. This module is strictly a quick
and clean solution to a very day-to-day use case: bridging python functions
to command-line.
3. Feedback would be greatly appreciated. I would like to know if I'm the
only one who's tired of sys.argv/optparse boilerplate.

-- 
Yuv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/89de5f51/attachment.htm>

From brett at python.org  Thu Sep 10 03:27:51 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 9 Sep 2009 18:27:51 -0700
Subject: [stdlib-sig] Pyopt - command-line options that are alot easier
	than optparse
In-Reply-To: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
Message-ID: <bbaeab100909091827s6e8b7c26hde11cf41704e339d@mail.gmail.com>

On Wed, Sep 9, 2009 at 17:38, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> Hi stdlib-sig,
> I wanted to ease exposing functions to the command-line using python 3
> features and the result is?http://code.google.com/p/pyopt/
> I think this could be good stuff for the standard library and I'm not sure
> how to go about it, should I work on integrating with optparse/optik (is
> that project still active?)

No, Optik simply became optparse.

> or should this be a separate module?

If you are simply asking for design advice, this is not the proper
place to ask. If you are wanting to get this module accepted into the
standard library it needs to have existed for some time and become
accepted by the Python community as the best solution for command-line
argument parsing.

-Brett

> This is an example usage of pyopt.0.71:
> ---
> import pyopt
> expose = pyopt.Exposer()
> @expose.args
> def possy(archer:str, boulder:float, magic:int=42):
> ?? ?"""Shows an example positional command-line function.
> ?? ? ? ?archer - is a str
> ?? ? ? ?boulder - should be a float
> ?? ? ? ?magic - a number that is magical"""
> ?? ?print(repr(archer), repr(boulder), repr(magic))
> if __name__ == "__main__":
> ?? ?expose.run()
> ---
> And this is what you get:
> C:\>example.py -h
> Usage: example.py archer boulder [magic]
> ?? ? ? ?Shows an example positional command-line function.
> ?? ? ? ?archer - is a str
> ?? ? ? ?boulder - should be a float
> ?? ? ? ?magic - a number that is magical
> C:\>example.py 1 2 3
> '1' 2.0 3
> C:\>example.py 1 2
> '1' 2.0 42
> C:\>example.py 5
> 2 arguments required, got only 1. Run with ? or -h for more help.
> ---
> For more examples and functionality (like multiple functions) look at:
> http://code.google.com/p/pyopt/wiki/Examples
> Notes:
> 1. I know the names aren't perfect?(module pyopt, class Exposer and the
> decorator methods args/kwargs/mixed). Please reply if you have better names
> in mind.
> 2. I realize this isn't as flexible as optparse and passing options around
> multiple different sub-functions is harder. This module is strictly a quick
> and clean solution to a very day-to-day use case: bridging python functions
> to command-line.
> 3. Feedback would be greatly appreciated. I would like to know if I'm the
> only one who's tired of sys.argv/optparse boilerplate.
> --
> Yuv
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>
>

From michael at voidspace.org.uk  Thu Sep 10 11:21:33 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 10:21:33 +0100
Subject: [stdlib-sig] Pyopt - command-line options that are alot easier
 than	optparse
In-Reply-To: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
Message-ID: <4AA8C51D.3080902@voidspace.org.uk>

Yuvgoog Greenle wrote:
> Hi stdlib-sig,
>
> I wanted to ease exposing functions to the command-line using python 3 
> features and the result is http://code.google.com/p/pyopt/
>
> I think this could be good stuff for the standard library and I'm not 
> sure how to go about it, should I work on integrating with 
> optparse/optik (is that project still active?) or should this be a 
> separate module?

For a module to be accepted into the standard library it typically 
already needs to be in use by the Python community and recognised as 
'best-of-breed'.

*Personally* I would like to see argparse in the standard library as I 
think it is currently the best Python argument parsing library, but that 
is just my opinion. As we already have two argument parsing libraries in 
the standard library (getopt and optparse) the addition of a third for 
the *same* task is highly unlikely.

On the other hand, if you can provide your functionality as an addon-for 
optparse - retaining full backwards compatibility - then it may be 
accepted as an extension of the existing module rather than as a new one 
(a much lower bar).

All the best,

Michael Foord

>
> This is an example usage of pyopt.0.71:
> ---
> import pyopt
> expose = pyopt.Exposer()
>
> @expose.args
> def possy(archer:str, boulder:float, magic:int=42):
>     """Shows an example positional command-line function.
>         archer - is a str
>         boulder - should be a float
>         magic - a number that is magical"""
>     print(repr(archer), repr(boulder), repr(magic))
>
> if __name__ == "__main__":
>     expose.run()
>
> ---
> And this is what you get:
>
> C:\>example.py -h
> Usage: example.py archer boulder [magic]
>         Shows an example positional command-line function.
>         archer - is a str
>         boulder - should be a float
>         magic - a number that is magical
>
> C:\>example.py 1 2 3
> '1' 2.0 3
>
> C:\>example.py 1 2
> '1' 2.0 42
>
> C:\>example.py 5
> 2 arguments required, got only 1. Run with ? or -h for more help.
> ---
>
> For more examples and functionality (like multiple functions) look at:
> http://code.google.com/p/pyopt/wiki/Examples
>
> Notes:
> 1. I know the names aren't perfect (module pyopt, class Exposer and 
> the decorator methods args/kwargs/mixed). Please reply if you have 
> better names in mind.
> 2. I realize this isn't as flexible as optparse and passing options 
> around multiple different sub-functions is harder. This module is 
> strictly a quick and clean solution to a very day-to-day use case: 
> bridging python functions to command-line.
> 3. Feedback would be greatly appreciated. I would like to know if I'm 
> the only one who's tired of sys.argv/optparse boilerplate.
>
> -- 
> Yuv
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/


From holger at merlinux.eu  Thu Sep 10 12:03:00 2009
From: holger at merlinux.eu (holger krekel)
Date: Thu, 10 Sep 2009 12:03:00 +0200
Subject: [stdlib-sig] Pyopt - command-line options that are alot
	easier	than optparse
In-Reply-To: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
Message-ID: <20090910100300.GQ15455@trillke.net>

Hi Yuvgoog!,

On Thu, Sep 10, 2009 at 03:38 +0300, Yuvgoog Greenle wrote:
> Hi stdlib-sig,
> I wanted to ease exposing functions to the command-line using python 3
> features and the result is http://code.google.com/p/pyopt/
> 
> <http://code.google.com/p/pyopt/>I think this could be good stuff for the
> standard library and I'm not sure how to go about it, should I work on
> integrating with optparse/optik (is that project still active?) or should
> this be a separate module?
> 
> This is an example usage of pyopt.0.71:
> ---
> import pyopt
> expose = pyopt.Exposer()
> 
> @expose.args
> def possy(archer:str, boulder:float, magic:int=42):
>     """Shows an example positional command-line function.
>         archer - is a str
>         boulder - should be a float
>         magic - a number that is magical"""
>     print(repr(archer), repr(boulder), repr(magic))
> 
> if __name__ == "__main__":
>     expose.run()

interesting!  If i were you i'd not bother with the standard
library but just publish the module and project on PyPI. 
With two other packages already in it si hard to add
a third and making an extension to optparse is probably 
a hassle. 

btw, myself i'd only use it if it also works <3.0. 
A similar idea i had was this, btw:

    @cmdline
    def main(archer=str, boulder=float, magic=42):

which would wrap a parser around the function 
and otherwise work like your example. This can
easily work down to Python2.3. 

cheers,
holger

From barry at python.org  Thu Sep 10 15:05:28 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 10 Sep 2009 09:05:28 -0400
Subject: [stdlib-sig] Pyopt - command-line options that are alot easier
	than	optparse
In-Reply-To: <4AA8C51D.3080902@voidspace.org.uk>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
	<4AA8C51D.3080902@voidspace.org.uk>
Message-ID: <05CAF8FF-B55D-4A71-85F6-318029B3D4B7@python.org>

On Sep 10, 2009, at 5:21 AM, Michael Foord wrote:

> *Personally* I would like to see argparse in the standard library as  
> I think it is currently the best Python argument parsing library,  
> but that is just my opinion. As we already have two argument parsing  
> libraries in the standard library (getopt and optparse) the addition  
> of a third for the *same* task is highly unlikely.

Huge +1 for argparse.  It exists, seems rock solid, easy to use  
(amazing for an argument parsing library), supports subcommands  
easily, is easy to port from optparse usage, and just full of  
awesomeness.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/bb454574/attachment.pgp>

From orsenthil at gmail.com  Thu Sep 10 15:16:56 2009
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Thu, 10 Sep 2009 18:46:56 +0530
Subject: [stdlib-sig] Pyopt - command-line options that are alot
	easier	than optparse
In-Reply-To: <05CAF8FF-B55D-4A71-85F6-318029B3D4B7@python.org>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
	<4AA8C51D.3080902@voidspace.org.uk>
	<05CAF8FF-B55D-4A71-85F6-318029B3D4B7@python.org>
Message-ID: <20090910131656.GC20066@ubuntu.ubuntu-domain>

On Thu, Sep 10, 2009 at 09:05:28AM -0400, Barry Warsaw wrote:
>
>> *Personally* I would like to see argparse in the standard library as I 
>> think it is currently the best Python argument parsing library, but 
>> that is just my opinion. As we already have two argument parsing  
>> libraries in the standard library (getopt and optparse) the addition  
>> of a third for the *same* task is highly unlikely.
>
> Huge +1 for argparse.  It exists, seems rock solid, easy to use (amazing 
> for an argument parsing library), supports subcommands easily, is easy to 
> port from optparse usage, and just full of awesomeness.

Based on this discussion, I think we need to reopen this bug:
http://bugs.python.org/issue6247

We already seem to have couple of +1s ( while personally I am 0, as I
have never had requirement to use argparse and most often my needs
were served by optparse).

It would be required to convince MvL, to include argparse in stdlib.
But I hate the very idea of three modules doing the same job! (even
if one does something better than the other).

-- 
Senthil
  I tripped over a hole that was sticking up out of the ground.

From brett at python.org  Thu Sep 10 20:56:36 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 11:56:36 -0700
Subject: [stdlib-sig] should we try to add argparse?
Message-ID: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>

Upfront people need to realize that we might have three argument
parsing libraries for a while, but it won't be forever. If we get
argparse accepted we would slowly deprecate at least optparse, if not
getopt (lat time I tried to ditch getopt for Python 3 some argued that
getopt supported stuff optparse didn't), out of the standard library
and toss them into PyPI for those who refuse to switch. The standard
library might not evolve a lot, but it isn't dead or in stone.

But before this can happen, people need to have a general consensus
that I should bug Steven about contributing as it will require a PEP
from him. Steven already has commit privileges to maintenance from him
will not be a problem.

So if you want this to actually happen and for me to start talking to
Steven just reply to this email w/ a vote.

I am +0

From michael at voidspace.org.uk  Thu Sep 10 21:30:06 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 20:30:06 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <4AA953BE.1080908@voidspace.org.uk>

Brett Cannon wrote:
> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't), out of the standard library
> and toss them into PyPI for those who refuse to switch. The standard
> library might not evolve a lot, but it isn't dead or in stone.
>
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.
>
> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.
>
> I am +0
>   

+1

Michael

> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From theller at ctypes.org  Thu Sep 10 21:59:27 2009
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 10 Sep 2009 21:59:27 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <4AA95A9F.9060209@ctypes.org>

Brett Cannon schrieb:
> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't), out of the standard library
> and toss them into PyPI for those who refuse to switch. The standard
> library might not evolve a lot, but it isn't dead or in stone.
> 
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.
> 
> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.
> 
> I am +0

I have not used argparse, but I am missing a feature that is needed
on Windows:  It should be possible to mark option flags
not only with a '-' sign (or a '--' sign) but also with a '/' sign,
plus it should be possible to have case-insensitive parsing of option
names.  So that '-regserver', '/regserver' and '/RegServer' all
mean the same thing.

Does argparse allow this?

-- 
Thanks,
Thomas

From jbaker at zyasoft.com  Thu Sep 10 22:07:07 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Thu, 10 Sep 2009 14:07:07 -0600
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA953BE.1080908@voidspace.org.uk>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<4AA953BE.1080908@voidspace.org.uk>
Message-ID: <d03bb4010909101307q4440c796q83918119b1a16864@mail.gmail.com>

+1 - modernization is good. optparse was very good in its time, but too
clunky now

On Thu, Sep 10, 2009 at 1:30 PM, Michael Foord <michael at voidspace.org.uk>wrote:

> Brett Cannon wrote:
>
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't), out of the standard library
>> and toss them into PyPI for those who refuse to switch. The standard
>> library might not evolve a lot, but it isn't dead or in stone.
>>
>> But before this can happen, people need to have a general consensus
>> that I should bug Steven about contributing as it will require a PEP
>> from him. Steven already has commit privileges to maintenance from him
>> will not be a problem.
>>
>> So if you want this to actually happen and for me to start talking to
>> Steven just reply to this email w/ a vote.
>>
>> I am +0
>>
>>
>
> +1
>
> Michael
>
>  _______________________________________________
>> stdlib-sig mailing list
>> stdlib-sig at python.org
>> http://mail.python.org/mailman/listinfo/stdlib-sig
>>
>>
>
>
> --
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/3286d936/attachment-0001.htm>

From barry at python.org  Thu Sep 10 22:27:35 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 10 Sep 2009 16:27:35 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <89E62C37-2ADF-4744-9056-9805345A1102@python.org>

On Sep 10, 2009, at 2:56 PM, Brett Cannon wrote:

> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.

+1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/04535e84/attachment.pgp>

From holger at merlinux.eu  Thu Sep 10 22:55:27 2009
From: holger at merlinux.eu (holger krekel)
Date: Thu, 10 Sep 2009 22:55:27 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <20090910205527.GB15455@trillke.net>

On Thu, Sep 10, 2009 at 11:56 -0700, Brett Cannon wrote:
> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't), out of the standard library
> and toss them into PyPI for those who refuse to switch. The standard
> library might not evolve a lot, but it isn't dead or in stone.
> 
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.
> 
> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.

-0 because i think it's a questionable effort/value ratio.

holger

From brett at python.org  Thu Sep 10 22:36:53 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 13:36:53 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA95A9F.9060209@ctypes.org>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<4AA95A9F.9060209@ctypes.org>
Message-ID: <bbaeab100909101336x76bbed81gc877e7bf3ee48de1@mail.gmail.com>

On Thu, Sep 10, 2009 at 12:59, Thomas Heller <theller at ctypes.org> wrote:
> Brett Cannon schrieb:
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't), out of the standard library
>> and toss them into PyPI for those who refuse to switch. The standard
>> library might not evolve a lot, but it isn't dead or in stone.
>>
>> But before this can happen, people need to have a general consensus
>> that I should bug Steven about contributing as it will require a PEP
>> from him. Steven already has commit privileges to maintenance from him
>> will not be a problem.
>>
>> So if you want this to actually happen and for me to start talking to
>> Steven just reply to this email w/ a vote.
>>
>> I am +0
>
> I have not used argparse, but I am missing a feature that is needed
> on Windows: ?It should be possible to mark option flags
> not only with a '-' sign (or a '--' sign) but also with a '/' sign,
> plus it should be possible to have case-insensitive parsing of option
> names. ?So that '-regserver', '/regserver' and '/RegServer' all
> mean the same thing.
>
> Does argparse allow this?

According to http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
it does.

-Brett

From theller at ctypes.org  Thu Sep 10 23:10:26 2009
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 10 Sep 2009 23:10:26 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101336x76bbed81gc877e7bf3ee48de1@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<4AA95A9F.9060209@ctypes.org>
	<bbaeab100909101336x76bbed81gc877e7bf3ee48de1@mail.gmail.com>
Message-ID: <4AA96B42.7070502@ctypes.org>

>> I have not used argparse, but I am missing a feature that is needed
>> on Windows:  It should be possible to mark option flags
>> not only with a '-' sign (or a '--' sign) but also with a '/' sign,
>> plus it should be possible to have case-insensitive parsing of option
>> names.  So that '-regserver', '/regserver' and '/RegServer' all
>> mean the same thing.
>>
>> Does argparse allow this?
> 
> According to http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
> it does.

Then it gets a +1 from me.

-- 
Thanks,
Thomas

From solipsis at pitrou.net  Thu Sep 10 23:14:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 10 Sep 2009 23:14:59 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <1252617299.5933.0.camel@localhost>


> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't),

+0 on deprecating getopt, -1 on deprecating optparse. Breaking a
perfectly functional and useful module is stupid.




From michael at voidspace.org.uk  Thu Sep 10 23:22:09 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 22:22:09 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252617299.5933.0.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
Message-ID: <4AA96E01.2020601@voidspace.org.uk>

Antoine Pitrou wrote:
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't),
>>     
>
> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
> perfectly functional and useful module is stupid.
>
>   

So we're stuck with inferior technology?

Michael

>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Thu Sep 10 23:25:53 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 10 Sep 2009 16:25:53 -0500
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA96E01.2020601@voidspace.org.uk>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk>
Message-ID: <1afaf6160909101425p14970423o3b202bd7207a16db@mail.gmail.com>

2009/9/10 Michael Foord <michael at voidspace.org.uk>:
> Antoine Pitrou wrote:
>>>
>>> Upfront people need to realize that we might have three argument
>>> parsing libraries for a while, but it won't be forever. If we get
>>> argparse accepted we would slowly deprecate at least optparse, if not
>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>> getopt supported stuff optparse didn't),
>>>
>>
>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>> perfectly functional and useful module is stupid.
>>
>>
>
> So we're stuck with inferior technology?

No, that's never been the case. Download and install argparse.



-- 
Regards,
Benjamin

From michael at voidspace.org.uk  Thu Sep 10 23:27:38 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 22:27:38 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1afaf6160909101425p14970423o3b202bd7207a16db@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>	
	<1252617299.5933.0.camel@localhost>	
	<4AA96E01.2020601@voidspace.org.uk>
	<1afaf6160909101425p14970423o3b202bd7207a16db@mail.gmail.com>
Message-ID: <4AA96F4A.9020804@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/9/10 Michael Foord <michael at voidspace.org.uk>:
>   
>> Antoine Pitrou wrote:
>>     
>>>> Upfront people need to realize that we might have three argument
>>>> parsing libraries for a while, but it won't be forever. If we get
>>>> argparse accepted we would slowly deprecate at least optparse, if not
>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>>> getopt supported stuff optparse didn't),
>>>>
>>>>         
>>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>>> perfectly functional and useful module is stupid.
>>>
>>>
>>>       
>> So we're stuck with inferior technology?
>>     
>
> No, that's never been the case. Download and install argparse.
>
>
>
>   
I was talking about in the standard library.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Thu Sep 10 23:29:17 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 14:29:17 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252617299.5933.0.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost>
Message-ID: <bbaeab100909101429m7f250563m8059ac17e0138a75@mail.gmail.com>

On Thu, Sep 10, 2009 at 14:14, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't),
>
> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
> perfectly functional and useful module is stupid.

Fine, but that doesn't say anything about whether you think adding
argparse is a good idea.

-Brett

From solipsis at pitrou.net  Thu Sep 10 23:33:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 10 Sep 2009 23:33:49 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA96E01.2020601@voidspace.org.uk>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost> <4AA96E01.2020601@voidspace.org.uk>
Message-ID: <1252618429.5933.10.camel@localhost>

Le jeudi 10 septembre 2009 ? 22:22 +0100, Michael Foord a ?crit :
> > +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
> > perfectly functional and useful module is stupid.
> >
> >   
> 
> So we're stuck with inferior technology?

We aren't! I'm theoretically ok with putting argparse in the stdlib (I
say theoretically because I didn't bother to look it up). I'm not ok
with deprecating optparse in the process, because it's a a perfectly
good and widely used alternative. Forcing users to rewrite their code
just because we think we can come up with something smarter than what
they use is *not* nice.

On the other hand, getopt's API is so archaic that deprecating it seems
warranted.



From barry at python.org  Thu Sep 10 23:30:43 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 10 Sep 2009 17:30:43 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252617299.5933.0.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
Message-ID: <A6B44862-C103-4668-9ABF-DA9038726929@python.org>

On Sep 10, 2009, at 5:14 PM, Antoine Pitrou wrote:

> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
> perfectly functional and useful module is stupid.

Why does deprecation == breaking?

If you want to continue using optparse, that'll be fine for what?  
18-36 months or so?  It's a trivial amount of work to port from  
optparse to argparse.  We have a process for deprecating modules, so  
let's follow that and migrate to the much better library.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/eabd3911/attachment.pgp>

From collinw at gmail.com  Thu Sep 10 23:31:28 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 10 Sep 2009 18:31:28 -0300
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252617299.5933.0.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
Message-ID: <43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>

On Thu, Sep 10, 2009 at 6:14 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't),
>
> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
> perfectly functional and useful module is stupid.

Do remember that if optparse is deprecated, it will still be available
for *years*. Code isn't going to suddenly break overnight. Users will
see this coming far, far ahead of time.

How many more ways do we need to do argument parsing? If argparse is
added, I'd like to see both getopt and optparse begin their
deprecation cycles in the same release. Three separate ways of doing
argument parsing in the stdlib is madness.

Collin Winter

From michael at voidspace.org.uk  Thu Sep 10 23:33:13 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 22:33:13 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>	<1252617299.5933.0.camel@localhost>
	<43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
Message-ID: <4AA97099.1020406@voidspace.org.uk>

Collin Winter wrote:
> On Thu, Sep 10, 2009 at 6:14 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>   
>>> Upfront people need to realize that we might have three argument
>>> parsing libraries for a while, but it won't be forever. If we get
>>> argparse accepted we would slowly deprecate at least optparse, if not
>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>> getopt supported stuff optparse didn't),
>>>       
>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>> perfectly functional and useful module is stupid.
>>     
>
> Do remember that if optparse is deprecated, it will still be available
> for *years*. Code isn't going to suddenly break overnight. Users will
> see this coming far, far ahead of time.
>
> How many more ways do we need to do argument parsing? If argparse is
> added, I'd like to see both getopt and optparse begin their
> deprecation cycles in the same release. Three separate ways of doing
> argument parsing in the stdlib is madness.
>   

+1 to deprecating both.

Michael

> Collin Winter
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Thu Sep 10 23:35:17 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 14:35:17 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost>
	<43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
Message-ID: <bbaeab100909101435g77c3fe21r504f1bcf90fe18ac@mail.gmail.com>

On Thu, Sep 10, 2009 at 14:31, Collin Winter <collinw at gmail.com> wrote:
> On Thu, Sep 10, 2009 at 6:14 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>>> Upfront people need to realize that we might have three argument
>>> parsing libraries for a while, but it won't be forever. If we get
>>> argparse accepted we would slowly deprecate at least optparse, if not
>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>> getopt supported stuff optparse didn't),
>>
>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>> perfectly functional and useful module is stupid.
>
> Do remember that if optparse is deprecated, it will still be available
> for *years*. Code isn't going to suddenly break overnight. Users will
> see this coming far, far ahead of time.
>

I suspect we would do at least one release as
PendingDeprecationWarning, and one with DeprecationWarning. But there
is a much bigger chance they will stay w/ DeprecationWarning for a
couple of releases. Plus the code will be made available on PyPI for
people to download and use on their own w/ no deprecation warning.

> How many more ways do we need to do argument parsing? If argparse is
> added, I'd like to see both getopt and optparse begin their
> deprecation cycles in the same release. Three separate ways of doing
> argument parsing in the stdlib is madness.

So would I. I looked up the original arguments against removing getopt
for Python 3 so we can make sure everyone is happy w/ ditching both
modules.

-Brett

From mal at egenix.com  Thu Sep 10 23:36:30 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 10 Sep 2009 23:36:30 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA96E01.2020601@voidspace.org.uk>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk>
Message-ID: <4AA9715E.8080403@egenix.com>

Michael Foord wrote:
> Antoine Pitrou wrote:
>>> Upfront people need to realize that we might have three argument
>>> parsing libraries for a while, but it won't be forever. If we get
>>> argparse accepted we would slowly deprecate at least optparse, if not
>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>> getopt supported stuff optparse didn't),
>>>     
>>
>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>> perfectly functional and useful module is stupid.
>>
>>   
> 
> So we're stuck with inferior technology?

If you have the choice between breaking backwards compatibility
and downloading other implementations from PyPI, I think backwards
compatibility counts more.

We've just had a major change in the stdlib for 3.x. Then the 3.0
release was ditched due to poor performance. If we now start
deprecating widely used modules in 3.2, we're going to lose
a major pro argument for using Python: that of a stable eco system
to write code for.

I don't think we should deprecate any commonly used module (in the
3.x branch) unless there's a clear and documented migration path
or -even better- a migration wrapper available for the deprecated
module.

The next major round of refactoring will have to wait for
Python 4.x.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 09 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From michael at voidspace.org.uk  Thu Sep 10 23:39:02 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 22:39:02 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA9715E.8080403@egenix.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
Message-ID: <4AA971F6.8060707@voidspace.org.uk>

M.-A. Lemburg wrote:
> Michael Foord wrote:
>   
>> Antoine Pitrou wrote:
>>     
>>>> Upfront people need to realize that we might have three argument
>>>> parsing libraries for a while, but it won't be forever. If we get
>>>> argparse accepted we would slowly deprecate at least optparse, if not
>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>>> getopt supported stuff optparse didn't),
>>>>     
>>>>         
>>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>>> perfectly functional and useful module is stupid.
>>>
>>>   
>>>       
>> So we're stuck with inferior technology?
>>     
>
> If you have the choice between breaking backwards compatibility
> and downloading other implementations from PyPI, I think backwards
> compatibility counts more.
>
> We've just had a major change in the stdlib for 3.x. Then the 3.0
> release was ditched due to poor performance. If we now start
> deprecating widely used modules in 3.2, we're going to lose
> a major pro argument for using Python: that of a stable eco system
> to write code for.
>
> I don't think we should deprecate any commonly used module (in the
> 3.x branch) unless there's a clear and documented migration path
> or -even better- a migration wrapper available for the deprecated
> module.
>
> The next major round of refactoring will have to wait for
> Python 4.x.
>
>   
Language refactoring can wait for 4.0. Library improvements should be 
ongoing.

As Brett says, even if we go down this road, optparse won't actually be 
removed for several years.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Thu Sep 10 23:44:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 10 Sep 2009 23:44:00 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <A6B44862-C103-4668-9ABF-DA9038726929@python.org>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org>
Message-ID: <1252619040.5933.19.camel@localhost>

Le jeudi 10 septembre 2009 ? 17:30 -0400, Barry Warsaw a ?crit :
> Why does deprecation == breaking?

Well, you have to rewrite your code one day. And since optparse is
*very* widely used, tons of users will be impacted. Python is often used
as a scripting language, even for small one-off scripts that nobody
wants or even knows to maintain.

Besides, any spurious message on the standard output breaks things like
cron scripts, so even the deprecation period can be annoying.

Please do not assume that every Python program is a well-maintained
application with dedicated full-time developers...

> If you want to continue using optparse, that'll be fine for what?  
> 18-36 months or so?  It's a trivial amount of work to port from  
> optparse to argparse.

Ok, if it's trivial then perhaps I'd reconsider my opinion.
Is there a compatibility API?

(please note we have e.g. a lot of XML parsing libraries in the stdlib,
and nobody seems in a hurry to deprecate them)



From collinw at gmail.com  Thu Sep 10 23:46:20 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 10 Sep 2009 18:46:20 -0300
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA9715E.8080403@egenix.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
Message-ID: <43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>

On Thu, Sep 10, 2009 at 6:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Michael Foord wrote:
>> So we're stuck with inferior technology?
>
> If you have the choice between breaking backwards compatibility
> and downloading other implementations from PyPI, I think backwards
> compatibility counts more.

Why doesn't this argument apply to getopt and optparse equally? Once
they've gone through the multi-year deprecation process and have been
finally removed, they can be put up on PyPI as stand-alone release to
support users who can't/won't migrate to argparse.

> We've just had a major change in the stdlib for 3.x. Then the 3.0
> release was ditched due to poor performance. If we now start
> deprecating widely used modules in 3.2, we're going to lose
> a major pro argument for using Python: that of a stable eco system
> to write code for.

The language can be stable, but if the entire 3.x series is just a
series of bugfixes on top of 3.0's stdlib, then that's a poor state of
affairs. That's the price users pay for shiny, new things: the old,
crufty things have to go away eventually, and I think it's unrealistic
to wait another 10 years for a hypothetical Py4k project to remove
getopt.

> I don't think we should deprecate any commonly used module (in the
> 3.x branch) unless there's a clear and documented migration path
> or -even better- a migration wrapper available for the deprecated
> module.

Agreed. There should be a documented cookbook for migrating from
getopt/optparse to argparse; I can't imagine that it's anyone's
intention to throw users in the deep end without any aid. Perhaps
simple cases could be handled automatically by a refactoring tool
based on 2to3 (in the same way that 3to2 is based on 2to3), which I
imagine many users would prefer.

Collin Winter

From brett at python.org  Thu Sep 10 23:52:24 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 14:52:24 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252619040.5933.19.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org> 
	<1252619040.5933.19.camel@localhost>
Message-ID: <bbaeab100909101452r27ffe2b1tc98dcf62408e65db@mail.gmail.com>

On Thu, Sep 10, 2009 at 14:44, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le jeudi 10 septembre 2009 ? 17:30 -0400, Barry Warsaw a ?crit :
>> Why does deprecation == breaking?
>
> Well, you have to rewrite your code one day. And since optparse is
> *very* widely used, tons of users will be impacted. Python is often used
> as a scripting language, even for small one-off scripts that nobody
> wants or even knows to maintain.
>
> Besides, any spurious message on the standard output breaks things like
> cron scripts, so even the deprecation period can be annoying.
>
> Please do not assume that every Python program is a well-maintained
> application with dedicated full-time developers...
>
>> If you want to continue using optparse, that'll be fine for what?
>> 18-36 months or so? ?It's a trivial amount of work to port from
>> optparse to argparse.
>
> Ok, if it's trivial then perhaps I'd reconsider my opinion.
> Is there a compatibility API?

Here is what the docs say for transitioning:
http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html#upgrading-optparse-code
. Also says why optparse was no longer being extended.

>
> (please note we have e.g. a lot of XML parsing libraries in the stdlib,
> and nobody seems in a hurry to deprecate them)

Because no one has brought forward an XML replacement module that
doesn't provide some huge reason to not use SAX, DOM, or ETree (and
there is a reason to have all three).

-Brett

From ubershmekel at gmail.com  Thu Sep 10 23:58:53 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 11 Sep 2009 00:58:53 +0300
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AA971F6.8060707@voidspace.org.uk>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<4AA971F6.8060707@voidspace.org.uk>
Message-ID: <9d153b7c0909101458l2254baf2jed9ff6e585b212e@mail.gmail.com>

+1 for deprecating optparse or at the very least building a timeline for
it's deprecation. Don't say no to progress, just say when.
I just wrote pyopt http://code.google.com/p/pyopt/ because I think exposing
functions to the command line is way too hard.

If Steven Bethard and the argparse community agree, I would really
appreciate a convenience method named "add_function" built into
the ArgumentParser. add_function would automagically read the docstring and
understand what are the arguments for a given function. I'm willing and
motivated to do the work for this. Annotations-type-casting could be awesome
as well, but I'm not greedy enough to ask.

--MrPyopt


On Fri, Sep 11, 2009 at 12:39 AM, Michael Foord <michael at voidspace.org.uk>wrote:

> M.-A. Lemburg wrote:
>
>> Michael Foord wrote:
>>
>>
>>> Antoine Pitrou wrote:
>>>
>>>
>>>> Upfront people need to realize that we might have three argument
>>>>> parsing libraries for a while, but it won't be forever. If we get
>>>>> argparse accepted we would slowly deprecate at least optparse, if not
>>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>>>> getopt supported stuff optparse didn't),
>>>>>
>>>>>
>>>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>>>> perfectly functional and useful module is stupid.
>>>>
>>>>
>>>>
>>> So we're stuck with inferior technology?
>>>
>>>
>>
>> If you have the choice between breaking backwards compatibility
>> and downloading other implementations from PyPI, I think backwards
>> compatibility counts more.
>>
>> We've just had a major change in the stdlib for 3.x. Then the 3.0
>> release was ditched due to poor performance. If we now start
>> deprecating widely used modules in 3.2, we're going to lose
>> a major pro argument for using Python: that of a stable eco system
>> to write code for.
>>
>> I don't think we should deprecate any commonly used module (in the
>> 3.x branch) unless there's a clear and documented migration path
>> or -even better- a migration wrapper available for the deprecated
>> module.
>>
>> The next major round of refactoring will have to wait for
>> Python 4.x.
>>
>>
>>
> Language refactoring can wait for 4.0. Library improvements should be
> ongoing.
>
> As Brett says, even if we go down this road, optparse won't actually be
> removed for several years.
>
> Michael
>
> --
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>



-- 
Yuv
http://code.google.com/p/pyopt/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090911/e6f42064/attachment-0001.htm>

From brett at python.org  Fri Sep 11 00:04:56 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 15:04:56 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <9d153b7c0909101458l2254baf2jed9ff6e585b212e@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost> <4AA96E01.2020601@voidspace.org.uk>
	<4AA9715E.8080403@egenix.com> <4AA971F6.8060707@voidspace.org.uk> 
	<9d153b7c0909101458l2254baf2jed9ff6e585b212e@mail.gmail.com>
Message-ID: <bbaeab100909101504r3dc666acs1d28821fca2e0fb0@mail.gmail.com>

On Thu, Sep 10, 2009 at 14:58, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> +1 for deprecating optparse or at the very least building a timeline for
> it's deprecation. Don't say no to progress, just say when.
> I just wrote pyopt?http://code.google.com/p/pyopt/?because I think exposing
> functions to the command line is way too hard.
> If?Steven Bethard and the argparse community agree, I?would really
> appreciate a convenience method named?"add_function"?built into
> the?ArgumentParser. add_function would automagically read the docstring and
> understand what are the arguments for a given function. I'm willing and
> motivated to do the work for this. Annotations-type-casting could be awesome
> as well, but I'm not greedy enough to ask.

If this moves forward there will be a PEP and you can bring this idea up then.

-Brett

> --MrPyopt
>
> On Fri, Sep 11, 2009 at 12:39 AM, Michael Foord <michael at voidspace.org.uk>
> wrote:
>>
>> M.-A. Lemburg wrote:
>>>
>>> Michael Foord wrote:
>>>
>>>>
>>>> Antoine Pitrou wrote:
>>>>
>>>>>>
>>>>>> Upfront people need to realize that we might have three argument
>>>>>> parsing libraries for a while, but it won't be forever. If we get
>>>>>> argparse accepted we would slowly deprecate at least optparse, if not
>>>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>>>>> getopt supported stuff optparse didn't),
>>>>>>
>>>>>
>>>>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>>>>> perfectly functional and useful module is stupid.
>>>>>
>>>>>
>>>>
>>>> So we're stuck with inferior technology?
>>>>
>>>
>>> If you have the choice between breaking backwards compatibility
>>> and downloading other implementations from PyPI, I think backwards
>>> compatibility counts more.
>>>
>>> We've just had a major change in the stdlib for 3.x. Then the 3.0
>>> release was ditched due to poor performance. If we now start
>>> deprecating widely used modules in 3.2, we're going to lose
>>> a major pro argument for using Python: that of a stable eco system
>>> to write code for.
>>>
>>> I don't think we should deprecate any commonly used module (in the
>>> 3.x branch) unless there's a clear and documented migration path
>>> or -even better- a migration wrapper available for the deprecated
>>> module.
>>>
>>> The next major round of refactoring will have to wait for
>>> Python 4.x.
>>>
>>>
>>
>> Language refactoring can wait for 4.0. Library improvements should be
>> ongoing.
>>
>> As Brett says, even if we go down this road, optparse won't actually be
>> removed for several years.
>>
>> Michael
>>
>> --
>> http://www.ironpythoninaction.com/
>> http://www.voidspace.org.uk/blog
>>
>>
>> _______________________________________________
>> stdlib-sig mailing list
>> stdlib-sig at python.org
>> http://mail.python.org/mailman/listinfo/stdlib-sig
>
>
>
> --
> Yuv
> http://code.google.com/p/pyopt/
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>
>

From holger at merlinux.eu  Fri Sep 11 00:07:37 2009
From: holger at merlinux.eu (holger krekel)
Date: Fri, 11 Sep 2009 00:07:37 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
Message-ID: <20090910220737.GC15455@trillke.net>

On Thu, Sep 10, 2009 at 18:31 -0300, Collin Winter wrote:
> On Thu, Sep 10, 2009 at 6:14 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >
> >> Upfront people need to realize that we might have three argument
> >> parsing libraries for a while, but it won't be forever. If we get
> >> argparse accepted we would slowly deprecate at least optparse, if not
> >> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> >> getopt supported stuff optparse didn't),
> >
> > +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
> > perfectly functional and useful module is stupid.
> 
> Do remember that if optparse is deprecated, it will still be available
> for *years*. Code isn't going to suddenly break overnight. Users will
> see this coming far, far ahead of time.

i'd like to write keep writing tools that work across several
python versions and python interpreters.  how is removing
optparse in say 2011 going to help me as a tool writer? 

holger

From solipsis at pitrou.net  Fri Sep 11 00:04:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 11 Sep 2009 00:04:32 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost> <4AA96E01.2020601@voidspace.org.uk>
	<4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
Message-ID: <1252620272.5933.38.camel@localhost>


> Why doesn't this argument apply to getopt and optparse equally?

Because optparse has been, for years, the de facto standard API for
parsing command-line arguments. It is reasonably Pythonic and flexible,
if perhaps not as sleek and concise as other alternatives.
getopt is much lower-level, archaic, and probably only appeals to people
who are used to the C version of it.

> The language can be stable, but if the entire 3.x series is just a
> series of bugfixes on top of 3.0's stdlib, then that's a poor state of
> affairs.

This is a false dilemma. There is nothing wrong with introducing new
things. Deprecation of old things, however, should be done carefully,
and with two things in mind:
- the interest of users
- our interest as Python developers and maintainers

It seems to me that optparse has been needing little maintenance, so at
least the second reason doesn't apply here. As for the first reason,
since optparse isn't inherently insecure, fragile, platform-specific, or
wildly under-performing, I don't see it being applicable either.

The main reason, as I understand, is to make the big picture clearer for
new users because there's only one advertised way to do it. While it is
very nice for a clean sheet approach, it certainly isn't a consolation
for people who will have to rewrite their recently broken code...



From barry at python.org  Fri Sep 11 00:11:21 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 10 Sep 2009 18:11:21 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252619040.5933.19.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org>
	<1252619040.5933.19.camel@localhost>
Message-ID: <1DDCB937-D5E1-4325-80B4-1957340BF714@python.org>

On Sep 10, 2009, at 5:44 PM, Antoine Pitrou wrote:

> Le jeudi 10 septembre 2009 ? 17:30 -0400, Barry Warsaw a ?crit :
>> Why does deprecation == breaking?
>
> Well, you have to rewrite your code one day. And since optparse is
> *very* widely used, tons of users will be impacted. Python is often  
> used
> as a scripting language, even for small one-off scripts that nobody
> wants or even knows to maintain.
>
> Besides, any spurious message on the standard output breaks things  
> like
> cron scripts, so even the deprecation period can be annoying.
>
> Please do not assume that every Python program is a well-maintained
> application with dedicated full-time developers...

Then we have a serious long term problem.  Taken to its logical  
conclusion, we're going to end up with a batteries included standard  
library with nothing but old, /effectively/ obsolete APIs with all the  
good new stuff that people want in the Cheeseshop.  Okay, in reality  
it will always be a mix, but I do think this policy means that the  
APIs in the stdlib will degrade and ultimately decay over time.

If that's the case, why even have a stdlib?

>> If you want to continue using optparse, that'll be fine for what?
>> 18-36 months or so?  It's a trivial amount of work to port from
>> optparse to argparse.
>
> Ok, if it's trivial then perhaps I'd reconsider my opinion.
> Is there a compatibility API?

I don't believe so, but I think the changes could be more or less  
mechanically applied. Maybe we should be extending the basic 2to3  
technology to include API upgrades?

> (please note we have e.g. a lot of XML parsing libraries in the  
> stdlib,
> and nobody seems in a hurry to deprecate them)

opening-that-old-can-of-worms-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/77953481/attachment.pgp>

From ubershmekel at gmail.com  Fri Sep 11 00:11:35 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 11 Sep 2009 01:11:35 +0300
Subject: [stdlib-sig] Pyopt - command-line options that are alot easier
	than optparse
In-Reply-To: <20090910100300.GQ15455@trillke.net>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
	<20090910100300.GQ15455@trillke.net>
Message-ID: <9d153b7c0909101511i2f6e76dfod55e276c8adb0aa4@mail.gmail.com>

>
>
>    @cmdline
>    def main(archer=str, boulder=float, magic=42):
>
> which would wrap a parser around the function
> and otherwise work like your example. This can
> easily work down to Python2.3.
>
> cheers,
> holger
>

That is a really clever idea, it took me a while just to understand how it'd
work. If it weren't so hacky-lookin I would indeed consider it. I mean, any
pythonista would look and say "dude, you've got a bug, that doesn't make
sense". Either way, I'm gonna add pyopt to PyPI

-- 
Yuv
http://code.google.com/p/pyopt/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090911/0778b268/attachment-0001.htm>

From brett at python.org  Fri Sep 11 00:16:08 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 15:16:08 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <20090910220737.GC15455@trillke.net>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost>
	<43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com> 
	<20090910220737.GC15455@trillke.net>
Message-ID: <bbaeab100909101516o411653a2uf8b7b0e3bb6f0e41@mail.gmail.com>

On Thu, Sep 10, 2009 at 15:07, holger krekel <holger at merlinux.eu> wrote:
> On Thu, Sep 10, 2009 at 18:31 -0300, Collin Winter wrote:
>> On Thu, Sep 10, 2009 at 6:14 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> >
>> >> Upfront people need to realize that we might have three argument
>> >> parsing libraries for a while, but it won't be forever. If we get
>> >> argparse accepted we would slowly deprecate at least optparse, if not
>> >> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> >> getopt supported stuff optparse didn't),
>> >
>> > +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>> > perfectly functional and useful module is stupid.
>>
>> Do remember that if optparse is deprecated, it will still be available
>> for *years*. Code isn't going to suddenly break overnight. Users will
>> see this coming far, far ahead of time.
>
> i'd like to write keep writing tools that work across several
> python versions and python interpreters. ?how is removing
> optparse in say 2011 going to help me as a tool writer?

 By letting you use argparse in new code that will eventually work
across several versions. And the deprecation can lead to moving
optparse to the Cheeseshop, meaning you only have to add a new
dependency to keep your tools running.

As Barry said in another email, if we are not going to occasionally
update the standard library to new code we are going to end up with
stuff that is rotting or odd because we had to patch around
backwards-compatibility. And if we are doing that we might as well
remove the standard library instead of wasting developer time
maintaining junk that no one uses. I for one am not ready to do that.

-Brett

From michael at voidspace.org.uk  Fri Sep 11 00:17:54 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 10 Sep 2009 23:17:54 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <20090910220737.GC15455@trillke.net>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>	<1252617299.5933.0.camel@localhost>	<43aa6ff70909101431g69d12ac4q6106621f6ce8e6a5@mail.gmail.com>
	<20090910220737.GC15455@trillke.net>
Message-ID: <4AA97B12.9010302@voidspace.org.uk>

holger krekel wrote:
> On Thu, Sep 10, 2009 at 18:31 -0300, Collin Winter wrote:
>   
>> On Thu, Sep 10, 2009 at 6:14 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>     
>>>> Upfront people need to realize that we might have three argument
>>>> parsing libraries for a while, but it won't be forever. If we get
>>>> argparse accepted we would slowly deprecate at least optparse, if not
>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>>> getopt supported stuff optparse didn't),
>>>>         
>>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>>> perfectly functional and useful module is stupid.
>>>       
>> Do remember that if optparse is deprecated, it will still be available
>> for *years*. Code isn't going to suddenly break overnight. Users will
>> see this coming far, far ahead of time.
>>     
>
> i'd like to write keep writing tools that work across several
> python versions and python interpreters.  how is removing
> optparse in say 2011 going to help me as a tool writer? 
>   

argparse works with current versions of Python, it will just no longer 
be an additional dependency on recent versions once we include it.

If you are determined to stick with optparse it becomes an external 
dependency if you are using a version of Python that doesn't include it.

Michael

> holger
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From barry at python.org  Fri Sep 11 00:18:28 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 10 Sep 2009 18:18:28 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252620272.5933.38.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>
Message-ID: <B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>

On Sep 10, 2009, at 6:04 PM, Antoine Pitrou wrote:

> The main reason, as I understand, is to make the big picture clearer  
> for
> new users because there's only one advertised way to do it. While it  
> is
> very nice for a clean sheet approach, it certainly isn't a consolation
> for people who will have to rewrite their recently broken code...

Perhaps this indicates a solution then.  Let's leave getopt and  
optparse in the stdlib as long as they aren't totally bitrotted into  
non-functionality, or as long as there is someone willing to maintain  
them as the language evolves.  But let's effectively deprecate them by  
moving their documentation to the backwaters so the average Python  
programmer will not find them, but will find argparse instead.  Once  
argparse is in the stdlib, all new code should use be using it  
exclusively.

Remember there's more new code that hasn't been written yet than old  
code needing to be maintained :).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/0dec33c0/attachment.pgp>

From brett at python.org  Fri Sep 11 00:35:25 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 15:35:25 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost> <4AA96E01.2020601@voidspace.org.uk>
	<4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com> 
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
Message-ID: <bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>

On Thu, Sep 10, 2009 at 15:18, Barry Warsaw <barry at python.org> wrote:
> On Sep 10, 2009, at 6:04 PM, Antoine Pitrou wrote:
>
>> The main reason, as I understand, is to make the big picture clearer for
>> new users because there's only one advertised way to do it. While it is
>> very nice for a clean sheet approach, it certainly isn't a consolation
>> for people who will have to rewrite their recently broken code...
>
> Perhaps this indicates a solution then. ?Let's leave getopt and optparse in
> the stdlib as long as they aren't totally bitrotted into non-functionality,
> or as long as there is someone willing to maintain them as the language
> evolves. ?But let's effectively deprecate them by moving their documentation
> to the backwaters so the average Python programmer will not find them, but
> will find argparse instead. ?Once argparse is in the stdlib, all new code
> should use be using it exclusively.
>

As long as they always raise a deprecation warning and it is clear
they are no longer maintained then fine. Modules could be in stages
of:

1. pending deprecation
2. deprecation
3. perpetual, unmaintained bitrot (w/ docs removed; deprecation can
point to last version w/ docs)
4. deletion when we think no one is using it anymore (maybe next major
revision of Python)

And this should only apply to pure Python code as I don't want build
dependency issues lying around forever. If we can make sure that the
Cheeseshop gets its version updated after every micro release while it
is in stage 1 or 2 this could actually work.

-Brett

From jbaker at zyasoft.com  Fri Sep 11 00:40:17 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Thu, 10 Sep 2009 16:40:17 -0600
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
Message-ID: <d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>

+1 - I don't want Python being like Java where the only deprecation that
occurs is when code is actively harmful - and it still won't be removed for
backwards compatibility's sake.
On Thu, Sep 10, 2009 at 4:35 PM, Brett Cannon <brett at python.org> wrote:

> On Thu, Sep 10, 2009 at 15:18, Barry Warsaw <barry at python.org> wrote:
> > On Sep 10, 2009, at 6:04 PM, Antoine Pitrou wrote:
> >
> >> The main reason, as I understand, is to make the big picture clearer for
> >> new users because there's only one advertised way to do it. While it is
> >> very nice for a clean sheet approach, it certainly isn't a consolation
> >> for people who will have to rewrite their recently broken code...
> >
> > Perhaps this indicates a solution then.  Let's leave getopt and optparse
> in
> > the stdlib as long as they aren't totally bitrotted into
> non-functionality,
> > or as long as there is someone willing to maintain them as the language
> > evolves.  But let's effectively deprecate them by moving their
> documentation
> > to the backwaters so the average Python programmer will not find them,
> but
> > will find argparse instead.  Once argparse is in the stdlib, all new code
> > should use be using it exclusively.
> >
>
> As long as they always raise a deprecation warning and it is clear
> they are no longer maintained then fine. Modules could be in stages
> of:
>
> 1. pending deprecation
> 2. deprecation
> 3. perpetual, unmaintained bitrot (w/ docs removed; deprecation can
> point to last version w/ docs)
> 4. deletion when we think no one is using it anymore (maybe next major
> revision of Python)
>
> And this should only apply to pure Python code as I don't want build
> dependency issues lying around forever. If we can make sure that the
> Cheeseshop gets its version updated after every micro release while it
> is in stage 1 or 2 this could actually work.
>
> -Brett
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090910/c5916530/attachment-0001.htm>

From brett at python.org  Fri Sep 11 00:47:52 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 15:47:52 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<1252617299.5933.0.camel@localhost> <4AA96E01.2020601@voidspace.org.uk>
	<4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com> 
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org> 
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com> 
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
Message-ID: <bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>

On Thu, Sep 10, 2009 at 15:40, Jim Baker <jbaker at zyasoft.com> wrote:
> +1 - I don't want Python being like Java where the only deprecation that
> occurs is when code is actively harmful - and it still won't be removed for
> backwards compatibility's sake.

The deprecation would be permanent, so it won't be like Java where we
support stuff only until we consider it harmful. And unlike Java we
would not keep the docs around forever, suggesting that you can just
simply ignore what the docs say. The modules would eventually
disappear from people's knowledge.

This idea is not far off from what we already do. There were a bunch
of deprecated modules in the standard library I ripped out for Python
3 that had been slated for removal for ages. It's just nobody bothered
to remove them.

> On Thu, Sep 10, 2009 at 4:35 PM, Brett Cannon <brett at python.org> wrote:
>>
>> On Thu, Sep 10, 2009 at 15:18, Barry Warsaw <barry at python.org> wrote:
>> > On Sep 10, 2009, at 6:04 PM, Antoine Pitrou wrote:
>> >
>> >> The main reason, as I understand, is to make the big picture clearer
>> >> for
>> >> new users because there's only one advertised way to do it. While it is
>> >> very nice for a clean sheet approach, it certainly isn't a consolation
>> >> for people who will have to rewrite their recently broken code...
>> >
>> > Perhaps this indicates a solution then. ?Let's leave getopt and optparse
>> > in
>> > the stdlib as long as they aren't totally bitrotted into
>> > non-functionality,
>> > or as long as there is someone willing to maintain them as the language
>> > evolves. ?But let's effectively deprecate them by moving their
>> > documentation
>> > to the backwaters so the average Python programmer will not find them,
>> > but
>> > will find argparse instead. ?Once argparse is in the stdlib, all new
>> > code
>> > should use be using it exclusively.
>> >
>>
>> As long as they always raise a deprecation warning and it is clear
>> they are no longer maintained then fine. Modules could be in stages
>> of:
>>
>> 1. pending deprecation
>> 2. deprecation
>> 3. perpetual, unmaintained bitrot (w/ docs removed; deprecation can
>> point to last version w/ docs)
>> 4. deletion when we think no one is using it anymore (maybe next major
>> revision of Python)
>>
>> And this should only apply to pure Python code as I don't want build
>> dependency issues lying around forever. If we can make sure that the
>> Cheeseshop gets its version updated after every micro release while it
>> is in stage 1 or 2 this could actually work.
>>
>> -Brett
>> _______________________________________________
>> stdlib-sig mailing list
>> stdlib-sig at python.org
>> http://mail.python.org/mailman/listinfo/stdlib-sig
>
>
>
> --
> Jim Baker
> jbaker at zyasoft.com
>

From dgou at mac.com  Thu Sep 10 23:49:51 2009
From: dgou at mac.com (Douglas Philips)
Date: Thu, 10 Sep 2009 17:49:51 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1252619040.5933.19.camel@localhost>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org>
	<1252619040.5933.19.camel@localhost>
Message-ID: <3A92C1F2-F57B-4F1C-8EE9-5CC9490D1355@mac.com>

On or about 2009 Sep 10, at 5:44 PM, Antoine Pitrou indited:
> Well, you have to rewrite your code one day. And since optparse is
> *very* widely used, tons of users will be impacted. Python is often  
> used
> as a scripting language, even for small one-off scripts that nobody
> wants or even knows to maintain.

If they don't even upgrade their Python environment, they can't  
possibly be affected by this.
In large corp. environments, moving from 2.4.4 to 2.5 or 2.6 is all  
based on "what NEW stuff can we leverage." Once the libraries are  
dead, there is no reason to change versions at all, it is just too  
many installations to bother with.

> (please note we have e.g. a lot of XML parsing libraries in the  
> stdlib,
> and nobody seems in a hurry to deprecate them)

If we can't even get traction on something virtually trivial like  
argparse, what's the point.

-Doug


From collinw at gmail.com  Fri Sep 11 00:55:42 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 10 Sep 2009 19:55:42 -0300
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
	<bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
Message-ID: <43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com>

On Thu, Sep 10, 2009 at 7:47 PM, Brett Cannon <brett at python.org> wrote:
> On Thu, Sep 10, 2009 at 15:40, Jim Baker <jbaker at zyasoft.com> wrote:
>> +1 - I don't want Python being like Java where the only deprecation that
>> occurs is when code is actively harmful - and it still won't be removed for
>> backwards compatibility's sake.
>
> The deprecation would be permanent, so it won't be like Java where we
> support stuff only until we consider it harmful. And unlike Java we
> would not keep the docs around forever, suggesting that you can just
> simply ignore what the docs say. The modules would eventually
> disappear from people's knowledge.

If we allow the code to survive, eventually someone will need to
maintain an application that uses the deprecated, now-docless module.
If I were that person, I would curse the names of whoever left the
code there but removed the docs and allowed bugs to creep in (if the
module disappears from people's knowledge, it will likely disappear
from our consciousness).

Honest question, having only read the docs about argparse: would it be
possible to merge the functionality of argparse into optparse and so
preserve a greater measure of backwards compatibility? Some of the
stuff I'm reading about in
http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
looks like it should be able to be integrated fairly easily into the
existing optparse structure.

Collin Winter

From brett at python.org  Fri Sep 11 01:06:52 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 10 Sep 2009 16:06:52 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com> 
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com> 
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org> 
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com> 
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com> 
	<bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com> 
	<43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com>
Message-ID: <bbaeab100909101606v1944797eq59a43832868cd657@mail.gmail.com>

On Thu, Sep 10, 2009 at 15:55, Collin Winter <collinw at gmail.com> wrote:
> On Thu, Sep 10, 2009 at 7:47 PM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Sep 10, 2009 at 15:40, Jim Baker <jbaker at zyasoft.com> wrote:
>>> +1 - I don't want Python being like Java where the only deprecation that
>>> occurs is when code is actively harmful - and it still won't be removed for
>>> backwards compatibility's sake.
>>
>> The deprecation would be permanent, so it won't be like Java where we
>> support stuff only until we consider it harmful. And unlike Java we
>> would not keep the docs around forever, suggesting that you can just
>> simply ignore what the docs say. The modules would eventually
>> disappear from people's knowledge.
>
> If we allow the code to survive, eventually someone will need to
> maintain an application that uses the deprecated, now-docless module.
> If I were that person, I would curse the names of whoever left the
> code there but removed the docs and allowed bugs to creep in (if the
> module disappears from people's knowledge, it will likely disappear
> from our consciousness).
>

The docs would live on w/ the Python release that last contained it.
Plus the Cheeseshop release would have a copy of the docs.

I think PEP 4 (http://www.python.org/dev/peps/pep-0004/) needs to be
tightened up to flat-out explain what a deprecation means and how long
the code and docs will survive at minimum.

> Honest question, having only read the docs about argparse: would it be
> possible to merge the functionality of argparse into optparse and so
> preserve a greater measure of backwards compatibility? Some of the
> stuff I'm reading about in
> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
> looks like it should be able to be integrated fairly easily into the
> existing optparse structure.

Don't know. Would have to ask Steven about that. I wanted to first see
if people liked the idea of argparse enough.

-Brett

From jnoller at gmail.com  Fri Sep 11 03:45:59 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 10 Sep 2009 21:45:59 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <4222a8490909101845k4aca8d8bl793a5f8da52ade86@mail.gmail.com>

On Thu, Sep 10, 2009 at 2:56 PM, Brett Cannon <brett at python.org> wrote:
> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't), out of the standard library
> and toss them into PyPI for those who refuse to switch. The standard
> library might not evolve a lot, but it isn't dead or in stone.
>
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.
>
> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.
>
> I am +0

+1 to deprecation; +1 for inclusion.

jesse

From lac at openend.se  Fri Sep 11 06:37:21 2009
From: lac at openend.se (Laura Creighton)
Date: Fri, 11 Sep 2009 06:37:21 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: Message from Barry Warsaw <barry@python.org> of "Thu,
	10 Sep 2009 18:11:21 EDT."
	<1DDCB937-D5E1-4325-80B4-1957340BF714@python.org> 
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org>
	<1252619040.5933.19.camel@localhost>
	<1DDCB937-D5E1-4325-80B4-1957340BF714@python.org> 
Message-ID: <200909110437.n8B4bMdE024567@theraft.openend.se>

In a message of Thu, 10 Sep 2009 18:11:21 EDT, Barry Warsaw writes:
>Then we have a serious long term problem.  Taken to its logical =20
>conclusion, we're going to end up with a batteries included standard =20
>library with nothing but old, /effectively/ obsolete APIs with all the =2
>0=
>
>good new stuff that people want in the Cheeseshop.  Okay, in reality =20
>it will always be a mix, but I do think this policy means that the =20
>APIs in the stdlib will degrade and ultimately decay over time.
>
>If that's the case, why even have a stdlib?

That is, actually, pretty much what I want a stdlib for.  I don't want
it to contain the newest, greatest, and best ways of doing things.  I
want it to contain the things that people are willing to maintain until
hell freezes over, which will largely be things that aren't ever going
to change until hell freezes over.

I want users to know that if they want the latest, greatest, and best
ways of doing things -- the way we recommend people do things now --
then they should not be assuming that the standard library has it.
The point of using things in the standard library is so that you
will have a reason to believe that if you write your code using
this stuff, then you don't have to worry (much, at any rate, there
is always Python 3.0) about having to change your code at some later
date because the standard library now has a new way of doing things
which some people like a lot better than the old way you used.

Right now there are a whole lot of users of python programs who have
no programming skills whatsoever.  Waking up one morning and discovering
that their computing environment has gone to hell because the default
Python is now 3.7 and now the option parsing in their programs don't
work any more -- and they will have to get a replacement from the
original author of the tool, or some other programmer seems a remarkably
uncivil way to treat your users.

If new python programmers cannot be trusted to read the standard library
manual and understand what a paragraph that says:

       XXX is included in the standard library for reasons of
       backwards compatibility.  We strongly recommend that you use YYY
       instead.

then they are too clueless to worry about.  But if, having read this,
they choose to use XXX anyway, as today many people use getopt instead
optparse, when they are writing python programs that will
only be deployed on windows and who want the /option syntax that optparse
denies them, well then, that's their choice too.

I don't know anything about argparse, but reading the docs leads me to
believe it is a big improvement on optparse, which I dislike.  But
Simon Willison's optfunc http://simonwillison.net/2009/May/28/optfunc/
looks interesting to me as well, as would a parser that took its information
from docstrings.

There are two foolish attitudes here.  The first is that being tidy, or
impossible for newbies to misunderstand, because we won't give them the
option of using the wrong stuff is a higher value than not breaking
perfectly good working code.  It's better to teach the newbies that
there are lots of options out there than to hide this fact from them.

The second foolish attitude is that nobody seems to be discussing
'do we like argparse enough to be willing to maintain it for the
indefinite future as part of the standard library'.  Because, to my
mind, that is the real question.  If we no longer care about that,
and things in the standard library can be removed 'whenever we have
a replacement that we like better', then the standard library is no
longer really a _standard_ library, but rather a _stuff we suggest
you use_ library.  And do we really want one of those?

I sure don't.

Laura


<snip>
>-Barry

From lac at openend.se  Fri Sep 11 06:48:21 2009
From: lac at openend.se (Laura Creighton)
Date: Fri, 11 Sep 2009 06:48:21 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: Message from Collin Winter <collinw@gmail.com> of "Thu,
	10 Sep 2009 19:55:42 -0300."
	<43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com> 
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
	<bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
	<43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com> 
Message-ID: <200909110448.n8B4mL5H025784@theraft.openend.se>

In a message of Thu, 10 Sep 2009 19:55:42 -0300, Collin Winter writes:
>Honest question, having only read the docs about argparse: would it be
>possible to merge the functionality of argparse into optparse and so
>preserve a greater measure of backwards compatibility? Some of the
>stuff I'm reading about in
>http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
>looks like it should be able to be integrated fairly easily into the
>existing optparse structure.
>
>Collin Winter

Having looked at the code for optparse, I would say, no, hacking the
optparse enhancements into optparse would be extremely difficult.  optparse
was designed by somebody who finds certain common patterns of option
parsing (required options, parsing options passed as /whatever) such 
a terrible idea that preventing people from doing these things, even
if they -really-, -really-, wanted to was a significant design goal.

Laura

From holger at merlinux.eu  Fri Sep 11 08:30:38 2009
From: holger at merlinux.eu (holger krekel)
Date: Fri, 11 Sep 2009 08:30:38 +0200
Subject: [stdlib-sig] Pyopt - command-line options that are alot
	easier	than optparse
In-Reply-To: <9d153b7c0909101511i2f6e76dfod55e276c8adb0aa4@mail.gmail.com>
References: <9d153b7c0909091738w6b05d1aeq9e70d58300a2daee@mail.gmail.com>
	<20090910100300.GQ15455@trillke.net>
	<9d153b7c0909101511i2f6e76dfod55e276c8adb0aa4@mail.gmail.com>
Message-ID: <20090911063038.GD15455@trillke.net>

On Fri, Sep 11, 2009 at 01:11 +0300, Yuvgoog Greenle wrote:
> >    @cmdline
> >    def main(archer=str, boulder=float, magic=42):
> >
> > which would wrap a parser around the function
> > and otherwise work like your example. This can
> > easily work down to Python2.3.
> >
> > cheers,
> > holger
> >
> 
> That is a really clever idea, it took me a while just to understand how it'd
> work. If it weren't so hacky-lookin I would indeed consider it. I mean, any
> pythonista would look and say "dude, you've got a bug, that doesn't make
> sense". 

hehe, sure.  one of the reasons it didn't go public. 

> Either way, I'm gonna add pyopt to PyPI

cheers,
holger

> -- 
> Yuv
> http://code.google.com/p/pyopt/

-- 
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu 

From steven.bethard at gmail.com  Fri Sep 11 09:24:54 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 11 Sep 2009 00:24:54 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <9d153b7c0909101458l2254baf2jed9ff6e585b212e@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<4AA971F6.8060707@voidspace.org.uk>
	<9d153b7c0909101458l2254baf2jed9ff6e585b212e@mail.gmail.com>
Message-ID: <d11dcfba0909110024l56bd5819lf146ecd0e868c122@mail.gmail.com>

On Thu, Sep 10, 2009 at 2:58 PM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> I just wrote pyopt?http://code.google.com/p/pyopt/?because I think exposing
> functions to the command line is way too hard.
> If?Steven Bethard and the argparse community agree, I?would really
> appreciate a convenience method named?"add_function"?built into
> the?ArgumentParser. add_function would automagically read the docstring and
> understand what are the arguments for a given function. I'm willing and
> motivated to do the work for this. Annotations-type-casting could be awesome
> as well, but I'm not greedy enough to ask.

Regardless of what happens for argparse in the standard library, if
you're willing to provide a patch to add this to argparse, I'd be
happy to review it and include it in argparse.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Fri Sep 11 09:38:25 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 11 Sep 2009 00:38:25 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
	<bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
	<43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com>
Message-ID: <d11dcfba0909110038i30de8e1dr58bcfc9bc7f3632b@mail.gmail.com>

On Thu, Sep 10, 2009 at 3:55 PM, Collin Winter <collinw at gmail.com> wrote:
> Honest question, having only read the docs about argparse: would it be
> possible to merge the functionality of argparse into optparse and so
> preserve a greater measure of backwards compatibility? Some of the
> stuff I'm reading about in
> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
> looks like it should be able to be integrated fairly easily into the
> existing optparse structure.

I tried this, and when I originally started argparse, it was my intent
to make it fully compatible with optparse. For a simple example,
consider the documented "parser.largs", "parser.rargs" and
"parser.values" attributes of OptionParser. Supporting these would not
allow argparse's current parsing strategy, which doesn't follow the
optparse approach of moving args from "largs" to "rargs".

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Fri Sep 11 09:52:30 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 11 Sep 2009 00:52:30 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <d11dcfba0909110052m60cbae3as73d401a0532ebeb3@mail.gmail.com>

On Thu, Sep 10, 2009 at 11:56 AM, Brett Cannon <brett at python.org> wrote:
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.

Just in case it wasn't already clear, I'm happy to write a PEP for
argparse if we come to some consensus on (1) whether or not it should
be included and (2) whether or not it should be accompanied by a
deprecation process for getopt and optparse.

FWIW, I currently sit in the camp of adding big glaring "this module
is deprecated" notes in the documentation for getopt and optparse, and
sticking PendingDeprecation warnings into them. PendingDeprecation
warnings are off by default so shouldn't cause problems for the vast
majority of current users of these modules. We can switch those
PendingDeprecation warnings to Deprecation warnings when, say, we see
more hits for argparse than for optparse on Google code search. Right
now, I see optparse at around 23,000 [1] and argparse at only around
400, so I imagine that would be at least a few years.

Steve

[1]http://www.google.com/codesearch?q=optparse++lang:python&ct=rr&cs_r=lang:python
[2]http://www.google.com/codesearch?q=argparse++lang:python&ct=rr&cs_r=lang:python
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Fri Sep 11 09:54:53 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 11 Sep 2009 00:54:53 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <d11dcfba0909110052m60cbae3as73d401a0532ebeb3@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<d11dcfba0909110052m60cbae3as73d401a0532ebeb3@mail.gmail.com>
Message-ID: <d11dcfba0909110054l6e014ee6qcd623a551883c9be@mail.gmail.com>

On Fri, Sep 11, 2009 at 12:52 AM, Steven Bethard
<steven.bethard at gmail.com> wrote:
> FWIW, I currently sit in the camp of adding big glaring "this module
> is deprecated" notes in the documentation for getopt and optparse, and
> sticking PendingDeprecation warnings into them. PendingDeprecation
> warnings are off by default so shouldn't cause problems for the vast
> majority of current users of these modules. We can switch those
> PendingDeprecation warnings to Deprecation warnings when, say, we see
> more hits for argparse than for optparse on Google code search. Right
> now, I see optparse at around 23,000 [1] and argparse at only around
> 400, so I imagine that would be at least a few years.

(And many of those 400 aren't even for the argparse package itself,
just other classes and methods with the same name.)

Steve

> [1]http://www.google.com/codesearch?q=optparse++lang:python&ct=rr&cs_r=lang:python
> [2]http://www.google.com/codesearch?q=argparse++lang:python&ct=rr&cs_r=lang:python
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From mal at egenix.com  Fri Sep 11 11:27:03 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Sep 2009 11:27:03 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk>	<4AA9715E.8080403@egenix.com>	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
	<bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
Message-ID: <4AAA17E7.7050509@egenix.com>

Brett Cannon wrote:
> On Thu, Sep 10, 2009 at 15:40, Jim Baker <jbaker at zyasoft.com> wrote:
>> +1 - I don't want Python being like Java where the only deprecation that
>> occurs is when code is actively harmful - and it still won't be removed for
>> backwards compatibility's sake.
> 
> The deprecation would be permanent, so it won't be like Java where we
> support stuff only until we consider it harmful. And unlike Java we
> would not keep the docs around forever, suggesting that you can just
> simply ignore what the docs say. The modules would eventually
> disappear from people's knowledge.
> 
> This idea is not far off from what we already do. There were a bunch
> of deprecated modules in the standard library I ripped out for Python
> 3 that had been slated for removal for ages. It's just nobody bothered
> to remove them.

Right and this was done when transitioning from 2.x to 3.x - which
is fine. Backwards compatibility can be broken in such major release
steps.

However, it should be noted that the deprecated modules were mostly
targeting obsolete architectures, their functionality was moved
to other modules, or they were so obscure that the number of people
using them warranted deprecating and then finally removing them.

None of the above applies to the getopt and optparse modules.

getopt mimics the C level function of the same name and provides
an easy entry for C programmers. It has this approach in common
with many other Python modules and exposed APIs.

optparse has a more Python-friendly approach to option parsing.

Both serve their needs and both are in common use.

Note that the purpose of the stdlib is to provide tools to be
able to get things working without having to rely on 3rd party
tools.

The main motivation behind the stdlib is stability and soundness,
much more than providing the latest and greatest gimmicks you can
possibly imagine. It provides the basic set of tools (and does
an very good job at this).

Users are free to use other modules or libraries, if they believe
that a certain API or way of doing things suits them better.

Just because one group of people thinks that the shiny brand
new method B suits them better, doesn't mean that it's OK to
put the extra burden of porting applications using method A
to method B on the happy users of method A.

Even less so, since nothing stops people from using method B
by downloading and installing their favorite tool from PyPI.

And another aspect to consider: instead of replacing existing
tools with new ones, it's often better to think a little harder
and add the missing functionality to the existing tools.
That approach buys you more features and more stability at
much lower maintenance costs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 11 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From p.f.moore at gmail.com  Fri Sep 11 12:51:49 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 11 Sep 2009 11:51:49 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <200909110437.n8B4bMdE024567@theraft.openend.se>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org>
	<1252619040.5933.19.camel@localhost> <barry@python.org>
	<1DDCB937-D5E1-4325-80B4-1957340BF714@python.org>
	<200909110437.n8B4bMdE024567@theraft.openend.se>
Message-ID: <79990c6b0909110351l257902bawaac76a1582cda08a@mail.gmail.com>

2009/9/11 Laura Creighton <lac at openend.se>:
> That is, actually, pretty much what I want a stdlib for. ?I don't want
> it to contain the newest, greatest, and best ways of doing things. ?I
> want it to contain the things that people are willing to maintain until
> hell freezes over, which will largely be things that aren't ever going
> to change until hell freezes over.

Hmm, interesting. What *I* want from the stdlib is a readily-available
library of best practice (or at least, sufficiently good and
production-quality) tools covering a wide range of common programming
tasks. In other words, unless I'm doing something specialised, I want
to be able to write code without external dependencies.

Simple examples:

- random uses Mersenne Twister, which is a suitable choice for "most
people". I don't want something like C's rand() (which isn't strong
enough - or wasn't last time I used it...) or Haskell's System.Random
(which provides stronger guarantees which aren't relevant to normal
use, at a performance cost).
- hashlib provides current best practice hash algorithms, implemented
efficiently. I wouldn't want to only have md5 in the stdlib.
- http.server (SimpleHTTPServer) may not be as good as Twisted, but
it's good enough for a simple web interface.

So, while I don't necessarily want "newest, greatest and best", I do
expect "up to date, production quality, and current good practice". If
I wanted unchanging, I wouldn't upgrade my Python installation.

Paul.

From solipsis at pitrou.net  Fri Sep 11 13:03:52 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 11 Sep 2009 13:03:52 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <79990c6b0909110351l257902bawaac76a1582cda08a@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<A6B44862-C103-4668-9ABF-DA9038726929@python.org>
	<1252619040.5933.19.camel@localhost> <barry@python.org>
	<1DDCB937-D5E1-4325-80B4-1957340BF714@python.org>
	<200909110437.n8B4bMdE024567@theraft.openend.se>
	<79990c6b0909110351l257902bawaac76a1582cda08a@mail.gmail.com>
Message-ID: <1252667032.5620.3.camel@localhost>

Le vendredi 11 septembre 2009 ? 11:51 +0100, Paul Moore a ?crit :
> So, while I don't necessarily want "newest, greatest and best", I do
> expect "up to date, production quality, and current good practice".

Well, optparse *is* production quality and rather good practice.
It may not be the best API out there but it's certainly robust,
functional, and doesn't have any glaring flaws (like security or
performance problems).

> If
> I wanted unchanging, I wouldn't upgrade my Python installation.

There are lots of reasons for upgrading a Python installation, some of
which having to do with upgrading your whole system (especially if
you're under Linux), some of which having to do that older versions
ultimately don't get supported anymore, even for security fixes.

Anyway, I think Barry finally hit the nail on the head: if we include
something "better" than optparse, then we simply have to deemphasize
optparse in the documentation and emphasize its new replacement, so that
people know what to use for new programs.



From lac at openend.se  Fri Sep 11 14:29:25 2009
From: lac at openend.se (Laura Creighton)
Date: Fri, 11 Sep 2009 14:29:25 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: Message from Laura Creighton <lac@openend.se> of "Fri,
	11 Sep 2009 06:48:21 +0200."
	<200909110448.n8B4mL5H025784@theraft.openend.se> 
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk> <4AA9715E.8080403@egenix.com>
	<43aa6ff70909101446g646dd43dia1c601e591b6c39b@mail.gmail.com>
	<1252620272.5933.38.camel@localhost>
	<B18CE463-70E1-45B6-BE42-A2ED36CDE491@python.org>
	<bbaeab100909101535p655adfc3kc5f6f4a8e7040f1b@mail.gmail.com>
	<d03bb4010909101540y3533424cwf322be3421b50525@mail.gmail.com>
	<bbaeab100909101547k5b9d3844vcfbe7ed27e4769b4@mail.gmail.com>
	<43aa6ff70909101555m64532c04ib63121d686761fe3@mail.gmail.com>
	<200909110448.n8B4mL5H025784@theraft.openend.se> 
Message-ID: <200909111229.n8BCTPwl026848@theraft.openend.se>

In a message of Fri, 11 Sep 2009 06:48:21 +0200, Laura Creighton writes:
>In a message of Thu, 10 Sep 2009 19:55:42 -0300, Collin Winter writes:
>>Honest question, having only read the docs about argparse: would it be
>>possible to merge the functionality of argparse into optparse and so
>>preserve a greater measure of backwards compatibility? Some of the
>>stuff I'm reading about in
>>http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
>>looks like it should be able to be integrated fairly easily into the
>>existing optparse structure.
>>
>>Collin Winter
>
>Having looked at the code for optparse, I would say, no, hacking the
>optparse enhancements into optparse would be extremely difficult.  optpar
>se
>was designed by somebody who finds certain common patterns of option
>parsing (required options, parsing options passed as /whatever) such 
>a terrible idea that preventing people from doing these things, even
>if they -really-, -really-, wanted to was a significant design goal.
>
>Laura

Ooops, I see now that I misread your question, as 'can we stick the
new argparse things into optparse' rather than the other way around.
Sorry about that.

Laura

From ronaldoussoren at mac.com  Sun Sep 13 09:31:03 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 13 Sep 2009 09:31:03 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <1afaf6160909101425p14970423o3b202bd7207a16db@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost> <4AA96E01.2020601@voidspace.org.uk>
	<1afaf6160909101425p14970423o3b202bd7207a16db@mail.gmail.com>
Message-ID: <2B5285AC-88EA-49B6-AAE3-ED305A806764@mac.com>


On 10 Sep, 2009, at 23:25, Benjamin Peterson wrote:

> 2009/9/10 Michael Foord <michael at voidspace.org.uk>:
>> Antoine Pitrou wrote:
>>>>
>>>> Upfront people need to realize that we might have three argument
>>>> parsing libraries for a while, but it won't be forever. If we get
>>>> argparse accepted we would slowly deprecate at least optparse, if  
>>>> not
>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued  
>>>> that
>>>> getopt supported stuff optparse didn't),
>>>>
>>>
>>> +0 on deprecating getopt, -1 on deprecating optparse. Breaking a
>>> perfectly functional and useful module is stupid.
>>>
>>>
>>
>> So we're stuck with inferior technology?
>
> No, that's never been the case. Download and install argparse.

optparse's source code isn't cast in stone, therefore its interface  
can evolve.

I'm -0 on deprecating getopt, -1 on deprecating optparse and -0 on  
adding argparse as is.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090913/d93be4f1/attachment.bin>

From steven.bethard at gmail.com  Sun Sep 13 19:52:23 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 13 Sep 2009 10:52:23 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <2B5285AC-88EA-49B6-AAE3-ED305A806764@mac.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<1252617299.5933.0.camel@localhost>
	<4AA96E01.2020601@voidspace.org.uk>
	<1afaf6160909101425p14970423o3b202bd7207a16db@mail.gmail.com>
	<2B5285AC-88EA-49B6-AAE3-ED305A806764@mac.com>
Message-ID: <d11dcfba0909131052h17552d3byfe9cb04a62ac9dbb@mail.gmail.com>

On Sun, Sep 13, 2009 at 12:31 AM, Ronald Oussoren
<ronaldoussoren at mac.com> wrote:
> optparse's source code isn't cast in stone, therefore its interface can
> evolve.

While I agree with this comment in general, because optparse
officially exposes lots of internal details (OptionParser.largs,
OptionParser.rargs, Option.TYPE_CHECKER, Option.ALWAYS_TYPED_ACTIONS,
etc.) in practice it's extremely hard to evolve the optparse codebase
without introducing backwards compatibilities. If you don't believe
me, I recommend trying to implement argparse's support for
type=<callable> in optparse, while respecting Option.TYPES, etc. This
is one of the reasons in argparse I've been extremely conservative on
what constitutes the public API - I don't want the compatibility
nightmare that optparse has.

I guess one option would be to deprecate optparse's entire extension
mechanism and eventually replace it with argparse's, but I don't see
how that would really be any better than just including argparse
alongside optparse.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From collinw at gmail.com  Sun Sep 13 21:43:54 2009
From: collinw at gmail.com (Collin Winter)
Date: Sun, 13 Sep 2009 16:43:54 -0300
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <43aa6ff70909131243y2f98f641ofa2803325c03473a@mail.gmail.com>

On Thu, Sep 10, 2009 at 3:56 PM, Brett Cannon <brett at python.org> wrote:
> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't), out of the standard library
> and toss them into PyPI for those who refuse to switch. The standard
> library might not evolve a lot, but it isn't dead or in stone.
>
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.
>
> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.
>
> I am +0

Brett, if you and Steven could work up a PEP on this subject, I think
that would go a long way toward moving the discussion about our option
parsing options forward. Having a concrete PEP to discuss will be more
useful and more focused than debating the abstract principles at play.

Thanks,
Collin Winter

From solipsis at pitrou.net  Sun Sep 13 21:54:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 13 Sep 2009 21:54:29 +0200
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909131243y2f98f641ofa2803325c03473a@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<43aa6ff70909131243y2f98f641ofa2803325c03473a@mail.gmail.com>
Message-ID: <1252871669.5630.45.camel@localhost>

Le dimanche 13 septembre 2009 ? 16:43 -0300, Collin Winter a ?crit :
> Brett, if you and Steven could work up a PEP on this subject, I think
> that would go a long way toward moving the discussion about our option
> parsing options forward.

Option parsing options?
Wouldn't you prefer an argument parsing argument?

;)



From brett at python.org  Sun Sep 13 22:01:43 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 13 Sep 2009 13:01:43 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <43aa6ff70909131243y2f98f641ofa2803325c03473a@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<43aa6ff70909131243y2f98f641ofa2803325c03473a@mail.gmail.com>
Message-ID: <bbaeab100909131301ibfffa06mc54d48872369a9c8@mail.gmail.com>

On Sun, Sep 13, 2009 at 12:43, Collin Winter <collinw at gmail.com> wrote:
> On Thu, Sep 10, 2009 at 3:56 PM, Brett Cannon <brett at python.org> wrote:
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't), out of the standard library
>> and toss them into PyPI for those who refuse to switch. The standard
>> library might not evolve a lot, but it isn't dead or in stone.
>>
>> But before this can happen, people need to have a general consensus
>> that I should bug Steven about contributing as it will require a PEP
>> from him. Steven already has commit privileges to maintenance from him
>> will not be a problem.
>>
>> So if you want this to actually happen and for me to start talking to
>> Steven just reply to this email w/ a vote.
>>
>> I am +0
>
> Brett, if you and Steven could work up a PEP on this subject, I think
> that would go a long way toward moving the discussion about our option
> parsing options forward. Having a concrete PEP to discuss will be more
> useful and more focused than debating the abstract principles at play.

I was actually planning on sending this email after I finished
checking my inbox. There seems to be enough support to warrant Steven
doing the PEP. We can then see what python-dev ends up thinking of the
whole thing.

-Brett

From steven.bethard at gmail.com  Mon Sep 14 00:08:06 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 13 Sep 2009 15:08:06 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909131301ibfffa06mc54d48872369a9c8@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<43aa6ff70909131243y2f98f641ofa2803325c03473a@mail.gmail.com>
	<bbaeab100909131301ibfffa06mc54d48872369a9c8@mail.gmail.com>
Message-ID: <d11dcfba0909131508x2001b75fq8d2587c779848d19@mail.gmail.com>

On Sun, Sep 13, 2009 at 1:01 PM, Brett Cannon <brett at python.org> wrote:
> On Sun, Sep 13, 2009 at 12:43, Collin Winter <collinw at gmail.com> wrote:
>> Brett, if you and Steven could work up a PEP on this subject, I think
>> that would go a long way toward moving the discussion about our option
>> parsing options forward. Having a concrete PEP to discuss will be more
>> useful and more focused than debating the abstract principles at play.
>
> I was actually planning on sending this email after I finished
> checking my inbox. There seems to be enough support to warrant Steven
> doing the PEP. We can then see what python-dev ends up thinking of the
> whole thing.

Sounds like a plan. I'll try to put a PEP together this week.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From jnoller at gmail.com  Mon Sep 14 17:13:12 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:13:12 -0400
Subject: [stdlib-sig] Breaking out the stdlib
Message-ID: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>

Note, since I drafted this, brett's posted some thought on evolution
as well: http://sayspy.blogspot.com/2009/09/evolving-standard-library.html


So, here's a small pile of thoughts - the main driving force of which
was the common sentiment that was shown in the language summit at last
pycon. I'm mainly sending this out to evoke some discussion.

Last pycon, many of us agreed that the stdlib needed to be "broken"
out of the core within source control. AFAIK, this is still pending
the mercurial migration. The goal of this would be to have one common
stdlib shared amongst the implementations (Jython, Ironpython, etc).
Modules which were cpython-only (such as multiprocessing) would be
kept near-core, or marked as Cpython only.

This means we would have an interpreter/builtins section for cpython,
one for Jython, etc while they could all consume the central stdlib
blob.

In thinking about this even more over the past year(ish) - I've
wondered if the stdlib, and python-core should actually be *really*
separated. I'm a huge fan of batteries included, but I can see the
draw to a breakdown like this:

python-core (no stdlib, interpreter, core language)
python-stdlib (no core)
python-full (the works)

(Note this may actually *help* OS package maintainers.)

Doing this - in my mind - lends itself to the stdlib evolving faster.
Sure, there's a lot of good things in the stdlib, but frankly, we have
over 216 packages in it, and only 113 developers
(http://www.python.org/dev/committers). Of those 113 developers, how
many are actually active? How many of the modules in the stdlib
actually have owners with vested interest in maintaining them? How
many of them are evolutionary dead ends? From a packaging standpoint -
it's a lot easier to spin a new stdlib package and get it into an OS
upstream then the entire interpreter.

The running joke is that the stdlib is where modules go to die -
personally, I don't think this should be true - although it is true to
a certain extent. It's also true that some of the modules within the
stdlib are not best-of-breed - httplib2 vs. urllib/httplib comes to
mind (mainly because I'm dealing with that right now).

We all know that the stdlib has evolved over a great deal of time -
and over time the quality bar has changed (for the better) - but I'd
ask how to we take that quality bar and beat some of the
packages/modules we have in there with it? How do we make sure that
the stdlib is stable, best of breed and high quality?

I would say that it's entirely possible that some things simply need
to be removed; not just platform specific things, but things which
don't have maintainers on the hook to review their patches, things
which have low-to-no test coverage/docs.

I would personally like to see every single stdlib library have an
"owner" - I know, that's a long shot, but I really feel it's needed.
Otherwise you potentially have people reviewing patches for code they
may not fully understand, or not understand the ramifications of.

Breaking out is 1/2 the fight - the other half is cleaning it up/out -
removing things with low test coverage, poor docs or things which
simply are *not* the best option. We need to take a really hard look
at things in there and ask ourselves if the things in there meet our
collective quality bar, and if they don't: remove it. We want a good,
solid and shared-amongst implementations stdlib.

jesse

From doug.hellmann at gmail.com  Mon Sep 14 17:20:31 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 14 Sep 2009 11:20:31 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
Message-ID: <8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>


On Sep 14, 2009, at 11:13 AM, Jesse Noller wrote:

> Note, since I drafted this, brett's posted some thought on evolution
> as well: http://sayspy.blogspot.com/2009/09/evolving-standard-library.html
>
>
> So, here's a small pile of thoughts - the main driving force of which
> was the common sentiment that was shown in the language summit at last
> pycon. I'm mainly sending this out to evoke some discussion.
>
> Last pycon, many of us agreed that the stdlib needed to be "broken"
> out of the core within source control. AFAIK, this is still pending
> the mercurial migration. The goal of this would be to have one common
> stdlib shared amongst the implementations (Jython, Ironpython, etc).
> Modules which were cpython-only (such as multiprocessing) would be
> kept near-core, or marked as Cpython only.
>
> This means we would have an interpreter/builtins section for cpython,
> one for Jython, etc while they could all consume the central stdlib
> blob.
>
> In thinking about this even more over the past year(ish) - I've
> wondered if the stdlib, and python-core should actually be *really*
> separated. I'm a huge fan of batteries included, but I can see the
> draw to a breakdown like this:
>
> python-core (no stdlib, interpreter, core language)
> python-stdlib (no core)
> python-full (the works)

It would be interesting to know what stdlib modules are a minimum  
requirement to install other packages with a tool like easy_install or  
pip.  Those might need to stay in "core" so that installing core gives  
a minimally functional system.

Otherwise, I like the idea.

Doug


From jnoller at gmail.com  Mon Sep 14 17:28:19 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:28:19 -0400
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
Message-ID: <4222a8490909140828q4a564520n973f87e0050f386f@mail.gmail.com>

On Thu, Sep 10, 2009 at 2:56 PM, Brett Cannon <brett at python.org> wrote:
> Upfront people need to realize that we might have three argument
> parsing libraries for a while, but it won't be forever. If we get
> argparse accepted we would slowly deprecate at least optparse, if not
> getopt (lat time I tried to ditch getopt for Python 3 some argued that
> getopt supported stuff optparse didn't), out of the standard library
> and toss them into PyPI for those who refuse to switch. The standard
> library might not evolve a lot, but it isn't dead or in stone.
>
> But before this can happen, people need to have a general consensus
> that I should bug Steven about contributing as it will require a PEP
> from him. Steven already has commit privileges to maintenance from him
> will not be a problem.
>
> So if you want this to actually happen and for me to start talking to
> Steven just reply to this email w/ a vote.
>
> I am +0

More fuel for the pep(fire):

http://blogg.ingspree.net/blog/2009/09/14/opster/

From p.f.moore at gmail.com  Mon Sep 14 17:35:54 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 14 Sep 2009 16:35:54 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
Message-ID: <79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>

2009/9/14 Doug Hellmann <doug.hellmann at gmail.com>:
>> In thinking about this even more over the past year(ish) - I've
>> wondered if the stdlib, and python-core should actually be *really*
>> separated. I'm a huge fan of batteries included, but I can see the
>> draw to a breakdown like this:
>>
>> python-core (no stdlib, interpreter, core language)
>> python-stdlib (no core)
>> python-full (the works)
>
> It would be interesting to know what stdlib modules are a minimum
> requirement to install other packages with a tool like easy_install or pip.
> ?Those might need to stay in "core" so that installing core gives a
> minimally functional system.
>
> Otherwise, I like the idea.

Please remember that some establishments have restrictions that mean
that tools like easy_install or pip cannot be used. In locked-down
corporate environments, python-full is potentially all that will be
available (and maybe very specific "blessed" environment-specific 3rd
party modules).

But if the stdlib can be split out in such a way that it doesn't
adversely impact those environments, then maybe the extra flexibility
to evolve it would be helpful. (I'd like to know how that aligns with
the stated goal of stdlib stability, though - seems to me like it's
one or the other...)

Paul.

From michael at voidspace.org.uk  Mon Sep 14 17:37:10 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 16:37:10 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
Message-ID: <4AAE6326.5090402@voidspace.org.uk>

Paul Moore wrote:
> 2009/9/14 Doug Hellmann <doug.hellmann at gmail.com>:
>   
>>> In thinking about this even more over the past year(ish) - I've
>>> wondered if the stdlib, and python-core should actually be *really*
>>> separated. I'm a huge fan of batteries included, but I can see the
>>> draw to a breakdown like this:
>>>
>>> python-core (no stdlib, interpreter, core language)
>>> python-stdlib (no core)
>>> python-full (the works)
>>>       
>> It would be interesting to know what stdlib modules are a minimum
>> requirement to install other packages with a tool like easy_install or pip.
>>  Those might need to stay in "core" so that installing core gives a
>> minimally functional system.
>>
>> Otherwise, I like the idea.
>>     
>
> Please remember that some establishments have restrictions that mean
> that tools like easy_install or pip cannot be used. In locked-down
> corporate environments, python-full is potentially all that will be
> available (and maybe very specific "blessed" environment-specific 3rd
> party modules).
>
> But if the stdlib can be split out in such a way that it doesn't
> adversely impact those environments, then maybe the extra flexibility
> to evolve it would be helpful. (I'd like to know how that aligns with
> the stated goal of stdlib stability, though - seems to me like it's
> one or the other...)
>   

For reference, where is that stated?

Michael

> Paul.
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Mon Sep 14 17:41:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 14 Sep 2009 17:41:53 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
Message-ID: <1252942913.5954.39.camel@localhost>

Le lundi 14 septembre 2009 ? 11:13 -0400, Jesse Noller a ?crit :
> Note, since I drafted this, brett's posted some thought on evolution
> as well: http://sayspy.blogspot.com/2009/09/evolving-standard-library.html

I tried to comment on it but, after a lot of pain editing my thoughts
(the text editor on that blog is completely broken in my browser), it
seems the comment got lost when I clicked "post"...

> In thinking about this even more over the past year(ish) - I've
> wondered if the stdlib, and python-core should actually be *really*
> separated. I'm a huge fan of batteries included, but I can see the
> draw to a breakdown like this:
> 
> python-core (no stdlib, interpreter, core language)
> python-stdlib (no core)

Note, however, that an interpreter without stdlib is useless. Even basic
I/O (on Python 3) may not function properly. Conversely, the stdlib will
depend on certain interpreter features, or even implementation details.
So, while we can put them in separate VCSes, the development of
python-core and python-stdlib will be still be quite tied.

> python-full (the works)

What is this? core + stdlib?

> From a packaging standpoint -
> it's a lot easier to spin a new stdlib package and get it into an OS
> upstream then the entire interpreter.

I'm not convinced. A new stdlib can lead to as many compatibility
problems as a new interpreter does. And it seems that compatibility /
dependency management is the #1 problem in packaging.

> I would personally like to see every single stdlib library have an
> "owner" - I know, that's a long shot, but I really feel it's needed.
> Otherwise you potentially have people reviewing patches for code they
> may not fully understand, or not understand the ramifications of.

On the other hand, having an owner can be detrimental to maintenance.
For example, nobody wants to touch ElementTree except Fredrik, and
Fredrik isn't very active these days.
We should also say to no to externally-maintained modules, because it
completely ruins maintenance for us core developers.



From jnoller at gmail.com  Mon Sep 14 17:42:57 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:42:57 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
Message-ID: <4222a8490909140842r61aae5eepd77c7349f48bd7d8@mail.gmail.com>

On Mon, Sep 14, 2009 at 11:35 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/14 Doug Hellmann <doug.hellmann at gmail.com>:
>>> In thinking about this even more over the past year(ish) - I've
>>> wondered if the stdlib, and python-core should actually be *really*
>>> separated. I'm a huge fan of batteries included, but I can see the
>>> draw to a breakdown like this:
>>>
>>> python-core (no stdlib, interpreter, core language)
>>> python-stdlib (no core)
>>> python-full (the works)
>>
>> It would be interesting to know what stdlib modules are a minimum
>> requirement to install other packages with a tool like easy_install or pip.
>> ?Those might need to stay in "core" so that installing core gives a
>> minimally functional system.

Brett mentions that in his post; a minimal bootstrap. I didn't think
terribly hard about that aspect.

>> Otherwise, I like the idea.
>
> Please remember that some establishments have restrictions that mean
> that tools like easy_install or pip cannot be used. In locked-down
> corporate environments, python-full is potentially all that will be
> available (and maybe very specific "blessed" environment-specific 3rd
> party modules).

Yup. But 90% of the time, you're getting python-full from your OS
vendor, and if not, you just click on the full url :)

> But if the stdlib can be split out in such a way that it doesn't
> adversely impact those environments, then maybe the extra flexibility
> to evolve it would be helpful. (I'd like to know how that aligns with
> the stated goal of stdlib stability, though - seems to me like it's
> one or the other...)
>
> Paul.
>

Note; I too still need a python-full implementation. My argument is to
split them within source, and be able to use a single "source" for all
the implementations.

Let me give a concrete example. Let's say Bob and Alice are working on
a device which is severely space constrained. Bob and Alice love
Python but they only use a tiny subset of the stdlib. Right now
getting *just* the interpreter is painful at best.

Bob and alice have a second issue though - they still want good,
useful things like sockets, threading, and other low-level and
standard libraries, but simply don't need mimetools, an http server,
14 xml processing libraries and a package which goes "ping!". They'd
like to "roll their own" using some bits from python-stdlib, and maybe
some third party bits here and there.

jesse

From solipsis at pitrou.net  Mon Sep 14 17:51:17 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 14 Sep 2009 17:51:17 +0200
Subject: [stdlib-sig] Breaking out the stdlib
Message-ID: <1252943477.5954.42.camel@localhost>


(sorry for the double post, Jesse)

-------- Message transf?r? --------

Le lundi 14 septembre 2009 ? 11:42 -0400, Jesse Noller a ?crit :
> 
> Bob and alice have a second issue though - they still want good,
> useful things like sockets, threading, and other low-level and
> standard libraries, but simply don't need mimetools, an http server,
> 14 xml processing libraries and a package which goes "ping!". They'd
> like to "roll their own" using some bits from python-stdlib, and maybe
> some third party bits here and there.

How is this easier with a separate stdlib? You have to do exactly the
same job you have to do nowadays: prune useless modules, add the useful
stuff you need.




From solipsis at pitrou.net  Mon Sep 14 17:53:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 14 Sep 2009 17:53:36 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909140846h4ad93cc6ie026b5ac63c935d2@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<1252942913.5954.39.camel@localhost>
	<4222a8490909140846h4ad93cc6ie026b5ac63c935d2@mail.gmail.com>
Message-ID: <1252943616.5954.45.camel@localhost>

Le lundi 14 septembre 2009 ? 11:46 -0400, Jesse Noller a ?crit :
> Then Fredrik is no longer the maintainer - I'm looking at this through
> rather harsh eyes, true, but my goal is progress and quality - not
> being nice, so I'm sorry if I'm overly harsh.

I can agree with that, but it may imply some social / relational
problems if there isn't some formal (or widely agreed upon) process to
transfer ownership of a neglected module.

> And externally maintained modules are an oxymoron, no? :)

Perhaps, but they exist (e.g. simplejson).




From jnoller at gmail.com  Mon Sep 14 17:53:17 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:53:17 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1252943477.5954.42.camel@localhost>
References: <1252943477.5954.42.camel@localhost>
Message-ID: <4222a8490909140853y3cbbc16aq23f1d25e52ddf360@mail.gmail.com>

On Mon, Sep 14, 2009 at 11:51 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> (sorry for the double post, Jesse)
>
> -------- Message transf?r? --------
>
> Le lundi 14 septembre 2009 ? 11:42 -0400, Jesse Noller a ?crit :
>>
>> Bob and alice have a second issue though - they still want good,
>> useful things like sockets, threading, and other low-level and
>> standard libraries, but simply don't need mimetools, an http server,
>> 14 xml processing libraries and a package which goes "ping!". They'd
>> like to "roll their own" using some bits from python-stdlib, and maybe
>> some third party bits here and there.
>
> How is this easier with a separate stdlib? You have to do exactly the
> same job you have to do nowadays: prune useless modules, add the useful
> stuff you need.
>

Ah! You see, by it's very nature, breaking out the stdlib requires a
modification to the build tools which we lack now with the tight
coupling. We're going to need to build/evolve tools that would allow
multiple implementations to pull in modules, mark them as "jython
only" and so on.

Second, Bob and Alice would only need to hack the build process for
the stdlib *itself*. They can just download python-core, and *use*
that, their customizations are only to the python-stdlib.

Off the cuff (and no; I haven't thought about this terribly hard) you
could have a stdlib includes file ala Sphinx' index.rst file:

foo
bar
blatz

Which, when parsed, pulls in the stdlib modules to the build process.
If an item is omitted, it is included. Actually, a sphinx-*like*
structure makes a small amount of sense.

jesse

From jnoller at gmail.com  Mon Sep 14 17:54:18 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:54:18 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1252943616.5954.45.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<1252942913.5954.39.camel@localhost>
	<4222a8490909140846h4ad93cc6ie026b5ac63c935d2@mail.gmail.com>
	<1252943616.5954.45.camel@localhost>
Message-ID: <4222a8490909140854y1ea1a250hcad1d1a13bf4744@mail.gmail.com>

On Mon, Sep 14, 2009 at 11:53 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le lundi 14 septembre 2009 ? 11:46 -0400, Jesse Noller a ?crit :
>> Then Fredrik is no longer the maintainer - I'm looking at this through
>> rather harsh eyes, true, but my goal is progress and quality - not
>> being nice, so I'm sorry if I'm overly harsh.
>
> I can agree with that, but it may imply some social / relational
> problems if there isn't some formal (or widely agreed upon) process to
> transfer ownership of a neglected module.

Agreed.

>> And externally maintained modules are an oxymoron, no? :)
>
> Perhaps, but they exist (e.g. simplejson).
>

That one seems like an odd exception to the rule. I don't know how
that one evolved.

From jnoller at gmail.com  Mon Sep 14 17:46:40 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:46:40 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1252942913.5954.39.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<1252942913.5954.39.camel@localhost>
Message-ID: <4222a8490909140846h4ad93cc6ie026b5ac63c935d2@mail.gmail.com>

On Mon, Sep 14, 2009 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le lundi 14 septembre 2009 ? 11:13 -0400, Jesse Noller a ?crit :
>> Note, since I drafted this, brett's posted some thought on evolution
>> as well: http://sayspy.blogspot.com/2009/09/evolving-standard-library.html
>
> I tried to comment on it but, after a lot of pain editing my thoughts
> (the text editor on that blog is completely broken in my browser), it
> seems the comment got lost when I clicked "post"...
>
>> In thinking about this even more over the past year(ish) - I've
>> wondered if the stdlib, and python-core should actually be *really*
>> separated. I'm a huge fan of batteries included, but I can see the
>> draw to a breakdown like this:
>>
>> python-core (no stdlib, interpreter, core language)
>> python-stdlib (no core)
>
> Note, however, that an interpreter without stdlib is useless. Even basic
> I/O (on Python 3) may not function properly. Conversely, the stdlib will
> depend on certain interpreter features, or even implementation details.
> So, while we can put them in separate VCSes, the development of
> python-core and python-stdlib will be still be quite tied.

Understood; there is a simple "basic set of widgets" we do *need* -
but it's terribly smaller than what we have now.

>> python-full (the works)
>
> What is this? core + stdlib?

Yes.

>> From a packaging standpoint -
>> it's a lot easier to spin a new stdlib package and get it into an OS
>> upstream then the entire interpreter.
>
> I'm not convinced. A new stdlib can lead to as many compatibility
> problems as a new interpreter does. And it seems that compatibility /
> dependency management is the #1 problem in packaging.

I'm not trying to solve that, Tarek is ;)

>> I would personally like to see every single stdlib library have an
>> "owner" - I know, that's a long shot, but I really feel it's needed.
>> Otherwise you potentially have people reviewing patches for code they
>> may not fully understand, or not understand the ramifications of.
>
> On the other hand, having an owner can be detrimental to maintenance.
> For example, nobody wants to touch ElementTree except Fredrik, and
> Fredrik isn't very active these days.
> We should also say to no to externally-maintained modules, because it
> completely ruins maintenance for us core developers.

Then Fredrik is no longer the maintainer - I'm looking at this through
rather harsh eyes, true, but my goal is progress and quality - not
being nice, so I'm sorry if I'm overly harsh.

And externally maintained modules are an oxymoron, no? :)

jesse

From jnoller at gmail.com  Mon Sep 14 17:58:49 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 11:58:49 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
Message-ID: <4222a8490909140858g1eccacdydadc1d71107fc615@mail.gmail.com>

On Mon, Sep 14, 2009 at 11:13 AM, Jesse Noller <jnoller at gmail.com> wrote:
> Note, since I drafted this, brett's posted some thought on evolution
> as well: http://sayspy.blogspot.com/2009/09/evolving-standard-library.html
>
>
> So, here's a small pile of thoughts - the main driving force of which
> was the common sentiment that was shown in the language summit at last
> pycon. I'm mainly sending this out to evoke some discussion.
>
> Last pycon, many of us agreed that the stdlib needed to be "broken"
> out of the core within source control. AFAIK, this is still pending
> the mercurial migration. The goal of this would be to have one common
> stdlib shared amongst the implementations (Jython, Ironpython, etc).
> Modules which were cpython-only (such as multiprocessing) would be
> kept near-core, or marked as Cpython only.
>
> This means we would have an interpreter/builtins section for cpython,
> one for Jython, etc while they could all consume the central stdlib
> blob.
>
> In thinking about this even more over the past year(ish) - I've
> wondered if the stdlib, and python-core should actually be *really*
> separated. I'm a huge fan of batteries included, but I can see the
> draw to a breakdown like this:
>
> python-core (no stdlib, interpreter, core language)
> python-stdlib (no core)
> python-full (the works)
>
> (Note this may actually *help* OS package maintainers.)
>
> Doing this - in my mind - lends itself to the stdlib evolving faster.
> Sure, there's a lot of good things in the stdlib, but frankly, we have
> over 216 packages in it, and only 113 developers
> (http://www.python.org/dev/committers). Of those 113 developers, how
> many are actually active? How many of the modules in the stdlib
> actually have owners with vested interest in maintaining them? How
> many of them are evolutionary dead ends? From a packaging standpoint -
> it's a lot easier to spin a new stdlib package and get it into an OS
> upstream then the entire interpreter.
>
> The running joke is that the stdlib is where modules go to die -
> personally, I don't think this should be true - although it is true to
> a certain extent. It's also true that some of the modules within the
> stdlib are not best-of-breed - httplib2 vs. urllib/httplib comes to
> mind (mainly because I'm dealing with that right now).
>
> We all know that the stdlib has evolved over a great deal of time -
> and over time the quality bar has changed (for the better) - but I'd
> ask how to we take that quality bar and beat some of the
> packages/modules we have in there with it? How do we make sure that
> the stdlib is stable, best of breed and high quality?
>
> I would say that it's entirely possible that some things simply need
> to be removed; not just platform specific things, but things which
> don't have maintainers on the hook to review their patches, things
> which have low-to-no test coverage/docs.
>
> I would personally like to see every single stdlib library have an
> "owner" - I know, that's a long shot, but I really feel it's needed.
> Otherwise you potentially have people reviewing patches for code they
> may not fully understand, or not understand the ramifications of.
>
> Breaking out is 1/2 the fight - the other half is cleaning it up/out -
> removing things with low test coverage, poor docs or things which
> simply are *not* the best option. We need to take a really hard look
> at things in there and ask ourselves if the things in there meet our
> collective quality bar, and if they don't: remove it. We want a good,
> solid and shared-amongst implementations stdlib.
>
> jesse
>

Oh, and when I talk about pulling it out - I mean tests, docs, etc.
This way all other implementations could use the same tests, docs,
etc.

From ctb at msu.edu  Mon Sep 14 18:07:14 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Mon, 14 Sep 2009 09:07:14 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
Message-ID: <20090914160714.GE15328@idyll.org>

On Mon, Sep 14, 2009 at 04:35:54PM +0100, Paul Moore wrote:
-> 2009/9/14 Doug Hellmann <doug.hellmann at gmail.com>:
-> >> In thinking about this even more over the past year(ish) - I've
-> >> wondered if the stdlib, and python-core should actually be *really*
-> >> separated. I'm a huge fan of batteries included, but I can see the
-> >> draw to a breakdown like this:
-> >>
-> >> python-core (no stdlib, interpreter, core language)
-> >> python-stdlib (no core)
-> >> python-full (the works)
-> >
-> > It would be interesting to know what stdlib modules are a minimum
-> > requirement to install other packages with a tool like easy_install or pip.
-> > ?Those might need to stay in "core" so that installing core gives a
-> > minimally functional system.
-> >
-> > Otherwise, I like the idea.
-> 
-> Please remember that some establishments have restrictions that mean
-> that tools like easy_install or pip cannot be used. In locked-down
-> corporate environments, python-full is potentially all that will be
-> available (and maybe very specific "blessed" environment-specific 3rd
-> party modules).
-> 
-> But if the stdlib can be split out in such a way that it doesn't
-> adversely impact those environments, then maybe the extra flexibility
-> to evolve it would be helpful. (I'd like to know how that aligns with
-> the stated goal of stdlib stability, though - seems to me like it's
-> one or the other...)

I'd like to -1 this whole discussion by saying that you guys are basing
this whole thing on your mature, competent skills of Python programming
and computer use.  Corporations and beginning programmers need something
straightforward, simple, with "batteries included" in order to do
something sane with Python in large multi-user environments.

We can discuss *which* batteries should be included -- bsddb was taken
out, sqlite3 looks like a long-term winner -- but IMVSO pruning the
stdlib should not be seriously consider until we have an excellent,
time-tested, battle-proven completely automatic package installation
system.

To me, this could be like the decision to simultaneously release python
3.x and python 2.x distributions, but much worse: even more confusion-
engendering and detrimental to short- and medium-term adoption of
Python.

<relurk>

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From michael at voidspace.org.uk  Mon Sep 14 18:09:16 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 17:09:16 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <20090914160714.GE15328@idyll.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
Message-ID: <4AAE6AAC.6090109@voidspace.org.uk>

C. Titus Brown wrote:
> On Mon, Sep 14, 2009 at 04:35:54PM +0100, Paul Moore wrote:
> -> 2009/9/14 Doug Hellmann <doug.hellmann at gmail.com>:
> -> >> In thinking about this even more over the past year(ish) - I've
> -> >> wondered if the stdlib, and python-core should actually be *really*
> -> >> separated. I'm a huge fan of batteries included, but I can see the
> -> >> draw to a breakdown like this:
> -> >>
> -> >> python-core (no stdlib, interpreter, core language)
> -> >> python-stdlib (no core)
> -> >> python-full (the works)
> -> >
> -> > It would be interesting to know what stdlib modules are a minimum
> -> > requirement to install other packages with a tool like easy_install or pip.
> -> > ?Those might need to stay in "core" so that installing core gives a
> -> > minimally functional system.
> -> >
> -> > Otherwise, I like the idea.
> -> 
> -> Please remember that some establishments have restrictions that mean
> -> that tools like easy_install or pip cannot be used. In locked-down
> -> corporate environments, python-full is potentially all that will be
> -> available (and maybe very specific "blessed" environment-specific 3rd
> -> party modules).
> -> 
> -> But if the stdlib can be split out in such a way that it doesn't
> -> adversely impact those environments, then maybe the extra flexibility
> -> to evolve it would be helpful. (I'd like to know how that aligns with
> -> the stated goal of stdlib stability, though - seems to me like it's
> -> one or the other...)
>
> I'd like to -1 this whole discussion by saying that you guys are basing
> this whole thing on your mature, competent skills of Python programming
> and computer use.  Corporations and beginning programmers need something
> straightforward, simple, with "batteries included" in order to do
> something sane with Python in large multi-user environments.
>
> We can discuss *which* batteries should be included -- bsddb was taken
> out, sqlite3 looks like a long-term winner -- but IMVSO pruning the
> stdlib should not be seriously consider until we have an excellent,
> time-tested, battle-proven completely automatic package installation
> system.
>
> To me, this could be like the decision to simultaneously release python
> 3.x and python 2.x distributions, but much worse: even more confusion-
> engendering and detrimental to short- and medium-term adoption of
> Python.
>
>   
I think we're discussing the organisation of development repositories 
and *not* changing the default distributions - just making alternative 
distributions easier. (at least -1 if that's not the context in which 
we're discussing this and +1 if it is.)

Michael



> <relurk>
>
> cheers,
> --titus
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Mon Sep 14 18:16:59 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 09:16:59 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4222a8490909140828q4a564520n973f87e0050f386f@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<4222a8490909140828q4a564520n973f87e0050f386f@mail.gmail.com>
Message-ID: <bbaeab100909140916t20079dy7c9bf54cd00ebe11@mail.gmail.com>

On Mon, Sep 14, 2009 at 08:28, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Sep 10, 2009 at 2:56 PM, Brett Cannon <brett at python.org> wrote:
>> Upfront people need to realize that we might have three argument
>> parsing libraries for a while, but it won't be forever. If we get
>> argparse accepted we would slowly deprecate at least optparse, if not
>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>> getopt supported stuff optparse didn't), out of the standard library
>> and toss them into PyPI for those who refuse to switch. The standard
>> library might not evolve a lot, but it isn't dead or in stone.
>>
>> But before this can happen, people need to have a general consensus
>> that I should bug Steven about contributing as it will require a PEP
>> from him. Steven already has commit privileges to maintenance from him
>> will not be a problem.
>>
>> So if you want this to actually happen and for me to start talking to
>> Steven just reply to this email w/ a vote.
>>
>> I am +0
>
> More fuel for the pep(fire):
>
> http://blogg.ingspree.net/blog/2009/09/14/opster/

It's only more fuel in terms of acknowledging there is another
approach using decorators. But since no library that takes that
approach is near to being considered best-of-breed by the community or
is as stable as argparse I don't consider it that big of a deal.

-Brett

From jnoller at gmail.com  Mon Sep 14 18:19:57 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 12:19:57 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE6AAC.6090109@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
Message-ID: <4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>

On Mon, Sep 14, 2009 at 12:09 PM, Michael Foord
<michael at voidspace.org.uk> wrote:
> C. Titus Brown wrote:
>>
>> On Mon, Sep 14, 2009 at 04:35:54PM +0100, Paul Moore wrote:
>> -> 2009/9/14 Doug Hellmann <doug.hellmann at gmail.com>:
>> -> >> In thinking about this even more over the past year(ish) - I've
>> -> >> wondered if the stdlib, and python-core should actually be *really*
>> -> >> separated. I'm a huge fan of batteries included, but I can see the
>> -> >> draw to a breakdown like this:
>> -> >>
>> -> >> python-core (no stdlib, interpreter, core language)
>> -> >> python-stdlib (no core)
>> -> >> python-full (the works)
>> -> >
>> -> > It would be interesting to know what stdlib modules are a minimum
>> -> > requirement to install other packages with a tool like easy_install
>> or pip.
>> -> > ?Those might need to stay in "core" so that installing core gives a
>> -> > minimally functional system.
>> -> >
>> -> > Otherwise, I like the idea.
>> -> -> Please remember that some establishments have restrictions that mean
>> -> that tools like easy_install or pip cannot be used. In locked-down
>> -> corporate environments, python-full is potentially all that will be
>> -> available (and maybe very specific "blessed" environment-specific 3rd
>> -> party modules).
>> -> -> But if the stdlib can be split out in such a way that it doesn't
>> -> adversely impact those environments, then maybe the extra flexibility
>> -> to evolve it would be helpful. (I'd like to know how that aligns with
>> -> the stated goal of stdlib stability, though - seems to me like it's
>> -> one or the other...)
>>
>> I'd like to -1 this whole discussion by saying that you guys are basing
>> this whole thing on your mature, competent skills of Python programming
>> and computer use. ?Corporations and beginning programmers need something
>> straightforward, simple, with "batteries included" in order to do
>> something sane with Python in large multi-user environments.
>>
>> We can discuss *which* batteries should be included -- bsddb was taken
>> out, sqlite3 looks like a long-term winner -- but IMVSO pruning the
>> stdlib should not be seriously consider until we have an excellent,
>> time-tested, battle-proven completely automatic package installation
>> system.
>>
>> To me, this could be like the decision to simultaneously release python
>> 3.x and python 2.x distributions, but much worse: even more confusion-
>> engendering and detrimental to short- and medium-term adoption of
>> Python.
>>
>>
>
> I think we're discussing the organisation of development repositories and
> *not* changing the default distributions - just making alternative
> distributions easier. (at least -1 if that's not the context in which we're
> discussing this and +1 if it is.)
>
> Michael
>

Michael is correct; Titus is incorrect.

Titus; I'm taking in the needs of those just starting with python - I
would not, WOULD NOT have started using python if it had not come with
batteries. That's why I left python-full in the email.

What I am proposing, however, is we:

1> Separate them in source
2> Offer "core, stdlib, full" downloads - so we keep the simple,
simple - and allow for the more complex/nuanced uses.
3> Be aggressive with the *quality* of the modules

Sure, having batteries is fantastic. Until you being to realize that
some modules are broken, not up to date with current technology, etc,
etc. We have to be aggressive with cleaning it up.

In short; leaving crap in there "just because" doesn't make it better
- it just makes it bigger.

I can't even begin to describe the pain I went through when - new to
python, I tried to deal with XML parsing. Or the joy I felt when I
sent email for the first time using smtplib.

Things within the stdlib must have a high quality bar, and should
really represent the "one way of doing something" - meaning best of
breed, high quality, idiomatic, etc.

jesse

From michael at voidspace.org.uk  Mon Sep 14 18:25:16 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 17:25:16 +0100
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909140916t20079dy7c9bf54cd00ebe11@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<4222a8490909140828q4a564520n973f87e0050f386f@mail.gmail.com>
	<bbaeab100909140916t20079dy7c9bf54cd00ebe11@mail.gmail.com>
Message-ID: <4AAE6E6C.7050601@voidspace.org.uk>

Brett Cannon wrote:
> On Mon, Sep 14, 2009 at 08:28, Jesse Noller <jnoller at gmail.com> wrote:
>   
>> On Thu, Sep 10, 2009 at 2:56 PM, Brett Cannon <brett at python.org> wrote:
>>     
>>> Upfront people need to realize that we might have three argument
>>> parsing libraries for a while, but it won't be forever. If we get
>>> argparse accepted we would slowly deprecate at least optparse, if not
>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>> getopt supported stuff optparse didn't), out of the standard library
>>> and toss them into PyPI for those who refuse to switch. The standard
>>> library might not evolve a lot, but it isn't dead or in stone.
>>>
>>> But before this can happen, people need to have a general consensus
>>> that I should bug Steven about contributing as it will require a PEP
>>> from him. Steven already has commit privileges to maintenance from him
>>> will not be a problem.
>>>
>>> So if you want this to actually happen and for me to start talking to
>>> Steven just reply to this email w/ a vote.
>>>
>>> I am +0
>>>       
>> More fuel for the pep(fire):
>>
>> http://blogg.ingspree.net/blog/2009/09/14/opster/
>>     
>
> It's only more fuel in terms of acknowledging there is another
> approach using decorators. But since no library that takes that
> approach is near to being considered best-of-breed by the community or
> is as stable as argparse I don't consider it that big of a deal.
>   

Although adding a decorator based approach to argparse shouldn't be out 
of the question. Then there really would be MTOWTDI...

Michael

> -Brett
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From jnoller at gmail.com  Mon Sep 14 18:30:11 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 12:30:11 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <20090914160714.GE15328@idyll.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
Message-ID: <4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>

On Mon, Sep 14, 2009 at 12:07 PM, C. Titus Brown <ctb at msu.edu> wrote:

> I'd like to -1 this whole discussion by saying that you guys are basing
> this whole thing on your mature, competent skills of Python programming
> and computer use. ?Corporations and beginning programmers need something
> straightforward, simple, with "batteries included" in order to do
> something sane with Python in large multi-user environments.

And just to hit on this specifically: Actually, I'm not. Try
explaining what modules in the stdlib are of questionable use, and why
not to use them to your boss, who is learning Python.

Them: "Why shouldn't I used httplib?"
Me: Reasons A,B,C - just use httplib2
Them: "Why don't they just fix that?"

That's for starters. If you want batteries included to mean something
more than "we got junk in our trunk" then we should raise the bar. Not
to mention over 216 modules? Less than a handful of active committers?
It's not sustainable.

> We can discuss *which* batteries should be included -- bsddb was taken
> out, sqlite3 looks like a long-term winner -- but IMVSO pruning the
> stdlib should not be seriously consider until we have an excellent,
> time-tested, battle-proven completely automatic package installation
> system.

Pruning must occur regardless of how easy it is to install something
"voted off the island".

> To me, this could be like the decision to simultaneously release python
> 3.x and python 2.x distributions, but much worse: even more confusion-
> engendering and detrimental to short- and medium-term adoption of
> Python.

All an end user sees:

click here to download Python 3.0-Full
*smaller print* click here for the interpreter-core
*smaller print* click here for the stdlib

Otherwise, they will see the eventual deprecation of things which are
*broken* or have *no tests* to even prove if they aren't between
releases.

jesse

From brett at python.org  Mon Sep 14 18:30:14 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 09:30:14 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <4AAE6E6C.7050601@voidspace.org.uk>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com> 
	<4222a8490909140828q4a564520n973f87e0050f386f@mail.gmail.com> 
	<bbaeab100909140916t20079dy7c9bf54cd00ebe11@mail.gmail.com> 
	<4AAE6E6C.7050601@voidspace.org.uk>
Message-ID: <bbaeab100909140930jda5b3bax2c44a0f8bdb58cf1@mail.gmail.com>

On Mon, Sep 14, 2009 at 09:25, Michael Foord <michael at voidspace.org.uk> wrote:
> Brett Cannon wrote:
>>
>> On Mon, Sep 14, 2009 at 08:28, Jesse Noller <jnoller at gmail.com> wrote:
>>
>>>
>>> On Thu, Sep 10, 2009 at 2:56 PM, Brett Cannon <brett at python.org> wrote:
>>>
>>>>
>>>> Upfront people need to realize that we might have three argument
>>>> parsing libraries for a while, but it won't be forever. If we get
>>>> argparse accepted we would slowly deprecate at least optparse, if not
>>>> getopt (lat time I tried to ditch getopt for Python 3 some argued that
>>>> getopt supported stuff optparse didn't), out of the standard library
>>>> and toss them into PyPI for those who refuse to switch. The standard
>>>> library might not evolve a lot, but it isn't dead or in stone.
>>>>
>>>> But before this can happen, people need to have a general consensus
>>>> that I should bug Steven about contributing as it will require a PEP
>>>> from him. Steven already has commit privileges to maintenance from him
>>>> will not be a problem.
>>>>
>>>> So if you want this to actually happen and for me to start talking to
>>>> Steven just reply to this email w/ a vote.
>>>>
>>>> I am +0
>>>>
>>>
>>> More fuel for the pep(fire):
>>>
>>> http://blogg.ingspree.net/blog/2009/09/14/opster/
>>>
>>
>> It's only more fuel in terms of acknowledging there is another
>> approach using decorators. But since no library that takes that
>> approach is near to being considered best-of-breed by the community or
>> is as stable as argparse I don't consider it that big of a deal.
>>
>
> Although adding a decorator based approach to argparse shouldn't be out of
> the question. Then there really would be MTOWTDI...

It's not, but then I would want it out in the wild first to work out
the API, so still don't find it reasonable for this PEP.

-Brett

From solipsis at pitrou.net  Mon Sep 14 18:53:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 14 Sep 2009 18:53:27 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
Message-ID: <1252947207.7398.8.camel@localhost>

Le lundi 14 septembre 2009 ? 12:19 -0400, Jesse Noller a ?crit :
> Sure, having batteries is fantastic. Until you being to realize that
> some modules are broken, not up to date with current technology, etc,
> etc. We have to be aggressive with cleaning it up.
[...]
> 
> Things within the stdlib must have a high quality bar, and should
> really represent the "one way of doing something" - meaning best of
> breed, high quality, idiomatic, etc.

Have you read the thread about argparse and the deprecation of other
modules?

You are conflating two things:
1. the presence of somewhat outdated or too low-level modules
2. the desire to have best-of-breed options available in the stdlib

We can do 2. without undoing 1. As Marc-Andr? pointed out, even in py3k
we only deprecated a handful of very marginal modules. Emphasizing the
right modules in the documentation is probably a reasonably good answer
to the problem of having several modules fulfilling the same needs. No
need to go on a deprecation frenzy and start annoying the two thirds of
our user base just because we decided to be pure rather than practical.




From jnoller at gmail.com  Mon Sep 14 19:00:15 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 13:00:15 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1252947207.7398.8.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
Message-ID: <4222a8490909141000i686bdeeaq9308d546afd7824b@mail.gmail.com>

On Mon, Sep 14, 2009 at 12:53 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le lundi 14 septembre 2009 ? 12:19 -0400, Jesse Noller a ?crit :
>> Sure, having batteries is fantastic. Until you being to realize that
>> some modules are broken, not up to date with current technology, etc,
>> etc. We have to be aggressive with cleaning it up.
> [...]
>>
>> Things within the stdlib must have a high quality bar, and should
>> really represent the "one way of doing something" - meaning best of
>> breed, high quality, idiomatic, etc.
>
> Have you read the thread about argparse and the deprecation of other
> modules?
>
> You are conflating two things:
> 1. the presence of somewhat outdated or too low-level modules
> 2. the desire to have best-of-breed options available in the stdlib
>
> We can do 2. without undoing 1. As Marc-Andr? pointed out, even in py3k
> we only deprecated a handful of very marginal modules. Emphasizing the
> right modules in the documentation is probably a reasonably good answer
> to the problem of having several modules fulfilling the same needs. No
> need to go on a deprecation frenzy and start annoying the two thirds of
> our user base just because we decided to be pure rather than practical.
>

IMHO, and in the opinions of others, the deprecation within py3k did
not go far enough.

This is not purity vs. practicality. In fact I'd argue (and will! I'm
warnin ya!) that leaving stuff in there because we don't want to
remove it for fear some other piece of code in the wild may depend on
it is completely impractical.

We do not have the people, the time, the funding, etc to handle a
great big Java-esque pile of "permanently deprecated, slightly
discouraged" code.

If we do not want it in the stdlib, it should be deprecated; and then
removed. Leaving it there in perpetuity means some person, 5 years
after we deprecate it, will write code against it - which, by this
reasoning we will have to support for another 10 years, even *after*
it's no longer maintained.

jesse

From steven.bethard at gmail.com  Mon Sep 14 19:04:19 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 14 Sep 2009 10:04:19 -0700
Subject: [stdlib-sig] should we try to add argparse?
In-Reply-To: <bbaeab100909140930jda5b3bax2c44a0f8bdb58cf1@mail.gmail.com>
References: <bbaeab100909101156m22f7ed1br93b3e940a74d8bd5@mail.gmail.com>
	<4222a8490909140828q4a564520n973f87e0050f386f@mail.gmail.com>
	<bbaeab100909140916t20079dy7c9bf54cd00ebe11@mail.gmail.com>
	<4AAE6E6C.7050601@voidspace.org.uk>
	<bbaeab100909140930jda5b3bax2c44a0f8bdb58cf1@mail.gmail.com>
Message-ID: <d11dcfba0909141004y27185874ld4e56eda55480e7d@mail.gmail.com>

On Mon, Sep 14, 2009 at 9:30 AM, Brett Cannon <brett at python.org> wrote:
> On Mon, Sep 14, 2009 at 09:25, Michael Foord <michael at voidspace.org.uk> wrote:
>> Although adding a decorator based approach to argparse shouldn't be out of
>> the question. Then there really would be MTOWTDI...
>
> It's not, but then I would want it out in the wild first to work out
> the API, so still don't find it reasonable for this PEP.

It's out of scope for the PEP, but it's definitely under consideration
as a feature enhancement request for the next major version of
argparse. But that's really just like a feature request for any
library.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From mal at egenix.com  Mon Sep 14 19:06:29 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 14 Sep 2009 19:06:29 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
Message-ID: <4AAE7815.9020101@egenix.com>

I don't really understand how breaking something as useful as
the stdlib into smaller pieces would help with anything.

The main purpose of a library is that you have an *integrated* set
of modules that are known and tested to work together.

The stdlib has gone a long way in trying to achieve that and it's
getting better at it with every release.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 14 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From michael at voidspace.org.uk  Mon Sep 14 19:06:53 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 18:06:53 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1252947207.7398.8.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
Message-ID: <4AAE782D.8020800@voidspace.org.uk>

Antoine Pitrou wrote:
> Le lundi 14 septembre 2009 ? 12:19 -0400, Jesse Noller a ?crit :
>   
>> Sure, having batteries is fantastic. Until you being to realize that
>> some modules are broken, not up to date with current technology, etc,
>> etc. We have to be aggressive with cleaning it up.
>>     
> [...]
>   
>> Things within the stdlib must have a high quality bar, and should
>> really represent the "one way of doing something" - meaning best of
>> breed, high quality, idiomatic, etc.
>>     
>
> Have you read the thread about argparse and the deprecation of other
> modules?
>
> You are conflating two things:
> 1. the presence of somewhat outdated or too low-level modules
> 2. the desire to have best-of-breed options available in the stdlib
>
> We can do 2. without undoing 1. As Marc-Andr? pointed out, even in py3k
> we only deprecated a handful of very marginal modules. Emphasizing the
> right modules in the documentation is probably a reasonably good answer
> to the problem of having several modules fulfilling the same needs. No
> need to go on a deprecation frenzy and start annoying the two thirds of
> our user base just because we decided to be pure rather than practical.
>
>   

I think both 1) and 2) are on topic for a discussion of how we deal with 
standard library quality.

No-one argues that the standard library should evolve quickly but there 
do seem to be those arguing that it should *never* evolve.

I agree with Jesse (FWIW) that the presence of obsolete modules is 
actually damaging to Python. Deprecating and removing modules over a 
four or five years (PendingDeprecation -> Deprecation -> removal) is 
more than slow enough for a stable system that is the Python standard 
library.

I would love to see a PEP about further standard library clean-ups, 
perhaps slating modules for *eventual* removal in Python 3.4 / 3.5 and 
only when they are clearly not useful, replaced or broken and not 
maintained - and preferably where there is a clear alternative.

Michael

>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From michael at voidspace.org.uk  Mon Sep 14 19:07:58 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 18:07:58 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE7815.9020101@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
	<4AAE7815.9020101@egenix.com>
Message-ID: <4AAE786E.6020207@voidspace.org.uk>

M.-A. Lemburg wrote:
> I don't really understand how breaking something as useful as
> the stdlib into smaller pieces would help with anything.
>
> The main purpose of a library is that you have an *integrated* set
> of modules that are known and tested to work together.
>
> The stdlib has gone a long way in trying to achieve that and it's
> getting better at it with every release.
>
>   
The particular motivation is to make it easier for other implementations 
to reuse the standard library. It was something discussed (with the 
maintainers of some of these implementations) at the Python Language Summit.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From orestis at orestis.gr  Mon Sep 14 19:13:30 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Mon, 14 Sep 2009 20:13:30 +0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE7815.9020101@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
	<4AAE7815.9020101@egenix.com>
Message-ID: <4019B3EC-7065-40F8-BD3B-7CD4000092B2@orestis.gr>

On Sep 14, 2009, at 8:06 PM, M.-A. Lemburg wrote:

> I don't really understand how breaking something as useful as
> the stdlib into smaller pieces would help with anything.
>
> The main purpose of a library is that you have an *integrated* set
> of modules that are known and tested to work together.

No one has suggested that - just that the core can be downloaded  
without the stdlib, for the people that might need it.

>
> The stdlib has gone a long way in trying to achieve that and it's
> getting better at it with every release.
>
> -- 
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Sep 14  
> 2009)
>>>> Python/Zope Consulting and Support ...        http:// 
>>>> www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http:// 
>>>> zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http:// 
>>>> python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for  
> free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig


From orestis at orestis.gr  Mon Sep 14 19:22:27 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Mon, 14 Sep 2009 20:22:27 +0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE786E.6020207@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
	<4AAE7815.9020101@egenix.com> <4AAE786E.6020207@voidspace.org.uk>
Message-ID: <E3C487FC-4D00-4B4F-884C-C2A24C8BA4EB@orestis.gr>


On Sep 14, 2009, at 8:07 PM, Michael Foord wrote:

> M.-A. Lemburg wrote:
>> I don't really understand how breaking something as useful as
>> the stdlib into smaller pieces would help with anything.
>>
>> The main purpose of a library is that you have an *integrated* set
>> of modules that are known and tested to work together.
>>
>> The stdlib has gone a long way in trying to achieve that and it's
>> getting better at it with every release.
>>
>>
> The particular motivation is to make it easier for other  
> implementations to reuse the standard library. It was something  
> discussed (with the maintainers of some of these implementations) at  
> the Python Language Summit.

I'd also like to point out the psychological ramifications of making  
the stdlib and optional component (even if most users will actually  
download the python-full bundle).

Right now when looking for a library to perform some task, the first  
place anyone (at least, newcomers) looks is the stdlib. If there's  
something there, then most people will stop looking. This is good  
because python should come with batteries included. However, a lot of  
people will then tend to think "stdlib is sacred" and will never  
evaluate the better solutions that are out there.

If the python core devs split out the stdlib into an optional  
download, they are making a statement - that is, the stdlib is just a  
set of components that mostly work together, nothing more, nothing  
less. No one should be compelled to use them just because they  
originate from python-core. Hopefully both outside and inside packages  
will be judged on merit, and a set of de facto standard packages will  
be crowdsourced over time.

Of course, making that statement without a simple way of finding,  
installing and uninstalling packages is dangerous, but as said before,  
other people are trying to solve that.

Orestis

>
> Michael
>
> -- 
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig


From mal at egenix.com  Mon Sep 14 19:28:56 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 14 Sep 2009 19:28:56 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE786E.6020207@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>	<4AAE7815.9020101@egenix.com>
	<4AAE786E.6020207@voidspace.org.uk>
Message-ID: <4AAE7D58.1040105@egenix.com>

Michael Foord wrote:
> M.-A. Lemburg wrote:
>> I don't really understand how breaking something as useful as
>> the stdlib into smaller pieces would help with anything.
>>
>> The main purpose of a library is that you have an *integrated* set
>> of modules that are known and tested to work together.
>>
>> The stdlib has gone a long way in trying to achieve that and it's
>> getting better at it with every release.
>>
>>   
> The particular motivation is to make it easier for other implementations
> to reuse the standard library. It was something discussed (with the
> maintainers of some of these implementations) at the Python Language
> Summit.

The stdlib already has a lot of support for different Python
implementation built right into the code.

There will always be some things that don't work in certain
implementations, but if that's the problem, we could just
add an assert at the top of the module or raise an ImportError
to warn the user.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 14 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fwierzbicki at gmail.com  Mon Sep 14 19:28:47 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 14 Sep 2009 13:28:47 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE786E.6020207@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
	<4AAE7815.9020101@egenix.com> <4AAE786E.6020207@voidspace.org.uk>
Message-ID: <4dab5f760909141028v62527232v8e549cbecafcc8fe@mail.gmail.com>

On Mon, Sep 14, 2009 at 1:07 PM, Michael Foord <michael at voidspace.org.uk> wrote:
> The particular motivation is to make it easier for other implementations to
> reuse the standard library. It was something discussed (with the maintainers
> of some of these implementations) at the Python Language Summit.
Speaking for Jython, it would be very helpful if the stdlib was
considered a separate entity in a more official way than it is now.
Jython already uses the stdlib this way (sort of), we have an
svn:externals entry that just pulls the stdlib down.  Today, we
maintain a largish number of the modules with local patches that
replace some modules as part of our build process.  It would be nicer
if we could do less patching at build time.  BTW - I don't mean that I
expect others to do this work for us, a couple of us have commit
rights to help this along.

-Frank

From mal at egenix.com  Mon Sep 14 19:34:08 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 14 Sep 2009 19:34:08 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4dab5f760909141028v62527232v8e549cbecafcc8fe@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>	<4AAE7815.9020101@egenix.com>
	<4AAE786E.6020207@voidspace.org.uk>
	<4dab5f760909141028v62527232v8e549cbecafcc8fe@mail.gmail.com>
Message-ID: <4AAE7E90.8030509@egenix.com>

Frank Wierzbicki wrote:
> On Mon, Sep 14, 2009 at 1:07 PM, Michael Foord <michael at voidspace.org.uk> wrote:
>> The particular motivation is to make it easier for other implementations to
>> reuse the standard library. It was something discussed (with the maintainers
>> of some of these implementations) at the Python Language Summit.
> Speaking for Jython, it would be very helpful if the stdlib was
> considered a separate entity in a more official way than it is now.
> Jython already uses the stdlib this way (sort of), we have an
> svn:externals entry that just pulls the stdlib down.  Today, we
> maintain a largish number of the modules with local patches that
> replace some modules as part of our build process.  It would be nicer
> if we could do less patching at build time.  BTW - I don't mean that I
> expect others to do this work for us, a couple of us have commit
> rights to help this along.

Why don't you try to get those changes into the stdlib ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 14 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From michael at voidspace.org.uk  Mon Sep 14 19:34:00 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 18:34:00 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE7D58.1040105@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>	<4AAE7815.9020101@egenix.com>
	<4AAE786E.6020207@voidspace.org.uk> <4AAE7D58.1040105@egenix.com>
Message-ID: <4AAE7E88.6040007@voidspace.org.uk>

M.-A. Lemburg wrote:
> Michael Foord wrote:
>   
>> M.-A. Lemburg wrote:
>>     
>>> I don't really understand how breaking something as useful as
>>> the stdlib into smaller pieces would help with anything.
>>>
>>> The main purpose of a library is that you have an *integrated* set
>>> of modules that are known and tested to work together.
>>>
>>> The stdlib has gone a long way in trying to achieve that and it's
>>> getting better at it with every release.
>>>
>>>   
>>>       
>> The particular motivation is to make it easier for other implementations
>> to reuse the standard library. It was something discussed (with the
>> maintainers of some of these implementations) at the Python Language
>> Summit.
>>     
>
> The stdlib already has a lot of support for different Python
> implementation built right into the code.
>
> There will always be some things that don't work in certain
> implementations, but if that's the problem, we could just
> add an assert at the top of the module or raise an ImportError
> to warn the user.
>
>   
The maintainers of the other implementations felt it would be simpler to 
track the standard library if it was maintained in a separate 
repository, which wasn't felt to be a problem by the Python 
core-developers. The standard library would continue to include code to 
maintain compatibility (where possible) with alternative implementations.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From collinw at gmail.com  Mon Sep 14 19:34:36 2009
From: collinw at gmail.com (Collin Winter)
Date: Mon, 14 Sep 2009 13:34:36 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE7D58.1040105@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
	<4AAE7815.9020101@egenix.com> <4AAE786E.6020207@voidspace.org.uk>
	<4AAE7D58.1040105@egenix.com>
Message-ID: <43aa6ff70909141034k19ad3db3o2f166516677e8c06@mail.gmail.com>

On Mon, Sep 14, 2009 at 1:28 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Michael Foord wrote:
>> M.-A. Lemburg wrote:
>>> I don't really understand how breaking something as useful as
>>> the stdlib into smaller pieces would help with anything.
>>>
>>> The main purpose of a library is that you have an *integrated* set
>>> of modules that are known and tested to work together.
>>>
>>> The stdlib has gone a long way in trying to achieve that and it's
>>> getting better at it with every release.
>>>
>>>
>> The particular motivation is to make it easier for other implementations
>> to reuse the standard library. It was something discussed (with the
>> maintainers of some of these implementations) at the Python Language
>> Summit.
>
> The stdlib already has a lot of support for different Python
> implementation built right into the code.
>
> There will always be some things that don't work in certain
> implementations, but if that's the problem, we could just
> add an assert at the top of the module or raise an ImportError
> to warn the user.

The proposal was to separate the stdlib into two logical components:
1) a shared, this-has-to-work-everywhere standard library that all
Python implementations share on equal terms;
2) an implementation-specific standard library for things that are
either implementation details or that require
Jython/IronPython/PyPy-specific implementations.

The test suite would be similarly exposed and shared between all
implementations on equal terms: one set of this-has-to-work-everywhere
tests, one set of implementation-specific tests layered on top of the
shared test suite (think garbage collection vs refcounting, etc). Same
idea for documentation.

The idea is to a) put CPython on a more equal footing with the other
implementations, and b) to remove the need to have
Jython/IronPython/PyPy-specific cases in the CPython standard library.

I believe there was a more formal explanation of the plan sent to this
list and/or python-dev shortly after PyCon US 2009; I'll see if I can
dig it up.

Collin Winter

From michael at voidspace.org.uk  Mon Sep 14 19:36:44 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 18:36:44 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4dab5f760909141028v62527232v8e549cbecafcc8fe@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	
	<20090914160714.GE15328@idyll.org>	
	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>	
	<4AAE7815.9020101@egenix.com> <4AAE786E.6020207@voidspace.org.uk>
	<4dab5f760909141028v62527232v8e549cbecafcc8fe@mail.gmail.com>
Message-ID: <4AAE7F2C.2010306@voidspace.org.uk>

Frank Wierzbicki wrote:
> On Mon, Sep 14, 2009 at 1:07 PM, Michael Foord <michael at voidspace.org.uk> wrote:
>   
>> The particular motivation is to make it easier for other implementations to
>> reuse the standard library. It was something discussed (with the maintainers
>> of some of these implementations) at the Python Language Summit.
>>     
> Speaking for Jython, it would be very helpful if the stdlib was
> considered a separate entity in a more official way than it is now.
> Jython already uses the stdlib this way (sort of), we have an
> svn:externals entry that just pulls the stdlib down.  Today, we
> maintain a largish number of the modules with local patches that
> replace some modules as part of our build process.  It would be nicer
> if we could do less patching at build time.  BTW - I don't mean that I
> expect others to do this work for us, a couple of us have commit
> rights to help this along.
>   

And indeed it was to help move this process along, alongside improving 
the Python test suite for alternative implementations, that many of us 
joined this otherwise-mostly-defunct mailing list.

Not a lot concrete has happened so far, so it is good that Jesse has 
brought it up. I guess the details will impact (and be impacted by) the 
move to Mercurial, so maybe Dirkjan should be involved?

Michael



> -Frank
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From fwierzbicki at gmail.com  Mon Sep 14 19:47:13 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 14 Sep 2009 13:47:13 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE782D.8020800@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
Message-ID: <4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>

On Mon, Sep 14, 2009 at 1:06 PM, Michael Foord <michael at voidspace.org.uk> wrote:
> No-one argues that the standard library should evolve quickly but there do
> seem to be those arguing that it should *never* evolve.
I'll add my voice against the folks who want the stdlib to never
evolve.  Java's standard library provides a nice cautionary example.

> I agree with Jesse (FWIW) that the presence of obsolete modules is actually
> damaging to Python. Deprecating and removing modules over a four or five
> years (PendingDeprecation -> Deprecation -> removal) is more than slow
> enough for a stable system that is the Python standard library.
>
> I would love to see a PEP about further standard library clean-ups, perhaps
> slating modules for *eventual* removal in Python 3.4 / 3.5 and only when
> they are clearly not useful, replaced or broken and not maintained - and
> preferably where there is a clear alternative.
I too would love to see this -- it would have the added advantage of
lowering the priority for the implementation of deprecated modules
that we haven't gotten around to implementing in Jython yet :)

-Frank

From fwierzbicki at gmail.com  Mon Sep 14 19:50:06 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 14 Sep 2009 13:50:06 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE7E90.8030509@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4222a8490909140930y39fe0e07i2a5d66eb6d9b92ef@mail.gmail.com>
	<4AAE7815.9020101@egenix.com> <4AAE786E.6020207@voidspace.org.uk>
	<4dab5f760909141028v62527232v8e549cbecafcc8fe@mail.gmail.com>
	<4AAE7E90.8030509@egenix.com>
Message-ID: <4dab5f760909141050h316501e1qa20e344661780cea@mail.gmail.com>

On Mon, Sep 14, 2009 at 1:34 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Why don't you try to get those changes into the stdlib ?
That is the current plan until a stdlib separation actually happens
(and will remain the plan if such a separation does not occur).

-Frank

From mal at egenix.com  Mon Sep 14 20:26:44 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 14 Sep 2009 20:26:44 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
Message-ID: <4AAE8AE4.30104@egenix.com>

Frank Wierzbicki wrote:
> On Mon, Sep 14, 2009 at 1:06 PM, Michael Foord <michael at voidspace.org.uk> wrote:
>> No-one argues that the standard library should evolve quickly but there do
>> seem to be those arguing that it should *never* evolve.
> I'll add my voice against the folks who want the stdlib to never
> evolve.  Java's standard library provides a nice cautionary example.

I don't think anyone has voiced such an opinion. There are different
views on what "evolve" should mean and when to apply what actions,
though.

Replacing prefectly fine working code just for the fun of it, does
not count much as argument for evolving the stdlib.

There have to be really good reasons to drop an existing tested and
proven solution in favor of a new implementation that only adds one
or two tricks to the set of already available features, but at the
same time breaks the published API.

The bar is a lot lower for modifying existing modules, e.g. to
enhance or add functionality:

http://www.python.org/dev/peps/pep-0387/

BTW and just because this line gets misquoted so often: the Zen
says "There should be one -- and preferably only one -- obvious way
to do it." ... People tend to forget the "obvious" in that line.

There is often more than one way to look at a problem, so it's
not against the Zen to have more than one implementation to tackle
a certain problem in the stdlib.

A good example of this is the XML package of the stdlib: the modules
all work on XML, but each in its own separate way, specifically
designed with a certain approach in mind.

The urllibs are another example: they both allow accessing URLs
in various ways, but the way they are adapted to fit a particular
need is completely different.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 14 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Mon Sep 14 20:54:55 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 14 Sep 2009 14:54:55 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
Message-ID: <95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>

On Sep 14, 2009, at 11:35 AM, Paul Moore wrote:

> Please remember that some establishments have restrictions that mean
> that tools like easy_install or pip cannot be used. In locked-down
> corporate environments, python-full is potentially all that will be
> available (and maybe very specific "blessed" environment-specific 3rd
> party modules).

Splitting things out for developers is not the same as keeping that  
split visible for distributions, either via tarball, binary from us,  
or through distros.  In fact, I'd venture to guess that most locked  
down establishments are not going to be installing Python from us;  
they'll get it through their operating system vendor (well, thank  
goodness I don't have to know what locked down Windows users have to  
go through ;).

Still, there's no reason why we couldn't ship sumo packages with all  
those batteries included again.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090914/667c91e9/attachment.pgp>

From jnoller at gmail.com  Mon Sep 14 20:56:30 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 14:56:30 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
Message-ID: <4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>

On Mon, Sep 14, 2009 at 2:54 PM, Barry Warsaw <barry at python.org> wrote:
> On Sep 14, 2009, at 11:35 AM, Paul Moore wrote:
>
>> Please remember that some establishments have restrictions that mean
>> that tools like easy_install or pip cannot be used. In locked-down
>> corporate environments, python-full is potentially all that will be
>> available (and maybe very specific "blessed" environment-specific 3rd
>> party modules).
>
> Splitting things out for developers is not the same as keeping that split
> visible for distributions, either via tarball, binary from us, or through
> distros. ?In fact, I'd venture to guess that most locked down establishments
> are not going to be installing Python from us; they'll get it through their
> operating system vendor (well, thank goodness I don't have to know what
> locked down Windows users have to go through ;).
>
> Still, there's no reason why we couldn't ship sumo packages with all those
> batteries included again.
>
> -Barry

Yup; that was spelled out in the OP - I would like: core, stdlib,
everything as 3 packages. 99% of people will download the 3rd.

From michael at voidspace.org.uk  Mon Sep 14 21:08:47 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 20:08:47 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE8AE4.30104@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com>
Message-ID: <4AAE94BF.8080407@voidspace.org.uk>

M.-A. Lemburg wrote:
> Frank Wierzbicki wrote:
>   
>> On Mon, Sep 14, 2009 at 1:06 PM, Michael Foord <michael at voidspace.org.uk> wrote:
>>     
>>> No-one argues that the standard library should evolve quickly but there do
>>> seem to be those arguing that it should *never* evolve.
>>>       
>> I'll add my voice against the folks who want the stdlib to never
>> evolve.  Java's standard library provides a nice cautionary example.
>>     
>
> I don't think anyone has voiced such an opinion. There are different
> views on what "evolve" should mean and when to apply what actions,
> though.
>
>   
 
 From a recent post in this thread:

"That is, actually, pretty much what I want a stdlib for.  I don't want
it to contain the newest, greatest, and best ways of doing things.  I
want it to contain the things that people are willing to maintain until
hell freezes over, which will largely be things that aren't ever going
to change until hell freezes over."

> Replacing prefectly fine working code just for the fun of it, does
> not count much as argument for evolving the stdlib.
>
>   

Unless you are attacking a complete strawman, which is unhelpful and 
pointless so please refrain, can you point out who is suggesting 
replacing working code "just for the fun of it"?

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From jnoller at gmail.com  Mon Sep 14 21:19:05 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 15:19:05 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
Message-ID: <4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>

On Mon, Sep 14, 2009 at 3:11 PM, Brett Cannon <brett at python.org> wrote:
> On Mon, Sep 14, 2009 at 11:56, Jesse Noller <jnoller at gmail.com> wrote:
>> On Mon, Sep 14, 2009 at 2:54 PM, Barry Warsaw <barry at python.org> wrote:
>>> On Sep 14, 2009, at 11:35 AM, Paul Moore wrote:
>>>
>>>> Please remember that some establishments have restrictions that mean
>>>> that tools like easy_install or pip cannot be used. In locked-down
>>>> corporate environments, python-full is potentially all that will be
>>>> available (and maybe very specific "blessed" environment-specific 3rd
>>>> party modules).
>>>
>>> Splitting things out for developers is not the same as keeping that split
>>> visible for distributions, either via tarball, binary from us, or through
>>> distros. ?In fact, I'd venture to guess that most locked down establishments
>>> are not going to be installing Python from us; they'll get it through their
>>> operating system vendor (well, thank goodness I don't have to know what
>>> locked down Windows users have to go through ;).
>>>
>>> Still, there's no reason why we couldn't ship sumo packages with all those
>>> batteries included again.
>>>
>>> -Barry
>>
>> Yup; that was spelled out in the OP - I would like: core, stdlib,
>> everything as 3 packages. 99% of people will download the 3rd.
>
> Just to toss in my opinion, I think the standard library should be
> broken out in the VCS to make it very obvious what all Python VMs
> should come with and work with, but I don't think we should package it
> up for distribution separately. CPython should probably shift to
> having a slightly less stranglehold on the standard library than it
> has now. This would also help legitimize the other VMs.
>
> But I see no benefit for the general populace in having a version of
> Python w/o a standard library. Anyone who has funky space requirements
> can just do the leg work needed prune down the standard library to
> what they need.
>
> -Brett
>

Yeah: Except for those people that means custom compiling an
interpreter too. The tight coupling is just painful. When I want to
trim the standard library, I should not have to hack the build
scripts, compile an interpreter, etc, etc.

I'm really strongly (duh) for massive decoupling between the two,
especially within the build system.

How is there any harm in offering 3 downloads? The obvious thing is to
click on the big "get some pythons on" button which gets what we know
as python today.

Then there are two little buttons: get "just this" or "just that".

From brett at python.org  Mon Sep 14 21:21:10 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 12:21:10 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE782D.8020800@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com> 
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk> 
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com> 
	<1252947207.7398.8.camel@localhost> <4AAE782D.8020800@voidspace.org.uk>
Message-ID: <bbaeab100909141221g1af442a8y2c71884665cdcc51@mail.gmail.com>

On Mon, Sep 14, 2009 at 10:06, Michael Foord <michael at voidspace.org.uk> wrote:
> Antoine Pitrou wrote:
>>
>> Le lundi 14 septembre 2009 ? 12:19 -0400, Jesse Noller a ?crit :
>>
>>>
>>> Sure, having batteries is fantastic. Until you being to realize that
>>> some modules are broken, not up to date with current technology, etc,
>>> etc. We have to be aggressive with cleaning it up.
>>>
>>
>> [...]
>>
>>>
>>> Things within the stdlib must have a high quality bar, and should
>>> really represent the "one way of doing something" - meaning best of
>>> breed, high quality, idiomatic, etc.
>>>
>>
>> Have you read the thread about argparse and the deprecation of other
>> modules?
>>
>> You are conflating two things:
>> 1. the presence of somewhat outdated or too low-level modules
>> 2. the desire to have best-of-breed options available in the stdlib
>>
>> We can do 2. without undoing 1. As Marc-Andr? pointed out, even in py3k
>> we only deprecated a handful of very marginal modules. Emphasizing the
>> right modules in the documentation is probably a reasonably good answer
>> to the problem of having several modules fulfilling the same needs. No
>> need to go on a deprecation frenzy and start annoying the two thirds of
>> our user base just because we decided to be pure rather than practical.
>>
>>
>
> I think both 1) and 2) are on topic for a discussion of how we deal with
> standard library quality.
>
> No-one argues that the standard library should evolve quickly but there do
> seem to be those arguing that it should *never* evolve.
>
> I agree with Jesse (FWIW) that the presence of obsolete modules is actually
> damaging to Python.

Both in reputation and in developer time. It doesn't help our image
when someone finds a module in our standard library that is
practically unmaintained and goes w/o having patches applied. I think
this is Jesse's motivation behind wanting owners of modules; since the
core devs are volunteers we only patch stuff we care about, which
leads to orphaned modules rotting in the stdlib. And tying into this,
having to patch and fix modules that are outdated and not widely used
is a waste of time when it could have been spent working on code that
is more impactful on the wider community.

> Deprecating and removing modules over a four or five
> years (PendingDeprecation -> Deprecation -> removal) is more than slow
> enough for a stable system that is the Python standard library.
>

Well, to get your four or five year warning you need to have three
releases (3 * 18 months = 4.5 years), so releases would have to go
PendingDeprecation -> Deprecation -> Deprecation -> removal.

> I would love to see a PEP about further standard library clean-ups, perhaps
> slating modules for *eventual* removal in Python 3.4 / 3.5 and only when
> they are clearly not useful, replaced or broken and not maintained - and
> preferably where there is a clear alternative.

You can go back and look at early drafts of PEP 3108 for ideas. To
simply keep myself from burning out and making sure something happened
I had to cut a lot out, but I bet there is stuff we can toss (I mean
do we really need to be supporting any multimedia formats like
sunau?). If there is enough support we can try to come up with
standard library inclusion guidelines (what we are striving for,
quality standards, etc.) and then apply those to the existing stdlib
to potentially clean it up.

-Brett

From jnoller at gmail.com  Mon Sep 14 21:29:27 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 15:29:27 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141226j6afa075ekea3572d96f7644ca@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
	<bbaeab100909141226j6afa075ekea3572d96f7644ca@mail.gmail.com>
Message-ID: <4222a8490909141229q7453123dua2e118a5c8e3f941@mail.gmail.com>

On Mon, Sep 14, 2009 at 3:26 PM, Brett Cannon <brett at python.org> wrote:
> On Mon, Sep 14, 2009 at 12:19, Jesse Noller <jnoller at gmail.com> wrote:
>> On Mon, Sep 14, 2009 at 3:11 PM, Brett Cannon <brett at python.org> wrote:
>>> On Mon, Sep 14, 2009 at 11:56, Jesse Noller <jnoller at gmail.com> wrote:
>>>> On Mon, Sep 14, 2009 at 2:54 PM, Barry Warsaw <barry at python.org> wrote:
>>>>> On Sep 14, 2009, at 11:35 AM, Paul Moore wrote:
>>>>>
>>>>>> Please remember that some establishments have restrictions that mean
>>>>>> that tools like easy_install or pip cannot be used. In locked-down
>>>>>> corporate environments, python-full is potentially all that will be
>>>>>> available (and maybe very specific "blessed" environment-specific 3rd
>>>>>> party modules).
>>>>>
>>>>> Splitting things out for developers is not the same as keeping that split
>>>>> visible for distributions, either via tarball, binary from us, or through
>>>>> distros. ?In fact, I'd venture to guess that most locked down establishments
>>>>> are not going to be installing Python from us; they'll get it through their
>>>>> operating system vendor (well, thank goodness I don't have to know what
>>>>> locked down Windows users have to go through ;).
>>>>>
>>>>> Still, there's no reason why we couldn't ship sumo packages with all those
>>>>> batteries included again.
>>>>>
>>>>> -Barry
>>>>
>>>> Yup; that was spelled out in the OP - I would like: core, stdlib,
>>>> everything as 3 packages. 99% of people will download the 3rd.
>>>
>>> Just to toss in my opinion, I think the standard library should be
>>> broken out in the VCS to make it very obvious what all Python VMs
>>> should come with and work with, but I don't think we should package it
>>> up for distribution separately. CPython should probably shift to
>>> having a slightly less stranglehold on the standard library than it
>>> has now. This would also help legitimize the other VMs.
>>>
>>> But I see no benefit for the general populace in having a version of
>>> Python w/o a standard library. Anyone who has funky space requirements
>>> can just do the leg work needed prune down the standard library to
>>> what they need.
>>>
>>> -Brett
>>>
>>
>> Yeah: Except for those people that means custom compiling an
>> interpreter too. The tight coupling is just painful. When I want to
>> trim the standard library, I should not have to hack the build
>> scripts, compile an interpreter, etc, etc.
>>
>> I'm really strongly (duh) for massive decoupling between the two,
>> especially within the build system.
>>
>
> Decoupling in the build system is a good idea and would naturally
> happen if we broke out the standard library in the VCS.

\o/

>> How is there any harm in offering 3 downloads? The obvious thing is to
>> click on the big "get some pythons on" button which gets what we know
>> as python today.
>>
>> Then there are two little buttons: get "just this" or "just that".
>
> Because I don't want to have to start telling people "download the
> full Python distribution, not the interpreter-only one; that's only
> for those folk who want to stuff Python on an embedded device." That
> seems silly. And you know some newbie will screw up, download only the
> interpreter version and wonder why he can't import some module. The
> amount of people who are going to screw up on what to download will
> most likely be larger than the people who are going to save some time
> downloading just the interpreter instead of having to tweak something
> for an embedded device.
>
> -Brett
>

What if the font for the "only get dis" is really, really small? Or on
a different "advanced yo" page?

We could put it in the developer's FAQ - no one reads that ;)

From jnoller at gmail.com  Mon Sep 14 21:30:10 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 15:30:10 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141221g1af442a8y2c71884665cdcc51@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<bbaeab100909141221g1af442a8y2c71884665cdcc51@mail.gmail.com>
Message-ID: <4222a8490909141230p194eee2ua11d43c8742e11c5@mail.gmail.com>

On Mon, Sep 14, 2009 at 3:21 PM, Brett Cannon <brett at python.org> wrote:

> Both in reputation and in developer time. It doesn't help our image
> when someone finds a module in our standard library that is
> practically unmaintained and goes w/o having patches applied. I think
> this is Jesse's motivation behind wanting owners of modules; since the
> core devs are volunteers we only patch stuff we care about, which
> leads to orphaned modules rotting in the stdlib. And tying into this,
> having to patch and fix modules that are outdated and not widely used
> is a waste of time when it could have been spent working on code that
> is more impactful on the wider community.

Brett is correct in my pain/motivation.

From michael at voidspace.org.uk  Mon Sep 14 21:31:13 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 20:31:13 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141221g1af442a8y2c71884665cdcc51@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<bbaeab100909141221g1af442a8y2c71884665cdcc51@mail.gmail.com>
Message-ID: <4AAE9A01.8000201@voidspace.org.uk>

Brett Cannon wrote:
> On Mon, Sep 14, 2009 at 10:06, Michael Foord <michael at voidspace.org.uk> wrote:
>   
>> Antoine Pitrou wrote:
>>     
>>> Le lundi 14 septembre 2009 ? 12:19 -0400, Jesse Noller a ?crit :
>>>
>>>       
>>>> Sure, having batteries is fantastic. Until you being to realize that
>>>> some modules are broken, not up to date with current technology, etc,
>>>> etc. We have to be aggressive with cleaning it up.
>>>>
>>>>         
>>> [...]
>>>
>>>       
>>>> Things within the stdlib must have a high quality bar, and should
>>>> really represent the "one way of doing something" - meaning best of
>>>> breed, high quality, idiomatic, etc.
>>>>
>>>>         
>>> Have you read the thread about argparse and the deprecation of other
>>> modules?
>>>
>>> You are conflating two things:
>>> 1. the presence of somewhat outdated or too low-level modules
>>> 2. the desire to have best-of-breed options available in the stdlib
>>>
>>> We can do 2. without undoing 1. As Marc-Andr? pointed out, even in py3k
>>> we only deprecated a handful of very marginal modules. Emphasizing the
>>> right modules in the documentation is probably a reasonably good answer
>>> to the problem of having several modules fulfilling the same needs. No
>>> need to go on a deprecation frenzy and start annoying the two thirds of
>>> our user base just because we decided to be pure rather than practical.
>>>
>>>
>>>       
>> I think both 1) and 2) are on topic for a discussion of how we deal with
>> standard library quality.
>>
>> No-one argues that the standard library should evolve quickly but there do
>> seem to be those arguing that it should *never* evolve.
>>
>> I agree with Jesse (FWIW) that the presence of obsolete modules is actually
>> damaging to Python.
>>     
>
> Both in reputation and in developer time. It doesn't help our image
> when someone finds a module in our standard library that is
> practically unmaintained and goes w/o having patches applied. I think
> this is Jesse's motivation behind wanting owners of modules; since the
> core devs are volunteers we only patch stuff we care about, which
> leads to orphaned modules rotting in the stdlib. And tying into this,
> having to patch and fix modules that are outdated and not widely used
> is a waste of time when it could have been spent working on code that
> is more impactful on the wider community.
>
>   

I totally agree. It wastes our time and it wastes the time of users who 
attempt to use these modules and get frustrated with Python as a result.

>> Deprecating and removing modules over a four or five
>> years (PendingDeprecation -> Deprecation -> removal) is more than slow
>> enough for a stable system that is the Python standard library.
>>
>>     
>
> Well, to get your four or five year warning you need to have three
> releases (3 * 18 months = 4.5 years), so releases would have to go
> PendingDeprecation -> Deprecation -> Deprecation -> removal.
>
>   

That sounds fine to me.

>> I would love to see a PEP about further standard library clean-ups, perhaps
>> slating modules for *eventual* removal in Python 3.4 / 3.5 and only when
>> they are clearly not useful, replaced or broken and not maintained - and
>> preferably where there is a clear alternative.
>>     
>
> You can go back and look at early drafts of PEP 3108 for ideas. To
> simply keep myself from burning out and making sure something happened
> I had to cut a lot out, but I bet there is stuff we can toss (I mean
> do we really need to be supporting any multimedia formats like
> sunau?). If there is enough support we can try to come up with
> standard library inclusion guidelines (what we are striving for,
> quality standards, etc.) and then apply those to the existing stdlib
> to potentially clean it up.
>
> -Brett
>   
Yep, sounds like a good way forward for Jesse. He has my full support. ;-)

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Mon Sep 14 21:11:45 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 12:11:45 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com> 
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
Message-ID: <bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>

On Mon, Sep 14, 2009 at 11:56, Jesse Noller <jnoller at gmail.com> wrote:
> On Mon, Sep 14, 2009 at 2:54 PM, Barry Warsaw <barry at python.org> wrote:
>> On Sep 14, 2009, at 11:35 AM, Paul Moore wrote:
>>
>>> Please remember that some establishments have restrictions that mean
>>> that tools like easy_install or pip cannot be used. In locked-down
>>> corporate environments, python-full is potentially all that will be
>>> available (and maybe very specific "blessed" environment-specific 3rd
>>> party modules).
>>
>> Splitting things out for developers is not the same as keeping that split
>> visible for distributions, either via tarball, binary from us, or through
>> distros. ?In fact, I'd venture to guess that most locked down establishments
>> are not going to be installing Python from us; they'll get it through their
>> operating system vendor (well, thank goodness I don't have to know what
>> locked down Windows users have to go through ;).
>>
>> Still, there's no reason why we couldn't ship sumo packages with all those
>> batteries included again.
>>
>> -Barry
>
> Yup; that was spelled out in the OP - I would like: core, stdlib,
> everything as 3 packages. 99% of people will download the 3rd.

Just to toss in my opinion, I think the standard library should be
broken out in the VCS to make it very obvious what all Python VMs
should come with and work with, but I don't think we should package it
up for distribution separately. CPython should probably shift to
having a slightly less stranglehold on the standard library than it
has now. This would also help legitimize the other VMs.

But I see no benefit for the general populace in having a version of
Python w/o a standard library. Anyone who has funky space requirements
can just do the leg work needed prune down the standard library to
what they need.

-Brett

From santagada at gmail.com  Mon Sep 14 21:46:35 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 14 Sep 2009 16:46:35 -0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
Message-ID: <26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>


On Sep 14, 2009, at 4:19 PM, Jesse Noller wrote:

> How is there any harm in offering 3 downloads? The obvious thing is to
> click on the big "get some pythons on" button which gets what we know
> as python today.

Confusing the hell out of users. Right now there is a lot more links  
on the download page than it should (why offer both a tar.gz and a bz2  
on the main download page?). Another improvement would be to divide it  
by operating system with an icon (and name, just the icon or the name  
is not enough to most people). But this is of topic.

IMHO we should divide this dicussion in 3:

1) Breaking the repo to help all the other python implementations.  
This means moving tests and let the python implementations start  
classifying which tests are platform dependent or not. (I would vote  
+1 if I could)

2) Having different python versions with various levels of  
completeness, which could be broken in two:
   a) only on the repository
   b) offer different distributions on the main python site (-1)

3) What should the stdlib be (should be discussed elsewhere)

I think the topic discussed on pycon 2009 (from all the accounts I  
have heard) is only about item 1, and it should not matter much to the  
other ones (well maybe 2a can change things a little).

--
Leonardo Santagada
santagada at gmail.com




From jnoller at gmail.com  Mon Sep 14 21:53:18 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 14 Sep 2009 15:53:18 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
Message-ID: <4222a8490909141253k69354f18vd5e35c6b79cc58f1@mail.gmail.com>

On Mon, Sep 14, 2009 at 3:46 PM, Leonardo Santagada <santagada at gmail.com> wrote:
>
> On Sep 14, 2009, at 4:19 PM, Jesse Noller wrote:
>
>> How is there any harm in offering 3 downloads? The obvious thing is to
>> click on the big "get some pythons on" button which gets what we know
>> as python today.
>
> Confusing the hell out of users. Right now there is a lot more links on the
> download page than it should (why offer both a tar.gz and a bz2 on the main
> download page?). Another improvement would be to divide it by operating
> system with an icon (and name, just the icon or the name is not enough to
> most people). But this is of topic.

The python.org website should be redesigned, completely. But then
again, that's something for another thread ;) It's also not a
fundamental issue for this thread.


> 2) Having different python versions with various levels of completeness,
> which could be broken in two:
> ?a) only on the repository
> ?b) offer different distributions on the main python site (-1)

Modulo a cleaner, simpler download page: Is it still -1?

I plan on bringing up the website design elsewhere, as that was
another thing at pycon 2009 which was brought up, but failed to escape
the atmosphere.

jesse

From michael at voidspace.org.uk  Mon Sep 14 21:55:26 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 20:55:26 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909141253k69354f18vd5e35c6b79cc58f1@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
	<4222a8490909141253k69354f18vd5e35c6b79cc58f1@mail.gmail.com>
Message-ID: <4AAE9FAE.4010200@voidspace.org.uk>

Jesse Noller wrote:
> On Mon, Sep 14, 2009 at 3:46 PM, Leonardo Santagada <santagada at gmail.com> wrote:
>   
>> On Sep 14, 2009, at 4:19 PM, Jesse Noller wrote:
>>
>>     
>>> How is there any harm in offering 3 downloads? The obvious thing is to
>>> click on the big "get some pythons on" button which gets what we know
>>> as python today.
>>>       
>> Confusing the hell out of users. Right now there is a lot more links on the
>> download page than it should (why offer both a tar.gz and a bz2 on the main
>> download page?). Another improvement would be to divide it by operating
>> system with an icon (and name, just the icon or the name is not enough to
>> most people). But this is of topic.
>>     
>
> The python.org website should be redesigned, completely. But then
> again, that's something for another thread ;) It's also not a
> fundamental issue for this thread.
>
>
>   
>> 2) Having different python versions with various levels of completeness,
>> which could be broken in two:
>>  a) only on the repository
>>  b) offer different distributions on the main python site (-1)
>>     
>
> Modulo a cleaner, simpler download page: Is it still -1?
>
> I plan on bringing up the website design elsewhere, as that was
> another thing at pycon 2009 which was brought up, but failed to escape
> the atmosphere.
>
>   
It was recently discussed on the pydotorg mailing list - with someone 
even volunteering their time and submitting an example design that was 
(in my opinion) excellent. Nonetheless, the pydotorg mailing list is the 
correct one for website issues.

Michael


> jesse
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Mon Sep 14 21:55:50 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 12:55:50 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com> 
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com> 
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com> 
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com> 
	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
Message-ID: <bbaeab100909141255m4f80f7cbsec29f17bc8fe9f@mail.gmail.com>

On Mon, Sep 14, 2009 at 12:46, Leonardo Santagada <santagada at gmail.com> wrote:
>
> On Sep 14, 2009, at 4:19 PM, Jesse Noller wrote:
>
>> How is there any harm in offering 3 downloads? The obvious thing is to
>> click on the big "get some pythons on" button which gets what we know
>> as python today.
>
> Confusing the hell out of users. Right now there is a lot more links on the
> download page than it should (why offer both a tar.gz and a bz2 on the main
> download page?). Another improvement would be to divide it by operating
> system with an icon (and name, just the icon or the name is not enough to
> most people). But this is of topic.
>
> IMHO we should divide this dicussion in 3:
>
> 1) Breaking the repo to help all the other python implementations. This
> means moving tests and let the python implementations start classifying
> which tests are platform dependent or not. (I would vote +1 if I could)
>

This actually doesn't really require a discussion until someone
volunteers to spearhead it because it will happen, it's just a
question of who will lead it and whether they will wait for the
Mercurial transition.

-Brett

From brett at python.org  Mon Sep 14 21:56:57 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 12:56:57 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909141253k69354f18vd5e35c6b79cc58f1@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com> 
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com> 
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com> 
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com> 
	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
	<4222a8490909141253k69354f18vd5e35c6b79cc58f1@mail.gmail.com>
Message-ID: <bbaeab100909141256g1009b9e3x3e134cf91b59c441@mail.gmail.com>

On Mon, Sep 14, 2009 at 12:53, Jesse Noller <jnoller at gmail.com> wrote:
> On Mon, Sep 14, 2009 at 3:46 PM, Leonardo Santagada <santagada at gmail.com> wrote:
>>
>> On Sep 14, 2009, at 4:19 PM, Jesse Noller wrote:
>>
>>> How is there any harm in offering 3 downloads? The obvious thing is to
>>> click on the big "get some pythons on" button which gets what we know
>>> as python today.
>>
>> Confusing the hell out of users. Right now there is a lot more links on the
>> download page than it should (why offer both a tar.gz and a bz2 on the main
>> download page?). Another improvement would be to divide it by operating
>> system with an icon (and name, just the icon or the name is not enough to
>> most people). But this is of topic.
>
> The python.org website should be redesigned, completely. But then
> again, that's something for another thread ;) It's also not a
> fundamental issue for this thread.
>
>
>> 2) Having different python versions with various levels of completeness,
>> which could be broken in two:
>> ?a) only on the repository
>> ?b) offer different distributions on the main python site (-1)
>
> Modulo a cleaner, simpler download page: Is it still -1?

I'm -1 even with a cleaner page. Google doesn't return results based
on what we want.

-Brett

From p.f.moore at gmail.com  Mon Sep 14 22:00:34 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 14 Sep 2009 21:00:34 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE6326.5090402@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<4AAE6326.5090402@voidspace.org.uk>
Message-ID: <79990c6b0909141300o6c61d03idc960bcf7f31a10d@mail.gmail.com>

2009/9/14 Michael Foord <michael at voidspace.org.uk>:
> For reference, where is that stated?

Sorry, it was stated by Jesse in his original post (a bit I clipped):
"How do we make sure that the stdlib is stable, best of breed and high
quality?". I didn't mean to imply that it was necessarily policy.

Paul.

From michael at voidspace.org.uk  Mon Sep 14 22:02:09 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 21:02:09 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909141300o6c61d03idc960bcf7f31a10d@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	
	<4AAE6326.5090402@voidspace.org.uk>
	<79990c6b0909141300o6c61d03idc960bcf7f31a10d@mail.gmail.com>
Message-ID: <4AAEA141.8080409@voidspace.org.uk>

Paul Moore wrote:
> 2009/9/14 Michael Foord <michael at voidspace.org.uk>:
>   
>> For reference, where is that stated?
>>     
>
> Sorry, it was stated by Jesse in his original post (a bit I clipped):
> "How do we make sure that the stdlib is stable, best of breed and high
> quality?". I didn't mean to imply that it was necessarily policy.
>
> Paul.
>   
You've also clipped enough of *this* conversation that I can't now 
remember what you're referring to...

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Mon Sep 14 21:26:38 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 12:26:38 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com> 
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com> 
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com> 
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
Message-ID: <bbaeab100909141226j6afa075ekea3572d96f7644ca@mail.gmail.com>

On Mon, Sep 14, 2009 at 12:19, Jesse Noller <jnoller at gmail.com> wrote:
> On Mon, Sep 14, 2009 at 3:11 PM, Brett Cannon <brett at python.org> wrote:
>> On Mon, Sep 14, 2009 at 11:56, Jesse Noller <jnoller at gmail.com> wrote:
>>> On Mon, Sep 14, 2009 at 2:54 PM, Barry Warsaw <barry at python.org> wrote:
>>>> On Sep 14, 2009, at 11:35 AM, Paul Moore wrote:
>>>>
>>>>> Please remember that some establishments have restrictions that mean
>>>>> that tools like easy_install or pip cannot be used. In locked-down
>>>>> corporate environments, python-full is potentially all that will be
>>>>> available (and maybe very specific "blessed" environment-specific 3rd
>>>>> party modules).
>>>>
>>>> Splitting things out for developers is not the same as keeping that split
>>>> visible for distributions, either via tarball, binary from us, or through
>>>> distros. ?In fact, I'd venture to guess that most locked down establishments
>>>> are not going to be installing Python from us; they'll get it through their
>>>> operating system vendor (well, thank goodness I don't have to know what
>>>> locked down Windows users have to go through ;).
>>>>
>>>> Still, there's no reason why we couldn't ship sumo packages with all those
>>>> batteries included again.
>>>>
>>>> -Barry
>>>
>>> Yup; that was spelled out in the OP - I would like: core, stdlib,
>>> everything as 3 packages. 99% of people will download the 3rd.
>>
>> Just to toss in my opinion, I think the standard library should be
>> broken out in the VCS to make it very obvious what all Python VMs
>> should come with and work with, but I don't think we should package it
>> up for distribution separately. CPython should probably shift to
>> having a slightly less stranglehold on the standard library than it
>> has now. This would also help legitimize the other VMs.
>>
>> But I see no benefit for the general populace in having a version of
>> Python w/o a standard library. Anyone who has funky space requirements
>> can just do the leg work needed prune down the standard library to
>> what they need.
>>
>> -Brett
>>
>
> Yeah: Except for those people that means custom compiling an
> interpreter too. The tight coupling is just painful. When I want to
> trim the standard library, I should not have to hack the build
> scripts, compile an interpreter, etc, etc.
>
> I'm really strongly (duh) for massive decoupling between the two,
> especially within the build system.
>

Decoupling in the build system is a good idea and would naturally
happen if we broke out the standard library in the VCS.

> How is there any harm in offering 3 downloads? The obvious thing is to
> click on the big "get some pythons on" button which gets what we know
> as python today.
>
> Then there are two little buttons: get "just this" or "just that".

Because I don't want to have to start telling people "download the
full Python distribution, not the interpreter-only one; that's only
for those folk who want to stuff Python on an embedded device." That
seems silly. And you know some newbie will screw up, download only the
interpreter version and wonder why he can't import some module. The
amount of people who are going to screw up on what to download will
most likely be larger than the people who are going to save some time
downloading just the interpreter instead of having to tweak something
for an embedded device.

-Brett

From fwierzbicki at gmail.com  Mon Sep 14 23:14:23 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 14 Sep 2009 17:14:23 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141226j6afa075ekea3572d96f7644ca@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
	<bbaeab100909141226j6afa075ekea3572d96f7644ca@mail.gmail.com>
Message-ID: <4dab5f760909141414t4262c2cbqcd1a4fed2f713de3@mail.gmail.com>

I remember some build time options in other projects (Dojo is the one
that pops to mind) that take a build time configuration file that
lists the modules to be included.  Would this be enough?  It would
make for a reasonably easy way to build stripped down distros without
giving people a confusion causing download link...

-Frank

From fwierzbicki at gmail.com  Mon Sep 14 23:32:43 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 14 Sep 2009 17:32:43 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141255m4f80f7cbsec29f17bc8fe9f@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
	<bbaeab100909141255m4f80f7cbsec29f17bc8fe9f@mail.gmail.com>
Message-ID: <4dab5f760909141432q5747febay5f864b9758d55fe9@mail.gmail.com>

On Mon, Sep 14, 2009 at 3:55 PM, Brett Cannon <brett at python.org> wrote:
> This actually doesn't really require a discussion until someone
> volunteers to spearhead it because it will happen, it's just a
> question of who will lead it and whether they will wait for the
> Mercurial transition.
Since I represent one of the other python implementations that need
help, I'd volunteer to spearhead the effort to break up the repo.  I
probably would prefer to wait until the mercurial transition has
happened to help with implementation, but that wouldn't stop me from
having the discussion and starting a PEP since we'll need to work out
which modules are which.

-Frank

From brett at python.org  Mon Sep 14 23:40:18 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Sep 2009 14:40:18 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4dab5f760909141432q5747febay5f864b9758d55fe9@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com> 
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com> 
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com> 
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com> 
	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
	<bbaeab100909141255m4f80f7cbsec29f17bc8fe9f@mail.gmail.com> 
	<4dab5f760909141432q5747febay5f864b9758d55fe9@mail.gmail.com>
Message-ID: <bbaeab100909141440j603edc1do7ece9019a0fcbb45@mail.gmail.com>

On Mon, Sep 14, 2009 at 14:32, Frank Wierzbicki <fwierzbicki at gmail.com> wrote:
> On Mon, Sep 14, 2009 at 3:55 PM, Brett Cannon <brett at python.org> wrote:
>> This actually doesn't really require a discussion until someone
>> volunteers to spearhead it because it will happen, it's just a
>> question of who will lead it and whether they will wait for the
>> Mercurial transition.
> Since I represent one of the other python implementations that need
> help, I'd volunteer to spearhead the effort to break up the repo. ?I
> probably would prefer to wait until the mercurial transition has
> happened to help with implementation, but that wouldn't stop me from
> having the discussion and starting a PEP since we'll need to work out
> which modules are which.

Great! PEP is going to be needed anyway, so that's a start. As you
said, it can say that the transition will not happen until we switch
to Mercurial. The PEP should probably specify what the criteria is for
a module to be expected to be available for all VMs, what modules
currently do not meet that and thus would be Python-only, how to
document that in Sphinx, structure of the code, and whether anything
beyond sys will have intra-module stuff that is restricted to any one
VM.

Do you want to discuss this on this list, another list, or take it
off-list initially until there is something more concrete to present
here or on python-dev?

-Brett

From fwierzbicki at gmail.com  Mon Sep 14 23:47:16 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 14 Sep 2009 17:47:16 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909141440j603edc1do7ece9019a0fcbb45@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<95CA9B37-A650-42C2-9B7B-958B32F53145@python.org>
	<4222a8490909141156h4140273eo9c97e5239bdf22aa@mail.gmail.com>
	<bbaeab100909141211x8ed68ffhcc60c5fc7b163a80@mail.gmail.com>
	<4222a8490909141219s16f04666ic64a711146821d06@mail.gmail.com>
	<26CA0FF9-4625-4FC4-BBA7-8A03C324BDC5@gmail.com>
	<bbaeab100909141255m4f80f7cbsec29f17bc8fe9f@mail.gmail.com>
	<4dab5f760909141432q5747febay5f864b9758d55fe9@mail.gmail.com>
	<bbaeab100909141440j603edc1do7ece9019a0fcbb45@mail.gmail.com>
Message-ID: <4dab5f760909141447v2f8df007k491517253647bb57@mail.gmail.com>

On Mon, Sep 14, 2009 at 5:40 PM, Brett Cannon <brett at python.org> wrote:
> Do you want to discuss this on this list, another list, or take it
> off-list initially until there is something more concrete to present
> here or on python-dev?
I'll take a crack at writing up a PEP and bring it back here when I
have something worth discussing.

-Frank

From mal at egenix.com  Tue Sep 15 00:07:58 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 00:07:58 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAE94BF.8080407@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>
Message-ID: <4AAEBEBE.1050105@egenix.com>

Michael Foord wrote:
> M.-A. Lemburg wrote:
>> Frank Wierzbicki wrote:
>>  
>>> On Mon, Sep 14, 2009 at 1:06 PM, Michael Foord
>>> <michael at voidspace.org.uk> wrote:
>>>    
>>>> No-one argues that the standard library should evolve quickly but
>>>> there do
>>>> seem to be those arguing that it should *never* evolve.
>>>>       
>>> I'll add my voice against the folks who want the stdlib to never
>>> evolve.  Java's standard library provides a nice cautionary example.
>>>     
>>
>> I don't think anyone has voiced such an opinion. There are different
>> views on what "evolve" should mean and when to apply what actions,
>> though.
>>
>>   
> 
> From a recent post in this thread:
> 
> "That is, actually, pretty much what I want a stdlib for.  I don't want
> it to contain the newest, greatest, and best ways of doing things.  I
> want it to contain the things that people are willing to maintain until
> hell freezes over, which will largely be things that aren't ever going
> to change until hell freezes over."

Sigh, when quoting people, could you please leave their name in
place. Thanks.

The above quote is from Laura and, yes, it can be read in many
ways.

People who have been working with Python for many many years and
have a huge code base to maintain do have a more reserved attitude
towards changes - simply because they know how much work is
involved in applying such major API changes to a code base, including
retesting it all and getting it to work in a production environment
again.

And no, it's not a good idea to stick with some old release of
Python for the sake of stability - you end up having to maintain
3-5 different Python installations on every single deployment
machine...

>> Replacing prefectly fine working code just for the fun of it, does
>> not count much as argument for evolving the stdlib.
>>
>>   
> 
> Unless you are attacking a complete strawman, which is unhelpful and
> pointless so please refrain, can you point out who is suggesting
> replacing working code "just for the fun of it"?

Just have a look at the various arguments for adding argparse to the
stdlib with the intention of replacing optparse and getopt.

On one hand you have this new API which is not compatible with optparse:

http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html#upgrading-optparse-code

On the other you have a rather short list of features that make
argparse different from optparse:

http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html

and the fact that argparse has been in the wild for 4.5 months.

optparse was added to Python in 2002 and had been around for
a few years before that.

getopt was added in 1990.

Now, take into consideration that most command line scripts you
write use one of the two modules and imagine the huge number of
scripts out there which would have to be updated because
of such a change in API.

If you put everything together you get what I call "replacing
working code just for the fun of it".

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 14 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From michael at voidspace.org.uk  Tue Sep 15 00:13:00 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 23:13:00 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAEBEBE.1050105@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com>
Message-ID: <4AAEBFEC.7060507@voidspace.org.uk>

M.-A. Lemburg wrote:
> [snip...]
>>> Replacing prefectly fine working code just for the fun of it, does
>>> not count much as argument for evolving the stdlib.
>>>
>>>   
>>>       
>> Unless you are attacking a complete strawman, which is unhelpful and
>> pointless so please refrain, can you point out who is suggesting
>> replacing working code "just for the fun of it"?
>>     
>
> Just have a look at the various arguments for adding argparse to the
> stdlib with the intention of replacing optparse and getopt.
>
> On one hand you have this new API which is not compatible with optparse:
>
> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html#upgrading-optparse-code
>
> On the other you have a rather short list of features that make
> argparse different from optparse:
>
> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
>
> and the fact that argparse has been in the wild for 4.5 months.
>   

Here's an email from 2007 asking when it will be in the standard library:

http://mail.python.org/pipermail/python-list/2007-January/592646.html


Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From mal at egenix.com  Tue Sep 15 00:37:16 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 00:37:16 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAEBFEC.7060507@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
Message-ID: <4AAEC59C.7060605@egenix.com>

Michael Foord wrote:
> M.-A. Lemburg wrote:
>> [snip...]
>>>> Replacing prefectly fine working code just for the fun of it, does
>>>> not count much as argument for evolving the stdlib.
>>>>
>>>>         
>>> Unless you are attacking a complete strawman, which is unhelpful and
>>> pointless so please refrain, can you point out who is suggesting
>>> replacing working code "just for the fun of it"?
>>>     
>>
>> Just have a look at the various arguments for adding argparse to the
>> stdlib with the intention of replacing optparse and getopt.
>>
>> On one hand you have this new API which is not compatible with optparse:
>>
>> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html#upgrading-optparse-code
>>
>>
>> On the other you have a rather short list of features that make
>> argparse different from optparse:
>>
>> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
>>
>> and the fact that argparse has been in the wild for 4.5 months.
>>   
> 
> Here's an email from 2007 asking when it will be in the standard library:
> 
> http://mail.python.org/pipermail/python-list/2007-January/592646.html

I was looking at this page:

http://code.google.com/p/argparse/downloads/list

It turned 1.0 in July and the first release (on Google Code) was on
April 1st this year.

That doesn't say anything about the robustness of the code, but then:
how much traction can you get in those few months and how likely are
API changes to the code in a 1.1 or 2.0 release ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From michael at voidspace.org.uk  Tue Sep 15 00:44:24 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Sep 2009 23:44:24 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAEC59C.7060605@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
Message-ID: <4AAEC748.4060406@voidspace.org.uk>

M.-A. Lemburg wrote:
> Michael Foord wrote:
>   
>> M.-A. Lemburg wrote:
>>     
>>> [snip...]
>>>       
>>>>> Replacing prefectly fine working code just for the fun of it, does
>>>>> not count much as argument for evolving the stdlib.
>>>>>
>>>>>         
>>>>>           
>>>> Unless you are attacking a complete strawman, which is unhelpful and
>>>> pointless so please refrain, can you point out who is suggesting
>>>> replacing working code "just for the fun of it"?
>>>>     
>>>>         
>>> Just have a look at the various arguments for adding argparse to the
>>> stdlib with the intention of replacing optparse and getopt.
>>>
>>> On one hand you have this new API which is not compatible with optparse:
>>>
>>> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html#upgrading-optparse-code
>>>
>>>
>>> On the other you have a rather short list of features that make
>>> argparse different from optparse:
>>>
>>> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
>>>
>>> and the fact that argparse has been in the wild for 4.5 months.
>>>   
>>>       
>> Here's an email from 2007 asking when it will be in the standard library:
>>
>> http://mail.python.org/pipermail/python-list/2007-January/592646.html
>>     
>
> I was looking at this page:
>
> http://code.google.com/p/argparse/downloads/list
>
> It turned 1.0 in July and the first release (on Google Code) was on
> April 1st this year.
>
> That doesn't say anything about the robustness of the code, but then:
> how much traction can you get in those few months and how likely are
> API changes to the code in a 1.1 or 2.0 release ?
>
>   
argparse has been available and in use in the community since at the 
*latest* 2006. As for API stability you will have to ask Steven.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From orestis at orestis.gr  Tue Sep 15 00:49:51 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Tue, 15 Sep 2009 01:49:51 +0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAEC748.4060406@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com> <4AAEC748.4060406@voidspace.org.uk>
Message-ID: <E6608558-EF47-4BE0-819B-F1D30FF7FA2C@orestis.gr>


On Sep 15, 2009, at 1:44 AM, Michael Foord wrote:

> M.-A. Lemburg wrote:
>> Michael Foord wrote:
>>
>>> M.-A. Lemburg wrote:
>>>
>>>> [snip...]
>>>>
>>>>>> Replacing prefectly fine working code just for the fun of it,  
>>>>>> does
>>>>>> not count much as argument for evolving the stdlib.
>>>>>>
>>>>>>
>>>>> Unless you are attacking a complete strawman, which is unhelpful  
>>>>> and
>>>>> pointless so please refrain, can you point out who is suggesting
>>>>> replacing working code "just for the fun of it"?
>>>>>
>>>> Just have a look at the various arguments for adding argparse to  
>>>> the
>>>> stdlib with the intention of replacing optparse and getopt.
>>>>
>>>> On one hand you have this new API which is not compatible with  
>>>> optparse:
>>>>
>>>> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html#upgrading-optparse-code
>>>>
>>>>
>>>> On the other you have a rather short list of features that make
>>>> argparse different from optparse:
>>>>
>>>> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
>>>>
>>>> and the fact that argparse has been in the wild for 4.5 months.
>>>>
>>> Here's an email from 2007 asking when it will be in the standard  
>>> library:
>>>
>>> http://mail.python.org/pipermail/python-list/2007-January/ 
>>> 592646.html
>>>
>>
>> I was looking at this page:
>>
>> http://code.google.com/p/argparse/downloads/list
>>
>> It turned 1.0 in July and the first release (on Google Code) was on
>> April 1st this year.
>>
>> That doesn't say anything about the robustness of the code, but then:
>> how much traction can you get in those few months and how likely are
>> API changes to the code in a 1.1 or 2.0 release ?
>>
>>
> argparse has been available and in use in the community since at the  
> *latest* 2006. As for API stability you will have to ask Steven.

Please, the suggestion for adding argparse has nothing to do with  
Jesse's suggestion of splitting the stdlib and making sure the stdlib  
modules are top-notch. Let's keep the discussion on topic.

Orestis


From steven.bethard at gmail.com  Tue Sep 15 04:03:07 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 14 Sep 2009 19:03:07 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAEBEBE.1050105@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
Message-ID: <d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>

On Mon, Sep 14, 2009 at 3:07 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> and the fact that argparse has been in the wild for 4.5 months.

Just to set the record straight, argparse's first release was in October 2006:

http://mail.python.org/pipermail/python-announce-list/2006-October/005304.html

On Mon, Sep 14, 2009 at 3:37 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> It turned 1.0 in July and the first release (on Google Code) was on
> April 1st this year.

There were many releases on python-hosting.com before argparse moved
repositories. A quick Google for "argparse release" would have turned
these up for you.

> how much traction can you get in those few months and how likely are
> API changes to the code in a 1.1 or 2.0 release ?

This is now completely off topic, but the the core APIs (e.g.
add_argument, parse_args) haven't really changed since 0.1, and there
haven't really been any backwards incompatible changes since about 0.5
(January 2007), and they were preceded by deprecations as in most
Python modules.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From mal at egenix.com  Tue Sep 15 10:41:33 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 10:41:33 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
Message-ID: <4AAF533D.9030908@egenix.com>

Steven Bethard wrote:
> On Mon, Sep 14, 2009 at 3:07 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> and the fact that argparse has been in the wild for 4.5 months.
> 
> Just to set the record straight, argparse's first release was in October 2006:
> 
> http://mail.python.org/pipermail/python-announce-list/2006-October/005304.html

Thanks for clarifying this.

> On Mon, Sep 14, 2009 at 3:37 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> It turned 1.0 in July and the first release (on Google Code) was on
>> April 1st this year.
> 
> There were many releases on python-hosting.com before argparse moved
> repositories. A quick Google for "argparse release" would have turned
> these up for you.

I did Google for argparse and it did bring up python-hosting.com.
However, the python-hosting.com page redirected straight to
the Google Code page. I also looked on PyPI, but that only listed
the 1.0 release.

>> how much traction can you get in those few months and how likely are
>> API changes to the code in a 1.1 or 2.0 release ?
> 
> This is now completely off topic, but the the core APIs (e.g.
> add_argument, parse_args) haven't really changed since 0.1, and there
> haven't really been any backwards incompatible changes since about 0.5
> (January 2007), and they were preceded by deprecations as in most
> Python modules.

Thanks, that's good to know. So argparse has been stable for
2 years now.

BTW: This is fact is not off-topic at all - we are discussing
evolving the stdlib and Jesse brought up the idea of breaking up the
stdlib as answer to the discussions we had about getopt/optparse/
argparse.

This thread then shifted into a different direction:
that of breaking up the stdlib to make life easier for other
Python implementations than CPython.

And that discussion then led back to the original discussion
about how to evolve the stdlib, since it included deprecation
of modules, which then resulted in my summary of the situation
w/r to the parsing libs.

The two threads "should we try to add argparse?" and
"Breaking out the stdlib" both target the same basic
topic: How much change is needed and wanted in the stdlib ?

Part of the answer involves looking at the maturity of new
code vs. that of the existing code and weighing added
features vs. breaking backwards compatibility.

So to correct my summary: argparse has been stable for 2 years,
optparse for 7 years, getopt for >9 years (the last major
change was in 1998).

Anyway, Frank is going to write a PEP covering splitting the
stdlib into chunks in order to simplify reuse in Jython,
IronPython, etc. so that's a good result.

Just to avoid any misunderstandings: I don't have anything
against argparse or adding it to the stdlib.

I'm only concerned about the effects of deprecating modules
that are in wide-spread use, in this case getopt and optparse,
without providing a drop-in API compatible replacement.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From orestis at orestis.gr  Tue Sep 15 10:57:58 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Tue, 15 Sep 2009 11:57:58 +0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF533D.9030908@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
Message-ID: <E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>

> BTW: This is fact is not off-topic at all - we are discussing
> evolving the stdlib and Jesse brought up the idea of breaking up the
> stdlib as answer to the discussions we had about getopt/optparse/
> argparse.
>
> This thread then shifted into a different direction:
> that of breaking up the stdlib to make life easier for other
> Python implementations than CPython.

This was the driving force - the discussions about deprecating and  
evolving stdlib came later. But yes, we are talking about a lot of  
orthogonal issues
.
>
> And that discussion then led back to the original discussion
> about how to evolve the stdlib, since it included deprecation
> of modules, which then resulted in my summary of the situation
> w/r to the parsing libs.
>
> The two threads "should we try to add argparse?" and
> "Breaking out the stdlib" both target the same basic
> topic: How much change is needed and wanted in the stdlib ?
>
> Part of the answer involves looking at the maturity of new
> code vs. that of the existing code and weighing added
> features vs. breaking backwards compatibility.
>
> So to correct my summary: argparse has been stable for 2 years,
> optparse for 7 years, getopt for >9 years (the last major
> change was in 1998).
>
> Anyway, Frank is going to write a PEP covering splitting the
> stdlib into chunks in order to simplify reuse in Jython,
> IronPython, etc. so that's a good result.
>
> Just to avoid any misunderstandings: I don't have anything
> against argparse or adding it to the stdlib.
>
> I'm only concerned about the effects of deprecating modules
> that are in wide-spread use, in this case getopt and optparse,
> without providing a drop-in API compatible replacement.

A deprecated -> removed module can always exist in PyPI and manually  
installed (or perhaps they can live in their own special place stdlib- 
legacy). When a new version of Python comes out, people *do* have to  
spend some time testing their apps, so if something has moved from  
stdlib to stdlib-legacy, they can just install that in their site  
packages and go on pretending nothing happened.

For me, deprecating modules and adding new ones is more of a social  
move - it shows that the Python community cares, and it moves Python  
closer to the bleeding edge. I think it can be done without alienating  
legacy users.

>
> Thanks,
> -- 
> Marc-Andre Lemburg
> eGenix.com
>


From mal at egenix.com  Tue Sep 15 11:21:12 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 11:21:12 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
Message-ID: <4AAF5C88.2000009@egenix.com>

Orestis Markou wrote:
>> M.-A. Lemburg wrote:
>> Just to avoid any misunderstandings: I don't have anything
>> against argparse or adding it to the stdlib.
>>
>> I'm only concerned about the effects of deprecating modules
>> that are in wide-spread use, in this case getopt and optparse,
>> without providing a drop-in API compatible replacement.
> 
> A deprecated -> removed module can always exist in PyPI and manually
> installed (or perhaps they can live in their own special place
> stdlib-legacy). When a new version of Python comes out, people *do* have
> to spend some time testing their apps, so if something has moved from
> stdlib to stdlib-legacy, they can just install that in their site
> packages and go on pretending nothing happened.

True, but the modules in question mainly target small scripts,
not big applications. And those typically use the default Python
version found on the system via "#!/usr/bin/env python".

However, perhaps we can reach some middle-ground: we split out
LTS legacy modules into a stdlib-legacy or stdlib-compat
collection, keep applying patches to them as needed and
only apply the deprecation mechanism to the documentation
of these modules (the deprecation warnings tend to upset
consumers of the script's output and thus cause breakage on
their own).

That way, new code will likely start to be written against
the new modules, while old tested and working code can continue
to run with the old tested and working modules.

> For me, deprecating modules and adding new ones is more of a social move
> - it shows that the Python community cares, and it moves Python closer
> to the bleeding edge. I think it can be done without alienating legacy
> users.

I think points to one of the differences we are seeing in this
discussion:

One group of people wants bleeding edge, the other is more
interested in a stable code base.

IMHO, it's fair to ask the first group to put in a little more
effort and download whatever they want from PyPI, than putting
the burden on the second group.

After all, people who use bleeding edge code, need to be aware
of the fact that things can break, so it's better to make that
choice explicit, than have it made implicit for everyone.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From lac at openend.se  Tue Sep 15 12:29:17 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Sep 2009 12:29:17 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: Message from "M.-A. Lemburg" <mal@egenix.com> 
	of "Tue, 15 Sep 2009 00:37:16 +0200." <4AAEC59C.7060605@egenix.com> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com> 
Message-ID: <200909151029.n8FATHmq028625@theraft.openend.se>


When I said that I wanted things in the stdlibrary to 'never change
until hell freezes over' I didn't mean that the std library shouldn't
evolve by adding new modules.  What I meant was that the old,
perfectly good working modules who haven't needed changes for years
and years and years don't get summarily yanked out because people
'don't want to maintain them any more'.  There is a case to be made
that code that refers to hardware that hasn't been used for years and
years can be removed, and perhaps other things which are similarily
unused at this point in time can be removed, but nobody is making that
claim about getopt and optparse.

Indeed, right now, if you write code that parses options you are
most likely doing one of 4 things:

a) using getopt
b) using optparse
c) using some third party module for parsing that is currently under
   development
d) writing your own option parser in python

Now, for years I have been telling people that d) was by far the poorest
way of doing things.  Don't reinvent the wheel, and all that.  But on
the day that you yank getopt and optparse out of the library, you will
have made this terrible advice to give my poor hospital clients.  And
you will have made all the time I spent ripping out hand-made option
parsers and replacing them with getopt instead wasted time that I billed
my clients for.  Instead of giving them a robust solution that will
require little or no maintainance, I have given them a zillion scripts,
_all of which will suddenly break one day_.  And since the pure python
parsers that I never got around to converting will probably still be
working, the people who did d) will get the last laugh.

So -- real use case time -- the hospitals where I have done a lot of
work have some really weird equipment.  And using them costs real
money, uses up real lab supplies, and conceivably can ruin a sample
that you will find it inconvenient or impossible to replace.

There are all sorts of weird dependencies among the various options
you can use to operate the devices.  If you have specified option K,
you may not also specify option M or N, and if you have specified
option L, you must also specify option Q, or option R and option S.

Thus the whole exercise of writing a script to use the equipment
becomes as matter of validating the options you selected are complete
and non-contradictory, and then going out and exercising the hardware.

You can build a validating option parser by hand (option d) or using
getopt (option a).  You will find it difficult to the point of near
impossibility to build one using optparse, because optparse specifically
rejects the notion of 'required options' which is the meat-and-potoatoes
part of this app.  I found this out when optparse went into the standard
library, and was touted at being superior to getopt.

I tried it, and tried to subclass it, and talked to its author about
whether it could be changed to support my use case, and even volunteered
to write code to change it, but was firmly told that optparse worked
the way it did, on purpose, in order to prevent the sort of use that
I wanted to make of it, and that patches to make that possible were
entirely unwelcome.

At which point I go back to using getopt.  There is no particular hard
feelings about this --  I figure that the people who want to use
optparse can use it, and getopt is here for the people who won't, or
can't.  But when later in time people suggested getting rid of getopt
because 'it was old' and 'optparse was better' I realise, a long time
when it was too late to do anything, that I should have been spearheading
the 'I don't want optparse in the standard library' effort, on the
grounds that it didn't support 'required arguments'.  That fight might
have become nasty.  Far better to allow multiple ways of doing things.

As far as I can tell most software packages go through three stages:

Stage A:  "rapid development"  Expect changes to the API.  New releases
          can and will break all the existing code out there.  There are
          lots of bugs, but it is hard to tell a bug from a thing that
          is incomplete sometimes.

Stage B:  "things have (mostly) settled down"  Major releases  may break 
          existing code.  Minor ones will not. The bugs are being shaken
          out.  Development slows.

	  There are two major variants in Stage B code.

	  B-1: has an active maintainer(s)
	  B-2: isn't being maintained any more

Stage C:  "until hell freezes over"  The author of the package has determined
          that he has solved his problem as well as he ever cares to.  He
          considers his package 'finished', 'done', or 'complete'. No more
	  bugs are being found.  No more development is planned.  

	  It also has two variants.

	  C-1:  The maintainer will fix major bugs should you ever find
                any.
          C-2: The maintainer won't.  He might even be dead.

Some software packages never reach stage C.  Forgetting those that are
abandoned by their authors, for the moment while they are still in
stage B, we are still left with many that deal with the rapidly
changing world in such a way that their development can never be said
to be done.

Now software in Stage A doesn't need maintenance so much as development
in the first place.  And software in stage B requires a lot of
maintainance.  But most software in stage C requires little or no
maintenance, precisely because it is unchanging.  So, if you decide to
change how Exceptions are inherited, for instance, you may break a
whole lot of Stage C: code, but fixing them is part of the general
problem of 'fixing exceptions everywhere' not part of 'fixing getopt'.

Now I think that there is some confusion here between packages who
are in Stage C:  and packages who are in Stage B-2:  around here.
The B-2 packages are probably the core developers greatest headache.
But I don't see the C packages as being troublesome at all.  If you
don't like them, it isn't because of the work that maintaining is
costing you.  It may be that hatred for the B-2's has become a general
hatred of all packages with no maintainers, which is an understandable
mistake.  But from reading this list I get the distinct impression that
some people just hate C:s _precisely because they are old and unchanging_,
and would continue to hate them for that reason even if I was in some
way able to guarantee that they would never need any maintenance ever
again.  These people are condemning the packages I love best for the
reason they are the packages I love best.  And that is the attitude
I would like to change.

It boils down to a matter of trust.  My customers trust me to not give
them ticking time bombs that will all stop working one day, and I trust
you not to go about gratuitously removing perfectly working code that
is quietly sitting there, not needing any changes, and not bothering
anybody.  When you break that contract with me, my customers suffer,
I suffer, and the people who said 'You shouldn't have coded it in
Python in the first place, but picked a mature language like Java'
are completely vindicated.  Deciding that we want to be more flexible
than Java and actually retire some old packages doesn't mean that
we should condone retiring old packages because they are old and
haven't changed in a long while.  There are a great number of packages
that should be kept for this very reason, and I think that getopt is
a great example of one of them.

So, if we are going to reorganise the standard library, even if only
conceptually, I'd like to toss in my suggestions for improvement.

I'd like to make it possible to tag modules.

One good set of tags would be 'CPython', 'Jython', 'PyPy', etc.
When you want to build your own version of the standard library you
just get the modules that are tagged for you.

Another good tag is 'who is maintaining this thing'.  Which could be
'no one'.  Along with this information I would like to know how long
this person has promised to maintain the thing, and at what point in
time (if ever) does he expect this module to become

And then I would like something along my A B and C scheme, though of
course the number of categories is likely to change and they should have
meaningful names (which is something I am particularly poor at coming
up with).  

This would mean as a developer I could take a look at the standard
library and find out:

getopt:  maintained by (nobody).  Entered the standard library in 1990.
         current status (dead as a doornail).  Expected to reach the
         status (dead as a doornail) by (already reached).  
Alternatives:
         optparse, argparse, optfunct* (not in the standard library)

argparse: maintained by (Steven Bethard)  Entered the standard library
          in 2009.  current status (a moderate amount of change is 
          happening).  Expected to reach the status (dead as a doornail)
          by (2012).  Has never reached the status 'dead as a doornail'.
	  Alternatives: optparse, getopt,  optfunct* (not in the standard 
          library)

or maybe we will see

argparse: maintained by (Steven Bethard)  Entered the standard library
          in (2009).  current status (a moderate amount of change is
          happening).  Expected to reach that status (dead as a doornail)
          by (never, Steve expects that this module will be under
          continuous development through its lifetime).  Has never reached
          the status 'dead as a doornail'. Alternatives: optparse, getopt, 
          optfunct* (not in the standard library)

we can also get things like:

unittest: maintained by (Michael Foord).  Entered the standard library in
          1999-or-whenever-the-real-date-was. Current status (a moderate 
          amount of change is happening).  Expected to reach the status 
          (dead as a doornail) by (2012). This module reached the status
          of 'dead as a doornail' in 2002 but was subsequently revived
          by Michael Foord in 2008.  Alternatives py.test*  (not in the 
          standard library) nose*  (not in the standard library)

elementtree: maintained by (noone) Entered the standard library in
          2005-or-whenever-the-real-date-was.  Current status (no changes
          are happening).  Expected to reach the status of 'dead as a
          doornail' by (unknown.  Fredrik Lundh, the original author
          of the module is no longer maintaining it.  It is incomplete,
          and a new maintainer is actively sought.)  No alternatives.

or maybe we get

elementtree: maintained by (noone) Entered the standard library in
          2005-or-whenever-the-real-date-was.  Current status (no changes
          are happening).  Expected to reach the status of 'dead as a
          doornail' by (unknown.  Fredrik Lundh, the original author
          of the module is no longer maintaining it.  It is quite finished,
          and the proposal has been made to call it dead as a doornail
          in 2010.) No alternatives.

I would find this very useful, as would anybody who wants to use dead-as-a
doornail whenever possible.  And I think that it would give a certain
breathing space for python core developers -- breaking things when the
library involved was tagged as 'under moderate development' is a much
less heinous sin than breaking the ones that are 'dead as a doornail'.
People like Michael would understand that when they re-open something
like unittest, they are taking on a responsibility which comes with a
much larger burden of 'don't break existing code'.  After all, if I
had wanted new unittesting features, I would be using py.test or nose.
When I use unittest at all, these days, you can take it a a very strong
indication that either I, or my customers, are vastly more interested
in stability than new features, and will be insensed if something breaks
in the name of 'this is better'.

So what do you think of this proposal?

Laura



From solipsis at pitrou.net  Tue Sep 15 13:07:44 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 13:07:44 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
Message-ID: <1253012864.5646.25.camel@localhost>

Le mardi 15 septembre 2009 ? 11:57 +0300, Orestis Markou a ?crit :
> 
> When a new version of Python comes out, people *do* have to  
> spend some time testing their apps, so if something has moved from  
> stdlib to stdlib-legacy, they can just install that in their site  
> packages and go on pretending nothing happened.

Who is "people" in that sentence?
It can be developers. But some applications or scripts have been
developed years ago, by someone who isn't there to maintain them
anymore.
It can be users. But it is unreasonable (and quite rude) to expect users
to fix compatibility problems by themselves.
(do you think the average user knows what a "site packages" is?)

There is the ideal world where every Python program (script,
application, library, etc.) has a dedicated and active maintainer and
regular releases to keep up-to-date with the state of the Python
ecosystem. It is also the visible part of the iceberg (PyPI, Linux
distros etc.), which is why some people assume it accurately describes
reality.

There is the real world where many programs are one-off solutions to
specific problems, coded years ago, not maintained anymore because the
coder has left for another place, and the users don't know Python.




From mal at egenix.com  Tue Sep 15 13:22:01 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 13:22:01 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151029.n8FATHmq028625@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
Message-ID: <4AAF78D9.20107@egenix.com>

Laura Creighton wrote:
> So what do you think of this proposal?

Good write-up and very much to the point.

[Executive Summary:
Code that hardly needs any changes, because it does what it's meant
to do, is good code, not bad code. And it causes only minimal
maintenance effort, so it's actually something core developer should
welcome rather than fight against.]

I'd only change the tag "dead-as-a-doornail" to "complete, proven and
stable". Sounds more accurate.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From orestis at orestis.gr  Tue Sep 15 13:23:27 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Tue, 15 Sep 2009 14:23:27 +0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253012864.5646.25.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<1253012864.5646.25.camel@localhost>
Message-ID: <021143DF-A979-44DA-9D45-296756F64749@orestis.gr>

On Sep 15, 2009, at 2:07 PM, Antoine Pitrou wrote:

> Le mardi 15 septembre 2009 ? 11:57 +0300, Orestis Markou a ?crit :
>>
>> When a new version of Python comes out, people *do* have to
>> spend some time testing their apps, so if something has moved from
>> stdlib to stdlib-legacy, they can just install that in their site
>> packages and go on pretending nothing happened.
>
> Who is "people" in that sentence?
> It can be developers. But some applications or scripts have been
> developed years ago, by someone who isn't there to maintain them
> anymore.
> It can be users. But it is unreasonable (and quite rude) to expect  
> users
> to fix compatibility problems by themselves.
> (do you think the average user knows what a "site packages" is?)
>
> There is the ideal world where every Python program (script,
> application, library, etc.) has a dedicated and active maintainer and
> regular releases to keep up-to-date with the state of the Python
> ecosystem. It is also the visible part of the iceberg (PyPI, Linux
> distros etc.), which is why some people assume it accurately describes
> reality.
>
> There is the real world where many programs are one-off solutions to
> specific problems, coded years ago, not maintained anymore because the
> coder has left for another place, and the users don't know Python.

People who don't know what Python or site-packages is, should not  
receive Python updates anyway (save security updates).  The same  
argument can be made against Python 3 (and it probably has been). Of  
course, I'm not saying that there won't be a lot of cases where  
breakage will happen. This can be mitigated by better packaging (ie,  
making standalone packages). And I'm not focusing on getopt/optparser  
here - there are loads of other modules in stdlib that are dinosaurs  
(mostly platform dependent ones) that could be hidden away neatly.

> <tagging proposal>
> So what do you think of this proposal?
> Laura

Seeing that this is an expansion of the stdlib-legacy idea, +1. I  
think this is information that's excellent to have, and should be  
extended to PyPI as well.

>
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig


From michael at voidspace.org.uk  Tue Sep 15 13:37:48 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 12:37:48 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
Message-ID: <4AAF7C8C.1060306@voidspace.org.uk>

Orestis Markou wrote:
> [snip...]
>>
>> And that discussion then led back to the original discussion
>> about how to evolve the stdlib, since it included deprecation
>> of modules, which then resulted in my summary of the situation
>> w/r to the parsing libs.
>>
>> The two threads "should we try to add argparse?" and
>> "Breaking out the stdlib" both target the same basic
>> topic: How much change is needed and wanted in the stdlib ?
>>
>> Part of the answer involves looking at the maturity of new
>> code vs. that of the existing code and weighing added
>> features vs. breaking backwards compatibility.
>>
>> So to correct my summary: argparse has been stable for 2 years,
>> optparse for 7 years, getopt for >9 years (the last major
>> change was in 1998).
>>
>> Anyway, Frank is going to write a PEP covering splitting the
>> stdlib into chunks in order to simplify reuse in Jython,
>> IronPython, etc. so that's a good result.
>>
>> Just to avoid any misunderstandings: I don't have anything
>> against argparse or adding it to the stdlib.
>>
>> I'm only concerned about the effects of deprecating modules
>> that are in wide-spread use, in this case getopt and optparse,
>> without providing a drop-in API compatible replacement.
>
> A deprecated -> removed module can always exist in PyPI and manually 
> installed (or perhaps they can live in their own special place 
> stdlib-legacy). When a new version of Python comes out, people *do* 
> have to spend some time testing their apps, so if something has moved 
> from stdlib to stdlib-legacy, they can just install that in their site 
> packages and go on pretending nothing happened.
>
> For me, deprecating modules and adding new ones is more of a social 
> move - it shows that the Python community cares, and it moves Python 
> closer to the bleeding edge. I think it can be done without alienating 
> legacy users.
>

I don't think it is a goal for the standard library to be bleeding edge. 
We definitely allow other libraries to be bleeding edge and then steal 
them for the standard library once they have proven widely used, stable 
and best-of-breed.

Part of the problem is that there are a lot of 'legacy' modules in the 
standard library that don't meet these criteria by a long way. Removing 
those is one issue that Jesse is about to embark on addressing with a PEP.

I also disagree with those who think that having a stable standard 
library means having a genuinely dead standard library. If I have time 
I'll compose a response to that - but I don't think that is *possible* 
(unless we also stop improving Python the language) or desirable 
socially or technologically.

Michael

>>
>> Thanks,
>> -- 
>> Marc-Andre Lemburg
>> eGenix.com
>>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig


-- 
http://www.ironpythoninaction.com/


From mal at egenix.com  Tue Sep 15 13:46:24 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 13:46:24 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <021143DF-A979-44DA-9D45-296756F64749@orestis.gr>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	<4AAF533D.9030908@egenix.com>	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>	<1253012864.5646.25.camel@localhost>
	<021143DF-A979-44DA-9D45-296756F64749@orestis.gr>
Message-ID: <4AAF7E90.5050402@egenix.com>

Orestis Markou wrote:
> On Sep 15, 2009, at 2:07 PM, Antoine Pitrou wrote:
>> <tagging proposal>
>> So what do you think of this proposal?
>> Laura
> 
> Seeing that this is an expansion of the stdlib-legacy idea, +1. I think
> this is information that's excellent to have, and should be extended to
> PyPI as well.

Uhm, we already have that: it's called "Development Status" and
part of the software package trove categories:

http://pypi.python.org/pypi?:action=browse

"Mature" would e.g. fit the getopt and optparse modules.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From michael at voidspace.org.uk  Tue Sep 15 13:53:38 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 12:53:38 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF5C88.2000009@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	<4AAF533D.9030908@egenix.com>	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com>
Message-ID: <4AAF8042.8000008@voidspace.org.uk>

M.-A. Lemburg wrote:
> Orestis Markou wrote:
>   
>>> M.-A. Lemburg wrote:
>>> Just to avoid any misunderstandings: I don't have anything
>>> against argparse or adding it to the stdlib.
>>>
>>> I'm only concerned about the effects of deprecating modules
>>> that are in wide-spread use, in this case getopt and optparse,
>>> without providing a drop-in API compatible replacement.
>>>       
>> A deprecated -> removed module can always exist in PyPI and manually
>> installed (or perhaps they can live in their own special place
>> stdlib-legacy). When a new version of Python comes out, people *do* have
>> to spend some time testing their apps, so if something has moved from
>> stdlib to stdlib-legacy, they can just install that in their site
>> packages and go on pretending nothing happened.
>>     
>
> True, but the modules in question mainly target small scripts,
> not big applications. And those typically use the default Python
> version found on the system via "#!/usr/bin/env python".
>
> However, perhaps we can reach some middle-ground: we split out
> LTS legacy modules into a stdlib-legacy or stdlib-compat
> collection, keep applying patches to them as needed and
> only apply the deprecation mechanism to the documentation
> of these modules (the deprecation warnings tend to upset
> consumers of the script's output and thus cause breakage on
> their own).
>
>   
Applying patches presumes we have tests (or bug reports). Maintaining 
dead code still has a cost. Of course whilst a module is in the standard 
library we should be committed to this (probably more than we are but 
then we're a volunteer based project) - but this is one of the reasons 
that we *won't* guarantee to keep any arbitrary module in the standard 
library forever without a maintainer. It is still an orthogonal point to 
any individual question of module removal however.


> That way, new code will likely start to be written against
> the new modules, while old tested and working code can continue
> to run with the old tested and working modules.
>
>   

This is an interesting idea, but hard to implement in a meaningful way. 
(i.e. if existing scripts using legacy modules are to continue working 
with new versions of Python then the legacy modules will have to be 
installed and on the path by default - and de-facto the situation is 
unchanged.)

I do agree that deprecating the documentation long before removing any 
modules is desirable - and probably achieves the same result as actually 
partitioning modules. I wouldn't be against a legacy package though as a 
'physical' partitioning has some benefits on the path to 'complete 
deprecation' (whatever that may mean).

>> For me, deprecating modules and adding new ones is more of a social move
>> - it shows that the Python community cares, and it moves Python closer
>> to the bleeding edge. I think it can be done without alienating legacy
>> users.
>>     
>
> I think points to one of the differences we are seeing in this
> discussion:
>
> One group of people wants bleeding edge, the other is more
> interested in a stable code base.
>
>   
I don't expect the standard library to get close to 'the-bleeding-edge'.

There seem to be three positions:

1) Virtually no changes or improvements to the standard library at all - 
nothing beyond maintaining compatibility with language changes. (Laura)
2) New modules are acceptable but old modules should remain forever. 
(Antoine)
3) New modules are acceptable and eventual removal of old modules is 
desirable. (Brett, myself, Jesse and Frank)

Marc-Andre seems to fall somewhere between 1 and 2 and Orestis wants the 
bleeding edge. (Sorry for these caricatures but it seems approximately 
right.)

I'd still like to write a longer piece on why I think 1) isn't possible 
and 3) is preferable to 2)  - but the basic points have all been covered 
in this thread anyway.

> IMHO, it's fair to ask the first group to put in a little more
> effort and download whatever they want from PyPI, than putting
> the burden on the second group.
>
> After all, people who use bleeding edge code, need to be aware
> of the fact that things can break, so it's better to make that
> choice explicit, than have it made implicit for everyone.
>
>   
It also seems fair to say that if people want stability to the point of 
no change at all then they should stick to a version of Python that 
meets their requirements. Language changes and even bugfixes introduce 
incompatibilities that can't and won't be avoided so there simply *is 
not* a guarantee that old code will continue to work forever.

Another idea is to have a standard library 'bleeding-edge' package, 
containing modules that we expect to move into the standard library in 
the next version of Python. This could be an additional install (with 
convenient installers for Mac OS X and Windows).

This slows down the introduction of new modules - whilst bringing them 
into the standard library but 'on probation'. We still insist on the PEP 
process for bringing in new modules and Steven will be writing a PEP for 
argparse (which will address the specific question of deprecation of 
optparse / getopt) so I don't know if a bleeding-edge package is worth 
the effort.

Michael

-- 
http://www.ironpythoninaction.com/


From michael at voidspace.org.uk  Tue Sep 15 14:03:15 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 13:03:15 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253012864.5646.25.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	<4AAF533D.9030908@egenix.com>	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<1253012864.5646.25.camel@localhost>
Message-ID: <4AAF8283.7090304@voidspace.org.uk>

Antoine Pitrou wrote:
> Le mardi 15 septembre 2009 ? 11:57 +0300, Orestis Markou a ?crit :
>   
>> When a new version of Python comes out, people *do* have to  
>> spend some time testing their apps, so if something has moved from  
>> stdlib to stdlib-legacy, they can just install that in their site  
>> packages and go on pretending nothing happened.
>>     
>
> Who is "people" in that sentence?
> It can be developers. But some applications or scripts have been
> developed years ago, by someone who isn't there to maintain them
> anymore.
> It can be users. But it is unreasonable (and quite rude) to expect users
> to fix compatibility problems by themselves.
> (do you think the average user knows what a "site packages" is?)
>
>   
To be fair, anyone relying on legacy Python scripts / applications that 
they have no desire or ability to maintain themselves will *soon* have 
to stick to the last version of Python 2.X anyway.

Within the foreseeable future vendors will have switched to Python 3 by 
default and Python 2 will finally be genuinely stable.

> There is the ideal world where every Python program (script,
> application, library, etc.) has a dedicated and active maintainer and
> regular releases to keep up-to-date with the state of the Python
> ecosystem. It is also the visible part of the iceberg (PyPI, Linux
> distros etc.), which is why some people assume it accurately describes
> reality.
>
> There is the real world where many programs are one-off solutions to
> specific problems, coded years ago, not maintained anymore because the
> coder has left for another place, and the users don't know Python.
>
>
>   
And eventually those programs stop working with the latest and greatest 
version of Python. That's the reality of bit-rot.

I read an article (on Planet Python I think) about code-examples from a 
C/C++ book that no longer compile with modern compilers (neither VS nor 
gcc). Languages change and Python has been better than others in this 
regard. The 2 to 3 transition will be very harsh from this point of view 
however.

Up until now we have been reluctant to remove modules, although several 
deprecated ones were removed in Python 3. My homebrew blogging system, 
which I haven't touched the code of in several years, now has several 
deprecation warnings from the standard library when I run it - because 
it uses the md5 and mimewriter modules that are now deprecated.

The question of removal is an interesting one. Dragging around legacy 
code that still needs to be maintained is a burden. We'll see what 
smattering of consensus we can reach when there is a PEP.

Michael


> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/


From solipsis at pitrou.net  Tue Sep 15 14:13:18 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 14:13:18 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF8042.8000008@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com>  <4AAF8042.8000008@voidspace.org.uk>
Message-ID: <1253016798.5646.40.camel@localhost>

Le mardi 15 septembre 2009 ? 12:53 +0100, Michael Foord a ?crit :
> 2) New modules are acceptable but old modules should remain forever. 
> (Antoine)

As far as I'm concerned, you could change this to ? New modules are
acceptable but old modules should remain as long as they are popular,
and aren't plagued by severe issues ?.

cheers

Antoine.



From michael at voidspace.org.uk  Tue Sep 15 14:12:52 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 13:12:52 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF78D9.20107@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com>
Message-ID: <4AAF84C4.20204@voidspace.org.uk>

M.-A. Lemburg wrote:
> Laura Creighton wrote:
>   
>> So what do you think of this proposal?
>>     
>
> Good write-up and very much to the point.
>
> [Executive Summary:
> Code that hardly needs any changes, because it does what it's meant
> to do, is good code, not bad code. And it causes only minimal
> maintenance effort, so it's actually something core developer should
> welcome rather than fight against.]
>
>   
Right - but part of the specific problem with optparse is that in many 
situations it does a very inadequate job (i.e. it "it does what it is 
meant to do" but not what many people "need it to do") and is designed 
in such a way that *required* functionality can't be added in a 
backwards compatible way.

That is not "good code" (by my reckoning).

Michael


> I'd only change the tag "dead-as-a-doornail" to "complete, proven and
> stable". Sounds more accurate.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Sep 15 2009)
>   
>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>>>         
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
>   


-- 
http://www.ironpythoninaction.com/


From rdmurray at bitdance.com  Tue Sep 15 14:20:03 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 08:20:03 -0400 (EDT)
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF8042.8000008@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org> <4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost> <4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
Message-ID: <Pine.LNX.4.64.0909150806160.6303@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 12:53, Michael Foord wrote:
> There seem to be three positions:
>
> 1) Virtually no changes or improvements to the standard library at all - 
> nothing beyond maintaining compatibility with language changes. (Laura)

I think you misrepresent Laura's opinion.  She wants modules that
are mature (stable) to remain in the library, but I did not hear her
object to the addition of improvements.  (She said perhaps she should
have objected to optparse, but that was a hypothetical conditioned on
getopt being removed...if getopt remains (as she expected it would)
she had/has no objection to optparse (or, presumably, argparse)).

> 2) New modules are acceptable but old modules should remain forever. 
> (Antoine)
> 3) New modules are acceptable and eventual removal of old modules is 
> desirable. (Brett, myself, Jesse and Frank)

Laura's objection was to this label of "old modules", as if all modules
beyond a certain age were automatically bad and should be removed.
There is an important distinction to be made between "old, broken,
and not maintained", and "old, mature, and functional".

> Marc-Andre seems to fall somewhere between 1 and 2 and Orestis wants the 
> bleeding edge. (Sorry for these caricatures but it seems approximately 
> right.)
>
> I'd still like to write a longer piece on why I think 1) isn't possible and 
> 3) is preferable to 2)  - but the basic points have all been covered in this 
> thread anyway.

Is anyone actually advocating (1)?  I doubt it.

I think Laura's post was excellent and is worth a careful (re)reading to
understand her points.

--David

From dstanek at dstanek.com  Tue Sep 15 14:28:12 2009
From: dstanek at dstanek.com (David Stanek)
Date: Tue, 15 Sep 2009 08:28:12 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF84C4.20204@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com> <4AAF84C4.20204@voidspace.org.uk>
Message-ID: <de32cc030909150528t686ea2f9hb26576bcea4bc123@mail.gmail.com>

Sorry if I am a bit off base, but I have only been browsing this thread...

I think that at some point the really old stuff that has a replacement
should get removed. Why not provide a LegacyStdlib egg on Pypi? This
could create a compatibility layer and help those that do not want to
change their code .

On 9/15/09, Michael Foord <michael at voidspace.org.uk> wrote:
> M.-A. Lemburg wrote:
>> Laura Creighton wrote:
>>
>>> So what do you think of this proposal?
>>>
>>
>> Good write-up and very much to the point.
>>
>> [Executive Summary:
>> Code that hardly needs any changes, because it does what it's meant
>> to do, is good code, not bad code. And it causes only minimal
>> maintenance effort, so it's actually something core developer should
>> welcome rather than fight against.]
>>
>>
> Right - but part of the specific problem with optparse is that in many
> situations it does a very inadequate job (i.e. it "it does what it is
> meant to do" but not what many people "need it to do") and is designed
> in such a way that *required* functionality can't be added in a
> backwards compatible way.
>
> That is not "good code" (by my reckoning).
>
> Michael
>
>
>> I'd only change the tag "dead-as-a-doornail" to "complete, proven and
>> stable". Sounds more accurate.
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>
>>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>>>>
>> ________________________________________________________________________
>>
>> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>>
>>
>>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>            Registered at Amtsgericht Duesseldorf: HRB 46611
>>                http://www.egenix.com/company/contact/
>>
>
>
> --
> http://www.ironpythoninaction.com/
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>

-- 
Sent from my mobile device

David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek

From orestis at orestis.gr  Tue Sep 15 14:28:38 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Tue, 15 Sep 2009 15:28:38 +0300
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF8042.8000008@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	<4AAF533D.9030908@egenix.com>	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
Message-ID: <1E9E6AF9-2813-47B5-8BEC-2809BD520EF2@orestis.gr>


On Sep 15, 2009, at 2:53 PM, Michael Foord wrote:
> There seem to be three positions:
>
> 1) Virtually no changes or improvements to the standard library at  
> all - nothing beyond maintaining compatibility with language  
> changes. (Laura)
> 2) New modules are acceptable but old modules should remain forever.  
> (Antoine)
> 3) New modules are acceptable and eventual removal of old modules is  
> desirable. (Brett, myself, Jesse and Frank)
>
> Marc-Andre seems to fall somewhere between 1 and 2 and Orestis wants  
> the bleeding edge. (Sorry for these caricatures but it seems  
> approximately right.)

To set the record straight:

I'm all for clearly marking 'old' modules in the documentation. When I  
started with Python, I remember not knowing which module to use from  
the selection in stdlib. I'm also all for pointing to alternatives on  
PyPI.

I'm also fine with removing truly obsolete modules (macpath, archaic  
audio and image support). I understand the need of keeping those  
around for people that truly need them, however I think they belong  
somewhere else, not in the stdlib.

As for adding new modules, I don't want the bleeding edge - let de  
facto standards emerge, then they can be blessed and included. I'm -1  
on creating a stdlib-bleeding. A pointer in the documentation to a  
module that is considered best of breed for a specific job should be  
adequate.

Sorry if I haven't expressed this clearly before.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090915/fe34835e/attachment.htm>

From jnoller at gmail.com  Tue Sep 15 14:42:12 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 08:42:12 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253016798.5646.40.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
Message-ID: <4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>

On Tue, Sep 15, 2009 at 8:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 12:53 +0100, Michael Foord a ?crit :
>> 2) New modules are acceptable but old modules should remain forever.
>> (Antoine)
>
> As far as I'm concerned, you could change this to ? New modules are
> acceptable but old modules should remain as long as they are popular,
> and aren't plagued by severe issues ?.
>
> cheers
>
> Antoine.

Define "popular" - it's purely subjective.

From solipsis at pitrou.net  Tue Sep 15 14:48:39 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 14:48:39 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
	<4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
Message-ID: <1253018919.5646.60.camel@localhost>

Le mardi 15 septembre 2009 ? 08:42 -0400, Jesse Noller a ?crit :
> On Tue, Sep 15, 2009 at 8:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Le mardi 15 septembre 2009 ? 12:53 +0100, Michael Foord a ?crit :
> >> 2) New modules are acceptable but old modules should remain forever.
> >> (Antoine)
> >
> > As far as I'm concerned, you could change this to ? New modules are
> > acceptable but old modules should remain as long as they are popular,
> > and aren't plagued by severe issues ?.
> >
> > cheers
> >
> > Antoine.
> 
> Define "popular" - it's purely subjective.

Used by many people.



From jnoller at gmail.com  Tue Sep 15 14:52:50 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 08:52:50 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253018919.5646.60.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
	<4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
	<1253018919.5646.60.camel@localhost>
Message-ID: <4222a8490909150552p687f68c2o783a933caa4f43b3@mail.gmail.com>

On Tue, Sep 15, 2009 at 8:48 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 08:42 -0400, Jesse Noller a ?crit :
>> On Tue, Sep 15, 2009 at 8:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > Le mardi 15 septembre 2009 ? 12:53 +0100, Michael Foord a ?crit :
>> >> 2) New modules are acceptable but old modules should remain forever.
>> >> (Antoine)
>> >
>> > As far as I'm concerned, you could change this to ? New modules are
>> > acceptable but old modules should remain as long as they are popular,
>> > and aren't plagued by severe issues ?.
>> >
>> > cheers
>> >
>> > Antoine.
>>
>> Define "popular" - it's purely subjective.
>
> Used by many people.
>

How do you define "many people"? What if it's used by MEGACORP
extensively, but by barely any modules on the cheeseshop?

From michael at voidspace.org.uk  Tue Sep 15 14:55:04 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 13:55:04 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <Pine.LNX.4.64.0909150806160.6303@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<Pine.LNX.4.64.0909150806160.6303@kimball.webabinitio.net>
Message-ID: <4AAF8EA8.4030807@voidspace.org.uk>

R. David Murray wrote:
> On Tue, 15 Sep 2009 at 12:53, Michael Foord wrote:
>> There seem to be three positions:
>>
>> 1) Virtually no changes or improvements to the standard library at 
>> all - nothing beyond maintaining compatibility with language changes. 
>> (Laura)
>
> I think you misrepresent Laura's opinion.  She wants modules that
> are mature (stable) to remain in the library, but I did not hear her
> object to the addition of improvements.  (She said perhaps she should
> have objected to optparse, but that was a hypothetical conditioned on
> getopt being removed...if getopt remains (as she expected it would)
> she had/has no objection to optparse (or, presumably, argparse)).
>
It's a caricature - but hard to express in a single line. ;-)

Her explicit preference however is for a standard library that is 
"dead-as-a doornail whenever possible" and that I think is something 
that is actively against what many of the core developers are working for.

>> 2) New modules are acceptable but old modules should remain forever. 
>> (Antoine)
>> 3) New modules are acceptable and eventual removal of old modules is 
>> desirable. (Brett, myself, Jesse and Frank)
>
> Laura's objection was to this label of "old modules", as if all modules
> beyond a certain age were automatically bad and should be removed.

That's a complete mis-characterisation. No one has said modules should 
be removed just because they are old. Modules that are unable to evolve 
to meet requirements because of API design are a problem though. Merely 
because a library meets *some* use cases doesn't make it *good*.

There is also the related issue of what to do about duplicate 
functionality - particularly in the case of argparse and optparse where 
argparse will completely 'replace' the functionality of optparse and 
no-one advocating for keeping it is also willing to maintain it.

> There is an important distinction to be made between "old, broken,
> and not maintained", and "old, mature, and functional".
>
>> Marc-Andre seems to fall somewhere between 1 and 2 and Orestis wants 
>> the bleeding edge. (Sorry for these caricatures but it seems 
>> approximately right.)
>>
>> I'd still like to write a longer piece on why I think 1) isn't 
>> possible and 3) is preferable to 2)  - but the basic points have all 
>> been covered in this thread anyway.
>
> Is anyone actually advocating (1)?  I doubt it.
>
> I think Laura's post was excellent and is worth a careful (re)reading to
> understand her points.
I think I understand her points and her motivation (and have sympathy 
for them), I just don't think they are really feasible (at face value) 
for a developing language. I think if you have a hard requirement for 
absolute stability then due to language changes alone you will have to 
stick to a specific version of Python.

Michael


>
> --David


-- 
http://www.ironpythoninaction.com/


From solipsis at pitrou.net  Tue Sep 15 15:06:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 15:06:36 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909150552p687f68c2o783a933caa4f43b3@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
	<4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
	<1253018919.5646.60.camel@localhost>
	<4222a8490909150552p687f68c2o783a933caa4f43b3@mail.gmail.com>
Message-ID: <1253019996.5646.68.camel@localhost>

Le mardi 15 septembre 2009 ? 08:52 -0400, Jesse Noller a ?crit :
> 
> How do you define "many people"? What if it's used by MEGACORP
> extensively, but by barely any modules on the cheeseshop?

Of course it's a matter of perception. But if you want some numerically
measurable input for everything, this discussion won't progress by a
single bit. And I don't think your criteria will hold either, because
things like "better", "up-to-date", "more idiomatic", etc., are wildly
subjective and non-measurable as well.

(besides, we can still conduct a survey on python-list beforehand, and
see how many people mind)



From rdmurray at bitdance.com  Tue Sep 15 15:06:23 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 09:06:23 -0400 (EDT)
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
	<4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909150901161.6303@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 08:42, Jesse Noller wrote:
> On Tue, Sep 15, 2009 at 8:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> As far as I'm concerned, you could change this to ? New modules are
>> acceptable but old modules should remain as long as they are popular,
>> and aren't plagued by severe issues ?.
>>
>> cheers
>>
>> Antoine.
>
> Define "popular" - it's purely subjective.

And as Laura pointed out, there are all those scripts and programs out
there that nobody knows about, because nobody is actively maintaining
them, that would affect the measurement that Antoine is trying to make.

--David

From collinw at gmail.com  Tue Sep 15 15:06:30 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 15 Sep 2009 13:06:30 +0000
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253019996.5646.68.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
	<4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
	<1253018919.5646.60.camel@localhost>
	<4222a8490909150552p687f68c2o783a933caa4f43b3@mail.gmail.com>
	<1253019996.5646.68.camel@localhost>
Message-ID: <43aa6ff70909150606j37c73b35s9398fa6a1f1f9354@mail.gmail.com>

On Tue, Sep 15, 2009 at 1:06 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 08:52 -0400, Jesse Noller a ?crit :
>>
>> How do you define "many people"? What if it's used by MEGACORP
>> extensively, but by barely any modules on the cheeseshop?
>
> Of course it's a matter of perception. But if you want some numerically
> measurable input for everything, this discussion won't progress by a
> single bit. And I don't think your criteria will hold either, because
> things like "better", "up-to-date", "more idiomatic", etc., are wildly
> subjective and non-measurable as well.

On the contrary: it's trivial to put numbers to "popular". Which
number would you prefer:

- Search results on Google Code Search?
- Search results on the Cheeseshop?
- A poll of python-{dev,list}?
- A poll of Guido?
- Grepping your company's source code?
- Grepping my company's source code?
- Number of questions about the module on python-list this month?
- Number of hits Google Analytics records for the module's
documentation this quarter?

Collin Winter

From jnoller at gmail.com  Tue Sep 15 15:12:42 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 09:12:42 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1E9E6AF9-2813-47B5-8BEC-2809BD520EF2@orestis.gr>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1E9E6AF9-2813-47B5-8BEC-2809BD520EF2@orestis.gr>
Message-ID: <4222a8490909150612i354ca466pa9b2fbce6b2269a2@mail.gmail.com>

Since I've obviously had a failure to communicate, let me try to
explain, once again, what my goals are, but first - here's the
assumptions:

a> The standard library is currently too large to maintain.
b> The standard library contains things which are *not* best of breed.
c> The standard library contains numerous modules which do not have
adequate tests; docs or maintainers.
d> The standard library is a monolith

So, based on these assumptions, my proposal(s) were to:

1> Break the standard library into it's own section within the
codebase; this solves D, and allows it to be used by the other
implementations more easily. It also (hopefully) allows for a
modularized build process by which individuals can be more selective
with what is included. A PEP is being drafted for this.

2> Take a hard look at the standard library; identify modules with low
test coverage, poor documentation, and without owners. Add these to a
"deprecation and removal" PEP for eventual removal. While this *will*
break legacy scripts and applications, the likelihood of this
happening in Python 2.x is infinitely small. This will more than
likely occur in py3k, which *already* breaks those scripts. This makes
me sad, as I'm handcuffed to python 2.x for some time, but I'm willing
to cry myself to sleep about that in private. This, in theory, helps
mitigate A and C

3> Identify modules which are not best of breed; and identify
replacements for them. I am not suggesting bleeding edge modules. No
one is. I am suggesting that we take a hard look at the modules and
ask ourselves "is there something else out there stable, clean and
mature, with an active maintainer which could be part of this
library?" This may end up being multiple PEPs.

If best of breed is too subjective - then how about "a viable, widely
used, high quality alternative exists".

Why?

Having a large, batteries included standard library is frankly what
makes Python Python - however, having something with old, unmaintained
modules with little-to-no tests and in the face of libraries in the
wild which are in a much better state serves no one.

I would like owners for every single module in the stdlib; we need
people who are willing to be domain experts for modules to review
patches, write tests and help us. Modules without owners with a vested
interest means we have to either take patches at face value, or simply
ignore the patches because we can not risk the change because we don't
fully understand it.

Note that "age" has nothing to do with this - older modules aren't
"bad" modules. Old, low test count, overly complex, no maintainer,
poor docs, non thread-safe, bad naming scheme modules are bad modules.

Having a bunch of crusty, leaky batteries doesn't help anyone. Needing
to explain to someone why they should not use something in the
standard library because it's either broke, or doesn't work with
current technology sucks.

Jesse

From solipsis at pitrou.net  Tue Sep 15 15:22:23 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 15:22:23 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <43aa6ff70909150606j37c73b35s9398fa6a1f1f9354@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1253016798.5646.40.camel@localhost>
	<4222a8490909150542t31e7cbdco5593684b058f45e0@mail.gmail.com>
	<1253018919.5646.60.camel@localhost>
	<4222a8490909150552p687f68c2o783a933caa4f43b3@mail.gmail.com>
	<1253019996.5646.68.camel@localhost>
	<43aa6ff70909150606j37c73b35s9398fa6a1f1f9354@mail.gmail.com>
Message-ID: <1253020943.5646.90.camel@localhost>

Le mardi 15 septembre 2009 ? 13:06 +0000, Collin Winter a ?crit :
> 
> On the contrary: it's trivial to put numbers to "popular". Which
> number would you prefer:
[snip]

Well, the fact that there are so many different numbers to choose from
makes them collectively useless, unless you have a magic recipe which
yields an "objective" and irrefutable truth by combining them :)

My point, more generally, was that most python-core discussions don't
really bother with objective numbers. They try to conflate perceptions
and see what comes out. When Marc-Andr? and others pointed out that
optparse and getopt were popular, noone tried to challenge them by
asking for numbers. Therefore, the view that these modules are popular
is accepted as truth within our small stdlib-sig community, even though
no objective proof was ever given.

IMO, most discussions about "outdated" modules would follow the same
path along the "popularity" issue: either someone points out that the
module is still popular and he is believed a priori, or nobody points it
out and chances are the module can be considered non-popular.



From michael at voidspace.org.uk  Tue Sep 15 15:23:08 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 14:23:08 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909150612i354ca466pa9b2fbce6b2269a2@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>	
	<4AAEBEBE.1050105@egenix.com>	
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>	
	<4AAF533D.9030908@egenix.com>	
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>	
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>	
	<1E9E6AF9-2813-47B5-8BEC-2809BD520EF2@orestis.gr>
	<4222a8490909150612i354ca466pa9b2fbce6b2269a2@mail.gmail.com>
Message-ID: <4AAF953C.4060907@voidspace.org.uk>

Jesse Noller wrote:
> Since I've obviously had a failure to communicate, let me try to
> explain, once again, what my goals are, but first - here's the
> assumptions:
>
> a> The standard library is currently too large to maintain.
> b> The standard library contains things which are *not* best of breed.
> c> The standard library contains numerous modules which do not have
> adequate tests; docs or maintainers.
> d> The standard library is a monolith
>
> So, based on these assumptions, my proposal(s) were to:
>
> 1> Break the standard library into it's own section within the
> codebase; this solves D, and allows it to be used by the other
> implementations more easily. It also (hopefully) allows for a
> modularized build process by which individuals can be more selective
> with what is included. A PEP is being drafted for this.
>
>   
This all sounds very good Jesse - but can you make sure that your 
proposal for splitting out the standard library for distribution is kept 
separate (preferably a separate PEP). It seems orthogonal and something 
that will attract a lot of debate on its own.

It would be a shame to stall either issue because of controversy around 
the other.

> 2> Take a hard look at the standard library; identify modules with low
> test coverage, poor documentation, and without owners. 
I would add to this "or of marginal utility". I think there are several 
modules that have very few users and would never be included in the 
standard library today (the calendar module springs to mind).

> Add these to a
> "deprecation and removal" PEP for eventual removal. While this *will*
> break legacy scripts and applications, the likelihood of this
> happening in Python 2.x is infinitely small. This will more than
> likely occur in py3k, which *already* breaks those scripts. This makes
> me sad, as I'm handcuffed to python 2.x for some time, but I'm willing
> to cry myself to sleep about that in private. This, in theory, helps
> mitigate A and C
>   

It would be wise to emphasise *eventual* removal. I think a long 
deprecation process leading to removal around Python 3.4 or 3.5 will 
give people sufficient notice and hopefully mitigate *some* of the 
problems people have with module removal.

Michael
> 3> Identify modules which are not best of breed; and identify
> replacements for them. I am not suggesting bleeding edge modules. No
> one is. I am suggesting that we take a hard look at the modules and
> ask ourselves "is there something else out there stable, clean and
> mature, with an active maintainer which could be part of this
> library?" This may end up being multiple PEPs.
>
> If best of breed is too subjective - then how about "a viable, widely
> used, high quality alternative exists".
>
> Why?
>
> Having a large, batteries included standard library is frankly what
> makes Python Python - however, having something with old, unmaintained
> modules with little-to-no tests and in the face of libraries in the
> wild which are in a much better state serves no one.
>
> I would like owners for every single module in the stdlib; we need
> people who are willing to be domain experts for modules to review
> patches, write tests and help us. Modules without owners with a vested
> interest means we have to either take patches at face value, or simply
> ignore the patches because we can not risk the change because we don't
> fully understand it.
>
> Note that "age" has nothing to do with this - older modules aren't
> "bad" modules. Old, low test count, overly complex, no maintainer,
> poor docs, non thread-safe, bad naming scheme modules are bad modules.
>
> Having a bunch of crusty, leaky batteries doesn't help anyone. Needing
> to explain to someone why they should not use something in the
> standard library because it's either broke, or doesn't work with
> current technology sucks.
>
> Jesse
>   


-- 
http://www.ironpythoninaction.com/


From jnoller at gmail.com  Tue Sep 15 15:33:01 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 09:33:01 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF953C.4060907@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1E9E6AF9-2813-47B5-8BEC-2809BD520EF2@orestis.gr>
	<4222a8490909150612i354ca466pa9b2fbce6b2269a2@mail.gmail.com>
	<4AAF953C.4060907@voidspace.org.uk>
Message-ID: <4222a8490909150633n7b2229eajff401a705cd35b@mail.gmail.com>

On Tue, Sep 15, 2009 at 9:23 AM, Michael Foord <michael at voidspace.org.uk> wrote:
> Jesse Noller wrote:
>>
>> Since I've obviously had a failure to communicate, let me try to
>> explain, once again, what my goals are, but first - here's the
>> assumptions:
>>
>> a> The standard library is currently too large to maintain.
>> b> The standard library contains things which are *not* best of breed.
>> c> The standard library contains numerous modules which do not have
>> adequate tests; docs or maintainers.
>> d> The standard library is a monolith
>>
>> So, based on these assumptions, my proposal(s) were to:
>>
>> 1> Break the standard library into it's own section within the
>> codebase; this solves D, and allows it to be used by the other
>> implementations more easily. It also (hopefully) allows for a
>> modularized build process by which individuals can be more selective
>> with what is included. A PEP is being drafted for this.
>>
>>
>
> This all sounds very good Jesse - but can you make sure that your proposal
> for splitting out the standard library for distribution is kept separate
> (preferably a separate PEP). It seems orthogonal and something that will
> attract a lot of debate on its own.

Yes, note I omitted the "separate for distribution" thing, as I'll
write that up post-other-peps

> It would be a shame to stall either issue because of controversy around the
> other.
>

Agreed.

>> 2> Take a hard look at the standard library; identify modules with low
>> test coverage, poor documentation, and without owners.
>
> I would add to this "or of marginal utility". I think there are several
> modules that have very few users and would never be included in the standard
> library today (the calendar module springs to mind).

Agreed.

>> Add these to a
>> "deprecation and removal" PEP for eventual removal. While this *will*
>> break legacy scripts and applications, the likelihood of this
>> happening in Python 2.x is infinitely small. This will more than
>> likely occur in py3k, which *already* breaks those scripts. This makes
>> me sad, as I'm handcuffed to python 2.x for some time, but I'm willing
>> to cry myself to sleep about that in private. This, in theory, helps
>> mitigate A and C
>>
>
> It would be wise to emphasise *eventual* removal. I think a long deprecation
> process leading to removal around Python 3.4 or 3.5 will give people
> sufficient notice and hopefully mitigate *some* of the problems people have
> with module removal.

I assumed that was implicit ;) I too must obey the deprecation guidelines!

From solipsis at pitrou.net  Tue Sep 15 15:36:42 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 15:36:42 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF953C.4060907@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk>
	<1E9E6AF9-2813-47B5-8BEC-2809BD520EF2@orestis.gr>
	<4222a8490909150612i354ca466pa9b2fbce6b2269a2@mail.gmail.com>
	<4AAF953C.4060907@voidspace.org.uk>
Message-ID: <1253021802.5646.108.camel@localhost>

Le mardi 15 septembre 2009 ? 14:23 +0100, Michael Foord a ?crit :
> This all sounds very good Jesse - but can you make sure that your 
> proposal for splitting out the standard library for distribution is kept 
> separate (preferably a separate PEP). It seems orthogonal and something 
> that will attract a lot of debate on its own.
> 
> It would be a shame to stall either issue because of controversy around 
> the other.

+1 :)
Trying to sell us a "big package" will only make the discussion more
tedious, confusing and confused.




From collinw at gmail.com  Tue Sep 15 16:38:20 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 15 Sep 2009 14:38:20 +0000
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151029.n8FATHmq028625@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
Message-ID: <43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:29 AM, Laura Creighton <lac at openend.se> wrote:
[snip]
> Indeed, right now, if you write code that parses options you are
> most likely doing one of 4 things:
>
> a) using getopt
> b) using optparse
> c) using some third party module for parsing that is currently under
> ? development
> d) writing your own option parser in python
>
> Now, for years I have been telling people that d) was by far the poorest
> way of doing things. ?Don't reinvent the wheel, and all that. ?But on
> the day that you yank getopt and optparse out of the library, you will
> have made this terrible advice to give my poor hospital clients. ?And
> you will have made all the time I spent ripping out hand-made option
> parsers and replacing them with getopt instead wasted time that I billed
> my clients for. ?Instead of giving them a robust solution that will
> require little or no maintainance, I have given them a zillion scripts,
> _all of which will suddenly break one day_. ?And since the pure python
> parsers that I never got around to converting will probably still be
> working, the people who did d) will get the last laugh.

This is incorrect. Code that runs correctly on Python x.y will not
"suddenly break one day": no-one from python-dev is going to reach
into your customers' systems and install some other version of Python
behind your/their back, against their will.

The code you have written will continue to work, indefinitely, until
the customer decides to upgrade their version of Python. When that
happens, they can choose to perform this migration correctly or
incorrectly. Upgrading correctly involves testing (your code comes
with a test suite, I assume) against the new version of Python, to
make sure things work, before deploying to production. This is no
different than upgrading versions of Postgres, gcc or Linux: it has
risks, but gcc doesn't upgrade itself on its own. You choose when to
do this, and you do so in a way that contains and mitigates the risk.

> So -- real use case time -- the hospitals where I have done a lot of
> work have some really weird equipment. ?And using them costs real
> money, uses up real lab supplies, and conceivably can ruin a sample
> that you will find it inconvenient or impossible to replace.
>
> There are all sorts of weird dependencies among the various options
> you can use to operate the devices. ?If you have specified option K,
> you may not also specify option M or N, and if you have specified
> option L, you must also specify option Q, or option R and option S.
>
> Thus the whole exercise of writing a script to use the equipment
> becomes as matter of validating the options you selected are complete
> and non-contradictory, and then going out and exercising the hardware.
>
> You can build a validating option parser by hand (option d) or using
> getopt (option a). ?You will find it difficult to the point of near
> impossibility to build one using optparse, because optparse specifically
> rejects the notion of 'required options' which is the meat-and-potoatoes
> part of this app. ?I found this out when optparse went into the standard
> library, and was touted at being superior to getopt.
>
> I tried it, and tried to subclass it, and talked to its author about
> whether it could be changed to support my use case, and even volunteered
> to write code to change it, but was firmly told that optparse worked
> the way it did, on purpose, in order to prevent the sort of use that
> I wanted to make of it, and that patches to make that possible were
> entirely unwelcome.

Frankly, this sounds like an *excellent* reason to get rid of optparse
and replace it with something more flexible. Steven Bethard has said
that he originally tried adding the features of argparse to optparse,
but found the code to be so poorly designed that it really couldn't be
changed in any meaningful way. If the Python community is finding
common argument-parsing scenarios that optparse doesn't support (such
as what you outlined above), to the point where people are hand-coding
argument parsers because that's easier than using optparse, then
optparse is a failure and we should look for something that better
serves the needs of developers like yourself.

> At which point I go back to using getopt. ?There is no particular hard
> feelings about this -- ?I figure that the people who want to use
> optparse can use it, and getopt is here for the people who won't, or
> can't. ?But when later in time people suggested getting rid of getopt
> because 'it was old' and 'optparse was better' I realise, a long time
> when it was too late to do anything, that I should have been spearheading
> the 'I don't want optparse in the standard library' effort, on the
> grounds that it didn't support 'required arguments'. ?That fight might
> have become nasty. ?Far better to allow multiple ways of doing things.

I sympathize with your experience, but disagree with your conclusion.
It doesn't sound like we have multiple ways of doing things at all: it
seems we have several different argument parsing libraries that,
despite starting from a common goal of "argument parsing", have
implemented divergent functionalities with only a relatively small
area of overlap. They actually do different things, and in different
ways.

This is a serious problem. If I've been happily using optparse in my
project, and one day discover that I need required arguments, I have
to switch to a totally different library? Libraries should be able to
scale to meet their users' needs. This is what Jacob Kaplan-Moss was
talking about in his recent PyCon Argentina/Brazil keynotes
(http://jacobian.org/TO): as the user has more complex/specific needs,
the library should be able to yield gracefully, either implementing
the desired functionality itself or making it possible for the user to
customize specific aspects of the library's behaviour without throwing
away/rewriting the entire library.

> As far as I can tell most software packages go through three stages:
[snip stage descriptions]
> Now software in Stage A doesn't need maintenance so much as development
> in the first place. ?And software in stage B requires a lot of
> maintainance. ?But most software in stage C requires little or no
> maintenance, precisely because it is unchanging. ?So, if you decide to
> change how Exceptions are inherited, for instance, you may break a
> whole lot of Stage C: code, but fixing them is part of the general
> problem of 'fixing exceptions everywhere' not part of 'fixing getopt'.
>
> Now I think that there is some confusion here between packages who
> are in Stage C: ?and packages who are in Stage B-2: ?around here.
> The B-2 packages are probably the core developers greatest headache.
> But I don't see the C packages as being troublesome at all. ?If you
> don't like them, it isn't because of the work that maintaining is
> costing you. ?It may be that hatred for the B-2's has become a general
> hatred of all packages with no maintainers, which is an understandable
> mistake. ?But from reading this list I get the distinct impression that
> some people just hate C:s _precisely because they are old and unchanging_,
> and would continue to hate them for that reason even if I was in some
> way able to guarantee that they would never need any maintenance ever
> again. ?These people are condemning the packages I love best for the
> reason they are the packages I love best. ?And that is the attitude
> I would like to change.

Speaking as a core developer, I disagree. The problem with
unmaintained (in your terms, B-2) or intentionally-frozen (C-*)
packages is that they make it difficult for us to evolve and adapt
Python the language and Python the standard library: if no-one is
willing/available to update the code to account for language/library
changes, the frozen package will become pinned to a specific
known-good version (or range of versions) of Python. Over time, that
version of Python will become uncommon (as distros phase it out) and
unsupported (as python-dev end-of-lifes it). This is a problem for
users of that package, who may wish to use newer version of Python for
performance or bug-fix reasons, and it is also a problem for
python-dev, since those frozen packages create inertia.

> It boils down to a matter of trust. ?My customers trust me to not give
> them ticking time bombs that will all stop working one day, and I trust
> you not to go about gratuitously removing perfectly working code that
> is quietly sitting there, not needing any changes, and not bothering
> anybody. ?When you break that contract with me, my customers suffer,
> I suffer, and the people who said 'You shouldn't have coded it in
> Python in the first place, but picked a mature language like Java'
> are completely vindicated.

The systems you have written for your customers are not autonomous
agents; presumably, you have not written code like "if today.year ==
2011: sudo apt-get upgrade python" into these systems that would
change the version of Python running without anyone asking. Human
beings control these upgrades. If the human being performs the upgrade
blindly, without taking appropriate risk mitigation steps, there's
very little that we can do to protect them. You left out a key element
in the chain of trust above: presumably, you trust your customers not
to violate the minimum requirements you've set out for the software
you've written for them. If you say "this software requires 2GB of
RAM" and the customer later decides to try running the machines with
only 256MB to save money instead, that's not your fault: it's theirs.
Likewise, if you say "this software runs on Python 2.5" and they
blindly install Python 3.1 instead, that's not your fault: it's
theirs.

As to the matter of Java's deprecation policy, I don't regard it as
"mature": I regard it as a sign of different requirements. Because you
can't know what browser version or JRE version a user's desktop is
running, stability is paramount for Java; "write once, run anywhere"
is not free, it has its costs. As Frank Wierzbicki has said (either in
this thread or the other one about argparse), the inability to ever
remove code from the libraries makes life difficult for Java's
developers -- who have to maintain this code -- as well as for
everyday Java engineers, who have to learn to navigate this maze of
deprecated vs non-deprecated solutions to the same problem. Java shops
generally end up with a list of Approved Java Classes so that new
hires and old pros alike don't get tripped up.

In Python, we don't have the luxury of a paid staff to work on our
libraries, to maintain the crufty, fragile,
we'd-like-to-get-rid-of-you-but-maybe-someone's-using-you-we-don't-really-know
modules. We rely almost exclusively on volunteer contributions, and
it's tough to find volunteers to work on crap code. It's one thing to
choose not to change something; it's another thing entirely not to be
able to change something, to have your hands tied by code you can't
see and no-one will change. As Python development slows, as stability
gets confused for permanence and stasis, I predict it will be harder
to attract enthusiastic, eager contributors. After all, who wants to
work on something you're not allowed to modify?

To speak more personally, and specifically to the issue of
getopt/optparse vs argparse: at Google, I'm part of the Python
readability team, which helps train the large numbers of Python
developers that the company produces. Part of this job involves
conducting detailed code reviews for new Python programmers,
explaining both Google style and idiomatic Python code generally,
suggesting library A over hand-written solution B. I am, frankly,
embarrassed whenever I have to explain the difference between getopt
and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess,
string.* vs str.*, etc. I cannot imagine how embarrassed I will be
when I have to explain why the standard library includes getopt,
optparse and argparse.

Collin Winter

From lac at openend.se  Tue Sep 15 16:55:03 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Sep 2009 16:55:03 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: Message from "M.-A. Lemburg" <mal@egenix.com> 
	of "Tue, 15 Sep 2009 13:22:01 +0200." <4AAF78D9.20107@egenix.com> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>
	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com> 
Message-ID: <200909151455.n8FEt3F1019480@theraft.openend.se>

I am all for breaking up the stdlib logically, via tagging.  I am
worried about physically breaking things up, though.  How are we
going to guarantee that imports still work after the breakup?

Laura


From jnoller at gmail.com  Tue Sep 15 16:59:44 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 10:59:44 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151455.n8FEt3F1019480@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <mal@egenix.com>
	<4AAF78D9.20107@egenix.com>
	<200909151455.n8FEt3F1019480@theraft.openend.se>
Message-ID: <4222a8490909150759w2440d1e0odc90b9fce68872cb@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:55 AM, Laura Creighton <lac at openend.se> wrote:
> I am all for breaking up the stdlib logically, via tagging. ?I am
> worried about physically breaking things up, though. ?How are we
> going to guarantee that imports still work after the breakup?
>
> Laura

What?

We run the same unit tests we run today.

From mal at egenix.com  Tue Sep 15 17:36:06 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 17:36:06 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151455.n8FEt3F1019480@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<8B5044E6-DA8A-4C25-BD49-D76EAC0D2091@gmail.com>	<79990c6b0909140835p5d387730jd48b33b7b2341c12@mail.gmail.com>	<20090914160714.GE15328@idyll.org>	<4AAE6AAC.6090109@voidspace.org.uk>	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>	<1252947207.7398.8.camel@localhost>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>
	<200909151455.n8FEt3F1019480@theraft.openend.se>
Message-ID: <4AAFB466.7080400@egenix.com>

Laura Creighton wrote:
> I am all for breaking up the stdlib logically, via tagging.  I am
> worried about physically breaking things up, though.  How are we
> going to guarantee that imports still work after the breakup?

I don't think CPython users will have to worry about this: the
main distribution will still ship with all batteries included.

However, Jython/IronPython/PyPy users will likely only see a subset.

Creating those should be easy using a bit of distutils. We'd only
need to put the tag information into some module that setup.py can
import to easily get at the tags and module names.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From lac at openend.se  Tue Sep 15 17:40:09 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Sep 2009 17:40:09 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: Message from Michael Foord <michael@voidspace.org.uk> 
	of "Tue, 15 Sep 2009 12:53:38 BST." <4AAF8042.8000008@voidspace.org.uk> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<20090914160714.GE15328@idyll.org>
	<4AAE6AAC.6090109@voidspace.org.uk>
	<4222a8490909140919m1c167f1fs26fb077ad9c6aa4a@mail.gmail.com>
	<1252947207.7398.8.camel@localhost>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<d11dcfba0909141903m76c2f439r172770329b4f7531@mail.gmail.com>
	<4AAF533D.9030908@egenix.com>
	<E97DEDBA-F978-485E-AEF2-2EF007454332@orestis.gr>
	<4AAF5C88.2000009@egenix.com> <4AAF8042.8000008@voidspace.org.uk> 
Message-ID: <200909151540.n8FFe9sm023348@theraft.openend.se>

In a message of Tue, 15 Sep 2009 12:53:38 BST, Michael Foord writes:

>1) Virtually no changes or improvements to the standard library at all - 
>nothing beyond maintaining compatibility with language changes. (Laura)

This isn't my position at all.  What I think is that there are
some modules in the standard library which have already reached
that state, as part of their natural evolution.  And these are
great modules, which should never be removed, even if there are
more modern ways to do things.  I've not been opposed to the addition
of new modules, though I do think that 'who is going to maintain
this module in 10 years' is a question that needs an answer before
we start sticking things into the standard library.  And 'well, if
nobody wants to maintain it, we can always drop it', is not an
acceptable answer.

There may be a need for a not-standard-but-we-love-you-anyway library
full of things that people would have put in the standard library
except that they strongly want the right to drop support for the
thing as soon as it is no longer actively maintained.  

Laura

From solipsis at pitrou.net  Tue Sep 15 17:49:02 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 17:49:02 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
Message-ID: <1253029742.5646.180.camel@localhost>

Le mardi 15 septembre 2009 ? 14:38 +0000, Collin Winter a ?crit :
> This is no
> different than upgrading versions of Postgres, gcc or Linux: it has
> risks, but gcc doesn't upgrade itself on its own.

But there *is* a difference between unforeseen bugs (which are an
unavoidable pitfall of software development), and deliberate removal of
widely used features.

Your reasoning is like saying: there will be unforeseen bugs anyway, so
we can just create deliberate bugs for the sake of it.

> Frankly, this sounds like an *excellent* reason to get rid of optparse
> and replace it with something more flexible.

It *would* have been an excellent reason to choose argparse over
optparse in the first place. But since optparse is already in the
stdlib, we have users to care about.

> This is a problem for
> users of that package, who may wish to use newer version of Python for
> performance or bug-fix reasons,

This is a reason for keeping that package in the stdlib, not for
removing it.

> and it is also a problem for
> python-dev, since those frozen packages create inertia.

I'm not sure what inertia you're talking about. The presence of the
Lib/optparse.py file does not look like a deterrent to contributing to
any other part of the interpreter or the stdlib. Lib/optparse.py itself
doesn't seem to receive a lot of checkins, and most are cosmetic. There
are very few open bugs concerning optparse in bugs.python.org, and most
seem normal/low priority. From this I gather that:
1) optparse is low maintenance (requires few checkins)
2) optparse is low annoyance (doesn't clutter the bug tracker)

Taking optparse as an example defeats your point more than it helps it,
IMO. You should choose a bug-ridden module with an awful API, not
something perfectly decent which fits lots of everyday use cases.



From collinw at gmail.com  Tue Sep 15 18:03:01 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 15 Sep 2009 12:03:01 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253029742.5646.180.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<1253029742.5646.180.camel@localhost>
Message-ID: <43aa6ff70909150903v3738d817iec5bfea8b2bb247@mail.gmail.com>

On Tue, Sep 15, 2009 at 3:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 14:38 +0000, Collin Winter a ?crit :
>> This is no
>> different than upgrading versions of Postgres, gcc or Linux: it has
>> risks, but gcc doesn't upgrade itself on its own.
>
> But there *is* a difference between unforeseen bugs (which are an
> unavoidable pitfall of software development), and deliberate removal of
> widely used features.
>
> Your reasoning is like saying: there will be unforeseen bugs anyway, so
> we can just create deliberate bugs for the sake of it.

I was specifically responding to the idea that deprecation/removal
creates "ticking time bombs". Willful deprecation is not a bug.

>> Frankly, this sounds like an *excellent* reason to get rid of optparse
>> and replace it with something more flexible.
>
> It *would* have been an excellent reason to choose argparse over
> optparse in the first place. But since optparse is already in the
> stdlib, we have users to care about.
>
>> This is a problem for
>> users of that package, who may wish to use newer version of Python for
>> performance or bug-fix reasons,
>
> This is a reason for keeping that package in the stdlib, not for
> removing it.
>
>> and it is also a problem for
>> python-dev, since those frozen packages create inertia.
>
> I'm not sure what inertia you're talking about. The presence of the
> Lib/optparse.py file does not look like a deterrent to contributing to
> any other part of the interpreter or the stdlib.

Sorry, I meant frozen/unmaintained packages that use optparse, rather
than optparse itself.

> Lib/optparse.py itself
> doesn't seem to receive a lot of checkins, and most are cosmetic. There
> are very few open bugs concerning optparse in bugs.python.org, and most
> seem normal/low priority. From this I gather that:
> 1) optparse is low maintenance (requires few checkins)
> 2) optparse is low annoyance (doesn't clutter the bug tracker)
>
> Taking optparse as an example defeats your point more than it helps it,
> IMO. You should choose a bug-ridden module with an awful API, not
> something perfectly decent which fits lots of everyday use cases.

Did you read what Laura wrote about her experience with optparse?

"""
So -- real use case time -- the hospitals where I have done a lot of
work have some really weird equipment.  And using them costs real
money, uses up real lab supplies, and conceivably can ruin a sample
that you will find it inconvenient or impossible to replace.

There are all sorts of weird dependencies among the various options
you can use to operate the devices.  If you have specified option K,
you may not also specify option M or N, and if you have specified
option L, you must also specify option Q, or option R and option S.

Thus the whole exercise of writing a script to use the equipment
becomes as matter of validating the options you selected are complete
and non-contradictory, and then going out and exercising the hardware.

You can build a validating option parser by hand (option d) or using
getopt (option a).  You will find it difficult to the point of near
impossibility to build one using optparse, because optparse specifically
rejects the notion of 'required options' which is the meat-and-potoatoes
part of this app.  I found this out when optparse went into the standard
library, and was touted at being superior to getopt.

I tried it, and tried to subclass it, and talked to its author about
whether it could be changed to support my use case, and even volunteered
to write code to change it, but was firmly told that optparse worked
the way it did, on purpose, in order to prevent the sort of use that
I wanted to make of it, and that patches to make that possible were
entirely unwelcome.
"""

Did you read what Steven Bethard wrote about his experience with optparse?

"""
On Thu, Sep 10, 2009 at 3:55 PM, Collin Winter <collinw at gmail.com> wrote:
> Honest question, having only read the docs about argparse: would it be
> possible to merge the functionality of argparse into optparse and so
> preserve a greater measure of backwards compatibility? Some of the
> stuff I'm reading about in
> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
> looks like it should be able to be integrated fairly easily into the
> existing optparse structure.

I tried this, and when I originally started argparse, it was my intent
to make it fully compatible with optparse. For a simple example,
consider the documented "parser.largs", "parser.rargs" and
"parser.values" attributes of OptionParser. Supporting these would not
allow argparse's current parsing strategy, which doesn't follow the
optparse approach of moving args from "largs" to "rargs".
"""

or

"""
On Sun, Sep 13, 2009 at 12:31 AM, Ronald Oussoren
<ronaldoussoren at mac.com> wrote:
> optparse's source code isn't cast in stone, therefore its interface can
> evolve.

While I agree with this comment in general, because optparse
officially exposes lots of internal details (OptionParser.largs,
OptionParser.rargs, Option.TYPE_CHECKER, Option.ALWAYS_TYPED_ACTIONS,
etc.) in practice it's extremely hard to evolve the optparse codebase
without introducing backwards compatibilities. If you don't believe
me, I recommend trying to implement argparse's support for
type=<callable> in optparse, while respecting Option.TYPES, etc. This
is one of the reasons in argparse I've been extremely conservative on
what constitutes the public API - I don't want the compatibility
nightmare that optparse has.
"""

It's also worth noting that optparse continues to have bugs filed
against it, so it is not perfect or low-maintenance in that respect:
http://bugs.python.org/issue2931 is an interesting example.

Collin Winter

From solipsis at pitrou.net  Tue Sep 15 18:18:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 18:18:37 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <43aa6ff70909150903v3738d817iec5bfea8b2bb247@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<1253029742.5646.180.camel@localhost>
	<43aa6ff70909150903v3738d817iec5bfea8b2bb247@mail.gmail.com>
Message-ID: <1253031517.5646.191.camel@localhost>

Le mardi 15 septembre 2009 ? 12:03 -0400, Collin Winter a ?crit :
> 
> Did you read what Laura wrote about her experience with optparse?
> [...]

Sure I did. These are still fringe use cases compared to the common uses
of an option parser. Perhaps argparse has a couple of nifty features
that make things slightly easier in the common cases, but the difference
doesn't seem earth-shattering to me.

The point is, optparse does useful things and it does them reasonably
well. You can't deny that, and talking about how that other library does
many other things is not a satisfying answer to the people whose code
you're gonna break.

> It's also worth noting that optparse continues to have bugs filed
> against it, so it is not perfect or low-maintenance in that respect:
> http://bugs.python.org/issue2931 is an interesting example.

Well, of course it has bugs filed against it (which software hasn't?).
The point is that there are few of them.
And, yes, being more unicode-compliant would be nice (although the bug
should be retried on py3k, because this whole discussion probably
targets 3.x anyway, not 2.x).



From p.f.moore at gmail.com  Tue Sep 15 18:16:41 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Sep 2009 17:16:41 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF78D9.20107@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com>
Message-ID: <79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>

2009/9/15 M.-A. Lemburg <mal at egenix.com>:
> Laura Creighton wrote:
>> So what do you think of this proposal?
>
> Good write-up and very much to the point.
>
> [Executive Summary:
> Code that hardly needs any changes, because it does what it's meant
> to do, is good code, not bad code. And it causes only minimal
> maintenance effort, so it's actually something core developer should
> welcome rather than fight against.]
>
> I'd only change the tag "dead-as-a-doornail" to "complete, proven and
> stable". Sounds more accurate.

Yes, I like both the summary, and the proposal (that standard library
code be tagged with details like its status and its maintainer).

I was in the "stdlib needs to evolve" camp, but this has clarified the
other side of the argument, and changed my mind.

I'm still in favour of new modules being added to the stdlib, and
existing modules being updated, but I support the idea of stage C
(dead as a doornail/complete, proven, stable) modules being retained
indefinitely. I'm not sure what the implications of this position
would be in the case of argparse vs optparse (optparse doesn't seem to
be stage C, so maybe removing it in favour of argparse is an option)
but I like the fact that this proposal gives us terminology on which
we can base the discussion.

Paul.

From jnoller at gmail.com  Tue Sep 15 18:21:53 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 12:21:53 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com>
	<79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>
Message-ID: <4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>

On Tue, Sep 15, 2009 at 12:16 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/15 M.-A. Lemburg <mal at egenix.com>:
>> Laura Creighton wrote:
>>> So what do you think of this proposal?
>>
>> Good write-up and very much to the point.
>>
>> [Executive Summary:
>> Code that hardly needs any changes, because it does what it's meant
>> to do, is good code, not bad code. And it causes only minimal
>> maintenance effort, so it's actually something core developer should
>> welcome rather than fight against.]
>>
>> I'd only change the tag "dead-as-a-doornail" to "complete, proven and
>> stable". Sounds more accurate.
>
> Yes, I like both the summary, and the proposal (that standard library
> code be tagged with details like its status and its maintainer).
>
> I was in the "stdlib needs to evolve" camp, but this has clarified the
> other side of the argument, and changed my mind.
>
> I'm still in favour of new modules being added to the stdlib, and
> existing modules being updated, but I support the idea of stage C
> (dead as a doornail/complete, proven, stable) modules being retained
> indefinitely. I'm not sure what the implications of this position
> would be in the case of argparse vs optparse (optparse doesn't seem to
> be stage C, so maybe removing it in favour of argparse is an option)
> but I like the fact that this proposal gives us terminology on which
> we can base the discussion.
>
> Paul.

There is no, no such thing as a "dead/complete" module. It does not
exist. Any time there is a grammar change, a new reserved keyword, or
some other functionality change a "dead" module comes back to life and
has to be maintained.

Can we please not treat "dead/complete" modules as if they have no
maintenance burden, or drag down the code base? The reason I avoided
that terminology in the first place is that there is no such thing as
code with zero cost.

jesse

From lac at openend.se  Tue Sep 15 18:28:50 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Sep 2009 18:28:50 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: Message from Jesse Noller <jnoller@gmail.com> of "Tue,
	15 Sep 2009 10:59:44 EDT."
	<4222a8490909150759w2440d1e0odc90b9fce68872cb@mail.gmail.com> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<mal@egenix.com> <4AAF78D9.20107@egenix.com>
	<200909151455.n8FEt3F1019480@theraft.openend.se>
	<4222a8490909150759w2440d1e0odc90b9fce68872cb@mail.gmail.com> 
Message-ID: <200909151628.n8FGSolQ026510@theraft.openend.se>

In a message of Tue, 15 Sep 2009 10:59:44 EDT, Jesse Noller writes:
>On Tue, Sep 15, 2009 at 10:55 AM, Laura Creighton <lac at openend.se> wrote:
>> I am all for breaking up the stdlib logically, via tagging. ?I am
>> worried about physically breaking things up, though. ?How are we
>> going to guarantee that imports still work after the breakup?
>>
>> Laura
>
>What?
>
>We run the same unit tests we run today.

I thought that people were proposing changing the directory structure
of the stdlib.

Laura

From solipsis at pitrou.net  Tue Sep 15 18:38:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 18:38:31 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com>
	<79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>
	<4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>
Message-ID: <1253032711.5646.199.camel@localhost>

Le mardi 15 septembre 2009 ? 12:21 -0400, Jesse Noller a ?crit :
> 
> Can we please not treat "dead/complete" modules as if they have no
> maintenance burden, or drag down the code base? The reason I avoided
> that terminology in the first place is that there is no such thing as
> code with zero cost.

Let's say "low maintenance", then. "Low" not necessarily meaning zero,
of course.

Regards

Antoine.



From mal at egenix.com  Tue Sep 15 18:37:47 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 18:37:47 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>	<79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>
	<4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>
Message-ID: <4AAFC2DB.30109@egenix.com>

Jesse Noller wrote:
> On Tue, Sep 15, 2009 at 12:16 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/9/15 M.-A. Lemburg <mal at egenix.com>:
>>> Laura Creighton wrote:
>>>> So what do you think of this proposal?
>>>
>>> Good write-up and very much to the point.
>>>
>>> [Executive Summary:
>>> Code that hardly needs any changes, because it does what it's meant
>>> to do, is good code, not bad code. And it causes only minimal
>>> maintenance effort, so it's actually something core developer should
>>> welcome rather than fight against.]
>>>
>>> I'd only change the tag "dead-as-a-doornail" to "complete, proven and
>>> stable". Sounds more accurate.
>>
>> Yes, I like both the summary, and the proposal (that standard library
>> code be tagged with details like its status and its maintainer).
>>
>> I was in the "stdlib needs to evolve" camp, but this has clarified the
>> other side of the argument, and changed my mind.
>>
>> I'm still in favour of new modules being added to the stdlib, and
>> existing modules being updated, but I support the idea of stage C
>> (dead as a doornail/complete, proven, stable) modules being retained
>> indefinitely. I'm not sure what the implications of this position
>> would be in the case of argparse vs optparse (optparse doesn't seem to
>> be stage C, so maybe removing it in favour of argparse is an option)
>> but I like the fact that this proposal gives us terminology on which
>> we can base the discussion.
>>
>> Paul.
> 
> There is no, no such thing as a "dead/complete" module. It does not
> exist. Any time there is a grammar change, a new reserved keyword, or
> some other functionality change a "dead" module comes back to life and
> has to be maintained.
> 
> Can we please not treat "dead/complete" modules as if they have no
> maintenance burden, or drag down the code base? The reason I avoided
> that terminology in the first place is that there is no such thing as
> code with zero cost.

We're not treating them like that. All along we said, there is
/very little/ maintenance needed.

Besides, changes such as keyword fixes, grammar changes, style
changes, etc. get applied to all stdlib modules, so these fixes
don't really count as module-specific maintenance. Most of these
fixes are done via a script or grep/sed anyway, so the cost per
module is very small.

You can have a look at the getopt SVN log to get an impression
of just how much effort it takes to keep that module running.

These are all changes applied to the module over a period of 7 years:

------------------------------------------------------------------------
r67572 | georg.brandl | 2008-12-05 10:23:14 +0100 (Fri, 05 Dec 2008) | 2 lines

#4458: recognize "-" as an argument, not a malformed option in gnu_getopt().

------------------------------------------------------------------------
r31261 | akuchling | 2003-02-06 20:52:56 +0100 (Thu, 06 Feb 2003) | 4 lines

A few naughty external scripts do 'raise getopt.error, "blah"', and
   now crash because two arguments are expected.  Add a default value
   to keep those scripts running.

------------------------------------------------------------------------
r27929 | loewis | 2002-08-04 19:22:59 +0200 (Sun, 04 Aug 2002) | 2 lines

Add encoding declaration.

------------------------------------------------------------------------
r27780 | jackjansen | 2002-07-26 13:34:49 +0200 (Fri, 26 Jul 2002) | 2 lines

Use os.environ.get() in stead of os.getenv() (which is platform-dependent).

------------------------------------------------------------------------
r27645 | tim_one | 2002-07-16 23:35:23 +0200 (Tue, 16 Jul 2002) | 2 lines

Whitespace normalization.

------------------------------------------------------------------------
r27095 | montanaro | 2002-06-07 05:26:43 +0200 (Fri, 07 Jun 2002) | 2 lines

gnu_getopt should be exported in __all__

------------------------------------------------------------------------
r27077 | loewis | 2002-06-06 20:14:50 +0200 (Thu, 06 Jun 2002) | 2 lines

Use isinstance for the type check, use booleans.

------------------------------------------------------------------------
r27068 | loewis | 2002-06-06 12:58:36 +0200 (Thu, 06 Jun 2002) | 2 lines

Patch 473512: add GNU style scanning as gnu_getopt.

------------------------------------------------------------------------
r26191 | tim_one | 2002-04-05 00:55:58 +0200 (Fri, 05 Apr 2002) | 2 lines

Convert a pile of obvious "yes/no" functions to return bool.

------------------------------------------------------------------------
r24680 | fdrake | 2001-12-12 07:20:34 +0100 (Wed, 12 Dec 2001) | 3 lines

Wrapped a long line.
Converted to use "".startswith() to avoid slicing (& temp string creation).

------------------------------------------------------------------------
r18934 | montanaro | 2001-01-21 00:34:12 +0100 (Sun, 21 Jan 2001) | 2 lines

more __all__ updates

...

The story is similar for optparse, only that the low maintenance
mode has started in 2006.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From brett at python.org  Tue Sep 15 18:47:03 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 09:47:03 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151628.n8FGSolQ026510@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4AAEBFEC.7060507@voidspace.org.uk> <4AAEC59C.7060605@egenix.com> 
	<200909151029.n8FATHmq028625@theraft.openend.se> <mal@egenix.com> 
	<4AAF78D9.20107@egenix.com>
	<200909151455.n8FEt3F1019480@theraft.openend.se> 
	<jnoller@gmail.com>
	<4222a8490909150759w2440d1e0odc90b9fce68872cb@mail.gmail.com> 
	<200909151628.n8FGSolQ026510@theraft.openend.se>
Message-ID: <bbaeab100909150947q5d4e59afq1ab3c52571e3f2f9@mail.gmail.com>

On Tue, Sep 15, 2009 at 09:28, Laura Creighton <lac at openend.se> wrote:
> In a message of Tue, 15 Sep 2009 10:59:44 EDT, Jesse Noller writes:
>>On Tue, Sep 15, 2009 at 10:55 AM, Laura Creighton <lac at openend.se> wrote:
>>> I am all for breaking up the stdlib logically, via tagging. ?I am
>>> worried about physically breaking things up, though. ?How are we
>>> going to guarantee that imports still work after the breakup?
>>>
>>> Laura
>>
>>What?
>>
>>We run the same unit tests we run today.
>
> I thought that people were proposing changing the directory structure
> of the stdlib.

People are getting ahead of themselves with the stdlib breakout
proposal. There will be a PEP explaining how we proposed to have it
work for CPython, what changes to the build will be, etc. But the
download for CPython will not change nor will a checkout in a major
way as we will simply require the forest extension or subrepos from
Mercurial.

For now, let's please refrain on worrying about this topic until there
is a proposal to discuss.

-Brett

From p.f.moore at gmail.com  Tue Sep 15 18:55:31 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Sep 2009 17:55:31 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAF84C4.20204@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<4AAF78D9.20107@egenix.com> <4AAF84C4.20204@voidspace.org.uk>
Message-ID: <79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>

2009/9/15 Michael Foord <michael at voidspace.org.uk>:
> Right - but part of the specific problem with optparse is that in many
> situations it does a very inadequate job (i.e. it "it does what it is meant
> to do" but not what many people "need it to do") and is designed in such a
> way that *required* functionality can't be added in a backwards compatible
> way.
>
> That is not "good code" (by my reckoning).

As such, optparse needs to change (assuming that the stdlib *should*
be "good code"). There are requirements (feature requests at least,
possibly even bugs) which need addressing.

If no maintainer can be found to do this, then optparse is de facto
dead and unmaintained.(Laura's B-2). No-one has made a statement about
what should be done with B-2 code in the stdlib, but I can see good
arguments for allowing the possibility of dropping it in favour of a
replacement library.

If someone wants to argue that optparse is class "C" (ie, it is
"finished" and "complete") then that's a different matter. Requests
for optparse to support required arguments, or to better support user
extension, would then be closed "won't fix - this is by design". And
in that case, it *is* more or less by definition "good code" - it
fulfills its aims, it just doesn't aim to do what Michael states many
people "need it to do" above. (Personally, I don't see that code can
be described as "good" if it doesn't aim to do what people need, but
maybe someone wants to argue this).

I think the issue with argparse vs optparse is that argparse appears
to be intended as "optparse done right" (I can't comment on whether it
is, just that's what it looks like). So having both in the stdlib
looks like duplication (far more so than having getopt along with
either). If optparse was still evolving, it should be possible to
devise some way of merging the two (possibly with API breakages,
admittedly). The dilemma here seems to be that optparse won't change,
and argparse offers extra functionality that people actually want (and
would like in the stdlib). Something needs to give.

Paul.

From michael at voidspace.org.uk  Tue Sep 15 18:59:32 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 17:59:32 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>	
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>	
	<4AAEC59C.7060605@egenix.com>	
	<200909151029.n8FATHmq028625@theraft.openend.se>	
	<4AAF78D9.20107@egenix.com> <4AAF84C4.20204@voidspace.org.uk>
	<79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>
Message-ID: <4AAFC7F4.8080907@voidspace.org.uk>

Paul Moore wrote:
> 2009/9/15 Michael Foord <michael at voidspace.org.uk>:
>   
>> Right - but part of the specific problem with optparse is that in many
>> situations it does a very inadequate job (i.e. it "it does what it is meant
>> to do" but not what many people "need it to do") and is designed in such a
>> way that *required* functionality can't be added in a backwards compatible
>> way.
>>
>> That is not "good code" (by my reckoning).
>>     
>
> As such, optparse needs to change (assuming that the stdlib *should*
> be "good code"). There are requirements (feature requests at least,
> possibly even bugs) which need addressing.
>
> If no maintainer can be found to do this, then optparse is de facto
> dead and unmaintained.(Laura's B-2). No-one has made a statement about
> what should be done with B-2 code in the stdlib, but I can see good
> arguments for allowing the possibility of dropping it in favour of a
> replacement library.
>
> If someone wants to argue that optparse is class "C" (ie, it is
> "finished" and "complete") then that's a different matter. Requests
> for optparse to support required arguments, or to better support user
> extension, would then be closed "won't fix - this is by design". And
> in that case, it *is* more or less by definition "good code" - it
> fulfills its aims, it just doesn't aim to do what Michael states many
> people "need it to do" above. (Personally, I don't see that code can
> be described as "good" if it doesn't aim to do what people need, but
> maybe someone wants to argue this).
>
> I think the issue with argparse vs optparse is that argparse appears
> to be intended as "optparse done right" (I can't comment on whether it
> is, just that's what it looks like). So having both in the stdlib
> looks like duplication (far more so than having getopt along with
> either). If optparse was still evolving, it should be possible to
> devise some way of merging the two (possibly with API breakages,
> admittedly). The dilemma here seems to be that optparse won't change,
> and argparse offers extra functionality that people actually want (and
> would like in the stdlib). Something needs to give.
>   


Seems like a good summary. I hope there will be a PEP from Steven for 
the inclusion of argparse. It's all a bit of a moot debate because we 
don't even need to decide whether to remove optparse now - we can start 
it down the deprecation road and *possibly* remove it eventually...

Personally I would like to see dead modules removed (whilst some would 
like to see the whole standard library dead ;-) - but our deprecation 
process is deliberately slow so that issues like this can be deliberated 
on a case by case basis over a period of years.

Michael

> Paul.
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From lac at openend.se  Tue Sep 15 19:02:09 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Sep 2009 19:02:09 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: Message from Collin Winter <collinw@gmail.com> of "Tue,
	15 Sep 2009 14:38:20 -0000."
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
Message-ID: <200909151702.n8FH29uv029039@theraft.openend.se>

In a message of Tue, 15 Sep 2009 14:38:20 -0000, Collin Winter writes:

<stuff that indicates that  my users shouldn't be changing versions
of Python without going through some sort of migration process>

This is what happens every time /usr/bin/env python changes.

<lots of stuff indicating that optparse really should be removed
 on the grounds that it is poorly written, still buggy, and 
 unmaintained>

Ok, so it's type B-2, the maintainer's nightmare.  It probably fits
in the category of 'things that never should be in the library in
the first place'.  It's an embarassment.  I notice that you aren't
talking about getopt, though.

And there may be things that are so badly written that we actually want
to throw them out of the standard library before people start using them.
A note 'this module only exists for backwards compatibility, we
recommend that you use XXX instead' will not be good enough for the
purpose, we really want to stamp out the use of this mistake as soon
and as well as possible.  This could happen.

But, as much as I dislike optparse, I don't think that this is the module
where that happened.  I will be happy when people stop using it, but
I don't think that using it is so hazardous that we should force all
the optparse users to rewrite all their code.

final quote:

>To speak more personally, and specifically to the issue of
>getopt/optparse vs argparse: at Google, I'm part of the Python
>readability team, which helps train the large numbers of Python
>developers that the company produces. Part of this job involves
>conducting detailed code reviews for new Python programmers,
>explaining both Google style and idiomatic Python code generally,
>suggesting library A over hand-written solution B. I am, frankly,
>embarrassed whenever I have to explain the difference between getopt
>and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess,
>string.* vs str.*, etc. I cannot imagine how embarrassed I will be
>when I have to explain why the standard library includes getopt,
>optparse and argparse.

You need to practice saying 'argparse is the preferred way to do
things; getopt and optparse are maintained for compatibility reasons'
until you can do this without embarassment.

It also sounds to me as if you would be far less embarassed explaining
to Google employees why they have to go off and rewrite all their old
code which uses getopt than explaining to them why we have getopt,
optparse, and argparse in the standard library.  I find this very odd.

Laura

>
>Collin Winter

From brett at python.org  Tue Sep 15 19:19:05 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 10:19:05 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com> 
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk> 
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk> 
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> 
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
Message-ID: <bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>

On Tue, Sep 15, 2009 at 07:38, Collin Winter <collinw at gmail.com> wrote:
> On Tue, Sep 15, 2009 at 10:29 AM, Laura Creighton <lac at openend.se> wrote:
[snip]
>> As far as I can tell most software packages go through three stages:
> [snip stage descriptions]
>> Now software in Stage A doesn't need maintenance so much as development
>> in the first place. ?And software in stage B requires a lot of
>> maintainance. ?But most software in stage C requires little or no
>> maintenance, precisely because it is unchanging. ?So, if you decide to
>> change how Exceptions are inherited, for instance, you may break a
>> whole lot of Stage C: code, but fixing them is part of the general
>> problem of 'fixing exceptions everywhere' not part of 'fixing getopt'.
>>
>> Now I think that there is some confusion here between packages who
>> are in Stage C: ?and packages who are in Stage B-2: ?around here.
>> The B-2 packages are probably the core developers greatest headache.
>> But I don't see the C packages as being troublesome at all. ?If you
>> don't like them, it isn't because of the work that maintaining is
>> costing you. ?It may be that hatred for the B-2's has become a general
>> hatred of all packages with no maintainers, which is an understandable
>> mistake. ?But from reading this list I get the distinct impression that
>> some people just hate C:s _precisely because they are old and unchanging_,
>> and would continue to hate them for that reason even if I was in some
>> way able to guarantee that they would never need any maintenance ever
>> again. ?These people are condemning the packages I love best for the
>> reason they are the packages I love best. ?And that is the attitude
>> I would like to change.
>
> Speaking as a core developer, I disagree. The problem with
> unmaintained (in your terms, B-2) or intentionally-frozen (C-*)
> packages is that they make it difficult for us to evolve and adapt
> Python the language and Python the standard library: if no-one is
> willing/available to update the code to account for language/library
> changes, the frozen package will become pinned to a specific
> known-good version (or range of versions) of Python. Over time, that
> version of Python will become uncommon (as distros phase it out) and
> unsupported (as python-dev end-of-lifes it). This is a problem for
> users of that package, who may wish to use newer version of Python for
> performance or bug-fix reasons, and it is also a problem for
> python-dev, since those frozen packages create inertia.
>
[snip]
>> It boils down to a matter of trust. ?My customers trust me to not give
>> them ticking time bombs that will all stop working one day, and I trust
>> you not to go about gratuitously removing perfectly working code that
>> is quietly sitting there, not needing any changes, and not bothering
>> anybody. ?When you break that contract with me, my customers suffer,
>> I suffer, and the people who said 'You shouldn't have coded it in
>> Python in the first place, but picked a mature language like Java'
>> are completely vindicated.
>
> The systems you have written for your customers are not autonomous
> agents; presumably, you have not written code like "if today.year ==
> 2011: sudo apt-get upgrade python" into these systems that would
> change the version of Python running without anyone asking. Human
> beings control these upgrades. If the human being performs the upgrade
> blindly, without taking appropriate risk mitigation steps, there's
> very little that we can do to protect them. You left out a key element
> in the chain of trust above: presumably, you trust your customers not
> to violate the minimum requirements you've set out for the software
> you've written for them. If you say "this software requires 2GB of
> RAM" and the customer later decides to try running the machines with
> only 256MB to save money instead, that's not your fault: it's theirs.
> Likewise, if you say "this software runs on Python 2.5" and they
> blindly install Python 3.1 instead, that's not your fault: it's
> theirs.
>
> As to the matter of Java's deprecation policy, I don't regard it as
> "mature": I regard it as a sign of different requirements. Because you
> can't know what browser version or JRE version a user's desktop is
> running, stability is paramount for Java; "write once, run anywhere"
> is not free, it has its costs. As Frank Wierzbicki has said (either in
> this thread or the other one about argparse), the inability to ever
> remove code from the libraries makes life difficult for Java's
> developers -- who have to maintain this code -- as well as for
> everyday Java engineers, who have to learn to navigate this maze of
> deprecated vs non-deprecated solutions to the same problem. Java shops
> generally end up with a list of Approved Java Classes so that new
> hires and old pros alike don't get tripped up.
>
> In Python, we don't have the luxury of a paid staff to work on our
> libraries, to maintain the crufty, fragile,
> we'd-like-to-get-rid-of-you-but-maybe-someone's-using-you-we-don't-really-know
> modules. We rely almost exclusively on volunteer contributions, and
> it's tough to find volunteers to work on crap code. It's one thing to
> choose not to change something; it's another thing entirely not to be
> able to change something, to have your hands tied by code you can't
> see and no-one will change. As Python development slows, as stability
> gets confused for permanence and stasis, I predict it will be harder
> to attract enthusiastic, eager contributors. After all, who wants to
> work on something you're not allowed to modify?
>
> To speak more personally, and specifically to the issue of
> getopt/optparse vs argparse: at Google, I'm part of the Python
> readability team, which helps train the large numbers of Python
> developers that the company produces. Part of this job involves
> conducting detailed code reviews for new Python programmers,
> explaining both Google style and idiomatic Python code generally,
> suggesting library A over hand-written solution B. I am, frankly,
> embarrassed whenever I have to explain the difference between getopt
> and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess,
> string.* vs str.*, etc. I cannot imagine how embarrassed I will be
> when I have to explain why the standard library includes getopt,
> optparse and argparse.

I agree with Collin in that we have to keep perspective on how this
impacts active core contributors and potential future ones. Like all
core developers I only have a small amount of free time in my life to
spend working on Python. That means I have to prioritize what patches
I review, what bugs I try to fix, etc., because frankly there are too
many for me and all of the active contributors to look at (I could say
how that is part of why I think the stdlib needs to slim down a
little, but that can wait for a PEP proposing what to trim and why).
So if I am browsing the issue tracker trying to help out, I can come
across getopt and optparse issues. Now I can obviously work on those
and fix bugs, but I also happen to know that argparse is out there,
generally well received by the community, and has been requested for
addition to the stdlib for years (the crowd was rather enthusiastic
during Steven's lightning talk at PyCon 2010 about argparse). It's a
little deflating to work on code that I know the community has deemed
mediocre compared to something out in the wild. Why should I spend my
time on getopt and optparse when I could try to get argparse into the
stdlib and spend my precious free time on some code I know will
ultimately serve the community better than the current code? Or how
about code that slipped in years ago and has not held up well in terms
of design? That's even worse because of the amount of time I have to
burn trying to untangle some coding mess that we have inherited from
Python's history.

I don't want to feel like I am wasting my time when I work on Python
stuff. And I am sure the other core developers don't either. And this
is partially why we have modules that end up with a laundry list of
bug reports and patches languishing in the issue tracker. Nor do I
want Python's reputation tarnished, as Collin pointed out, by having
old modules sitting there in the standard library.

Which gives another reason why argparse is attractive; Steven is
already a core developer and will keep argparse maintained, letting me
slowly ignore getopt/optparse issues guilt-free while I work on other
issues that actually bring me some form of joy.

-Brett

From michael at voidspace.org.uk  Tue Sep 15 19:29:42 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 18:29:42 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFC2DB.30109@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>	<79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>	<4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>
	<4AAFC2DB.30109@egenix.com>
Message-ID: <4AAFCF06.5000802@voidspace.org.uk>

M.-A. Lemburg wrote:
> Jesse Noller wrote:
>   
>> On Tue, Sep 15, 2009 at 12:16 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>     
>>> 2009/9/15 M.-A. Lemburg <mal at egenix.com>:
>>>       
>>>> Laura Creighton wrote:
>>>>         
>>>>> So what do you think of this proposal?
>>>>>           
>>>> Good write-up and very much to the point.
>>>>
>>>> [Executive Summary:
>>>> Code that hardly needs any changes, because it does what it's meant
>>>> to do, is good code, not bad code. And it causes only minimal
>>>> maintenance effort, so it's actually something core developer should
>>>> welcome rather than fight against.]
>>>>
>>>> I'd only change the tag "dead-as-a-doornail" to "complete, proven and
>>>> stable". Sounds more accurate.
>>>>         
>>> Yes, I like both the summary, and the proposal (that standard library
>>> code be tagged with details like its status and its maintainer).
>>>
>>> I was in the "stdlib needs to evolve" camp, but this has clarified the
>>> other side of the argument, and changed my mind.
>>>
>>> I'm still in favour of new modules being added to the stdlib, and
>>> existing modules being updated, but I support the idea of stage C
>>> (dead as a doornail/complete, proven, stable) modules being retained
>>> indefinitely. I'm not sure what the implications of this position
>>> would be in the case of argparse vs optparse (optparse doesn't seem to
>>> be stage C, so maybe removing it in favour of argparse is an option)
>>> but I like the fact that this proposal gives us terminology on which
>>> we can base the discussion.
>>>
>>> Paul.
>>>       
>> There is no, no such thing as a "dead/complete" module. It does not
>> exist. Any time there is a grammar change, a new reserved keyword, or
>> some other functionality change a "dead" module comes back to life and
>> has to be maintained.
>>
>> Can we please not treat "dead/complete" modules as if they have no
>> maintenance burden, or drag down the code base? The reason I avoided
>> that terminology in the first place is that there is no such thing as
>> code with zero cost.
>>     
>
> We're not treating them like that. All along we said, there is
> /very little/ maintenance needed.
>
> Besides, changes such as keyword fixes, grammar changes, style
> changes, etc. get applied to all stdlib modules, so these fixes
> don't really count as module-specific maintenance. Most of these
> fixes are done via a script or grep/sed anyway, so the cost per
> module is very small.
>
> You can have a look at the getopt SVN log to get an impression
> of just how much effort it takes to keep that module running.
>
> These are all changes applied to the module over a period of 7 years:
>   

This doesn't begin to cover the whole cost of keeping a library and its 
tests in Python. There are all the closed issues that someone has had to 
look into and respond to. Every time a developer runs all of the Python 
tests he pays some cost for every test of every module. Every time 
someone packages, distributes or downloads Python they pay some cost for 
every byte in Python. Every time someone translates the documentation, 
or updates the documentation (which Georg did recently for *every* 
module in the standard library) they pay a cost for every language 
feature and module they touch - or have to make a conscious decision not 
to touch.

There is also a cost in keeping bad modules in the library (something 
that has already been touched on) in user frustration. There is of 
course a cost in removing modules too - documentation and tutorials go 
out of date and need changing. There is definitely an inertia in favour 
of not removing modules - but there is also a hidden cost in keeping 
them. All of this needs to be weighed (carefully).

Michael

> ------------------------------------------------------------------------
> r67572 | georg.brandl | 2008-12-05 10:23:14 +0100 (Fri, 05 Dec 2008) | 2 lines
>
> #4458: recognize "-" as an argument, not a malformed option in gnu_getopt().
>
> ------------------------------------------------------------------------
> r31261 | akuchling | 2003-02-06 20:52:56 +0100 (Thu, 06 Feb 2003) | 4 lines
>
> A few naughty external scripts do 'raise getopt.error, "blah"', and
>    now crash because two arguments are expected.  Add a default value
>    to keep those scripts running.
>
> ------------------------------------------------------------------------
> r27929 | loewis | 2002-08-04 19:22:59 +0200 (Sun, 04 Aug 2002) | 2 lines
>
> Add encoding declaration.
>
> ------------------------------------------------------------------------
> r27780 | jackjansen | 2002-07-26 13:34:49 +0200 (Fri, 26 Jul 2002) | 2 lines
>
> Use os.environ.get() in stead of os.getenv() (which is platform-dependent).
>
> ------------------------------------------------------------------------
> r27645 | tim_one | 2002-07-16 23:35:23 +0200 (Tue, 16 Jul 2002) | 2 lines
>
> Whitespace normalization.
>
> ------------------------------------------------------------------------
> r27095 | montanaro | 2002-06-07 05:26:43 +0200 (Fri, 07 Jun 2002) | 2 lines
>
> gnu_getopt should be exported in __all__
>
> ------------------------------------------------------------------------
> r27077 | loewis | 2002-06-06 20:14:50 +0200 (Thu, 06 Jun 2002) | 2 lines
>
> Use isinstance for the type check, use booleans.
>
> ------------------------------------------------------------------------
> r27068 | loewis | 2002-06-06 12:58:36 +0200 (Thu, 06 Jun 2002) | 2 lines
>
> Patch 473512: add GNU style scanning as gnu_getopt.
>
> ------------------------------------------------------------------------
> r26191 | tim_one | 2002-04-05 00:55:58 +0200 (Fri, 05 Apr 2002) | 2 lines
>
> Convert a pile of obvious "yes/no" functions to return bool.
>
> ------------------------------------------------------------------------
> r24680 | fdrake | 2001-12-12 07:20:34 +0100 (Wed, 12 Dec 2001) | 3 lines
>
> Wrapped a long line.
> Converted to use "".startswith() to avoid slicing (& temp string creation).
>
> ------------------------------------------------------------------------
> r18934 | montanaro | 2001-01-21 00:34:12 +0100 (Sun, 21 Jan 2001) | 2 lines
>
> more __all__ updates
>
> ...
>
> The story is similar for optparse, only that the low maintenance
> mode has started in 2006.
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From collinw at gmail.com  Tue Sep 15 19:48:18 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 15 Sep 2009 13:48:18 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151702.n8FH29uv029039@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
Message-ID: <43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>

On Tue, Sep 15, 2009 at 1:02 PM, Laura Creighton <lac at openend.se> wrote:
> In a message of Tue, 15 Sep 2009 14:38:20 -0000, Collin Winter writes:
>
> <stuff that indicates that ?my users shouldn't be changing versions
> of Python without going through some sort of migration process>
>
> This is what happens every time /usr/bin/env python changes.

I do not understand. Why is "/usr/bin/env python" changing so
frequently on these systems? Why is it being changed without
consideration of the requirements of the software running on top of
it?

> final quote:
>
>>To speak more personally, and specifically to the issue of
>>getopt/optparse vs argparse: at Google, I'm part of the Python
>>readability team, which helps train the large numbers of Python
>>developers that the company produces. Part of this job involves
>>conducting detailed code reviews for new Python programmers,
>>explaining both Google style and idiomatic Python code generally,
>>suggesting library A over hand-written solution B. I am, frankly,
>>embarrassed whenever I have to explain the difference between getopt
>>and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess,
>>string.* vs str.*, etc. I cannot imagine how embarrassed I will be
>>when I have to explain why the standard library includes getopt,
>>optparse and argparse.
>
> You need to practice saying 'argparse is the preferred way to do
> things; getopt and optparse are maintained for compatibility reasons'
> until you can do this without embarassment.

I am embarrassed explaining why the core Python team, of which I am a
member, would consent to indefinitely maintain multiple, separate,
mutually-incompatible systems that all have the same purpose without
deprecating any of them. I do not understand why getopt was not
deprecated when optparse was introduced seven years ago; perhaps
someone can shed historical perspective on that decision. Pending that
elaboration, I do not understand why we would repeat that mistake
again with argparse now, or with some other module in the future.

> It also sounds to me as if you would be far less embarassed explaining
> to Google employees why they have to go off and rewrite all their old
> code which uses getopt than explaining to them why we have getopt,
> optparse, and argparse in the standard library. ?I find this very odd.

I don't find "we found a better way" embarrassing. I don't find having
a single, unified, best-of-breed solution embarrassing. I don't find
having multiple years of advance notice to update your code
embarrassing (which is vastly longer than the deprecation windows we
have internally). I don't find consolidating developer time, energy
and attention embarrassing. I don't find progress embarrassing.

The commonly-expressed idea behind the stdlib is that it represents
best-of-breed code: it should have an independent userbase first, it
should have proven itself in the wild before it is allowed into the
stdlib. "Best" comes with baggage: it means that other solutions are
worse, and it admits the possibility that other code will someday
overtake the current solution to *become* best-of-breed. That was the
motivation then for including optparse: it was a more Pythonic way of
doing argument parsing, as opposed to getopt's faithful recreation of
C's limitations and restrictions. That is the motivation now for
including argparse: optparse is inflexible and doesn't support useful
functionality. We have to be open to the idea that we may have found a
new "best".

Collin Winter

From solipsis at pitrou.net  Tue Sep 15 19:54:23 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 19:54:23 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
Message-ID: <1253037263.5678.12.camel@localhost>

Le mardi 15 septembre 2009 ? 10:19 -0700, Brett Cannon a ?crit :
> (the crowd was rather enthusiastic
> during Steven's lightning talk at PyCon 2010 about argparse).

Is it a typo or a rhetorical trick? ;)

> Nor do I
> want Python's reputation tarnished, as Collin pointed out, by having
> old modules sitting there in the standard library.

Python's reputation will certainly get tarnished, however, if
widely-used modules get deprecated and removed. We will already get some
damage with all of the py3k changes. No need to worsen our case, IMO.

Regards

Antoine.


PS : Perhaps I'm the only one bothered by this, but it would be nice if
people trimmed quoted messages a bit more. Having to scroll 3 pages
before reading the actual reply is tedious.



From mal at egenix.com  Tue Sep 15 20:06:18 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 20:06:18 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>
	<4AAF84C4.20204@voidspace.org.uk>
	<79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>
Message-ID: <4AAFD79A.5050103@egenix.com>

Paul Moore wrote:
> 2009/9/15 Michael Foord <michael at voidspace.org.uk>:
>> Right - but part of the specific problem with optparse is that in many
>> situations it does a very inadequate job (i.e. it "it does what it is meant
>> to do" but not what many people "need it to do") and is designed in such a
>> way that *required* functionality can't be added in a backwards compatible
>> way.
>>
>> That is not "good code" (by my reckoning).
> 
> As such, optparse needs to change (assuming that the stdlib *should*
> be "good code"). There are requirements (feature requests at least,
> possibly even bugs) which need addressing.
> 
> If no maintainer can be found to do this, then optparse is de facto
> dead and unmaintained.(Laura's B-2). No-one has made a statement about
> what should be done with B-2 code in the stdlib, but I can see good
> arguments for allowing the possibility of dropping it in favour of a
> replacement library.
> 
> If someone wants to argue that optparse is class "C" (ie, it is
> "finished" and "complete") then that's a different matter. Requests
> for optparse to support required arguments, or to better support user
> extension, would then be closed "won't fix - this is by design". And
> in that case, it *is* more or less by definition "good code" - it
> fulfills its aims, it just doesn't aim to do what Michael states many
> people "need it to do" above. (Personally, I don't see that code can
> be described as "good" if it doesn't aim to do what people need, but
> maybe someone wants to argue this).

Please note that adding functionality to such class "C" module
is still allowed - provided it doesn't break anything. This has
been done with optparse and getopt a few times already and
is common practice with other such modules as well.

In Laura's particular case, there does appear to be an easy
solution which adds just that one feature:

http://code.activestate.com/recipes/573441/

Things do get problematic if you want to approach a problem
from a whole new angle, e.g. say you want to have XML DOM
parsing instead of SAX parsing.

In such a case, adding a whole new module would be better.
Even though you're still parsing XML in both cases, the new
module would provide a new method to do so. Much like the
urllib2 provides a new and different way to handle URL
fetching compared to urllib.

There's also a third case: If a module cannot be updated
or extended to support a major new feature mandated by
the Python language, such as Unicode support, then there
is little choice other than to replace it with a new module.

This has happened with the pcre module that used to drive
the re module - there was no way to get Unicode into pcre.
The change was not really noticeable to the users, though,
since the new module implemented the same API (and extended
it).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Tue Sep 15 20:15:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 20:15:14 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
Message-ID: <1253038514.5678.32.camel@localhost>


Le mardi 15 septembre 2009 ? 13:48 -0400, Collin Winter a ?crit :
> 
> The commonly-expressed idea behind the stdlib is that it represents
> best-of-breed code: it should have an independent userbase first, it
> should have proven itself in the wild before it is allowed into the
> stdlib.

Asking libraries to be proven in the wild sounds like a good idea, but
it promotes disruptive changes (replacing a module with another one)
rather than evolutionary changes. It also means that we get potentially
bloated or bizarrely idiomatic packages, because they weren't subject to
the same standards of simplicity / austerity that we value in the stdlib
(multiprocessing comes to mind, doesn't it? how much time did we lose
because of its byzantine API and implementation? how robust and
maintainable is it, even now, although the original PyPI package was
acclaimed?).

Therefore, I think it shouldn't be imposed as a general rule, rather a
guideline.

For example, Michael's work on unittest (a work of evolutionary changes)
is much better news, IMO, that someone proposing to include nose in the
stdlib. And I say that as a nose user. I don't want to maintain nose in
the stdlib.

>  "Best" comes with baggage: it means that other solutions are
> worse, and it admits the possibility that other code will someday
> overtake the current solution to *become* best-of-breed.

Of course. But it is also a double-sided argument because, if each new
package remains the best-of-breed during only 2 years after it is
integrated into the stdlib, we will spend a lot of time considering new
packages for inclusion, deprecating other ones, and ultimately confusing
the hell of our users.

Regards

Antoine.



From rdmurray at bitdance.com  Tue Sep 15 20:16:05 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 14:16:05 -0400 (EDT)
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151702.n8FH29uv029039@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<200909151702.n8FH29uv029039@theraft.openend.se>
Message-ID: <Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 19:02, Laura Creighton wrote:
> And there may be things that are so badly written that we actually want
> to throw them out of the standard library before people start using them.
> A note 'this module only exists for backwards compatibility, we
> recommend that you use XXX instead' will not be good enough for the
> purpose, we really want to stamp out the use of this mistake as soon
> and as well as possible.  This could happen.

On a smaller scale, it did a while ago.  Consider ConfigParser and
SafeConfigParser.  Although in that case the docs about the
deprecation are not even as good as they could be (see
http://bugs.python.org/issue6517).

--David

From brett at python.org  Tue Sep 15 20:21:26 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 11:21:26 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253037263.5678.12.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com> 
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com> 
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
Message-ID: <bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:54, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 10:19 -0700, Brett Cannon a ?crit :
>> (the crowd was rather enthusiastic
>> during Steven's lightning talk at PyCon 2010 about argparse).
>
> Is it a typo or a rhetorical trick? ;)
>

Typo. =) I meant 2009.

>> Nor do I
>> want Python's reputation tarnished, as Collin pointed out, by having
>> old modules sitting there in the standard library.
>
> Python's reputation will certainly get tarnished, however, if
> widely-used modules get deprecated and removed. We will already get some
> damage with all of the py3k changes. No need to worsen our case, IMO.
>

I don't think slow deprecations will do that. This happened with
md5/sha when we introduced hashlib and the Mercurial guys were the
only ones I ever heard complain.

> Regards
>
> Antoine.
>
>
> PS : Perhaps I'm the only one bothered by this, but it would be nice if
> people trimmed quoted messages a bit more. Having to scroll 3 pages
> before reading the actual reply is tedious.

Gmail color-codes so I simply scroll until I no longer see purple.

-Brett

From michael at voidspace.org.uk  Tue Sep 15 20:21:55 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 19:21:55 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFD79A.5050103@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>
	<4AAF84C4.20204@voidspace.org.uk>
	<79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>
	<4AAFD79A.5050103@egenix.com>
Message-ID: <4AAFDB43.8050509@voidspace.org.uk>

M.-A. Lemburg wrote:
> [snip...]
> Please note that adding functionality to such class "C" module
> is still allowed - provided it doesn't break anything. This has
> been done with optparse and getopt a few times already and
> is common practice with other such modules as well.
>
> In Laura's particular case, there does appear to be an easy
> solution which adds just that one feature:
>
> http://code.activestate.com/recipes/573441/
>
> Things do get problematic if you want to approach a problem
> from a whole new angle, e.g. say you want to have XML DOM
> parsing instead of SAX parsing.
>
> In such a case, adding a whole new module would be better.
> Even though you're still parsing XML in both cases, the new
> module would provide a new method to do so. Much like the
> urllib2 provides a new and different way to handle URL
> fetching compared to urllib.
>
> There's also a third case: If a module cannot be updated
> or extended to support a major new feature mandated by
> the Python language, such as Unicode support, then there
> is little choice other than to replace it with a new module.
>
> This has happened with the pcre module that used to drive
> the re module - there was no way to get Unicode into pcre.
> The change was not really noticeable to the users, though,
> since the new module implemented the same API (and extended
> it).
>
>   

Modules are also deprecated wholesale where their functionality is 
replaced by an alternative API.

    md5 -> hashlib
    mimewriter -> email

This is already a normal part of the evolution of the Python standard 
library.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From michael at voidspace.org.uk  Tue Sep 15 20:23:41 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 19:23:41 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253038514.5678.32.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
Message-ID: <4AAFDBAD.7040906@voidspace.org.uk>

Antoine Pitrou wrote:
> [snip...]
> For example, Michael's work on unittest (a work of evolutionary changes)
> is much better news, IMO, that someone proposing to include nose in the
> stdlib. And I say that as a nose user. I don't want to maintain nose in
> the stdlib.
>   

Evolving an existing library, as a rule, is definitely better than 
replacing it with a new one. There is a cost involved in removing a 
library. It isn't always possible though to meet requirements with an 
existing API - as is the case with optparse / argparse.

>   
>>  "Best" comes with baggage: it means that other solutions are
>> worse, and it admits the possibility that other code will someday
>> overtake the current solution to *become* best-of-breed.
>>     
>
> Of course. But it is also a double-sided argument because, if each new
> package remains the best-of-breed during only 2 years after it is
> integrated into the stdlib, we will spend a lot of time considering new
> packages for inclusion, deprecating other ones, and ultimately confusing
> the hell of our users.
>   

The bar for including a module in the standard library is high (which is 
where the best of breed requirement comes from) because *we* do commit 
to maintain it. That doesn't mean we commit to maintaining things 
forever though (at least I can't find that promise in the documentation 
anywhere...).

Flipping modules every two years would of course be ridiculous - but we 
still don't guarantee that any module will remain forever.

I don't think I would go as far as Collin hinted (without necessarily 
meaning anything so radical) in maintaining the standard library as a 
constantly changing "current best of breed", but I agree with his other 
sentiments.

Michael


> Regards
>
> Antoine.
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Tue Sep 15 20:26:52 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 11:26:52 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253038514.5678.32.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com> 
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com> 
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com> 
	<1253038514.5678.32.camel@localhost>
Message-ID: <bbaeab100909151126m65e3c67as2d012da757c3c64a@mail.gmail.com>

On Tue, Sep 15, 2009 at 11:15, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Le mardi 15 septembre 2009 ? 13:48 -0400, Collin Winter a ?crit :
>>
>> The commonly-expressed idea behind the stdlib is that it represents
>> best-of-breed code: it should have an independent userbase first, it
>> should have proven itself in the wild before it is allowed into the
>> stdlib.
>
> Asking libraries to be proven in the wild sounds like a good idea, but
> it promotes disruptive changes (replacing a module with another one)
> rather than evolutionary changes. It also means that we get potentially
> bloated or bizarrely idiomatic packages, because they weren't subject to
> the same standards of simplicity / austerity that we value in the stdlib
> (multiprocessing comes to mind, doesn't it? how much time did we lose
> because of its byzantine API and implementation? how robust and
> maintainable is it, even now, although the original PyPI package was
> acclaimed?).
>
> Therefore, I think it shouldn't be imposed as a general rule, rather a
> guideline.
>
> For example, Michael's work on unittest (a work of evolutionary changes)
> is much better news, IMO, that someone proposing to include nose in the
> stdlib. And I say that as a nose user. I don't want to maintain nose in
> the stdlib.
>

I don't think anyone is suggesting that we always reach outside of the
standard library when evolution is desired. It happens that for
argparse that occurred, and that's partially because optparse did not
lend itself to extension. Michael's work was great that it fit right
in, but that also occurred because he was given commit privileges to
do it. Without it he would either have needed to wait for someone to
check his stuff in or start a fork to get things done. Another reason
we need to do what we can to get through patches on the issue tracker.

>> ?"Best" comes with baggage: it means that other solutions are
>> worse, and it admits the possibility that other code will someday
>> overtake the current solution to *become* best-of-breed.
>
> Of course. But it is also a double-sided argument because, if each new
> package remains the best-of-breed during only 2 years after it is
> integrated into the stdlib, we will spend a lot of time considering new
> packages for inclusion, deprecating other ones, and ultimately confusing
> the hell of our users.

We are not about to change modules every release.

-Brett

From michael at voidspace.org.uk  Tue Sep 15 20:27:25 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 19:27:25 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAE782D.8020800@voidspace.org.uk>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>
Message-ID: <4AAFDC8D.5050401@voidspace.org.uk>

R. David Murray wrote:
> On Tue, 15 Sep 2009 at 19:02, Laura Creighton wrote:
>> And there may be things that are so badly written that we actually want
>> to throw them out of the standard library before people start using 
>> them.
>> A note 'this module only exists for backwards compatibility, we
>> recommend that you use XXX instead' will not be good enough for the
>> purpose, we really want to stamp out the use of this mistake as soon
>> and as well as possible.  This could happen.
>
> On a smaller scale, it did a while ago.  Consider ConfigParser and
> SafeConfigParser.  Although in that case the docs about the
> deprecation are not even as good as they could be (see
> http://bugs.python.org/issue6517).
Two examples of deprecated modules in the Python documentation:

http://docs.python.org/library/md5.html
http://docs.python.org/library/mimewriter.html

The deprecation notice could be stronger / more prominent.

Michael

>
> --David
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From p.f.moore at gmail.com  Tue Sep 15 20:31:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Sep 2009 19:31:11 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253038514.5678.32.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
Message-ID: <79990c6b0909151131h1979d398h3ade03b1f995191@mail.gmail.com>

2009/9/15 Antoine Pitrou <solipsis at pitrou.net>:
>
> Le mardi 15 septembre 2009 ? 13:48 -0400, Collin Winter a ?crit :
>>
>> The commonly-expressed idea behind the stdlib is that it represents
>> best-of-breed code: it should have an independent userbase first, it
>> should have proven itself in the wild before it is allowed into the
>> stdlib.
>
> Asking libraries to be proven in the wild sounds like a good idea, but
> it promotes disruptive changes (replacing a module with another one)
> rather than evolutionary changes.

Two points here:

1. In response to Collin - the idea that the stdlib is best of breed
is common, and commonly stated, but it is actually not always the
case. For example, wsgiref is not intended as a best of breed WSGI
implementation, but as a reference implementation. Similarly with
SimpleHTTPServer - it's there for "batteries included" rather than
because it is better than Apache :-)

2. Antoine's point is a very good one - the "proven in the wild"
principle really only applies to libraries providing entirely new
functionality (and maybe not even there). Where there's an overlap
with existing stdlib functionality, having the stdlib steal ideas from
competing 3rd party solutions is probably better. (Of course, if the
stdlib module has no maintainer, there's a problem...)

Paul.

From solipsis at pitrou.net  Tue Sep 15 20:35:04 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 20:35:04 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFDC8D.5050401@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE782D.8020800@voidspace.org.uk>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>
	<4AAFDC8D.5050401@voidspace.org.uk>
Message-ID: <1253039704.5678.36.camel@localhost>

Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
> Two examples of deprecated modules in the Python documentation:
> 
> http://docs.python.org/library/md5.html
> http://docs.python.org/library/mimewriter.html
> 
> The deprecation notice could be stronger / more prominent.

Perhaps deprecated modules can get some kind of greyish style
(background and/or font). Georg, is it possible? :)

They could also be moved to a dedicated section. For example, md5 would
be inside "XX - deprecated modules" rather than "15 - Cryptographic
services".



From mal at egenix.com  Tue Sep 15 20:33:14 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 20:33:14 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFCF06.5000802@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>	<79990c6b0909150916s7c67bd6dte16f2beeb5b52ce5@mail.gmail.com>	<4222a8490909150921h595ff10byaa3004f1a778fbc3@mail.gmail.com>	<4AAFC2DB.30109@egenix.com>
	<4AAFCF06.5000802@voidspace.org.uk>
Message-ID: <4AAFDDEA.5070200@egenix.com>

Michael Foord wrote:
> M.-A. Lemburg wrote:
>>> Can we please not treat "dead/complete" modules as if they have no
>>> maintenance burden, or drag down the code base? The reason I avoided
>>> that terminology in the first place is that there is no such thing as
>>> code with zero cost.
>>>     
>>
>> We're not treating them like that. All along we said, there is
>> /very little/ maintenance needed.
>>
>> Besides, changes such as keyword fixes, grammar changes, style
>> changes, etc. get applied to all stdlib modules, so these fixes
>> don't really count as module-specific maintenance. Most of these
>> fixes are done via a script or grep/sed anyway, so the cost per
>> module is very small.
>>
>> You can have a look at the getopt SVN log to get an impression
>> of just how much effort it takes to keep that module running.
>>
>> These are all changes applied to the module over a period of 7 years:
>>   
> 
> This doesn't begin to cover the whole cost of keeping a library and its
> tests in Python.

It was meant a proof for the code being mature and causing
low maintenance costs.

> There are all the closed issues that someone has had to
> look into and respond to. Every time a developer runs all of the Python
> tests he pays some cost for every test of every module. Every time
> someone packages, distributes or downloads Python they pay some cost for
> every byte in Python. Every time someone translates the documentation,
> or updates the documentation (which Georg did recently for *every*
> module in the standard library) they pay a cost for every language
> feature and module they touch - or have to make a conscious decision not
> to touch.

Right, I didn't say there were no costs, but then how does the above
time compare to e.g. fixing two nasty bugs in a newly adopted module ?

You can easily spend a day or two trying to track down such bugs.
Certainly more than you spend on doing the things you mentioned
above for a module that doesn't have all that many issues.

> There is also a cost in keeping bad modules in the library (something
> that has already been touched on) in user frustration.

Nobody forces a user to use a "bad" module (for whatever meaning
the user associates with "bad"). That's what so great about Python:
there are a gazillion 3rd party modules out there waiting to
get used.

> There is of
> course a cost in removing modules too - documentation and tutorials go
> out of date and need changing. There is definitely an inertia in favour
> of not removing modules - but there is also a hidden cost in keeping
> them. All of this needs to be weighed (carefully).

Sure.

What I'm trying to say is that keeping a module in the stdlib
costs less than removing it - for everyone.

I've been maintaining our eGenix mx Series for more than a decade
now, so I do know a little about such costs, where to expect them
and where not.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From p.f.moore at gmail.com  Tue Sep 15 20:41:25 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Sep 2009 19:41:25 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFDBAD.7040906@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
Message-ID: <79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>

2009/9/15 Michael Foord <michael at voidspace.org.uk>:
> Evolving an existing library, as a rule, is definitely better than replacing
> it with a new one. There is a cost involved in removing a library. It isn't
> always possible though to meet requirements with an existing API - as is the
> case with optparse / argparse.

MAL pointed out http://code.activestate.com/recipes/573441/ - extended
optparse to allow definition of required options. Given that one of
the requirements that argparse is claimed to meet where optparse
doesn't is supporting required arguments, how come this simple recipe
hasn't been incorporated into optparse?

The optparse/argparse case seems to rest on the argument that optparse
cannot be extended to do what argparse does. It seems like this isn't
true for all requirements. (And maybe some others could be addressed
by judiciously deprecating support for specific internal details that
maybe should not have been documented in the first place...)

Paul.

From mal at egenix.com  Tue Sep 15 20:42:21 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 20:42:21 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFDB43.8050509@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>	<4AAF84C4.20204@voidspace.org.uk>	<79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>	<4AAFD79A.5050103@egenix.com>
	<4AAFDB43.8050509@voidspace.org.uk>
Message-ID: <4AAFE00D.2070800@egenix.com>

Michael Foord wrote:
> M.-A. Lemburg wrote:
>> [snip...]
>> Please note that adding functionality to such class "C" module
>> is still allowed - provided it doesn't break anything. This has
>> been done with optparse and getopt a few times already and
>> is common practice with other such modules as well.
>>
>> In Laura's particular case, there does appear to be an easy
>> solution which adds just that one feature:
>>
>> http://code.activestate.com/recipes/573441/
>>
>> Things do get problematic if you want to approach a problem
>> from a whole new angle, e.g. say you want to have XML DOM
>> parsing instead of SAX parsing.
>>
>> In such a case, adding a whole new module would be better.
>> Even though you're still parsing XML in both cases, the new
>> module would provide a new method to do so. Much like the
>> urllib2 provides a new and different way to handle URL
>> fetching compared to urllib.
>>
>> There's also a third case: If a module cannot be updated
>> or extended to support a major new feature mandated by
>> the Python language, such as Unicode support, then there
>> is little choice other than to replace it with a new module.
>>
>> This has happened with the pcre module that used to drive
>> the re module - there was no way to get Unicode into pcre.
>> The change was not really noticeable to the users, though,
>> since the new module implemented the same API (and extended
>> it).
>>
>>   
> 
> Modules are also deprecated wholesale where their functionality is
> replaced by an alternative API.
> 
>    md5 -> hashlib
>    mimewriter -> email
> 
> This is already a normal part of the evolution of the Python standard
> library.

And that's perfectly ok.

If the change is a single import statement, then I don't have a
problem with that.

I also don't have a problem with having a module called
optparse, which, under the hood, uses argparse to provide
the optparse API.

I do have a problem with having to revisit the design of
all scripts using a deprecated module in order to adapt
it to some new, apparently deemed better, stdlib module,
without any benefit, just in order to get the
implementation or script working again.

That's wasted time, really. And given that some modules are
in wide-spread use that scales up wasting the time of
thousands of other programmers out there.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From brett at python.org  Tue Sep 15 20:42:27 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 11:42:27 -0700
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253039704.5678.36.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com> 
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net> 
	<4AAFDC8D.5050401@voidspace.org.uk>
	<1253039704.5678.36.camel@localhost>
Message-ID: <bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>

If Twitter is any indication Georg is now ignoring this mailing list,
so I am adding him directly so he can answer Antoine's questions.

On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>> Two examples of deprecated modules in the Python documentation:
>>
>> http://docs.python.org/library/md5.html
>> http://docs.python.org/library/mimewriter.html
>>
>> The deprecation notice could be stronger / more prominent.
>
> Perhaps deprecated modules can get some kind of greyish style
> (background and/or font). Georg, is it possible? :)
>
> They could also be moved to a dedicated section. For example, md5 would
> be inside "XX - deprecated modules" rather than "15 - Cryptographic
> services".
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>

From jnoller at gmail.com  Tue Sep 15 20:44:08 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 14:44:08 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253038514.5678.32.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
Message-ID: <4222a8490909151144o332cd67eu5d00bb37ebe57c1@mail.gmail.com>

On Tue, Sep 15, 2009 at 2:15 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Le mardi 15 septembre 2009 ? 13:48 -0400, Collin Winter a ?crit :
>>
>> The commonly-expressed idea behind the stdlib is that it represents
>> best-of-breed code: it should have an independent userbase first, it
>> should have proven itself in the wild before it is allowed into the
>> stdlib.
>
> Asking libraries to be proven in the wild sounds like a good idea, but
> it promotes disruptive changes (replacing a module with another one)
> rather than evolutionary changes. It also means that we get potentially
> bloated or bizarrely idiomatic packages, because they weren't subject to
> the same standards of simplicity / austerity that we value in the stdlib
> (multiprocessing comes to mind, doesn't it? how much time did we lose
> because of its byzantine API and implementation? how robust and
> maintainable is it, even now, although the original PyPI package was
> acclaimed?).

Yup, multiprocessing is a perfect example of something popular in the
wild, but was rushed to inclusion because I proposed it late in the
cycle. If I had to do it again - as the guy who is still on the hook
for bug fixes, evolving it, and general maintenance, I'd not have
gotten it in there in the state it went in. I should have proposed it
earlier in the 2.6/3.0 process, and spent more time working on it.

Thanks for your vote though.

Today, I still fix bugs, work on improvements, etc. In my mind,
multiprocessing is a poor example because it was pulled in *so
quickly* - not because it was pulled in, and not because it has bugs.
But rather a bar that should have been met, was not due to time.

On the other hand - at least it has a maintainer (me) unlike some 50%
of the rest of the libraries. So it has some small thing going for it.
It also has tests. And documentation. And I continue to add to those
when I can.

How much of the rest of the standard libs can claim that?

jesse

From michael at voidspace.org.uk  Tue Sep 15 20:49:15 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 19:49:15 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFE00D.2070800@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>	<4AAF84C4.20204@voidspace.org.uk>	<79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>	<4AAFD79A.5050103@egenix.com>
	<4AAFDB43.8050509@voidspace.org.uk> <4AAFE00D.2070800@egenix.com>
Message-ID: <4AAFE1AB.6000303@voidspace.org.uk>

M.-A. Lemburg wrote:
> Michael Foord wrote:
>   
>> M.-A. Lemburg wrote:
>>     
>>> [snip...]
>>> Please note that adding functionality to such class "C" module
>>> is still allowed - provided it doesn't break anything. This has
>>> been done with optparse and getopt a few times already and
>>> is common practice with other such modules as well.
>>>
>>> In Laura's particular case, there does appear to be an easy
>>> solution which adds just that one feature:
>>>
>>> http://code.activestate.com/recipes/573441/
>>>
>>> Things do get problematic if you want to approach a problem
>>> from a whole new angle, e.g. say you want to have XML DOM
>>> parsing instead of SAX parsing.
>>>
>>> In such a case, adding a whole new module would be better.
>>> Even though you're still parsing XML in both cases, the new
>>> module would provide a new method to do so. Much like the
>>> urllib2 provides a new and different way to handle URL
>>> fetching compared to urllib.
>>>
>>> There's also a third case: If a module cannot be updated
>>> or extended to support a major new feature mandated by
>>> the Python language, such as Unicode support, then there
>>> is little choice other than to replace it with a new module.
>>>
>>> This has happened with the pcre module that used to drive
>>> the re module - there was no way to get Unicode into pcre.
>>> The change was not really noticeable to the users, though,
>>> since the new module implemented the same API (and extended
>>> it).
>>>
>>>   
>>>       
>> Modules are also deprecated wholesale where their functionality is
>> replaced by an alternative API.
>>
>>    md5 -> hashlib
>>    mimewriter -> email
>>
>> This is already a normal part of the evolution of the Python standard
>> library.
>>     
>
> And that's perfectly ok.
>
> If the change is a single import statement, then I don't have a
> problem with that.
>   

For md5 the API basically moved (although md5.new became hashlib.md5).

For MimeWriter there were (what looks like from a relatively casual 
look) fairly big API changes as well - even if they were only name 
changes (which is substantially similar to the requirements for moving 
from optparse to argparse).

Michael

> I also don't have a problem with having a module called
> optparse, which, under the hood, uses argparse to provide
> the optparse API.
>
> I do have a problem with having to revisit the design of
> all scripts using a deprecated module in order to adapt
> it to some new, apparently deemed better, stdlib module,
> without any benefit, just in order to get the
> implementation or script working again.
>
> That's wasted time, really. And given that some modules are
> in wide-spread use that scales up wasting the time of
> thousands of other programmers out there.
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From armin.ronacher at active-4.com  Tue Sep 15 20:44:04 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Tue, 15 Sep 2009 11:44:04 -0700
Subject: [stdlib-sig] Maintenance of optparse
Message-ID: <4AAFE074.5070107@active-4.com>

Hello list,

What started as a joke yesterday in the Bug Tracker and on Twitter now
evolved into a serious consideration.  In case the only reason for
optparse to go away is that it does not have a maintainer I would take
over that task.

In fact I would also implement missing features based on real-world
needs.  That would most likely mean that some of the changes in argparse
would end up in optparse as well.  Furthermore I would release a
Python-independent version on PyPI for compatibility with older Python
versions.

I would take over this task if the following criteria are met:

- argparse would not enter the standard library
- I'm allowed to modernize optparse after a discussion with python-dev
  in a backwards compatible way.
- I'm allowed to refactor the code
- make the i18n support of the module more pluggable which would allow
  specifying a custom translations instance instead of using the global
  gettext function.


Regards,
Armin

From brett at python.org  Tue Sep 15 20:51:43 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 11:51:43 -0700
Subject: [stdlib-sig] PEP time: "put up or shut up" on the stuff being
	discussed
Message-ID: <bbaeab100909151151u63d6b27ema2e2fcf96e514297@mail.gmail.com>

OK, I think everyone's opinions on adding argparse (Steven), breaking
out the standard library from python-dev (Frank), and potentially
pruning the standard library (Jesse?) has been heard and we are
starting to go in circles. At this point it's time for the respective
parties to go off and write their PEPs before these threads start to
get nasty (e.g. Antoine's comments about multiprocessing, while I am
sure not meant to be insulting, were probably not welcome by Jesse; I
know I would not have liked them about importlib).

People are free to keep discussing if they want obviously, but I am
going to stop participating and focus on helping with the PEPs where I
can.

-Brett

From michael at voidspace.org.uk  Tue Sep 15 21:01:14 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 20:01:14 +0100
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <4AAFE074.5070107@active-4.com>
References: <4AAFE074.5070107@active-4.com>
Message-ID: <4AAFE47A.40201@voidspace.org.uk>

Armin Ronacher wrote:
> Hello list,
>
> What started as a joke yesterday in the Bug Tracker and on Twitter now
> evolved into a serious consideration.  In case the only reason for
> optparse to go away is that it does not have a maintainer I would take
> over that task.
>
> In fact I would also implement missing features based on real-world
> needs.  That would most likely mean that some of the changes in argparse
> would end up in optparse as well.  Furthermore I would release a
> Python-independent version on PyPI for compatibility with older Python
> versions.
>
> I would take over this task if the following criteria are met:
>
> - argparse would not enter the standard library
>   

That can't be a pre-requisite as we can't tell if you will succeed until 
you actually look at the problem and see how possible it is. I would do 
a feasibility check.

For me the important features of argparse are:

* handling positional arguments including required positional arguments
* required options (which it looks like can be added easily - the above 
is less easy though)
* subcommands
> - I'm allowed to modernize optparse after a discussion with python-dev
>   
That would be true anyway, unless we had already decided to deprecate.

>   in a backwards compatible way.
> - I'm allowed to refactor the code
>   

Ditto.

> - make the i18n support of the module more pluggable which would allow
>   specifying a custom translations instance instead of using the global
>   gettext function.
>
>   
That sounds good to me. I don't know how 'mandatory' others feel the use 
of gettext is in il8n support in the standard library.

Michael

> Regards,
> Armin
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Tue Sep 15 21:09:11 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 21:09:11 +0200
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <4AAFE074.5070107@active-4.com>
References: <4AAFE074.5070107@active-4.com>
Message-ID: <1253041751.5678.46.camel@localhost>

Le mardi 15 septembre 2009 ? 11:44 -0700, Armin Ronacher a ?crit :
> 
> I would take over this task if the following criteria are met:
> 
> - I'm allowed to modernize optparse after a discussion with python-dev
>   in a backwards compatible way.
> - I'm allowed to refactor the code
> - make the i18n support of the module more pluggable which would allow
>   specifying a custom translations instance instead of using the global
>   gettext function.

These all look fine and even welcome.

> - argparse would not enter the standard library

This one doesn't. This is a kind of "you are either with me, or against
me" argument. While I would favour improving optparse over including
argparse, I don't think we can commit to never include argparse, just
because you asked for it.

Please note that if you do improve optparse, the bar for including
argparse will be much higher anyway. Perhaps people won't even ask for
it, seeing that optparse has become sufficient for them.

Regards

Antoine.



From armin.ronacher at active-4.com  Tue Sep 15 21:08:16 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Tue, 15 Sep 2009 12:08:16 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <4AAFE5EB.2040207@active-4.com>
References: <4AAFE074.5070107@active-4.com> <4AAFE47A.40201@voidspace.org.uk>
	<4AAFE5EB.2040207@active-4.com>
Message-ID: <4AAFE620.5090805@active-4.com>

Armin Ronacher wrote:
> Hi,
> 
> Michael Foord wrote:
>> That can't be a pre-requisite as we can't tell if you will succeed until
>> you actually look at the problem and see how possible it is. I would do
>> a feasibility check.
> Well assuming I succeed before feature freeze.  If I do not I failed and
> you're welcome to do whatever you want (of course including my patches
> to be part of the stdlib, but I would not continue to maintain it
> afterwards because I do not see the point then)
> 
>> * handling positional arguments including required positional arguments
>> * required options (which it looks like can be added easily - the above
>>   is less easy though)
>> * subcommands
> I can see not reason why it should not be possible to implement that in
> optparse without breaking backwards compatibility.
> 
>> That sounds good to me. I don't know how 'mandatory' others feel the use
>> of gettext is in il8n support in the standard library.
> Obviously I'm biased towards Web development where you have to keep
> everything stateless as good as possible, to allow different languages
> in the same process.  And there are good reasons why you would want to
> use optparse from inside a website (interactive shells come to mind).
> 
> I'm deeply disturbed by how the standard library is currently depending
> on global state (the locale module for example does not work at all in a
> web application).
> 
> 
> Regards,
> Armin
> 


From armin.ronacher at active-4.com  Tue Sep 15 21:09:43 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Tue, 15 Sep 2009 12:09:43 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <1253041751.5678.46.camel@localhost>
References: <4AAFE074.5070107@active-4.com>
	<1253041751.5678.46.camel@localhost>
Message-ID: <4AAFE677.3030501@active-4.com>

Hi,

Antoine Pitrou wrote:
> This one doesn't. This is a kind of "you are either with me, or against
> me" argument. While I would favour improving optparse over including
> argparse, I don't think we can commit to never include argparse, just
> because you asked for it.
I can see strong disagreement with me here which is why I would love to
change my point to "I would only *continue* to maintain optparse is
argparse does not end up in the standard library".  If both end up
there, maintenance from my side ends.


Regards,
Armin

From lac at openend.se  Tue Sep 15 21:10:58 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Sep 2009 21:10:58 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: Message from Paul Moore <p.f.moore@gmail.com> of "Tue,
	15 Sep 2009 19:41:25 BST."
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com> 
Message-ID: <200909151910.n8FJAwag005132@theraft.openend.se>

In a message of Tue, 15 Sep 2009 19:41:25 BST, Paul Moore writes:
>MAL pointed out http://code.activestate.com/recipes/573441/ - extended
>optparse to allow definition of required options. Given that one of
>the requirements that argparse is claimed to meet where optparse
>doesn't is supporting required arguments, how come this simple recipe
>hasn't been incorporated into optparse?
>
>The optparse/argparse case seems to rest on the argument that optparse
>cannot be extended to do what argparse does. It seems like this isn't
>true for all requirements. (And maybe some others could be addressed
>by judiciously deprecating support for specific internal details that
>maybe should not have been documented in the first place...)

Maybe, but this simple recipe only works for options which are
unconditionally required.  What I needed was a solution for options
that are conditionally required -- if you have option X then you
must have option Q.

But optparse is particularly unhappy about this, and about other
things.  If you type command --verbose --brief, and these are
toggles, then you will get the last one processed, brief in this
case.  There is no easy way to say 'now please roll over and die'
because you have an incompatible set of options.  I wrote code 
that changed optparse to do what I wanted, but boy, was it ugly.
I can maybe dig it up somewhere.  And when I went knocking, asking 
optparse's author about how he would recommend that I solve my 
problem using optparse in a way that was less ugly, I was told
that my problem wasn't to be solved using optparse.

So I would surmise that the reason this recipe isn't in the standard
library is optparse's author didn't want it in optparse.

>
>Paul.
>_______________________________________________
>stdlib-sig mailing list
>stdlib-sig at python.org
>http://mail.python.org/mailman/listinfo/stdlib-sig

From barry at python.org  Tue Sep 15 21:13:24 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Sep 2009 15:13:24 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253037263.5678.12.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
Message-ID: <5115B35A-DFE6-4093-82BF-AC76F99846F0@python.org>

On Sep 15, 2009, at 1:54 PM, Antoine Pitrou wrote:

> Python's reputation will certainly get tarnished, however, if
> widely-used modules get deprecated and removed. We will already get  
> some
> damage with all of the py3k changes. No need to worsen our case, IMO.

I don't view this any differently than language evolution.  We have a  
__future__ mechanism and policy for how language features change in  
incompatible ways between versions.  Application developers may not  
like it, but it's published, they have forewarning, and a long period  
in which to migrate.  New versions of Python break applications every  
time despite this, and despite lengthy beta releases.  Yes, we get  
grief for that despite the policy, but tough.  This is an all- 
volunteer loosely-organized herd of cats who work on this stuff for  
fun, taking time away from family, hobbies and sleep.  Put up $10M/yr  
if you want to change that ('course, that's been tried <wink>).   
Still, we try to be responsible and responsive to our users, which is  
why we have a long deprecation period.

The stdlib is the same IMO.  It must evolve and that means cleaning  
out the cobwebs from time to time.  We can do it in a way that gives  
full disclosure and ample time to adjust, but I really don't have any  
sympathy for libraries and applications that /never/ want to update,  
even though I've been on that side many times.  If that's the case,  
then you will probably have a problem with upgrading Python /at all/  
stdlib stagnation or not.  Maintaining your own version of Python is a  
perfectly reasonable and viable option in that case.

OTOH, we don't have to be rash or uncaring about it.  A policy that  
provides an orderly, lengthy migration to better standard libraries  
seems responsible to me.  The first steps can be to add the new  
library and put a big red warning in the docs of the old code,  
followed in later years by deprecation warnings in the library and  
hiding of the documentation, followed by its complete removal many  
years later.  Heck, you probably won't even have to worry about it,  
it'll be your kids that have to actually remove it from Python :).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090915/542b1b7d/attachment.pgp>

From michael at voidspace.org.uk  Tue Sep 15 21:13:35 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 20:13:35 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <200909151910.n8FJAwag005132@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<200909151910.n8FJAwag005132@theraft.openend.se>
Message-ID: <4AAFE75F.8020709@voidspace.org.uk>

Laura Creighton wrote:
> In a message of Tue, 15 Sep 2009 19:41:25 BST, Paul Moore writes:
>   
>> MAL pointed out http://code.activestate.com/recipes/573441/ - extended
>> optparse to allow definition of required options. Given that one of
>> the requirements that argparse is claimed to meet where optparse
>> doesn't is supporting required arguments, how come this simple recipe
>> hasn't been incorporated into optparse?
>>
>> The optparse/argparse case seems to rest on the argument that optparse
>> cannot be extended to do what argparse does. It seems like this isn't
>> true for all requirements. (And maybe some others could be addressed
>> by judiciously deprecating support for specific internal details that
>> maybe should not have been documented in the first place...)
>>     
>
> Maybe, but this simple recipe only works for options which are
> unconditionally required.  What I needed was a solution for options
> that are conditionally required -- if you have option X then you
> must have option Q.
>
> But optparse is particularly unhappy about this, and about other
> things.  If you type command --verbose --brief, and these are
> toggles, then you will get the last one processed, brief in this
> case.  There is no easy way to say 'now please roll over and die'
> because you have an incompatible set of options.  I wrote code 
> that changed optparse to do what I wanted, but boy, was it ugly.
> I can maybe dig it up somewhere.  And when I went knocking, asking 
> optparse's author about how he would recommend that I solve my 
> problem using optparse in a way that was less ugly, I was told
> that my problem wasn't to be solved using optparse.
>
> So I would surmise that the reason this recipe isn't in the standard
> library is optparse's author didn't want it in optparse.
>
>   

Plus, more useful than required options are required positional 
arguments - which optparse doesn't handle at all (and exposes features 
that makes handling them difficult to do in a backwards incompatible way).

Required options are easy to do yourself - just check for their presence 
and error out if they aren't there. It is the other features that are 
more difficult.

Michael

>> Paul.
>> _______________________________________________
>> stdlib-sig mailing list
>> stdlib-sig at python.org
>> http://mail.python.org/mailman/listinfo/stdlib-sig
>>     


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Tue Sep 15 21:17:11 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 21:17:11 +0200
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <4AAFE677.3030501@active-4.com>
References: <4AAFE074.5070107@active-4.com>
	<1253041751.5678.46.camel@localhost>  <4AAFE677.3030501@active-4.com>
Message-ID: <1253042231.5678.49.camel@localhost>

Le mardi 15 septembre 2009 ? 12:09 -0700, Armin Ronacher a ?crit :
> I can see strong disagreement with me here which is why I would love to
> change my point to "I would only *continue* to maintain optparse is
> argparse does not end up in the standard library".  If both end up
> there, maintenance from my side ends.

Then it is probably fine, IMHO. The optparse situation, if you ended up
maintenance, wouldn't be worse than today's.




From mal at egenix.com  Tue Sep 15 21:15:31 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 21:15:31 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFE1AB.6000303@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<4AAF78D9.20107@egenix.com>	<4AAF84C4.20204@voidspace.org.uk>	<79990c6b0909150955g688c23aekfed7833862cf0449@mail.gmail.com>	<4AAFD79A.5050103@egenix.com>	<4AAFDB43.8050509@voidspace.org.uk>
	<4AAFE00D.2070800@egenix.com> <4AAFE1AB.6000303@voidspace.org.uk>
Message-ID: <4AAFE7D3.7010101@egenix.com>

Michael Foord wrote:
> M.-A. Lemburg wrote:
>> Michael Foord wrote:
>>> Modules are also deprecated wholesale where their functionality is
>>> replaced by an alternative API.
>>>
>>>    md5 -> hashlib
>>>    mimewriter -> email
>>>
>>> This is already a normal part of the evolution of the Python standard
>>> library.
>>>     
>>
>> And that's perfectly ok.
>>
>> If the change is a single import statement, then I don't have a
>> problem with that.
>>   
> 
> For md5 the API basically moved (although md5.new became hashlib.md5).

Right - you only have to write a line or two to support both
ways in your code.

> For MimeWriter there were (what looks like from a relatively casual
> look) fairly big API changes as well - even if they were only name
> changes (which is substantially similar to the requirements for moving
> from optparse to argparse).

True, but it's fairly easy writing a compatibility module which
does what MimeWriter did in terms of the email package.

I wonder why that wasn't added to the email package - I would
certainly have complained, but I've never used MimeWriter myself
and didn't notice the deprecation at the time.

Here's an example of migrating from MimeWriter to the email
package:

http://thomas.apestaart.org/moap/trac/changeset/401/trunk/moap/util/mail.py

Note that the MimeWriter module still exists in Python 2.7:

"""
This module is present only to maintain backward compatibility.
"""

and that's good.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From georg at python.org  Tue Sep 15 21:18:35 2009
From: georg at python.org (Georg Brandl)
Date: Tue, 15 Sep 2009 21:18:35 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>
	<4AAFDC8D.5050401@voidspace.org.uk>
	<1253039704.5678.36.camel@localhost>
	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>
Message-ID: <4AAFE88B.4050704@python.org>

Brett Cannon schrieb:
> If Twitter is any indication Georg is now ignoring this mailing list,
> so I am adding him directly so he can answer Antoine's questions.

Doesn't help because your mail's still sorted into the same folder with
all the other mails having [stdlib-sig] in the title :)

FWIW, I asked gmane today to add the stdlib-sig; but I fear that this
period of high volume is over before they get to it.

> On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>> Two examples of deprecated modules in the Python documentation:
>>>
>>> http://docs.python.org/library/md5.html
>>> http://docs.python.org/library/mimewriter.html
>>>
>>> The deprecation notice could be stronger / more prominent.
>>
>> Perhaps deprecated modules can get some kind of greyish style
>> (background and/or font). Georg, is it possible? :)

The whole document? Seems a bit too intrusive to me, and there is no obvious
connection to the deprecation if the deprecation notice is still hard to see.

But still possible, if you want it :)

>> They could also be moved to a dedicated section. For example, md5 would
>> be inside "XX - deprecated modules" rather than "15 - Cryptographic
>> services".

It would still have an ordinary number, but that's of course possible.

Georg


From barry at python.org  Tue Sep 15 21:21:45 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Sep 2009 15:21:45 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
Message-ID: <117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>

On Sep 15, 2009, at 2:41 PM, Paul Moore wrote:

> MAL pointed out http://code.activestate.com/recipes/573441/ - extended
> optparse to allow definition of required options. Given that one of
> the requirements that argparse is claimed to meet where optparse
> doesn't is supporting required arguments, how come this simple recipe
> hasn't been incorporated into optparse?

That's an excellent question which kind of says something about  
people's enthusiasm for maintaining optparse, eh?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090915/02e659d5/attachment.pgp>

From brett at python.org  Tue Sep 15 21:27:05 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 12:27:05 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <4AAFE074.5070107@active-4.com>
References: <4AAFE074.5070107@active-4.com>
Message-ID: <bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>

On Tue, Sep 15, 2009 at 11:44, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Hello list,
>
> What started as a joke yesterday in the Bug Tracker and on Twitter now
> evolved into a serious consideration. ?In case the only reason for
> optparse to go away is that it does not have a maintainer I would take
> over that task.
>
> In fact I would also implement missing features based on real-world
> needs. ?That would most likely mean that some of the changes in argparse
> would end up in optparse as well. ?Furthermore I would release a
> Python-independent version on PyPI for compatibility with older Python
> versions.
>
> I would take over this task if the following criteria are met:
>
> - argparse would not enter the standard library

Now I know you said in another email that you meant for this to be ""I
would only *continue* to maintain optparse is argparse does not end up
in the standard library", but that still feels like I am being
politically pushed around and I don't like that, especially when
argparse already exists and this proposal is hypothetical.

> - I'm allowed to modernize optparse after a discussion with python-dev
> ?in a backwards compatible way.

That's always been allowed.

> - I'm allowed to refactor the code

As long as the unit tests pass (but that assumes optparse has thorough
tests) and you do not change the public API, then that has always been
allowed as well.

> - make the i18n support of the module more pluggable which would allow
> ?specifying a custom translations instance instead of using the global
> ?gettext function.

Depends on how you want to do that. If you are suggesting creating
your own i18n solution that precludes gettext, then no as that is the
current solution in the standard library and you shouldn't try to
exclude it. If you want it to be pluggable enough such that gettext or
any other solution can be used, then fine.

But ignoring your criteria, Armin, python-dev has the criteria that
you get commit privileges and I am not sure if you will get them
because of lack of participation on python-dev and the issue tracker.
Until you get commit privileges I can't take this seriously.

-Brett

From benjamin at python.org  Tue Sep 15 21:29:49 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 15 Sep 2009 14:29:49 -0500
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
References: <4AAFE074.5070107@active-4.com>
	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
Message-ID: <1afaf6160909151229q534270e7p55bde7158830489c@mail.gmail.com>

2009/9/15 Brett Cannon <brett at python.org>:

> But ignoring your criteria, Armin, python-dev has the criteria that
> you get commit privileges and I am not sure if you will get them
> because of lack of participation on python-dev and the issue tracker.
> Until you get commit privileges I can't take this seriously.

He has them, though.



-- 
Regards,
Benjamin

From brett at python.org  Tue Sep 15 21:31:56 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 12:31:56 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <1afaf6160909151229q534270e7p55bde7158830489c@mail.gmail.com>
References: <4AAFE074.5070107@active-4.com>
	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com> 
	<1afaf6160909151229q534270e7p55bde7158830489c@mail.gmail.com>
Message-ID: <bbaeab100909151231l26b1f287l7d2805bacc3479bf@mail.gmail.com>

On Tue, Sep 15, 2009 at 12:29, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/15 Brett Cannon <brett at python.org>:
>
>> But ignoring your criteria, Armin, python-dev has the criteria that
>> you get commit privileges and I am not sure if you will get them
>> because of lack of participation on python-dev and the issue tracker.
>> Until you get commit privileges I can't take this seriously.
>
> He has them, though.

Yeah, just found that out.

-Brett

From g.brandl at gmx.net  Tue Sep 15 21:30:14 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 21:30:14 +0200
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
References: <4AAFE074.5070107@active-4.com>
	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
Message-ID: <h8oq42$eku$1@ger.gmane.org>

Brett Cannon schrieb:

> But ignoring your criteria, Armin, python-dev has the criteria that
> you get commit privileges and I am not sure if you will get them
> because of lack of participation on python-dev and the issue tracker.
> Until you get commit privileges I can't take this seriously.

In fact, Armin already had commit privileges since July 07, originally for
work on doctools (now Sphinx), which started out in the python.org repo.
Later he contributed the ast module and kept the privileges as its
maintainer.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From armin.ronacher at active-4.com  Tue Sep 15 21:35:38 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Tue, 15 Sep 2009 12:35:38 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <bbaeab100909151231l26b1f287l7d2805bacc3479bf@mail.gmail.com>
References: <4AAFE074.5070107@active-4.com>
	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
	<1afaf6160909151229q534270e7p55bde7158830489c@mail.gmail.com>
	<bbaeab100909151231l26b1f287l7d2805bacc3479bf@mail.gmail.com>
Message-ID: <4AAFEC8A.3010709@active-4.com>

Hi,

Brett Cannon wrote:
> Now I know you said in another email that you meant for this to be ""I
> would only *continue* to maintain optparse is argparse does not end up
> in the standard library", but that still feels like I am being
> politically pushed around and I don't like that, especially when
> argparse already exists and this proposal is hypothetical.
Open Source turns out to be very political unfortunately, and I'm of
course driven my politics as well.  I have not chosen my words carefully
and I'm sorry if that causes you to think I try to push around anyone.

> Depends on how you want to do that. If you are suggesting creating
> your own i18n solution that precludes gettext, then no as that is the
> current solution in the standard library and you shouldn't try to
> exclude it. If you want it to be pluggable enough such that gettext or
> any other solution can be used, then fine.
Django and Babel still use gettext without a global context, the way
this would change is that you can provide your own translator when
creating the option parser.

> But ignoring your criteria, Armin, python-dev has the criteria that
> you get commit privileges and I am not sure if you will get them
> because of lack of participation on python-dev and the issue tracker.
> Until you get commit privileges I can't take this seriously.
I ended up with commit privileges a while ago, but don't take me serious
because of that.  No matter if I would have commit privileges or not, my
offer is serious and I would do so either way *because* I want to see
Python improve.  So far I only complained about the standard library and
it would only be fair to start improving it.


Regards,
Armin

From g.brandl at gmx.net  Tue Sep 15 21:35:09 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 21:35:09 +0200
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <h8oq42$eku$1@ger.gmane.org>
References: <4AAFE074.5070107@active-4.com>	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
	<h8oq42$eku$1@ger.gmane.org>
Message-ID: <h8oqd9$g87$1@ger.gmane.org>

Georg Brandl schrieb:
> Brett Cannon schrieb:
> 
>> But ignoring your criteria, Armin, python-dev has the criteria that
>> you get commit privileges and I am not sure if you will get them
>> because of lack of participation on python-dev and the issue tracker.
>> Until you get commit privileges I can't take this seriously.
> 
> In fact, Armin already had commit privileges since July 07, originally for
> work on doctools (now Sphinx), which started out in the python.org repo.

This is also the reason why he's not listed in developers.txt, in case
someone is wondering.

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From mal at egenix.com  Tue Sep 15 21:43:39 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 21:43:39 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<collinw@gmail.com>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>	<1253038514.5678.32.camel@localhost>	<4AAFDBAD.7040906@voidspace.org.uk>	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
Message-ID: <4AAFEE6B.3070900@egenix.com>

Barry Warsaw wrote:
> On Sep 15, 2009, at 2:41 PM, Paul Moore wrote:
> 
>> MAL pointed out http://code.activestate.com/recipes/573441/ - extended
>> optparse to allow definition of required options. Given that one of
>> the requirements that argparse is claimed to meet where optparse
>> doesn't is supporting required arguments, how come this simple recipe
>> hasn't been incorporated into optparse?
> 
> That's an excellent question which kind of says something about people's
> enthusiasm for maintaining optparse, eh?

It says something about the apparent importance of this particular
feature in an argument parsing module ;-)

Seriously, I wouldn't expect an argument parsing tool to do any
complicated option dependency checking for me. That's something
you do after having parsed the command line and it's likely
different for every single script that needs such checks.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From benjamin at python.org  Tue Sep 15 21:24:20 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 15 Sep 2009 14:24:20 -0500
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFE88B.4050704@python.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>
	<4AAFDC8D.5050401@voidspace.org.uk>
	<1253039704.5678.36.camel@localhost>
	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>
	<4AAFE88B.4050704@python.org>
Message-ID: <1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>

2009/9/15 Georg Brandl <georg at python.org> schrieb:
>> On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>>> Two examples of deprecated modules in the Python documentation:
>>>>
>>>> http://docs.python.org/library/md5.html
>>>> http://docs.python.org/library/mimewriter.html
>>>>
>>>> The deprecation notice could be stronger / more prominent.
>>>
>>> Perhaps deprecated modules can get some kind of greyish style
>>> (background and/or font). Georg, is it possible? :)
>
> The whole document? Seems a bit too intrusive to me, and there is no obvious
> connection to the deprecation if the deprecation notice is still hard to see.

Wouldn't a flashing red background with a scrolling marquee in a
gothic font be more effective?



-- 
Regards,
Benjamin

From michael at voidspace.org.uk  Tue Sep 15 21:52:55 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 20:52:55 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>	<4AAFDC8D.5050401@voidspace.org.uk>	<1253039704.5678.36.camel@localhost>	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>	<4AAFE88B.4050704@python.org>
	<1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>
Message-ID: <4AAFF097.5040402@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/9/15 Georg Brandl <georg at python.org> schrieb:
>   
>>> On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>       
>>>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>>>         
>>>>> Two examples of deprecated modules in the Python documentation:
>>>>>
>>>>> http://docs.python.org/library/md5.html
>>>>> http://docs.python.org/library/mimewriter.html
>>>>>
>>>>> The deprecation notice could be stronger / more prominent.
>>>>>           
>>>> Perhaps deprecated modules can get some kind of greyish style
>>>> (background and/or font). Georg, is it possible? :)
>>>>         
>> The whole document? Seems a bit too intrusive to me, and there is no obvious
>> connection to the deprecation if the deprecation notice is still hard to see.
>>     
>
> Wouldn't a flashing red background with a scrolling marquee in a
> gothic font be more effective?
>
>
>
>   
+1 for <blink><marquee> on the whole document...

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From mal at egenix.com  Tue Sep 15 21:53:46 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 21:53:46 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <5115B35A-DFE6-4093-82BF-AC76F99846F0@python.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4dab5f760909141047y36c90274o4ceb4f21c8324ea8@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>	<4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>
	<5115B35A-DFE6-4093-82BF-AC76F99846F0@python.org>
Message-ID: <4AAFF0CA.3010604@egenix.com>

Barry Warsaw wrote:
> On Sep 15, 2009, at 1:54 PM, Antoine Pitrou wrote:
> 
>> Python's reputation will certainly get tarnished, however, if
>> widely-used modules get deprecated and removed. We will already get some
>> damage with all of the py3k changes. No need to worsen our case, IMO.
> 
> I don't view this any differently than language evolution. [...]
> 
> The stdlib is the same IMO.  It must evolve and that means cleaning out
> the cobwebs from time to time.  We can do it in a way that gives full
> disclosure and ample time to adjust, but I really don't have any
> sympathy for libraries and applications that /never/ want to update,
> even though I've been on that side many times. 

I think that the stdlib needs a slightly different approach -
the changes we apply in the language hardly ever cause serious
breakage of code (except maybe the Py_ssize_t change, but folks
won't notice until everyone starts running Python on Linux/x64
machines ;-).

Removing a complete set of APIs or even a complete concept to
approaching a problem is different, though. Those are major
changes.

I guess its time for a PEP outlining how such changes should
be handled and which time frames to apply (release cycles can
change, so years is probably a better unit).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From g.brandl at gmx.net  Tue Sep 15 21:51:37 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 21:51:37 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253031517.5646.191.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<1253029742.5646.180.camel@localhost>	<43aa6ff70909150903v3738d817iec5bfea8b2bb247@mail.gmail.com>
	<1253031517.5646.191.camel@localhost>
Message-ID: <h8orc6$juk$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Le mardi 15 septembre 2009 ? 12:03 -0400, Collin Winter a ?crit :
>> 
>> Did you read what Laura wrote about her experience with optparse?
>> [...]
> 
> Sure I did. These are still fringe use cases compared to the common uses
> of an option parser. Perhaps argparse has a couple of nifty features
> that make things slightly easier in the common cases, but the difference
> doesn't seem earth-shattering to me.
> 
> The point is, optparse does useful things and it does them reasonably
> well. You can't deny that, and talking about how that other library does
> many other things is not a satisfying answer to the people whose code
> you're gonna break.

Of course it does them reasonably well.  But for most of these "common"
use cases you talk about above, a replacement library will be able to
offer the same or a very similar API, so the breakage is not much worse
than with md5/hashlib.

>> It's also worth noting that optparse continues to have bugs filed
>> against it, so it is not perfect or low-maintenance in that respect:
>> http://bugs.python.org/issue2931 is an interesting example.
> 
> Well, of course it has bugs filed against it (which software hasn't?).
> The point is that there are few of them.
> And, yes, being more unicode-compliant would be nice (although the bug
> should be retried on py3k, because this whole discussion probably
> targets 3.x anyway, not 2.x).

I wouldn't say it has few bugs filed against it; sadly I have no hard data,
but since I read each new tracker item (or at least the title), I would say
that there are many modules that have fewer bugs filed than optparse.

Of course, this also is correlated with the module's size (it's currently
the 12th largest single module in trunk/Lib).

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Tue Sep 15 21:46:11 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 12:46:11 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <4AAFEC8A.3010709@active-4.com>
References: <4AAFE074.5070107@active-4.com>
	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com> 
	<1afaf6160909151229q534270e7p55bde7158830489c@mail.gmail.com> 
	<bbaeab100909151231l26b1f287l7d2805bacc3479bf@mail.gmail.com> 
	<4AAFEC8A.3010709@active-4.com>
Message-ID: <bbaeab100909151246w1afa4f50l61888249efe2c5f2@mail.gmail.com>

On Tue, Sep 15, 2009 at 12:35, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Hi,
>
> Brett Cannon wrote:
>> Now I know you said in another email that you meant for this to be ""I
>> would only *continue* to maintain optparse is argparse does not end up
>> in the standard library", but that still feels like I am being
>> politically pushed around and I don't like that, especially when
>> argparse already exists and this proposal is hypothetical.

> Open Source turns out to be very political unfortunately,

Tell me about it.

> and I'm of
> course driven my politics as well. ?I have not chosen my words carefully
> and I'm sorry if that causes you to think I try to push around anyone.

Glad you aren't.

>
>> Depends on how you want to do that. If you are suggesting creating
>> your own i18n solution that precludes gettext, then no as that is the
>> current solution in the standard library and you shouldn't try to
>> exclude it. If you want it to be pluggable enough such that gettext or
>> any other solution can be used, then fine.

> Django and Babel still use gettext without a global context, the way
> this would change is that you can provide your own translator when
> creating the option parser.
>

As long as it can be used w/o modification then that's fine.

>> But ignoring your criteria, Armin, python-dev has the criteria that
>> you get commit privileges and I am not sure if you will get them
>> because of lack of participation on python-dev and the issue tracker.
>> Until you get commit privileges I can't take this seriously.

> I ended up with commit privileges a while ago, but don't take me serious
> because of that. ?No matter if I would have commit privileges or not, my
> offer is serious and I would do so either way *because* I want to see
> Python improve. ?So far I only complained about the standard library and
> it would only be fair to start improving it.

I'm glad this is motivating you!

Here is my perspective. I want Steven to move forward w/ trying to get
argparse in, which means he has to write a PEP to propose his
inclusion. If you manage to get optparse into a place where argprase
is not needed, then at worst Steven has written a PEP that will be
rejected and he can potentially stop working on argparse and switch to
your refactored optparse. But I have seen too many well-intentioned
people come along and say they really want to do something and not
finish (because of time, turned out not to be doable, etc.). I am not
saying I think you personally will more than likely fail, Armin, as I
don't know you personally. But as a rule, I assume no one ever follows
through until they have proven to me otherwise. That's why I am not
willing to promise you that argparse will not go in or that I am
willing to tell Steven to not try to get argparse in. But if you pull
off what you are suggesting and you and Steven can get optparse to get
what argparse has then there won't be a need to add argparse.

-Brett

From mal at egenix.com  Tue Sep 15 21:58:25 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Sep 2009 21:58:25 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFF097.5040402@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>	<4AAFDC8D.5050401@voidspace.org.uk>	<1253039704.5678.36.camel@localhost>	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>	<4AAFE88B.4050704@python.org>	<1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>
	<4AAFF097.5040402@voidspace.org.uk>
Message-ID: <4AAFF1E1.50309@egenix.com>

Michael Foord wrote:
> Benjamin Peterson wrote:
>> 2009/9/15 Georg Brandl <georg at python.org> schrieb:
>>  
>>>> On Tue, Sep 15, 2009 at 11:35, Antoine Pitrou <solipsis at pitrou.net>
>>>> wrote:
>>>>      
>>>>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>>>>        
>>>>>> Two examples of deprecated modules in the Python documentation:
>>>>>>
>>>>>> http://docs.python.org/library/md5.html
>>>>>> http://docs.python.org/library/mimewriter.html
>>>>>>
>>>>>> The deprecation notice could be stronger / more prominent.
>>>>>>           
>>>>> Perhaps deprecated modules can get some kind of greyish style
>>>>> (background and/or font). Georg, is it possible? :)
>>>>>         
>>> The whole document? Seems a bit too intrusive to me, and there is no
>>> obvious
>>> connection to the deprecation if the deprecation notice is still hard
>>> to see.
>>>     
>>
>> Wouldn't a flashing red background with a scrolling marquee in a
>> gothic font be more effective?
>>
>>
>>
>>   
> +1 for <blink><marquee> on the whole document...

Those were deprecated a long long time ago by the W3C:
http://www.w3.org/TR/WCAG10/

Today, you have to use a different concept for this called "CSS"
and the new syntax goes like this:

<body style="text-decoration:blink;">...</body>

That's just as annoying, but hey, it's standards conform ;-)

Hmm, now why didn't they just remove it altogether...

(sorry, couldn't resist)
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 15 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Tue Sep 15 22:04:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 22:04:20 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <h8orc6$juk$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE8AE4.30104@egenix.com> <4AAE94BF.8080407@voidspace.org.uk>
	<4AAEBEBE.1050105@egenix.com> <4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<1253029742.5646.180.camel@localhost>
	<43aa6ff70909150903v3738d817iec5bfea8b2bb247@mail.gmail.com>
	<1253031517.5646.191.camel@localhost>  <h8orc6$juk$1@ger.gmane.org>
Message-ID: <1253045060.5678.61.camel@localhost>

Le mardi 15 septembre 2009 ? 21:51 +0200, Georg Brandl a ?crit :
> 
> Of course it does them reasonably well.  But for most of these "common"
> use cases you talk about above, a replacement library will be able to
> offer the same or a very similar API, so the breakage is not much worse
> than with md5/hashlib.

Well I do think that md5 shouldn't have been deprecated in 2.x. AFAIU,
it was easy to continue supporting it (by creating a simple stub);
annoying users wasn't worth what we gained in maintenance, IMO.

But regardless, this is not the same situation. Migrating from md5 to
hashlib is simply changing an import line and a module name. Migrating
from optparse to argparse involves an API change.

> I wouldn't say it has few bugs filed against it; sadly I have no hard data,
> but since I read each new tracker item (or at least the title), I would say
> that there are many modules that have fewer bugs filed than optparse.
> 
> Of course, this also is correlated with the module's size (it's currently
> the 12th largest single module in trunk/Lib).

It is also correlated with the module's usefulness. Parsing command line
options is certainly much more common than, say, normalizing unicode
strings or adding fractions together.

Regards

Antoine.



From g.brandl at gmx.net  Tue Sep 15 22:01:44 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 22:01:44 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFF097.5040402@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>	<4AAFDC8D.5050401@voidspace.org.uk>	<1253039704.5678.36.camel@localhost>	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>	<4AAFE88B.4050704@python.org>	<1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>
	<4AAFF097.5040402@voidspace.org.uk>
Message-ID: <h8orv4$m2k$1@ger.gmane.org>

Michael Foord schrieb:

>>>>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>>>>         
>>>>>> Two examples of deprecated modules in the Python documentation:
>>>>>>
>>>>>> http://docs.python.org/library/md5.html
>>>>>> http://docs.python.org/library/mimewriter.html
>>>>>>
>>>>>> The deprecation notice could be stronger / more prominent.
>>>>>>           
>>>>> Perhaps deprecated modules can get some kind of greyish style
>>>>> (background and/or font). Georg, is it possible? :)
>>>>>         
>>> The whole document? Seems a bit too intrusive to me, and there is no obvious
>>> connection to the deprecation if the deprecation notice is still hard to see.
>>>     
>>
>> Wouldn't a flashing red background with a scrolling marquee in a
>> gothic font be more effective?
>>
> +1 for <blink><marquee> on the whole document...

And now for something completely reasonable: same markup as for warnings?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Tue Sep 15 22:05:46 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 22:05:46 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253045060.5678.61.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAE8AE4.30104@egenix.com>
	<4AAE94BF.8080407@voidspace.org.uk>	<4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>	<mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<1253029742.5646.180.camel@localhost>	<43aa6ff70909150903v3738d817iec5bfea8b2bb247@mail.gmail.com>	<1253031517.5646.191.camel@localhost>
	<h8orc6$juk$1@ger.gmane.org> <1253045060.5678.61.camel@localhost>
Message-ID: <h8os6n$mrq$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Le mardi 15 septembre 2009 ? 21:51 +0200, Georg Brandl a ?crit :
>> 
>> Of course it does them reasonably well.  But for most of these "common"
>> use cases you talk about above, a replacement library will be able to
>> offer the same or a very similar API, so the breakage is not much worse
>> than with md5/hashlib.
> 
> Well I do think that md5 shouldn't have been deprecated in 2.x. AFAIU,
> it was easy to continue supporting it (by creating a simple stub);
> annoying users wasn't worth what we gained in maintenance, IMO.
> 
> But regardless, this is not the same situation. Migrating from md5 to
> hashlib is simply changing an import line and a module name. Migrating
> from optparse to argparse involves an API change.

Maybe. (I don't know argparse.)  But I can imagine that providing a simple
compatibility API that supports creating OptionParsers and calling
add_option() and parse() on them would be easy if not trivial, and that's
what most users of optparse use.

>> I wouldn't say it has few bugs filed against it; sadly I have no hard data,
>> but since I read each new tracker item (or at least the title), I would say
>> that there are many modules that have fewer bugs filed than optparse.
>> 
>> Of course, this also is correlated with the module's size (it's currently
>> the 12th largest single module in trunk/Lib).
> 
> It is also correlated with the module's usefulness. Parsing command line
> options is certainly much more common than, say, normalizing unicode
> strings or adding fractions together.

This is true, but it doesn't make the bugs less or less important, but
potentially even more important.  Endly, that results either in more work
or in dissatisfied users.

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Tue Sep 15 21:41:06 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 12:41:06 -0700
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <h8oqd9$g87$1@ger.gmane.org>
References: <4AAFE074.5070107@active-4.com>
	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com> 
	<h8oq42$eku$1@ger.gmane.org> <h8oqd9$g87$1@ger.gmane.org>
Message-ID: <bbaeab100909151241u7c9e2f61rc4cf0101e9995551@mail.gmail.com>

On Tue, Sep 15, 2009 at 12:35, Georg Brandl <g.brandl at gmx.net> wrote:
> Georg Brandl schrieb:
>> Brett Cannon schrieb:
>>
>>> But ignoring your criteria, Armin, python-dev has the criteria that
>>> you get commit privileges and I am not sure if you will get them
>>> because of lack of participation on python-dev and the issue tracker.
>>> Until you get commit privileges I can't take this seriously.
>>
>> In fact, Armin already had commit privileges since July 07, originally for
>> work on doctools (now Sphinx), which started out in the python.org repo.
>
> This is also the reason why he's not listed in developers.txt, in case
> someone is wondering.
>

He should probably be added stating this.

-Brett

> cheers,
> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>

From benjamin at python.org  Tue Sep 15 22:15:33 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 15 Sep 2009 15:15:33 -0500
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <h8orv4$m2k$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>
	<4AAFDC8D.5050401@voidspace.org.uk>
	<1253039704.5678.36.camel@localhost>
	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>
	<4AAFE88B.4050704@python.org>
	<1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>
	<4AAFF097.5040402@voidspace.org.uk> <h8orv4$m2k$1@ger.gmane.org>
Message-ID: <1afaf6160909151315r48136c74ib2fcd69ee3be131d@mail.gmail.com>

2009/9/15 Georg Brandl <g.brandl at gmx.net>:
> Michael Foord schrieb:
>
>>>>>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>>>>>
>>>>>>> Two examples of deprecated modules in the Python documentation:
>>>>>>>
>>>>>>> http://docs.python.org/library/md5.html
>>>>>>> http://docs.python.org/library/mimewriter.html
>>>>>>>
>>>>>>> The deprecation notice could be stronger / more prominent.
>>>>>>>
>>>>>> Perhaps deprecated modules can get some kind of greyish style
>>>>>> (background and/or font). Georg, is it possible? :)
>>>>>>
>>>> The whole document? Seems a bit too intrusive to me, and there is no obvious
>>>> connection to the deprecation if the deprecation notice is still hard to see.
>>>>
>>>
>>> Wouldn't a flashing red background with a scrolling marquee in a
>>> gothic font be more effective?
>>>
>> +1 for <blink><marquee> on the whole document...
>
> And now for something completely reasonable: same markup as for warnings?

Isn't there a "deprecated" directive?



-- 
Regards,
Benjamin

From g.brandl at gmx.net  Tue Sep 15 22:21:30 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 22:21:30 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909151144o332cd67eu5d00bb37ebe57c1@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>	<1253038514.5678.32.camel@localhost>
	<4222a8490909151144o332cd67eu5d00bb37ebe57c1@mail.gmail.com>
Message-ID: <h8ot46$q1s$1@ger.gmane.org>

Jesse Noller schrieb:

> Yup, multiprocessing is a perfect example of something popular in the
> wild, but was rushed to inclusion because I proposed it late in the
> cycle. If I had to do it again - as the guy who is still on the hook
> for bug fixes, evolving it, and general maintenance, I'd not have
> gotten it in there in the state it went in. I should have proposed it
> earlier in the 2.6/3.0 process, and spent more time working on it.
> 
> Thanks for your vote though.
> 
> Today, I still fix bugs, work on improvements, etc. In my mind,
> multiprocessing is a poor example because it was pulled in *so
> quickly* - not because it was pulled in, and not because it has bugs.
> But rather a bar that should have been met, was not due to time.
> 
> On the other hand - at least it has a maintainer (me) unlike some 50%
> of the rest of the libraries. So it has some small thing going for it.
> It also has tests. And documentation. And I continue to add to those
> when I can.
> 
> How much of the rest of the standard libs can claim that?

Not much, and this is a problem.  How often have I wished for an official
maintainer for some module, who I could defer a tracker issue to and say
"please decide NOW whether this is a valid bug/request".  There are literally
hundreds of issues that could either be closed immediately as rejected/
wontfix or fixed with a small effort as soon as somebody makes the decision.

Of course, for modules without a specific maintainer, we're all supposed to
share the job, but it simply doesn't work out.  I do fix bugs that are obviously
bugs, but in more involved situations I often simply don't feel "qualified"
enough to do so.  Often a maintainer wouldn't even need special skills, but
simply be there to take the blame and make decisions, complete with fixing the
problems that arise from them later.

I realize we can't have a maintainer for every module, but we should be glad
about every one that does.

So, bottom line: We love you, Jesse, and we need more of you!

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Tue Sep 15 22:25:14 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 22:25:14 +0200
Subject: [stdlib-sig] Maintenance of optparse
In-Reply-To: <bbaeab100909151241u7c9e2f61rc4cf0101e9995551@mail.gmail.com>
References: <4AAFE074.5070107@active-4.com>	<bbaeab100909151227y51ec1ab6v6ddce409ef22342e@mail.gmail.com>
	<h8oq42$eku$1@ger.gmane.org> <h8oqd9$g87$1@ger.gmane.org>
	<bbaeab100909151241u7c9e2f61rc4cf0101e9995551@mail.gmail.com>
Message-ID: <h8otb6$q1s$3@ger.gmane.org>

Brett Cannon schrieb:
> On Tue, Sep 15, 2009 at 12:35, Georg Brandl <g.brandl at gmx.net> wrote:
>> Georg Brandl schrieb:
>>> Brett Cannon schrieb:
>>>
>>>> But ignoring your criteria, Armin, python-dev has the criteria that
>>>> you get commit privileges and I am not sure if you will get them
>>>> because of lack of participation on python-dev and the issue tracker.
>>>> Until you get commit privileges I can't take this seriously.
>>>
>>> In fact, Armin already had commit privileges since July 07, originally for
>>> work on doctools (now Sphinx), which started out in the python.org repo.
>>
>> This is also the reason why he's not listed in developers.txt, in case
>> someone is wondering.
>>
> 
> He should probably be added stating this.

Makes sense, done.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Tue Sep 15 22:23:27 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 22:23:27 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1afaf6160909151315r48136c74ib2fcd69ee3be131d@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<Pine.LNX.4.64.0909151411530.6303@kimball.webabinitio.net>	<4AAFDC8D.5050401@voidspace.org.uk>	<1253039704.5678.36.camel@localhost>	<bbaeab100909151142x43887b7dwf9a5c1ff15324554@mail.gmail.com>	<4AAFE88B.4050704@python.org>	<1afaf6160909151224r59638ec1l79a38acd9aa7bb95@mail.gmail.com>	<4AAFF097.5040402@voidspace.org.uk>
	<h8orv4$m2k$1@ger.gmane.org>
	<1afaf6160909151315r48136c74ib2fcd69ee3be131d@mail.gmail.com>
Message-ID: <h8ot7r$q1s$2@ger.gmane.org>

Benjamin Peterson schrieb:
> 2009/9/15 Georg Brandl <g.brandl at gmx.net>:
>> Michael Foord schrieb:
>>
>>>>>>> Le mardi 15 septembre 2009 ? 19:27 +0100, Michael Foord a ?crit :
>>>>>>>
>>>>>>>> Two examples of deprecated modules in the Python documentation:
>>>>>>>>
>>>>>>>> http://docs.python.org/library/md5.html
>>>>>>>> http://docs.python.org/library/mimewriter.html
>>>>>>>>
>>>>>>>> The deprecation notice could be stronger / more prominent.
>>>>>>>>
>>>>>>> Perhaps deprecated modules can get some kind of greyish style
>>>>>>> (background and/or font). Georg, is it possible? :)
>>>>>>>
>>>>> The whole document? Seems a bit too intrusive to me, and there is no obvious
>>>>> connection to the deprecation if the deprecation notice is still hard to see.
>>>>>
>>>>
>>>> Wouldn't a flashing red background with a scrolling marquee in a
>>>> gothic font be more effective?
>>>>
>>> +1 for <blink><marquee> on the whole document...
>>
>> And now for something completely reasonable: same markup as for warnings?
> 
> Isn't there a "deprecated" directive?

Of course, I meant "styling", not "markup". Shame on you for not reading
my mind :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From rdmurray at bitdance.com  Tue Sep 15 23:14:01 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 17:14:01 -0400 (EDT)
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AAFEE6B.3070900@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
Message-ID: <Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 21:43, M.-A. Lemburg wrote:
> Barry Warsaw wrote:
>> On Sep 15, 2009, at 2:41 PM, Paul Moore wrote:
>>
>>> MAL pointed out http://code.activestate.com/recipes/573441/ - extended
>>> optparse to allow definition of required options. Given that one of
>>> the requirements that argparse is claimed to meet where optparse
>>> doesn't is supporting required arguments, how come this simple recipe
>>> hasn't been incorporated into optparse?
>>
>> That's an excellent question which kind of says something about people's
>> enthusiasm for maintaining optparse, eh?
>
> It says something about the apparent importance of this particular
> feature in an argument parsing module ;-)

Actually I believe I heard from someone other than Laura that required
options were explicitly rejected.  And then there's this from the
documentation for optparse:

required option
     an option that must be supplied on the command-line; note that the
     phrase ???required option??? is self-contradictory in English. optparse
     doesn???t prevent you from implementing required options, but doesn???t
     give you much help at it either. See examples/required_1.py and
     examples/required_2.py in the optparse source distribution for two
     ways to implement required options with optparse.

--David

From rdmurray at bitdance.com  Tue Sep 15 23:17:16 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 17:17:16 -0400 (EDT)
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com> 
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 11:21, Brett Cannon wrote:
> On Tue, Sep 15, 2009 at 10:54, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> PS : Perhaps I'm the only one bothered by this, but it would be nice if
>> people trimmed quoted messages a bit more. Having to scroll 3 pages
>> before reading the actual reply is tedious.
>
> Gmail color-codes so I simply scroll until I no longer see purple.

My editor color codes, too, which helps, but I still find masses of
quoted context tedious.

--David

From solipsis at pitrou.net  Tue Sep 15 23:22:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 23:22:28 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
Message-ID: <1253049748.5678.75.camel@localhost>

Le mardi 15 septembre 2009 ? 17:14 -0400, R. David Murray a ?crit :
> 
> Actually I believe I heard from someone other than Laura that required
> options were explicitly rejected.

This is one of the reasons why I'm against exclusive module ownership.
If a reasonable number of people think a feature would benefit the
community, the module owner shouldn't be able to veto it on ideological
(or whatever other personal) grounds.

In any case, this is not directly an argument against optparse itself,
if someone (Armin?) decides to maintain it with a more open attitude.



From michael at voidspace.org.uk  Tue Sep 15 23:22:24 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Sep 2009 22:22:24 +0100
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253049748.5678.75.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>
	<collinw@gmail.com>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>	<1253038514.5678.32.camel@localhost>	<4AAFDBAD.7040906@voidspace.org.uk>	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>	<4AAFEE6B.3070900@egenix.com>	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
Message-ID: <4AB00590.7040000@voidspace.org.uk>

Antoine Pitrou wrote:
> Le mardi 15 septembre 2009 ? 17:14 -0400, R. David Murray a ?crit :
>   
>> Actually I believe I heard from someone other than Laura that required
>> options were explicitly rejected.
>>     
>
> This is one of the reasons why I'm against exclusive module ownership.
> If a reasonable number of people think a feature would benefit the
> community, the module owner shouldn't be able to veto it on ideological
> (or whatever other personal) grounds.
>
>   

I dislike exclusive module ownership too. We end up in situations where 
modules (like ElementTree) are 'owned' by someone who is absent and 
no-one else is able to (or dares to) touch the code.

Michael

> In any case, this is not directly an argument against optparse itself,
> if someone (Armin?) decides to maintain it with a more open attitude.
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From g.brandl at gmx.net  Tue Sep 15 23:49:43 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Sep 2009 23:49:43 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AB00590.7040000@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<collinw@gmail.com>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>	<1253038514.5678.32.camel@localhost>	<4AAFDBAD.7040906@voidspace.org.uk>	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>	<4AAFEE6B.3070900@egenix.com>	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>	<1253049748.5678.75.camel@localhost>
	<4AB00590.7040000@voidspace.org.uk>
Message-ID: <h8p29j$9qk$1@ger.gmane.org>

Michael Foord schrieb:
> Antoine Pitrou wrote:
>> Le mardi 15 septembre 2009 ? 17:14 -0400, R. David Murray a ?crit :
>>   
>>> Actually I believe I heard from someone other than Laura that required
>>> options were explicitly rejected.
>>>     
>>
>> This is one of the reasons why I'm against exclusive module ownership.
>> If a reasonable number of people think a feature would benefit the
>> community, the module owner shouldn't be able to veto it on ideological
>> (or whatever other personal) grounds.
>>
>>   
> 
> I dislike exclusive module ownership too. We end up in situations where 
> modules (like ElementTree) are 'owned' by someone who is absent and 
> no-one else is able to (or dares to) touch the code.

Well, one part of module ownership would be to complain about and/or
revert unwanted changes.  Which one obviously cannot do when absent
or inactive :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From solipsis at pitrou.net  Wed Sep 16 00:00:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 00:00:53 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <h8p29j$9qk$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>	<collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
	<4AB00590.7040000@voidspace.org.uk> <h8p29j$9qk$1@ger.gmane.org>
Message-ID: <1253052053.5678.77.camel@localhost>

Le mardi 15 septembre 2009 ? 23:49 +0200, Georg Brandl a ?crit :
> 
> Well, one part of module ownership would be to complain about and/or
> revert unwanted changes.

We don't need ownership for that. Anyone can complain about a change and
ask for it to be reverted.




From jnoller at gmail.com  Wed Sep 16 00:22:19 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 18:22:19 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253049748.5678.75.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
Message-ID: <4222a8490909151522w5e99c1dfw63c34a9454c3252b@mail.gmail.com>

On Tue, Sep 15, 2009 at 5:22 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 17:14 -0400, R. David Murray a ?crit :
>>
>> Actually I believe I heard from someone other than Laura that required
>> options were explicitly rejected.
>
> This is one of the reasons why I'm against exclusive module ownership.
> If a reasonable number of people think a feature would benefit the
> community, the module owner shouldn't be able to veto it on ideological
> (or whatever other personal) grounds.
>
> In any case, this is not directly an argument against optparse itself,
> if someone (Armin?) decides to maintain it with a more open attitude.

And I can find at least 176 reasons why owners are a good idea:

http://bugs.python.org/issue?%40search_text=&title=&%40columns=title&id=&%40columns=id&stage=4&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search

There will always be some owners who insist on idealistic purity over
serving external users - just look at every PEP that crosses
python-dev.

However; this is not a case against owners - this is a case against bad owners.

The fact is, we need people who feel responsibility for every one of
these modules to review patches, and have some amount of mental design
integrity to ensure modules don't just wander off into the sunset and
die.

jesse

From jnoller at gmail.com  Wed Sep 16 00:23:30 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 18:23:30 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253052053.5678.77.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
	<4AB00590.7040000@voidspace.org.uk> <h8p29j$9qk$1@ger.gmane.org>
	<1253052053.5678.77.camel@localhost>
Message-ID: <4222a8490909151523w16d1888bsd3b72ed633f1ce61@mail.gmail.com>

On Tue, Sep 15, 2009 at 6:00 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 23:49 +0200, Georg Brandl a ?crit :
>>
>> Well, one part of module ownership would be to complain about and/or
>> revert unwanted changes.
>
> We don't need ownership for that. Anyone can complain about a change and
> ask for it to be reverted.
>

And anyone can complain about an owners decision. I've backed off on
things I've not wanted in multiprocessing because of user feedback and
compelling use cases.

I would expect the same of everyone with +commit.

jesse

From jnoller at gmail.com  Wed Sep 16 00:27:27 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 18:27:27 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4AB00590.7040000@voidspace.org.uk>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
	<4AB00590.7040000@voidspace.org.uk>
Message-ID: <4222a8490909151527y583b864hbafdc93b64cde386@mail.gmail.com>

On Tue, Sep 15, 2009 at 5:22 PM, Michael Foord <michael at voidspace.org.uk> wrote:
> Antoine Pitrou wrote:
>>
>> Le mardi 15 septembre 2009 ? 17:14 -0400, R. David Murray a ?crit :
>>
>>>
>>> Actually I believe I heard from someone other than Laura that required
>>> options were explicitly rejected.
>>>
>>
>> This is one of the reasons why I'm against exclusive module ownership.
>> If a reasonable number of people think a feature would benefit the
>> community, the module owner shouldn't be able to veto it on ideological
>> (or whatever other personal) grounds.
>>
>>
>
> I dislike exclusive module ownership too. We end up in situations where
> modules (like ElementTree) are 'owned' by someone who is absent and no-one
> else is able to (or dares to) touch the code.
>
> Michael

There is no such thing as "exclusive" ownership, and there can not be.
I am advocating for owners in as much as I'd like (like Georg) someone
to assign bugs, patches and other things to for a given module.

If an owner, such as ElementTree's chooses to be absent forever, or no
longer be involved - then they are replaced.

We've all(?) worked in business settings and most of us probably
understand the "drawbacks" to "exclusive" ownership. Ergo, that's not
what I am advocating.

However, having someone be the "thought leader, patch reviewer and guy
to send angry emails to when something is so broken it causes
convulsions" would be nice.

jesse

From jnoller at gmail.com  Wed Sep 16 00:30:47 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 18:30:47 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <h8ot46$q1s$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se> <collinw@gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4222a8490909151144o332cd67eu5d00bb37ebe57c1@mail.gmail.com>
	<h8ot46$q1s$1@ger.gmane.org>
Message-ID: <4222a8490909151530w1159d14dm26a8a3bc8d696884@mail.gmail.com>

On Tue, Sep 15, 2009 at 4:21 PM, Georg Brandl <g.brandl at gmx.net> wrote:

>> How much of the rest of the standard libs can claim that?
>
> Not much, and this is a problem. ?How often have I wished for an official
> maintainer for some module, who I could defer a tracker issue to and say
> "please decide NOW whether this is a valid bug/request". ?There are literally
> hundreds of issues that could either be closed immediately as rejected/
> wontfix or fixed with a small effort as soon as somebody makes the decision.

Bingo. Yes. Correct.

> Of course, for modules without a specific maintainer, we're all supposed to
> share the job, but it simply doesn't work out. ?I do fix bugs that are obviously
> bugs, but in more involved situations I often simply don't feel "qualified"
> enough to do so. ?Often a maintainer wouldn't even need special skills, but
> simply be there to take the blame and make decisions, complete with fixing the
> problems that arise from them later.

Also correct - this and the latter issue is exactly why I am
advocating owners, and ultimately a smaller, cleaner standard library
(but that's a different PEP)

> I realize we can't have a maintainer for every module, but we should be glad
> about every one that does.

Except those owners who are perma-afk. That's bad. Just as bad as
having none at all.

> So, bottom line: We love you, Jesse, and we need more of you!

Yeah, well - I don't know about that ;)

jesse

From rdmurray at bitdance.com  Wed Sep 16 00:38:49 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 18:38:49 -0400 (EDT)
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com> 
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
Message-ID: <Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>

It has been mentioned here that some bugs languish in the tracker because
there is no one willing to say "yes" or "no" to them.  In at least some
cases this may be because it is unclear who the best person is to ask
for a decision when the participants in the issue don't feel qualified
to decide.  And in some cases, some of the people it may be unclear to
may be the very people who _could_ decide...if only they knew that they
were the closest thing to an expert on that module that we have.

In a discussion on IRC we came up with a proposal for a simple tool that
might help out in this situation.  I would like to propose that we create
a file, tentatively named MISC/maintainers.txt, that contains two tables:
(1) a table of all the modules in the standard library and (2) a table of
'areas of expertise' (things like Unicode, arithmetic, etc).  Table (2)
would be the simpler, and would just list people who felt they had
enough expertise in the given area to be willing to make judgement
calls on relevant issues on request.

Table (1) would list, I propose, three categories of people:
(a) 'official maintainer(s)', (b) experts, and (c) contributors.
An 'official maintainer' would be someone willing to take more-or-less
full responsibility for a module (such as Jesse for Multiprocessing).
Experts would be people who feel they have a good working knowledge of
the module and would not be afraid to sign off on the advisability and
quality of a feature/bug fix when there is a question.  Contributors would
be anyone else with a more than casual knowledge of the module, but who
aren't comfortable with signing off on the advisability of non-trivial
patches/feature requests.  My rational for including the third category is
to have a pool of people who can self-promote as needed.  These people
can decide that it is OK to make the decision once they see that
there is no one willing to declare themselves an expert in that module.
I unfortunately expect a non-trivial number of modules to fall into
this category.

The listing in the table would be by tracker id, to facilitate making
people nosy on issues.  The tracker id can be used to discover names
and email addresses if those are needed instead.

Obviously one problem would be keeping this up to date, since when someone
stops contributing they are fairly likely to not remove their name from
the list.  This task could be handled by anyone who does issue triage:
if a triage person notices a maintainer or expert who has not responded
to a request for decision on an issue, they can try pinging the person
directly, and if they still get no response, mark the person (or request
that the person be marked, if the triage person is not a committer) as
'deprecated'(*) in the maintainers file.

If this proposal or a modification of it is accepted I will volunteer
to create the file and canvas python-dev for names.

--David

(*) I mean 'inactive' of course. :)

From solipsis at pitrou.net  Wed Sep 16 00:52:15 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 00:52:15 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909151522w5e99c1dfw63c34a9454c3252b@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<200909151702.n8FH29uv029039@theraft.openend.se>
	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
	<4222a8490909151522w5e99c1dfw63c34a9454c3252b@mail.gmail.com>
Message-ID: <1253055135.5678.100.camel@localhost>

Le mardi 15 septembre 2009 ? 18:22 -0400, Jesse Noller a ?crit :
> 
> And I can find at least 176 reasons why owners are a good idea:
> 
> http://bugs.python.org/issue?%40search_text=&title=&%40columns=title&id=&%40columns=id&stage=4&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search

Sorry, what's that URL supposed to prove? What does it even represent?
It is a populist argument at best, because I won't skim through those
176 bugs to try and make sense of your argument. If you want to make a
point, please don't try to leave the burden of proof on me.

Actually, I'll just take one of them, because I know it quite well:
http://bugs.python.org/issue4967

This is a bug in _ssl for which I had to write a patch, although I knew
nothing about _ssl, because the owner wouldn't react. He wouldn't react
for review either. The bug *had* to be fixed because it blocked the
whole process of including the C IO library.
Finally, Benjamin committed the patch. The owner didn't give a sign of
life *during the whole process*. He isn't dead, he still sometimes
contributes to python-dev, but he was totally unavailable when his
presence was needed.

So much for owners being a good thing.

> The fact is, we need people who feel responsibility for every one of
> these modules to review patches, and have some amount of mental design
> integrity to ensure modules don't just wander off into the sunset and
> die.

But this is not the same as having an owner.

Perhaps it wasn't clear, but I draw a clear separation between exclusive
owners and maintainers.

I'm all for non-exclusive maintainership, with people having reasonable
authority over code they understand thoroughly. You taking care of
multiprocessing falls into this category (as long as you don't demand to
approve of all changes before they are committed).

I'm against ownership, however. I'm against mandatory signoffs
(imprimaturs), and I'm against the possessive sentiment that some might
develop towards a piece of code they contributed. Any core developer
should be allowed to commit a patch if he thinks the patch is reasonable
enough and serves an useful purpose.

Ownership prevents proper maintenance when the owner is absent (which
*will* happen, since we are all unpaid volunteers). It discourages other
people from getting acquainted with the code, which gradually makes the
problem worse. Furthermore, it is often correlated with strange
(personal) idioms, coding styles and design principles.

Regards

Antoine.



From jnoller at gmail.com  Wed Sep 16 00:51:47 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 18:51:47 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
Message-ID: <4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>

On Tue, Sep 15, 2009 at 6:38 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> It has been mentioned here that some bugs languish in the tracker because
> there is no one willing to say "yes" or "no" to them. ?In at least some
> cases this may be because it is unclear who the best person is to ask
> for a decision when the participants in the issue don't feel qualified
> to decide. ?And in some cases, some of the people it may be unclear to
> may be the very people who _could_ decide...if only they knew that they
> were the closest thing to an expert on that module that we have.

+1 I had started this as a googlesheet as part of a pycon talk, and
was planning on an email later on asking for "owners".

> In a discussion on IRC we came up with a proposal for a simple tool that
> might help out in this situation. ?I would like to propose that we create
> a file, tentatively named MISC/maintainers.txt, that contains two tables:
> (1) a table of all the modules in the standard library and (2) a table of
> 'areas of expertise' (things like Unicode, arithmetic, etc). ?Table (2)
> would be the simpler, and would just list people who felt they had
> enough expertise in the given area to be willing to make judgement
> calls on relevant issues on request.

Hmm, tables in a text file? I can see it, it's just always wacky.

+1

> Table (1) would list, I propose, three categories of people:
> (a) 'official maintainer(s)', (b) experts, and (c) contributors.
> An 'official maintainer' would be someone willing to take more-or-less
> full responsibility for a module (such as Jesse for Multiprocessing).
> Experts would be people who feel they have a good working knowledge of
> the module and would not be afraid to sign off on the advisability and
> quality of a feature/bug fix when there is a question. ?Contributors would
> be anyone else with a more than casual knowledge of the module, but who
> aren't comfortable with signing off on the advisability of non-trivial
> patches/feature requests. ?My rational for including the third category is
> to have a pool of people who can self-promote as needed. ?These people
> can decide that it is OK to make the decision once they see that
> there is no one willing to declare themselves an expert in that module.
> I unfortunately expect a non-trivial number of modules to fall into
> this category.

+1, provided we can get good information into this, this could help
out a lot. Can I ask that instead of just a misc file, we put this in
the official documentation, in ReST format? I'd like to be able to
point to all official-like.

Not to mention, sphinx and ReST are the One True Way.

> The listing in the table would be by tracker id, to facilitate making
> people nosy on issues. ?The tracker id can be used to discover names
> and email addresses if those are needed instead.

+1

> Obviously one problem would be keeping this up to date, since when someone
> stops contributing they are fairly likely to not remove their name from
> the list. ?This task could be handled by anyone who does issue triage:
> if a triage person notices a maintainer or expert who has not responded
> to a request for decision on an issue, they can try pinging the person
> directly, and if they still get no response, mark the person (or request
> that the person be marked, if the triage person is not a committer) as
> 'deprecated'(*) in the maintainers file.

+1

> If this proposal or a modification of it is accepted I will volunteer
> to create the file and canvas python-dev for names.

I accept it :)

From benjamin at python.org  Wed Sep 16 00:55:35 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 15 Sep 2009 17:55:35 -0500
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
Message-ID: <1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>

2009/9/15 Jesse Noller <jnoller at gmail.com>:
> On Tue, Sep 15, 2009 at 6:38 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> Hmm, tables in a text file? I can see it, it's just always wacky.

emacs table mode!

>
> +1
>
>> Table (1) would list, I propose, three categories of people:
>> (a) 'official maintainer(s)', (b) experts, and (c) contributors.
>> An 'official maintainer' would be someone willing to take more-or-less
>> full responsibility for a module (such as Jesse for Multiprocessing).
>> Experts would be people who feel they have a good working knowledge of
>> the module and would not be afraid to sign off on the advisability and
>> quality of a feature/bug fix when there is a question. ?Contributors would
>> be anyone else with a more than casual knowledge of the module, but who
>> aren't comfortable with signing off on the advisability of non-trivial
>> patches/feature requests. ?My rational for including the third category is
>> to have a pool of people who can self-promote as needed. ?These people
>> can decide that it is OK to make the decision once they see that
>> there is no one willing to declare themselves an expert in that module.
>> I unfortunately expect a non-trivial number of modules to fall into
>> this category.
>
> +1, provided we can get good information into this, this could help
> out a lot. Can I ask that instead of just a misc file, we put this in
> the official documentation, in ReST format? I'd like to be able to
> point to all official-like.

Why? It's not meant to be all official-like, just to help people
triaging bugs or having questions about the code. I don't see how it
would benefit the person trying to find out how to use module X in the
official docs.



-- 
Regards,
Benjamin

From solipsis at pitrou.net  Wed Sep 16 01:01:05 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 01:01:05 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk> <mal@egenix.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
Message-ID: <1253055665.5678.108.camel@localhost>

Le mardi 15 septembre 2009 ? 18:38 -0400, R. David Murray a ?crit :
> 
> Table (1) would list, I propose, three categories of people:
> (a) 'official maintainer(s)', (b) experts, and (c) contributors.

This is too complicated IMO.
(a) + (b) is very sufficient and perhaps still not simple enough.
I don't see any strong difference between maintainers and experts.
As for casual contributors, I don't see any point in an exhaustive
listing of them (which, depending on the module, may be very long and
tedious and maintain).

> An 'official maintainer' would be someone willing to take more-or-less
> full responsibility for a module (such as Jesse for Multiprocessing).

I don't think "full responsibility" is a good thing. See my other
message (at 00:52 CEST) for why I think so.

As a summary:

A maintainer is someone who has a reasonable authority over a piece of
code; it may be worth asking him for review or permission, but no core
developer should be *required* to do so. On the contrary, we must
encourage other people to be autonomous, learn the code, and be able to
make decisions on their own. This is the only way to avoid dead ends
like we nowadays have with several modules. It also discourages
possessive behaviours (wrt. feature set, coding style, etc.), which is
always a good thing.

Regards

Antoine.



From jnoller at gmail.com  Wed Sep 16 00:58:04 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 18:58:04 -0400
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253055135.5678.100.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
	<4222a8490909151522w5e99c1dfw63c34a9454c3252b@mail.gmail.com>
	<1253055135.5678.100.camel@localhost>
Message-ID: <4222a8490909151558k5d77146fg6a273cb4a083536f@mail.gmail.com>

On Tue, Sep 15, 2009 at 6:52 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 18:22 -0400, Jesse Noller a ?crit :

> Sorry, what's that URL supposed to prove? What does it even represent?
> It is a populist argument at best, because I won't skim through those
> 176 bugs to try and make sense of your argument. If you want to make a
> point, please don't try to leave the burden of proof on me.
>
> Actually, I'll just take one of them, because I know it quite well:
> http://bugs.python.org/issue4967
>
> This is a bug in _ssl for which I had to write a patch, although I knew
> nothing about _ssl, because the owner wouldn't react. He wouldn't react
> for review either. The bug *had* to be fixed because it blocked the
> whole process of including the C IO library.
> Finally, Benjamin committed the patch. The owner didn't give a sign of
> life *during the whole process*. He isn't dead, he still sometimes
> contributes to python-dev, but he was totally unavailable when his
> presence was needed.

And there are over 176 bugs with patches, and more needing patches
which could use patches, docs, or tests. I guess that makes all of us
negligent and bad maintainers.

> So much for owners being a good thing.

So much for bad owners. Owners, by the very definition, must be
active, and responsive (even if it's not real-time). People who are
domain experts, but who are largely MIA are not a benefit.

>> The fact is, we need people who feel responsibility for every one of
>> these modules to review patches, and have some amount of mental design
>> integrity to ensure modules don't just wander off into the sunset and
>> die.
>
> But this is not the same as having an owner.

yes it is, but I think we're arguing the semantics of the word "owner"
- how about "person on the hook"

> Perhaps it wasn't clear, but I draw a clear separation between exclusive
> owners and maintainers.

Maybe we can agree on "maintainer" than "owner" - I did not mean
exclusive ownership, and I apologize if I gave that impression.

> I'm all for non-exclusive maintainership, with people having reasonable
> authority over code they understand thoroughly. You taking care of
> multiprocessing falls into this category (as long as you don't demand to
> approve of all changes before they are committed).

If I did that, I'd go insane.

> I'm against ownership, however. I'm against mandatory signoffs
> (imprimaturs), and I'm against the possessive sentiment that some might
> develop towards a piece of code they contributed. Any core developer
> should be allowed to commit a patch if he thinks the patch is reasonable
> enough and serves an useful purpose.

Agreed.

> Ownership prevents proper maintenance when the owner is absent (which
> *will* happen, since we are all unpaid volunteers). It discourages other
> people from getting acquainted with the code, which gradually makes the
> problem worse. Furthermore, it is often correlated with strange
> (personal) idioms, coding styles and design principles.

Agreed; but the maintainer should at least have a chance to say
something, or be +noisy on issues at very least. I completely agree
code dictatorship is bad, and I've seen it harm open source, and
business code bases *a lot*.

jesse

From jnoller at gmail.com  Wed Sep 16 01:00:43 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 19:00:43 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
Message-ID: <4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>

On Tue, Sep 15, 2009 at 6:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/15 Jesse Noller <jnoller at gmail.com>:
>> On Tue, Sep 15, 2009 at 6:38 PM, R. David Murray <rdmurray at bitdance.com> wrote:
>> Hmm, tables in a text file? I can see it, it's just always wacky.
>
> emacs table mode!

while emacs might be the One True Editor, some of us are Unwashed
Heathens who don't use it. So saying "emacs table mode" reads as "as
useful as a beef jerky bikini" ;)


> Why? It's not meant to be all official-like, just to help people
> triaging bugs or having questions about the code. I don't see how it
> would benefit the person trying to find out how to use module X in the
> official docs.
>

Having a table in the official docs at least gives people an idea who
to +noisy on bugs. How many multiprocessing bugs have you had to
reassign, or even *add* me to because people outside of our group
don't know?

From ziade.tarek at gmail.com  Wed Sep 16 01:01:37 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 16 Sep 2009 01:01:37 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
Message-ID: <94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>

On Wed, Sep 16, 2009 at 12:38 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> It has been mentioned here that some bugs languish in the tracker because
> there is no one willing to say "yes" or "no" to them. ?In at least some
> cases this may be because it is unclear who the best person is to ask
> for a decision when the participants in the issue don't feel qualified
> to decide. ?And in some cases, some of the people it may be unclear to
> may be the very people who _could_ decide...if only they knew that they
> were the closest thing to an expert on that module that we have.
>
> In a discussion on IRC we came up with a proposal for a simple tool that
> might help out in this situation. ?I would like to propose that we create
> a file, tentatively named MISC/maintainers.txt, that contains two tables:
> (1) a table of all the modules in the standard library and (2) a table of
> 'areas of expertise' (things like Unicode, arithmetic, etc). ?Table (2)
> would be the simpler, and would just list people who felt they had
> enough expertise in the given area to be willing to make judgement
> calls on relevant issues on request.

That would be a third source of info about who maintains what.

If this file is created it should maybe override and cover what PEP
360 and PEP 291 provides

- some modules/packages backward compatibility infos
- a list of externally maintained packages

(some of these info are a bit outdated though)

Regards
Tarek

From jnoller at gmail.com  Wed Sep 16 01:02:24 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 19:02:24 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <1253055665.5678.108.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<1253055665.5678.108.camel@localhost>
Message-ID: <4222a8490909151602r3b37b8b8n33dd0e22ac0b435a@mail.gmail.com>

On Tue, Sep 15, 2009 at 7:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 18:38 -0400, R. David Murray a ?crit :
>>
>> Table (1) would list, I propose, three categories of people:
>> (a) 'official maintainer(s)', (b) experts, and (c) contributors.
>
> This is too complicated IMO.
> (a) + (b) is very sufficient and perhaps still not simple enough.
> I don't see any strong difference between maintainers and experts.
> As for casual contributors, I don't see any point in an exhaustive
> listing of them (which, depending on the module, may be very long and
> tedious and maintain).

but hey, if they're willing to write out all that info antoine ;)

I think Antoine has a good point here. Can we start shorter (and simpler)?

>> An 'official maintainer' would be someone willing to take more-or-less
>> full responsibility for a module (such as Jesse for Multiprocessing).
>
> I don't think "full responsibility" is a good thing. See my other
> message (at 00:52 CEST) for why I think so.

Let's just stick to "tinpot maintainer" ;)

jesse

From benjamin at python.org  Wed Sep 16 01:03:51 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 15 Sep 2009 18:03:51 -0500
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
Message-ID: <1afaf6160909151603h29f10ffflf6d50d4083d19b99@mail.gmail.com>

2009/9/15 Jesse Noller <jnoller at gmail.com>:
> On Tue, Sep 15, 2009 at 6:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> Why? It's not meant to be all official-like, just to help people
>> triaging bugs or having questions about the code. I don't see how it
>> would benefit the person trying to find out how to use module X in the
>> official docs.
>>
>
> Having a table in the official docs at least gives people an idea who
> to +noisy on bugs. How many multiprocessing bugs have you had to
> reassign, or even *add* me to because people outside of our group
> don't know?

I see your point, but I think it could be better solved by a auto-nosy
for the "Components" menu or the ability to tag issues.



-- 
Regards,
Benjamin

From solipsis at pitrou.net  Wed Sep 16 01:07:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 01:07:47 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <4222a8490909151558k5d77146fg6a273cb4a083536f@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<1253038514.5678.32.camel@localhost>
	<4AAFDBAD.7040906@voidspace.org.uk>
	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>
	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>
	<4AAFEE6B.3070900@egenix.com>
	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>
	<1253049748.5678.75.camel@localhost>
	<4222a8490909151522w5e99c1dfw63c34a9454c3252b@mail.gmail.com>
	<1253055135.5678.100.camel@localhost>
	<4222a8490909151558k5d77146fg6a273cb4a083536f@mail.gmail.com>
Message-ID: <1253056067.5678.113.camel@localhost>

> > Perhaps it wasn't clear, but I draw a clear separation between
exclusive
> > owners and maintainers.
> 
> Maybe we can agree on "maintainer" than "owner" - I did not mean
> exclusive ownership, and I apologize if I gave that impression.
> 
[...]
> Agreed; but the maintainer should at least have a chance to say
> something, or be +noisy on issues at very least. I completely agree
> code dictatorship is bad, and I've seen it harm open source, and
> business code bases *a lot*.

Ok, so we do agree after all :)

To elaborate a bit, I think the word "owner" is too strong and would
easily give those "owners" the impression that they have full power over
their respective modules. It is important to stress that code inside the
CPython repository (or the stdlib one, once it is split) can be modified
by any core developer with sufficiently good reasons.

It is even more important to stress it when including a new module in
the stdlib (and to refuse external maintainership).

Regards

Antoine.



From jnoller at gmail.com  Wed Sep 16 01:10:23 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 19:10:23 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <1afaf6160909151603h29f10ffflf6d50d4083d19b99@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
	<1afaf6160909151603h29f10ffflf6d50d4083d19b99@mail.gmail.com>
Message-ID: <4222a8490909151610w5b471d92sfbd4dcd7a8fbfe@mail.gmail.com>

On Tue, Sep 15, 2009 at 7:03 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/15 Jesse Noller <jnoller at gmail.com>:
>> On Tue, Sep 15, 2009 at 6:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
>>> Why? It's not meant to be all official-like, just to help people
>>> triaging bugs or having questions about the code. I don't see how it
>>> would benefit the person trying to find out how to use module X in the
>>> official docs.
>>>
>>
>> Having a table in the official docs at least gives people an idea who
>> to +noisy on bugs. How many multiprocessing bugs have you had to
>> reassign, or even *add* me to because people outside of our group
>> don't know?
>
> I see your point, but I think it could be better solved by a auto-nosy
> for the "Components" menu or the ability to tag issues.
>

Well, modulo tracker changes - I still think this could be useful in
the main docs. Not to mention sphinx just makes them so pretty.

jesse

From jnoller at gmail.com  Wed Sep 16 01:11:40 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 19:11:40 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
Message-ID: <4222a8490909151611y284cf2a3sf94f5c6cc7d50f68@mail.gmail.com>

On Tue, Sep 15, 2009 at 7:01 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Wed, Sep 16, 2009 at 12:38 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>> It has been mentioned here that some bugs languish in the tracker because
>> there is no one willing to say "yes" or "no" to them. ?In at least some
>> cases this may be because it is unclear who the best person is to ask
>> for a decision when the participants in the issue don't feel qualified
>> to decide. ?And in some cases, some of the people it may be unclear to
>> may be the very people who _could_ decide...if only they knew that they
>> were the closest thing to an expert on that module that we have.
>>
>> In a discussion on IRC we came up with a proposal for a simple tool that
>> might help out in this situation. ?I would like to propose that we create
>> a file, tentatively named MISC/maintainers.txt, that contains two tables:
>> (1) a table of all the modules in the standard library and (2) a table of
>> 'areas of expertise' (things like Unicode, arithmetic, etc). ?Table (2)
>> would be the simpler, and would just list people who felt they had
>> enough expertise in the given area to be willing to make judgement
>> calls on relevant issues on request.
>
> That would be a third source of info about who maintains what.
>
> If this file is created it should maybe override and cover what PEP
> 360 and PEP 291 provides
>
> - some modules/packages backward compatibility infos
> - a list of externally maintained packages
>
> (some of these info are a bit outdated though)

Oh man, those are dated. json isn't even in there.

From rdmurray at bitdance.com  Wed Sep 16 01:24:28 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 19:24:28 -0400 (EDT)
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4222a8490909151602r3b37b8b8n33dd0e22ac0b435a@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<1253055665.5678.108.camel@localhost>
	<4222a8490909151602r3b37b8b8n33dd0e22ac0b435a@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909151917260.17020@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 19:02, Jesse Noller wrote:
> On Tue, Sep 15, 2009 at 7:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le mardi 15 septembre 2009 ? 18:38 -0400, R. David Murray a ?crit :
>>>
>>> Table (1) would list, I propose, three categories of people:
>>> (a) 'official maintainer(s)', (b) experts, and (c) contributors.
>>
>> This is too complicated IMO.
>> (a) + (b) is very sufficient and perhaps still not simple enough.
>> I don't see any strong difference between maintainers and experts.
>> As for casual contributors, I don't see any point in an exhaustive
>> listing of them (which, depending on the module, may be very long and
>> tedious and maintain).
>
> but hey, if they're willing to write out all that info antoine ;)
>
> I think Antoine has a good point here. Can we start shorter (and simpler)?

OK, less work is fine by me.

So instead of a full blown table, we have a file with two sections, and
in each section we have a keyword (module name in the first section,
expertise area in the second) followed by a list of tracker ids for
people willing to make judgement calls on demand.  Any lines that are
blank means the person who doesn't feel qualified to make the judgement
call gets to make it anyway :)

This would also mean that this file serves a completely different
purpose than the PEPs, and is thus not redundant with them.

--David

From rdmurray at bitdance.com  Wed Sep 16 01:39:06 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 19:39:06 -0400 (EDT)
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>

On Wed, 16 Sep 2009 at 01:01, Tarek Ziad? wrote:
> That would be a third source of info about who maintains what.
>
> If this file is created it should maybe override and cover what PEP
> 360 and PEP 291 provides
>
> - some modules/packages backward compatibility infos

I think this (PEP 291) should be referenced in the header of the file,
since it would help inform various decisions.  What it lists is
something different from what maintainers.txt is proposed to list,
since the PEP is talking about the maintenance of the non-stdlib
versions of those modules.  (Granted, that's relevant for the
maintenance of the stdlib version, but not conclusive.)

> - a list of externally maintained packages

Antoine should hate this one (PEP 360) :)  And it is, essentially, a
deprecated PEP.  (Which opens the question of what we should do about
the modules it lists...though apparently we can now remove optik/optparse
from it.)

> (some of these info are a bit outdated though)

Indeed.

--David

From ubershmekel at gmail.com  Wed Sep 16 03:15:46 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Wed, 16 Sep 2009 04:15:46 +0300
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
Message-ID: <9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com>

In the age of wikis and mega-collaboration-web-toys, we're talking about a
text file? Of course having 2 different information sources sounds wacky,
but somehow somewhere this responsibility matrix should be googlable
and colourful.
Or at least linked ie -
http://en.wikipedia.org/wiki/Knights_of_the_Round_Table#List_of_Knights

On Wed, Sep 16, 2009 at 2:39 AM, R. David Murray <rdmurray at bitdance.com>wrote:

> On Wed, 16 Sep 2009 at 01:01, Tarek Ziad? wrote:
>
>> That would be a third source of info about who maintains what.
>>
>> If this file is created it should maybe override and cover what PEP
>> 360 and PEP 291 provides
>>
>> - some modules/packages backward compatibility infos
>>
>
> I think this (PEP 291) should be referenced in the header of the file,
> since it would help inform various decisions.  What it lists is
> something different from what maintainers.txt is proposed to list,
> since the PEP is talking about the maintenance of the non-stdlib
> versions of those modules.  (Granted, that's relevant for the
> maintenance of the stdlib version, but not conclusive.)
>
>  - a list of externally maintained packages
>>
>
> Antoine should hate this one (PEP 360) :)  And it is, essentially, a
> deprecated PEP.  (Which opens the question of what we should do about
> the modules it lists...though apparently we can now remove optik/optparse
> from it.)
>
>  (some of these info are a bit outdated though)
>>
>
> Indeed.
>
> --David
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090916/96e6faee/attachment.htm>

From ctb at msu.edu  Wed Sep 16 03:16:26 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Tue, 15 Sep 2009 18:16:26 -0700
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
References: <200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
Message-ID: <20090916011626.GF29739@idyll.org>

On Tue, Sep 15, 2009 at 07:00:43PM -0400, Jesse Noller wrote:
> On Tue, Sep 15, 2009 at 6:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
> > 2009/9/15 Jesse Noller <jnoller at gmail.com>:
> >> On Tue, Sep 15, 2009 at 6:38 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> >> Hmm, tables in a text file? I can see it, it's just always wacky.
> >
> > emacs table mode!
> 
> while emacs might be the One True Editor, some of us are Unwashed
> Heathens who don't use it. So saying "emacs table mode" reads as "as
> useful as a beef jerky bikini" ;)

DIVERSITY!

Sorry, had to say it.  Nothing against beef jerky bikinis.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From jnoller at gmail.com  Wed Sep 16 03:29:16 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 21:29:16 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com>
Message-ID: <4222a8490909151829t47eca92dhb9499913966ffcfe@mail.gmail.com>

On Tue, Sep 15, 2009 at 9:15 PM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> In the age of wikis and mega-collaboration-web-toys, we're talking about a
> text file? Of course having 2 different information sources sounds wacky,
> but somehow somewhere this responsibility matrix should be googlable
> and?colourful.
> Or at least linked ie
> -?http://en.wikipedia.org/wiki/Knights_of_the_Round_Table#List_of_Knights

I would rather a sharp poke in the eye than a wiki. Besides; we use
source control to do things like this, it's collaborative!

From rdmurray at bitdance.com  Wed Sep 16 03:40:34 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 21:40:34 -0400 (EDT)
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<200909151029.n8FATHmq028625@theraft.openend.se> 
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net> 
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com> 
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909152134170.17020@kimball.webabinitio.net>

On Wed, 16 Sep 2009 at 04:15, Yuvgoog Greenle wrote:
> In the age of wikis and mega-collaboration-web-toys, we're talking about a
> text file? Of course having 2 different information sources sounds wacky,
> but somehow somewhere this responsibility matrix should be googlable
> and colourful.

Well, it'll be Googlable via

http://svn.python.org/view/python/trunk/Misc/maintainers.txt

or the hg equivalent.  It won't be colorful or linked that way.

Perhaps a script that uploads a hyperlinked version to a page linked from
the developer FAQ would be worthwhile.  Care to volunteer to write one?

As Jesse said, the reason for putting it in the repository is because
that's where the core team does its collaboration.

--David

From brett at python.org  Wed Sep 16 03:57:49 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 18:57:49 -0700
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com> 
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net> 
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com> 
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
Message-ID: <bbaeab100909151857r41fcf33dwde0e77c7d9dbf000@mail.gmail.com>

On Tue, Sep 15, 2009 at 16:39, R. David Murray <rdmurray at bitdance.com> wrote:
> On Wed, 16 Sep 2009 at 01:01, Tarek Ziad? wrote:
>>
>> That would be a third source of info about who maintains what.
>>
>> If this file is created it should maybe override and cover what PEP
>> 360 and PEP 291 provides
>>
>> - some modules/packages backward compatibility infos
>
> I think this (PEP 291) should be referenced in the header of the file,
> since it would help inform various decisions. ?What it lists is
> something different from what maintainers.txt is proposed to list,
> since the PEP is talking about the maintenance of the non-stdlib
> versions of those modules. ?(Granted, that's relevant for the
> maintenance of the stdlib version, but not conclusive.)
>
>> - a list of externally maintained packages
>
> Antoine should hate this one (PEP 360) :) ?And it is, essentially, a
> deprecated PEP. ?(Which opens the question of what we should do about
> the modules it lists...though apparently we can now remove optik/optparse
> from it.)

The people who brought them in are the experts for those modules so
just list them as such.

-Brett

From ubershmekel at gmail.com  Wed Sep 16 03:59:01 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Wed, 16 Sep 2009 04:59:01 +0300
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909152134170.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com>
	<Pine.LNX.4.64.0909152134170.17020@kimball.webabinitio.net>
Message-ID: <9d153b7c0909151859x62170b39o96d5aad8f8caafed@mail.gmail.com>

Gladly :)
I would need a few pointers though, where to write/upload, in what format
and where is the developer's information (contact etc)? I hope passing this
info won't be as bad as writing the script.

On Wed, Sep 16, 2009 at 4:40 AM, R. David Murray <rdmurray at bitdance.com>wrote:

> On Wed, 16 Sep 2009 at 04:15, Yuvgoog Greenle wrote:
>
>> In the age of wikis and mega-collaboration-web-toys, we're talking about a
>> text file? Of course having 2 different information sources sounds wacky,
>> but somehow somewhere this responsibility matrix should be googlable
>> and colourful.
>>
>
> Well, it'll be Googlable via
>
> http://svn.python.org/view/python/trunk/Misc/maintainers.txt
>
> or the hg equivalent.  It won't be colorful or linked that way.
>
> Perhaps a script that uploads a hyperlinked version to a page linked from
> the developer FAQ would be worthwhile.  Care to volunteer to write one?
>
> As Jesse said, the reason for putting it in the repository is because
> that's where the core team does its collaboration.
>
> --David
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090916/c979a12f/attachment.htm>

From brett at python.org  Wed Sep 16 04:00:02 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Sep 2009 19:00:02 -0700
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151917260.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com> 
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net> 
	<1253055665.5678.108.camel@localhost>
	<4222a8490909151602r3b37b8b8n33dd0e22ac0b435a@mail.gmail.com> 
	<Pine.LNX.4.64.0909151917260.17020@kimball.webabinitio.net>
Message-ID: <bbaeab100909151900x4a5a636bodb876f0b27fa9932@mail.gmail.com>

On Tue, Sep 15, 2009 at 16:24, R. David Murray <rdmurray at bitdance.com> wrote:
> On Tue, 15 Sep 2009 at 19:02, Jesse Noller wrote:
>>
>> On Tue, Sep 15, 2009 at 7:01 PM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>>>
>>> Le mardi 15 septembre 2009 ? 18:38 -0400, R. David Murray a ?crit :
>>>>
>>>> Table (1) would list, I propose, three categories of people:
>>>> (a) 'official maintainer(s)', (b) experts, and (c) contributors.
>>>
>>> This is too complicated IMO.
>>> (a) + (b) is very sufficient and perhaps still not simple enough.
>>> I don't see any strong difference between maintainers and experts.
>>> As for casual contributors, I don't see any point in an exhaustive
>>> listing of them (which, depending on the module, may be very long and
>>> tedious and maintain).
>>
>> but hey, if they're willing to write out all that info antoine ;)
>>
>> I think Antoine has a good point here. Can we start shorter (and simpler)?
>
> OK, less work is fine by me.
>
> So instead of a full blown table, we have a file with two sections, and
> in each section we have a keyword (module name in the first section,
> expertise area in the second) followed by a list of tracker ids for
> people willing to make judgement calls on demand. ?Any lines that are
> blank means the person who doesn't feel qualified to make the judgement
> call gets to make it anyway :)
>

Sounds great to me!

> This would also mean that this file serves a completely different
> purpose than the PEPs, and is thus not redundant with them.

Right. And at some point we can write some code to mine the stdlib to
make sure the list is not missing any modules and to potentially
suggest maintainers for the blank modules.

And for sanity reasons you might want to only do this in py3k.

-Brett

From rdmurray at bitdance.com  Wed Sep 16 04:23:23 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 22:23:23 -0400 (EDT)
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <9d153b7c0909151859x62170b39o96d5aad8f8caafed@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net> 
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com> 
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net> 
	<9d153b7c0909151815x78658354ve0038f71a5768fd3@mail.gmail.com> 
	<Pine.LNX.4.64.0909152134170.17020@kimball.webabinitio.net>
	<9d153b7c0909151859x62170b39o96d5aad8f8caafed@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909152216060.17020@kimball.webabinitio.net>

On Wed, 16 Sep 2009 at 04:59, Yuvgoog Greenle wrote:
> Gladly :)
> I would need a few pointers though, where to write/upload, in what format
> and where is the developer's information (contact etc)? I hope passing this
> info won't be as bad as writing the script.

I think you'll want to make sure there's consensus that this is a good
idea, first.

I'm not sure what the details would be, actually, since I haven't yet
been involved in the Python web site.  I know the docs themselves are
automatically uploaded...but we'd have to get buy-in from the pydotorg
people to do this (and they'd be who would know how to do it).

As for the linking, there you'd have to figure out how to map tracker ids
into links into the tracker user page, which probably involves doing an
query into the tracker database.  The place to go for info there would
be the tracker development list (tracker-discuss).

So finding out everything you need to know is probably harder than
writing the script :)

--David

From python at rcn.com  Wed Sep 16 04:42:45 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 15 Sep 2009 19:42:45 -0700
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com>
	<4AAEC59C.7060605@egenix.com><200909151029.n8FATHmq028625@theraft.openend.se><43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost><bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com><Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net><94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
Message-ID: <0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>


> That would be a third source of info about who maintains what.

Is this just a formalization of what we already do now?


I.E.. Georg "just-knows" that I maintain sets, collections, random, itertools;
he assigns those bugs or feature requests to me; and
if i dissappear, then other developers step in and rearrange all my code ;-)


Raymond


From jnoller at gmail.com  Wed Sep 16 04:48:20 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Sep 2009 22:48:20 -0400
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
Message-ID: <4222a8490909151948m506eb85v89d6984218e29f71@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:42 PM, Raymond Hettinger <python at rcn.com> wrote:
>
>> That would be a third source of info about who maintains what.
>
> Is this just a formalization of what we already do now?
>
>
> I.E.. Georg "just-knows" that I maintain sets, collections, random,
> itertools;
> he assigns those bugs or feature requests to me; and
> if i dissappear, then other developers step in and rearrange all my code ;-)

Somewhat, yes - it is a formalization. But it also helps some of us
who don't spend enough time in the tracker to know *where* to move
things. Ideally, this also helps us identify gaps in "personnel"
coverage so we can:

1> Find someone who loves it enough to maintain it
2> jettison it into space if needed (I kid... sorta!)

So yeah, it's a formalization + a data mining operation, and
hopefully, a resource to the community to help identify people who
should be added to bugs so poor Georg doesn't have to reassign
everything.

jesse

From g.brandl at gmx.net  Wed Sep 16 10:04:00 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 10:04:00 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
Message-ID: <h8q69c$q8i$1@ger.gmane.org>

Jesse Noller schrieb:
> On Tue, Sep 15, 2009 at 6:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2009/9/15 Jesse Noller <jnoller at gmail.com>:
>>> On Tue, Sep 15, 2009 at 6:38 PM, R. David Murray <rdmurray at bitdance.com> wrote:
>>> Hmm, tables in a text file? I can see it, it's just always wacky.
>>
>> emacs table mode!
> 
> while emacs might be the One True Editor, some of us are Unwashed
> Heathens who don't use it. So saying "emacs table mode" reads as "as
> useful as a beef jerky bikini" ;)

Think of it that way: Emacs is the bacon of editors.

>> Why? It's not meant to be all official-like, just to help people
>> triaging bugs or having questions about the code. I don't see how it
>> would benefit the person trying to find out how to use module X in the
>> official docs.
>>
> 
> Having a table in the official docs at least gives people an idea who
> to +noisy on bugs. How many multiprocessing bugs have you had to
> reassign, or even *add* me to because people outside of our group
> don't know?

Hey, I got another idea! What about we simply add a second set of docs, for
developers?

We have lots of information scattered throughout the website, the source
and the heads of important people (the most volatile sort), that, put
together, should make a nice set of docs that could be collected (and made
a sphinx tree).

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From lac at openend.se  Wed Sep 16 10:10:21 2009
From: lac at openend.se (Laura Creighton)
Date: Wed, 16 Sep 2009 10:10:21 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: Message from Georg Brandl <g.brandl@gmx.net> 
	of "Wed, 16 Sep 2009 10:04:00 +0200." <h8q69c$q8i$1@ger.gmane.org> 
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>
	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>
	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
	<h8q69c$q8i$1@ger.gmane.org> 
Message-ID: <200909160810.n8G8ALmM023008@theraft.openend.se>

In a message of Wed, 16 Sep 2009 10:04:00 +0200, Georg Brandl writes:
>Hey, I got another idea! What about we simply add a second set of docs, f
>or
>developers?
>
>We have lots of information scattered throughout the website, the source
>and the heads of important people (the most volatile sort), that, put
>together, should make a nice set of docs that could be collected (and mad
>e
>a sphinx tree).
>
>Georg

I really like this idea, which also accomplishes the goal of making
the python development process more transparent to non-core-developers.
This, of course, assumes that you would let us read such a thing.  Is
that a fair assumption?

Laura

From mal at egenix.com  Wed Sep 16 10:20:54 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Sep 2009 10:20:54 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <h8q69c$q8i$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>
	<h8q69c$q8i$1@ger.gmane.org>
Message-ID: <4AB09FE6.6030308@egenix.com>

Georg Brandl wrote:
>> Having a table in the official docs at least gives people an idea who
>> to +noisy on bugs. How many multiprocessing bugs have you had to
>> reassign, or even *add* me to because people outside of our group
>> don't know?

As general note: I have the impression that the noisy list feature
of the bug tracker is not exactly ideal to get attention for certain
bugs from the right people.

Perhaps it's better to just ask the various developers to setup
email filters for this purpose; or maybe in addition to using the
noisy lists.

I've done that and while it's not perfect, it works a lot better than
relying on someone putting me on e.g. the platform.py-related ticket
noisy list.

> Hey, I got another idea! What about we simply add a second set of docs, for
> developers?
> 
> We have lots of information scattered throughout the website, the source
> and the heads of important people (the most volatile sort), that, put
> together, should make a nice set of docs that could be collected (and made
> a sphinx tree).

Wouldn't a developer wiki (readable for everyone, writable only
for developers) be a better and more direct way to handle this ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 16 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From g.brandl at gmx.net  Wed Sep 16 10:43:05 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 10:43:05 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <200909160810.n8G8ALmM023008@theraft.openend.se>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>	<h8q69c$q8i$1@ger.gmane.org>
	<200909160810.n8G8ALmM023008@theraft.openend.se>
Message-ID: <h8q8il$1v6$1@ger.gmane.org>

Laura Creighton schrieb:
> In a message of Wed, 16 Sep 2009 10:04:00 +0200, Georg Brandl writes:
>>Hey, I got another idea! What about we simply add a second set of docs, f
>>or
>>developers?
>>
>>We have lots of information scattered throughout the website, the source
>>and the heads of important people (the most volatile sort), that, put
>>together, should make a nice set of docs that could be collected (and mad
>>e
>>a sphinx tree).
>>
>>Georg
> 
> I really like this idea, which also accomplishes the goal of making
> the python development process more transparent to non-core-developers.
> This, of course, assumes that you would let us read such a thing.  Is
> that a fair assumption?

Yes, certainly -- all the resources are public now, and the new thing
would be in the source checkout, and I'd set up a regular build as for
the docs, e.g. at http://devdocs.python.org/.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Wed Sep 16 10:47:26 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 10:47:26 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4AB09FE6.6030308@egenix.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>	<h8q69c$q8i$1@ger.gmane.org>
	<4AB09FE6.6030308@egenix.com>
Message-ID: <h8q8qq$37t$1@ger.gmane.org>

M.-A. Lemburg schrieb:
> Georg Brandl wrote:
>>> Having a table in the official docs at least gives people an idea who
>>> to +noisy on bugs. How many multiprocessing bugs have you had to
>>> reassign, or even *add* me to because people outside of our group
>>> don't know?
> 
> As general note: I have the impression that the noisy list feature
> of the bug tracker is not exactly ideal to get attention for certain
> bugs from the right people.
> 
> Perhaps it's better to just ask the various developers to setup
> email filters for this purpose; or maybe in addition to using the
> noisy lists.
> 
> I've done that and while it's not perfect, it works a lot better than
> relying on someone putting me on e.g. the platform.py-related ticket
> noisy list.

That's why at PyCon (or was it Europython?) we thought about a "tags"
field for issues.  The main use for tags would be module names (and
it should be advertised as such in the UI for submitting bugs)
with auto-nosy associations for module maintainers.

(I don't want to call it "module name" because I could envision other
useful tags as well.)

>> Hey, I got another idea! What about we simply add a second set of docs, for
>> developers?
>> 
>> We have lots of information scattered throughout the website, the source
>> and the heads of important people (the most volatile sort), that, put
>> together, should make a nice set of docs that could be collected (and made
>> a sphinx tree).
> 
> Wouldn't a developer wiki (readable for everyone, writable only
> for developers) be a better and more direct way to handle this ?

The source would be in the repo, so that's already writable for all
developers (without them having to sign up to just *another* system,
with another password etc.), and for the public an automated regular
build, like I suggested in response to Laura, would be even nicer
than a read-only wiki.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From michael at voidspace.org.uk  Wed Sep 16 11:43:03 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 10:43:03 +0100
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<4AAE94BF.8080407@voidspace.org.uk> <4AAEBEBE.1050105@egenix.com>
	<4AAEBFEC.7060507@voidspace.org.uk>
	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
Message-ID: <4AB0B327.6040807@voidspace.org.uk>

Late to the party.

+1 for a list
+1 for maintainers not owners

Michael

R. David Murray wrote:
> It has been mentioned here that some bugs languish in the tracker because
> there is no one willing to say "yes" or "no" to them.  In at least some
> cases this may be because it is unclear who the best person is to ask
> for a decision when the participants in the issue don't feel qualified
> to decide.  And in some cases, some of the people it may be unclear to
> may be the very people who _could_ decide...if only they knew that they
> were the closest thing to an expert on that module that we have.
>
> In a discussion on IRC we came up with a proposal for a simple tool that
> might help out in this situation.  I would like to propose that we create
> a file, tentatively named MISC/maintainers.txt, that contains two tables:
> (1) a table of all the modules in the standard library and (2) a table of
> 'areas of expertise' (things like Unicode, arithmetic, etc).  Table (2)
> would be the simpler, and would just list people who felt they had
> enough expertise in the given area to be willing to make judgement
> calls on relevant issues on request.
>
> Table (1) would list, I propose, three categories of people:
> (a) 'official maintainer(s)', (b) experts, and (c) contributors.
> An 'official maintainer' would be someone willing to take more-or-less
> full responsibility for a module (such as Jesse for Multiprocessing).
> Experts would be people who feel they have a good working knowledge of
> the module and would not be afraid to sign off on the advisability and
> quality of a feature/bug fix when there is a question.  Contributors 
> would
> be anyone else with a more than casual knowledge of the module, but who
> aren't comfortable with signing off on the advisability of non-trivial
> patches/feature requests.  My rational for including the third 
> category is
> to have a pool of people who can self-promote as needed.  These people
> can decide that it is OK to make the decision once they see that
> there is no one willing to declare themselves an expert in that module.
> I unfortunately expect a non-trivial number of modules to fall into
> this category.
>
> The listing in the table would be by tracker id, to facilitate making
> people nosy on issues.  The tracker id can be used to discover names
> and email addresses if those are needed instead.
>
> Obviously one problem would be keeping this up to date, since when 
> someone
> stops contributing they are fairly likely to not remove their name from
> the list.  This task could be handled by anyone who does issue triage:
> if a triage person notices a maintainer or expert who has not responded
> to a request for decision on an issue, they can try pinging the person
> directly, and if they still get no response, mark the person (or request
> that the person be marked, if the triage person is not a committer) as
> 'deprecated'(*) in the maintainers file.
>
> If this proposal or a modification of it is accepted I will volunteer
> to create the file and canvas python-dev for names.
>
> --David
>
> (*) I mean 'inactive' of course. :)
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig


-- 
http://www.ironpythoninaction.com/


From armin.ronacher at active-4.com  Wed Sep 16 14:28:13 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 16 Sep 2009 14:28:13 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<mal@egenix.com>	<4AAEC59C.7060605@egenix.com><200909151029.n8FATHmq028625@theraft.openend.se><43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost><bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com><Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net><94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
Message-ID: <4AB0D9DD.6030800@active-4.com>

Hi,

Raymond Hettinger wrote:
> if i dissappear, then other developers step in and rearrange all my code
s/other developers/me/
s/rearrange all my code/rename all my classes/

FTFY


Regards,
Armin

From armin.ronacher at active-4.com  Wed Sep 16 14:29:50 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 16 Sep 2009 14:29:50 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <h8q8qq$37t$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>	<h8q69c$q8i$1@ger.gmane.org>	<4AB09FE6.6030308@egenix.com>
	<h8q8qq$37t$1@ger.gmane.org>
Message-ID: <4AB0DA3E.6030400@active-4.com>

Hi,

Georg Brandl wrote:
> That's why at PyCon (or was it Europython?) we thought about a "tags"
> field for issues.  The main use for tags would be module names (and
> it should be advertised as such in the UI for submitting bugs)
> with auto-nosy associations for module maintainers.
Arbitrary tags in a bug tracker are annoying.  People will not try to
find exiting tags but make up their own, making it hard to query for it.
 I would rather see an automatically generated module list based on the
hg tip / svn head.


Regards,
Armin

From armin.ronacher at active-4.com  Wed Sep 16 14:19:34 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 16 Sep 2009 14:19:34 +0200
Subject: [stdlib-sig] Evolving the Standard Library
Message-ID: <4AB0D7D6.7030204@active-4.com>

Hi everybody,

I'm known for my dislike of the standard libray.  In the past I wrote
some blog posts about this topic into my personal blog already.  However
as many people pointed out earlier, a blog is not the place for this
kind of criticism.  Not only that, also just ranting about a topic does
not help at all.  Yesterday I subscribed to the stdlib-sig and
immediately tons of mails ended up in my inbox.  A quick look at the
mail archives confirms what I was afraid of: this list is really high
traffic.  I tried to read up some of the discussions I missed but it's
nearly impossible to do that.

I would love to sum up my thoughts about the standard library here and
my ideas to improve it.  This list of ideas and improvements does not
include any unrealistic plans such as rewriting the standard library, an
approach I was a big fan of.

I can see a couple of problems with the standard library currently, and
some reasons why that is the case.  If we look back on the history of
Python it's obvious that a large number of modules in the standard
library appeared out of the need of a single developer or company a
while ago.  Many of these libraries finally disappered or where renamed
in the big standard library reorganization in Python 3 and I'm very
happy that this happened.  However at the same time a large number of
the modules still continue to show their age.

Python is currently heading into a new direction many people would not
have thought about a few years ago.  And that are web applications.
For web applications different rules apply than for desktop
applications.  Command line scripts or GUI applications are mostly fine
with shared state on module level, web applications are not.  It is true
that Python currently has some issues with high concurrency and people
try to fix that by forking and spawning new processes which certainly
hides away the problem of shared state, but that does not solve it.  In
fact, very recently Facebook open sourced the Tornado framework which
does very well at high concurrency by using async IO.  Also this recent
interest in Tornado will probably also motivate Twisted developers to
improve their project's documentation and performance, because
competition is often the what causes projects to improve.

Now if we look at the standard library, we can see many modules that
just do not work in such environments because they have some sort of
shared state.  The most obvious ones are certainly the `locale` module
and all the other modules that change behavior based on the locale
settings.  Did you know that every major Python framework reimplements
time formatting even for something as simple as HTTP headers, because
Python does not provide a way to format the time to english strings
reliably?  But there are certainly more modules that have this sort of
problem.

Also we have many modules in the standard library that in my opinion
just do not belong there.  From my point of view, stuff like XML does
not belong into the standard library.  But it appears that not many
people agree with me on this one.  But even if everybody would,
backwards compatibility would still be a good reason to keep these
modules around.

Besides modules that do not work in every environment or modules that
were probably a mistake to include, we also have modules in the standard
library with a hideous implementation or no reusability, forcing people
to reinvent what's already there.  For a long time, `urllib` was a
module I would have listed there, but as of Python 2.6, the module
largely improved by exposing the underlaying socket more which finally
alllows us to set the timeout in a reliable way.  But there are still a
ton of modules in the library that cause troubles for people.  `dis` is
one of them.  The implementation of dis prints to stdout no matter what
you do.  Of course you can replace sys.stdout with something else for a
brief moment, but again: this is not something we should aim for or
advertise because it breaks for many people.  `Cookie` is a module
people monkey patched for a while (badly) to support the http only flag.
 Not only does the code expose a weird API, it is also nearly impossible
to extend and even ships cookie subclasses that use unsigned pickles and
trust the client.  `cgi` has again, shared state on the global namespace
that alters the behavior of the lirbary.  Of course it was never
intended to be used by anything but `cgi`, but that leaves people
reimplementing it or abusing it.

So when the discussion started replacing `optparse` with `argparse`,
because the former is unmaintained I became alerted.  My wishes have
always been the standard library to be a reliable fallback to be used if
everything else fails.  Something I can rely on which will not change,
except for maybe some additions or modules moved to different locations.
As Python developers we became used to moving import locations a lot.
It it's `cPickle` or any of the element tree implementations, you name it.

I wonder if the solution to this problem wouldn't be a largely improved
packaging system and some sort of standardized reviewing process for the
standard library.  Currently there is not even an accepted style for
modules ending up in the Python distribution.  That, and a group of
people, dedicated to standard library refactoring.  The majority of
libraries in the standard library are small and easy to understand, I'm
sure they are perfectly suited for students on projects like GSOC or
GHOP to work on.  They could even be used as some sort of "playground"
for new Python developers.

Ubuntu recently started the "100 paper cuts" project.  There people work
on tiny little patches to improve the system, rather to replace
components.  Even though a large place of the standard library appears
to be broken by design they could still be redesigned on the small
scale, without breaking backwards compatibility.

Of course libraries like `locale` and `logging` are hard to change, but
it would still be possible.  For `locale` it would probably a useful
idea to go into the direction of datetime, where the timezone
information is left to a 3rd party library.  `locale` could provide some
hooks for libraries like `babel` to fill the gap.  On the other hand
`Cookie` would be very easy to fix by moving the parsing code into a
separate function and refactoring the cookie objects.

We could probably also start a poll out there with well-selected
questions of what users think about parts of the library.  And for that
poll it would make a lot of sense to not just ask the questions and
evaluating the results, but also track the area the user is coming from
(small size company, open / closed source, web development etc.).
Because we all are biased and seeing results grouped by some of these
factoids could be enlightening.  That said, it could tell us that I'm
completely wrong with my ideas of how the state of the standard library.

But how realistic is it to refactor the standard library?  I don't know.
For a long time people were pretty sure Python will not get any faster
and yet Unleaden Swallow is doing some really amazing progress.

If we want to push Python foward into new areas, and the web is one of
them, it is necessary to jump into the cold water and start things.

Any maybe we should have some elected task forces for things like the
standard library.  Judging from the mailinglist it appears that far too
many people are discussing *every detail* of it.  It is a good idea to
ask as many people as possible, but I am not sure if the mailinglist is
the way to do that.  It is currently very hard to see the direction in
which development is heading.


Please think of this email just as a suggestion.  I don't have too much
trust into myself to follow the discussions on this list camely enough
to become a real part of a solution, but I would love to help shifting
the development into a better direction, no matter which one it will be.


Regards,
Armin

From g.brandl at gmx.net  Wed Sep 16 14:46:49 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 14:46:49 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <4AB0DA3E.6030400@active-4.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151029.n8FATHmq028625@theraft.openend.se>	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>	<1253037263.5678.12.camel@localhost>	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com>	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com>	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com>	<h8q69c$q8i$1@ger.gmane.org>	<4AB09FE6.6030308@egenix.com>	<h8q8qq$37t$1@ger.gmane.org>
	<4AB0DA3E.6030400@active-4.com>
Message-ID: <h8qmrl$fnk$1@ger.gmane.org>

Armin Ronacher schrieb:
> Hi,
> 
> Georg Brandl wrote:
>> That's why at PyCon (or was it Europython?) we thought about a "tags"
>> field for issues.  The main use for tags would be module names (and
>> it should be advertised as such in the UI for submitting bugs)
>> with auto-nosy associations for module maintainers.
> Arbitrary tags in a bug tracker are annoying.  People will not try to
> find exiting tags but make up their own, making it hard to query for it.

I disagree. Again: we wouldn't ask submitters to populate that list with
tags they make up, but the module name.  In the process as I envision it
now, further tags are added/changed by developers.

> I would rather see an automatically generated module list based on the
> hg tip / svn head.

That's not so easy to do.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From john at szakmeister.net  Wed Sep 16 15:16:21 2009
From: john at szakmeister.net (John Szakmeister)
Date: Wed, 16 Sep 2009 09:16:21 -0400
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0D7D6.7030204@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>
Message-ID: <a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>

On Wed, Sep 16, 2009 at 8:19 AM, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Hi everybody,
>
> I'm known for my dislike of the standard libray. ?In the past I wrote
> some blog posts about this topic into my personal blog already. ?However
> as many people pointed out earlier, a blog is not the place for this
> kind of criticism. ?Not only that, also just ranting about a topic does
> not help at all. ?Yesterday I subscribed to the stdlib-sig and
> immediately tons of mails ended up in my inbox. ?A quick look at the
> mail archives confirms what I was afraid of: this list is really high
> traffic. ?I tried to read up some of the discussions I missed but it's
> nearly impossible to do that.

I think that's out of the norm... the list has been quiet until recently. :-)

[snip]
> Any maybe we should have some elected task forces for things like the
> standard library. ?Judging from the mailinglist it appears that far too
> many people are discussing *every detail* of it. ?It is a good idea to
> ask as many people as possible, but I am not sure if the mailinglist is
> the way to do that. ?It is currently very hard to see the direction in
> which development is heading.
>
>
> Please think of this email just as a suggestion. ?I don't have too much
> trust into myself to follow the discussions on this list camely enough
> to become a real part of a solution, but I would love to help shifting
> the development into a better direction, no matter which one it will be.

Just a thought: PyCon 2010 is around the corner.  Open space?
Something more formalized?  Is that too late (because we will have
lost momentum)?

-John

From michael at voidspace.org.uk  Wed Sep 16 15:18:32 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 14:18:32 +0100
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
Message-ID: <4AB0E5A8.1010005@voidspace.org.uk>

John Szakmeister wrote:
> On Wed, Sep 16, 2009 at 8:19 AM, Armin Ronacher
> <armin.ronacher at active-4.com> wrote:
>   
>> Hi everybody,
>>
>> I'm known for my dislike of the standard libray.  In the past I wrote
>> some blog posts about this topic into my personal blog already.  However
>> as many people pointed out earlier, a blog is not the place for this
>> kind of criticism.  Not only that, also just ranting about a topic does
>> not help at all.  Yesterday I subscribed to the stdlib-sig and
>> immediately tons of mails ended up in my inbox.  A quick look at the
>> mail archives confirms what I was afraid of: this list is really high
>> traffic.  I tried to read up some of the discussions I missed but it's
>> nearly impossible to do that.
>>     
>
> I think that's out of the norm... the list has been quiet until recently. :-)
>
> [snip]
>   
>> Any maybe we should have some elected task forces for things like the
>> standard library.  Judging from the mailinglist it appears that far too
>> many people are discussing *every detail* of it.  It is a good idea to
>> ask as many people as possible, but I am not sure if the mailinglist is
>> the way to do that.  It is currently very hard to see the direction in
>> which development is heading.
>>
>>
>> Please think of this email just as a suggestion.  I don't have too much
>> trust into myself to follow the discussions on this list camely enough
>> to become a real part of a solution, but I would love to help shifting
>> the development into a better direction, no matter which one it will be.
>>     
>
> Just a thought: PyCon 2010 is around the corner.  Open space?
> Something more formalized?  Is that too late (because we will have
> lost momentum)?
>   

It looks like it will be something covered at the language summit, but 
an open space is a good idea. Backwards compatibility is a *big* problem 
for any major refactoring though.

Michael

> -John
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/


From solipsis at pitrou.net  Wed Sep 16 15:25:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 15:25:25 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0D7D6.7030204@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>
Message-ID: <1253107525.5829.35.camel@localhost>


Hello,

I'll just comment on some specific points:

> A quick look at the
> mail archives confirms what I was afraid of: this list is really high
> traffic.

Actually, it was very low traffic until those recent threads were
spawned.
I'm probably guilty of some of the traffic :-)

> It is true
> that Python currently has some issues with high concurrency and people
> try to fix that by forking and spawning new processes which certainly
> hides away the problem of shared state, but that does not solve it.  In
> fact, very recently Facebook open sourced the Tornado framework which
> does very well at high concurrency by using async IO.  Also this recent
> interest in Tornado will probably also motivate Twisted developers to
> improve their project's documentation and performance, because
> competition is often the what causes projects to improve.

First, I'm not sure what it has to do with the stdlib.
Second, if you look at the HTTP implementation in Tornado, it does not
handle 1/10th of the spec. Basically, it parses headers and handles a
couple of them (Content-Length, perhaps another one). It's not difficult
to write a fast HTTP server if you only need to support one smallish
part of the spec, and then to show impressive "Hello, World" benchmarks.
(besides, Tornado seems platform-specific since it explicitly uses
epoll)

The way Tornado was promoted looks like a marketing stunt. Glyph
Lefkowitz had a very reasonable answer to it on his blog.

(and, in any case, if you need speedy HTTP, just use mod_wsgi. There's
no need to try and look fancy by using a pure-Python async server, which
will always be much less tested, supported and documented than Apache
is; not to mention the wealth of plugins which are available to
customize Apache behaviour)

> The most obvious ones are certainly the `locale` module
> and all the other modules that change behavior based on the locale
> settings.  Did you know that every major Python framework reimplements
> time formatting even for something as simple as HTTP headers, because
> Python does not provide a way to format the time to english strings
> reliably?

Yes, it is very annoying.
Please note, however, that the locale module addresses a specific need,
which is to interface with the system-level locale mechanism. The global
state comes from this and is not caused by the design of the module
itself. While it does limit its uses a lot (and makes it fragile because
of system variations), it is still useful precisely when what you want
is to rely on the system's locale mechanism.

I don't know if including something like Babel in the stdlib would be a
good thing. It depends on the size of it, and the required maintenance
(I suppose there is a continuous flow of patches, as long as new
languages/cultures get supported?).
Making locale being able to delegate to Babel sounds awkward. Just tell
people to use Babel if they need to (whether it is in the stdlib, or
not).

> Also we have many modules in the standard library that in my opinion
> just do not belong there.  From my point of view, stuff like XML does
> not belong into the standard library.  But it appears that not many
> people agree with me on this one.

I would disagree indeed :)
Things like XML and JSON in the standard library are very useful,
because they provide a proven and reliable way to parse standardized
formats without having to install any third-party library.
Being able to do this kind of thing without installing additional stuff
is especially useful when writing small scripts.
(moreover, those libraries often have C accelerators, which might be
non-trivial to package properly or install manually on Windows
platforms)

> `dis` is
> one of them.  The implementation of dis prints to stdout no matter what
> you do.  Of course you can replace sys.stdout with something else for a
> brief moment, but again: this is not something we should aim for or
> advertise because it breaks for many people.

Sure, but `dis` is used mainly by the core developers themselves, for
testing and development purposes, and for these uses it is fine.
Besides, it is certainly possible to propose an extension of the API so
as to direct the output to another file-like object.

> Ubuntu recently started the "100 paper cuts" project.  There people work
> on tiny little patches to improve the system, rather to replace
> components.  Even though a large place of the standard library appears
> to be broken by design they could still be redesigned on the small
> scale, without breaking backwards compatibility.

This "call to arms" can be a good idea. But we have to be able to
channel it and appropriately review / validate the submitted changes.

> But how realistic is it to refactor the standard library?  I don't know.

It depends what you mean by "refactor". It doesn't sound very precise :)
I think it's better to discuss proposed changes case by case rather than
trying to reach a consensus on such vague terms.

> It is a good idea to
> ask as many people as possible, but I am not sure if the mailinglist is
> the way to do that.

If you have precise feature requests or bugs to report, the bug tracker
might indeed be a better place.
Especially if you have patches ready :-)

Regards

Antoine.



From michael at voidspace.org.uk  Wed Sep 16 15:25:40 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 14:25:40 +0100
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0D7D6.7030204@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>
Message-ID: <4AB0E754.50201@voidspace.org.uk>

Armin Ronacher wrote:
> Hi everybody,
>
> I'm known for my dislike of the standard libray.  In the past I wrote
> some blog posts about this topic into my personal blog already.  However
> as many people pointed out earlier, a blog is not the place for this
> kind of criticism.  Not only that, also just ranting about a topic does
> not help at all.  Yesterday I subscribed to the stdlib-sig and
> immediately tons of mails ended up in my inbox.  A quick look at the
> mail archives confirms what I was afraid of: this list is really high
> traffic.  I tried to read up some of the discussions I missed but it's
> nearly impossible to do that.
>
>   

And so you repeat a lot of points that have been discussed already in 
the last few days (the only few days in recent years that this list has 
been high traffic).

> [snip...]
>
> Also we have many modules in the standard library that in my opinion
> just do not belong there.  From my point of view, stuff like XML does
> not belong into the standard library.  But it appears that not many
> people agree with me on this one.  

You're right, a lot of people disagree with you. :-)

> But even if everybody would,
> backwards compatibility would still be a good reason to keep these
> modules around.
>
> Besides modules that do not work in every environment or modules that
> were probably a mistake to include, we also have modules in the standard
> library with a hideous implementation or no reusability, forcing people
> to reinvent what's already there.  For a long time, `urllib` was a
> module I would have listed there, but as of Python 2.6, the module
> largely improved by exposing the underlaying socket more which finally
> alllows us to set the timeout in a reliable way.  But there are still a
> ton of modules in the library that cause troubles for people.  `dis` is
> one of them.  The implementation of dis prints to stdout no matter what
> you do.  Of course you can replace sys.stdout with something else for a
> brief moment, but again: this is not something we should aim for or
> advertise because it breaks for many people.  

That particular problem sounds easy to fix.

> `Cookie` is a module
> people monkey patched for a while (badly) to support the http only flag.
>  Not only does the code expose a weird API, it is also nearly impossible
> to extend and even ships cookie subclasses that use unsigned pickles and
> trust the client.  `cgi` has again, shared state on the global namespace
> that alters the behavior of the lirbary.  Of course it was never
> intended to be used by anything but `cgi`, but that leaves people
> reimplementing it or abusing it.
>
>   

cgi and Cookie would both be *excellent* targets for refactoring / 
improving. This is *hugely* preferable to complaining about them. ;-)

I *hope* that Python-dev would be willing to accept some measure of 
backwards incompatibility in the name of improving what few people will 
disagree are potentially useful but horribly outdated APIs.

> So when the discussion started replacing `optparse` with `argparse`,
> because the former is unmaintained I became alerted.  My wishes have
> always been the standard library to be a reliable fallback to be used if
> everything else fails.  Something I can rely on which will not change,
> except for maybe some additions or modules moved to different locations.
> As Python developers we became used to moving import locations a lot.
> It it's `cPickle` or any of the element tree implementations, you name it.
>   

For many things that is an admirable goal. But as you point out, and we 
have already discussed at great length, there is a problem with what to 
do with modules that can't be evolved to meet new requirements because 
of original API design.


> I wonder if the solution to this problem wouldn't be a largely improved
> packaging system and some sort of standardized reviewing process for the
> standard library.  Currently there is not even an accepted style for
> modules ending up in the Python distribution. 

Yes there is - the standard procedure for getting new mdoules into the 
standard library is via the PEP process.

>  That, and a group of
> people, dedicated to standard library refactoring.  The majority of
> libraries in the standard library are small and easy to understand, I'm
> sure they are perfectly suited for students on projects like GSOC or
> GHOP to work on.  They could even be used as some sort of "playground"
> for new Python developers.
>   

It would be a great project for GHOP *if* we have some experienced 
developers, like yourself, dedicated to working out what the things that 
need fixing are.

I think that one of the best ways to achieve the changes you discuss 
below may well be 'one-step-at-a-time' rather than a huge project.

Michael

> Ubuntu recently started the "100 paper cuts" project.  There people work
> on tiny little patches to improve the system, rather to replace
> components.  Even though a large place of the standard library appears
> to be broken by design they could still be redesigned on the small
> scale, without breaking backwards compatibility.
>
> Of course libraries like `locale` and `logging` are hard to change, but
> it would still be possible.  For `locale` it would probably a useful
> idea to go into the direction of datetime, where the timezone
> information is left to a 3rd party library.  `locale` could provide some
> hooks for libraries like `babel` to fill the gap.  On the other hand
> `Cookie` would be very easy to fix by moving the parsing code into a
> separate function and refactoring the cookie objects.
>
> We could probably also start a poll out there with well-selected
> questions of what users think about parts of the library.  And for that
> poll it would make a lot of sense to not just ask the questions and
> evaluating the results, but also track the area the user is coming from
> (small size company, open / closed source, web development etc.).
> Because we all are biased and seeing results grouped by some of these
> factoids could be enlightening.  That said, it could tell us that I'm
> completely wrong with my ideas of how the state of the standard library.
>
> But how realistic is it to refactor the standard library?  I don't know.
> For a long time people were pretty sure Python will not get any faster
> and yet Unleaden Swallow is doing some really amazing progress.
>
> If we want to push Python foward into new areas, and the web is one of
> them, it is necessary to jump into the cold water and start things.
>
> Any maybe we should have some elected task forces for things like the
> standard library.  Judging from the mailinglist it appears that far too
> many people are discussing *every detail* of it.  It is a good idea to
> ask as many people as possible, but I am not sure if the mailinglist is
> the way to do that.  It is currently very hard to see the direction in
> which development is heading.
>
>
> Please think of this email just as a suggestion.  I don't have too much
> trust into myself to follow the discussions on this list camely enough
> to become a real part of a solution, but I would love to help shifting
> the development into a better direction, no matter which one it will be.
>
>
> Regards,
> Armin
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/


From jnoller at gmail.com  Wed Sep 16 15:27:23 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 16 Sep 2009 09:27:23 -0400
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0D7D6.7030204@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>
Message-ID: <4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>

[snip]
> with shared state on module level, web applications are not. ?It is true
> that Python currently has some issues with high concurrency and people
> try to fix that by forking and spawning new processes which certainly
> hides away the problem of shared state, but that does not solve it.

FWIW: Multiprocessing doesn't care about shared state; nor is it an
attempt to "get around" the shared state within the standard library.
Conflating concurrency issues with shared state within standard
library modules is not quite right. I do agree, however, that there
are some modules whose shared state is undesirable.

You do have to understand though; while a large portion of the world
is moving into the web; there are many of us still, who simply don't
do "the online thing" - we should strive to improve the web-story, but
we can not do so in a way which cripples or makes the lives of people
who are *not* web-heads more difficult.

> Now if we look at the standard library, we can see many modules that
> just do not work in such environments because they have some sort of
> shared state. ?The most obvious ones are certainly the `locale` module
> and all the other modules that change behavior based on the locale
> settings. ?Did you know that every major Python framework reimplements
> time formatting even for something as simple as HTTP headers, because
> Python does not provide a way to format the time to english strings
> reliably? ?But there are certainly more modules that have this sort of
> problem.

Part of my motivation in starting the other thread are issues such as this.

> Also we have many modules in the standard library that in my opinion
> just do not belong there. ?From my point of view, stuff like XML does
> not belong into the standard library. ?But it appears that not many
> people agree with me on this one. ?But even if everybody would,
> backwards compatibility would still be a good reason to keep these
> modules around.

Each of us comes from a different problem domain - You might be
focused on the web, but I'm focused on daemons, tools, and networking
and glue. This difference between us exemplifies the problem of a
common, objective "smell test" for what really belongs in the standard
library. Take your example - XML parsing. I would prefer One Way To Do
It in the standard library. I feel XML parsing (and JSON, and YAML)
are critical things to have in the standard library for a variety of
reasons.

> Besides modules that do not work in every environment or modules that
> were probably a mistake to include, we also have modules in the standard
> library with a hideous implementation or no reusability, forcing people
> to reinvent what's already there.
[snip]

And SimpleHTTPServer, and logging, and... Armin, some of us agree with
you, and again, this was part of my driving force in starting the
other thread proposing the logical break out and subsequent cleanup.
Fred, I and Brett have gone off to write PEPs outlining these tasks.
If you would like to contribute to those peps, email me off list and I
will give you access. But you have to be nice ;)

> I wonder if the solution to this problem wouldn't be a largely improved
> packaging system and some sort of standardized reviewing process for the
> standard library. ?Currently there is not even an accepted style for
> modules ending up in the Python distribution. ?That, and a group of
> people, dedicated to standard library refactoring. ?The majority of
> libraries in the standard library are small and easy to understand, I'm
> sure they are perfectly suited for students on projects like GSOC or
> GHOP to work on. ?They could even be used as some sort of "playground"
> for new Python developers.

This was another point in the other thread; we need maintainers for
all of the modules. While there is not "guideline" for the code which
goes in per-se, the process by which something gets in is outlined,
and the code is typically reviewed prior to inclusion by Python-Dev.

As for the packaging system: Tarek and Company are working on this,
and it is outside of the boundaries of the discussions on this list so
far. If you really want to help with packaging, you need to go over to
disutils-sig (and report back to us the traffic levels there ;)) or
contact Tarek directly.

> Ubuntu recently started the "100 paper cuts" project. ?There people work
> on tiny little patches to improve the system, rather to replace
> components. ?Even though a large place of the standard library appears
> to be broken by design they could still be redesigned on the small
> scale, without breaking backwards compatibility.

We have over 170 patches in the tracker needing reviews. We have more
issues with patches that need docs and tests. More patches, while
welcome, still need someone to review them, apply them, and ensure
that they don't side-effect everything else, conceptually break
everything, and so on.

> Of course libraries like `locale` and `logging` are hard to change, but
> it would still be possible. ?For `locale` it would probably a useful
> idea to go into the direction of datetime, where the timezone
> information is left to a 3rd party library. ?`locale` could provide some
> hooks for libraries like `babel` to fill the gap. ?On the other hand
> `Cookie` would be very easy to fix by moving the parsing code into a
> separate function and refactoring the cookie objects.

And a 3rd party library adds a dependency to all the build bots,
consumers, apps, etc out there. That dependency may not work on
windows, OS/X, or IRIX. This is partially the reason something like an
libxml dependency is right on out (sadly).

Again, agreed - but these modules need maintainers, people who care
enough about them to do the things you talk about. That's why I
started this tempest in a mailpot in the first place. It's not like I
enjoy replying to emails - I don't even get paid for it.

> We could probably also start a poll out there with well-selected
> questions of what users think about parts of the library. ?And for that
> poll it would make a lot of sense to not just ask the questions and
> evaluating the results, but also track the area the user is coming from
> (small size company, open / closed source, web development etc.).
> Because we all are biased and seeing results grouped by some of these
> factoids could be enlightening. ?That said, it could tell us that I'm
> completely wrong with my ideas of how the state of the standard library.

There are two things conflated here. One is "what do the users want"
and "what can we maintain". They are not the same thing. Brett already
tried an informal poll:

http://sayspy.blogspot.com/2009/07/results-of-informal-poll-about-standard.html

While not entirely representative of the hundreds of companies, and
thousands of people out there using Python, it's a good place to
start. In fact, it's one of the data points I'm using in my "cleanup
PEP". Would you like to help?

> But how realistic is it to refactor the standard library? ?I don't know.
> For a long time people were pretty sure Python will not get any faster
> and yet Unleaden Swallow is doing some really amazing progress.

refactoring of the standard library, and it's continued evolution are
requirements for Python 's survival. This is why I started the other
thread, and others contributed to it.

> Any maybe we should have some elected task forces for things like the
> standard library. ?Judging from the mailinglist it appears that far too
> many people are discussing *every detail* of it. ?It is a good idea to
> ask as many people as possible, but I am not sure if the mailinglist is
> the way to do that. ?It is currently very hard to see the direction in
> which development is heading.

Those of us who care about this are off writing PEPs. If you want to
help, you can. The discussion of every detail is a necessary "evil" -
and it comes with the territory. There is a time for discussion
though, and a time for work. David, Georg, Brett, Frank, and I are all
taking action items to go off and do, because you're right: actions
speak louder.

>
> Please think of this email just as a suggestion. ?I don't have too much
> trust into myself to follow the discussions on this list camely enough
> to become a real part of a solution, but I would love to help shifting
> the development into a better direction, no matter which one it will be.

If you can not follow this mailing list calmly to find the good
information, filter the fluff, and ultimately cherry pick and extract
the work necessary to move forward, you're going to dread the PEP
process.

Changes affect everyone, we can not go and do them in a smokey dimly
lit room. It runs counter to who and what we are. It's fine to be a
dictator when it's your own project (Jinja vs. Jinja2 come to mind)
but discussion is needed, and healthy. You just need to filter the
good from the bad.

Armin, I agree with your sentiment, the feeling that is contained
within it is the motivation for me starting the original discussion in
the *first place*. Yes, it caused a fair amount of discussion, some
good, some bad, some circular. But we also got some people working on
solid deliverables, which was the point.

If you would like to help write some PEPs, I'm open to collaborating.

Jesse

From jnoller at gmail.com  Wed Sep 16 15:28:35 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 16 Sep 2009 09:28:35 -0400
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0E5A8.1010005@voidspace.org.uk>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
Message-ID: <4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>

On Wed, Sep 16, 2009 at 9:18 AM, Michael Foord <michael at voidspace.org.uk> wrote:
>
> It looks like it will be something covered at the language summit, but an
> open space is a good idea. Backwards compatibility is a *big* problem for
> any major refactoring though.
>
> Michael
>

Yup, language summit. I'm hoping to cover some amount of this in a
pending talk proposal I have in the pycon system too.

From ziade.tarek at gmail.com  Wed Sep 16 15:32:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 16 Sep 2009 15:32:47 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0D7D6.7030204@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>
Message-ID: <94bdd2610909160632x29fff91cp6e4ea63291a90927@mail.gmail.com>

On Wed, Sep 16, 2009 at 2:19 PM, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> I wonder if the solution to this problem wouldn't be a largely improved
> packaging system and some sort of standardized reviewing process for the
> standard library.

FYI

The work we've started in PEP 376 and PEP 386 (and some elements not
formalized in PEPs) is trying
to improve the situation.

Distutils provides one part of what would be a full packaging system,
and this work tries to fill
the gap:

- uninstall feature
- version comparison
- extension of metadata (to include dependencies)

The plan is to finish this work and provide a basis to build a package manager.
(which could be used by the stdlib as well if the packages and modules
in there were at the same level
than third party packages)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????!

From armin.ronacher at active-4.com  Wed Sep 16 15:40:04 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 16 Sep 2009 15:40:04 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <1253107525.5829.35.camel@localhost>
References: <4AB0D7D6.7030204@active-4.com>
	<1253107525.5829.35.camel@localhost>
Message-ID: <4AB0EAB4.3090105@active-4.com>

Hi,

Antoine Pitrou wrote:
> First, I'm not sure what it has to do with the stdlib.
Preamble.

> I don't know if including something like Babel in the stdlib would be a
> good thing. It depends on the size of it, and the required maintenance
> (I suppose there is a continuous flow of patches, as long as new
> languages/cultures get supported?).
I'm not saying babel should go into the stdlib, it's just not a good
idea.  Maybe unicodedata could expose more information but that's where
it ends.

> Making locale being able to delegate to Babel sounds awkward. Just tell
> people to use Babel if they need to (whether it is in the stdlib, or
> not).
Then at least some time and friends should have a flag to *ignore* the
locale if that's somehow possible.

> I would disagree indeed :)
Yes, I already gave up.

> Sure, but `dis` is used mainly by the core developers themselves, for
> testing and development purposes, and for these uses it is fine.
> Besides, it is certainly possible to propose an extension of the API so
> as to direct the output to another file-like object.
But that's something that can be changed as a paper-cut project, it's
not hard.  Just nobody really has the urge and time to do it.

> This "call to arms" can be a good idea. But we have to be able to
> channel it and appropriately review / validate the submitted changes.
Of course.  But code review should happen in general, not just for
external contributions.

> It depends what you mean by "refactor". It doesn't sound very precise :)
> I think it's better to discuss proposed changes case by case rather than
> trying to reach a consensus on such vague terms.
That would have to be decided on a papercut-by-papercut base.  And
someone would have to select this modules first, which is why I
mentioned the poll.

Regards,
Armin

From armin.ronacher at active-4.com  Wed Sep 16 15:42:14 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 16 Sep 2009 15:42:14 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <94bdd2610909160632x29fff91cp6e4ea63291a90927@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<94bdd2610909160632x29fff91cp6e4ea63291a90927@mail.gmail.com>
Message-ID: <4AB0EB36.2010107@active-4.com>

Hi,

Tarek Ziad? wrote:
> The work we've started in PEP 376 and PEP 386 (and some elements not
> formalized in PEPs) is trying
> to improve the situation.
And I'm incredible thankful for that.  It's heading into the right
direction.

Regards,
Armin

From ctb at msu.edu  Wed Sep 16 16:43:15 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Sep 2009 07:43:15 -0700
Subject: [stdlib-sig] Backwards compat (was: Evolving the Standard
	Library)
In-Reply-To: <4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
	<4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>
Message-ID: <20090916144315.GH19160@idyll.org>

On Wed, Sep 16, 2009 at 09:28:35AM -0400, Jesse Noller wrote:
> On Wed, Sep 16, 2009 at 9:18 AM, Michael Foord <michael at voidspace.org.uk> wrote:
> >
> > It looks like it will be something covered at the language summit, but an
> > open space is a good idea. Backwards compatibility is a *big* problem for
> > any major refactoring though.
> >
> > Michael
> >
> 
> Yup, language summit. I'm hoping to cover some amount of this in a
> pending talk proposal I have in the pycon system too.

One interesting thought for backwards compatibility -- why not take all
of the PyPI packages and try importing and/or testing them across
versions, and then trying to build automatic classifiers to highlight
the "interesting" breakages?  A first pass filter would be "breakages we
know about" vs "breakages we don't."

Then you could build these breakages into a compatibility diagnostics
package.

Sounds like a fun PyCon sprint to me...

--titus
-- 
C. Titus Brown, ctb at msu.edu

From solipsis at pitrou.net  Wed Sep 16 16:47:02 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 16:47:02 +0200
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<mal@egenix.com> <4AAEC59C.7060605@egenix.com>
	<200909151029.n8FATHmq028625@theraft.openend.se>
	<43aa6ff70909150738q44044b5eo795c58850ec5b36c@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
Message-ID: <1253112422.5829.50.camel@localhost>

Le mardi 15 septembre 2009 ? 19:42 -0700, Raymond Hettinger a ?crit : 
> > That would be a third source of info about who maintains what.
> 
> Is this just a formalization of what we already do now?

I cannot speak for David, but IMHO it should be a bit more than that.

The underlying idea is to promote a broader (and therefore less
exclusive) view of maintenership. We should extend the meaning of
"maintainer" (or "expert" as also stated) to people who are 1) competent
enough to give useful advice on (bug/feature) requests concerning a
module and 2) at least moderately willing to do so; this rather than the
supposed "owner" of a module, which is a notion we should discourage.

Therefore, we can promote collective appropriation of code, to avoid the
kind of deadlocks / gradual obsolescence we sometimes have with some
specific pieces of code for which nobody wants to take responsibility
without receiving an imprimatur from the "owner".

But of course MISC/maintainers.txt is /already/ useful as just a
centralized piece of information.

Regards

Antoine.



From ctb at msu.edu  Wed Sep 16 16:49:08 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Sep 2009 07:49:08 -0700
Subject: [stdlib-sig] GHOP / py3k (was: Evolving the Standard Library)
In-Reply-To: <4AB0E754.50201@voidspace.org.uk>
References: <4AB0D7D6.7030204@active-4.com> <4AB0E754.50201@voidspace.org.uk>
Message-ID: <20090916144908.GI19160@idyll.org>

>Michael said:
> Armin said:
> > That, and a group of
> >people, dedicated to standard library refactoring.  The majority of
> >libraries in the standard library are small and easy to understand, I'm
> >sure they are perfectly suited for students on projects like GSOC or
> >GHOP to work on.  They could even be used as some sort of "playground"
> >for new Python developers.
> 
> It would be a great project for GHOP *if* we have some experienced 
> developers, like yourself, dedicated to working out what the things that 
> need fixing are.

Testing and doc updates worked reasonably well within GHOP last time,
and surprisingly little in the way of "experienced developers" were
needed.  Faced with the responsibility of coming up with dozens of tasks
on short notice, I picked a dozen stdlib modules and said

  test this
  integrate doug hellmann's documentation
  run through the existing examples and write more

...and voila, it happened and attracted positive notice from the BDFL,
which is saying something.

By far the most important part of that process was not my role in
putting the tasks up, but Georg's role in reviewing the patches and
committing them in a timely manner.  I can't speak for how much time he
spent doing that, however, and I certainly don't expect that level of
effort from HIM this time; perhaps with Mercurial we can get
non-committers to act as first-pass filters to reduce the strain on
Georg or whoever steps into his shoes. [0]

Perhaps this time we can focus on py3k stuff with GHOP; that'd be great,
and a real community boost, IMO.

cheers,
--titus

[0] I'm nominating Brett; he seems to have plenty of time.

From dickinsm at gmail.com  Wed Sep 16 17:05:23 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 16 Sep 2009 16:05:23 +0100
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <1253112422.5829.50.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>
	<bbaeab100909151019x1a7cbdebv1a7866a553077332@mail.gmail.com>
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com>
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net>
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com>
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net>
	<0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
	<1253112422.5829.50.camel@localhost>
Message-ID: <5c6f2a5d0909160805p36e91dc2yc9e70e286919419a@mail.gmail.com>

On Wed, Sep 16, 2009 at 3:47 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 ? 19:42 -0700, Raymond Hettinger a ?crit :
>> > That would be a third source of info about who maintains what.
>>
>> Is this just a formalization of what we already do now?
>
> I cannot speak for David, but IMHO it should be a bit more than that.
>
> The underlying idea is to promote a broader (and therefore less
> exclusive) view of maintenership. We should extend the meaning of
> "maintainer" (or "expert" as also stated) to people who are 1) competent
> enough to give useful advice on (bug/feature) requests concerning a
> module and 2) at least moderately willing to do so; this rather than the
> supposed "owner" of a module, which is a notion we should discourage.

+1 for this interpretation of maintainer.

And a big +1 for Misc/maintainers.txt, or equivalent information elsewhere
(a wiki page, perhaps).  I've frequently found myself wanting this
information when commenting on an issue (most recently for the
curses module:  the source says that AMK is the current maintainer,
but I don't know whether that information is still in date, and if not,
who else might be familiar enough to fix curses issues).  I've not been
around long enough to know the history of the various bits and pieces
of Python.

Mark

From orestis at orestis.gr  Wed Sep 16 17:35:36 2009
From: orestis at orestis.gr (Orestis Markou)
Date: Wed, 16 Sep 2009 18:35:36 +0300
Subject: [stdlib-sig] Python-the-platform vs Python-the-framework
Message-ID: <FECFFCAB-A154-46BB-8A30-2F78B3098BC5@orestis.gr>

The recent discussions on breaking the standard library made something  
click for me. I wrote a long blog post about it, which I am pasting  
here. The original is at http://orestis.gr/blog/2009/09/16/backwards-compatibility-straw-man/

Hopefully it will help the discussion go forward, and perhaps help the  
people who are writing PEPs.

TL;DR was TL;DR: By acknowledging the different ways Python is used,  
we will be at a better position to evolve it.

TL;DR version: Python can be separated into two parts: Python-the- 
platform and Python-the-framework. The former is mainly the language  
and some core services, the latter is the "batteries". Some people  
code against frameworks that bypass the batteries, some code against  
the batteries themselves. While it's important that the batteries are  
there and relatively stable, since they are mostly domain-specific,  
they are superseded by newer, better and more powerful ones every once  
in a while. Python-core are experts on the language, they are not on  
the various domains. We should let domain experts develop solutions  
for their domains outside the stdlib, and not confuse users by keeping  
outdated and cranky modules in the stdlib.


Original:

== Backwards compatibility is a straw man==

I recently signed up to stdlib-sig so I could just nod in agreement to  
the people that suggested that the stdlib needs to evolve. In the  
discussions that ensued, the backwards compatibility argument came up  
often. I think it's not a valid argument for the specific discussion,  
though. Here are my thoughts.

## What is stdlib?

The standard library is a set of packages and modules that get shipped  
by default with the Python interpreter.  Despite what I thought when I  
was starting with Python, it doesn't represent the so-called "best- 
practices" modules. There is no overarching design that dictates why  
things are the way they are. Instead, it just represents a set of  
modules that have been picked up at some point in time. Some of them  
are close to the language (such as collections, itertools and others)  
and some are domain-specific tools. The difference is significant.

Here is an attempt to divide the ones visible at [the docs](http://docs.python.org/library/ 
). Of course, I'm choosing here based on personal preference, YMMV.  
Here we go:

### Language

String Services:

 > `string, re, struct, StringIO, cStringIO, codecs, unicodedata`

Data types:

 > `datetime, collections, heapq, bisect, array, sets, sched, mutex,  
queue, weakref, UserDict, UserList, UserString, types, new, copy,  
pprint, repr`

Numeric and Mathematical Modules:

 > `numbers, math, cmath, decimal, fractions, random, itertools,  
functools, operator`

(why itertools, functools and operator are here is beyond me)

File and Directory Access:

 > `os.path, stat, statvfs, filecmp, tempfile, glob, fnmatch, shutil`

Data Persistence:

 > `pickle, cPickle, copy_reg, shelve, marshal`

Data Compression and Archiving:

 > `zlib, gzip, bz2, zipfile, tarfile`

* this category can argued to be both domain-speficic and close to  
language.

Cryptographic Services:

 > `hashlib, hmac, md5, sha`

Generic Operating System Services:

 > `os, io, time, getpass, platform, errno, ctypes`

Optional Operating System Services:

 > `select, threading, thread, dummy_threading, dummy_thread,  
multiprocessing, mmap`

Interprocess Communication and Networking:

 > `subprocess, socket, ssl, signal, popen2, asyncore, asynchat`

  * asyncore and asynchat can be said to be domain specific, but async  
io is fundamental IMO

Internet Data Handling:

 > `base64, binhex, binascii, uu`

  * these should probably live alongside codecs

Internet Protocols and Support:

 > `wsgiref, uuid`

  * wsgi is meant as an interop protocol, so I put it close to the  
language.

Internationalization:

 > `gettext, locale`

Development Tools:

 > `pydoc, doctest, unittest, 2to2, test`

Debugging and Profiling

 > `bdb, pdb, hotshot, timeit, trace`

Python Runtime Services

 > `sys, __builtin__, future_builtins, __main__, warnings, contextlib,  
abc, atexit, traceback, __future__, gc, inspect, site, user, fpectl`

Importing Modules:

 > `imp, imputil, zipimport, pkgutil, modulefinder, runpy`

Python Language Services:

 > `parser, ast, symtable, symbol, token, keyword, tokenize, tabnanny,  
py_compile, compileall, dis, pickletools, distutils, pyclbr, compiler`

  * a lot of these arguably are domain-speficic, but given the domain  
is Python...
  * the compiler package is included here as well




### Domain specific

String Services:

 > `difflib, textwrap, stringprep, fpformat`

Data types:

 > `calendar`

File and Directory Access:

 > `fileinput, linecache, dircache, macpath`

Data Persistence:

 > `anydbm, whichdbm, dbm, gdbm, dbhash, bsddb, dubmdbm, sqlite3`

File Formats:

 > `csv, ConfigParser, robotparser, nterc, xdrlib, plistlib`

Generic Operating System Services:

 > `optparse, getopt, logging, curses, curses.*`

Optional Operating System Services:

 > `readline, rlcompleter`

Internet Data Handling:

 > `email, json, mailcap, mailbox, mhlib, mimetools, mimetypes,  
MimeWriter, mimify, multifile, rfc822, quopri`

Structured Markup Processing Tools:

 > `HTMLParser, sgmllib, htmllib, htmlentitydefs, xml.*`

Internet Protocols and Support:

 > `webbrowser, cgi, cgitb, urllib, urllib2, httplib, ftplib, poplib,  
imaplib, nntplib, smtplib, smtpd, telnetlib, urlparse, SocketServer,  
BaseHTTPServer, SimpleHTTPServer, CGIHTTPServer, cookielib, Cookie,  
xmlrpclib, SimpleXMLRPCServer, DocXMLRPCServer`

Multimedia Services:

 > `audioop, imageop, aifc, sunau, wave, chunk, colorsys, imghdr,  
sndhdr, ossaudiodev`

Program Frameworks

 > `cmd, shlex`

GUI with Tk:

 > `Tkinter, Tix, ScrolledText, turtle, IDLE, Others`

Custom Python Interpreters:

 > `code, codeop`

Restricted Execution:

 > `rexec, Bastion`

  * Both have been removed from Python 3.0

Miscellaneous Services:

 > `formatter`

MS Windows Specific Services:

 > `msilib, msvcrt, _winreg, winsound`

Unix Specific Services:

 > `posix, pwd, spwd, grp, crypt, dl, termios, tty, pty, fcntl, pipes,  
posixfile, resource, nis, syslog, commands`

Mac OS X specific services:

 > `ic, MacOS, macostools, findertools, EasyDialogs, Framework,  
autoGIL, ColorPicker`

MacPython OSA Modules:

 > `gensuitemodule, aetools, aepack, aetypes, MiniAEFrame`

SGI IRIX Specific Services:

 > `al, AL, cd, dl, DL, flp, fm, gl, DEVICE, GL, imgfile, jpeg`

SunOS Specific Services:

 > `sunaudiodev, SUNAUDIODEV`

### GRAND TOTAL

130 language-related

151 domain-specific

Damn isn't that a lot of packages. For reference, PyPI currently hosts  
~7500 of them. Truly, Python has a lot of batteries.

## A platform, or a framework?

I can easily see a neat split there - the first half is Python, the  
platform. The second half are the batteries. However nowadays the  
batteries are not enough. While you may be able to write a quick and  
dirty script with them, if you're doing web stuff you're probably  
using another framework, if you're doing desktop stuff you're probably  
using another toolkit as well. Of course, there are other uses I  
probably don't know nothing about, and for them Python *becomes* the  
framework.

I know that many frameworks built on top of Python-the-platform start  
with the batteries, and then they start writing their own  
implementations to fix bugs or add features. Django-the-framework runs  
on Python-the-platform 2.3-2.6 so it can't rely on features being  
present or bugs fixed in the batteries - it has its own.

## Backwards compatible

My issue with the backwards compatibility argument is this: No one  
forces anyone to update to any version of Python. Developers make a  
conscious decision - to develop software for a specific (or a range  
of) version of Python, and specific versions for all the other  
libraries they depend on. *Any* change to the dependencies of a piece  
of software may lead to breakage. I see no reason why Python should be  
different for that purpose.

I can't see backwards compatibility as an argument against upgrading  
Python, adding features, deprecating and removing modules, and of  
course fixing bugs. (Aside: Microsoft is so backwards compatible so as  
to emulate bugs if important programs need it. We don't want to do  
that!). Instead, I see backwards compatibility as an argument *for*  
better isolation of Python-the-framework. If a program needs specific  
versions of Python and libraries, it should be trivial to guard them  
against change. If an operating system depends on a specific version  
of Python, it should hide it away and not allow modifications.

On the other hand, I would argue against radical changes to Python-the- 
platform. Of course, this has been the case so far, with one exception  
in Python 3.0 to fix issues that needed to be fixed. In fact, there's  
a nice forwards-compatible feature for changes to the platform -  
`__future__`. People have been upgrading to new Python features with  
minor complaints, so I don't see why changing the batteries part of  
stdlib is tickling people so much.

## Best of breed

When I started with Python, I only used modules from stdlib - I had no  
idea about PyPI, and I assumed that things from python-core would be  
more high-quality. However, this is only true for the language  
modules, not the domain specific modules. The reason is simple -  
python-core are experts on Python and language design, but not experts  
on the numerous domains the batteries cover. There are now  
replacements for most, if not all (os-specific stuff probably  
excluded) domain-specific modules. People trying to get a GUI running  
with Tk and not knowing about wx, Qt, Gtk, or the platform-specific  
choices is bad. People trying to do image manipulation and not knowing  
about PIL is bad.

I would argue that domain-specific parts should be spun off the stdlib  
and be released as separate PyPI modules. We can keep Python-the- 
framework going by having a download with the kitchen sink provided  
(as [Jesse Noller proposed](http://mail.python.org/pipermail/stdlib-sig/2009-September/000398.html 
)), and cooperate with packagers/distributions so that they can  
fortify their installations against change.

## Conclusion

The argument on stdlib-sig is huge, and thankfully it seems that  
something is getting done in the end. I expect a some people to agree  
with me, and some to disagree. Writing my thoughts makes me think, so  
please keep in mind that I am willing to be persuaded otherwise, with  
the correct arguments.

As far as my day to day use is concerned, 99% of the batteries could  
disappear from my site-packages, and I would not care. Of course,  
packages I actually use and import (twisted, pyobjc) _would_ care  
(actually, both of those will most likely use their own batteries). I  
wonder for how many people is this situation familiar. Find your  
imports, and see what the results are.






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090916/69125133/attachment-0001.htm>

From ziade.tarek at gmail.com  Wed Sep 16 17:43:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 16 Sep 2009 17:43:18 +0200
Subject: [stdlib-sig] Backwards compat (was: Evolving the Standard
	Library)
In-Reply-To: <20090916144315.GH19160@idyll.org>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
	<4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>
	<20090916144315.GH19160@idyll.org>
Message-ID: <94bdd2610909160843l240eb250ia755ef2f48d7dc5e@mail.gmail.com>

On Wed, Sep 16, 2009 at 4:43 PM, C. Titus Brown <ctb at msu.edu> wrote:
> On Wed, Sep 16, 2009 at 09:28:35AM -0400, Jesse Noller wrote:
>> On Wed, Sep 16, 2009 at 9:18 AM, Michael Foord <michael at voidspace.org.uk> wrote:
>> >
>> > It looks like it will be something covered at the language summit, but an
>> > open space is a good idea. Backwards compatibility is a *big* problem for
>> > any major refactoring though.
>> >
>> > Michael
>> >
>>
>> Yup, language summit. I'm hoping to cover some amount of this in a
>> pending talk proposal I have in the pycon system too.
>
> One interesting thought for backwards compatibility -- why not take all
> of the PyPI packages and try importing and/or testing them across
> versions, and then trying to build automatic classifiers to highlight
> the "interesting" breakages? ?A first pass filter would be "breakages we
> know about" vs "breakages we don't."
>
> Then you could build these breakages into a compatibility diagnostics
> package.
>
> Sounds like a fun PyCon sprint to me...

I don't know if you remember my message on the snakebite mailing list
some times ago on a related topic.

That's the same process I would like to do to test distutils over
PyPI, by grabbing
packages there and running some commands using their setup.py.
and say "this package is Distutils certified !"

But this requires some work to make sure there are no security
problems I/O-wise, unless you work with a list of trusted packages (which is
not what we would want if we want to do QA tests)

And the environment has to be reseted after each run to make sure
there are no problems created by the package.

Quite a work, but I am in for some brainstroming at Pycon on this topic
if you are interested :)

From ctb at msu.edu  Wed Sep 16 17:54:14 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Sep 2009 08:54:14 -0700
Subject: [stdlib-sig] Backwards compat (was: Evolving the Standard
	Library)
In-Reply-To: <94bdd2610909160843l240eb250ia755ef2f48d7dc5e@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
	<4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>
	<20090916144315.GH19160@idyll.org>
	<94bdd2610909160843l240eb250ia755ef2f48d7dc5e@mail.gmail.com>
Message-ID: <20090916155414.GC2523@idyll.org>

On Wed, Sep 16, 2009 at 05:43:18PM +0200, Tarek Ziad? wrote:
> On Wed, Sep 16, 2009 at 4:43 PM, C. Titus Brown <ctb at msu.edu> wrote:
> > On Wed, Sep 16, 2009 at 09:28:35AM -0400, Jesse Noller wrote:
> >> On Wed, Sep 16, 2009 at 9:18 AM, Michael Foord <michael at voidspace.org.uk> wrote:
> >> >
> >> > It looks like it will be something covered at the language summit, but an
> >> > open space is a good idea. Backwards compatibility is a *big* problem for
> >> > any major refactoring though.
> >> >
> >> > Michael
> >> >
> >>
> >> Yup, language summit. I'm hoping to cover some amount of this in a
> >> pending talk proposal I have in the pycon system too.
> >
> > One interesting thought for backwards compatibility -- why not take all
> > of the PyPI packages and try importing and/or testing them across
> > versions, and then trying to build automatic classifiers to highlight
> > the "interesting" breakages? ?A first pass filter would be "breakages we
> > know about" vs "breakages we don't."
> >
> > Then you could build these breakages into a compatibility diagnostics
> > package.
> >
> > Sounds like a fun PyCon sprint to me...
> 
> I don't know if you remember my message on the snakebite mailing list
> some times ago on a related topic.
> 
> That's the same process I would like to do to test distutils over
> PyPI, by grabbing
> packages there and running some commands using their setup.py.
> and say "this package is Distutils certified !"
> 
> But this requires some work to make sure there are no security
> problems I/O-wise, unless you work with a list of trusted packages (which is
> not what we would want if we want to do QA tests)
> 
> And the environment has to be reseted after each run to make sure
> there are no problems created by the package.
> 
> Quite a work, but I am in for some brainstroming at Pycon on this topic
> if you are interested :)

yep, absolutely!  I think I've got the execution and reporting end
handled; now we just need to get some virtual environments running so we
can do untrusted packages.

Hmm, might be worth an AWS account just to do the basic stuff.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From ziade.tarek at gmail.com  Wed Sep 16 18:11:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 16 Sep 2009 18:11:22 +0200
Subject: [stdlib-sig] Backwards compat (was: Evolving the Standard
	Library)
In-Reply-To: <20090916155414.GC2523@idyll.org>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
	<4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>
	<20090916144315.GH19160@idyll.org>
	<94bdd2610909160843l240eb250ia755ef2f48d7dc5e@mail.gmail.com>
	<20090916155414.GC2523@idyll.org>
Message-ID: <94bdd2610909160911o1682f519p561d503985f282cc@mail.gmail.com>

On Wed, Sep 16, 2009 at 5:54 PM, C. Titus Brown <ctb at msu.edu> wrote:
>> Quite a work, but I am in for some brainstroming at Pycon on this topic
>> if you are interested :)
>
> yep, absolutely! ?I think I've got the execution and reporting end
> handled; now we just need to get some virtual environments running so we
> can do untrusted packages.
>

Ah nice ! for the virtual environment, I am not sure about the proper
way to handle
network access if some packages work with sockets. They suppose
to fake I/O in tests and not acces the network in the code we might call,
but that's never sure,

> Hmm, might be worth an AWS account just to do the basic stuff.

absolutely,

maybe we could continue this topic off-list at snakebite's ?

Regards
Tarek

From mal at egenix.com  Wed Sep 16 18:12:39 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Sep 2009 18:12:39 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB0E5A8.1010005@voidspace.org.uk>
References: <4AB0D7D6.7030204@active-4.com>	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
Message-ID: <4AB10E77.4070706@egenix.com>

Michael Foord wrote:
> Backwards compatibility is a *big* problem
> for any major refactoring though.

Sigh.

I sometimes get the feeling that people on this list don't
know Python's history, how it was developed over the past decade
and what our goals were.

Maintaining as much backwards compatibility as reasonably possible
has always been a key goal and we've done a pretty good job at
it (if I may say so).

As Py3k approached, it was deemed ok to break with the past and
that was accepted by the core developers and the users. However,
that time has past now and we're running in non-breaking mode
again.

As we're starting to establish the Py3k branch as new stable
Python branch, we're not suddenly going to change the goals
we've established over the years in the Python 2.x branch.

Backwards compatibility is one of the key arguments for using
Python as a development platform. As such it's not a problem,
it's a feature of Python.

And while it may not mean much to developers who prefer to run
bleeding edge code, it does mean a lot to the established
Python user base.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 16 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Wed Sep 16 18:19:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 18:19:00 +0200
Subject: [stdlib-sig] Python-the-platform vs Python-the-framework
In-Reply-To: <FECFFCAB-A154-46BB-8A30-2F78B3098BC5@orestis.gr>
References: <FECFFCAB-A154-46BB-8A30-2F78B3098BC5@orestis.gr>
Message-ID: <1253117940.5829.95.camel@localhost>

Le mercredi 16 septembre 2009 ? 18:35 +0300, Orestis Markou a ?crit :
> 
> Hopefully it will help the discussion go forward, and perhaps help the
> people who are writing PEPs.

What would already help would be properly formatting your email. Most of
it is unreadable here... A good guideline is probably to send only plain
text email, so that your email editor's peculiarities don't fool you.

> We should let domain experts develop solutions for their domains
> outside the stdlib,

Nobody forbids them to do so.

Besides, you overlooked that not every domain is a fast-evolving
ecosystem in which everyone runs after the bleeding edge.

XML (the format, not the surrounding dialects and so-called
"technologies"), for example, has been quite stable. JSON, once it is
normalized, will probably not evolve a lot. URL quoting has been the
same for years. Even HTTP, to my knowledge, hasn't evolved a lot, and
the stdlib's weakness on that matter seems largely orthogonal to recent
evolutions.

For all these things, it seems quite reasonable and helpful to provide
solutions in the stdlib, rather than requiring the user to evaluate,
choose and install third-party libraries. Yes, the solutions must be
well-written, robust, flexible. There's no reason, however, why they
shouldn't be, except accidental (historical) reasons which were also
correlated with looser inclusion procedures.

> == Backwards compatibility is a straw man==

Starting your text like this will not encourage people to read what
follows... Since it is so badly formatted, I've stopped anyway.

Regards

Antoine.



From michael at voidspace.org.uk  Wed Sep 16 18:15:39 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 17:15:39 +0100
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB10E77.4070706@egenix.com>
References: <4AB0D7D6.7030204@active-4.com>	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk> <4AB10E77.4070706@egenix.com>
Message-ID: <4AB10F2B.3090600@voidspace.org.uk>

M.-A. Lemburg wrote:
> Michael Foord wrote:
>   
>> Backwards compatibility is a *big* problem
>> for any major refactoring though.
>>     
>
> Sigh.
>   

*sigh*

Don't you just love emails that start with a sigh. Anyway, yes. That is 
why I said it was a problem. Good grief.

Michael

> I sometimes get the feeling that people on this list don't
> know Python's history, how it was developed over the past decade
> and what our goals were.
>
> Maintaining as much backwards compatibility as reasonably possible
> has always been a key goal and we've done a pretty good job at
> it (if I may say so).
>
> As Py3k approached, it was deemed ok to break with the past and
> that was accepted by the core developers and the users. However,
> that time has past now and we're running in non-breaking mode
> again.
>
> As we're starting to establish the Py3k branch as new stable
> Python branch, we're not suddenly going to change the goals
> we've established over the years in the Python 2.x branch.
>
> Backwards compatibility is one of the key arguments for using
> Python as a development platform. As such it's not a problem,
> it's a feature of Python.
>
> And while it may not mean much to developers who prefer to run
> bleeding edge code, it does mean a lot to the established
> Python user base.
>
>   


-- 
http://www.ironpythoninaction.com/


From mal at egenix.com  Wed Sep 16 19:02:18 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Sep 2009 19:02:18 +0200
Subject: [stdlib-sig] Breaking out the stdlib
In-Reply-To: <1253055135.5678.100.camel@localhost>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com>	<200909151702.n8FH29uv029039@theraft.openend.se>	<43aa6ff70909151048l678a012ek4523e90676827cd9@mail.gmail.com>	<1253038514.5678.32.camel@localhost>	<4AAFDBAD.7040906@voidspace.org.uk>	<79990c6b0909151141o40c402eaw8445ff1c4bf9addc@mail.gmail.com>	<117125CC-8D8D-4DE4-9C72-8A1BCBA0430D@python.org>	<4AAFEE6B.3070900@egenix.com>	<Pine.LNX.4.64.0909151710290.17020@kimball.webabinitio.net>	<1253049748.5678.75.camel@localhost>	<4222a8490909151522w5e99c1dfw63c34a9454c3252b@mail.gmail.com>
	<1253055135.5678.100.camel@localhost>
Message-ID: <4AB11A1A.1010704@egenix.com>

Antoine Pitrou wrote:
> I'm all for non-exclusive maintainership, with people having reasonable
> authority over code they understand thoroughly. You taking care of
> multiprocessing falls into this category (as long as you don't demand to
> approve of all changes before they are committed).
> 
> I'm against ownership, however. I'm against mandatory signoffs
> (imprimaturs), and I'm against the possessive sentiment that some might
> develop towards a piece of code they contributed. Any core developer
> should be allowed to commit a patch if he thinks the patch is reasonable
> enough and serves an useful purpose.
> 
> Ownership prevents proper maintenance when the owner is absent (which
> *will* happen, since we are all unpaid volunteers). It discourages other
> people from getting acquainted with the code, which gradually makes the
> problem worse. Furthermore, it is often correlated with strange
> (personal) idioms, coding styles and design principles.

I don't see things as negative as you apparently do.

Owning a piece usually means that you wrote it and have good reasons
for the design and code being what it is. It's a matter of respect
towards the author to ask for review and discussion of a patch
or new idea.

If maintenance is passed on to someone else, the maintainer
becomes the authority to discuss a patch with.

If a maintainer or owner doesn't respond within 2-3 weeks
(yes, people do go on vacation sometimes ;-), then I think
it's ok to get review of the patch by some other core developer
and then check it in - the maintainer or owner can always go
back and revert the patch, if it doesn't fit for some reason.

If this happens for longer periods of time, the maintainer
or owner should be asked to pass on maintenance to someone
else.

Now, what to do in case of conflicts ? I.e. a core developer
wants to add some new feature, change the design, etc. and
the maintainer or owner disagrees.

Well, we do what we've always done in the past: discuss the
proposed changes on python-dev. If things go well, a compromise
is found, otherwise the BDFL decides as tie-breaker.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 16 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Wed Sep 16 19:12:52 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 16 Sep 2009 19:12:52 +0200
Subject: [stdlib-sig] Backwards compat (was: Evolving the Standard
	Library)
In-Reply-To: <D5ADE9B8-BAF2-40B5-A836-B70640CE212D@masklinn.net>
References: <4AB0D7D6.7030204@active-4.com>
	<a1c1f4750909160616o5fbd9326he62af32dd98afe63@mail.gmail.com>
	<4AB0E5A8.1010005@voidspace.org.uk>
	<4222a8490909160628i71cf794cmbbb7bc35769bded8@mail.gmail.com>
	<20090916144315.GH19160@idyll.org>
	<94bdd2610909160843l240eb250ia755ef2f48d7dc5e@mail.gmail.com>
	<20090916155414.GC2523@idyll.org>
	<D5ADE9B8-BAF2-40B5-A836-B70640CE212D@masklinn.net>
Message-ID: <94bdd2610909161012s71a9e4d0p367b0b8b5986c760@mail.gmail.com>

On Wed, Sep 16, 2009 at 7:07 PM, Masklinn <masklinn at masklinn.net> wrote:
> A basic FreeBSD machine and automatically creating and destroying BSD jails
> would probably work as well (and with less limitations) wouldn't it? (or
> another kind of virtualization, but jails seem to work pretty well for
> running untrusted code securely)

If we want a full system, we will need something that works on every
tested platform, so basically
Windows, Linux or BSD and Mac OS X.

But maybe what you are mentioning could be used for version #1

Tarek

From g.brandl at gmx.net  Wed Sep 16 21:37:02 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 21:37:02 +0200
Subject: [stdlib-sig] GHOP / py3k
In-Reply-To: <20090916144908.GI19160@idyll.org>
References: <4AB0D7D6.7030204@active-4.com> <4AB0E754.50201@voidspace.org.uk>
	<20090916144908.GI19160@idyll.org>
Message-ID: <h8resr$8l9$1@ger.gmane.org>

C. Titus Brown schrieb:
>>Michael said:
>> Armin said:
>> > That, and a group of
>> >people, dedicated to standard library refactoring.  The majority of
>> >libraries in the standard library are small and easy to understand, I'm
>> >sure they are perfectly suited for students on projects like GSOC or
>> >GHOP to work on.  They could even be used as some sort of "playground"
>> >for new Python developers.
>> 
>> It would be a great project for GHOP *if* we have some experienced 
>> developers, like yourself, dedicated to working out what the things that 
>> need fixing are.
> 
> Testing and doc updates worked reasonably well within GHOP last time,
> and surprisingly little in the way of "experienced developers" were
> needed.  Faced with the responsibility of coming up with dozens of tasks
> on short notice, I picked a dozen stdlib modules and said
> 
>   test this
>   integrate doug hellmann's documentation
>   run through the existing examples and write more
> 
> ...and voila, it happened and attracted positive notice from the BDFL,
> which is saying something.

It also means that it's going to be a bit harder to find new tasks this
year -- not much has changed in terms of the kind of work available.

Most libraries might seem "small and easy to understand" to Armin these
days, but certainly not to high school students from all over the world.
Combine that with the fact that the code quality in the stdlib is supposed
to *improve* due to these efforts, *and* that we need to keep a keen eye
on backwards compatibility, and we have the same non-trivial problem of
finding tasks.  In fact, I'd rather mentor two tasks than find one (good)
additional task :)

> By far the most important part of that process was not my role in
> putting the tasks up, but Georg's role in reviewing the patches and
> committing them in a timely manner.  I can't speak for how much time he
> spent doing that,

Enough for one lifetime!  No, seriously, I'm willing to help as a mentor
in this year's GHOP again, but I don't want to end up as the only core
developer involved.

> however, and I certainly don't expect that level of
> effort from HIM this time; perhaps with Mercurial we can get
> non-committers to act as first-pass filters to reduce the strain on
> Georg or whoever steps into his shoes. [0]

I sincerely hope that it could work that way. Not having to reject sub-
standard work is already a relief.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Wed Sep 16 22:08:28 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 16 Sep 2009 13:08:28 -0700
Subject: [stdlib-sig] GHOP / py3k (was: Evolving the Standard Library)
In-Reply-To: <20090916144908.GI19160@idyll.org>
References: <4AB0D7D6.7030204@active-4.com> <4AB0E754.50201@voidspace.org.uk> 
	<20090916144908.GI19160@idyll.org>
Message-ID: <bbaeab100909161308o552334d3tc0c10e82069f8f48@mail.gmail.com>

On Wed, Sep 16, 2009 at 07:49, C. Titus Brown <ctb at msu.edu> wrote:
>>Michael said:
>> Armin said:
>> > That, and a group of
>> >people, dedicated to standard library refactoring. ?The majority of
>> >libraries in the standard library are small and easy to understand, I'm
>> >sure they are perfectly suited for students on projects like GSOC or
>> >GHOP to work on. ?They could even be used as some sort of "playground"
>> >for new Python developers.
>>
>> It would be a great project for GHOP *if* we have some experienced
>> developers, like yourself, dedicated to working out what the things that
>> need fixing are.
>
> Testing and doc updates worked reasonably well within GHOP last time,
> and surprisingly little in the way of "experienced developers" were
> needed. ?Faced with the responsibility of coming up with dozens of tasks
> on short notice, I picked a dozen stdlib modules and said
>
> ?test this
> ?integrate doug hellmann's documentation
> ?run through the existing examples and write more
>
> ...and voila, it happened and attracted positive notice from the BDFL,
> which is saying something.
>

We can also run figleaf or coverage.py across the standard library and
see which modules have horrible test coverage.

> By far the most important part of that process was not my role in
> putting the tasks up, but Georg's role in reviewing the patches and
> committing them in a timely manner. ?I can't speak for how much time he
> spent doing that, however, and I certainly don't expect that level of
> effort from HIM this time; perhaps with Mercurial we can get
> non-committers to act as first-pass filters to reduce the strain on
> Georg or whoever steps into his shoes. [0]
>

Hey, I helped a lot last time (and plan to do so again). As for the
Mercurial thing, it might not happen by December, so don't rely on
that (although we always have the mirrors so we can instruct the
students on how to use those).

> Perhaps this time we can focus on py3k stuff with GHOP; that'd be great,
> and a real community boost, IMO.

One possibility is to simply only care about the test and doc changes for py3k.

-Brett

From michael at voidspace.org.uk  Wed Sep 16 22:10:06 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 21:10:06 +0100
Subject: [stdlib-sig] GHOP / py3k
In-Reply-To: <bbaeab100909161308o552334d3tc0c10e82069f8f48@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com> <4AB0E754.50201@voidspace.org.uk>
	<20090916144908.GI19160@idyll.org>
	<bbaeab100909161308o552334d3tc0c10e82069f8f48@mail.gmail.com>
Message-ID: <4AB1461E.9080807@voidspace.org.uk>

Brett Cannon wrote:
> On Wed, Sep 16, 2009 at 07:49, C. Titus Brown <ctb at msu.edu> wrote:
>   
>>> Michael said:
>>> Armin said:
>>>       
>>>> That, and a group of
>>>> people, dedicated to standard library refactoring.  The majority of
>>>> libraries in the standard library are small and easy to understand, I'm
>>>> sure they are perfectly suited for students on projects like GSOC or
>>>> GHOP to work on.  They could even be used as some sort of "playground"
>>>> for new Python developers.
>>>>         
>>> It would be a great project for GHOP *if* we have some experienced
>>> developers, like yourself, dedicated to working out what the things that
>>> need fixing are.
>>>       
>> Testing and doc updates worked reasonably well within GHOP last time,
>> and surprisingly little in the way of "experienced developers" were
>> needed.  Faced with the responsibility of coming up with dozens of tasks
>> on short notice, I picked a dozen stdlib modules and said
>>
>>  test this
>>  integrate doug hellmann's documentation
>>  run through the existing examples and write more
>>
>> ...and voila, it happened and attracted positive notice from the BDFL,
>> which is saying something.
>>
>>     
>
> We can also run figleaf or coverage.py across the standard library and
> see which modules have horrible test coverage.
>
>   
>> By far the most important part of that process was not my role in
>> putting the tasks up, but Georg's role in reviewing the patches and
>> committing them in a timely manner.  I can't speak for how much time he
>> spent doing that, however, and I certainly don't expect that level of
>> effort from HIM this time; perhaps with Mercurial we can get
>> non-committers to act as first-pass filters to reduce the strain on
>> Georg or whoever steps into his shoes. [0]
>>
>>     
>
> Hey, I helped a lot last time (and plan to do so again). As for the
> Mercurial thing, it might not happen by December, so don't rely on
> that (although we always have the mirrors so we can instruct the
> students on how to use those).
>
>   

I'll try and help.

Michael

>> Perhaps this time we can focus on py3k stuff with GHOP; that'd be great,
>> and a real community boost, IMO.
>>     
>
> One possibility is to simply only care about the test and doc changes for py3k.
>
> -Brett
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From g.brandl at gmx.net  Wed Sep 16 22:10:39 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 22:10:39 +0200
Subject: [stdlib-sig] GHOP / py3k
In-Reply-To: <h8resr$8l9$1@ger.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4AB0E754.50201@voidspace.org.uk>	<20090916144908.GI19160@idyll.org>
	<h8resr$8l9$1@ger.gmane.org>
Message-ID: <h8rgrs$fe3$1@ger.gmane.org>

Georg Brandl schrieb:

>> By far the most important part of that process was not my role in
>> putting the tasks up, but Georg's role in reviewing the patches and
>> committing them in a timely manner.  I can't speak for how much time he
>> spent doing that,
> 
> Enough for one lifetime!  No, seriously, I'm willing to help as a mentor
> in this year's GHOP again, but I don't want to end up as the only core
> developer involved.

Not that this has happened it the past, of course ;)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ctb at msu.edu  Wed Sep 16 22:17:08 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Sep 2009 13:17:08 -0700
Subject: [stdlib-sig] GHOP / py3k (was: Evolving the Standard Library)
In-Reply-To: <bbaeab100909161308o552334d3tc0c10e82069f8f48@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com> <4AB0E754.50201@voidspace.org.uk>
	<20090916144908.GI19160@idyll.org>
	<bbaeab100909161308o552334d3tc0c10e82069f8f48@mail.gmail.com>
Message-ID: <20090916201708.GA28516@idyll.org>

On Wed, Sep 16, 2009 at 01:08:28PM -0700, Brett Cannon wrote:
> On Wed, Sep 16, 2009 at 07:49, C. Titus Brown <ctb at msu.edu> wrote:
> >>Michael said:
> >> Armin said:
> >> > That, and a group of
> >> >people, dedicated to standard library refactoring. ??The majority of
> >> >libraries in the standard library are small and easy to understand, I'm
> >> >sure they are perfectly suited for students on projects like GSOC or
> >> >GHOP to work on. ??They could even be used as some sort of "playground"
> >> >for new Python developers.
> >>
> >> It would be a great project for GHOP *if* we have some experienced
> >> developers, like yourself, dedicated to working out what the things that
> >> need fixing are.
> >
> > Testing and doc updates worked reasonably well within GHOP last time,
> > and surprisingly little in the way of "experienced developers" were
> > needed. ??Faced with the responsibility of coming up with dozens of tasks
> > on short notice, I picked a dozen stdlib modules and said
> >
> > ??test this
> > ??integrate doug hellmann's documentation
> > ??run through the existing examples and write more
> >
> > ...and voila, it happened and attracted positive notice from the BDFL,
> > which is saying something.
> >
> 
> We can also run figleaf or coverage.py across the standard library and
> see which modules have horrible test coverage.

Yes, and I expect to integrate the GSoC work on C code coverage, too,
along with multi-platform build results.  (Titus's Motto: "coercing
other people into building my own testing infrastructure from scratch
since 2006".)

> > By far the most important part of that process was not my role in
> > putting the tasks up, but Georg's role in reviewing the patches and
> > committing them in a timely manner. ??I can't speak for how much time he
> > spent doing that, however, and I certainly don't expect that level of
> > effort from HIM this time; perhaps with Mercurial we can get
> > non-committers to act as first-pass filters to reduce the strain on
> > Georg or whoever steps into his shoes. [0]
> 
> Hey, I helped a lot last time (and plan to do so again). As for the
> Mercurial thing, it might not happen by December, so don't rely on
> that (although we always have the mirrors so we can instruct the
> students on how to use those).

Yes, I remember -- no denigration intended, just that Georg devoted an
absolutely insane amount of time to it ;)

> > Perhaps this time we can focus on py3k stuff with GHOP; that'd be great,
> > and a real community boost, IMO.
> 
> One possibility is to simply only care about the test and doc changes for py3k.

OK, sounds good.

--t
-- 
C. Titus Brown, ctb at msu.edu

From brett at python.org  Wed Sep 16 22:12:39 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 16 Sep 2009 13:12:39 -0700
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <5c6f2a5d0909160805p36e91dc2yc9e70e286919419a@mail.gmail.com>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<1253037263.5678.12.camel@localhost>
	<bbaeab100909151121r5375236dob8472381f6efcfe8@mail.gmail.com> 
	<Pine.LNX.4.64.0909151715120.17020@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net> 
	<94bdd2610909151601p73fb08c8qea8ba962d5164a2b@mail.gmail.com> 
	<Pine.LNX.4.64.0909151924450.17020@kimball.webabinitio.net> 
	<0C696A692D3E4F229FC3EC345CAAA6A3@RaymondLaptop1>
	<1253112422.5829.50.camel@localhost> 
	<5c6f2a5d0909160805p36e91dc2yc9e70e286919419a@mail.gmail.com>
Message-ID: <bbaeab100909161312r3289b217jbfae0d914ccc6683@mail.gmail.com>

On Wed, Sep 16, 2009 at 08:05, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Wed, Sep 16, 2009 at 3:47 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le mardi 15 septembre 2009 ? 19:42 -0700, Raymond Hettinger a ?crit :
>>> > That would be a third source of info about who maintains what.
>>>
>>> Is this just a formalization of what we already do now?
>>
>> I cannot speak for David, but IMHO it should be a bit more than that.
>>
>> The underlying idea is to promote a broader (and therefore less
>> exclusive) view of maintenership. We should extend the meaning of
>> "maintainer" (or "expert" as also stated) to people who are 1) competent
>> enough to give useful advice on (bug/feature) requests concerning a
>> module and 2) at least moderately willing to do so; this rather than the
>> supposed "owner" of a module, which is a notion we should discourage.
>
> +1 for this interpretation of maintainer.
>
> And a big +1 for Misc/maintainers.txt, or equivalent information elsewhere
> (a wiki page, perhaps). ?I've frequently found myself wanting this
> information when commenting on an issue (most recently for the
> curses module: ?the source says that AMK is the current maintainer,
> but I don't know whether that information is still in date, and if not,
> who else might be familiar enough to fix curses issues). ?I've not been
> around long enough to know the history of the various bits and pieces
> of Python.

This is the interpretation I have always held and this is the reason
we should do this (don't wait, David, just make the list; people who
don't like it can just ignore it). If we can help people who do issue
triage by writing down knowledge that the rest of us have just
instilled over the years then it is a benefit.

-Brett

From brett at python.org  Wed Sep 16 22:14:54 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 16 Sep 2009 13:14:54 -0700
Subject: [stdlib-sig] MISC/maintainers.txt anyone?
In-Reply-To: <h8qmrl$fnk$1@ger.gmane.org>
References: <4222a8490909140813n715a6350s6b647ab8555e424b@mail.gmail.com> 
	<Pine.LNX.4.64.0909151717520.17020@kimball.webabinitio.net> 
	<4222a8490909151551m12f0c598v14ad078d1b8db61c@mail.gmail.com> 
	<1afaf6160909151555y669f5f78x447100cdf4afc9a@mail.gmail.com> 
	<4222a8490909151600s45bdef8fk6b32492d80b869cb@mail.gmail.com> 
	<h8q69c$q8i$1@ger.gmane.org> <4AB09FE6.6030308@egenix.com> 
	<h8q8qq$37t$1@ger.gmane.org> <4AB0DA3E.6030400@active-4.com> 
	<h8qmrl$fnk$1@ger.gmane.org>
Message-ID: <bbaeab100909161314k758fb79fke90f6de16e7029e2@mail.gmail.com>

On Wed, Sep 16, 2009 at 05:46, Georg Brandl <g.brandl at gmx.net> wrote:
> Armin Ronacher schrieb:
>> Hi,
>>
>> Georg Brandl wrote:
>>> That's why at PyCon (or was it Europython?)

PyCon.

> we thought about a "tags"
>>> field for issues. ?The main use for tags would be module names (and
>>> it should be advertised as such in the UI for submitting bugs)
>>> with auto-nosy associations for module maintainers.
>> Arbitrary tags in a bug tracker are annoying. ?People will not try to
>> find exiting tags but make up their own, making it hard to query for it.
>
> I disagree. Again: we wouldn't ask submitters to populate that list with
> tags they make up, but the module name. ?In the process as I envision it
> now, further tags are added/changed by developers.
>

Sounds reasonable to me. That would allow us to pull out some of the
stuff we have in the Components list to make that clearer to use.

>> I would rather see an automatically generated module list based on the
>> hg tip / svn head.
>
> That's not so easy to do.
>

Nor really necessary once the initial list is done; we don't exactly
add modules that frequently.

-Brett

> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>

From mats at sun.com  Thu Sep 17 18:53:42 2009
From: mats at sun.com (Mats Kindahl)
Date: Thu, 17 Sep 2009 18:53:42 +0200
Subject: [stdlib-sig] ConfigParser - parsing options with no value
Message-ID: <4AB26996.8030707@sun.com>

Hi all,

I am currently using the ConfigParser module to parse MySQL configuration files.
 Just plain config files, nothing fancy except... options without values.

There is a number of options here that does not require a value, they are
basically just turning on a feature. They are also common in the standard
template files for the server. Options that are for mysqld can have a value even
though it is not required and the option file parser will not complain, but for
some of the client tools, values may not be given or there will be a error.

Looking at the tests of ConfigParser, I see that it is a deliberate design
decision to not allow options without values (or I am misunderstanding
something). Why?

Being able to parse these files without having to write a new module would be
very valuable.

Best wishes,
Mats Kindahl
-- 
Mats Kindahl
Senior Software Engineer
Database Technology Group
Sun Microsystems

From vinay_sajip at yahoo.co.uk  Thu Sep 17 18:56:40 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 17 Sep 2009 16:56:40 +0000 (UTC)
Subject: [stdlib-sig] Evolving the Standard Library
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
Message-ID: <loom.20090917T173746-561@post.gmane.org>

Hi everyone,

I'm all for improving the standard library, and as the author of the logging
package, have a keen interest in making sure that it is relevant and usable by
most if not all of the Python community, and that it evolves with changing
needs. However, this objective is made much harder by what I see as some
shortcomings in the way we all communicate about these issues. For example, from
a post earlier in this thread, here are two snippets...

[Jesse]

And SimpleHTTPServer, and logging, and... Armin, some of us agree with
you, and again, this was part of my driving force in starting the
other thread proposing the logical break out and subsequent cleanup.

[Armin]
Of course libraries like `locale` and `logging` are hard to change, but
it would still be possible.  For `locale` it would probably a useful

Now, it's not hard to find out that I'm the author of the logging package -
apart from being co-author of the PEP which introduced it, I'm fairly active on
python-list when logging-related issues crop up, as well as promptly addressing
issues in the bug tracker. I'm also not that hard to find via Google when you
search for "python logging".

>From the above snippets, one could infer that both Armin and Jesse have some
"issues" with Python's logging package. In Brett's informal poll, logging was
one of the packages which people raised as "needs to change" - it came third in
the "hall of shame". When Brett posted about it, at

http://sayspy.blogspot.com/2009/07/results-of-informal-poll-about-standard.html

I followed up there at some length. It seems a lot of this stuff gets discussed
on Twitter, which makes it very easy for meanings to be misinterpreted because
you can't always be clear about what you mean in 140 chars. (I don't use Twitter
myself, as for me the noise to signal ratio is far too high.) I was at one point
given to understand that in some tweet or other, Andrii Mishkovskyi apparently
offered to rewrite the logging package. Andrii has assured me that he hadn't
actually meant to cause offence, but surely you can see it comes across as a tad
impolite, given that the package has an active maintainer.

Jesse's and Armin's comments above epitomise the problem. As far as I know (with
Google's help), neither has ever bothered to post on python-list, python-dev,
their own blogs or anywhere else what these "issues" with logging are. Nor has
either ever contacted me directly. Yet they talk blithely about changing
logging, as if it has no maintainer. What exactly is the difficulty in
articulating your issues? Armin has done a fair job on describing bad points
about other parts of the library, and fair points they are, too. Armin did once
mention logging in a post about singletons, because logging does contain
singletons. However, as far as I know it does not cause problems in practice - I
use it with Django on numerous websites and the Tornado webserver of
FriendFeed/Facebook uses it too, apparently without the sky falling on its head.

Andrii Mishkovskyi set up a page on the Python wiki,

http://wiki.python.org/moin/LoggingPackage

where he posted his criticisms of the logging package and invited comments.
Great! Something specific to work on. I responded to all his points, and waited
for others to weigh in. Since 8 August, when I made my last changes to it, that
page has not been changed - by Andrii, Jesse or anyone else.

I'm not expecting logging to be anyone's hot button except mine, but I am
committed to maintaining it. If you're not interested in improving it, don't
mention it in the offhand way I quoted above - it's not the type of criticism
that I can work from. And if you are interested in improving it, take the time
to articulate the issues. For example, I recently came across the Opster library
(which wraps getopt) and really liked some aspects of it, though at the moment
argparse is my package of choice for command line parsing. I contacted both
Steven Bethard, argparse author and Alexander Solovyov, author of Opster, about
trying to get some synergy going between the two approaches. I did this using
the argparse Google code project issue tracker (for contact with Steven) and
(for Alexander) by commenting on his blog entry about Opster. Both contacts have
been fruitful, at least from my point of view as a user/potential user of their
work.

This might sound a bit like a rant, but it's not meant to be - I just speak as I
find. I'm not overly sensitive to criticism about the logging package; in fact I
welcome *constructive* criticism which can help to improve it. All of you,
please feel free to head over to Andrii's page to post your criticisms/comments
there.

If my expectations are that:

- I'm not the dead parrot - think of me more as the elephant in the room. If you
have issues with my work, talk to me.
- Use a platform where meanings have the potential to be clear - i.e. let's not
make being on Twitter a pre-requisite for discourse.
- Avoid the general snide-sounding "Logging sucks." "Yes, doesn't it just?" kind
of comments. It's great to vent, but is that the best you want to aim for?
- Remember the exigencies of backward compatibility. Root and branch changes to
the public API are clearly out, at least for now - not just for logging but for
the whole stdlib.

Am I expecting too much?

Regards,


Vinay Sajip


From brett at python.org  Thu Sep 17 20:14:36 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Sep 2009 11:14:36 -0700
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <4AB26996.8030707@sun.com>
References: <4AB26996.8030707@sun.com>
Message-ID: <bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>

On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
> Hi all,
>
> I am currently using the ConfigParser module to parse MySQL configuration files.
> ?Just plain config files, nothing fancy except... options without values.
>
> There is a number of options here that does not require a value, they are
> basically just turning on a feature. They are also common in the standard
> template files for the server. Options that are for mysqld can have a value even
> though it is not required and the option file parser will not complain, but for
> some of the client tools, values may not be given or there will be a error.
>
> Looking at the tests of ConfigParser, I see that it is a deliberate design
> decision to not allow options without values (or I am misunderstanding
> something). Why?
>

Who knows. =) Module is old enough it's quite possible no one
remembers why. Maybe the original creator thought it would help catch
errors when people accidentally left off a value.

-Brett

From rdmurray at bitdance.com  Thu Sep 17 20:21:44 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 14:21:44 -0400 (EDT)
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <loom.20090917T173746-561@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0909171418220.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 16:56, Vinay Sajip wrote:
> Now, it's not hard to find out that I'm the author of the logging package -
> apart from being co-author of the PEP which introduced it, I'm fairly active on
> python-list when logging-related issues crop up, as well as promptly addressing
> issues in the bug tracker. I'm also not that hard to find via Google when you
> search for "python logging".

I spend more time watching the bug tracker than I should, and I can
confirm that Vinay is _very_ responsive to tracker issues concerning
logging.

--David

From brett at python.org  Thu Sep 17 20:23:48 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Sep 2009 11:23:48 -0700
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <loom.20090917T173746-561@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com> 
	<loom.20090917T173746-561@post.gmane.org>
Message-ID: <bbaeab100909171123o63b8def9hfe2d010be9ce173@mail.gmail.com>

On Thu, Sep 17, 2009 at 09:56, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
[snip]
> If my expectations are that:
>
> - I'm not the dead parrot - think of me more as the elephant in the room. If you
> have issues with my work, talk to me.
> - Use a platform where meanings have the potential to be clear - i.e. let's not
> make being on Twitter a pre-requisite for discourse.
> - Avoid the general snide-sounding "Logging sucks." "Yes, doesn't it just?" kind
> of comments. It's great to vent, but is that the best you want to aim for?
> - Remember the exigencies of backward compatibility. Root and branch changes to
> the public API are clearly out, at least for now - not just for logging but for
> the whole stdlib.
>
> Am I expecting too much?

Nope. And hopefully David's maintainer list will help make sure people
talk to you directly.

-Brett

From michael at voidspace.org.uk  Thu Sep 17 21:03:17 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Thu, 17 Sep 2009 20:03:17 +0100
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <loom.20090917T173746-561@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
Message-ID: <4AB287F5.9030107@voidspace.org.uk>

This is a great response from a standard library module maintainer.

Michael

Vinay Sajip wrote:
> Hi everyone,
>
> I'm all for improving the standard library, and as the author of the logging
> package, have a keen interest in making sure that it is relevant and usable by
> most if not all of the Python community, and that it evolves with changing
> needs. However, this objective is made much harder by what I see as some
> shortcomings in the way we all communicate about these issues. For example, from
> a post earlier in this thread, here are two snippets...
>
> [Jesse]
>
> And SimpleHTTPServer, and logging, and... Armin, some of us agree with
> you, and again, this was part of my driving force in starting the
> other thread proposing the logical break out and subsequent cleanup.
>
> [Armin]
> Of course libraries like `locale` and `logging` are hard to change, but
> it would still be possible.  For `locale` it would probably a useful
>
> Now, it's not hard to find out that I'm the author of the logging package -
> apart from being co-author of the PEP which introduced it, I'm fairly active on
> python-list when logging-related issues crop up, as well as promptly addressing
> issues in the bug tracker. I'm also not that hard to find via Google when you
> search for "python logging".
>
> >From the above snippets, one could infer that both Armin and Jesse have some
> "issues" with Python's logging package. In Brett's informal poll, logging was
> one of the packages which people raised as "needs to change" - it came third in
> the "hall of shame". When Brett posted about it, at
>
> http://sayspy.blogspot.com/2009/07/results-of-informal-poll-about-standard.html
>
> I followed up there at some length. It seems a lot of this stuff gets discussed
> on Twitter, which makes it very easy for meanings to be misinterpreted because
> you can't always be clear about what you mean in 140 chars. (I don't use Twitter
> myself, as for me the noise to signal ratio is far too high.) I was at one point
> given to understand that in some tweet or other, Andrii Mishkovskyi apparently
> offered to rewrite the logging package. Andrii has assured me that he hadn't
> actually meant to cause offence, but surely you can see it comes across as a tad
> impolite, given that the package has an active maintainer.
>
> Jesse's and Armin's comments above epitomise the problem. As far as I know (with
> Google's help), neither has ever bothered to post on python-list, python-dev,
> their own blogs or anywhere else what these "issues" with logging are. Nor has
> either ever contacted me directly. Yet they talk blithely about changing
> logging, as if it has no maintainer. What exactly is the difficulty in
> articulating your issues? Armin has done a fair job on describing bad points
> about other parts of the library, and fair points they are, too. Armin did once
> mention logging in a post about singletons, because logging does contain
> singletons. However, as far as I know it does not cause problems in practice - I
> use it with Django on numerous websites and the Tornado webserver of
> FriendFeed/Facebook uses it too, apparently without the sky falling on its head.
>
> Andrii Mishkovskyi set up a page on the Python wiki,
>
> http://wiki.python.org/moin/LoggingPackage
>
> where he posted his criticisms of the logging package and invited comments.
> Great! Something specific to work on. I responded to all his points, and waited
> for others to weigh in. Since 8 August, when I made my last changes to it, that
> page has not been changed - by Andrii, Jesse or anyone else.
>
> I'm not expecting logging to be anyone's hot button except mine, but I am
> committed to maintaining it. If you're not interested in improving it, don't
> mention it in the offhand way I quoted above - it's not the type of criticism
> that I can work from. And if you are interested in improving it, take the time
> to articulate the issues. For example, I recently came across the Opster library
> (which wraps getopt) and really liked some aspects of it, though at the moment
> argparse is my package of choice for command line parsing. I contacted both
> Steven Bethard, argparse author and Alexander Solovyov, author of Opster, about
> trying to get some synergy going between the two approaches. I did this using
> the argparse Google code project issue tracker (for contact with Steven) and
> (for Alexander) by commenting on his blog entry about Opster. Both contacts have
> been fruitful, at least from my point of view as a user/potential user of their
> work.
>
> This might sound a bit like a rant, but it's not meant to be - I just speak as I
> find. I'm not overly sensitive to criticism about the logging package; in fact I
> welcome *constructive* criticism which can help to improve it. All of you,
> please feel free to head over to Andrii's page to post your criticisms/comments
> there.
>
> If my expectations are that:
>
> - I'm not the dead parrot - think of me more as the elephant in the room. If you
> have issues with my work, talk to me.
> - Use a platform where meanings have the potential to be clear - i.e. let's not
> make being on Twitter a pre-requisite for discourse.
> - Avoid the general snide-sounding "Logging sucks." "Yes, doesn't it just?" kind
> of comments. It's great to vent, but is that the best you want to aim for?
> - Remember the exigencies of backward compatibility. Root and branch changes to
> the public API are clearly out, at least for now - not just for logging but for
> the whole stdlib.
>
> Am I expecting too much?
>
> Regards,
>
>
> Vinay Sajip
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/


From mal at egenix.com  Thu Sep 17 22:08:23 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 17 Sep 2009 22:08:23 +0200
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <4AB26996.8030707@sun.com>
References: <4AB26996.8030707@sun.com>
Message-ID: <4AB29737.2020107@egenix.com>

Mats Kindahl wrote:
> Hi all,
> 
> I am currently using the ConfigParser module to parse MySQL configuration files.
>  Just plain config files, nothing fancy except... options without values.
> 
> There is a number of options here that does not require a value, they are
> basically just turning on a feature. They are also common in the standard
> template files for the server. Options that are for mysqld can have a value even
> though it is not required and the option file parser will not complain, but for
> some of the client tools, values may not be given or there will be a error.
> 
> Looking at the tests of ConfigParser, I see that it is a deliberate design
> decision to not allow options without values (or I am misunderstanding
> something). Why?

Probably to provide better error reporting and avoid casual
user errors.

> Being able to parse these files without having to write a new module would be
> very valuable.

Please submit a feature request for this, ideally with patch to
the Python tracker.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 17 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mats at sun.com  Thu Sep 17 22:28:50 2009
From: mats at sun.com (Mats Kindahl)
Date: Thu, 17 Sep 2009 22:28:50 +0200
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
Message-ID: <4AB29C02.8080409@sun.com>



Brett Cannon wrote:
> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>> Hi all,
>>
>> I am currently using the ConfigParser module to parse MySQL configuration files.
>>  Just plain config files, nothing fancy except... options without values.
>>
>> There is a number of options here that does not require a value, they are
>> basically just turning on a feature. They are also common in the standard
>> template files for the server. Options that are for mysqld can have a value even
>> though it is not required and the option file parser will not complain, but for
>> some of the client tools, values may not be given or there will be a error.
>>
>> Looking at the tests of ConfigParser, I see that it is a deliberate design
>> decision to not allow options without values (or I am misunderstanding
>> something). Why?
>>
> 
> Who knows. =) Module is old enough it's quite possible no one
> remembers why. Maybe the original creator thought it would help catch
> errors when people accidentally left off a value.

Hmmm....

If one should allow value-less options, I see two ways to handle that:

- For options lacking values, set the value to the empty string, or

- For options lacking values, set the value to None.

I have a patch for the first case, but I really think that the second approach
is better. That would make it possible to distinguish between options that were
given the empty string value and that were not given a value at all.

Where should I post such patches and what is people opinion of these to
alternatives?

Another consideration is how to allow this extension. It would be possible to
add flags to allow new and old behaviour, but I don't know how much value people
attribute to be able to get an exception if there is no value for the option.

I am also thinking that some option files have extensions to be able to include
files (MySQL option parsing package has), so it might be a good idea to add
callbacks to handle unrecognized lines, allowing the callback to inject text
into the parsing stream to, for example, support include files or even a macro
language.

Just my few cents,
Mats Kindahl
-- 
Mats Kindahl
Senior Software Engineer
Database Technology Group
Sun Microsystems

From rdmurray at bitdance.com  Thu Sep 17 22:34:20 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 16:34:20 -0400 (EDT)
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <4AB29C02.8080409@sun.com>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<4AB29C02.8080409@sun.com>
Message-ID: <Pine.LNX.4.64.0909171631030.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 22:28, Mats Kindahl wrote:
> Where should I post such patches and what is people opinion of these to
> alternatives?

The bug tracker (bugs.python.org).

> Another consideration is how to allow this extension. It would be possible to
> add flags to allow new and old behaviour, but I don't know how much value people
> attribute to be able to get an exception if there is no value for the option.

Python's philosophy is to remain backward compatible, so this new
behavior should be controlled by an option of some sort whose default
value is the old behavior.

> I am also thinking that some option files have extensions to be able to include
> files (MySQL option parsing package has), so it might be a good idea to add
> callbacks to handle unrecognized lines, allowing the callback to inject text
> into the parsing stream to, for example, support include files or even a macro
> language.

That should be a separate feature request/tracker issue/patch.

--David

From g.brandl at gmx.net  Thu Sep 17 23:58:13 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Sep 2009 23:58:13 +0200
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
Message-ID: <h8ubhl$t69$1@ger.gmane.org>

Brett Cannon schrieb:
> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>> Hi all,
>>
>> I am currently using the ConfigParser module to parse MySQL configuration files.
>>  Just plain config files, nothing fancy except... options without values.
>>
>> There is a number of options here that does not require a value, they are
>> basically just turning on a feature. They are also common in the standard
>> template files for the server. Options that are for mysqld can have a value even
>> though it is not required and the option file parser will not complain, but for
>> some of the client tools, values may not be given or there will be a error.
>>
>> Looking at the tests of ConfigParser, I see that it is a deliberate design
>> decision to not allow options without values (or I am misunderstanding
>> something). Why?
>>
> 
> Who knows. =) Module is old enough it's quite possible no one
> remembers why.

Good point.  There's also an awful shortage of config parsers in the standard
library; we should add one or two.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From armin.ronacher at active-4.com  Fri Sep 18 00:32:05 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Fri, 18 Sep 2009 00:32:05 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <loom.20090917T173746-561@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
Message-ID: <4AB2B8E5.1070500@active-4.com>

Hi,

Vinay Sajip schrieb:
> Jesse's and Armin's comments above epitomise the problem. As far as I
> know (with Google's help), neither has ever bothered to post on
> python-list, python-dev, their own blogs or anywhere else what these
> "issues" with logging are.
I agree that nobody did.  And there is a reason for it, and that reason
is probably something that could be discussed in a separate thread too,
because it affects a lof stuff that is currently in the standard libary.

I looked at the logging library and my point of view is that logging is
"broken by design".  Eg: you can't fix it without breaking backwards
compatibility.

> Armin has done a fair job on describing bad points about other parts
> of the library, and fair points they are, too. Armin did once mention
> logging in a post about singletons, because logging does contain 
> singletons. However, as far as I know it does not cause problems in
> practice - I use it with Django on numerous websites and the Tornado
> webserver of FriendFeed/Facebook uses it too, apparently without the
> sky falling on its head.
I'm sorry for the wait I was expressing my disagreement with design
decisions made in the standard library and I would love to change that.
 One of the biggest griefs I have with how the standard library works it
that it does not work for me and everytime I discuss that topic with
anyone else I immediately get the feedback that "this does not happen in
the real world" or "works for me" etc.

I consider any kind of shared state a design mistake, no matter if it
may work for some people or not, the logging package is no exception.
However I have to admit that global loggers appears to be one of the
easiest solutions for the problem.

If I would have to create my own logging library, I would go a total
different part from design to implementation.  I would create a system
where the "sender" is an arbitrary Python object and the system that
handles it would have to check for its own if it may process that
message.  Only *if* there was any handler that wants to do anything with
that message, it would pull the details form the stack and format the
message.  That also would get rid of the formatters and all the other
stuff we currently have and avoids a global registry of loggers.

How would I configure a logger in a library then?  I library would
*never* by default print anything that is logged.  Default
configurations for the logging system currently are the biggest reason
for me to hate that library.  Many libraries do not even give you the
ability to turn of logging in general, and documentation for the logging
system was probably one of the reasons.  I don't know when the
"NullHandler" example appeared in the docs, but I'm sure it was not
there when I started using the logging system.

I cannot use logging for anything serious because it is slow, it has
shared state that often causes frustration, you cannot delete loggers
and much more.  Of course I do use logging for libraries because it's
the standard and the best we have got.

The only reasons (in my opinion) that logging is still around is that
it's in the stdlib, not because it's any good.  Especially when it comes
to highly optimized code in web applications you will quickly discover
that half a dozen log calls take up more CPU cycles than the actual
application code.

> This might sound a bit like a rant, but it's not meant to be - I just
> speak as  find. I'm not overly sensitive to criticism about the
> logging package; in fact I welcome *constructive* criticism which can
> help to improve it.  All of you, please feel free to head over to
> Andrii's page to post your criticisms/comments there.
I don't know how I could contribute constructive criticism.  I'm sorry
for that.

> Am I expecting too much?
I'm afraid you are.  It's hard to accept that some people think of your
system as "it just sucks", but you can't change that.  I know that
feeling from some of the stuff I wrote.  It's just a little bit worse
for you because logging is the standard (and only) one, everybody uses.
 For the stuff I wrote*, people have choice.  If they don't want to use
it, they don't have to.


Regards,
Armin

* external libraries, not distributed as part of Python except for the
  rather simple "ast" module.

From python at rcn.com  Fri Sep 18 00:40:54 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 17 Sep 2009 15:40:54 -0700
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB2B8E5.1070500@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
Message-ID: <D42F4700-0FF8-4C46-8269-BA845076201D@rcn.com>


On Sep 17, 2009, at 3:32 PM, Armin Ronacher wrote:

> Hi,
>
> Vinay Sajip schrieb:
>> Jesse's and Armin's comments above epitomise the problem. As far as I
>> know (with Google's help), neither has ever bothered to post on
>> python-list, python-dev, their own blogs or anywhere else what these
>> "issues" with logging are.
> I agree that nobody did.  And there is a reason for it, and that  
> reason
> is probably something that could be discussed in a separate thread  
> too,
> because it affects a lof stuff that is currently in the standard  
> libary.
>
> I looked at the logging library and my point of view is that logging  
> is
> "broken by design".  Eg: you can't fix it without breaking backwards
> compatibility.

I think we way just have to live with that one.
While I'm no fan of the logging module, it is widely used.
It was based on a Java version and Guido blessed it early-on.


Raymond



From armin.ronacher at active-4.com  Fri Sep 18 00:50:57 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Fri, 18 Sep 2009 00:50:57 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <D42F4700-0FF8-4C46-8269-BA845076201D@rcn.com>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<D42F4700-0FF8-4C46-8269-BA845076201D@rcn.com>
Message-ID: <4AB2BD51.9010500@active-4.com>

Hi,

Raymond Hettinger schrieb:
> I think we way just have to live with that one.
> While I'm no fan of the logging module, it is widely used.
> It was based on a Java version and Guido blessed it early-on.
I'm not insane enough to seriously consider replacing it.  I just wanted
to point out why I never wrote a ticket / mail to a mailing list in the
first place.

Regards,
Armin

From python at rcn.com  Fri Sep 18 01:00:41 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 17 Sep 2009 16:00:41 -0700
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB2BD51.9010500@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<D42F4700-0FF8-4C46-8269-BA845076201D@rcn.com>
	<4AB2BD51.9010500@active-4.com>
Message-ID: <20B0B846-DE80-4DBC-8C86-30AF01271594@rcn.com>


On Sep 17, 2009, at 3:50 PM, Armin Ronacher wrote:

> Hi,
>
> Raymond Hettinger schrieb:
>> I think we way just have to live with that one.
>> While I'm no fan of the logging module, it is widely used.
>> It was based on a Java version and Guido blessed it early-on.
> I'm not insane enough to seriously consider replacing it.  I just  
> wanted
> to point out why I never wrote a ticket / mail to a mailing list in  
> the
> first place.


That's good.  For a moment, I thought you had lost your marbles ;-)


Raymond

From michael at voidspace.org.uk  Fri Sep 18 01:34:27 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Fri, 18 Sep 2009 00:34:27 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <h8ubhl$t69$1@ger.gmane.org>
References: <4AB26996.8030707@sun.com>	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org>
Message-ID: <4AB2C783.3040105@voidspace.org.uk>

Georg Brandl wrote:
> Brett Cannon schrieb:
>   
>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>>     
>>> Hi all,
>>>
>>> I am currently using the ConfigParser module to parse MySQL configuration files.
>>>  Just plain config files, nothing fancy except... options without values.
>>>
>>> There is a number of options here that does not require a value, they are
>>> basically just turning on a feature. They are also common in the standard
>>> template files for the server. Options that are for mysqld can have a value even
>>> though it is not required and the option file parser will not complain, but for
>>> some of the client tools, values may not be given or there will be a error.
>>>
>>> Looking at the tests of ConfigParser, I see that it is a deliberate design
>>> decision to not allow options without values (or I am misunderstanding
>>> something). Why?
>>>
>>>       
>> Who knows. =) Module is old enough it's quite possible no one
>> remembers why.
>>     
>
> Good point.  There's also an awful shortage of config parsers in the standard
> library; we should add one or two.
>
> Georg
>
>   
I happen to know a good one we could add. ;-)

Michael

-- 
http://www.ironpythoninaction.com/


From jnoller at gmail.com  Fri Sep 18 01:59:27 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Sep 2009 19:59:27 -0400
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <4AB2C783.3040105@voidspace.org.uk>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <4AB2C783.3040105@voidspace.org.uk>
Message-ID: <CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>



On Sep 17, 2009, at 7:34 PM, Michael Foord <michael at voidspace.org.uk>  
wrote:

> Georg Brandl wrote:
>> Brett Cannon schrieb:
>>
>>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am currently using the ConfigParser module to parse MySQL  
>>>> configuration files.
>>>> Just plain config files, nothing fancy except... options without  
>>>> values.
>>>>
>>>> There is a number of options here that does not require a value,  
>>>> they are
>>>> basically just turning on a feature. They are also common in the  
>>>> standard
>>>> template files for the server. Options that are for mysqld can  
>>>> have a value even
>>>> though it is not required and the option file parser will not  
>>>> complain, but for
>>>> some of the client tools, values may not be given or there will  
>>>> be a error.
>>>>
>>>> Looking at the tests of ConfigParser, I see that it is a  
>>>> deliberate design
>>>> decision to not allow options without values (or I am  
>>>> misunderstanding
>>>> something). Why?
>>>>
>>>>
>>> Who knows. =) Module is old enough it's quite possible no one
>>> remembers why.
>>>
>>
>> Good point.  There's also an awful shortage of config parsers in  
>> the standard
>> library; we should add one or two.
>>
>> Georg
>>
>>
> I happen to know a good one we could add. ;-)
>
> Michael
>

+1 I eagerly await a PEP

From ubershmekel at gmail.com  Fri Sep 18 02:07:12 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 18 Sep 2009 03:07:12 +0300
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <20B0B846-DE80-4DBC-8C86-30AF01271594@rcn.com>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<D42F4700-0FF8-4C46-8269-BA845076201D@rcn.com>
	<4AB2BD51.9010500@active-4.com>
	<20B0B846-DE80-4DBC-8C86-30AF01271594@rcn.com>
Message-ID: <9d153b7c0909171707i447c6abbv262f40b3f4065e69@mail.gmail.com>

I'm not sure exactly how your logging API would look, but is there a PyPI
logging module that you would recommend or something similar? It sounds as
though the logging module could have a Class for handling logging
differently as you would propose. Though I'm not sure what that is.

On Fri, Sep 18, 2009 at 2:00 AM, Raymond Hettinger <python at rcn.com> wrote:

>
> On Sep 17, 2009, at 3:50 PM, Armin Ronacher wrote:
>
>  Hi,
>>
>> Raymond Hettinger schrieb:
>>
>>> I think we way just have to live with that one.
>>> While I'm no fan of the logging module, it is widely used.
>>> It was based on a Java version and Guido blessed it early-on.
>>>
>> I'm not insane enough to seriously consider replacing it.  I just wanted
>> to point out why I never wrote a ticket / mail to a mailing list in the
>> first place.
>>
>
>
> That's good.  For a moment, I thought you had lost your marbles ;-)
>
>
> Raymond
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090918/e002300c/attachment-0001.htm>

From solipsis at pitrou.net  Fri Sep 18 02:11:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Sep 2009 02:11:16 +0200
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <h8ubhl$t69$1@ger.gmane.org>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org>
Message-ID: <1253232676.5889.28.camel@localhost>

Le jeudi 17 septembre 2009 ? 23:58 +0200, Georg Brandl a ?crit :
> > 
> > Who knows. =) Module is old enough it's quite possible no one
> > remembers why.
> 
> Good point.  There's also an awful shortage of config parsers in the standard
> library; we should add one or two.

If so, can it have a .network attribute?



From michael at voidspace.org.uk  Fri Sep 18 02:09:12 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Fri, 18 Sep 2009 01:09:12 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <4AB2C783.3040105@voidspace.org.uk>
	<CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>
Message-ID: <2015A3C0-8E2B-4575-93C7-60B3E408EF4C@voidspace.org.uk>


Sorry to top-post, sent from phone-like-device.

Actually Guido has shot down the inclusion of ConfigObj on the past,  
so it doesn't seem worth a PEP.

He doesn't like that sections inherit from dict (although they are  
functionally key-value mappings) as it gives them an over-wide API.  
This is fair enough.

He also fundamentally disliked the idea of nested sections, doesn't  
see the need and suggested people should be using XML for this kind of  
use case! It is an often requested feature, so this I thought was odd.  
Anyway, so be it.

Michael

--
http://www.ironpythoninaction.com

On 18 Sep 2009, at 00:59, Jesse Noller <jnoller at gmail.com> wrote:

>
>
> On Sep 17, 2009, at 7:34 PM, Michael Foord  
> <michael at voidspace.org.uk> wrote:
>
>> Georg Brandl wrote:
>>> Brett Cannon schrieb:
>>>
>>>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am currently using the ConfigParser module to parse MySQL  
>>>>> configuration files.
>>>>> Just plain config files, nothing fancy except... options without  
>>>>> values.
>>>>>
>>>>> There is a number of options here that does not require a value,  
>>>>> they are
>>>>> basically just turning on a feature. They are also common in the  
>>>>> standard
>>>>> template files for the server. Options that are for mysqld can  
>>>>> have a value even
>>>>> though it is not required and the option file parser will not  
>>>>> complain, but for
>>>>> some of the client tools, values may not be given or there will  
>>>>> be a error.
>>>>>
>>>>> Looking at the tests of ConfigParser, I see that it is a  
>>>>> deliberate design
>>>>> decision to not allow options without values (or I am  
>>>>> misunderstanding
>>>>> something). Why?
>>>>>
>>>>>
>>>> Who knows. =) Module is old enough it's quite possible no one
>>>> remembers why.
>>>>
>>>
>>> Good point.  There's also an awful shortage of config parsers in  
>>> the standard
>>> library; we should add one or two.
>>>
>>> Georg
>>>
>>>
>> I happen to know a good one we could add. ;-)
>>
>> Michael
>>
>
> +1 I eagerly await a PEP

From fuzzyman at gmail.com  Fri Sep 18 02:14:29 2009
From: fuzzyman at gmail.com (Michael)
Date: Fri, 18 Sep 2009 01:14:29 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <1253232676.5889.28.camel@localhost>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <1253232676.5889.28.camel@localhost>
Message-ID: <2BDE5C13-D90A-401A-8B6A-ECF2A5E91467@gmail.com>

Adding a network attribute is easy. Plus I should have added that the  
core parser in ConfigObj (using and abusing regexes) is pretty hairy  
and I try to avoid touching it. :-)

The rest of the code is basically fine (working on a new release now  
~25% perf improvement thanks to Christian Heimes) although the module  
is waay too long. Doesn't take much maintenance though.

Will add the network attribute as an undocumented new feature in  
honour of Antoine...

Michael


--
http://www.ironpythoninaction.com

On 18 Sep 2009, at 01:11, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le jeudi 17 septembre 2009 ? 23:58 +0200, Georg Brandl a ?crit :
>>>
>>> Who knows. =) Module is old enough it's quite possible no one
>>> remembers why.
>>
>> Good point.  There's also an awful shortage of config parsers in  
>> the standard
>> library; we should add one or two.
>
> If so, can it have a .network attribute?
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig

From jnoller at gmail.com  Fri Sep 18 02:20:44 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Sep 2009 20:20:44 -0400
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <2015A3C0-8E2B-4575-93C7-60B3E408EF4C@voidspace.org.uk>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <4AB2C783.3040105@voidspace.org.uk>
	<CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>
	<2015A3C0-8E2B-4575-93C7-60B3E408EF4C@voidspace.org.uk>
Message-ID: <14EC63F4-F22C-4711-A618-E5850C48D395@gmail.com>

I was being ever slightly tongue in cheek, but I for one would like to  
see bits of configobj get into configparser, having used the latter  
several times until I discovered the magical world of json/YAML

On Sep 17, 2009, at 8:09 PM, Michael Foord <michael at voidspace.org.uk>  
wrote:

>
> Sorry to top-post, sent from phone-like-device.
>
> Actually Guido has shot down the inclusion of ConfigObj on the past,  
> so it doesn't seem worth a PEP.
>
> He doesn't like that sections inherit from dict (although they are  
> functionally key-value mappings) as it gives them an over-wide API.  
> This is fair enough.
>
> He also fundamentally disliked the idea of nested sections, doesn't  
> see the need and suggested people should be using XML for this kind  
> of use case! It is an often requested feature, so this I thought was  
> odd. Anyway, so be it.
>
> Michael
>
> --
> http://www.ironpythoninaction.com
>
> On 18 Sep 2009, at 00:59, Jesse Noller <jnoller at gmail.com> wrote:
>
>>
>>
>> On Sep 17, 2009, at 7:34 PM, Michael Foord  
>> <michael at voidspace.org.uk> wrote:
>>
>>> Georg Brandl wrote:
>>>> Brett Cannon schrieb:
>>>>
>>>>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am currently using the ConfigParser module to parse MySQL  
>>>>>> configuration files.
>>>>>> Just plain config files, nothing fancy except... options  
>>>>>> without values.
>>>>>>
>>>>>> There is a number of options here that does not require a  
>>>>>> value, they are
>>>>>> basically just turning on a feature. They are also common in  
>>>>>> the standard
>>>>>> template files for the server. Options that are for mysqld can  
>>>>>> have a value even
>>>>>> though it is not required and the option file parser will not  
>>>>>> complain, but for
>>>>>> some of the client tools, values may not be given or there will  
>>>>>> be a error.
>>>>>>
>>>>>> Looking at the tests of ConfigParser, I see that it is a  
>>>>>> deliberate design
>>>>>> decision to not allow options without values (or I am  
>>>>>> misunderstanding
>>>>>> something). Why?
>>>>>>
>>>>>>
>>>>> Who knows. =) Module is old enough it's quite possible no one
>>>>> remembers why.
>>>>>
>>>>
>>>> Good point.  There's also an awful shortage of config parsers in  
>>>> the standard
>>>> library; we should add one or two.
>>>>
>>>> Georg
>>>>
>>>>
>>> I happen to know a good one we could add. ;-)
>>>
>>> Michael
>>>
>>
>> +1 I eagerly await a PEP

From michael at voidspace.org.uk  Fri Sep 18 02:30:04 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Fri, 18 Sep 2009 01:30:04 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <14EC63F4-F22C-4711-A618-E5850C48D395@gmail.com>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <4AB2C783.3040105@voidspace.org.uk>
	<CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>
	<2015A3C0-8E2B-4575-93C7-60B3E408EF4C@voidspace.org.uk>
	<14EC63F4-F22C-4711-A618-E5850C48D395@gmail.com>
Message-ID: <7E15303D-33E6-4B52-91E5-09953851B59A@voidspace.org.uk>

Yeah - I just never miss an opportunity to promote ConfigObj. :-)

The nice thing about it is that the basic API is *so* much nicer than  
ConfigParser. Parts of that would be easy to get on and other parts  
would require major surgery and result on a hybrid there's-now-two- 
ways-to-do-it API. ConfigObj is bizarrely widely used so I'm happy  
with it as a third party module. Fact of the day though, the first  
version (about five years ago now) was only written because I didn't  
know ConfigParser existed...

Michael


--
http://www.ironpythoninaction.com

On 18 Sep 2009, at 01:20, Jesse Noller <jnoller at gmail.com> wrote:

> I was being ever slightly tongue in cheek, but I for one would like  
> to see bits of configobj get into configparser, having used the  
> latter several times until I discovered the magical world of json/YAML
>
> On Sep 17, 2009, at 8:09 PM, Michael Foord  
> <michael at voidspace.org.uk> wrote:
>
>>
>> Sorry to top-post, sent from phone-like-device.
>>
>> Actually Guido has shot down the inclusion of ConfigObj on the  
>> past, so it doesn't seem worth a PEP.
>>
>> He doesn't like that sections inherit from dict (although they are  
>> functionally key-value mappings) as it gives them an over-wide API.  
>> This is fair enough.
>>
>> He also fundamentally disliked the idea of nested sections, doesn't  
>> see the need and suggested people should be using XML for this kind  
>> of use case! It is an often requested feature, so this I thought  
>> was odd. Anyway, so be it.
>>
>> Michael
>>
>> --
>> http://www.ironpythoninaction.com
>>
>> On 18 Sep 2009, at 00:59, Jesse Noller <jnoller at gmail.com> wrote:
>>
>>>
>>>
>>> On Sep 17, 2009, at 7:34 PM, Michael Foord  
>>> <michael at voidspace.org.uk> wrote:
>>>
>>>> Georg Brandl wrote:
>>>>> Brett Cannon schrieb:
>>>>>
>>>>>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I am currently using the ConfigParser module to parse MySQL  
>>>>>>> configuration files.
>>>>>>> Just plain config files, nothing fancy except... options  
>>>>>>> without values.
>>>>>>>
>>>>>>> There is a number of options here that does not require a  
>>>>>>> value, they are
>>>>>>> basically just turning on a feature. They are also common in  
>>>>>>> the standard
>>>>>>> template files for the server. Options that are for mysqld can  
>>>>>>> have a value even
>>>>>>> though it is not required and the option file parser will not  
>>>>>>> complain, but for
>>>>>>> some of the client tools, values may not be given or there  
>>>>>>> will be a error.
>>>>>>>
>>>>>>> Looking at the tests of ConfigParser, I see that it is a  
>>>>>>> deliberate design
>>>>>>> decision to not allow options without values (or I am  
>>>>>>> misunderstanding
>>>>>>> something). Why?
>>>>>>>
>>>>>>>
>>>>>> Who knows. =) Module is old enough it's quite possible no one
>>>>>> remembers why.
>>>>>>
>>>>>
>>>>> Good point.  There's also an awful shortage of config parsers in  
>>>>> the standard
>>>>> library; we should add one or two.
>>>>>
>>>>> Georg
>>>>>
>>>>>
>>>> I happen to know a good one we could add. ;-)
>>>>
>>>> Michael
>>>>
>>>
>>> +1 I eagerly await a PEP

From michael at voidspace.org.uk  Fri Sep 18 02:30:23 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Fri, 18 Sep 2009 01:30:23 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <14EC63F4-F22C-4711-A618-E5850C48D395@gmail.com>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <4AB2C783.3040105@voidspace.org.uk>
	<CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>
	<2015A3C0-8E2B-4575-93C7-60B3E408EF4C@voidspace.org.uk>
	<14EC63F4-F22C-4711-A618-E5850C48D395@gmail.com>
Message-ID: <5DDDE6C2-5BF0-4612-BF16-DDB20038F846@voidspace.org.uk>

And YAML can suck it...


--
http://www.ironpythoninaction.com

On 18 Sep 2009, at 01:20, Jesse Noller <jnoller at gmail.com> wrote:

> I was being ever slightly tongue in cheek, but I for one would like  
> to see bits of configobj get into configparser, having used the  
> latter several times until I discovered the magical world of json/YAML
>
> On Sep 17, 2009, at 8:09 PM, Michael Foord  
> <michael at voidspace.org.uk> wrote:
>
>>
>> Sorry to top-post, sent from phone-like-device.
>>
>> Actually Guido has shot down the inclusion of ConfigObj on the  
>> past, so it doesn't seem worth a PEP.
>>
>> He doesn't like that sections inherit from dict (although they are  
>> functionally key-value mappings) as it gives them an over-wide API.  
>> This is fair enough.
>>
>> He also fundamentally disliked the idea of nested sections, doesn't  
>> see the need and suggested people should be using XML for this kind  
>> of use case! It is an often requested feature, so this I thought  
>> was odd. Anyway, so be it.
>>
>> Michael
>>
>> --
>> http://www.ironpythoninaction.com
>>
>> On 18 Sep 2009, at 00:59, Jesse Noller <jnoller at gmail.com> wrote:
>>
>>>
>>>
>>> On Sep 17, 2009, at 7:34 PM, Michael Foord  
>>> <michael at voidspace.org.uk> wrote:
>>>
>>>> Georg Brandl wrote:
>>>>> Brett Cannon schrieb:
>>>>>
>>>>>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I am currently using the ConfigParser module to parse MySQL  
>>>>>>> configuration files.
>>>>>>> Just plain config files, nothing fancy except... options  
>>>>>>> without values.
>>>>>>>
>>>>>>> There is a number of options here that does not require a  
>>>>>>> value, they are
>>>>>>> basically just turning on a feature. They are also common in  
>>>>>>> the standard
>>>>>>> template files for the server. Options that are for mysqld can  
>>>>>>> have a value even
>>>>>>> though it is not required and the option file parser will not  
>>>>>>> complain, but for
>>>>>>> some of the client tools, values may not be given or there  
>>>>>>> will be a error.
>>>>>>>
>>>>>>> Looking at the tests of ConfigParser, I see that it is a  
>>>>>>> deliberate design
>>>>>>> decision to not allow options without values (or I am  
>>>>>>> misunderstanding
>>>>>>> something). Why?
>>>>>>>
>>>>>>>
>>>>>> Who knows. =) Module is old enough it's quite possible no one
>>>>>> remembers why.
>>>>>>
>>>>>
>>>>> Good point.  There's also an awful shortage of config parsers in  
>>>>> the standard
>>>>> library; we should add one or two.
>>>>>
>>>>> Georg
>>>>>
>>>>>
>>>> I happen to know a good one we could add. ;-)
>>>>
>>>> Michael
>>>>
>>>
>>> +1 I eagerly await a PEP

From jnoller at gmail.com  Fri Sep 18 02:35:21 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Sep 2009 20:35:21 -0400
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <5DDDE6C2-5BF0-4612-BF16-DDB20038F846@voidspace.org.uk>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org> <4AB2C783.3040105@voidspace.org.uk>
	<CDB380E3-FF2E-4DEC-8E87-C75E9CB51C3E@gmail.com>
	<2015A3C0-8E2B-4575-93C7-60B3E408EF4C@voidspace.org.uk>
	<14EC63F4-F22C-4711-A618-E5850C48D395@gmail.com>
	<5DDDE6C2-5BF0-4612-BF16-DDB20038F846@voidspace.org.uk>
Message-ID: <B355A603-4794-44BE-BBD2-7B5E9F9D5D88@gmail.com>

Go back to your Visual studio and XML there beard boy!

On Sep 17, 2009, at 8:30 PM, Michael Foord <michael at voidspace.org.uk>  
wrote:

> And YAML can suck it...
>
>
> --
> http://www.ironpythoninaction.com
>
> On 18 Sep 2009, at 01:20, Jesse Noller <jnoller at gmail.com> wrote:
>
>> I was being ever slightly tongue in cheek, but I for one would like  
>> to see bits of configobj get into configparser, having used the  
>> latter several times until I discovered the magical world of json/ 
>> YAML
>>
>> On Sep 17, 2009, at 8:09 PM, Michael Foord  
>> <michael at voidspace.org.uk> wrote:
>>
>>>
>>> Sorry to top-post, sent from phone-like-device.
>>>
>>> Actually Guido has shot down the inclusion of ConfigObj on the  
>>> past, so it doesn't seem worth a PEP.
>>>
>>> He doesn't like that sections inherit from dict (although they are  
>>> functionally key-value mappings) as it gives them an over-wide  
>>> API. This is fair enough.
>>>
>>> He also fundamentally disliked the idea of nested sections,  
>>> doesn't see the need and suggested people should be using XML for  
>>> this kind of use case! It is an often requested feature, so this I  
>>> thought was odd. Anyway, so be it.
>>>
>>> Michael
>>>
>>> --
>>> http://www.ironpythoninaction.com
>>>
>>> On 18 Sep 2009, at 00:59, Jesse Noller <jnoller at gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Sep 17, 2009, at 7:34 PM, Michael Foord <michael at voidspace.org.uk 
>>>> > wrote:
>>>>
>>>>> Georg Brandl wrote:
>>>>>> Brett Cannon schrieb:
>>>>>>
>>>>>>> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats at sun.com>  
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I am currently using the ConfigParser module to parse MySQL  
>>>>>>>> configuration files.
>>>>>>>> Just plain config files, nothing fancy except... options  
>>>>>>>> without values.
>>>>>>>>
>>>>>>>> There is a number of options here that does not require a  
>>>>>>>> value, they are
>>>>>>>> basically just turning on a feature. They are also common in  
>>>>>>>> the standard
>>>>>>>> template files for the server. Options that are for mysqld  
>>>>>>>> can have a value even
>>>>>>>> though it is not required and the option file parser will not  
>>>>>>>> complain, but for
>>>>>>>> some of the client tools, values may not be given or there  
>>>>>>>> will be a error.
>>>>>>>>
>>>>>>>> Looking at the tests of ConfigParser, I see that it is a  
>>>>>>>> deliberate design
>>>>>>>> decision to not allow options without values (or I am  
>>>>>>>> misunderstanding
>>>>>>>> something). Why?
>>>>>>>>
>>>>>>>>
>>>>>>> Who knows. =) Module is old enough it's quite possible no one
>>>>>>> remembers why.
>>>>>>>
>>>>>>
>>>>>> Good point.  There's also an awful shortage of config parsers  
>>>>>> in the standard
>>>>>> library; we should add one or two.
>>>>>>
>>>>>> Georg
>>>>>>
>>>>>>
>>>>> I happen to know a good one we could add. ;-)
>>>>>
>>>>> Michael
>>>>>
>>>>
>>>> +1 I eagerly await a PEP

From vinay_sajip at yahoo.co.uk  Fri Sep 18 10:18:44 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 08:18:44 +0000 (UTC)
Subject: [stdlib-sig] Evolving the Standard Library
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
Message-ID: <loom.20090918T101815-54@post.gmane.org>

Armin Ronacher <armin.ronacher at ...> writes:

> I agree that nobody did.  And there is a reason for it, and that reason
> is probably something that could be discussed in a separate thread too,
> because it affects a lof stuff that is currently in the standard libary.
> 
> I looked at the logging library and my point of view is that logging is
> "broken by design".  Eg: you can't fix it without breaking backwards
> compatibility.

It's easy for you to just say "it's broken by design", and that's only a more
polite way of saying "it sucks". It doesn'nt strike me as a basis for
constructive dialogue, unless you provide some more specifics.

> I'm sorry for the wait I was expressing my disagreement with design
> decisions made in the standard library and I would love to change that.
>  One of the biggest griefs I have with how the standard library works it
> that it does not work for me and everytime I discuss that topic with
> anyone else I immediately get the feedback that "this does not happen in
> the real world" or "works for me" etc.

Well, I'm happy to discuss any problems you are having with logging in a
practical sense. Obviously I can't do much with "I just don't like how it's
designed", but if you want to spell out a specific problem - something you want
to do with it that you just can't - I'll gladly listen and see if I can help
with it.

> I consider any kind of shared state a design mistake, no matter if it
> may work for some people or not, the logging package is no exception.
> However I have to admit that global loggers appears to be one of the
> easiest solutions for the problem.

I think you are being dogmatic, rather than pragmatic. The Zen of Python says,
"practicality beats purity." Your bugbear here seems to be how shared state
causes problems in web applications. Despite having shared state, AFAIK the
logging module is quite usable in a web context - as well as the usage with
Django and Tornado that I mentioned earlier, Google App Engine uses it too
(meaning, all the web applications developed with GAE can use it). So if it
doesn't work for you in a practical way, give me some details. But the logging
design isn't meant to be a candidate in a beauty contest, and I don't claim it's
perfect. You're a very smart guy, Armin, but you perhaps need to consider that
it is possible to not like a design because it doesn't suit your personal taste
- but that doesn't necessarily make it a bad design. 

> 
> If I would have to create my own logging library, I would go a total
> different part from design to implementation.  I would create a system
> where the "sender" is an arbitrary Python object and the system that
> handles it would have to check for its own if it may process that
> message.  Only *if* there was any handler that wants to do anything with
> that message, it would pull the details form the stack and format the

Currently, Python logging doesn't do formatting of stack traces etc. until it's
sure that the message is severe enough to require handling, based on the current
logger configuration. When handling a message, each handler checks its
configuration before formatting and dealing with the formatted message. The
system tries not to do unnecessary work, and if you have found some cases where
it does unnecessary work, please tell me. So at present, Python logging conforms
to your statement "Only if there was any handler that wants to do anything with
that message, it would pull the details form the stack and format ..."

That's been there from day one, so if you got the idea it worked differently,
I'm not sure why.

> message.  That also would get rid of the formatters and all the other
> stuff we currently have and avoids a global registry of loggers.

In Python logging, you never have to instantiate a Formatter in your code unless
you want some specific formatting functionality or format. In your scheme, if a
user wanted certain messages to be formatted in certain ways (and other messages
in other ways - e.g. for a log file as opposed to console display), you would do
this without any formatter classes - how, exactly?

The global registry of loggers is there to avoid the need to pass loggers around
the system - you just access them by name. It's a bit like thread locals - do
you have a problem with thread locals, too? Sure, it's state shared across
threads, unlike thread locals. But if it's causing you a specific problem
because you've found some non-thread-safe behaviour, I'd really like to know.

> How would I configure a logger in a library then?  I library would
> *never* by default print anything that is logged.  Default
> configurations for the logging system currently are the biggest reason
> for me to hate that library.  Many libraries do not even give you the
> ability to turn of logging in general, and documentation for the logging
> system was probably one of the reasons.  I don't know when the
> "NullHandler" example appeared in the docs, but I'm sure it was not
> there when I started using the logging system.

Agreed, the logging from a library leading to warning messages was an annoyance
in the library as first released. But *you* never raised an issue about it. When
other people did (I think it was Thomas Heller), I immediately updated the
documentation to explain how to use a NullHandler to avoid annoying messages,
and NullHandler appeared in a subsequent release soon afterwards.

If default configuration for the logging system is the biggest reason for you to
hate that package, don't get mad - get even. Tell me how I messed up and how to
make it better.

> 
> I cannot use logging for anything serious because it is slow, it has
> shared state that often causes frustration, you cannot delete loggers
> and much more.  Of course I do use logging for libraries because it's
> the standard and the best we have got.

Slow? Sure, everything has a cost. It's all about tradeoffs. What specific
performance problems have you come up against? What mitigating strategies did
you use? What were the observed performance metrics, as against what you
expected? Did you do any profiling to be sure that logging was definitely the
culprit? Show me the numbers.

If your frustration with shared state is from an aesthetic point of view, I
can't really help. After all, sys.modules (say) is shared state too. If the
frustration comes from some specific thing you're trying to do, please do tell
what that is.

You cannot delete loggers because in a multithreaded application, other threads
may still be using them. You can, however, disable loggers - which, from a
functional point of view, seems as good. Or are you finding that loggers are
taking up too much memory?

> The only reasons (in my opinion) that logging is still around is that
> it's in the stdlib, not because it's any good.  Especially when it comes

That's a cheap remark, I would have expected better from you. So everybody who
uses logging (and is smart, like you) hates it but uses it because they have no
choice, right? It *is* widely used, AFAIK. If everyone shared your opinion, I
don't think that would be the case.

> to highly optimized code in web applications you will quickly discover
> that half a dozen log calls take up more CPU cycles than the actual
> application code.
>

Please show me the numbers.

> I don't know how I could contribute constructive criticism.  I'm sorry
> for that.

You can't contribute constructive criticism because you don't know how? Well,
you've already made some criticisms in your post, so how about putting some in
more detail? I really like Jinja2 (a lot), but if someone said to you "Jinja2
sucks, it's really slow, it does all that conversion to bytecode stuff, that's
really gross. I can't really say any more than that, excuse me while I vomit"
then how would you feel? That's pretty much how you're coming across right now.

> > Am I expecting too much?
> I'm afraid you are.  It's hard to accept that some people think of your
> system as "it just sucks", but you can't change that.  I know that

Actually I have no problem with that - people thinking that it sucks. You can't
please everyone, and that goes for when you have a roomful of smart people, too.
The problem I have is when people just vent without trying to make it better,
talk about replacing it or removing it without even having the courtesy to talk
to me first. (I'm not talking here about what you post on your blog - those are
your opinions and of course you're entitled to say whatever you like. I'm
talking about this discussion and perhaps on python-dev and other other SIG
mailing lists.)

Perhaps you think I'm being oldskool by mentioning "courtesy", but I prefer to
think if it as an attribute of being a grown-up. Open source is very political,
but that doesn't mean it doesn't have to be polite.

> feeling from some of the stuff I wrote.  It's just a little bit worse
> for you because logging is the standard (and only) one, everybody uses.
>  For the stuff I wrote*, people have choice.  If they don't want to use
> it, they don't have to.

Actually they don't have to use Python's built-in logging either. Who's twisting
your arm? Talking of twisting, I recently had a discussion with Glyph Lefkowitz
of Twisted Matrix about twisted.log after he posted some of his thoughts on
logging in general and Python's implementation in particular. I believe I
rebutted, or at least addressed, all of his points. They will continue to use
twisted.log for aesthetic and pragmatic reasons, which is fine by me. They've
built a bridge to Python logging, presumably because some customers wanted that.

By the way, if you think that having choice has no downsides, perhaps you should
step back and take a look at the Python web application framework space -
Django, Pylons, TurboGears, web.py, Werkzeug, ... wow, so much choice! Sorry if
I missed one. Each one doing it a little differently, even for something as
basic as HTTP request/response objects. (I know the issues aren't trivial, but
are they rocket surgery? ;-) Look at Graham Dumpleton's frustration over WSGI
and Python 3.0 - I feel more than a little sympathy for his situation. He's
actually trying to create something useful, just as I am. There's no shortage of
approaches, and opinions, and in general, I believe competition is good. But
some of us are just trying to get some work done here.


From vinay_sajip at yahoo.co.uk  Fri Sep 18 10:31:29 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 08:31:29 +0000 (UTC)
Subject: [stdlib-sig] Evolving the Standard Library
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<D42F4700-0FF8-4C46-8269-BA845076201D@rcn.com>
Message-ID: <loom.20090918T101943-9@post.gmane.org>

Raymond Hettinger <python at ...> writes:

> While I'm no fan of the logging module, it is widely used.
> It was based on a Java version and Guido blessed it early-on.

Raymond, I took *some ideas* from log4j. In that sense it was "based on", but it
is not a port. While Guido did bless it, I do believe it went through a
reasonable review process on python-dev (where you could certainly have given
some feedback - I don't remember that you did) and I changed the package in
response to various concerns from various people. It certainly didn't feel to me
like a rubber-stamping exercise - do you feel that it was?

Logging's current design is based on the premise that logging is concerned with
"What happened?", "Where did it happen?", "How important is it?" and "Who wants
to know?", and this is completely general and not tied to any language or
environment. Of course there are many designs which could be developed from such
a premise, and the present design is just one such. The abstractions in log4j
(such as a hierarchical namespace for loggers, and handlers as orthogonal to
loggers, whereas many systems conflate them) made sense to me (and to a lot of
others), and it's no surprise that I named my abstractions similarly where that
made sense. Beyond that, there's little correspondence between log4j's code and
the code in Python's logging.

I'm sorry you're not a fan. Is it an aesthetic thing, or have you had specific
problems?


From solipsis at pitrou.net  Fri Sep 18 10:40:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Sep 2009 10:40:46 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T101815-54@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
Message-ID: <1253263246.5710.6.camel@localhost>

Le vendredi 18 septembre 2009 ? 08:18 +0000, Vinay Sajip a ?crit :
> Despite having shared state, AFAIK the
> logging module is quite usable in a web context - as well as the usage with
> Django and Tornado that I mentioned earlier, Google App Engine uses it too
> (meaning, all the web applications developed with GAE can use it).

Pylons uses it too.

While logging may not look extremely pretty, I have yet to see another
logging library that has a pretty *and* powerful API. Applications /
libraries which reinvent their own logging API usually end up with
something which is both less powerful and *not* prettier (see
twisted.log for an example).

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Fri Sep 18 10:43:59 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 08:43:59 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
Message-ID: <loom.20090918T104047-293@post.gmane.org>

Antoine Pitrou <solipsis at ...> writes:


> Pylons uses it too.
> 
> While logging may not look extremely pretty, I have yet to see another
> logging library that has a pretty *and* powerful API. Applications /
> libraries which reinvent their own logging API usually end up with
> something which is both less powerful and *not* prettier (see
> twisted.log for an example).
> 

Thanks. If you had the time to write your ideal, "pretty" API - hypothetically,
say, to wrap logging so you wouldn't use the underlying power - what would this
API look like? I'm open to ideas from all of you.


From ziade.tarek at gmail.com  Fri Sep 18 10:54:00 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 18 Sep 2009 10:54:00 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T104047-293@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
Message-ID: <94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>

On Fri, Sep 18, 2009 at 10:43 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Antoine Pitrou <solipsis at ...> writes:
>
>
>> Pylons uses it too.
>>
>> While logging may not look extremely pretty, I have yet to see another
>> logging library that has a pretty *and* powerful API. Applications /
>> libraries which reinvent their own logging API usually end up with
>> something which is both less powerful and *not* prettier (see
>> twisted.log for an example).
>>
>
> Thanks. If you had the time to write your ideal, "pretty" API - hypothetically,
> say, to wrap logging so you wouldn't use the underlying power - what would this
> API look like? I'm open to ideas from all of you.

I think logging is fine, but it misses a few pythonic functions on the
top of it to work with.

Right now, if you want to set up a logging output on a file or on
stdout with some options,
you have to write 5 or 6 lines of code.

These 5/6 lines could probably be put in a function in the logging module,
and be used with a few arguments. This function would return a logger
ready to be used.

Once a logger is created, logging is dead simple to use.

If you think it's a good idea I can try to work on a proposal for this
function signature.


>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>



-- 
Tarek Ziad? | http://ziade.org |????????????!

From ziade.tarek at gmail.com  Fri Sep 18 11:18:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 18 Sep 2009 11:18:30 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <486C2B71-15EA-4CC8-974C-C8CB2BC11EAE@masklinn.net>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
	<486C2B71-15EA-4CC8-974C-C8CB2BC11EAE@masklinn.net>
Message-ID: <94bdd2610909180218y5e9793d2i8883b110b33380e9@mail.gmail.com>

On Fri, Sep 18, 2009 at 11:08 AM, Masklinn <masklinn at masklinn.net> wrote:

>
> Isn't that what logging.basicConfig does?
>
> Allows you to log to either a file or a custom stream, specify your logging
> format, and sets up the logging level in a single call.

Except that it will only configure the root logger. I was thinking about an api
that would return a new logger (or reconfigure one that exists and return it)

Notice that looking at this function code, it wouldn't be too far from
what I would like to have. e.g:

create_logger(name, level=logging.INFO, stream=sys.stdout,
filename=None) -> logger

From vinay_sajip at yahoo.co.uk  Fri Sep 18 11:18:35 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 09:18:35 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
Message-ID: <loom.20090918T105659-185@post.gmane.org>

Tarek Ziad? <ziade.tarek at ...> writes:

> I think logging is fine, but it misses a few pythonic functions on the
> top of it to work with.
> 
> Right now, if you want to set up a logging output on a file or on
> stdout with some options,
> you have to write 5 or 6 lines of code.

There's basicConfig, which can be used like

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(levelname)s %(message)s',
                    filename='/tmp/myapp.log',
                    filemode='w')

for a file and

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(levelname)s %(message)s',
                    stream=sys.stdout)

for stdout. This configures the root logger, but you could obviously add a
loggerName argument that configured and returned a configured logger with that
name. That wouldn't even break backward compatibility.

It's not in general a desirable pattern to have handlers associated with every
logger - which is why I haven't provided that additional argument. It's more
common to attach handlers at the root and at certain specific points in the
hierarchy - for example, attach an SMTP logger to the root module for a
subsystem so that emails about errors can be sent to the team looking after that
subsystem.

Of course it's perfectly valid to have handers attached to multiple loggers -
but if you do that for lots of loggers, you get multiple messages and increased
processing time. If that's what is wanted, fine - it's just not the norm. But
having a convenience function which makes it too convenient to configure
multiple handlers could lead to lots of "I'm getting messages multiple times,
please help!" traffic on c.l.py.

The basic premise is - loggers map to areas in the application ("Where did it
happen?") and handlers to the audience ("Who wants to know?"). Apart from in
scripts intended to be run from the command line, in general you don't find a
one-to-one mapping between loggers and handlers.



From vinay_sajip at yahoo.co.uk  Fri Sep 18 11:20:18 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 09:20:18 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
	<486C2B71-15EA-4CC8-974C-C8CB2BC11EAE@masklinn.net>
	<94bdd2610909180218y5e9793d2i8883b110b33380e9@mail.gmail.com>
Message-ID: <loom.20090918T111941-994@post.gmane.org>

Tarek Ziad? <ziade.tarek at ...> writes:

> 
> On Fri, Sep 18, 2009 at 11:08 AM, Masklinn <masklinn at ...> wrote:
> 
> >
> > Isn't that what logging.basicConfig does?
> >

Tarek, see my answer to your other post.




From ziade.tarek at gmail.com  Fri Sep 18 11:28:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 18 Sep 2009 11:28:11 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T105659-185@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
	<loom.20090918T105659-185@post.gmane.org>
Message-ID: <94bdd2610909180228y1f3b6a9av370edeb2e11eabdd@mail.gmail.com>

On Fri, Sep 18, 2009 at 11:18 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> but you could obviously add a
> loggerName argument that configured and returned a configured logger with that
> name. That wouldn't even break backward compatibility.

That would be great. With an option to make the logger instance 'standalone'
(e.g. blocking the  propagation of the messages to the other handler
to avoid the problem
you've described below)

> It's not in general a desirable pattern to have handlers associated with every
> logger - which is why I haven't provided that additional argument. It's more
> common to attach handlers at the root and at certain specific points in the
> hierarchy - for example, attach an SMTP logger to the root module for a
> subsystem so that emails about errors can be sent to the team looking after that
> subsystem.
>
> Of course it's perfectly valid to have handers attached to multiple loggers -
> but if you do that for lots of loggers, you get multiple messages and increased
> processing time. If that's what is wanted, fine - it's just not the norm. But
> having a convenience function which makes it too convenient to configure
> multiple handlers could lead to lots of "I'm getting messages multiple times,
> please help!" traffic on c.l.py.
>
> The basic premise is - loggers map to areas in the application ("Where did it
> happen?") and handlers to the audience ("Who wants to know?"). Apart from in
> scripts intended to be run from the command line, in general you don't find a
> one-to-one mapping between loggers and handlers.
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>



-- 
Tarek Ziad? | http://ziade.org |????????????!

From vinay_sajip at yahoo.co.uk  Fri Sep 18 11:25:46 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 09:25:46 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
Message-ID: <loom.20090918T112521-679@post.gmane.org>

Vinay Sajip <vinay_sajip at ...> writes:


> say, to wrap logging so you wouldn't use the underlying power - what would this

s/use/lose/

Sorry.




From vinay_sajip at yahoo.co.uk  Fri Sep 18 11:52:00 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 09:52:00 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
	<loom.20090918T105659-185@post.gmane.org>
	<94bdd2610909180228y1f3b6a9av370edeb2e11eabdd@mail.gmail.com>
Message-ID: <loom.20090918T114806-502@post.gmane.org>

Tarek Ziad? <ziade.tarek at ...> writes:

> That would be great. With an option to make the logger instance 'standalone'
> (e.g. blocking the  propagation of the messages to the other handler
> to avoid the problem
> you've described below)
> 

That could work. The additional keyword argument would be propagate, defaulting
to None, and (if not specified) internally set as True if no logger name is
provided, or False if it is. Let's just wait to see if any other ideas pop up.

Regards,

Vinay Sajip


From john at szakmeister.net  Fri Sep 18 12:11:39 2009
From: john at szakmeister.net (John Szakmeister)
Date: Fri, 18 Sep 2009 06:11:39 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<94bdd2610909180154y50d25f90w141cfc8bcc3b5b4@mail.gmail.com>
Message-ID: <a1c1f4750909180311u6faedc1rbded007fd670cc46@mail.gmail.com>

On Fri, Sep 18, 2009 at 4:54 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Fri, Sep 18, 2009 at 10:43 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> Antoine Pitrou <solipsis at ...> writes:
>>
>>
>>> Pylons uses it too.
>>>
>>> While logging may not look extremely pretty, I have yet to see another
>>> logging library that has a pretty *and* powerful API. Applications /
>>> libraries which reinvent their own logging API usually end up with
>>> something which is both less powerful and *not* prettier (see
>>> twisted.log for an example).
>>>
>>
>> Thanks. If you had the time to write your ideal, "pretty" API - hypothetically,
>> say, to wrap logging so you wouldn't use the underlying power - what would this
>> API look like? I'm open to ideas from all of you.
>
> I think logging is fine, but it misses a few pythonic functions on the
> top of it to work with.
>
> Right now, if you want to set up a logging output on a file or on
> stdout with some options,
> you have to write 5 or 6 lines of code.

That would be exactly my complaint.  It feels like I'm writing Java
instead of Python. :-)

FWIW, logging has been quite flexible for my needs.  Just getting it
all configured is a bit of work.

> These 5/6 lines could probably be put in a function in the logging module,
> and be used with a few arguments. This function would return a logger
> ready to be used.

That would definitely be an improvement!

Now back to lurk mode...

-John

From armin.ronacher at active-4.com  Fri Sep 18 13:20:41 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Fri, 18 Sep 2009 13:20:41 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <loom.20090918T101815-54@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
Message-ID: <4AB36D09.4040701@active-4.com>

Hi,

Vinay Sajip schrieb:
> It's easy for you to just say "it's broken by design", and that's only a more
> polite way of saying "it sucks". It doesn'nt strike me as a basis for
> constructive dialogue, unless you provide some more specifics.
Of course it is easier.

> I think you are being dogmatic, rather than pragmatic. The Zen of Python says,
> "practicality beats purity." Your bugbear here seems to be how shared state
> causes problems in web applications. Despite having shared state, AFAIK the
> logging module is quite usable in a web context - as well as the usage with
> Django and Tornado that I mentioned earlier, Google App Engine uses it too
> (meaning, all the web applications developed with GAE can use it).
Django has a global settings module and yet there are tons of developers
hating that one.  Even Simon Willison agrees that the settings module
was the biggest mistake made in Django.

> You're a very smart guy, Armin, but you perhaps need to consider that
> it is possible to not like a design because it doesn't suit your personal taste
> - but that doesn't necessarily make it a bad design. 
That does not have anything to do with personal taste or not.  Some
things cannot work in some situations, and logging is currently one of
them.  Logging was designed to be based on persistent loggers.  You get
one, it's there, you can log into it.  SQLAlchemy for example does an
incredible dance to get separate loggers for temporary database connections.

> Currently, Python logging doesn't do formatting of stack traces etc. until it's
> sure that the message is severe enough to require handling, based on the current
> logger configuration.
That is actually something where I had a small problem with the logging
implementation that could be fixed: the exc_text is currently only set
in the format() function of the formatter, instead of being an attribute
on the log record.  And yes, it could be an attribute of the log record
if it was a property.  (I guess the reason it isn't currently is that
the logging module is backwards compatible down to 1.5.2 or something)

> So at present, Python logging conforms to your statement "Only if there
> was any handler that wants to do anything with that message, it would
> pull the details form the stack and format ..."
My point was that there are no loggers, no registry of loggers, just
senders and senders are arbitrary Python objects.

> In Python logging, you never have to instantiate a Formatter in your code unless
> you want some specific formatting functionality or format. In your scheme, if a
> user wanted certain messages to be formatted in certain ways (and other messages
> in other ways - e.g. for a log file as opposed to console display), you would do
> this without any formatter classes - how, exactly?
In my personal experience the formatting is based on the handler.  For
example if I want to log into a text file, i have one formatting, if I
log to stderr I have a different one.  If I log into an email I have a
completely different one.  For example I tried a long ago to log my
exceptions from a logging handler into a new ticket in the trac.  I
ended up replacing a log of code from both log record and formatter
inside the log handler because of the way the message was assembled.  I
had to escape text and could not rely on string formatting to work.

> The global registry of loggers is there to avoid the need to pass loggers around
> the system - you just access them by name. It's a bit like thread locals - do
> you have a problem with thread locals, too?
I have a problem with unnecessarily used thread locals.  Not with the
concept in general.

> Sure, it's state shared across threads, unlike thread locals. But if
> it's causing you a specific problem because you've found some
> non-thread-safe behaviour, I'd really like to know.
I don't complain that it's not threadsafe, I'm pretty sure if logging
was thread unsafe someone would have noticed by now.

> Agreed, the logging from a library leading to warning messages was an annoyance
> in the library as first released. But *you* never raised an issue about it.
That is true and I have to change that.

> If default configuration for the logging system is the biggest reason for you to
> hate that package, don't get mad - get even. Tell me how I messed up and how to
> make it better.
If I would know how to improve it so that I'm happy with it, I would
have told you.

> Slow? Sure, everything has a cost. It's all about tradeoffs. What specific
> performance problems have you come up against? What mitigating strategies did
> you use? What were the observed performance metrics, as against what you
> expected? Did you do any profiling to be sure that logging was definitely the
> culprit? Show me the numbers.


> If your frustration with shared state is from an aesthetic point of view, I
> can't really help. After all, sys.modules (say) is shared state too.
http://lucumr.pocoo.org/2009/7/24/singletons-and-their-problems-in-python
*cough*

> You cannot delete loggers because in a multithreaded application, other threads
> may still be using them. You can, however, disable loggers - which, from a
> functional point of view, seems as good.
Which is why I would not design a logging library based on loggers.

> Or are you finding that loggers are taking up too much memory?
Even if a logger would be just 8 bytes in size, it would steal leak if
you cannot control the number of loggers created.  (see SQLAlchemy for a
nice example).

> That's a cheap remark, I would have expected better from you. So everybody who
> uses logging (and is smart, like you) hates it but uses it because they have no
> choice, right? It *is* widely used, AFAIK. If everyone shared your opinion, I
> don't think that would be the case.
A while ago when I was blogging about logging I wrote this:

> [...] It's called "logging" and does exactly that --- it logs errors.
> I don't know why so many people miss it or just don't use it, but it's
> really one of the good things in the python standard library.

to which Robert Brewer replied:

> That's also why it's so bad: it's so extensible and configurable
> that's it's far too slow for high-performance websites.

And I'm pretty sure Rober knows what he's talking about.

> You can't contribute constructive criticism because you don't know how? Well,
> you've already made some criticisms in your post, so how about putting some in
> more detail? I really like Jinja2 (a lot), but if someone said to you "Jinja2
> sucks, it's really slow, it does all that conversion to bytecode stuff, that's
> really gross. I can't really say any more than that, excuse me while I vomit"
> then how would you feel? That's pretty much how you're coming across right now.
I gave up on defending Jinja2 a while ago.  Because people from the
Django world constantly call me names for enabling logic in templates ;)

I'm sorry for the way I expressed my disagreement with the library's
design here and in what way.  To make you feel better: from all the
modules in the standard library, logging is still one of the best
designed and implemented, despite my disagreement with it.

> Perhaps you think I'm being oldskool by mentioning "courtesy", but I prefer to
> think if it as an attribute of being a grown-up. Open source is very political,
> but that doesn't mean it doesn't have to be polite.
I agree.

> Look at Graham Dumpleton's frustration over WSGI
> and Python 3.0 - I feel more than a little sympathy for his situation. He's
> actually trying to create something useful, just as I am. There's no shortage of
> approaches, and opinions, and in general, I believe competition is good. But
> some of us are just trying to get some work done here.
and WSGI for Python 3 is something that has to be discussed.  Graham's
master plan is one proposal, but in my opinion not the best one.


Regards,
Armi

From barry at python.org  Fri Sep 18 13:57:03 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 18 Sep 2009 07:57:03 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T104047-293@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
Message-ID: <BD98B237-A3BA-44C5-8BA9-28305CD56159@python.org>

On Sep 18, 2009, at 4:43 AM, Vinay Sajip wrote:

> Thanks. If you had the time to write your ideal, "pretty" API -  
> hypothetically,
> say, to wrap logging so you wouldn't use the underlying power - what  
> would this
> API look like? I'm open to ideas from all of you.

So, I'm a big fan of the logging package and thank Vinay for his work  
on it over the years.  It was quite a joy to chuck all the hacky  
Mailman 2 logging crap in favor of the standard logging package for MM3.

The one thing I (very) occasionally want is to ask a logger for a file- 
like object suitable for print.  There are some situations where I  
have a 3rd party API that requires a file-like object to output to,  
but I really want that output to go to a log.  I'm pretty sure I've  
wrangled it out of a file-based logger, but it would be nice have this  
as an official API.  Maybe it's there and I've just missed it though.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090918/f093929b/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Fri Sep 18 15:37:25 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 13:37:25 +0000 (UTC)
Subject: [stdlib-sig] Evolving the Standard Library
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<4AB36D09.4040701@active-4.com>
Message-ID: <loom.20090918T141927-390@post.gmane.org>

Armin Ronacher <armin.ronacher at ...> writes:

> Django has a global settings module and yet there are tons of developers
> hating that one.  Even Simon Willison agrees that the settings module
> was the biggest mistake made in Django.

No system is perfect, and even systems that are good to start with improve
over time. I don't say that every use of shared state is justified. There's no
connection I can see between haters of Django's global settings module and
problems with the logging package. Care to stop waving your arms about over
general principles like "shared state", and get down to specifics about
logging?

> That does not have anything to do with personal taste or not.  Some
> things cannot work in some situations, and logging is currently one of
> them.  Logging was designed to be based on persistent loggers.  You get

You're still talking in very general terms - "some things cannot work in some
situations" - that could really be said about most APIs. What things? What
situations? I'm not saying we have to do this on this thread, else it might
become off-topic - just that it should be done somewhere.

> one, it's there, you can log into it.  SQLAlchemy for example does an
> incredible dance to get separate loggers for temporary database connections.

Neither Mike Bayer nor anyone else on the SQLAlchemy team have raised this as
an issue, AFAIK. Mike, if you or other SQLAlchemy committers are reading
this, please get in touch. I'm not sure why they'd need to get separate
loggers for temporary database connections - logging has mechanisms to log to
the same logger while separating out different contextual information for
transient entities (for example, database or socket connections).

> That is actually something where I had a small problem with the logging
> implementation that could be fixed: the exc_text is currently only set
> in the format() function of the formatter, instead of being an attribute
> on the log record.  And yes, it could be an attribute of the log record
> if it was a property.  (I guess the reason it isn't currently is that
> the logging module is backwards compatible down to 1.5.2 or something)

It *is* an attribute of the LogRecord, and always has been - not sure where
you get your "facts" ;-). The reason it is not set in the constructor is to
avoid doing unnecessary work - something else you complained about. The
reason the exc_text is set in the formatter is this: the logger receives
the event/message, conditionally passes it on to handlers (i.e. only when
necessary), the handlers format and send out the message (again, only when
necessary). The exc_text is stored as an attribute of the LogRecord, but only
computed when the exception is formatted for output by the overridable
formatException method of the Formatter. If I hadn't done it that way, you
might well have said, "Bah! There's no way to control how an exception gets
formatted. Logging's not reusable." Once computed, it's stored there so that
later handling operations don't need to compute it again (again, to save
unnecessary work).

> My point was that there are no loggers, no registry of loggers, just
> senders and senders are arbitrary Python objects.

The point of having logger names is to control, by easy configuration,
logging verbosity at different levels in your application. Different levels
imply a hierarchy. A dotted-namespace is a reasonable way of codifying a
hierarchy - Java, Python and many other systems use it. If you don't want to
pass loggers around between function calls (and in case you say a logger
could be an instance attribute, remember logging works for non-OOP
function-based scripts, too), then you need some way to get hold of a
certain logger from anywhere in your application. A registry provides that.
It means that at runtime, without bringing down a running application, it is
possible to change the verbosity of any logger.

If you want arbitrary objects to send notifications to arbitrary objects, you
can use something like pydispatcher. It works, and it fulfills a need, but
it's not, to my mind, a logging system. You've merely stated the germ of an
idea ("no loggers, no registry of loggers, just senders, and senders are
arbitrary Python objects"), but I see no evidence that you've thought it
through in terms of addressing a whole host of practical issues,
such as configuration.

> In my personal experience the formatting is based on the handler.  For
> example if I want to log into a text file, i have one formatting, if I
> log to stderr I have a different one.  If I log into an email I have a
> completely different one.  For example I tried a long ago to log my

Yes, that's how it is because the audiences are different. The end user may
not need a timestamp and probably will not want a stack trace.

> exceptions from a logging handler into a new ticket in the trac.  I
> ended up replacing a log of code from both log record and formatter
> inside the log handler because of the way the message was assembled. I
> had to escape text and could not rely on string formatting to work.

I don't know, without knowing your problem in more detail, if you could have
avoided copying and changing code from LogRecord and Formatter. Obviously
I've tried to provide enough hooks so that people can subclass and override
methods for specific requirements, such as adding to a Trac ticket. If you
describe the problem in more detail, I may be able to indicate a better
solution. If it turns out that I need to provide more hooks where people can
override functionality, I'm open to doing that.

> I have a problem with unnecessarily used thread locals.  Not with the
> concept in general.

That's okay then. I don't feel that the use of shared state in logging is
unnecessary, because of the benefits it confers. It was done in a thoughtful
way, not just because I don't know any better. But then, I would say that,
right?

> I don't complain that it's not threadsafe, I'm pretty sure if logging
> was thread unsafe someone would have noticed by now.

So your complaint is really just a philosophical diatribe against shared
state? 

> If I would know how to improve it so that I'm happy with it, I would
> have told you.

Then you're really saying something roughly like "It's Not Invented Here, so
I don't like it. The only way it could be better is if I had thought of it.
Period."

> > can't really help. After all, sys.modules (say) is shared state too.
> http://lucumr.pocoo.org/2009/7/24/singletons-and-their-problems-in-python
> *cough*

*cough* *cough*. I've already read that post, as I referred to it earlier in
this thread. Since sys.modules is shared state at a much more fundamental
level than logging's logger registry, why not focus your energies on getting
that changed first? If you're successful at pulling it off, it'll no doubt
lead to a whole slew of changes in the Python ecosystem, of which logging is
just a tiny part.

> > ... because in a multithreaded application, other threads
> > ... be using them. You can, however, disable loggers - which, from a
> > functional point of view, seems as good.
> Which is why I would not design a logging library based on loggers.

You didn't really answer the point that you don't need to delete loggers,
since disabling them is just as good.

> > Or are you finding that loggers are taking up too much memory?
> Even if a logger would be just 8 bytes in size, it would steal leak if
> you cannot control the number of loggers created.  (see SQLAlchemy for a
> nice example).

You (as the application developer or library developer) *can* control it,
because you decide exactly which loggers get created. I'll ask Mike Bayer
about the specifics of the SQLAlchemy issue.

> A while ago when I was blogging about logging I wrote this:
> 
> > [...] It's called "logging" and does exactly that --- it logs errors.
> > I don't know why so many people miss it or just don't use it, but it's
> > really one of the good things in the python standard library.
> 
> to which Robert Brewer replied:
> 
> > That's also why it's so bad: it's so extensible and configurable
> > that's it's far too slow for high-performance websites.
> 
> And I'm pretty sure Rober knows what he's talking about.
>

No doubt he does. Logging's extensible and configurable because it has to
work for *lots* of different scenarios - not *just* high-performance web
sites. Obviously, that generality involves tradeoffs which may make the
logging package less suitable for high-performance scenarios. Did Robert
confide in what they *do* use for logging functionality?

Robert - if you're reading, do you have numbers? Obviously *any* logging
will involve *some* penalty. I'll be very happy if someone says, "we profiled
the application with logging enabled, and we found that there was a penalty
of X%. That's hard for us, but we can live with an overhead of Y%. Here are
the profiler reports, can you do anything to improve it?" I might *not* be
able to make much, if any, improvement, but I'd certainly take a shot at it.

> I gave up on defending Jinja2 a while ago.  Because people from the
> Django world constantly call me names for enabling logic in templates ;)
>

There you go. Is Jinja2 "broken by design", then?

Aso, I prefer Python to Ruby. Should I say "Ruby? Broken, by design"?

> I'm sorry for the way I expressed my disagreement with the library's
> design here and in what way.  To make you feel better: from all the
> modules in the standard library, logging is still one of the best
> designed and implemented, despite my disagreement with it.

What, sarcasm now? From "broken by design" to "paragon of virtue, exemplar
to the world"? Please. I've been around the block a few times, no longer the
new kid, and my skin is reasonably thick. I'm not crying. But keep it
constructive (i.e. with improvement as a goal), that's all I'm asking.

> and WSGI for Python 3 is something that has to be discussed.  Graham's
> master plan is one proposal, but in my opinion not the best one.

My point was not about whether Graham's plan was or was not the best. It's
that he wants to get *something* reasonable out there, knows the issues and
is frustrated with the constant back-and-forth between conflicting opinions.
We could be waiting forever for a resolution, what good is that to anybody?

Remember Voltaire: "Le mieux est l'ennemi du bien." (The best is the enemy of
the good.) Am I showing my age, or what? ;-)

Regards,

Vinay Sajip




From vinay_sajip at yahoo.co.uk  Fri Sep 18 15:54:36 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 13:54:36 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<BD98B237-A3BA-44C5-8BA9-28305CD56159@python.org>
Message-ID: <loom.20090918T153911-846@post.gmane.org>

Barry Warsaw <barry at ...> writes:

> 
> So, I'm a big fan of the logging package and thank Vinay for his work  
> on it over the years.  It was quite a joy to chuck all the hacky  
> Mailman 2 logging crap in favor of the standard logging package for MM3.
>

Yay, a thumbs up! Thanks, Barry, you're very welcome!
 
> The one thing I (very) occasionally want is to ask a logger for a file- 
> like object suitable for print.  There are some situations where I  
> have a 3rd party API that requires a file-like object to output to,  
> but I really want that output to go to a log.  I'm pretty sure I've  
> wrangled it out of a file-based logger, but it would be nice have this  
> as an official API.  Maybe it's there and I've just missed it though.
> 

A StreamHandler (which includes file-based handlers) has a stream attribute
which is a file-like object. Just pass that to your 3rd-party API.

N.B. If it's a file-based handler and you specify a delay argument to the
handler constructor, the file isn't actually opened until you first log to it,
so stream will be None. So you might need to be careful about that (by default,
a file-based handler doesn't delay opening the file).

Regards,

Vinay Sajip


From jnoller at gmail.com  Fri Sep 18 15:57:08 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 18 Sep 2009 09:57:08 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T104047-293@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
Message-ID: <4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>

On Fri, Sep 18, 2009 at 4:43 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Antoine Pitrou <solipsis at ...> writes:
>
>
>> Pylons uses it too.
>>
>> While logging may not look extremely pretty, I have yet to see another
>> logging library that has a pretty *and* powerful API. Applications /
>> libraries which reinvent their own logging API usually end up with
>> something which is both less powerful and *not* prettier (see
>> twisted.log for an example).
>>
>
> Thanks. If you had the time to write your ideal, "pretty" API - hypothetically,
> say, to wrap logging so you wouldn't use the underlying power - what would this
> API look like? I'm open to ideas from all of you.

Hi Vinay;

I've kept my mouth shut (well, on this subject) simply due to the fact
I tend to feel API design is a bit of a "smell" thing. First off;
thank you for the package. As much as I might not like the API - I use
the logging package an *insane* amount. I also know you're responsive,
and you care strongly about it. I myself might come off as slightly
defensive for my own module (multiprocessing) if someone where to just
say "lol it sux hahaha" (in fact, I have).

You are right; it is flexible, and it is meant for a wide-range of use
cases, and when you really "get it" it can be wildly powerful (I think
David Beazley wrote a twitter handler!).

However, my particular gut feeling when dealing with it stems from
something I can't quite communicate properly. I *want* something
"simpler" - for example, something which logs messages at a certain
level to stderr (but not stdout) and stdout to stdout (but not stderr)
- but also has a file logger.

I don't want to have to write more than a few lines of code to do this
- in *my* mind this is something so fundamental to
unixy-scripts/daemons than it should be as simple as:

import logging

log = logging.get_log('mylog')
log.warning('hay guys')

What I end up doing in most of my projects (sorry, not public ones) is
wrapping this in a "jesse.log" module that offers that API. The user
does not see the complexity of the underlying logging module's APIs.
In fact, I have a nasty tendency to create one "log" object which also
has a fair amount of the logging module's API pushed into it, e.g.:

    from jesse import log

    log = log()
    log.critical('yay!')
    log.set_level(log.WARNING)  # I loathe BouncyNames
    log.add_handler(log.file_handler(level=log.CRITICAL))


The other aspect of this is my experience trying to explain logging to
people who have never dealt with that module before. Recently, I was
trying to explain it to someone who has limited python knowledge, and
really just wanted something like what I describe above. They read the
docs, re-read the docs, re-re-re-read the docs, and still came to me
and said "how on earth do I do this?!". The API doesn't (and this
again, is a smell thing) feel python-y - it feels very Java like
(having experience log4j, I can say it really does feel like it). I
think this trips newbies up quite a bit.

Part of the newb-to-not-newb transition would be helped by possibly
simplifying the docs (something I am *still* working on for
multiprocessing) - the examples can be like drinking from a fire hose.
Doug Hellmann - the author of the Python Module of the Week is an
*excellent* doc writer (see his logging write up here:
http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
and might be willing to help, give pointers, what have you.

Like I said at the start - this is all a "smell" thing, and it
obviously varies from person to person. This is fundamentally why I
was interested in encouraging Mishok and others to put together
concrete ideas together (I would be interested in seeing an
alternative implementation as a thought exercise). I know others
besides me have written little wrappers around logging, for example:

http://pypi.python.org/pypi/easylog/
http://pypi.python.org/pypi/autolog/
http://pypi.python.org/pypi/sensible/

Perhaps that's a good place to start - higher level
functions/methods/etc to "scale down" loggings perceived complexity? I
know I'm trying to do bits of that for multiprocessing.

Then of course, there's time for something completely different ;)

http://code.zacharyvoase.com/lumberjack/src/

jesse

From armin.ronacher at active-4.com  Fri Sep 18 16:06:40 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Fri, 18 Sep 2009 16:06:40 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <loom.20090918T141927-390@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<4AB36D09.4040701@active-4.com>
	<loom.20090918T141927-390@post.gmane.org>
Message-ID: <4AB393F0.70301@active-4.com>

Hi,

Vinay Sajip wrote:
> Care to stop waving your arms about over general principles like
> "shared state", and get down to specifics about logging?
I was pointing out that a similar problem exists elsewhere.  And you
will never bring me to a point where I say something along the lines of
"shared state is okay because we cannot avoid it".

> It *is* an attribute of the LogRecord, and always has been - not sure where
> you get your "facts" ;-).
s/attribute/property/

> I don't know, without knowing your problem in more detail, if you could have
> avoided copying and changing code from LogRecord and Formatter. Obviously
> I've tried to provide enough hooks so that people can subclass and override
> methods for specific requirements, such as adding to a Trac ticket. If you
> describe the problem in more detail, I may be able to indicate a better
> solution. If it turns out that I need to provide more hooks where people can
> override functionality, I'm open to doing that.
It's in the standard library, modifications are not that useful.  If I
could pull an updated version from an external URL that adds these
hooks, fine.  But until then I can't use any of the modifications done
in there because it has to be compatible Python 2.4.

> So your complaint is really just a philosophical diatribe against shared
> state?
It's not philosophical, it's practical.

> Then you're really saying something roughly like "It's Not Invented Here, so
> I don't like it. The only way it could be better is if I had thought of it.
> Period."
I really don't know how to reply to that...

> *cough* *cough*. I've already read that post, as I referred to it earlier in
> this thread. Since sys.modules is shared state at a much more fundamental
> level than logging's logger registry, why not focus your energies on getting
> that changed first? If you're successful at pulling it off, it'll no doubt
> lead to a whole slew of changes in the Python ecosystem, of which logging is
> just a tiny part.
I think you're missing something:  I do not intent do change it.  I
complain about both logging and sys.modules because in my opinion those
are broken by design.  However there is also no way we could possibly
change those, so I don't even try to think about solutions.  Please
acknowledge that.

> There you go. Is Jinja2 "broken by design", then?
For many people it certainly is.  Also there are design decisions I made
in Jinja2 early on that are certainly broken, such as the scoping and I
try to fix those by adding deprecation warning for edge cases.  However,
you cannot do that in the standard library, different rules apply there.

> Aso, I prefer Python to Ruby. Should I say "Ruby? Broken, by design"?
I will just ignore that.

> What, sarcasm now? From "broken by design" to "paragon of virtue, exemplar
> to the world"? Please. I've been around the block a few times, no longer the
> new kid, and my skin is reasonably thick. I'm not crying. But keep it
> constructive (i.e. with improvement as a goal), that's all I'm asking.
That was not sarcastic.

> My point was not about whether Graham's plan was or was not the best. It's
> that he wants to get *something* reasonable out there, knows the issues and
> is frustrated with the constant back-and-forth between conflicting opinions.
> We could be waiting forever for a resolution, what good is that to anybody?
What good is a broken specification.  It's not about changing the
specification, it's about fixing it.  And there is a lot of things that
have to be thought through.  I love Graham's efforts in changing it, but
just deciding on one the proposals without thinking about the
consequences it has is not very wise.


Regards,
Armin

From doug.hellmann at gmail.com  Fri Sep 18 16:07:05 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Fri, 18 Sep 2009 10:07:05 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
Message-ID: <27643029-78B1-4ECF-B2EE-7B1F8D0ACE0B@gmail.com>


On Sep 18, 2009, at 9:57 AM, Jesse Noller wrote:

> Part of the newb-to-not-newb transition would be helped by possibly
> simplifying the docs (something I am *still* working on for
> multiprocessing) - the examples can be like drinking from a fire hose.
> Doug Hellmann - the author of the Python Module of the Week is an
> *excellent* doc writer (see his logging write up here:
> http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
> and might be willing to help, give pointers, what have you.

We use the logging module extensively at work and have found the  
shared state (even in a multi-threaded/multi-process situation) *much*  
nicer than passing around log handles.  In fact, we recently  
overhauled several process that, for legacy reasons, had been using  
log handles and now the code is simpler and the logs are easier to  
follow.

As far as code verbosity goes, we've found the "secret" to be using a  
configuration file.  That makes it easy to set up default logging and  
if our integrators want to tie in to their "enterprise" logging  
facility they can simply reconfigure it.  We don't have to build  
custom hooks for them because they're already included.  We *love*  
writing less code!

Vinay, contact me off list if you want to talk about documentation.  
I'd be happy to help in any way I can.

Doug


From g.brandl at gmx.net  Fri Sep 18 16:15:57 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 18 Sep 2009 16:15:57 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<1253263246.5710.6.camel@localhost>	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
Message-ID: <h904qt$6in$1@ger.gmane.org>

Jesse Noller schrieb:

> Part of the newb-to-not-newb transition would be helped by possibly
> simplifying the docs (something I am *still* working on for
> multiprocessing) - the examples can be like drinking from a fire hose.
> Doug Hellmann - the author of the Python Module of the Week is an
> *excellent* doc writer (see his logging write up here:
> http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
> and might be willing to help, give pointers, what have you.

Uh, the logging docs already contain most of Doug's MOTW for logging
in the "Tutorial/Simple examples" section.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From jnoller at gmail.com  Fri Sep 18 16:22:30 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 18 Sep 2009 10:22:30 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <h904qt$6in$1@ger.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<h904qt$6in$1@ger.gmane.org>
Message-ID: <4222a8490909180722m7f84d5cfy52a84436988dceb9@mail.gmail.com>

On Fri, Sep 18, 2009 at 10:15 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Jesse Noller schrieb:
>
>> Part of the newb-to-not-newb transition would be helped by possibly
>> simplifying the docs (something I am *still* working on for
>> multiprocessing) - the examples can be like drinking from a fire hose.
>> Doug Hellmann - the author of the Python Module of the Week is an
>> *excellent* doc writer (see his logging write up here:
>> http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
>> and might be willing to help, give pointers, what have you.
>
> Uh, the logging docs already contain most of Doug's MOTW for logging
> in the "Tutorial/Simple examples" section.
>
> Georg
>

I know, but I think that further work doesn't hurt - it only helps. In
fact I've still got a checkout where I'm trying to mind-meld
multiprocessing's docs with Doug's writing.

We should just give doug a commit bit.

jesse

From p.f.moore at gmail.com  Fri Sep 18 16:23:42 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Sep 2009 15:23:42 +0100
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB393F0.70301@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<4AB36D09.4040701@active-4.com>
	<loom.20090918T141927-390@post.gmane.org>
	<4AB393F0.70301@active-4.com>
Message-ID: <79990c6b0909180723y5b2680k26000fbeb14a7dc2@mail.gmail.com>

2009/9/18 Armin Ronacher <armin.ronacher at active-4.com>:
> I think you're missing something: ?I do not intent do change it. ?I
> complain about both logging and sys.modules because in my opinion those
> are broken by design. ?However there is also no way we could possibly
> change those, so I don't even try to think about solutions. ?Please
> acknowledge that.

Please, if you're not looking for change, do not complain about the
existing situation. At least, do not complain on this mailing list -
this whole discussion is tedious  (with the exception that I have been
impressed with Vinay's patience and willingness to listen to your
complaints and do something practical where possible) as well as being
off-topic for this list.

Can this thread end now, please?

Paul.

From armin.ronacher at active-4.com  Fri Sep 18 16:27:56 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Fri, 18 Sep 2009 16:27:56 +0200
Subject: [stdlib-sig] Evolving the Standard Library
In-Reply-To: <4AB393F0.70301@active-4.com>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<4AB36D09.4040701@active-4.com>	<loom.20090918T141927-390@post.gmane.org>
	<4AB393F0.70301@active-4.com>
Message-ID: <4AB398EC.2020209@active-4.com>

Hi,

Armin Ronacher schrieb:
> I think you're missing something:  I do not intent do change it.  I
> complain about both logging and sys.modules because in my opinion those
> are broken by design.  However there is also no way we could possibly
> change those, so I don't even try to think about solutions.  Please
> acknowledge that.
Just to explain myself a little bit: I got the impression you're
complaining that I'm not filing bugs in the bug tracker about logging.
Now you know the reason: what I want from logging is something that will
never happen.

As far as I'm concerned if this discussion should continue, then
probably around a table with some beer ;)


Regards,
Armin

From g.brandl at gmx.net  Fri Sep 18 16:37:45 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 18 Sep 2009 16:37:45 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <4222a8490909180722m7f84d5cfy52a84436988dceb9@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<1253263246.5710.6.camel@localhost>	<loom.20090918T104047-293@post.gmane.org>	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>	<h904qt$6in$1@ger.gmane.org>
	<4222a8490909180722m7f84d5cfy52a84436988dceb9@mail.gmail.com>
Message-ID: <h9063p$b0g$1@ger.gmane.org>

Jesse Noller schrieb:
> On Fri, Sep 18, 2009 at 10:15 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Jesse Noller schrieb:
>>
>>> Part of the newb-to-not-newb transition would be helped by possibly
>>> simplifying the docs (something I am *still* working on for
>>> multiprocessing) - the examples can be like drinking from a fire hose.
>>> Doug Hellmann - the author of the Python Module of the Week is an
>>> *excellent* doc writer (see his logging write up here:
>>> http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
>>> and might be willing to help, give pointers, what have you.
>>
>> Uh, the logging docs already contain most of Doug's MOTW for logging
>> in the "Tutorial/Simple examples" section.
>>
>> Georg
>>
> 
> I know, but I think that further work doesn't hurt - it only helps. In
> fact I've still got a checkout where I'm trying to mind-meld
> multiprocessing's docs with Doug's writing.
> 
> We should just give doug a commit bit.

Fine by me, but does he want it?  (And if he had it, he'd probably not know
where to start...)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From jnoller at gmail.com  Fri Sep 18 16:53:25 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 18 Sep 2009 10:53:25 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <h9063p$b0g$1@ger.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<h904qt$6in$1@ger.gmane.org>
	<4222a8490909180722m7f84d5cfy52a84436988dceb9@mail.gmail.com>
	<h9063p$b0g$1@ger.gmane.org>
Message-ID: <4222a8490909180753l6e2c9d57t9ba150bc07941fc4@mail.gmail.com>

On Fri, Sep 18, 2009 at 10:37 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Jesse Noller schrieb:
>> On Fri, Sep 18, 2009 at 10:15 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>>> Jesse Noller schrieb:
>>>
>>>> Part of the newb-to-not-newb transition would be helped by possibly
>>>> simplifying the docs (something I am *still* working on for
>>>> multiprocessing) - the examples can be like drinking from a fire hose.
>>>> Doug Hellmann - the author of the Python Module of the Week is an
>>>> *excellent* doc writer (see his logging write up here:
>>>> http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
>>>> and might be willing to help, give pointers, what have you.
>>>
>>> Uh, the logging docs already contain most of Doug's MOTW for logging
>>> in the "Tutorial/Simple examples" section.
>>>
>>> Georg
>>>
>>
>> I know, but I think that further work doesn't hurt - it only helps. In
>> fact I've still got a checkout where I'm trying to mind-meld
>> multiprocessing's docs with Doug's writing.
>>
>> We should just give doug a commit bit.
>
> Fine by me, but does he want it? ?(And if he had it, he'd probably not know
> where to start...)
>
> Georg

Yes - I think he is open to it, and could be an excellent contributor
to the docs at very least.

jesse

From p.f.moore at gmail.com  Fri Sep 18 17:21:29 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Sep 2009 16:21:29 +0100
Subject: [stdlib-sig] logging
In-Reply-To: <4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
Message-ID: <79990c6b0909180821y688af122qed5eaee18efc368c@mail.gmail.com>

2009/9/18 Jesse Noller <jnoller at gmail.com>:
> I've kept my mouth shut (well, on this subject) simply due to the fact
> I tend to feel API design is a bit of a "smell" thing. First off;
> thank you for the package.
[...]
> Perhaps that's a good place to start - higher level
> functions/methods/etc to "scale down" loggings perceived complexity? I
> know I'm trying to do bits of that for multiprocessing.

Basically, exactly what Jesse said applies with me - except that I
don't use the logging module intensively, and when I do use it the
lack of a simple approach makes it a struggle (nagging "why don't I
just use print statements" questions in the back of my mind :-))

If I try to recall my last experience, things that struck me as "too hard" were:

- logging.getLogger().warning(...) is irritatingly verbose ("feels
like Java"). Hmm, looks like I missed the module-level warning()
function and its cousins - don't know how I managed that! That's
basically my mistake (I can't even really claim that the documentation
is hard to find, looks like I just missed it). It might be nice if the
module-level functions had a logger='whatever' argument to ease the
change from "simple" requirements (root logger only) to more complex
ones (multiple loggers), but that's hardly critical.

- The file configuration seems very complex, for simple uses. I looked
at using it, but ultimately rolled my own, because I only wanted a few
simple options (which, no surprise, grew as time went on :-)).
Currently my application config consists of

[Log]
level: ERROR
file: {APPDIR}\{APP}.log
maxsize: 1M

log level, file to log to (with a couple of simple templating
parameters and special cases of "stdout" and "stderr") and a max size
(if set, use a rotating file handler).

Of course, writing a simpler version of logging.config.fileConfig can
be done as a 3rd party addition, so it's not fundamental to the
logging module.

Oh, and finally I *hate* Java-style camelCase but that's purely a
preference thing, and it's not going to change for compatibility
reasons, so let's ignore that.

Anyway, ultimately have no significant issues with the logging module.
Maybe it doesn't make the simple cases as simple as I'd like, but it
gives me all the power I'm ever likely to need, and then some. So
thanks for all your work on it!

Paul

From vinay_sajip at yahoo.co.uk  Fri Sep 18 17:59:35 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 15:59:35 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
Message-ID: <loom.20090918T161555-854@post.gmane.org>

Jesse Noller <jnoller at ...> writes:

> I've kept my mouth shut (well, on this subject) simply due to the fact
> I tend to feel API design is a bit of a "smell" thing. First off;
> thank you for the package. As much as I might not like the API - I use
> the logging package an *insane* amount. I also know you're responsive,
> and you care strongly about it. I myself might come off as slightly
> defensive for my own module (multiprocessing) if someone where to just
> say "lol it sux hahaha" (in fact, I have).
> 
> You are right; it is flexible, and it is meant for a wide-range of use
> cases, and when you really "get it" it can be wildly powerful (I think
> David Beazley wrote a twitter handler!).

It was only a matter of time. Next, a Google Wave handler. ;-)

> However, my particular gut feeling when dealing with it stems from
> something I can't quite communicate properly. I *want* something
> "simpler" - for example, something which logs messages at a certain
> level to stderr (but not stdout) and stdout to stdout (but not stderr)
> - but also has a file logger.
> 
> I don't want to have to write more than a few lines of code to do this
> - in *my* mind this is something so fundamental to
> unixy-scripts/daemons than it should be as simple as:
> 
> import logging
> 
> log = logging.get_log('mylog')
> log.warning('hay guys')
> 
> What I end up doing in most of my projects (sorry, not public ones) is
> wrapping this in a "jesse.log" module that offers that API. The user
> does not see the complexity of the underlying logging module's APIs.
> In fact, I have a nasty tendency to create one "log" object which also
> has a fair amount of the logging module's API pushed into it, e.g.:
> 
>     from jesse import log
> 
>     log = log()
>     log.critical('yay!')
>     log.set_level(log.WARNING)  # I loathe BouncyNames
>     log.add_handler(log.file_handler(level=log.CRITICAL))

One reason why people think logging is complicated is that they typically
have to contend with two things - loggers *and* handlers - whereas they have
perhaps in just used a logger, which either logs to file, or to console, or
whatever. For simple applications, this might work. However, for more
sophisticated requirements, it really helps to keep the two concepts
separate: "Where did it happen?" (the logger) and "Who wants to know?" (the
handler). In my experience, these are in general not mapped one-to-one. In a
typical enterprise use case, a critical error might page an on-call member of
the support team, write a detailed message to log, print a simplified message
to console, and email the incident to a developer team mailbox.

The logger is the domain of the developer. The handler is the domain of the
deployer. In many scenarios, of course, they are the same constituency - but
in many other scenarios, it's not so.

On the BouncyName thing - yes I know it's a religious thing for some people.
I've worked in Python, Java and C# amongst others and I've seen it all. My
cardinal rule is to fit in with whatever's already there, and although PEP 8
mandates names with no bounce, the logging package had already been released
and was in use before being proposed for inclusion in Python, and was using
camelCaseNames. Furthermore, the stdlib didn't at the time religiously stick
to PEP 8 in this regard. As I'm agnostic on this point, I've no problem in
providing all the method names using unix_style_underscore_notation, though
of course the other names will still be there for backwards compatibility :-)

Where you say "smell", I say "taste". It has a more positive connotation, but
means more or less the same thing in this context. For instance, I would not
naturally use "log" to name a logger, as in my mind, a logger is something
that facilitates getting some information into a log, and the log is the
final substrate for that information - the file, the console screen, the
email or whatever. So IMO there are elements of personal taste which we just
have to pass over when considering API design in general and naming in
particular, and when working with others who may not think exactly as we do.

I mentioned that the handler is the domain of the deployer, by which I mean
that using hard-coded set_level and add_handler calls in code is not always
the right thing to do. (To be sure, it is the right thing to do on many
occasions). Of course your simple basic example is doable as

import logging, sys

logging.basicConfig(level=logging.DEBUG, stream=sys.stderr)
logger = logging.getLogger('myapp')
logger.warning('hay guys')

which is only one line longer than your snippet - but in general, it's better
if the verbosity of logging in an application can be turned up and down
without having to change the application. It's not uncommon for people to use
configuration files to configure levels and handlers, and in the case of
long-running daemon processes this can even be done without stopping the
process.

The Python logging package's configuration format uses ConfigParser, which
doesn't float everybody's boat. If it is a cause of extreme hatred or even
mild distaste, you can always come up with your own configuration file format
which is more to your liking, and configure programmatically from that.

Why ConfigParser? one may ask. I don't believe that introducing my own,
different, ad-hoc configuration file format would have been the right thing
to do. TOOWTDI, and all that. Also, at the time, it was important not to add
too many APIs to the system, using the guiding principle of YAGNI (You Aren't
Gonna Need It), particularly if they were things that would perhaps be more
open to subjective judgements. Feel free to search the python-dev mail
archives for YAGNI in the late 2001-early 2002 timeframe. If I come up with
a simple easy-to-use wrapper that you like, then it's quite possible that
some other smart-and-opinionated developer will complain that it doesn't do
it for him/her. So - if it's just five lines of code in a project, even if
it's for every project, then it's hardly worth secreting much bile over, is
it? (I'm not saying *you* are.)

There are also some archaic constructs which result from trying to provide
Python 1.5.2 compatibility. At the time, there was a good reason for this -
most Linux distros were shipping with 1.5.2, although 2.2 had been around for
some time. I felt at the time that was important to support developers and
sysadmins who would have had to use the system's Python rather than upgrading
to 2.3. Of course the landscape has now changed considerably, and I am not of
the opinion that 1.5.2 support is stll important. The archaic relics will
disappear over time, no doubt.

> The other aspect of this is my experience trying to explain logging to
> people who have never dealt with that module before. Recently, I was
> trying to explain it to someone who has limited python knowledge, and
> really just wanted something like what I describe above. They read the
> docs, re-read the docs, re-re-re-read the docs, and still came to me
> and said "how on earth do I do this?!". The API doesn't (and this
> again, is a smell thing) feel python-y - it feels very Java like
> (having experience log4j, I can say it really does feel like it). I
> think this trips newbies up quite a bit.

I don't know why, particularly, apart from the conflating-handlers-and-loggers
thing. But, as I explained above, that separation of concerns is there for a
reason. I'm well aware of the Java predilection for over-complicating things,
and I have not drunk that particular Kool-Aid.

I don't blog, but following a recent discussion with Glyph Lefkowitz of
Twisted Matrix, I created a blog where I will put information related to
Python logging which perhaps doesn't belong in the documentation. The first
post is, reasonably enough, entitled "Python Logging 101" and is available at

http://plumberjack.blogspot.com/2009/09/python-logging-101.html

Perhaps you could take a look at it. Is really is a bit of a 101 in terms of
going from first principles; it's distilled from an article draft I had
prepared for DDJ back in 2002, but which never got accepted :-( [In those
days, DDJ was a prestigious print magazine, not vendor-aligned, rather than a
website-with-mailshots which is hard to distinguish from a Microsoft
tentacle.] It's only a five-minute-or-so read, and perhaps might explain why
logging's broad design is as it is.

> Part of the newb-to-not-newb transition would be helped by possibly
> simplifying the docs (something I am *still* working on for
> multiprocessing) - the examples can be like drinking from a fire hose.

Well the basic example is

import logging

logging.debug('A debug message')
logging.info('Some information')
logging.warning('A shot across the bows')

which is hardly challenging, and it builds up from there.

> Doug Hellmann - the author of the Python Module of the Week is an
> *excellent* doc writer (see his logging write up here:
> http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
> and might be willing to help, give pointers, what have you.

I'll definitely take it up with Doug. AFAIK, the current introductory
documentation on logging owes, I believe, something to Doug's PyMOTW
entry for logging, but I'm not sure as I didn't write that material.
(I can't remember who did, but it was in response to "logging is
complicated" feedback.)

> Like I said at the start - this is all a "smell" thing, and it
> obviously varies from person to person. This is fundamentally why I
> was interested in encouraging Mishok and others to put together
> concrete ideas together (I would be interested in seeing an
> alternative implementation as a thought exercise). I know others
> besides me have written little wrappers around logging, for example:
> 
> http://pypi.python.org/pypi/easylog/
> http://pypi.python.org/pypi/autolog/
> http://pypi.python.org/pypi/sensible/
> 
> Perhaps that's a good place to start - higher level
> functions/methods/etc to "scale down" loggings perceived complexity? I
> know I'm trying to do bits of that for multiprocessing.

I'm not sure how much traction they've got. Anybody know? The very fact that
there are three seems to indicate that this is an area where subjective
judgements play a part. So, people can pick whichever one they want, and off
they go. If I incorporated one of their versions into the stdlib, presumably
the others would still be around, along with your variation on the same theme.

> Then of course, there's time for something completely different ;)
> 
> http://code.zacharyvoase.com/lumberjack/src/
> 

Is Zachary actually serious about this as an alternative to Python logging?
(I'm not dissing it - just asking if he is seriously committed to getting it
to have the same level of functionality.) From the screencast, I got the
impression (perhaps mistakenly) that it was motivated at least in part by "Oh
look! Coroutines! Shiny new Python toys. Let's see what we can do with them!"
After an initial flurry of work on it, things have gone quiet over the last
four weeks. Perhaps he's not pushed his changes to BitBucket because he's
still working on them.

Coroutines are undeniably nice for some things - but I can't say I see a
no-brainer fit with logging. I know coroutines are new in Python but I've
seen them come and go in popularity a couple of times in different
environments.

Regards,

Vinay Sajip




From vinay_sajip at yahoo.co.uk  Fri Sep 18 18:03:56 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 16:03:56 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<27643029-78B1-4ECF-B2EE-7B1F8D0ACE0B@gmail.com>
Message-ID: <loom.20090918T180015-770@post.gmane.org>

Doug Hellmann <doug.hellmann at ...> writes:

> We use the logging module extensively at work and have found the  
> shared state (even in a multi-threaded/multi-process situation) *much*  
> nicer than passing around log handles...
> ...now the code is simpler and the logs are easier to follow.

Thanks for sharing your experience.

> As far as code verbosity goes, we've found the "secret" to be using a  
> configuration file... We don't have to build custom hooks for them because
> they're already included.  We *love* writing less code!
 
> Vinay, contact me off list if you want to talk about documentation.  

Thanks, Doug, I'll definitely do that.

Regards,


Vinay Sajip



From vinay_sajip at yahoo.co.uk  Fri Sep 18 18:10:56 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 16:10:56 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<79990c6b0909180821y688af122qed5eaee18efc368c@mail.gmail.com>
Message-ID: <loom.20090918T180635-509@post.gmane.org>

Paul Moore <p.f.moore at ...> writes:

> - The file configuration seems very complex, for simple uses. I looked
> at using it, but ultimately rolled my own, because I only wanted a few
> simple options (which, no surprise, grew as time went on ).
[snip]
> Oh, and finally I *hate* Java-style camelCase but that's purely a
> preference thing, and it's not going to change for compatibility
> reasons, so let's ignore that.

Okay, I've commented elsewhere that I'm happy to provide
unix_style_underscored_names.

> Anyway, ultimately have no significant issues with the logging module.
> Maybe it doesn't make the simple cases as simple as I'd like, but it
> gives me all the power I'm ever likely to need, and then some. So
> thanks for all your work on it!

For non-rotating files, it's really as simple as 

import logging

logging.basicConfig(filename='/tmp/logging_example.out',
                    level=logging.DEBUG,)
logging.debug('This message should go to the log file')

Of course for rotating it's more involved, but that's where there's
really not much of a common denominator. IMO The best approach would be
to add recipes to the Python Cookbook on ActiveState.



From theller at ctypes.org  Fri Sep 18 18:21:09 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 18 Sep 2009 18:21:09 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T161555-854@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<1253263246.5710.6.camel@localhost>	<loom.20090918T104047-293@post.gmane.org>	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<loom.20090918T161555-854@post.gmane.org>
Message-ID: <4AB3B375.2010704@ctypes.org>

Vinay, you get a bug thumbs-up from me too, for the logging package
and your maintainance and support.

Vinay Sajip schrieb:
> The Python logging package's configuration format uses ConfigParser, which
> doesn't float everybody's boat. If it is a cause of extreme hatred or even
> mild distaste, you can always come up with your own configuration file format
> which is more to your liking, and configure programmatically from that.

Although I like and use logging, I belong more to the hatred people
for the logging configuration.

What I am really missing is a way to do basic logging configuration (calling
logging.basicConfig(...)) with some command line args for the Python
interpreter.  I have suggested that before IIRC, but it doesn't seem to fly.

-- 
Thanks,
Thomas

From vinay_sajip at yahoo.co.uk  Fri Sep 18 18:58:49 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 16:58:49 +0000 (UTC)
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<1253263246.5710.6.camel@localhost>	<loom.20090918T104047-293@post.gmane.org>	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<loom.20090918T161555-854@post.gmane.org>
	<4AB3B375.2010704@ctypes.org>
Message-ID: <loom.20090918T185838-730@post.gmane.org>

Thomas Heller <theller at ...> writes:

> 
> Vinay, you get a bug thumbs-up from me too, for the logging package
> and your maintainance and support.

Thanks.
 
> 
> Although I like and use logging, I belong more to the hatred people
> for the logging configuration.
> 

I don't blame you. It's partly, though not all, a ConfigParser thing. The
difficulty for me with configuration is, there's very little chance of a
consensus for how it should work. Everybody will have their opinion, they'll all
have good points, but nobody will agree :-( Backwards compatibility might be
manageable, if we stick to a ConfigParser format and introduce a "version" key
somewhere.

> What I am really missing is a way to do basic logging configuration (calling
> logging.basicConfig(...)) with some command line args for the Python
> interpreter.  I have suggested that before IIRC, but it doesn't seem to fly.

Can you point me to a ticket or post about it?

Thanks and regards,

Vinay Sajip



From barry at python.org  Fri Sep 18 19:17:40 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 18 Sep 2009 13:17:40 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <27643029-78B1-4ECF-B2EE-7B1F8D0ACE0B@gmail.com>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<27643029-78B1-4ECF-B2EE-7B1F8D0ACE0B@gmail.com>
Message-ID: <AF4B885C-A6F8-4266-8F1A-9B18995AFDDF@python.org>

On Sep 18, 2009, at 10:07 AM, Doug Hellmann wrote:

> We use the logging module extensively at work and have found the  
> shared state (even in a multi-threaded/multi-process situation)  
> *much* nicer than passing around log handles.

Here, here.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090918/6be70d3f/attachment.pgp>

From jnoller at gmail.com  Fri Sep 18 19:26:12 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 18 Sep 2009 13:26:12 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <AF4B885C-A6F8-4266-8F1A-9B18995AFDDF@python.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<27643029-78B1-4ECF-B2EE-7B1F8D0ACE0B@gmail.com>
	<AF4B885C-A6F8-4266-8F1A-9B18995AFDDF@python.org>
Message-ID: <4222a8490909181026g653ad9cfo2bf8f2f5dea57261@mail.gmail.com>

On Fri, Sep 18, 2009 at 1:17 PM, Barry Warsaw <barry at python.org> wrote:
> On Sep 18, 2009, at 10:07 AM, Doug Hellmann wrote:
>
>> We use the logging module extensively at work and have found the shared
>> state (even in a multi-threaded/multi-process situation) *much* nicer than
>> passing around log handles.
>
> Here, here.
> -Barry
>
>

Also +1; while the concept of shared state is bad in some contexts,
for apps where you want one central logger the global registry *is*
nice.

From theller at ctypes.org  Fri Sep 18 19:40:21 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 18 Sep 2009 19:40:21 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T185838-730@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<1253263246.5710.6.camel@localhost>	<loom.20090918T104047-293@post.gmane.org>	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>	<loom.20090918T161555-854@post.gmane.org>	<4AB3B375.2010704@ctypes.org>
	<loom.20090918T185838-730@post.gmane.org>
Message-ID: <4AB3C605.7070504@ctypes.org>

Vinay Sajip schrieb:
> Thomas Heller <theller at ...> writes:
>> 
>> Although I like and use logging, I belong more to the hatred people
>> for the logging configuration.
>> 
> 
> I don't blame you. It's partly, though not all, a ConfigParser thing. The
> difficulty for me with configuration is, there's very little chance of a
> consensus for how it should work. Everybody will have their opinion, they'll all
> have good points, but nobody will agree :-( Backwards compatibility might be
> manageable, if we stick to a ConfigParser format and introduce a "version" key
> somewhere.

You should be fine with me: I don't care because I don't the config files.

>> What I am really missing is a way to do basic logging configuration (calling
>> logging.basicConfig(...)) with some command line args for the Python
>> interpreter.  I have suggested that before IIRC, but it doesn't seem to fly.
> 
> Can you point me to a ticket or post about it?

I'm not sure there is a ticket or feature request for it - probably not.
What I would like to see is something like this:

python -L level=DEBUG script.py scriptargs ...

which then calls
import logging; logging.basicConfig(level=logging.DEBUG)

Unfortunately it's not possible to access command line arguments in sitecustomize.py,
otherwise I would have hacked it there...

-- 
Thanks,
Thomas

From vinay_sajip at yahoo.co.uk  Sat Sep 19 00:28:25 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Sep 2009 22:28:25 +0000 (UTC)
Subject: [stdlib-sig] ConfigParser - parsing options with no value
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<h8ubhl$t69$1@ger.gmane.org>
Message-ID: <loom.20090919T002421-440@post.gmane.org>

Georg Brandl <g.brandl at ...> writes:

> Good point.  There's also an awful shortage of config parsers in the standard
> library; we should add one or two.
> 

Way back in the mists of time, a ConfigParserShootout was mooted and a wiki page
dedicated to looking at alternative implementations - a whole bunch of
candidates were nominated (disclosure: including one that I wrote, but also
Michael's ConfigObj) but then it never went anywhere. Here's the link to the
Wiki page:

http://wiki.python.org/moin/ConfigParserShootout

Perhaps we should have a vote to select the one or two to include ;-)

Regards,

Vinay Sajip



From michael at voidspace.org.uk  Sat Sep 19 00:30:49 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Fri, 18 Sep 2009 23:30:49 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <loom.20090919T002421-440@post.gmane.org>
References: <4AB26996.8030707@sun.com>	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>	<h8ubhl$t69$1@ger.gmane.org>
	<loom.20090919T002421-440@post.gmane.org>
Message-ID: <4AB40A19.7030102@voidspace.org.uk>

Vinay Sajip wrote:
> Georg Brandl <g.brandl at ...> writes:
>
>   
>> Good point.  There's also an awful shortage of config parsers in the standard
>> library; we should add one or two.
>>
>>     
>
> Way back in the mists of time, a ConfigParserShootout was mooted and a wiki page
> dedicated to looking at alternative implementations - a whole bunch of
> candidates were nominated (disclosure: including one that I wrote, but also
> Michael's ConfigObj) but then it never went anywhere. Here's the link to the
> Wiki page:
>
> http://wiki.python.org/moin/ConfigParserShootout
>
> Perhaps we should have a vote to select the one or two to include ;-)
>   

It did go somewhere - it was discussed on Python-dev and Guido didn't 
see the need to replace ConfigParser (and specifically didn't see the 
need for nested sections that was at least in part the motivation for 
the shootout).

Michael

> Regards,
>
> Vinay Sajip
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From vinay_sajip at yahoo.co.uk  Sat Sep 19 09:54:57 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 19 Sep 2009 07:54:57 +0000 (UTC)
Subject: [stdlib-sig] ConfigParser - parsing options with no value
References: <4AB26996.8030707@sun.com>	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>	<h8ubhl$t69$1@ger.gmane.org>
	<loom.20090919T002421-440@post.gmane.org>
	<4AB40A19.7030102@voidspace.org.uk>
Message-ID: <loom.20090919T095336-977@post.gmane.org>

Michael Foord <michael at ...> writes:

> It did go somewhere - it was discussed on Python-dev and Guido didn't 
> see the need to replace ConfigParser (and specifically didn't see the 
> need for nested sections that was at least in part the motivation for 
> the shootout).
> 

Oh - must've missed that. Perhaps the Wiki page should be updated.

Thanks, Michael.

Regards,


Vinay Sajip


From michael at voidspace.org.uk  Sat Sep 19 11:40:56 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Sat, 19 Sep 2009 10:40:56 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <loom.20090919T095336-977@post.gmane.org>
References: <4AB26996.8030707@sun.com>	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>	<h8ubhl$t69$1@ger.gmane.org>
	<loom.20090919T002421-440@post.gmane.org>
	<4AB40A19.7030102@voidspace.org.uk>
	<loom.20090919T095336-977@post.gmane.org>
Message-ID: <FFB7F448-B346-4EA0-B824-EE5537E6A01A@voidspace.org.uk>

I'll see if I can dig out the discussion and update the page.

It might also be worth adding feature requests or doc examples for  
ConfigParser to cover the more common use cases / features that other  
config file modules provide that you don't get out of the box with  
ConfigParser.

Michael


--
http://www.ironpythoninaction.com

On 19 Sep 2009, at 08:54, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

> Michael Foord <michael at ...> writes:
>
>> It did go somewhere - it was discussed on Python-dev and Guido didn't
>> see the need to replace ConfigParser (and specifically didn't see the
>> need for nested sections that was at least in part the motivation for
>> the shootout).
>>
>
> Oh - must've missed that. Perhaps the Wiki page should be updated.
>
> Thanks, Michael.
>
> Regards,
>
>
> Vinay Sajip
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig

From vinay_sajip at yahoo.co.uk  Sat Sep 19 14:27:29 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 19 Sep 2009 12:27:29 +0000 (UTC)
Subject: [stdlib-sig] ConfigParser - parsing options with no value
References: <4AB26996.8030707@sun.com>	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>	<h8ubhl$t69$1@ger.gmane.org>
	<loom.20090919T002421-440@post.gmane.org>
	<4AB40A19.7030102@voidspace.org.uk>
	<loom.20090919T095336-977@post.gmane.org>
	<FFB7F448-B346-4EA0-B824-EE5537E6A01A@voidspace.org.uk>
Message-ID: <loom.20090919T142511-689@post.gmane.org>

Michael Foord <michael at ...> writes:

> I'll see if I can dig out the discussion and update the page.

Good idea.

> It might also be worth adding feature requests or doc examples for  
> ConfigParser to cover the more common use cases / features that other  
> config file modules provide that you don't get out of the box with  
> ConfigParser.
> 

There's already a ConfigParser examples page

http://wiki.python.org/moin/ConfigParserExamples

which has only some basic stuff - but perhaps any additional examples/recipes
should be added there.

Regards,

Vinay Sajip


From michael at voidspace.org.uk  Sat Sep 19 14:33:10 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Sat, 19 Sep 2009 13:33:10 +0100
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <loom.20090919T142511-689@post.gmane.org>
References: <4AB26996.8030707@sun.com>	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>	<h8ubhl$t69$1@ger.gmane.org>	<loom.20090919T002421-440@post.gmane.org>	<4AB40A19.7030102@voidspace.org.uk>	<loom.20090919T095336-977@post.gmane.org>	<FFB7F448-B346-4EA0-B824-EE5537E6A01A@voidspace.org.uk>
	<loom.20090919T142511-689@post.gmane.org>
Message-ID: <4AB4CF86.4070002@voidspace.org.uk>

Vinay Sajip wrote:
> Michael Foord <michael at ...> writes:
>
>   
>> I'll see if I can dig out the discussion and update the page.
>>     
>
> Good idea.
>
>   
>> It might also be worth adding feature requests or doc examples for  
>> ConfigParser to cover the more common use cases / features that other  
>> config file modules provide that you don't get out of the box with  
>> ConfigParser.
>>
>>     
>
> There's already a ConfigParser examples page
>
> http://wiki.python.org/moin/ConfigParserExamples
>
> which has only some basic stuff - but perhaps any additional examples/recipes
> should be added there.
>   

For common use cases that *can* be achieved with simple recipes it would 
be nice to have them in the documentation. Anyway - I'll put this in my 
'list' of things to do.

Michael

> Regards,
>
> Vinay Sajip
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From armin.ronacher at active-4.com  Sat Sep 19 21:27:58 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sat, 19 Sep 2009 21:27:58 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
Message-ID: <4AB530BE.1090104@active-4.com>

Aye, me mateys,

In Python 3 the parrrsing function of urllib do not work with bytes.
What's the prrrroblem?  I tell you: U'RLs only have a charrrrrset
rrecommendation and sometimes you have to deal with URL encoded stuff
that does not contain unicode data.

I tried to crrreate a patch for urllib but it appears that you have to
rrrrrreplicate ParseResult for byte strrrrings which seems wrong to me.
 Does anyone rrremember the rrreasons why urllib was not designed to
work on bytes interrrnaly and only convert to unicode before/after
converrrrrsion?

But maybe we could also add IRI functions to that module, or add a
irilib that allows the conversion between U'RIs and I'RIs.

urllib depending on unicode strrrrings is for me the biggest rrreason to
base WSGI for Python 3 exlusively on unicode.

Yo ho, that's it frrom me.


Shiver my timbers!
Arrrrrmin

From brett at python.org  Sat Sep 19 22:56:21 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 19 Sep 2009 13:56:21 -0700
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB530BE.1090104@active-4.com>
References: <4AB530BE.1090104@active-4.com>
Message-ID: <bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>

Initially I thought your 'r' key was having sticking issues. I really
do hate Talk Like A Pirate Day.

On Sat, Sep 19, 2009 at 12:27, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Aye, me mateys,
>
> In Python 3 the parrrsing function of urllib do not work with bytes.
> What's the prrrroblem? ?I tell you: U'RLs only have a charrrrrset
> rrecommendation and sometimes you have to deal with URL encoded stuff
> that does not contain unicode data.
>
> I tried to crrreate a patch for urllib but it appears that you have to
> rrrrrreplicate ParseResult for byte strrrrings which seems wrong to me.
> ?Does anyone rrremember the rrreasons why urllib was not designed to
> work on bytes interrrnaly and only convert to unicode before/after
> converrrrrsion?
>

See, you are assuming any design went into other than to make the
thing pass the unit tests. Most modules did not go through some
rigorous design discussion to decide how to make it work with bytes.
Someone just took it upon themselves to make the thing work and that
was that. I am willing to guess this is more or less what happened
with urllib, especially since it was a bit tricky to get it merged
between urllib and urllib2.

-Brett

From armin.ronacher at active-4.com  Sat Sep 19 22:59:40 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sat, 19 Sep 2009 22:59:40 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
Message-ID: <4AB5463C.5010505@active-4.com>

Hi,

Brett Cannon schrieb:
> See, you are assuming any design went into other than to make the
> thing pass the unit tests. Most modules did not go through some
> rigorous design discussion to decide how to make it work with bytes.
So I suppose there would be nothing wrong with providing a patch that
makes it work with bytes internally?


Regards,
Armin

From g.brandl at gmx.net  Sat Sep 19 23:10:47 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 19 Sep 2009 23:10:47 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB5463C.5010505@active-4.com>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com>
Message-ID: <h93hgp$idi$1@ger.gmane.org>

Armin Ronacher schrieb:
> Hi,
> 
> Brett Cannon schrieb:
>> See, you are assuming any design went into other than to make the
>> thing pass the unit tests. Most modules did not go through some
>> rigorous design discussion to decide how to make it work with bytes.
> So I suppose there would be nothing wrong with providing a patch that
> makes it work with bytes internally?

Aye, assuming no one else speaks up in defense of the current one.
(But please don't provide the patch until tomorrrrow.)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Sun Sep 20 00:24:10 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 19 Sep 2009 15:24:10 -0700
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB5463C.5010505@active-4.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com> 
	<4AB5463C.5010505@active-4.com>
Message-ID: <bbaeab100909191524x3c733602x3c5f3d49477fe1f3@mail.gmail.com>

On Sat, Sep 19, 2009 at 13:59, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Hi,
>
> Brett Cannon schrieb:
>> See, you are assuming any design went into other than to make the
>> thing pass the unit tests. Most modules did not go through some
>> rigorous design discussion to decide how to make it work with bytes.
> So I suppose there would be nothing wrong with providing a patch that
> makes it work with bytes internally?

As long as the external API continues to work as expected, then no,
there are no problems with making the code more bytes/unicode friendly
internally.

-Brett

From solipsis at pitrou.net  Sun Sep 20 13:49:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 13:49:25 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB5463C.5010505@active-4.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com>
Message-ID: <1253447365.5341.23.camel@localhost>

Le samedi 19 septembre 2009 ? 22:59 +0200, Armin Ronacher a ?crit :
> Hi,
> 
> Brett Cannon schrieb:
> > See, you are assuming any design went into other than to make the
> > thing pass the unit tests. Most modules did not go through some
> > rigorous design discussion to decide how to make it work with bytes.
> So I suppose there would be nothing wrong with providing a patch that
> makes it work with bytes internally?

Can you please precise what you are trying to do?

If you just want to replace the implementation without changing the API
to support bytes, I'm not sure what the point is.

If you want the public API to support bytes, perhaps it would be worth
discussing it (on python-dev perhaps, since I'm not sure everyone
concerned is here)?
Although since HTTP is bytes at the protocol level anyway, it doesn't
seem very controversial to me...

Regards

Antoine.



From michael at voidspace.org.uk  Sun Sep 20 13:49:05 2009
From: michael at voidspace.org.uk (Michael Foord)
Date: Sun, 20 Sep 2009 12:49:05 +0100
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <1253447365.5341.23.camel@localhost>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>
Message-ID: <4AB616B1.2080806@voidspace.org.uk>

Antoine Pitrou wrote:
> Le samedi 19 septembre 2009 ? 22:59 +0200, Armin Ronacher a ?crit :
>   
>> Hi,
>>
>> Brett Cannon schrieb:
>>     
>>> See, you are assuming any design went into other than to make the
>>> thing pass the unit tests. Most modules did not go through some
>>> rigorous design discussion to decide how to make it work with bytes.
>>>       
>> So I suppose there would be nothing wrong with providing a patch that
>> makes it work with bytes internally?
>>     
>
> Can you please precise what you are trying to do?
>
> If you just want to replace the implementation without changing the API
> to support bytes, I'm not sure what the point is.
>
>   
Uhm... the point being to support bytes? :-)

> If you want the public API to support bytes, perhaps it would be worth
> discussing it (on python-dev perhaps, since I'm not sure everyone
> concerned is here)?
> Although since HTTP is bytes at the protocol level anyway, it doesn't
> seem very controversial to me...
>   

I don't see why it should be controversial at all - and I'm sure Armin 
is aware enough of the issues to do a good job. Probably posting a patch 
and having a discussion on the bug tracker.

Michael

> Regards
>
> Antoine.
>
>
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From armin.ronacher at active-4.com  Sun Sep 20 20:48:04 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 20 Sep 2009 20:48:04 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <1253447365.5341.23.camel@localhost>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>
Message-ID: <4AB678E4.1040704@active-4.com>

Hi,

Antoine Pitrou schrieb:
> Can you please precise what you are trying to do?
> 
> If you just want to replace the implementation without changing the API
> to support bytes, I'm not sure what the point is.
What I'm trying to do and the point of my changes is support for bytes.
 A feature that disappared in Python 3.0 and is widely used.


Problems I've seen in the code so far that go beyond the point of what I
intend to change:

unused and undocumented interfaces:

- urllib.parse.unwrap     not sure what that is for, I guess testing
- urllib.parse.to_bytes   also unused, I guess it was in use for an
                          older implementation
- urllib.parse.test and
  urllib.parse.test_data  unused test suite that is also in the real
                          test suite.  Can we delete it?

It's also pretty inconsistent currently:

urllib.parse.quote and urllib.parse.quote_plus already work with bytes
(the code adds explicit byte support, one just calls into
urllib.parse.quote_from_bytes).  However the reverse functions do not
work with bytes at all.

What is quote_from_bytes used for when quote just calls into it?  Why
did this ever become a public interface?  Also once unquote and
unquote_plus are fixed, unquote_to_bytes is similarly useless,
especially because both of them perform some weird utf-8 conversion if
non-bytes are passed to them.

I would just fix those and deprecate the explicit unquote_to_bytes and
quote_from_bytes for future versions of Python 3.

Any comments on that?


Regards,
Armin

From solipsis at pitrou.net  Sun Sep 20 20:55:51 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 20:55:51 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB678E4.1040704@active-4.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com> <1253447365.5341.23.camel@localhost>
	<4AB678E4.1040704@active-4.com>
Message-ID: <1253472951.5341.32.camel@localhost>

Le dimanche 20 septembre 2009 ? 20:48 +0200, Armin Ronacher a ?crit :
> 
> What is quote_from_bytes used for when quote just calls into it?  Why
> did this ever become a public interface?  Also once unquote and
> unquote_plus are fixed, unquote_to_bytes is similarly useless,
> especially because both of them perform some weird utf-8 conversion if
> non-bytes are passed to them.

You should try a search in the python-dev archives. It was introduced
recently, and after quite a bit of discussion I think.

> I would just fix those and deprecate the explicit unquote_to_bytes and
> quote_from_bytes for future versions of Python 3.
> 
> Any comments on that?

Regardless of whether the semantics would be better or not, we can't
change APIs every 2 years, it will make our users furous.

Regards

Antoine.



From armin.ronacher at active-4.com  Sun Sep 20 20:58:30 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 20 Sep 2009 20:58:30 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <1253472951.5341.32.camel@localhost>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>	<4AB678E4.1040704@active-4.com>
	<1253472951.5341.32.camel@localhost>
Message-ID: <4AB67B56.5000408@active-4.com>

Hi,

Antoine Pitrou schrieb:
> You should try a search in the python-dev archives. It was introduced
> recently, and after quite a bit of discussion I think.
Just found the discussion.  Apparently the reason there are different
functions is something Guido wanted.

I guess the best idea is to finish the patch and discuss that one on
python-devel then.

> Regardless of whether the semantics would be better or not, we can't
> change APIs every 2 years, it will make our users furous.
Right, but how many people are using Python 3?

Regards,
Armin

From orsenthil at gmail.com  Sun Sep 20 20:59:34 2009
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Mon, 21 Sep 2009 00:29:34 +0530
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <1253472951.5341.32.camel@localhost>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>
	<4AB678E4.1040704@active-4.com>
	<1253472951.5341.32.camel@localhost>
Message-ID: <20090920185934.GD8962@ubuntu.ubuntu-domain>

On Sun, Sep 20, 2009 at 08:55:51PM +0200, Antoine Pitrou wrote:
> Le dimanche 20 septembre 2009 ? 20:48 +0200, Armin Ronacher a ?crit :
> > 
> > What is quote_from_bytes used for when quote just calls into it?  Why
> > did this ever become a public interface?  Also once unquote and
> > unquote_plus are fixed, unquote_to_bytes is similarly useless,
> > especially because both of them perform some weird utf-8 conversion if
> > non-bytes are passed to them.
> 
> You should try a search in the python-dev archives. It was introduced

Well, I was discussing with Armin Ronacher at IRC and I pointed him to 
this: http://bugs.python.org/issue3300 

Yes, it did go through considerable amount of discussion. 

> recently, and after quite a bit of discussion I think.
> 
> > I would just fix those and deprecate the explicit unquote_to_bytes and
> > quote_from_bytes for future versions of Python 3.
> > 
> > Any comments on that?
> 
> Regardless of whether the semantics would be better or not, we can't
> change APIs every 2 years, it will make our users furous.

More so for commonly used ones.. 


-- 
Senthil
Smoking is one of the leading causes of statistics.
		-- Fletcher Knebel

From ronaldoussoren at mac.com  Sun Sep 20 21:03:32 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 20 Sep 2009 21:03:32 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB67B56.5000408@active-4.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com> <1253447365.5341.23.camel@localhost>
	<4AB678E4.1040704@active-4.com> <1253472951.5341.32.camel@localhost>
	<4AB67B56.5000408@active-4.com>
Message-ID: <01146B88-B720-444E-A329-E9E7A472B88C@mac.com>


On 20 Sep, 2009, at 20:58, Armin Ronacher wrote:
>
>
>> Regardless of whether the semantics would be better or not, we can't
>> change APIs every 2 years, it will make our users furous.
> Right, but how many people are using Python 3?

This irrelevant. Python 3.x is bound by the same API stability rules
as the 2.x releases.

One way to ensure that 3.x isn't used a lot is to keep making
incompatible changes, that way anyone that wants to use
Python 3 in production code will stay away.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090920/ab5e43b4/attachment-0001.bin>

From orsenthil at gmail.com  Sun Sep 20 21:20:44 2009
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Mon, 21 Sep 2009 00:50:44 +0530
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB678E4.1040704@active-4.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>
	<4AB678E4.1040704@active-4.com>
Message-ID: <20090920192044.GE8962@ubuntu.ubuntu-domain>

On Sun, Sep 20, 2009 at 08:48:04PM +0200, Armin Ronacher wrote:
> unused and undocumented interfaces:

Not really unused.

> - urllib.parse.unwrap     not sure what that is for, I guess testing

Head over to request.py and see how Request class unwraps the URL
presented in the format it handles.  

This has been there since 2.x urlparse and this is parsing function
and not does belong to urllibx.py.

I am not sure, if people are not using this interface in any of their
codes, because this has been present in urlparse for a long time.

> - urllib.parse.to_bytes   also unused, I guess it was in use for an

Again used in request.py to validate that URL as str

> - urllib.parse.test and
>   urllib.parse.test_data  unused test suite that is also in the real
>                           test suite.  Can we delete it?
> 

These are just hanging around with as No-Harm thing :) Can be removed.
As we have tests which cover them.  Its been there, because there is
'no-harm' in keeping them around.


-- 
Senthil
I believe a little incompatibility is the spice of life, particularly if he
has income and she is pattable.
		-- Ogden Nash

From armin.ronacher at active-4.com  Sun Sep 20 21:42:45 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 20 Sep 2009 21:42:45 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <01146B88-B720-444E-A329-E9E7A472B88C@mac.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>
	<4AB678E4.1040704@active-4.com>
	<1253472951.5341.32.camel@localhost>
	<4AB67B56.5000408@active-4.com>
	<01146B88-B720-444E-A329-E9E7A472B88C@mac.com>
Message-ID: <4AB685B5.6010108@active-4.com>

Hi,

Ronald Oussoren schrieb:
> This irrelevant. Python 3.x is bound by the same API stability rules
> as the 2.x releases.
Right now the number of Python 3 adopters that are using Python for web
related application is a *lot* lower than the number of Python 2 users.

When it comes to actual users using it I think it's better to compare it
to the early Python 1.5 days.

> One way to ensure that 3.x isn't used a lot is to keep making
> incompatible changes, that way anyone that wants to use
> Python 3 in production code will stay away.
So you think the broken behavior is the best way to encourage people
working with Python 3?  Sounds about right.


Regards,
Armin

From armin.ronacher at active-4.com  Sun Sep 20 21:44:20 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 20 Sep 2009 21:44:20 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <20090920185934.GD8962@ubuntu.ubuntu-domain>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>	<4AB5463C.5010505@active-4.com>	<1253447365.5341.23.camel@localhost>	<4AB678E4.1040704@active-4.com>	<1253472951.5341.32.camel@localhost>
	<20090920185934.GD8962@ubuntu.ubuntu-domain>
Message-ID: <4AB68614.1070708@active-4.com>

Hi,

Senthil Kumaran schrieb:
> Well, I was discussing with Armin Ronacher at IRC and I pointed him to 
> this: http://bugs.python.org/issue3300 
However this is a different issue.  That ticket covers the *unicode*
related parts of urllib, I only work on the *byte* parts which currently
either do not work or are not enough tested and break easily.

I just aim to make sure urllib in Python 3 will also work on bytes.


Regards,
Armin

From solipsis at pitrou.net  Sun Sep 20 21:55:17 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 21:55:17 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <4AB685B5.6010108@active-4.com>
References: <4AB530BE.1090104@active-4.com>
	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>
	<4AB5463C.5010505@active-4.com> <1253447365.5341.23.camel@localhost>
	<4AB678E4.1040704@active-4.com> <1253472951.5341.32.camel@localhost>
	<4AB67B56.5000408@active-4.com>
	<01146B88-B720-444E-A329-E9E7A472B88C@mac.com>
	<4AB685B5.6010108@active-4.com>
Message-ID: <1253476517.5341.35.camel@localhost>


> So you think the broken behavior is the best way to encourage people
> working with Python 3?  Sounds about right.

You know, it would be better if you demonstrated that the behaviour is
broken, rather than asserting it. The fact that a long discussion led to
the current API is a good hint that it is probably not as broken as you
make it to be.




From mal at egenix.com  Sun Sep 20 22:09:50 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun, 20 Sep 2009 22:09:50 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <1253476517.5341.35.camel@localhost>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>	<4AB678E4.1040704@active-4.com>
	<1253472951.5341.32.camel@localhost>	<4AB67B56.5000408@active-4.com>	<01146B88-B720-444E-A329-E9E7A472B88C@mac.com>	<4AB685B5.6010108@active-4.com>
	<1253476517.5341.35.camel@localhost>
Message-ID: <4AB68C0E.1080509@egenix.com>

Antoine Pitrou wrote:
> 
>> So you think the broken behavior is the best way to encourage people
>> working with Python 3?  Sounds about right.
> 
> You know, it would be better if you demonstrated that the behaviour is
> broken, rather than asserting it. The fact that a long discussion led to
> the current API is a good hint that it is probably not as broken as you
> make it to be.

Agreed.

The quote/unquote() APIs implement everything you need to
support non-ASCII URLs - they even give you a choice of using
a different encoding for the %-escaped parts of the URL.

Armin, what else do you think you need ?

If you do know the encoding of the URL, then you can easily
convert a bytes URL into a Unicode one.

If not, then it's better to stick to the standards and have the
functions raise an exception if needed.

If you don't want to see errors, use the errors="replace"
error handler or just use "latin-1" as encoding for both
unquote() and quote() - while it's not necessarily correct,
it should get you pretty close to the bytes-behavior you
see in Python 2.x.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 20 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From armin.ronacher at active-4.com  Sun Sep 20 22:21:13 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 20 Sep 2009 22:21:13 +0200
Subject: [stdlib-sig] urllib.parrrrse does not supporrrrt bytes
In-Reply-To: <1253476517.5341.35.camel@localhost>
References: <4AB530BE.1090104@active-4.com>	<bbaeab100909191356l398784few607ced87ae4b3e09@mail.gmail.com>	<4AB5463C.5010505@active-4.com>
	<1253447365.5341.23.camel@localhost>	<4AB678E4.1040704@active-4.com>
	<1253472951.5341.32.camel@localhost>	<4AB67B56.5000408@active-4.com>	<01146B88-B720-444E-A329-E9E7A472B88C@mac.com>	<4AB685B5.6010108@active-4.com>
	<1253476517.5341.35.camel@localhost>
Message-ID: <4AB68EB9.3080704@active-4.com>

Hi,

Antoine Pitrou schrieb:
> You know, it would be better if you demonstrated that the behaviour is
> broken, rather than asserting it. The fact that a long discussion led to
> the current API is a good hint that it is probably not as broken as you
> make it to be.
So if you look at a current version of urllib.parse on Python 3.1 you
can observe the following behavior:

- quote and quote_plus are written to support bytes by forwarding the
  call to

- the unquote and unquote_plus functions do not work with bytes at all.
  You can currently quote to bytes, but don't go the other way.

- None of the URL parsing functions currently work with bytes.

- None of the URL joining functions currently work with bytes.

- You cannot specifiy the URL encoding to urlencode, parse_qs and
  parse_qsl.  Currently you can only decode/encode utf-8 data with
  these functions.


Regards,
Armin

From theller at ctypes.org  Mon Sep 21 13:26:18 2009
From: theller at ctypes.org (Thomas Heller)
Date: Mon, 21 Sep 2009 13:26:18 +0200
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T185838-730@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>	<loom.20090917T173746-561@post.gmane.org>	<4AB2B8E5.1070500@active-4.com>	<loom.20090918T101815-54@post.gmane.org>	<1253263246.5710.6.camel@localhost>	<loom.20090918T104047-293@post.gmane.org>	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>	<loom.20090918T161555-854@post.gmane.org>	<4AB3B375.2010704@ctypes.org>
	<loom.20090918T185838-730@post.gmane.org>
Message-ID: <4AB762DA.6050904@ctypes.org>

Vinay Sajip schrieb:
> > Thomas Heller <theller at ...> writes:
>> >> What I am really missing is a way to do basic logging configuration (calling
>> >> logging.basicConfig(...)) with some command line args for the Python
>> >> interpreter.  I have suggested that before IIRC, but it doesn't seem to fly.
> > 
> > Can you point me to a ticket or post about it?

I've opened a feature request at the Python tracker:

http://bugs.python.org/issue6958

-- 
Thanks,
Thomas

From barry at python.org  Mon Sep 21 16:20:44 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 21 Sep 2009 10:20:44 -0400
Subject: [stdlib-sig] logging
In-Reply-To: <loom.20090918T153911-846@post.gmane.org>
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<BD98B237-A3BA-44C5-8BA9-28305CD56159@python.org>
	<loom.20090918T153911-846@post.gmane.org>
Message-ID: <4954F790-C658-441B-825A-66C100BFB8C7@python.org>

On Sep 18, 2009, at 9:54 AM, Vinay Sajip wrote:

> A StreamHandler (which includes file-based handlers) has a stream  
> attribute
> which is a file-like object. Just pass that to your 3rd-party API.

Thanks Vinay.  If that's part of the public (i.e. supported) API,  
could you add that to the documentation?

> N.B. If it's a file-based handler and you specify a delay argument  
> to the
> handler constructor, the file isn't actually opened until you first  
> log to it,
> so stream will be None. So you might need to be careful about that  
> (by default,
> a file-based handler doesn't delay opening the file).

I see.  Is .stream used internally?  Could it be a property that  
ensures its underlying file is open when its accessed?

Also, if the handler is not StreamHandler, is it insane to want to  
print to it?  IOW, would it make sense to implement .write() on the  
base Handler?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/stdlib-sig/attachments/20090921/c93df036/attachment.pgp>

From mats at sun.com  Sun Sep 27 08:19:13 2009
From: mats at sun.com (Mats Kindahl)
Date: Sun, 27 Sep 2009 08:19:13 +0200
Subject: [stdlib-sig] ConfigParser - parsing options with no value
In-Reply-To: <Pine.LNX.4.64.0909171631030.17020@kimball.webabinitio.net>
References: <4AB26996.8030707@sun.com>
	<bbaeab100909171114h573b0d13k40936bf1c394a4f7@mail.gmail.com>
	<4AB29C02.8080409@sun.com>
	<Pine.LNX.4.64.0909171631030.17020@kimball.webabinitio.net>
Message-ID: <4ABF03E1.3090606@sun.com>



R. David Murray wrote:
> On Thu, 17 Sep 2009 at 22:28, Mats Kindahl wrote:
>> Where should I post such patches and what is people opinion of these to
>> alternatives?
> 
> The bug tracker (bugs.python.org).
> 
>> Another consideration is how to allow this extension. It would be
>> possible to
>> add flags to allow new and old behaviour, but I don't know how much
>> value people
>> attribute to be able to get an exception if there is no value for the
>> option.
> 
> Python's philosophy is to remain backward compatible, so this new
> behavior should be controlled by an option of some sort whose default
> value is the old behavior.

OK. This suggestion for improvement is now registered as Issue 7005. It contain
changes to ConfigParser.py as well as an extended test case. I elected to add a
parameter allow_no_value to __init__ and use the default False. Also, options
without values are registered with the value None, so that they can be
distinguished from options that have the empty string as value.

> 
>> I am also thinking that some option files have extensions to be able
>> to include
>> files (MySQL option parsing package has), so it might be a good idea
>> to add
>> callbacks to handle unrecognized lines, allowing the callback to
>> inject text
>> into the parsing stream to, for example, support include files or even
>> a macro
>> language.
> 
> That should be a separate feature request/tracker issue/patch.

It's on my todo list.

Best wishes,
Mats Kindahl

> 
> --David

-- 
Mats Kindahl
Senior Software Engineer
Database Technology Group
Sun Microsystems

From ben+python at benfinney.id.au  Mon Sep 28 06:03:14 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 28 Sep 2009 14:03:14 +1000
Subject: [stdlib-sig] logging
References: <4AB0D7D6.7030204@active-4.com>
	<4222a8490909160627v4da76637k39373fb1a2a6a788@mail.gmail.com>
	<loom.20090917T173746-561@post.gmane.org>
	<4AB2B8E5.1070500@active-4.com>
	<loom.20090918T101815-54@post.gmane.org>
	<1253263246.5710.6.camel@localhost>
	<loom.20090918T104047-293@post.gmane.org>
	<4222a8490909180657i1abaddb7o1545e18d0a159d96@mail.gmail.com>
	<79990c6b0909180821y688af122qed5eaee18efc368c@mail.gmail.com>
	<loom.20090918T180635-509@post.gmane.org>
Message-ID: <878wfz4sxp.fsf@benfinney.id.au>

Vinay Sajip <vinay_sajip at yahoo.co.uk>
writes:

> Paul Moore <p.f.moore at ...> writes:
> > Oh, and finally I *hate* Java-style camelCase [?]
>
> Okay, I've commented elsewhere that I'm happy to provide
> unix_style_underscored_names.

I think you mean ?python_pep8_style_underscored_names? :-)

How would these names be provided? As simple ?fooName = foo_name?? Or
would the PEP-8 one be preferred, deprecating the non-conformant name?

-- 
 \        ?I filled my humidifier with wax. Now my room is all shiny.? |
  `\                                                    ?Steven Wright |
_o__)                                                                  |
Ben Finney