From ben+python at benfinney.id.au  Thu Oct  1 00:18:57 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 01 Oct 2009 08:18:57 +1000
Subject: [Python-Dev] Python 2.6.3rc1 available
References: <B132EC23-B905-4BEC-B845-5E19BD4BE9A1@python.org>
Message-ID: <87pr98yt2m.fsf@benfinney.id.au>

Barry Warsaw <barry at python.org> writes:

> The first (and hopefully last) release candidate for Python 2.6.3 is
> now available
[?]

Thank you for this announcement, and the efforts that go into making
this work available.

*Especially* thank you for avoiding the oxymoronic ?Released: 2.6.3
release candidate? or similar. I hope this signifies a trend toward
using the more accurate term ?available? for announcing availability of
anything that's not an actual release.

-- 
 \     ?Guaranteed to work throughout its useful life.? ?packaging for |
  `\                                          clockwork toy, Hong Kong |
_o__)                                                                  |
Ben Finney
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091001/82bb11f6/attachment.pgp>

From amcnabb at mcnabbs.org  Thu Oct  1 00:58:00 2009
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Wed, 30 Sep 2009 16:58:00 -0600
Subject: [Python-Dev] PEP 389: argparse - new command line parsing	module
In-Reply-To: <d11dcfba0909301440r35d30edbk584c8ee6958c7df0@mail.gmail.com>
References: <d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<loom.20090930T181855-247@post.gmane.org>
	<ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>
	<ha02nc$gtt$1@ger.gmane.org>
	<212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org>
	<20090930201735.GD27796@mcnabbs.org>
	<d11dcfba0909301440r35d30edbk584c8ee6958c7df0@mail.gmail.com>
Message-ID: <20090930225800.GF27796@mcnabbs.org>

On Wed, Sep 30, 2009 at 02:40:20PM -0700, Steven Bethard wrote:
> 
> > Also, is it possible to add these subparsers dynamically? ?For example,
> > you would want to be able to load a module immediately after parsing the
> > name instead of having to keep a predetermined list of all module names.
> > I'm pretty sure that bzr dynamically loads modules this way. ?Can
> > argparse help with this?
> 
> You can probably already do this. I'm not 100% sure what you want to
> do, but it's certainly possible to define an argparse.Action that
> loads a module when it's invoked. It might look something like::
> 
>     class MyAction(argparse.Action):
>         def __call__(self, parser, namespace, value, option_string=None):
>             mod = __import__(value) # or whatever

This looks much easier than what I was able to do in optparse.  Cool.

-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868

From vinay_sajip at yahoo.co.uk  Thu Oct  1 01:03:16 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Sep 2009 23:03:16 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
Message-ID: <loom.20091001T010253-42@post.gmane.org>

Steven Bethard <steven.bethard <at> gmail.com> writes:

> There's a lot of code already out there (in the standard library and
> other places) that uses %-style formatting, when in Python 3.0 we
> should be encouraging {}-style formatting. We should really provide
> some sort of transition plan. Consider an example from the logging
> docs:
> 
> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
> 
> We'd like to support both this style as well as the following style:
> 
> logging.Formatter("{asctime} - {name} - {levelname} - {message}")
> 

In logging at least, there are two different places where the formatting issue
crops up.

The first is creating the "message" part of the the logging event, which is
made up of a format string and arguments.

The second is the one Steven's mentioned: formatting the message along with
other event data such as time of occurrence, level, logger name etc. into the
final text which is output.

Support for both % and {} forms in logging would need to be considered in
these two places. I sort of liked Martin's proposal about using different
keyword arguments, but apart from the ugliness of "dicttemplate" and the fact
that "fmt" is already used in Formatter.__init__ as a keyword argument, it's
possible that two different keyword arguments "fmt" and "format" both referring
to format strings might be confusing to some users.

Benjamin's suggestion of providing a flag to Formatter seems slightly better,
as it doesn't change what existing positional or keyword parameters do, and
just adds an additional, optional parameter which can start off with a default
of False and transition to a default of True.

However, AFAICT these approaches only cover the second area where formatting
options are chosen - not the creation of the message from the parameters passed
to the logging call itself. 

Of course one can pass arbitrary objects as messages which contain their own
formatting logic. This has been possible since the very first release but I'm
not sure that it's widely used, as it's usually easier to pass strings. So
instead of passing a string and arguments such as

logger.debug("The %s is %d", "answer", 42)

one can currently pass, for a fictitious class PercentMessage,

logger.debug(PercentMessage("The %s is %d", "answer", 42))

and when the time comes to obtain the formatted message, LogRecord.getMessage
calls str() on the PercentMessage instance, whose __str__ will use %-formatting
to get the actual message.

Of course, one can also do for example

logger.debug(BraceMessage("The {} is {}", "answer", 42))

where the __str__() method on the BraceMessage will do {} formatting.

Of course, I'm not suggesting we actually use the names PercentMessage and
BraceMessage, I've just used them there for clarity.

Also, although Raymond has pointed out that it seems likely that no one ever
needs *both* types of format string, what about the case where application A
depends on libraries B and C, and they don't all share the same preferences
regarding which format style to use? ISTM no-one's brought this up yet, but it
seems to me like a real issue. It would certainly appear to preclude any
approach that configured a logging-wide or logger-wide flag to determine how to
interpret the format string.

Another potential issue is where logging events are pickled and sent over
sockets to be finally formatted and output on different machines. What if a
sending machine has a recent version of Python, which supports {} formatting,
but a receiving machine doesn't? It seems that at the very least, it would
require a change to SocketHandler and DatagramHandler to format the "message"
part into the LogRecord before pickling and sending. While making this change
is simple, it represents a potential backwards-incompatible problem for users
who have defined their own handlers for doing something similar.

Apart from thinking through the above issues, the actual formatting only
happens in two locations - LogRecord.getMessage and Formatter.format - so
making the code do either %- or {} formatting would be simple, as long as it
knows which of % and {} to pick.

Does it seems too onerous to expect people to pass an additional "use_format"
keyword argument with every logging call to indicate how to interpret the
message format string? Or does the PercentMessage/BraceMessage type approach
have any mileage? What do y'all think?

Regards,

Vinay Sajip



From tjreedy at udel.edu  Thu Oct  1 01:47:21 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 30 Sep 2009 19:47:21 -0400
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
References: <20090929202035.GA5492@gmail.com>	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
	<9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
Message-ID: <ha0qm9$g9$1@ger.gmane.org>

Yuvgoog Greenle wrote:
> I like how python has a minimalistic and powerful syntax (-1 for the 
> break ___ PEP).
> 
> Also, I really dislike the for/else ambiguity "butterflies".

Properly understood, no ambiguity.

while c:
   x()

is equivalent to hypothetical

label z:
if c:
   x()
   goto: z

So

while c:
   x()
else:
   y()

is equivalent to

label 1:
if c:
   x()
   goto: 1
else"
   y()

The else clause fires (once) if and when the if/while condition 
evaluates as false. Break and continue are restricted *unconditional* 
goto statements, and so *cannot* trigger an else clause.

In for loops, the implied condition is 'there is another item in the 
collection represented by the iterable'.

For any more, move to another list.

Terry Jan Reedy


From brett at python.org  Thu Oct  1 02:29:04 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 30 Sep 2009 17:29:04 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T010253-42@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<loom.20091001T010253-42@post.gmane.org>
Message-ID: <bbaeab100909301729o2e1b11barb5690e5a09184a38@mail.gmail.com>

On Wed, Sep 30, 2009 at 16:03, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Steven Bethard <steven.bethard <at> gmail.com> writes:
>
>> There's a lot of code already out there (in the standard library and
>> other places) that uses %-style formatting, when in Python 3.0 we
>> should be encouraging {}-style formatting. We should really provide
>> some sort of transition plan. Consider an example from the logging
>> docs:
>>
>> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
>>
>> We'd like to support both this style as well as the following style:
>>
>> logging.Formatter("{asctime} - {name} - {levelname} - {message}")
>>
>
> In logging at least, there are two different places where the formatting issue
> crops up.
>
> The first is creating the "message" part of the the logging event, which is
> made up of a format string and arguments.
>
> The second is the one Steven's mentioned: formatting the message along with
> other event data such as time of occurrence, level, logger name etc. into the
> final text which is output.
>
> Support for both % and {} forms in logging would need to be considered in
> these two places. I sort of liked Martin's proposal about using different
> keyword arguments, but apart from the ugliness of "dicttemplate" and the fact
> that "fmt" is already used in Formatter.__init__ as a keyword argument, it's
> possible that two different keyword arguments "fmt" and "format" both referring
> to format strings might be confusing to some users.
>
> Benjamin's suggestion of providing a flag to Formatter seems slightly better,
> as it doesn't change what existing positional or keyword parameters do, and
> just adds an additional, optional parameter which can start off with a default
> of False and transition to a default of True.
>
> However, AFAICT these approaches only cover the second area where formatting
> options are chosen - not the creation of the message from the parameters passed
> to the logging call itself.
>
> Of course one can pass arbitrary objects as messages which contain their own
> formatting logic. This has been possible since the very first release but I'm
> not sure that it's widely used, as it's usually easier to pass strings. So
> instead of passing a string and arguments such as
>
> logger.debug("The %s is %d", "answer", 42)
>
> one can currently pass, for a fictitious class PercentMessage,
>
> logger.debug(PercentMessage("The %s is %d", "answer", 42))
>
> and when the time comes to obtain the formatted message, LogRecord.getMessage
> calls str() on the PercentMessage instance, whose __str__ will use %-formatting
> to get the actual message.
>
> Of course, one can also do for example
>
> logger.debug(BraceMessage("The {} is {}", "answer", 42))
>
> where the __str__() method on the BraceMessage will do {} formatting.
>
> Of course, I'm not suggesting we actually use the names PercentMessage and
> BraceMessage, I've just used them there for clarity.
>
> Also, although Raymond has pointed out that it seems likely that no one ever
> needs *both* types of format string, what about the case where application A
> depends on libraries B and C, and they don't all share the same preferences
> regarding which format style to use? ISTM no-one's brought this up yet, but it
> seems to me like a real issue. It would certainly appear to preclude any
> approach that configured a logging-wide or logger-wide flag to determine how to
> interpret the format string.
>
> Another potential issue is where logging events are pickled and sent over
> sockets to be finally formatted and output on different machines. What if a
> sending machine has a recent version of Python, which supports {} formatting,
> but a receiving machine doesn't? It seems that at the very least, it would
> require a change to SocketHandler and DatagramHandler to format the "message"
> part into the LogRecord before pickling and sending. While making this change
> is simple, it represents a potential backwards-incompatible problem for users
> who have defined their own handlers for doing something similar.
>
> Apart from thinking through the above issues, the actual formatting only
> happens in two locations - LogRecord.getMessage and Formatter.format - so
> making the code do either %- or {} formatting would be simple, as long as it
> knows which of % and {} to pick.
>
> Does it seems too onerous to expect people to pass an additional "use_format"
> keyword argument with every logging call to indicate how to interpret the
> message format string? Or does the PercentMessage/BraceMessage type approach
> have any mileage? What do y'all think?

I personally prefer the keyword argument approach to act as a flag,
but that's me.

As for the PercentMessage/BraceMessage, I would make sure that you
just simply take the string format and simply apply the arguments
later to cut down on the amount of parentheses butting up against each
other: ``logger.debug(BraceMessage("The {} is {}"), "answer", 42)``.
It's still an acceptable solution that provides a clear transition:
simply provide the two classes, deprecate PercentMessage or bare
string usage, require BraceMessage, remove requirement. This wrapper
approach also provides a way for libraries that have not shifted over
to still work with PEP 3101 strings by letting the user wrap the
string to be interpolated themselves and then to pass it in to the
libraries. It's just unfortunate that any transition would have this
cost of wrapping all strings for a while. I suspect most people will
simply import the wrapping class and give it some short name like
people do with gettext.

-Brett

From solipsis at pitrou.net  Thu Oct  1 02:47:09 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 1 Oct 2009 00:47:09 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
Message-ID: <loom.20091001T024614-990@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
> 
> Does it seems too onerous to expect people to pass an additional "use_format"
> keyword argument with every logging call to indicate how to interpret the
> message format string? Or does the PercentMessage/BraceMessage type approach
> have any mileage? What do y'all think?

What about the proposal I made earlier?
(support for giving a callable, so that you pass the "{foobar}".format method
when you want new-style formatting)




From stephen at xemacs.org  Thu Oct  1 04:53:25 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 01 Oct 2009 11:53:25 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <eae285400909301459y445d021fh254a129eb34fc578@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
	<Pine.LNX.4.64.0909301618550.18193@kimball.webabinitio.net>
	<eae285400909301459y445d021fh254a129eb34fc578@mail.gmail.com>
Message-ID: <87fxa3keoq.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Stutzbach writes:

 > People have voiced support for the IPNetwork class

The obvious use cases are 

1.  parsing and validating strings that purport to describe networks,
2.  providing a received for the result of a function that deduces the
    network from an arbitrary address and a mask
3.  (controversial) providing the function in (2) as a method or
    constructor
4.  holding certain information associated with a network such as
    a. broadcast address
    b. (controversial) the arbitrary address from (2) (I count this as
       obvious because I'm a packrat and hate to throw away any data,
       but really if generated as in 2 I see no use case)
5.  providing an iterator for the addresses in the network.

Note that (1) and (2) have analogous use cases for IPv?Address.

 > and for the occasional utility of an .ip field.  I assume they have
 > good use cases.

[Please don't do that, not even ironically.  90% of the heat in this
thread has come because people discussed proposed syntax without
agreeing on a common use case to compare the semantics to.]

The only use case I can imagine for .ip is to represent a single
gateway:

def make_gateway(network, gateway):
    gw = IPv4Network(network)            # eg network = '192.168.1.0/0'
    gw.ip = IPv4Address(gateway).ip      # eg gateway = '192.168.2.1'
    return gw

Note that this usage *cannot* be served by parsing "denormalized
network descriptions" as the gateway address (ie, make_gateway cannot
be a one-argument function).  I did think about a use case like that
for DNS SRV records, but servers are looked up by service, not by
network, in most cases.  So gateways are really the only service that
fits this model.

Also, it seems to me that this usage is well-served by the change to
the definition of __eq__ to compare .network and .prefixlen, viz.

def find_gateway_address(network, gateway_list):
    "raises ValueError if no gateway known for network"
    # it's interesting that there's no sequence method for this
    return gateway_list[gateway_list.index(network)].ip

 > I worry that this discussion has focused too much on the details of
 > ipaddr (and the false dichotomy of "ipaddr versus nothing"),

It's not a false dichotomy; this PEP proposes adding ipaddr, not any
other library.  Of course if ipaddr needs additional functionality, it
should be added before approval.  But you seem to question whether any
of the functionality is necessary, and I don't recall any posts of the
form, "this is really sweet, but it needs salt".

 > without properly tackling the question of "What use-cases for IP
 > addresses are sufficiently universal* that they belong in the
 > standard library?"

I think that the use cases given above are pretty universal.  The
string parsing cases alone give sufficient rationale for existence of
at least one class, and since addresses and networks are conceptually
different (for me, anyway), I am happy to have two different classes.


From vinay_sajip at yahoo.co.uk  Thu Oct  1 08:15:35 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 06:15:35 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
Message-ID: <loom.20091001T075736-620@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> Why not allow logging.Formatter to take a callable, which would in turn call
> the callable with keyword arguments?
> 
> Therefore, you could write:
>    logging.Formatter("{asctime} - {name} - {level} - {msg}".format)
> 
> and then:
>    logging.critical(name="Python", msg="Buildbots are down")

This seems perhaps usable for a Formatter instantiation (infrequent) but a
problem for the case where you want to convert format_str + args -> message
(potentially frequent, and less readable). Another problem is that logging
calls already use keyword arguments (extra, exc_info) and so backward
compatibility might be compromised. It also feels like passing a callable could
encourage patterns of usage which restrict our flexibility for future changes:
we want for now to just allow choosing between % and {}, but a callable can do
anything. That's more flexible, to be sure, but more specialized formatting
requirements are already catered for using e.g. the PercentMessage/BraceMessage
approach.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Thu Oct  1 08:51:34 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 06:51:34 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<5353D5BD-1862-42C2-913F-C75AD73002DC@python.org>
Message-ID: <loom.20091001T081826-133@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:

> 
> This is a very interesting idea.
> 
> Note that one of the reasons to /at least/ support {}-strings also is  
> that %-strings are simply too error prone in many situations.  For  
> example, if I decide to support internationalization of log format  
> strings, and all I can use is %-strings, it's almost guaranteed that I  
> will have bugs because a translator forgot the trailing 's'.  This  
> exactly the motivation that led to PEP 292 $-strings.
> 
> In fact, while we're at it, it would be kind of cool if I could use $- 
> strings in log templates.  Antoine's idea of accepting a callable  
> might fit that bill nicely.

You're already covered if you use the PercentMessage/BraceMessage approach I
mentioned elsewhere in this thread. Suppose:

#Just typing this in, it's not tested or anything
class DollarMessage:
    def __init__(self, fmt, *args, **kwargs):
        self.fmt = fmt
        self.args = args
        self.kwargs = kwargs

    def __str__(self):
        return string.Template(self.fmt).substitute(*args, **kwargs)




From vinay_sajip at yahoo.co.uk  Thu Oct  1 08:59:40 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 06:59:40 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<5353D5BD-1862-42C2-913F-C75AD73002DC@python.org>
	<loom.20091001T081826-133@post.gmane.org>
Message-ID: <loom.20091001T085454-26@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:

> #Just typing this in, it's not tested or anything
> class DollarMessage:
>     def __init__(self, fmt, *args, **kwargs):
>         self.fmt = fmt
>         self.args = args
>         self.kwargs = kwargs
> 
>     def __str__(self):
>         return string.Template(self.fmt).substitute(*args, **kwargs)
> 

Whoops, sorry, pressed the "post" button by accident on my previous post. The
above substitute call should of course say

string.Template(self.fmt).substitute(*self.args, **self.kwargs)

and you can alias DollarMessage (or whatever name you choose) as _ or __, say.

As far as the Formatter formatting goes, it's easy enough to subclass Formatter
to format using whatever approach you want.

Regards,

Vinay Sajip




From vinay_sajip at yahoo.co.uk  Thu Oct  1 09:07:52 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 07:07:52 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<bbaeab100909301729o2e1b11barb5690e5a09184a38@mail.gmail.com>
Message-ID: <loom.20091001T090100-950@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> As for the PercentMessage/BraceMessage, I would make sure that you
> just simply take the string format and simply apply the arguments
> later to cut down on the amount of parentheses butting up against each
> other: ``logger.debug(BraceMessage("The {} is {}"), "answer", 42)``.

The problem with that is that BraceMessage.__str__() wouldn't know what
arguments to use to produce the message.

> cost of wrapping all strings for a while. I suspect most people will
> simply import the wrapping class and give it some short name like
> people do with gettext.
> 

Yes,

logger.debug(__("The {} is {}", "answer", 42))

isn't ideal but perhaps liveable with. And hopefully with a decent editor, the
paren-butting annoyance will be minimized.

Regards,

Vinay Sajip




From scott+python-dev at scottdial.com  Thu Oct  1 09:36:26 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 01 Oct 2009 03:36:26 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
Message-ID: <4AC45BFA.3020802@scottdial.com>

Barry Warsaw wrote:
> If not, I'll try to
> spend some time over the next few days looking at outstanding bugs, and
> marking release blockers, etc.

I'd like to draw attention to:

http://bugs.python.org/issue5949

Which is a regression of imaplib.py introduced in r57680.

Ever since I switched to python 2.6 on my box, I have had issues with
getmail[1] getting stuck in an infinite loop swallowing memory (note:
only applies to IMAP SSL connections). While this code is present in
older versions of python, it seems to have become a problem recently
(2009-05-06 is the earliest report on the issue) perhaps due to a
version bump of OpenSSL? I never noticed the problem in python2.5 even
though the code is unchanged. Nevertheless, the code is clearly wrong
and should be corrected.

I would appreciate this bug being resolved before the next release as it
effects me on a daily basis. I have submitted a patch, which reflects my
local solution.

-Scott

[1] http://pyropus.ca/software/getmail/

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From scott+python-dev at scottdial.com  Thu Oct  1 09:55:28 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 01 Oct 2009 03:55:28 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC45BFA.3020802@scottdial.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AC45BFA.3020802@scottdial.com>
Message-ID: <4AC46070.1010706@scottdial.com>

Scott Dial wrote:
> While this code is present in
> older versions of python, it seems to have become a problem recently
> (2009-05-06 is the earliest report on the issue) perhaps due to a
> version bump of OpenSSL? I never noticed the problem in python2.5 even
> though the code is unchanged.

To answer my own question, this was introduced in r64578 in ssl.py by
the addition of the suppress_ragged_eofs feature (seems to originate
with issue1251). While it is a change in the behavior of the ssl module,
the change falls in line with other file-like objects and their handling
of EOF, so the bug still falls to imaplib.

In other words, this bug appeared in 2.6 and 3.0, and is present in all
subsequent versions.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From p.f.moore at gmail.com  Thu Oct  1 10:58:59 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 1 Oct 2009 09:58:59 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <ha01t7$e2j$1@ger.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<ha01t7$e2j$1@ger.gmane.org>
Message-ID: <79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>

2009/9/30 Robert Kern <robert.kern at gmail.com>:
> I am blissfully unaware of the problems Paul mentioned about Windows GUI-mode programs.

:-)

> I'm not sure what would make a program "GUI-mode" or not. Certainly, I have written
> Python programs that use wxPython and PyQt on Windows that print to stdout/stderr,
> and they appear to work fine.

It's the difference between using python.exe and pythonw.exe. There's
a flag in the executable header saying whether the executable is
"console mode" or "gui mode".

GUI mode programs are run (by the OS) without a console (or if run
from a console prompt, they automatically detach from that console,
much like Unix programs which fork themselves to leave the terminal
group (did I get the terminology right?) but done by the OS). As a
result, the program has no valid stdin/out/err handles. Any attempt to
write to them causes the program to crash.

Traceback (most recent call last):
  File "hello.py", line 13, in <module>
    main()
  File "hello.py", line 7, in main
    sys.stdout.flush()
IOError: [Errno 9] Bad file descriptor

(Question - is it *ever* possible for a Unix program to have invalid
file descriptors 0,1 and 2? At startup - I'm assuming anyone who does
os.close(1) knows what they are doing!)

Paul.

From mal at egenix.com  Thu Oct  1 11:37:26 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 01 Oct 2009 11:37:26 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC33DE1.3040304@trueblade.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<4AC2CD65.30905@v.loewis.de>
	<4AC33DE1.3040304@trueblade.com>
Message-ID: <4AC47856.2090307@egenix.com>

Eric Smith wrote:
> Martin v. L?wis wrote:
>> Steven Bethard wrote:
>>> There's a lot of code already out there (in the standard library and
>>> other places) that uses %-style formatting, when in Python 3.0 we
>>> should be encouraging {}-style formatting. 
>>
>> I don't agree that we should do that. I see nothing wrong with using
>> % substitution.

I agree with Martin.

Both approaches have their ups and downs, but forcing users to move
from %-formatting to .format()-formatting will just frustrate them:
having to convert several thousand such (working) uses in their code
with absolutely no benefit simply doesn't look like a good way to
spend your time.

In addition to the code changes, such a move would also render
existing translations of the %-formatted string templates useless.

> It's a maintenance burden. There are several outstanding bugs with it,
> admittedly not of any great significance. I've been putting time into
> fixing at least one of them. When Mark and I did short-float-repr, at
> least half of my time was consumed with %-formatting, mostly because of
> how it does memory management.

Why not allow both and use .format() for those cases where %-formatting
doesn't work too well ?

> On the plus side, %-formatting is (and always will be) faster than
> str.format(). Its very limitations make it possible for it to be fast.
> 
> I'd note that PEP 3101 calls str.format() a replacement for
> %-formatting, not an alternate mechanism to achieve the same end.

I think that's a wording we should change.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 01 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From floris.bruynooghe at gmail.com  Thu Oct  1 11:38:25 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 1 Oct 2009 10:38:25 +0100
Subject: [Python-Dev] OT: detatching from terminal (was: Re: PEP 389:
 argparse - new command line parsing module)
In-Reply-To: <79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
References: <d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<ha01t7$e2j$1@ger.gmane.org>
	<79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
Message-ID: <20091001093825.GA8945@laurie.devork>

On Thu, Oct 01, 2009 at 09:58:59AM +0100, Paul Moore wrote:
> (Question - is it *ever* possible for a Unix program to have invalid
> file descriptors 0,1 and 2? At startup - I'm assuming anyone who does
> os.close(1) knows what they are doing!)

Normally you don't close fd 0, 1 & 2 but rather redirect them to
/dev/null (a black hole).  That saves you from nastiness when a
library or something misbahaves and tries to use one of those.  It
would have been nice if windows GUI-mode programs as you describe them
would do something similar.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From jon+python-dev at unequivocal.co.uk  Thu Oct  1 12:39:38 2009
From: jon+python-dev at unequivocal.co.uk (Jon Ribbens)
Date: Thu, 1 Oct 2009 11:39:38 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing	module
In-Reply-To: <79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
References: <d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<ha01t7$e2j$1@ger.gmane.org>
	<79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
Message-ID: <20091001103938.GI642@snowy.squish.net>

On Thu, Oct 01, 2009 at 09:58:59AM +0100, Paul Moore wrote:
> (Question - is it *ever* possible for a Unix program to have invalid
> file descriptors 0,1 and 2? At startup - I'm assuming anyone who does
> os.close(1) knows what they are doing!)

Yes, at startup you just have the file descriptors your parent process
gave you. This may or may not include 0, 1, and 2, and may or may not
include other descriptors too.

I seem to recall there have been security holes caused in the past by
people assuming 0-2 will exist.

From solipsis at pitrou.net  Thu Oct  1 12:48:44 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 1 Oct 2009 10:48:44 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
Message-ID: <loom.20091001T124105-392@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
> 
> This seems perhaps usable for a Formatter instantiation (infrequent) but a
> problem for the case where you want to convert format_str + args -> message
> (potentially frequent, and less readable).

Why is it a problem? I don't understand. It certainly is less pleasant to write
"{foo}".format or "{0} {1}".format than it is to write "{0} {1}" alone, but it's
still prettier and easier to remember than the special wrappers people are
proposing here.

> Another problem is that logging
> calls already use keyword arguments (extra, exc_info) and so backward
> compatibility might be compromised.

Then logging can just keep recognizing those special keyword arguments, and
forward the others to the formatting function.

> It also feels like passing a callable could
> encourage patterns of usage which restrict our flexibility for future changes:

Which future changes are you thinking about? AFAIK, there hasn't been a single
change in logging output formatting in years. Rejecting a present change on the
basis that it "restricts our flexibility for future changes" sounds like the
worst kind of argument to me :-)

> That's more flexible, to be sure, but more specialized formatting
> requirements are already catered for using e.g. the PercentMessage/BraceMessage
> approach.

Except that having to wrap format strings with "PercentMessage" or
"BraceMessage" is horrible. Python is not Java.

Regards

Antoine.



From ncoghlan at gmail.com  Thu Oct  1 13:09:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 01 Oct 2009 21:09:49 +1000
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC45BFA.3020802@scottdial.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AC45BFA.3020802@scottdial.com>
Message-ID: <4AC48DFD.7020607@gmail.com>

Scott Dial wrote:
> I would appreciate this bug being resolved before the next release as it
> effects me on a daily basis. I have submitted a patch, which reflects my
> local solution.

Unfortunately, it's almost certainly too late to get this into 2.6.3. It
really needed to be brought up back when Barry called for identification
of significant 2.6 bugs and patches rather than after the RC had already
been released.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Thu Oct  1 14:21:56 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 1 Oct 2009 08:21:56 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T081826-133@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<5353D5BD-1862-42C2-913F-C75AD73002DC@python.org>
	<loom.20091001T081826-133@post.gmane.org>
Message-ID: <034CA2BD-6DA0-43C4-A52C-66DD3885EBF4@python.org>

On Oct 1, 2009, at 2:51 AM, Vinay Sajip wrote:

> You're already covered if you use the PercentMessage/BraceMessage  
> approach I
> mentioned elsewhere in this thread. Suppose:
>
> #Just typing this in, it's not tested or anything
> class DollarMessage:
>    def __init__(self, fmt, *args, **kwargs):
>        self.fmt = fmt
>        self.args = args
>        self.kwargs = kwargs
>
>    def __str__(self):
>        return string.Template(self.fmt).substitute(*args, **kwargs)

Right, thanks.  This is a very cool feature that I didn't know logging  
supported!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091001/90caaca9/attachment.pgp>

From barry at python.org  Thu Oct  1 14:25:56 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 1 Oct 2009 08:25:56 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC48DFD.7020607@gmail.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AC45BFA.3020802@scottdial.com> <4AC48DFD.7020607@gmail.com>
Message-ID: <7A758FD5-5052-4E9F-B164-397EF19947A3@python.org>

On Oct 1, 2009, at 7:09 AM, Nick Coghlan wrote:

> Scott Dial wrote:
>> I would appreciate this bug being resolved before the next release  
>> as it
>> effects me on a daily basis. I have submitted a patch, which  
>> reflects my
>> local solution.
>
> Unfortunately, it's almost certainly too late to get this into  
> 2.6.3. It
> really needed to be brought up back when Barry called for  
> identification
> of significant 2.6 bugs and patches rather than after the RC had  
> already
> been released.

Right.  Scott, you can get reviews and support for the patch now so  
that the bug can be addressed in 2.6.4.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091001/2f9b457b/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Thu Oct  1 14:42:38 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 12:42:38 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
Message-ID: <loom.20091001T140503-467@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> 
> Why is it a problem? I don't understand. It certainly is less pleasant to
> write "{foo}".format or "{0} {1}".format than it is to write "{0} {1}" alone,
> but it's still prettier and easier to remember than the special wrappers
> people are proposing here.

Well, it's less readable, as I said in parentheses. It would work, of course.
And the special wrappers needn't be too intrusive:

__ = BraceMessage

logger.debug(__("Message with {0} {1}", 1, "argument"))

> Then logging can just keep recognizing those special keyword arguments, and
> forward the others to the formatting function.

It just means that you can't pass those values through, and what if some of
them are used somewhere in existing code?

> > It also feels like passing a callable could
> > encourage patterns of usage which restrict our flexibility for future
> > changes:
> 
> Which future changes are you thinking about? AFAIK, there hasn't been a

It's the Rumsfeldian "We don't know what we don't know" ;-)

> single change in logging output formatting in years. Rejecting a present
> change on the basis that it "restricts our flexibility for future changes"
> sounds like the worst kind of argument to me 

Now don't get upset and take it as a rejection, as we're still in the
kicking-ideas-around stage ;-) I'm just saying how it feels to me.

I agree that logging output formatting hasn't changed in years, and that's
because there's been no particular need for it to change (some changes *were*
made in the very early days to support a single dict argument). Now that time
for change has perhaps come.

I'm just trying to think ahead, and can't claim to have got a definitive answer
up my sleeve. Passing a callable has upsides and downsides, and ISTM it's
always worth focusing on the downsides to make sure they don't come back and
bite you later. I don't foresee any specific problem - I'm just uneasy about
it.

> Except that having to wrap format strings with "PercentMessage" or
> "BraceMessage" is horrible. Python is not Java.

Amen. I'd say "Yeccchh!" too, if it literally had to be like that. And I also
note that there are voices here saying that support for %-formatting shouldn't,
or doesn't need to, change, at least until Python 4.0.

So consider the following tentative suggestion, which is off the top of my head
and offered as a discussion point:

Suppose that if you want to use %-formatting, everything stays as is. No
backward-compatibility headaches.

To support {}-formatting, add an extra class which I've called BraceMessage.
Consider this name a "working title", as no doubt a better name will suggest
itself, but for now this name makes it clear what we're talking about.

If any module wants to use {} formatting for their logging, they can add the
line

from logging import BraceMessage as __

I've used two underscores, since _ might be being used for gettext, but
obviously the importer can use whatever name they want.

and then they can use

logger.debug(__("The {0} is {1}", "answer", 42))

which I think is more readable than putting in ".format" following the string
literal. It's not a *huge* point, perhaps, but "Readability counts".

This has the side benefit that if e.g. Barry wanted to use string.Template for
formatting, he's just got to replace the above import with something like

from logging import DollarMessage as __

Another "working title", please note. And while I've shown these classes being
imported from logging, it doesn't make sense to put them there if this idea
were to fly in a more general context. Then, perhaps string would be a better
home for these classes.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Thu Oct  1 15:07:50 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 1 Oct 2009 13:07:50 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
Message-ID: <loom.20091001T150010-10@post.gmane.org>


Hello,

> Well, it's less readable, as I said in parentheses. It would work, of course.
> And the special wrappers needn't be too intrusive:
> 
> __ = BraceMessage
> 
> logger.debug(__("Message with {0} {1}", 1, "argument"))

Ah, I hadn't thought about that. It looks a bit less awful indeed.
I'm of the opinion, however, that %-formatting should remain the default and
shouldn't need a wrapper.

There's another possibility, which is to build the wrapping directly around the
logger. That is, if I want a %-style logger, I do:

   logger = logging.getLogger("smtp")
   logger.debug("incoming email from %s", sender_address)

and I want a {}-style logger, I do:

   logger = logging.getLogger("smtp", style="{}")
   logger.debug("incoming email from {addr}", addr=sender_address)

(of course, different users of the "smtp" logger can request different
formatting styles when calling getLogger().)

We could combine the various proposals to give users flexible APIs. Of course,
it generally smells of "there's more than one way to do it".

> It's the Rumsfeldian "We don't know what we don't know" 

Is this guy in the Python community? :-)

> I'm just trying to think ahead, and can't claim to have got a definitive answer
> up my sleeve.

Sure, we have some time until 2.7/3.2 anyway.

Regards

Antoine.



From p.f.moore at gmail.com  Thu Oct  1 15:11:27 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 1 Oct 2009 14:11:27 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T140503-467@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
Message-ID: <79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>

2009/10/1 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> If any module wants to use {} formatting for their logging, they can add the
> line
>
> from logging import BraceMessage as __
>
> I've used two underscores, since _ might be being used for gettext, but
> obviously the importer can use whatever name they want.
>
> and then they can use
>
> logger.debug(__("The {0} is {1}", "answer", 42))
>
> which I think is more readable than putting in ".format" following the string
> literal. It's not a *huge* point, perhaps, but "Readability counts".
>
> This has the side benefit that if e.g. Barry wanted to use string.Template for
> formatting, he's just got to replace the above import with something like
>
> from logging import DollarMessage as __
>
> Another "working title", please note. And while I've shown these classes being
> imported from logging, it doesn't make sense to put them there if this idea
> were to fly in a more general context. Then, perhaps string would be a better
> home for these classes.

This seems to me to be almost the same as the previous suggestion of
having a string subclass:

class BraceFormatter(str):
    def __mod__(self, other):
        # Needs more magic here to cope with dict argument
        return self.format(*other)

__ = BraceFormatter

logger.debug(__("The {0} is {1}"), "answer", 42)

The only real differences are

1. The positioning of the closing parenthesis
2. The internal implementation of logger.debug needs to preserve
string subclasses properly

But the benefit is that the approach allows anyone to use brace
formatting in any API that currently accepts % format (assuming string
subclasses don't get mangled).

On the one hand, I'd prefer a more general solution. On the other, I'm
nervous about that "assuming string subclasses..." proviso.

I've no real answer, just offering the point up for consideration.

Paul.

From vinay_sajip at yahoo.co.uk  Thu Oct  1 15:29:29 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 13:29:29 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
Message-ID: <loom.20091001T151606-886@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> This seems to me to be almost the same as the previous suggestion of
> having a string subclass:
> 
> class BraceFormatter(str):
>     def __mod__(self, other):
>         # Needs more magic here to cope with dict argument
>         return self.format(*other)
> 
> __ = BraceFormatter
> 
> logger.debug(__("The {0} is {1}"), "answer", 42)
> 
> The only real differences are
> 
> 1. The positioning of the closing parenthesis
> 2. The internal implementation of logger.debug needs to preserve
> string subclasses properly
> 

The other difference is that my suggestion supports Barry's desire to use
string.Template with no muss, no fuss ;-) Plus, very little additional work is
required compared to your items 1 and 2. ISTM BraceMessage would be something
like this,

clsss BraceMessage:
    def __init__(self, fmt, *args, **kwargs):
        self.fmt = fmt
        self.args = args
        self.kwargs = kwargs

    def __str__(self):
        return self.fmt.format(*self.args, **self.kwargs)


Regards,

Vinay


From vinay_sajip at yahoo.co.uk  Thu Oct  1 15:39:04 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 13:39:04 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
Message-ID: <loom.20091001T153627-621@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> 2. The internal implementation of logger.debug needs to preserve
> string subclasses properly
> 
> But the benefit is that the approach allows anyone to use brace
> formatting in any API that currently accepts % format (assuming string
> subclasses don't get mangled).
> 
> On the one hand, I'd prefer a more general solution. On the other, I'm
> nervous about that "assuming string subclasses..." proviso.
> 

You're right to be nervous: it's not hard to mangle subtypes.

>>> class mystr(str): pass
...
>>> s = mystr("Abc")
>>> s
'Abc'
>>> type(s)
<class '__main__.mystr'>
>>> s2 = s.replace("A", "a")
>>> s2
'abc'
>>> type(s2)
<type 'str'>
>>>


From vinay_sajip at yahoo.co.uk  Thu Oct  1 16:37:29 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 14:37:29 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<loom.20091001T150010-10@post.gmane.org>
Message-ID: <loom.20091001T161717-343@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> There's another possibility, which is to build the wrapping directly around the
> logger. That is, if I want a %-style logger, I do:
> 
>    logger = logging.getLogger("smtp")
>    logger.debug("incoming email from %s", sender_address)
> 
> and I want a {}-style logger, I do:
> 
>    logger = logging.getLogger("smtp", style="{}")
>    logger.debug("incoming email from {addr}", addr=sender_address)
> 
> (of course, different users of the "smtp" logger can request different
> formatting styles when calling getLogger().)

There's a LoggerAdapter class already in the system which is used to wrap
loggers so that additional contextual information (e.g. network or database
connection information) can be added to logs. The LoggerAdapter could fulfill
this "wrapping" function.

> We could combine the various proposals to give users flexible APIs. Of course,
> it generally smells of "there's more than one way to do it".

Yeah, that bothers me too.

> > It's the Rumsfeldian "We don't know what we don't know" 
> 
> Is this guy in the Python community? 

Not sure, but I believe he's a piece of work and not a guy to get on the wrong
side of ;-)

Regards,

Vinay Sajip


From foom at fuhm.net  Thu Oct  1 17:42:38 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 1 Oct 2009 11:42:38 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
Message-ID: <2650CF0D-E8B8-42C0-9063-9B88E634DBF6@fuhm.net>


On Oct 1, 2009, at 9:11 AM, Paul Moore wrote:
> This seems to me to be almost the same as the previous suggestion of
> having a string subclass:
>
> class BraceFormatter(str):
>    def __mod__(self, other):
>        # Needs more magic here to cope with dict argument
>        return self.format(*other)
>
> __ = BraceFormatter
>
> logger.debug(__("The {0} is {1}"), "answer", 42)


I'd rather make that:

class BraceFormatter:
     def __init__(self, s):
         self.s = s
     def __mod__(self, other):
         # Needs more magic here to cope with dict argument
         return s.format(*other)

__ = BraceFormatter

That is, *not* a string subclass. Then if someone attempts to mangle  
it, or use it for anything but %, it fails loudly.

James

From python at rcn.com  Thu Oct  1 17:50:28 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 1 Oct 2009 08:50:28 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com><A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net><200910010034.27427.steve@pearwood.info><ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net><loom.20090930T185811-479@post.gmane.org><loom.20091001T075736-620@post.gmane.org><loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
Message-ID: <68A57C0BA3E84957A968BE16CA3E5ADC@RaymondLaptop1>


[Vinay Sajip]
> And the special wrappers needn't be too intrusive:
> 
> __ = BraceMessage
> 
> logger.debug(__("Message with {0} {1}", 1, "argument"))

It looks like the BraceMessage would have to re-instantiate on every invocation.


Raymond

From janssen at parc.com  Thu Oct  1 18:01:21 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 1 Oct 2009 09:01:21 PDT
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC46070.1010706@scottdial.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AC45BFA.3020802@scottdial.com> <4AC46070.1010706@scottdial.com>
Message-ID: <59160.1254412881@parc.com>

Scott Dial <scott+python-dev at scottdial.com> wrote:

> Scott Dial wrote:
> > While this code is present in
> > older versions of python, it seems to have become a problem recently
> > (2009-05-06 is the earliest report on the issue) perhaps due to a
> > version bump of OpenSSL? I never noticed the problem in python2.5 even
> > though the code is unchanged.
> 
> To answer my own question, this was introduced in r64578 in ssl.py by
> the addition of the suppress_ragged_eofs feature (seems to originate
> with issue1251). While it is a change in the behavior of the ssl module,
> the change falls in line with other file-like objects and their handling
> of EOF, so the bug still falls to imaplib.
> 
> In other words, this bug appeared in 2.6 and 3.0, and is present in all
> subsequent versions.

Thanks for bringing this up, Scott.

Bill

From foom at fuhm.net  Thu Oct  1 18:15:15 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 1 Oct 2009 12:15:15 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20090930T185811-479@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
Message-ID: <894C0271-B3CC-49CD-B098-41EFBD27EF49@fuhm.net>

On Sep 30, 2009, at 1:01 PM, Antoine Pitrou wrote:
> Why not allow logging.Formatter to take a callable, which would in  
> turn call the
> callable with keyword arguments?
>
> Therefore, you could write:
>   logging.Formatter("{asctime} - {name} - {level} - {msg}".format)
>
> and then:
>   logging.critical(name="Python", msg="Buildbots are down")
>
> All this without having to learn about a separate "compatibility  
> wrapper object".

It's a nice idea -- but I think it's better for the wrapper (whatever  
form it takes) to support __mod__ so that logging.Formatter (and  
everything else) doesn't need to be modified to be able to know about  
how to use both callables and "%"ables.

Is it possible for a C function like str.format to have other methods  
defined on its function type?

James

From vinay_sajip at yahoo.co.uk  Thu Oct  1 18:21:00 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 16:21:00 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com><A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net><200910010034.27427.steve@pearwood.info><ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net><loom.20090930T185811-479@post.gmane.org><loom.20091001T075736-620@post.gmane.org><loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<68A57C0BA3E84957A968BE16CA3E5ADC@RaymondLaptop1>
Message-ID: <loom.20091001T180506-846@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:

> It looks like the BraceMessage would have to re-instantiate on every
> invocation.

True, because the arguments to the instantiation are kept around as a
BraceMessage instance until the time comes to actually format the message
(which might be never). Since typically in performance-sensitive code, the
isEnabledFor pattern is used to avoid doing unnecessary work, as in

if logger.isEnabledFor(logging.DEBUG):
    logger.debug(__("The {0} is {1}", "answer", 42))

The BraceMessage invocation overhead is only incurred when needed, as is the
cost of computing the additional arguments.

As I understand it {}-formatting is slower than %-formatting anyway, and if
this pattern is used only for {}-formatting, then there will be no additional
overhead for %-formatting and some additional overhead for {}-formatting. I'm
not sure what that instantiation cost will be relative to the overall time for
an "average" call - whatever that is ;-) - though.

Other approaches to avoid instantiation could be considered: for example,
making __ a callable which remembers previous calls and caches instances keyed
by the call arguments. But this will incur memory overhead and some processing
overhead and I'm not sure if it really buys you enough to warrant doing it.

Regards,

Vinay Sajip


From glenn at nevcal.com  Thu Oct  1 18:49:24 2009
From: glenn at nevcal.com (Glenn Linderman)
Date: Thu, 01 Oct 2009 09:49:24 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T010253-42@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
Message-ID: <4AC4DD94.1010205@nevcal.com>

On approximately 9/30/2009 4:03 PM, came the following characters from
the keyboard of Vinay Sajip:
> Steven Bethard <steven.bethard <at> gmail.com> writes:
> 
>> There's a lot of code already out there (in the standard library and
>> other places) that uses %-style formatting, when in Python 3.0 we
>> should be encouraging {}-style formatting. We should really provide
>> some sort of transition plan. Consider an example from the logging
>> docs:
>>
>> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
>>
>> We'd like to support both this style as well as the following style:
>>
>> logging.Formatter("{asctime} - {name} - {levelname} - {message}")
>>
> 
> In logging at least, there are two different places where the formatting issue
> crops up.
> 
> The first is creating the "message" part of the the logging event, which is
> made up of a format string and arguments.
> 
> The second is the one Steven's mentioned: formatting the message along with
> other event data such as time of occurrence, level, logger name etc. into the
> final text which is output.


It seems to me that most of the discussion of in this thread is
concerned with the first issue... and yet I see the second as the harder
issue, and it has gotten less press.

Abstracting this away from logger, I think the problem has three cases:

1) Both the format message and all the parameters are supplied in a
single API call.  This is really a foolish API, because

    def API( fmt, p1, p2, p3 ):
	str = fmt % (p1, p2, p3)

could have just as easily been documented originally as

    def API( str ):

where the user is welcome to supply a string such as

    API( fmt % (p1, p2, p3 ))

and if done this way, the conversion to .format is obvious... and all
under the users control.


2) The format message and the parameters are supplied to separate APIs,
because the format message is common to many invocations of the other
APIs that supply parameters, and is cached by the API.  This is
sufficient to break the foolishness of #1, but is really just a subset
of #3, so any solutions to #3 apply here.

3) The format message and the parameters for it may be supplied by the
same or separate APIs, but one or both are incomplete, and are augmented
by the API.  In other words, one or both of the following cases:

3a) The user supplied format message may include references to named
parameters that are documented by the API, and supplied by the API,
rather than by the user.

3b) The user supplied format string may be embedded into a larger format
string by the API, which contains references to other values that the
user must also supply.

In either case of 3a or 3b, the user has insufficient information to
perform the whole format operation and pass the result to the API.

In both cases, the API that accepts the format string must be informed
whether it is a % or {} string, somehow.  This could be supplied to the
API that accepts the string, or to some other related API that sets a
format mode.  Internally, the code would have to be able to manipulate
both types of formats.



> Support for both % and {} forms in logging would need to be considered in
> these two places. I sort of liked Martin's proposal about using different
> keyword arguments, but apart from the ugliness of "dicttemplate" and the fact
> that "fmt" is already used in Formatter.__init__ as a keyword argument, it's
> possible that two different keyword arguments "fmt" and "format" both referring
> to format strings might be confusing to some users.
> 
> Benjamin's suggestion of providing a flag to Formatter seems slightly better,
> as it doesn't change what existing positional or keyword parameters do, and
> just adds an additional, optional parameter which can start off with a default
> of False and transition to a default of True.
>
> However, AFAICT these approaches only cover the second area where formatting
> options are chosen - not the creation of the message from the parameters passed
> to the logging call itself. 


The above three paragraphs are unclear to me.  I think they might be
referring to case 2 or 3, though.



> Of course one can pass arbitrary objects as messages which contain their own
> formatting logic. This has been possible since the very first release but I'm
> not sure that it's widely used, as it's usually easier to pass strings. So
> instead of passing a string and arguments such as
> 
> logger.debug("The %s is %d", "answer", 42)
> 
> one can currently pass, for a fictitious class PercentMessage,
> 
> logger.debug(PercentMessage("The %s is %d", "answer", 42))
> 
> and when the time comes to obtain the formatted message, LogRecord.getMessage
> calls str() on the PercentMessage instance, whose __str__ will use %-formatting
> to get the actual message.
> 
> Of course, one can also do for example
> 
> logger.debug(BraceMessage("The {} is {}", "answer", 42))
> 
> where the __str__() method on the BraceMessage will do {} formatting.
> 
> Of course, I'm not suggesting we actually use the names PercentMessage and
> BraceMessage, I've just used them there for clarity.


It seems that the above is only referring to case 1?  And doesn't help
with case 2 or 3?


> Also, although Raymond has pointed out that it seems likely that no one ever
> needs *both* types of format string, what about the case where application A
> depends on libraries B and C, and they don't all share the same preferences
> regarding which format style to use? ISTM no-one's brought this up yet, but it
> seems to me like a real issue. It would certainly appear to preclude any
> approach that configured a logging-wide or logger-wide flag to determine how to
> interpret the format string.


Agreed here... a single global state would not make modular upgrades to
a complex program easy... the state would be best included with
particular instance objects, especially when such instance objects exist
already.  The format type parameter could be provided to the instance,
instead of globally.


> Another potential issue is where logging events are pickled and sent over
> sockets to be finally formatted and output on different machines. What if a
> sending machine has a recent version of Python, which supports {} formatting,
> but a receiving machine doesn't? It seems that at the very least, it would
> require a change to SocketHandler and DatagramHandler to format the "message"
> part into the LogRecord before pickling and sending. While making this change
> is simple, it represents a potential backwards-incompatible problem for users
> who have defined their own handlers for doing something similar.
> 
> Apart from thinking through the above issues, the actual formatting only
> happens in two locations - LogRecord.getMessage and Formatter.format - so
> making the code do either %- or {} formatting would be simple, as long as it
> knows which of % and {} to pick.
> 
> Does it seems too onerous to expect people to pass an additional "use_format"
> keyword argument with every logging call to indicate how to interpret the
> message format string? Or does the PercentMessage/BraceMessage type approach
> have any mileage? What do y'all think?


These last 3 paragraphs seem to be very related to logger, specifically.
  The first of the 3 does point out a concern for systems that
interoperate across networks: if the format strings and parameters are
exposed separately across networks, whatever types are sent must be
usable at the receiver, or at least appropriate version control must be
required so that incompatible systems can be detected and reported.


On approximately 9/30/2009 5:47 PM, came the following characters from
the keyboard of Antoine Pitrou:
> Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
>> Does it seems too onerous to expect people to pass an additional
>> "use_format" keyword argument with every logging call to indicate 
how >> to interpret the message format string? Or does the
>> PercentMessage/BraceMessage type approach have any mileage? What do
>> y'all think?
>
> What about the proposal I made earlier?
> (support for giving a callable, so that you pass the "{foobar}".format
> method when you want new-style formatting)


This "callable" technique seems to only support case 1 and 2, but not 3,
unless I misunderstand it.


-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking


From scott+python-dev at scottdial.com  Thu Oct  1 19:47:08 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 01 Oct 2009 13:47:08 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC48DFD.7020607@gmail.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>	<4AC45BFA.3020802@scottdial.com>
	<4AC48DFD.7020607@gmail.com>
Message-ID: <4AC4EB1C.7030302@scottdial.com>

Nick Coghlan wrote:
> Scott Dial wrote:
>> I would appreciate this bug being resolved before the next release as it
>> effects me on a daily basis. I have submitted a patch, which reflects my
>> local solution.
> 
> Unfortunately, it's almost certainly too late to get this into 2.6.3. It
> really needed to be brought up back when Barry called for identification
> of significant 2.6 bugs and patches rather than after the RC had already
> been released.
> 

I understand. I didn't figure out the bug until after rc1 landed. It was
only happening spuriously with my mail server, never when I manually
tried to invoke the problem. I was forced to let my cronjob run until
the kernel killed getmail for OOM, giving me the trace shown in the issue.

It is impossible to break anything with this patch, as no program could
proceed. The 3-line patch merely converts it back into the exception
that was originally raised prior to 2.6, so it's not a new behavior in
that respect either. I wouldn't have anticipated this much resistance to
removing an infinite loop from the standard library. I could also for
this patch from the angle that it allows a remote host the ability to
execute a denial-of-service attack (even by accident!) since the
infinite loop appends an empty string to a list on every loop, taking
CPU time and memory with it. Allow me to be naive for a moment and say,
is this not the point of rc1 but to catch bugs that should not be in the
final?

Of course, it's Barry's call.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From brett at python.org  Thu Oct  1 20:03:39 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 1 Oct 2009 11:03:39 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T151606-886@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info> 
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org> 
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org> 
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com> 
	<loom.20091001T151606-886@post.gmane.org>
Message-ID: <bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>

On Thu, Oct 1, 2009 at 06:29, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>
>> This seems to me to be almost the same as the previous suggestion of
>> having a string subclass:
>>
>> class BraceFormatter(str):
>> ? ? def __mod__(self, other):
>> ? ? ? ? # Needs more magic here to cope with dict argument
>> ? ? ? ? return self.format(*other)
>>
>> __ = BraceFormatter
>>
>> logger.debug(__("The {0} is {1}"), "answer", 42)
>>

So I created this last night:

import collections

class braces_fmt(str):

    def __mod__(self, stuff):
        if isinstance(stuff, tuple):
            return self.__class__(self.format(*stuff))
        elif isinstance(stuff, collections.Mapping):
            return self.__class__(self.format(**stuff))
        else:
            return self.__class__(self.format(stuff))

The biggest issue is that ``"%s" % {'a': 42}`` substitutes the dict
instead of throwing an error that str.format() would do with the code
above. But what's nice about this is I think I can use this now w/ any
library that expects % interpolation and it should basically work.

>> The only real differences are
>>
>> 1. The positioning of the closing parenthesis
>> 2. The internal implementation of logger.debug needs to preserve
>> string subclasses properly
>>
>
> The other difference is that my suggestion supports Barry's desire to use
> string.Template with no muss, no fuss ;-)

I don't think Paul's suggestion requires much more work to support
string.Template, simply a subclass that implements __mod__>

> Plus, very little additional work is
> required compared to your items 1 and 2. ISTM BraceMessage would be something
> like this,
>
> clsss BraceMessage:
> ? ?def __init__(self, fmt, *args, **kwargs):
> ? ? ? ?self.fmt = fmt
> ? ? ? ?self.args = args
> ? ? ? ?self.kwargs = kwargs
>
> ? ?def __str__(self):
> ? ? ? ?return self.fmt.format(*self.args, **self.kwargs)

I guess my question is what's the point of the class if you are simply
converting it before you pass it in to the logger? To be lazy about
the formatting call? Otherwise you could simply call str.format() with
your arguments before you pass the string into the logger and not have
to wrap anything.

-Brett

From barry at python.org  Thu Oct  1 21:28:22 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 1 Oct 2009 15:28:22 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC4EB1C.7030302@scottdial.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>	<4AC45BFA.3020802@scottdial.com>
	<4AC48DFD.7020607@gmail.com> <4AC4EB1C.7030302@scottdial.com>
Message-ID: <FEBF1BC1-6C35-42E4-9F6C-A39FAC6437A2@python.org>

On Oct 1, 2009, at 1:47 PM, Scott Dial wrote:

> Allow me to be naive for a moment and say,
> is this not the point of rc1 but to catch bugs that should not be in  
> the
> final?

Actually, no.  The point of an rc is to avoid a brown paper bag  
mistake, i.e. something we really fscked up in the release like  
breaking "import sys" or the build process, etc.

We're always gonna have bugs, so that can't be it. :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091001/32c28ca7/attachment.pgp>

From ncoghlan at gmail.com  Thu Oct  1 23:15:14 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 02 Oct 2009 07:15:14 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC4DD94.1010205@nevcal.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>
	<4AC4DD94.1010205@nevcal.com>
Message-ID: <4AC51BE2.8040206@gmail.com>

Glenn Linderman wrote:
> 1) Both the format message and all the parameters are supplied in a
> single API call.  This is really a foolish API, because
> 
>    def API( fmt, p1, p2, p3 ):
>     str = fmt % (p1, p2, p3)
> 
> could have just as easily been documented originally as
> 
>    def API( str ):
> 
> where the user is welcome to supply a string such as
> 
>    API( fmt % (p1, p2, p3 ))
> 
> and if done this way, the conversion to .format is obvious... and all
> under the users control.

The lazy APIs actually make a lot of sense, particularly when there is a
chance that the function being called may be able to avoid the
formatting call altogether.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From vinay_sajip at yahoo.co.uk  Thu Oct  1 23:33:41 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 21:33:41 +0000 (GMT)
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
Message-ID: <69124.66598.qm@web25807.mail.ukl.yahoo.com>

> So I created this last night:

> 
> import collections
> 
> class braces_fmt(str):
> 
>     def __mod__(self, stuff):
>         if isinstance(stuff, tuple):
>             return self.__class__(self.format(*stuff))
>         elif isinstance(stuff, collections.Mapping):
>             return self.__class__(self.format(**stuff))
>         else:
>             return self.__class__(self.format(stuff))
> 
> The biggest issue is that ``"%s" % {'a': 42}`` substitutes the dict
> instead of throwing an error that str.format() would do with the code
> above. But what's nice about this is I think I can use this now w/ any
> library that expects % interpolation and it should basically work.

So there's no need to change modules like logging to explicitly provide support for {}-formatting? What's not to like? ;-) Something like this perhaps should have been added in at the same time as str.format went in.

> I don't think Paul's suggestion requires much more work to support
> string.Template, simply a subclass that implements __mod__

True.

> I guess my question is what's the point of the class if you are simply
> converting it before you pass it in to the logger? To be lazy about
> the formatting call? Otherwise you could simply call str.format() with
> your arguments before you pass the string into the logger and not have
> to wrap anything.

That's exactly the reason - to defer the formatting until it's needed. Otherwise you can always format the string yourself,as you say, and pass it as the single argument in the logging call - logging won't know or care if it was passed in as a literal, or was computed by %-, {}-, $- or any other formatting approach.

Regards,

Vinay Sajip



      

From ncoghlan at gmail.com  Thu Oct  1 23:11:57 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 02 Oct 2009 07:11:57 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T153627-621@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>	<loom.20090930T185811-479@post.gmane.org>	<loom.20091001T075736-620@post.gmane.org>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T153627-621@post.gmane.org>
Message-ID: <4AC51B1D.7040203@gmail.com>

Vinay Sajip wrote:
> You're right to be nervous: it's not hard to mangle subtypes.
> 
>>>> class mystr(str): pass
> ...
>>>> s = mystr("Abc")
>>>> s
> 'Abc'
>>>> type(s)
> <class '__main__.mystr'>
>>>> s2 = s.replace("A", "a")
>>>> s2
> 'abc'
>>>> type(s2)
> <type 'str'>

It's also difficult for the subclass to prevent this without creating an
infinite loop... (I only spent about 10 minutes looking into it the
other day, but that's what happened in all of my naive attempts at doing
it in pure Python code).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From sridharr at activestate.com  Thu Oct  1 23:48:19 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Thu, 01 Oct 2009 14:48:19 -0700
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <op.u0255laz1q4jlt@whymac.activestate.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-FC243A.21293229092009@news.gmane.org>
	<21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
	<op.u024qbnl1q4jlt@whymac.activestate.com>
	<E67D3FEF-CF1A-48F1-8E4B-A614794C57CF@python.org>
	<op.u0255laz1q4jlt@whymac.activestate.com>
Message-ID: <op.u045ite41q4jlt@whymac>

On Wed, 30 Sep 2009 13:06:47 -0700, Sridhar Ratnakumar  
<sridharr at activestate.com> wrote:

> On Wed, 30 Sep 2009 12:44:14 -0700, Barry Warsaw <barry at python.org>  
> wrote:
>
>> 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows  
>> 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did.
>> Thanks for the feedback!  Did you run the test suite on any of these?
>
> I will run the tests sometime tonight and let you know.

No new significant failures have been found. (Some trivial issues have  
been reported in the bug tracker).

-srid


From ncoghlan at gmail.com  Thu Oct  1 23:54:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 02 Oct 2009 07:54:20 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <69124.66598.qm@web25807.mail.ukl.yahoo.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>	<loom.20090930T185811-479@post.gmane.org>	<loom.20091001T075736-620@post.gmane.org>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>	<loom.20091001T151606-886@post.gmane.org>	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
Message-ID: <4AC5250C.8030403@gmail.com>

Vinay Sajip wrote:
> So there's no need to change modules like logging to explicitly
> provide support for {}-formatting? What's not to like? ;-) Something
> like this perhaps should have been added in at the same time as
> str.format went in.

I believe classes like fmt_braces/fmt_dollar/fmt_percent will be part of
a solution, but they aren't a complete solution on their own. (Naming
the three major string formatting techniques by the key symbols involved
is a really good idea though)

The two major problems with them:

1. It's easy to inadvertently convert them back to normal strings. If a
formatting API even calls "str" on the format string then we end up with
a problem (and switching to containment instead of inheritance doesn't
really help, since all objects implement __str__).

2. They don't help with APIs that expect a percent-formatted string and
do more with it than just pass it to str.__mod__ (e.g. inspecting it for
particular values such as '%(asctime)s')

Still, it's worth considering adding the three fmt_* classes to the
string module to see how far they can get us in adapting the formats for
different APIs.

Note that I don't think these concepts are fully baked yet, so we
shouldn't do anything in a hurry - and anything that does happen should
be via a PEP so we can flush out more issues.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Oct  1 23:25:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 02 Oct 2009 07:25:19 +1000
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AC4EB1C.7030302@scottdial.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>	<4AC45BFA.3020802@scottdial.com>
	<4AC48DFD.7020607@gmail.com> <4AC4EB1C.7030302@scottdial.com>
Message-ID: <4AC51E3F.9070601@gmail.com>

Scott Dial wrote:
> Allow me to be naive for a moment and say,
> is this not the point of rc1 but to catch bugs that should not be in the
> final?

For an x.y.0 rc I would usually agree with you, but for x.y.z (where z >
0), the intended situation is for there to be zero functional changes
between the rc and the release. (This actually holds true for the final
rc in x.y.0 release as well).

It's unfortunate, but the imminent point release gets people's attention
in a way that the original "there is going to be a point release around
this date" never seems to. If the release managers didn't draw a line
and said "no more bug fixes, even minor ones" then we'd never get point
releases out even close to on time.

This is particularly so for bugs that aren't a regression from x.y.(z-1)
- it's most often regressions which are marked as release blockers
rather than newly discovered (or analysed) bugs which have existed in
the series since the x.y.0 release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From foom at fuhm.net  Fri Oct  2 00:03:47 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 1 Oct 2009 18:03:47 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC5250C.8030403@gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>	<loom.20090930T185811-479@post.gmane.org>	<loom.20091001T075736-620@post.gmane.org>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>	<loom.20091001T151606-886@post.gmane.org>	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
	<4AC5250C.8030403@gmail.com>
Message-ID: <51684EC1-96B7-480E-A01D-871879308E2F@fuhm.net>

On Oct 1, 2009, at 5:54 PM, Nick Coghlan wrote:
> I believe classes like fmt_braces/fmt_dollar/fmt_percent will be  
> part of
> a solution, but they aren't a complete solution on their own. (Naming
> the three major string formatting techniques by the key symbols  
> involved
> is a really good idea though)
>
> 1. It's easy to inadvertently convert them back to normal strings.  
> If a
> formatting API even calls "str" on the format string then we end up  
> with
> a problem (and switching to containment instead of inheritance doesn't
> really help, since all objects implement __str__).

Using containment instead of inheritance makes sure none of the  
*other* operations people do on strings will appear to work, at least  
(substring, contains, etc). I bet explicitly calling str() on a format  
string is even more rare than attempting to do those things.

> 2. They don't help with APIs that expect a percent-formatted string  
> and
> do more with it than just pass it to str.__mod__ (e.g. inspecting it  
> for
> particular values such as '%(asctime)s')

True, but I don't think there's many such cases in the first place,  
and such places can be fixed to not do that as they're found.

Until they are fixed, fmt_braces will loudly fail when used with that  
API (assuming fmt_braces is not a subclass of str).

James

From vinay_sajip at yahoo.co.uk  Fri Oct  2 00:12:03 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 22:12:03 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>	<loom.20090930T185811-479@post.gmane.org>	<loom.20091001T075736-620@post.gmane.org>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>	<loom.20091001T151606-886@post.gmane.org>	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
	<4AC5250C.8030403@gmail.com>
	<51684EC1-96B7-480E-A01D-871879308E2F@fuhm.net>
Message-ID: <loom.20091002T000835-624@post.gmane.org>

James Y Knight <foom <at> fuhm.net> writes:

> Using containment instead of inheritance makes sure none of the  
> *other* operations people do on strings will appear to work, at least  
> (substring, contains, etc). I bet explicitly calling str() on a format  
> string is even more rare than attempting to do those things.

Actually, logging calls str() on the object passed as the first argument in a
logging call such as logger.debug(), which can either be a format string or an
arbitrary object whose __str__() returns the format string.

Regards,

Vinay Sajip


From steven.bethard at gmail.com  Fri Oct  2 00:19:38 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 1 Oct 2009 15:19:38 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
Message-ID: <d11dcfba0910011519r249d5ee8ie7217b025b134e60@mail.gmail.com>

On Thu, Oct 1, 2009 at 11:03 AM, Brett Cannon <brett at python.org> wrote:
> So I created this last night:
>
> import collections
>
> class braces_fmt(str):
>
> ? ?def __mod__(self, stuff):
> ? ? ? ?if isinstance(stuff, tuple):
> ? ? ? ? ? ?return self.__class__(self.format(*stuff))
> ? ? ? ?elif isinstance(stuff, collections.Mapping):
> ? ? ? ? ? ?return self.__class__(self.format(**stuff))
> ? ? ? ?else:
> ? ? ? ? ? ?return self.__class__(self.format(stuff))
>
> The biggest issue is that ``"%s" % {'a': 42}`` substitutes the dict
> instead of throwing an error that str.format() would do with the code
> above. But what's nice about this is I think I can use this now w/ any
> library that expects % interpolation and it should basically work.

I see how this could allow a user to supply a {}-format string to an
API that accepts only %-format strings. But I still don't see the
transition strategy for the API itself. That is, how does the %-format
API use this to eventually switch to {}-format strings? Could someone
please lay it out for me, step by step, showing what happens in each
version?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From vinay_sajip at yahoo.co.uk  Fri Oct  2 00:39:03 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 1 Oct 2009 22:39:03 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>	<loom.20090930T185811-479@post.gmane.org>	<loom.20091001T075736-620@post.gmane.org>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>	<loom.20091001T151606-886@post.gmane.org>	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
	<4AC5250C.8030403@gmail.com>
Message-ID: <loom.20091002T001232-270@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> I believe classes like fmt_braces/fmt_dollar/fmt_percent will be part of
> a solution, but they aren't a complete solution on their own. (Naming
> the three major string formatting techniques by the key symbols involved
> is a really good idea though)
> 
> The two major problems with them:
> 
> 1. It's easy to inadvertently convert them back to normal strings. If a
> formatting API even calls "str" on the format string then we end up with
> a problem (and switching to containment instead of inheritance doesn't
> really help, since all objects implement __str__).
> 
> 2. They don't help with APIs that expect a percent-formatted string and
> do more with it than just pass it to str.__mod__ (e.g. inspecting it for
> particular values such as '%(asctime)s')

Good point as far as the general case is concerned, though it's perhaps not that
critical for logging. By which I mean, it's not unreasonable for
Formatter.__init__ to grow a "style" keyword parameter which determines whether
it uses %-, {}- or $-formatting. Then the formatter can look for '%(asctime)s',
'{asctime}' or '$asctime' according to the style. 

Just to clarify - LogRecord.getMessage *will* call str() on a message object if
it's not a string or Unicode object. For 2.x the logic is

if type(msg) not in (unicode, str):
    msg = str(msg)

and for 3.x the check is for isinstance(msg, str).


> Still, it's worth considering adding the three fmt_* classes to the
> string module to see how far they can get us in adapting the formats for
> different APIs.
> 
> Note that I don't think these concepts are fully baked yet, so we
> shouldn't do anything in a hurry - and anything that does happen should
> be via a PEP so we can flush out more issues.

Yes, we're just "kicking the tires" on the various ideas. There are things still
a bit up in the air such as what happens when pickling and sending to an older
version of Python, etc. which still need to be resolved for logging, at least.

Regards,

Vinay Sajip


From barry at python.org  Fri Oct  2 00:40:51 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 1 Oct 2009 18:40:51 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <op.u045ite41q4jlt@whymac>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-FC243A.21293229092009@news.gmane.org>
	<21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
	<op.u024qbnl1q4jlt@whymac.activestate.com>
	<E67D3FEF-CF1A-48F1-8E4B-A614794C57CF@python.org>
	<op.u0255laz1q4jlt@whymac.activestate.com>
	<op.u045ite41q4jlt@whymac>
Message-ID: <FAA6DCFB-D427-4C0A-8EF3-06226367A535@python.org>

On Oct 1, 2009, at 5:48 PM, Sridhar Ratnakumar wrote:

> No new significant failures have been found. (Some trivial issues  
> have been reported in the bug tracker).

Fantastic, thanks.  I'll be tagging the final release soon.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091001/4c338225/attachment.pgp>

From eric at trueblade.com  Fri Oct  2 00:58:31 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 01 Oct 2009 18:58:31 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091002T001232-270@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>	<loom.20090930T185811-479@post.gmane.org>	<loom.20091001T075736-620@post.gmane.org>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>	<loom.20091001T151606-886@post.gmane.org>	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>	<69124.66598.qm@web25807.mail.ukl.yahoo.com>	<4AC5250C.8030403@gmail.com>
	<loom.20091002T001232-270@post.gmane.org>
Message-ID: <4AC53417.1040405@trueblade.com>

Vinay Sajip wrote:
> Good point as far as the general case is concerned, though it's perhaps not that
> critical for logging. By which I mean, it's not unreasonable for
> Formatter.__init__ to grow a "style" keyword parameter which determines whether
> it uses %-, {}- or $-formatting. Then the formatter can look for '%(asctime)s',
> '{asctime}' or '$asctime' according to the style. 

It's tangential, but in the str.format case you don't want to check for 
just '{asctime}', because you might want '{asctime:%Y-%m-%d}', for example.

But there are ways to delay computing the time until you're sure it's 
actually being used in the format string, without parsing the format 
string. Now that I think of it, the same technique could be used with 
%-formatting:

import datetime

class DelayedStr:
     def __init__(self, fn):
         self.fn = fn
         self.obj = None
     def __str__(self):
         if self.obj is None:
             self.obj = self.fn()
         return self.obj.__str__()

def current_time():
     print "calculating time"
     return datetime.datetime.now()

# will not compute current time
print '%(msg)s' % {'asctime':DelayedStr(current_time),
                    'msg':'test'}

# will compute current time: same dict used as before
print '%(asctime)s %(msg)s' % {'asctime':DelayedStr(current_time),
                                'msg':'test'}

Eric.



From foom at fuhm.net  Fri Oct  2 01:13:28 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 1 Oct 2009 19:13:28 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910011519r249d5ee8ie7217b025b134e60@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<d11dcfba0910011519r249d5ee8ie7217b025b134e60@mail.gmail.com>
Message-ID: <5EC96CF6-C48F-41E1-B32C-816045DBE91D@fuhm.net>

On Oct 1, 2009, at 6:19 PM, Steven Bethard wrote:
> I see how this could allow a user to supply a {}-format string to an
> API that accepts only %-format strings. But I still don't see the
> transition strategy for the API itself. That is, how does the %-format
> API use this to eventually switch to {}-format strings? Could someone
> please lay it out for me, step by step, showing what happens in each
> version?


Here's what I said in my first message, suggesting this change.  
Copy&pasted below:

I wrote:
> 1) introduce the above feature, and recommend in docs that people  
> only ever use new-style format strings, wrapping the string in  
> newstyle_formatstr() when necessary for passing to an API which uses  
> % internally.
> 2) A long time later...deprecate str.__mod__; don't deprecate  
> newstyle_formatstr.__mod__.
> 3) A while after that (maybe), remove str.__mod__ and replace all  
> calls in Python to % (used as a formatting operator) with .format()  
> so that the default is to use newstyle format strings for all APIs  
> from then on.

So do (1) in 3.2. Then do (2) in 3.4, and (3) in 3.6. I skipped two  
versions each time because of how widely this API is used, and the  
likely pain that doing the transition quickly would cause. But I guess  
you *could* do it in one version each step.

James

From brett at python.org  Fri Oct  2 01:39:49 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 1 Oct 2009 16:39:49 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC5250C.8030403@gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org> 
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org> 
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com> 
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com> 
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
	<4AC5250C.8030403@gmail.com>
Message-ID: <bbaeab100910011639y7bcaf2a3xc4e4f4de79a790db@mail.gmail.com>

On Thu, Oct 1, 2009 at 14:54, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Vinay Sajip wrote:
>> So there's no need to change modules like logging to explicitly
>> provide support for {}-formatting? What's not to like? ;-) Something
>> like this perhaps should have been added in at the same time as
>> str.format went in.
>
> I believe classes like fmt_braces/fmt_dollar/fmt_percent will be part of
> a solution, but they aren't a complete solution on their own.

I agree. I view them more as a band-aid over APIs that only accept %
formatting but the user of the library wants to use {} formatting.

> (Naming
> the three major string formatting techniques by the key symbols involved
> is a really good idea though)
>
> The two major problems with them:
>
> 1. It's easy to inadvertently convert them back to normal strings. If a
> formatting API even calls "str" on the format string then we end up with
> a problem (and switching to containment instead of inheritance doesn't
> really help, since all objects implement __str__).
>

Well, you can override the methods on str to always return the proper
thing, e.g. ``def __str__(self): return self``. Do the same for
__add__() and all other methods on strings that return a string
themselves. It should be possible to prevent Python code from
stripping off the class.

> 2. They don't help with APIs that expect a percent-formatted string and
> do more with it than just pass it to str.__mod__ (e.g. inspecting it for
> particular values such as '%(asctime)s')
>

Nope, they don't and people would need to be warned against this.

> Still, it's worth considering adding the three fmt_* classes to the
> string module to see how far they can get us in adapting the formats for
> different APIs.
>
> Note that I don't think these concepts are fully baked yet, so we
> shouldn't do anything in a hurry - and anything that does happen should
> be via a PEP so we can flush out more issues.

Having a PEP that lays out how we think people should consider
transitioning their code would be good.

-Brett

From steven.bethard at gmail.com  Fri Oct  2 03:13:12 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 1 Oct 2009 18:13:12 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <bbaeab100910011635o37845c8dlb197a5880f29781c@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20090930T185811-479@post.gmane.org>
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<d11dcfba0910011519r249d5ee8ie7217b025b134e60@mail.gmail.com>
	<bbaeab100910011635o37845c8dlb197a5880f29781c@mail.gmail.com>
Message-ID: <d11dcfba0910011813n58e3dd33jf75593e3b4c6a4f1@mail.gmail.com>

On Thu, Oct 1, 2009 at 4:35 PM, Brett Cannon <brett at python.org> wrote:
> On Thu, Oct 1, 2009 at 15:19, Steven Bethard <steven.bethard at gmail.com> wrote:
>> On Thu, Oct 1, 2009 at 11:03 AM, Brett Cannon <brett at python.org> wrote:
>>> class braces_fmt(str):
>>>
>>> ? ?def __mod__(self, stuff):
>>> ? ? ? ?if isinstance(stuff, tuple):
>>> ? ? ? ? ? ?return self.__class__(self.format(*stuff))
>>> ? ? ? ?elif isinstance(stuff, collections.Mapping):
>>> ? ? ? ? ? ?return self.__class__(self.format(**stuff))
>>> ? ? ? ?else:
>>> ? ? ? ? ? ?return self.__class__(self.format(stuff))
>>>
>>> The biggest issue is that ``"%s" % {'a': 42}`` substitutes the dict
>>> instead of throwing an error that str.format() would do with the code
>>> above. But what's nice about this is I think I can use this now w/ any
>>> library that expects % interpolation and it should basically work.
>>
>> Could someone please lay it out for me, step by step, showing what
>> happens in each version?
>
> 1. Nothing changes as hopefully the wrapper works fine (as people are
> pointing out, though, my approach needs to override __str__() to
> return 'self', else the str type will just return what it has
> internally in its buffer).
>
> 2. Raise a deprecation warning when ``isinstance(ob, brace_fmt)`` is
> false. When a class is passed in that is a subclass of brace_fmt, call
> ob.format() on it.
>
> 3. Require the subclass.
>
> 4. Remove the requirement and always call ob.format().

Thanks Brett, that's clear. So you save one version over the proposal
of adding a format= flag to the API.


On Thu, Oct 1, 2009 at 4:13 PM, James Y Knight <foom at fuhm.net> wrote:
> Here's what I said in my first message, suggesting this change. Copy&pasted
> below:
[snip steps that only talk about str.__mod__, not an API that uses it]

I didn't understand how you wanted to apply your suggestion to an API
(instead of str.__mod__) the first time and I still don't understand
it. Is what Brett has proposed the same thing?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From songofacandy at gmail.com  Fri Oct  2 04:07:57 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Fri, 2 Oct 2009 11:07:57 +0900
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on Mac
	OS X?
Message-ID: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>

I found this hg's issue.
http://mercurial.selenic.com/bts/msg8375

I think below fix is not enabled on Mac OS X.
http://svn.python.org/view/python/trunk/Include/pyport.h?view=diff&pathrev=43219&r1=36792&r2=36793

I can't confirm it because I am not Mac OS X user.
Can anyone confirm it?

-- 
Naoki INADA  <songofacandy at gmail.com>

From tjreedy at udel.edu  Fri Oct  2 07:49:34 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 02 Oct 2009 01:49:34 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091001T024614-990@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org>
Message-ID: <ha449d$9n5$1@ger.gmane.org>

Antoine Pitrou wrote:
> Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
>> Does it seems too onerous to expect people to pass an additional "use_format"
>> keyword argument with every logging call to indicate how to interpret the
>> message format string? Or does the PercentMessage/BraceMessage type approach
>> have any mileage? What do y'all think?
> 
> What about the proposal I made earlier?
> (support for giving a callable, so that you pass the "{foobar}".format method
> when you want new-style formatting)

As someone who likes .format() and who already uses such bound methods 
to print, such as in

emsg = "...".format
...
    if c: print(emsg(arg, barg))

I find this **MUCH** preferable to the ugly and seemingly unnecessary 
wrapper class idea being bandied about. This would be scarcely worse 
than passing the string itself.

Terry Jan Reedy


From steven.bethard at gmail.com  Fri Oct  2 09:12:28 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 2 Oct 2009 00:12:28 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ha449d$9n5$1@ger.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
Message-ID: <d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>

On Thu, Oct 1, 2009 at 10:49 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> As someone who likes .format() and who already uses such bound methods to
> print, such as in
>
> emsg = "...".format
> ...
> ? if c: print(emsg(arg, barg))
>
> I find this **MUCH** preferable to the ugly and seemingly unnecessary
> wrapper class idea being bandied about. This would be scarcely worse than
> passing the string itself.

But it's not much of a transition plan. Or are you suggesting:

(1) Make API accept callables
(2) Issue warnings for regular strings
(3) Throw exceptions for regular strings
(4) Allow regular strings again, but assume {}-style formatting

?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From ronaldoussoren at mac.com  Fri Oct  2 09:39:21 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 02 Oct 2009 09:39:21 +0200
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on
 Mac	OS X?
In-Reply-To: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
References: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
Message-ID: <92839421634652657618189239787554831247-Webmail@me.com>

 
On Friday, 02 October, 2009, at 04:07AM, "INADA Naoki" <songofacandy at gmail.com> wrote:
>I found this hg's issue.
>http://mercurial.selenic.com/bts/msg8375
>
>I think below fix is not enabled on Mac OS X.
>http://svn.python.org/view/python/trunk/Include/pyport.h?view=diff&pathrev=43219&r1=36792&r2=36793
>
>I can't confirm it because I am not Mac OS X user.
>Can anyone confirm it?

Do you have a testcase that shows what the problem is?

Ronald


From brett at python.org  Fri Oct  2 01:35:44 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 1 Oct 2009 16:35:44 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910011519r249d5ee8ie7217b025b134e60@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org> 
	<loom.20091001T075736-620@post.gmane.org>
	<loom.20091001T124105-392@post.gmane.org> 
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com> 
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com> 
	<d11dcfba0910011519r249d5ee8ie7217b025b134e60@mail.gmail.com>
Message-ID: <bbaeab100910011635o37845c8dlb197a5880f29781c@mail.gmail.com>

On Thu, Oct 1, 2009 at 15:19, Steven Bethard <steven.bethard at gmail.com> wrote:
> On Thu, Oct 1, 2009 at 11:03 AM, Brett Cannon <brett at python.org> wrote:
>> So I created this last night:
>>
>> import collections
>>
>> class braces_fmt(str):
>>
>> ? ?def __mod__(self, stuff):
>> ? ? ? ?if isinstance(stuff, tuple):
>> ? ? ? ? ? ?return self.__class__(self.format(*stuff))
>> ? ? ? ?elif isinstance(stuff, collections.Mapping):
>> ? ? ? ? ? ?return self.__class__(self.format(**stuff))
>> ? ? ? ?else:
>> ? ? ? ? ? ?return self.__class__(self.format(stuff))
>>
>> The biggest issue is that ``"%s" % {'a': 42}`` substitutes the dict
>> instead of throwing an error that str.format() would do with the code
>> above. But what's nice about this is I think I can use this now w/ any
>> library that expects % interpolation and it should basically work.
>
> I see how this could allow a user to supply a {}-format string to an
> API that accepts only %-format strings. But I still don't see the
> transition strategy for the API itself. That is, how does the %-format
> API use this to eventually switch to {}-format strings? Could someone
> please lay it out for me, step by step, showing what happens in each
> version?

First off, a wrapper like this I think is a temporary solution for
libraries that do not have any transition strategy, not a replacement
for one that is thought out (e.g. using a flag when appropriate). With
that said, you could transition by:

1. Nothing changes as hopefully the wrapper works fine (as people are
pointing out, though, my approach needs to override __str__() to
return 'self', else the str type will just return what it has
internally in its buffer).

2. Raise a deprecation warning when ``isinstance(ob, brace_fmt)`` is
false. When a class is passed in that is a subclass of brace_fmt, call
ob.format() on it.

3. Require the subclass.

4. Remove the requirement and always call ob.format().

-Brett

From songofacandy at gmail.com  Fri Oct  2 10:29:39 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Fri, 2 Oct 2009 17:29:39 +0900
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on
	Mac OS X?
In-Reply-To: <92839421634652657618189239787554831247-Webmail@me.com>
References: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
	<92839421634652657618189239787554831247-Webmail@me.com>
Message-ID: <ee9557410910020129w1ed7be0fie2c45d099209189e@mail.gmail.com>

> Do you have a testcase that shows what the problem is?
>
> Ronald

>>> s = '\xa0'
>>> assert s.strip() == s
>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
'en_US.UTF-8'
>>> assert s.strip() == s

Second assert failed on Snow Leopard.

-- 
Naoki INADA  <songofacandy at gmail.com>

From solipsis at pitrou.net  Fri Oct  2 11:34:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 2 Oct 2009 09:34:30 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
Message-ID: <loom.20091002T113352-479@post.gmane.org>

Steven Bethard <steven.bethard <at> gmail.com> writes:
> 
> But it's not much of a transition plan. Or are you suggesting:

The question is why we want a transition plan that will bother everyone with no
tangible benefits for the user.

Regards

Antoine.



From barry at python.org  Fri Oct  2 12:56:58 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 2 Oct 2009 06:56:58 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091002T113352-479@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
Message-ID: <ECC08A6A-846E-4334-899C-F42D9D8BC811@python.org>

On Oct 2, 2009, at 5:34 AM, Antoine Pitrou wrote:

> Steven Bethard <steven.bethard <at> gmail.com> writes:
>>
>> But it's not much of a transition plan. Or are you suggesting:
>
> The question is why we want a transition plan that will bother  
> everyone with no
> tangible benefits for the user.

Forcing a transition to {}-strings might have limited usefulness, but  
enabling their use has clear benefits.  You just then have to decide  
how important TOOWTDI is.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091002/8867707f/attachment.pgp>

From status at bugs.python.org  Fri Oct  2 18:07:38 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  2 Oct 2009 18:07:38 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091002160738.226BE78644@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (09/25/09 - 10/02/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2410 open (+17) / 16447 closed (+24) / 18857 total (+41)

Open issues with patches:   958

Average duration of open issues: 673 days.
Median duration of open issues: 430 days.

Open Issues Breakdown
   open  2376 (+17)
pending    33 ( +0)

Issues Created Or Reopened (44)
_______________________________

Mismatching use of memory APIs                                   09/28/09
CLOSED http://bugs.python.org/issue6836    reopened r.david.murray                
       patch                                                                   

threading.local subclasses don't cleanup their state and it gets 09/29/09
CLOSED http://bugs.python.org/issue6990    reopened pitrou                        
       patch, needs review                                                     

PEP 314 inconsistency (authors/author/maintainer)                09/28/09
       http://bugs.python.org/issue6992    reopened techtonik                     
                                                                               

Python 3.1 interpreter Fatal Error on windows with unknown encod 09/25/09
CLOSED http://bugs.python.org/issue6995    created  bahaelaila7                   
                                                                               

Python 2.6.2 & 3.1.1 Installer                                   09/25/09
CLOSED http://bugs.python.org/issue6996    created  deepakp                       
                                                                               

python26 compiled on snow leopard x86_64                         09/25/09
CLOSED http://bugs.python.org/issue6997    created  monway                        
                                                                               

Bug in Tutorial (introduction.rst)                               09/25/09
CLOSED http://bugs.python.org/issue6998    created  cofi                          
                                                                               

TypeError in pathfix.py                                          09/25/09
CLOSED http://bugs.python.org/issue6999    created  amcnabb                       
       patch                                                                   

optional second argument of string.capwords not documented       09/25/09
CLOSED http://bugs.python.org/issue7000    created  MLModel                       
                                                                               

turtle.py bug-fixes, backported from 3.1 [issue5923]             09/25/09
CLOSED http://bugs.python.org/issue7001    created  gregorlingl                   
       patch                                                                   

help me to set variables - 64 bit                                09/26/09
CLOSED http://bugs.python.org/issue7002    created  ashish                        
                                                                               

finish documentation of user local paths                         09/26/09
       http://bugs.python.org/issue7003    created  benjamin.peterson             
                                                                               

Py_RETURN_BOOL() convenience macro                               09/27/09
CLOSED http://bugs.python.org/issue7004    created  jon                           
       patch                                                                   

ConfigParser does not handle options without values              09/27/09
       http://bugs.python.org/issue7005    created  mkindahl                      
       patch                                                                   

The replacement suggested for callable(x) in py3k is not equival 09/27/09
       http://bugs.python.org/issue7006    created  milko.krachounov              
                                                                               

Tiny inconsistency in the orthography of "url encoded" in the do 09/27/09
       http://bugs.python.org/issue7007    created  MLModel                       
                                                                               

str.title() misbehaves with apostrophes                          09/27/09
CLOSED http://bugs.python.org/issue7008    created  nickd                         
                                                                               

random.randint docs                                              09/28/09
CLOSED http://bugs.python.org/issue7009    created  jgsack                        
                                                                               

Rounding when converting float tuple to str                      09/28/09
CLOSED http://bugs.python.org/issue7010    created  scientist47                   
                                                                               

tarinfo has problems with longlinks                              09/28/09
CLOSED http://bugs.python.org/issue7011    created  do3cc                         
                                                                               

Tabs is better than spaces for indentation                       09/28/09
CLOSED http://bugs.python.org/issue7012    created  rohdef                        
                                                                               

Httplib read routine is not tolerant to not well-formed chunked  09/28/09
       http://bugs.python.org/issue7013    created  Andrei Korostelev             
       patch                                                                   

Logging incompatible with IronPython                             09/28/09
CLOSED http://bugs.python.org/issue7014    created  michael.foord                 
                                                                               

Getting call trace while executing "modules spam" at help prompt 09/29/09
       http://bugs.python.org/issue7015    created  rishikesh                     
       patch                                                                   

.pyc files are set executable if the .py file is too             09/29/09
CLOSED http://bugs.python.org/issue7016    created  stevenjd                      
                                                                               

os.listdir behaviour                                             09/29/09
CLOSED http://bugs.python.org/issue7017    created  bigaddo                       
                                                                               

Recommend "*" over "#" in getargs.c typecodes                    09/29/09
       http://bugs.python.org/issue7018    created  pitrou                        
                                                                               

unmarshaling of artificial strings can produce funny longs.      09/29/09
       http://bugs.python.org/issue7019    created  cfbolz                        
       patch                                                                   

regression in pywin32 build due to 2.6.3rc1                      09/30/09
CLOSED http://bugs.python.org/issue7020    created  srid                          
                                                                               

subprocess.Popen doesn't work without "executable="              09/30/09
CLOSED http://bugs.python.org/issue7021    created  hc                            
                                                                               

Doc update for io module                                         09/30/09
CLOSED http://bugs.python.org/issue7022    created  pakal                         
                                                                               

Marshal doesn't release GIL while dumping                        09/30/09
CLOSED http://bugs.python.org/issue7023    created  zengke                        
       patch                                                                   

webbrowser : Could not open ftp server using webbrowser.open()   09/30/09
       http://bugs.python.org/issue7024    created  hmo                           
                                                                               

Python 3.1 and 3.2 (dev) opensearch.xml still references 3.0     09/30/09
CLOSED http://bugs.python.org/issue7025    created  jon                           
                                                                               

test_urllib: unsetting missing 'env' variable                    10/01/09
       http://bugs.python.org/issue7026    created  srid                          
                                                                               

test_io.py: codecs.IncrementalDecoder is sometimes None          10/01/09
       http://bugs.python.org/issue7027    created  srid                          
                                                                               

Add int.hex for symmetry with float.hex                          10/02/09
       http://bugs.python.org/issue7028    reopened mark.dickinson                
                                                                               

Improve pybench                                                  10/01/09
       http://bugs.python.org/issue7029    created  krisvale                      
       patch                                                                   

Update version{added,changed} entries in py3k unittest docs      10/01/09
       http://bugs.python.org/issue7030    created  mark.dickinson                
                                                                               

add assertIsInstance                                             10/01/09
CLOSED http://bugs.python.org/issue7031    created  georg.brandl                  
       patch, patch, easy, needs review                                        

Make assertMultilineEqual default for unicode string comparison  10/01/09
       http://bugs.python.org/issue7032    created  michael.foord                 
                                                                               

C/API - Document exceptions                                      10/02/09
       http://bugs.python.org/issue7033    created  lekma                         
                                                                               

While Loop Bug on a game of 21, help!                            10/02/09
CLOSED http://bugs.python.org/issue7034    created  PixelHead777                  
                                                                               

codecs error handlers lack documentation                         10/02/09
       http://bugs.python.org/issue7035    created  olau                          
                                                                               



Issues Now Closed (42)
______________________

Subprocess error with I/O redirection to Pipes                    341 days
       http://bugs.python.org/issue4192    pakal                         
                                                                               

os.popen2 and os.popen3 in python 2.6 incompatible with os.popen  221 days
       http://bugs.python.org/issue5329    pjenvey                       
       patch                                                                   

Python 3.0 grammar is ambiguous with the addition of star_expr    201 days
       http://bugs.python.org/issue5460    benjamin.peterson             
                                                                               

http://docs.python.org/dev/library/unittest.html#load-tests-prot   71 days
       http://bugs.python.org/issue6515    michael.foord                 
                                                                               

csv.Sniffer.sniff on data with doublequotes doesn't set up the d   60 days
       http://bugs.python.org/issue6606    jtate                         
       patch                                                                   

Integer & Long types:  Performance improvement of 1.6x to 2x for   42 days
       http://bugs.python.org/issue6713    mark.dickinson                
       patch                                                                   

httplib and array do not play together well                        33 days
       http://bugs.python.org/issue6790    pitrou                        
       patch, easy                                                             

Mismatching use of memory APIs                                      0 days
       http://bugs.python.org/issue6836    krisvale                      
       patch                                                                   

Test creation in unittest.TestProgram should be done in one plac    6 days
       http://bugs.python.org/issue6956    michael.foord                 
                                                                               

Extension modules fail to build on OS X 10.6 using python.org 2.    8 days
       http://bugs.python.org/issue6957    ronaldoussoren                
                                                                               

Add Python command line flags to configure logging                 10 days
       http://bugs.python.org/issue6958    vinay.sajip                   
       patch                                                                   

numpy extensions to distutils... are a source of improvements fo    3 days
       http://bugs.python.org/issue6968    tjreedy                       
                                                                               

Add the SIO_KEEPALIVE_VALS option to socket.ioctl                   5 days
       http://bugs.python.org/issue6971    krisvale                      
       patch, patch, easy                                                      

Compile error  for Python-2.6.2 on Solaris                          1 days
       http://bugs.python.org/issue6989    ashish                        
                                                                               

threading.local subclasses don't cleanup their state and it gets    0 days
       http://bugs.python.org/issue6990    pitrou                        
       patch, needs review                                                     

logging encoding failes some situation                              1 days
       http://bugs.python.org/issue6991    naoki                         
       patch                                                                   

importing of "time" module is terribly slow                         0 days
       http://bugs.python.org/issue6993    ndyankov                      
                                                                               

enumerate dosctring has a typo                                      0 days
       http://bugs.python.org/issue6994    ezio.melotti                  
                                                                               

Python 3.1 interpreter Fatal Error on windows with unknown encod    0 days
       http://bugs.python.org/issue6995    r.david.murray                
                                                                               

Python 2.6.2 & 3.1.1 Installer                                      0 days
       http://bugs.python.org/issue6996    deepakp                       
                                                                               

python26 compiled on snow leopard x86_64                            0 days
       http://bugs.python.org/issue6997    monway                        
                                                                               

Bug in Tutorial (introduction.rst)                                  0 days
       http://bugs.python.org/issue6998    ezio.melotti                  
                                                                               

TypeError in pathfix.py                                             0 days
       http://bugs.python.org/issue6999    benjamin.peterson             
       patch                                                                   

optional second argument of string.capwords not documented          1 days
       http://bugs.python.org/issue7000    ezio.melotti                  
                                                                               

turtle.py bug-fixes, backported from 3.1 [issue5923]                3 days
       http://bugs.python.org/issue7001    r.david.murray                
       patch                                                                   

help me to set variables - 64 bit                                   0 days
       http://bugs.python.org/issue7002    brett.cannon                  
                                                                               

Py_RETURN_BOOL() convenience macro                                  1 days
       http://bugs.python.org/issue7004    jon                           
       patch                                                                   

str.title() misbehaves with apostrophes                             2 days
       http://bugs.python.org/issue7008    gvanrossum                    
                                                                               

random.randint docs                                                 0 days
       http://bugs.python.org/issue7009    orsenthil                     
                                                                               

Rounding when converting float tuple to str                         0 days
       http://bugs.python.org/issue7010    rhettinger                    
                                                                               

tarinfo has problems with longlinks                                 0 days
       http://bugs.python.org/issue7011    do3cc                         
                                                                               

Tabs is better than spaces for indentation                          1 days
       http://bugs.python.org/issue7012    gvanrossum                    
                                                                               

Logging incompatible with IronPython                                1 days
       http://bugs.python.org/issue7014    michael.foord                 
                                                                               

.pyc files are set executable if the .py file is too                0 days
       http://bugs.python.org/issue7016    r.david.murray                
                                                                               

os.listdir behaviour                                                1 days
       http://bugs.python.org/issue7017    bigaddo                       
                                                                               

regression in pywin32 build due to 2.6.3rc1                         2 days
       http://bugs.python.org/issue7020    srid                          
                                                                               

subprocess.Popen doesn't work without "executable="                 0 days
       http://bugs.python.org/issue7021    hc                            
                                                                               

Doc update for io module                                            1 days
       http://bugs.python.org/issue7022    pitrou                        
                                                                               

Marshal doesn't release GIL while dumping                           0 days
       http://bugs.python.org/issue7023    benjamin.peterson             
       patch                                                                   

Python 3.1 and 3.2 (dev) opensearch.xml still references 3.0        0 days
       http://bugs.python.org/issue7025    benjamin.peterson             
                                                                               

add assertIsInstance                                                0 days
       http://bugs.python.org/issue7031    georg.brandl                  
       patch, patch, easy, needs review                                        

While Loop Bug on a game of 21, help!                               0 days
       http://bugs.python.org/issue7034    PixelHead777                  
                                                                               



Top Issues Most Discussed (10)
______________________________

 27 str.title() misbehaves with apostrophes                            2 days
closed  http://bugs.python.org/issue7008   

 27 PEP 314 inconsistency (authors/author/maintainer)                  4 days
open    http://bugs.python.org/issue6992   

 23 zipfile.ZipFile overwrites files outside destination path         10 days
open    http://bugs.python.org/issue6972   

 14 no longer possible to hash arrays                                135 days
open    http://bugs.python.org/issue6071   

 11 ssl.SSLSocket.recv() implementation may not work with non-block  380 days
open    http://bugs.python.org/issue3890   

 10 regression in pywin32 build due to 2.6.3rc1                        2 days
closed  http://bugs.python.org/issue7020   

  9 unmarshaling of artificial strings can produce funny longs.        3 days
open    http://bugs.python.org/issue7019   

  9 httplib and array do not play together well                       33 days
closed  http://bugs.python.org/issue6790   

  7 Add int.hex for symmetry with float.hex                            0 days
open    http://bugs.python.org/issue7028   

  6 threading.local subclasses don't cleanup their state and it get    0 days
closed  http://bugs.python.org/issue6990   




From steven.bethard at gmail.com  Fri Oct  2 19:23:47 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 2 Oct 2009 10:23:47 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091002T113352-479@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
Message-ID: <d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>

On Fri, Oct 2, 2009 at 2:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Steven Bethard <steven.bethard <at> gmail.com> writes:
>>
>> But it's not much of a transition plan. Or are you suggesting:
>
> The question is why we want a transition plan that will bother everyone with no
> tangible benefits for the user.

I think Guido expressed my feelings pretty well:

On Wed, Sep 30, 2009 at 10:37 AM, Guido van Rossum <guido at python.org> wrote:
> In the long run, say Python 4, I think we don't need both, and we
> should get rid of one. My preference is still getting rid of
> %-formatting, due to the problems with it that prompted the design of
> {}-formatting (no need to reiterate the list here).

I agree with this 100% but I can't see it working unless we have some
sort of transition plan. Just saying "ok, switch your format strings
from % to {}" didn't work in Python 3.0 for various good reasons, and
I can't imagine it will work in Python 4.0 unless we have a transition
plan.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From kmtracey at gmail.com  Fri Oct  2 20:46:29 2009
From: kmtracey at gmail.com (Karen Tracey)
Date: Fri, 02 Oct 2009 14:46:29 -0400
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.co
 m>
References: <20090929202035.GA5492@gmail.com>
	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
	<9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
Message-ID: <4ac64a8d.1a135e0a.7580.11d0@mx.google.com>

At 05:15 PM 9/30/2009, Yuvgoog Greenle wrote:
>I like how python has a minimalistic and powerful syntax (-1 for the 
>break ___ PEP).
>
>Also, I really dislike the for/else ambiguity "butterflies".
>
>When is the else after a loop executed?
>1. When the loop isn't entered at all.
>2. When the loop terminates through exhaustion of the list (does 
>this include when the list was empty?)
>3. When the loop didn't exit because of a break statement.
>
>HINTS: The way django does it is opposite the way python does it and 
>there may be more than one correct answer.

Django's template language does not have for/else, it has for/empty: 
<http://docs.djangoproject.com/en/dev/ref/templates/builtins/#for-empty>http://docs.djangoproject.com/en/dev/ref/templates/builtins/#for-empty

This construct did come from an external snippet that used 'else' 
instead of 'empty'.  However when it was integrated into Django the 
'else' name was specifically rejected because it did the opposite of 
what for/else does in Python.

Karen


From python at rcn.com  Fri Oct  2 20:56:54 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 2 Oct 2009 11:56:54 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><loom.20091001T010253-42@post.gmane.org><loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com><loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
Message-ID: <ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>


[Steven Bethard]
>. Just saying "ok, switch your format strings
> from % to {}" didn't work in Python 3.0 for various good reasons, and
> I can't imagine it will work in Python 4.0 unless we have a transition
> plan.

Do the users get any say in this?
I imagine that some people are heavily invested in %-formatting.

Because there has been limited uptake on {}-formatting (afaict),
we still have limited experience with knowing that it is actually
better, less error-prone, easier to learn/rember, etc.   Outside
a handful of people on this list, I have yet to see anyone adopt
it as the preferred syntax. 

Raymond



From solipsis at pitrou.net  Fri Oct  2 21:08:21 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 2 Oct 2009 19:08:21 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><loom.20091001T010253-42@post.gmane.org><loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com><loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
Message-ID: <loom.20091002T210446-929@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> Because there has been limited uptake on {}-formatting (afaict),
> we still have limited experience with knowing that it is actually
> better, less error-prone, easier to learn/rember, etc.

It is known to be quite slower.

The following timings are on the py3k branch:

- with positional arguments:
$ ./python -m timeit -s "s='%s %s'; t = ('hello', 'world')" "s % t"
1000000 loops, best of 3: 0.313 usec per loop
$ ./python -m timeit -s "f='{} {}'.format; t = ('hello', 'world')" "f(*t)"
1000000 loops, best of 3: 0.572 usec per loop

- with named arguments:
$ ./python -m timeit -s "s='%(a)s %(b)s'; d = dict(a='hello', b='world')" "s % d"
1000000 loops, best of 3: 0.387 usec per loop
$ ./python -m timeit -s "f='{a} {b}'.format; d = dict(a='hello', b='world')"
"f(**d)"
1000000 loops, best of 3: 0.581 usec per loop


Regards

Antoine.



From mg at lazybytes.net  Fri Oct  2 21:43:48 2009
From: mg at lazybytes.net (Martin Geisler)
Date: Fri, 02 Oct 2009 21:43:48 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
Message-ID: <87y6nt37kb.fsf@hbox.dyndns.org>

"Raymond Hettinger" <python at rcn.com> writes:

> [Steven Bethard]
>>. Just saying "ok, switch your format strings
>> from % to {}" didn't work in Python 3.0 for various good reasons, and
>> I can't imagine it will work in Python 4.0 unless we have a transition
>> plan.
>
> Do the users get any say in this?

I'm a user! :-)

I hate calling methods on string literals, I think it looks very odd to
have code like this:

  "Displaying {0} of {1} revisions".format(x, y)

Will we be able to write this as

  "Displaying {0} of {1} revisions" % (x, y)

too?

> I imagine that some people are heavily invested in %-formatting.
>
> Because there has been limited uptake on {}-formatting (afaict), we
> still have limited experience with knowing that it is actually better,
> less error-prone, easier to learn/rember, etc. Outside a handful of
> people on this list, I have yet to see anyone adopt it as the
> preferred syntax.

I've skimmed over the PEP, and the new {}-syntax seems to have some nice
features. But I've not seen it used anywhere yet.

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091002/d78d80ea/attachment.pgp>

From barry at python.org  Fri Oct  2 22:21:43 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 2 Oct 2009 16:21:43 -0400
Subject: [Python-Dev] RELEASED Python 2.6.3
Message-ID: <5B2596C4-1128-4B5B-8471-FE0409369D2D@python.org>

On behalf of the Python community, I'm happy to announce the  
availability of Python 2.6.3.  This is the latest production-ready  
version in the Python 2.6 series.  Somewhere near 100 bugs have been  
fixed since Python 2.6.2 was released in April 2009.  Please see the  
NEWS file for all the gory details.

     http://www.python.org/download/releases/2.6.3/NEWS.txt

Source tarballs, Windows installers and the Mac OS X disk image can be  
downloaded from the Python 2.6.3 page.

     http://www.python.org/download/releases/2.6.3/

For more information on Python 2.6 in general, please see

     http://docs.python.org/whatsnew/2.6.html

Please report bugs for any Python version in the Python tracker.

     http://bugs.python.org

Enjoy,
-Barry

Barry Warsaw
barry at python.org
Python 2.6 Release Manager
(on behalf of the entire python-dev team)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091002/359432f5/attachment-0001.pgp>

From steven.bethard at gmail.com  Fri Oct  2 22:50:36 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 2 Oct 2009 13:50:36 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <87y6nt37kb.fsf@hbox.dyndns.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<87y6nt37kb.fsf@hbox.dyndns.org>
Message-ID: <d11dcfba0910021350p4769b2c2x3ec564e63262cde5@mail.gmail.com>

On Fri, Oct 2, 2009 at 12:43 PM, Martin Geisler <mg at lazybytes.net> wrote:
> I hate calling methods on string literals, I think it looks very odd to
> have code like this:
>
> ?"Displaying {0} of {1} revisions".format(x, y)
>
> Will we be able to write this as
>
> ?"Displaying {0} of {1} revisions" % (x, y)
>
> too?

I doubt it. One of the major complaints about the %-style formatting
was that the use of % produced (somewhat) unexpected errors because of
how operator precedence works::

>>> '{0}'.format(4 + 1)
'5'
>>> '%s' % 4 + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot concatenate 'str' and 'int' objects

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From jan.hosang at gmail.com  Fri Oct  2 23:32:38 2009
From: jan.hosang at gmail.com (Jan Hosang)
Date: Fri, 2 Oct 2009 23:32:38 +0200
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on
	Mac OS X?
In-Reply-To: <ee9557410910020129w1ed7be0fie2c45d099209189e@mail.gmail.com>
References: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
	<92839421634652657618189239787554831247-Webmail@me.com>
	<ee9557410910020129w1ed7be0fie2c45d099209189e@mail.gmail.com>
Message-ID: <2ECCFBA0-1A70-4D12-925C-CE620D430AAD@gmail.com>

Confirmed on 10.6 for 2.6.3 and 2.7a0. For 3.2a0 both asserts fail.

On 02.10.2009, at 10:29, INADA Naoki wrote:

>> Do you have a testcase that shows what the problem is?
>>
>> Ronald
>
>>>> s = '\xa0'
>>>> assert s.strip() == s
>>>> import locale
>>>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
> 'en_US.UTF-8'
>>>> assert s.strip() == s
>
> Second assert failed on Snow Leopard.
>
> -- 
> Naoki INADA  <songofacandy at gmail.com>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jan.hosang%40gmail.com


From steven.bethard at gmail.com  Fri Oct  2 23:59:25 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 2 Oct 2009 14:59:25 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
Message-ID: <d11dcfba0910021459s1f663040he8505079d57e7c9e@mail.gmail.com>

On Fri, Oct 2, 2009 at 11:56 AM, Raymond Hettinger <python at rcn.com> wrote:
> [Steven Bethard]
>> Just saying "ok, switch your format strings
>> from % to {}" didn't work in Python 3.0 for various good reasons, and
>> I can't imagine it will work in Python 4.0 unless we have a transition
>> plan.
>
> Do the users get any say in this?
> I imagine that some people are heavily invested in %-formatting.
>
> Because there has been limited uptake on {}-formatting (afaict),
> we still have limited experience with knowing that it is actually
> better, less error-prone, easier to learn/rember, etc. ? Outside
> a handful of people on this list, I have yet to see anyone adopt
> it as the preferred syntax.

Sure, I guess this is a possibility too, and it could make the
transition process I have to work through for argparse much easier.
;-)

To be clear, are you suggesting that APIs that currently support only
%-formatting shouldn't bother supporting {}-formatting at all? Or are
you suggesting that they should support both, but support for
%-formatting should never go away?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From songofacandy at gmail.com  Sat Oct  3 01:40:52 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Sat, 3 Oct 2009 08:40:52 +0900
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on
	Mac OS X?
In-Reply-To: <2ECCFBA0-1A70-4D12-925C-CE620D430AAD@gmail.com>
References: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
	<92839421634652657618189239787554831247-Webmail@me.com>
	<ee9557410910020129w1ed7be0fie2c45d099209189e@mail.gmail.com>
	<2ECCFBA0-1A70-4D12-925C-CE620D430AAD@gmail.com>
Message-ID: <ee9557410910021640o2e2ae1ceyf48cf6736fc11c8@mail.gmail.com>

> Confirmed on 10.6 for 2.6.3 and 2.7a0. For 3.2a0 both asserts fail.

OK.
`s = '\xa0'` should be `s = b'\xa0'`.

Should I file a bug?

-- 
Naoki INADA  <songofacandy at gmail.com>

From andrew.svetlov at gmail.com  Sat Oct  3 03:06:24 2009
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Fri, 2 Oct 2009 21:06:24 -0400
Subject: [Python-Dev] PEP 3121
Message-ID: <270ea8310910021806v667776f6x419470ec25d5ba56@mail.gmail.com>

What real status of this document?

As I figured out py3k trunk uses this ones.
Most part of 'battery included' modules don't use this feature,
leaving m_traverse, m_clear and m_free NULL pointers.

There are only exception: _io.

But looks like all underground python machinery is already ported to
support PEP.

Maybe it's good point to update PEP?
"accepted; may not be implemented yet" sounds a bit confusing.
Just insert somethere close to top of content: "Implemented by Python
itself, but builtin modules should be converted".

Next question: are where all 'python standard library' modules looks
like "not converted yet" really need to be refactored?
Or we can split this huge set to 'looks already good' and 'requires some work'?
And just add comment for every PyModuleDef definition like:
"Completely satisfy PEP 3121, signed and stamped by Martin von Loewis
and GvR".?

Last question: PEP refers to "multiple interpreters" feature of python.
This feature presented at least in 2.xx branch (don't remember about
1.xx - up to 2003 I used python as scripting language only and not
care about C interface).
I never saw usage of this one in real use cases.
Is there projects (preferable with code sources, of course)  what runs
multiple interpreter in same process?

From tjreedy at udel.edu  Sat Oct  3 03:39:11 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 02 Oct 2009 21:39:11 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
Message-ID: <ha69vt$qso$1@ger.gmane.org>

Steven Bethard wrote:
> On Thu, Oct 1, 2009 at 10:49 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> As someone who likes .format() and who already uses such bound methods to
>> print, such as in
>>
>> emsg = "...".format
>> ...
>>   if c: print(emsg(arg, barg))
>>
>> I find this **MUCH** preferable to the ugly and seemingly unnecessary
>> wrapper class idea being bandied about. This would be scarcely worse than
>> passing the string itself.
> 
> But it's not much of a transition plan.

It is a 'plan' to transition from not being able to use the new 
formatting, which I prefer, throughout the stdlib, to being able to do so.

I believe most, even if not all, find that acceptable. Certainly, I 
think you should be able to implement the above for argparse before 
submitting it. And I would hope that 3.2, in a year, is generally 
.format usable.

This is the first step in a possible long-term replacement, but there is 
currently no consensus to do any more than this. So I think it premature 
to do any more. I would agree, for instance, that an auto-translation 
tool is needed.

Terry Jan Reedy


From python at rcn.com  Sat Oct  3 05:03:58 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 2 Oct 2009 20:03:58 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
Message-ID: <837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>


[Terry Reedy]
> I would agree, for instance, that an auto-translation 
> tool is needed.

We should get one written.  ISTM, every %-formatting
string is directly translatable to an equivalent {}-formatting string.


Raymond


From benjamin at python.org  Sat Oct  3 05:11:35 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 2 Oct 2009 22:11:35 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
Message-ID: <1afaf6160910022011w46ff842ax4ee0b56a9b86cc45@mail.gmail.com>

2009/10/2 Raymond Hettinger <python at rcn.com>:
>
> [Terry Reedy]
>>
>> I would agree, for instance, that an auto-translation tool is needed.
>
> We should get one written. ?ISTM, every %-formatting
> string is directly translatable to an equivalent {}-formatting string.

Yes, but not all are possible to guess with mere static analysis.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sat Oct  3 07:15:00 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 03 Oct 2009 15:15:00 +1000
Subject: [Python-Dev] PEP 3121
In-Reply-To: <270ea8310910021806v667776f6x419470ec25d5ba56@mail.gmail.com>
References: <270ea8310910021806v667776f6x419470ec25d5ba56@mail.gmail.com>
Message-ID: <4AC6DDD4.1040608@gmail.com>

Andrew Svetlov wrote:
> Maybe it's good point to update PEP?
> "accepted; may not be implemented yet" sounds a bit confusing.

That's the status though - the PEP is accepted, implementation is ongoing.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Oct  3 07:15:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 03 Oct 2009 15:15:53 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC29F71.3000307@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	<4AC2846E.8060304@g.nevcal.com>	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<4AC29F71.3000307@gmail.com>
Message-ID: <4AC6DE09.3030003@gmail.com>

Toshio Kuratomi wrote:
> About exit(), I agree with others about wanting to catch the exception
> myself and then choosing to exit from the code.  I'm not sure that it's
> actually useful in practice, though...it might just feel cleaner but not
> actually be that helpful.

As others have pointed out, if you want to do that then you can just
catch SystemExit.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Oct  3 07:19:55 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 03 Oct 2009 15:19:55 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910021350p4769b2c2x3ec564e63262cde5@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<loom.20091002T113352-479@post.gmane.org>	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>	<87y6nt37kb.fsf@hbox.dyndns.org>
	<d11dcfba0910021350p4769b2c2x3ec564e63262cde5@mail.gmail.com>
Message-ID: <4AC6DEFB.4040402@gmail.com>

Steven Bethard wrote:
> On Fri, Oct 2, 2009 at 12:43 PM, Martin Geisler <mg at lazybytes.net> wrote:
>> I hate calling methods on string literals, I think it looks very odd to
>> have code like this:
>>
>>  "Displaying {0} of {1} revisions".format(x, y)
>>
>> Will we be able to write this as
>>
>>  "Displaying {0} of {1} revisions" % (x, y)
>>
>> too?
> 
> I doubt it. One of the major complaints about the %-style formatting
> was that the use of % produced (somewhat) unexpected errors because of
> how operator precedence works::
> 
>>>> '{0}'.format(4 + 1)
> '5'
>>>> '%s' % 4 + 1
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: cannot concatenate 'str' and 'int' objects
> 
> Steve

The other major problem with the use of the mod operator is the bugs
encountered with "fmt % obj" when obj happened to be a tuple or a dict.

So no, the switch to a method rather than an operator was deliberate.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tjreedy at udel.edu  Sat Oct  3 07:23:33 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 03 Oct 2009 01:23:33 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <1afaf6160910022011w46ff842ax4ee0b56a9b86cc45@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<1afaf6160910022011w46ff842ax4ee0b56a9b86cc45@mail.gmail.com>
Message-ID: <ha6n4i$i2i$1@ger.gmane.org>

Benjamin Peterson wrote:
> 2009/10/2 Raymond Hettinger <python at rcn.com>:
>> [Terry Reedy]
>>> I would agree, for instance, that an auto-translation tool is needed.
>> We should get one written.  ISTM, every %-formatting
>> string is directly translatable to an equivalent {}-formatting string.
> 
> Yes, but not all are possible to guess with mere static analysis.

Even if perfect is not possible, decently good would probably cover a 
lot of use cases.


From ncoghlan at gmail.com  Sat Oct  3 07:24:41 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 03 Oct 2009 15:24:41 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><loom.20091001T010253-42@post.gmane.org><loom.20091001T024614-990@post.gmane.org>	<ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com><loom.20091002T113352-479@post.gmane.org>	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
Message-ID: <4AC6E019.4000701@gmail.com>

Raymond Hettinger wrote:
> Because there has been limited uptake on {}-formatting (afaict),
> we still have limited experience with knowing that it is actually
> better, less error-prone, easier to learn/rember, etc.   Outside
> a handful of people on this list, I have yet to see anyone adopt
> it as the preferred syntax.

A self-fulfilling prophecy if ever I heard one... uptake is limited
because there's a large legacy code base that doesn't use it and many
APIs don't support it, so we shouldn't bother trying to increase the
number of APIs that *do* support it?

I'm starting to think that a converter between the two format
mini-languages may be the way to go though.

fmt_braces is meant to provide a superset of the capabilites of
fmt_percent, so a forward converter shouldn't be too hard. A reverse
converter may have to punt with ValueError when it finds things that
cannot be expressed in the fmt_percent mini language though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From stephen at xemacs.org  Sat Oct  3 07:37:55 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 03 Oct 2009 14:37:55 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC33BA1.8050203@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
Message-ID: <878wfthwb0.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > However, while I'd still be a little happier if the .ip attribute
 > went away all together and another means was found to conveniently
 > associate an IPAddress and an IPNetwork, keeping it doesn't bother
 > me anywhere near as much as having network equivalence defined in
 > terms of something other than the network ID and the netmask.

s/equivalence/equality/

There are several notions of equivalence in play, and all can be
implemented as methods.  The question is which one is going to be "==".

From larry.bugbee at boeing.com  Sat Oct  3 08:08:10 2009
From: larry.bugbee at boeing.com (Bugbee, Larry)
Date: Fri, 2 Oct 2009 23:08:10 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
Message-ID: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>

> > Do the users get any say in this?
> 
> I'm a user! :-)
> 
> I hate calling methods on string literals, I think it looks very odd
> to have code like this:
> 
>   "Displaying {0} of {1} revisions".format(x, y)

Ugh!  Good point.  

Is Python to be an easy-to-learn-and-remember language?  I submit we are losing that one.  To a user, this will be confusing.  To a C programmer coming over to Python, especially so.  Some of what makes %-formatting easy to remember is its parallel in C.  

I'm conflicted.  Philosophically I like the idea of mnemonic names over positional variables and allowing variable values determined elsewhere to be inserted in print strings.  It is appealing.  

Unless the benefit is at least 2x, a change should not be made, and I don't think this benefit rises to where it is worth the confusion and problems.  ...and converting the legacy base.  And forget pretty, not that %-formatting is pretty either.  Besides, according to the bench, it is slower too.  And it will take editors a while before the new syntax is supported and colorized, thus some errors for a while.  

...and if one wants a "{" or a "}" in the printed output, one has to escape it?  That is -2x over wanting a "%" in the output.  

So until I see a *significant* benefit, my vote is *not* remove %-formatting.  Make both available and if {} is to win, it will.  


From foom at fuhm.net  Sat Oct  3 08:36:38 2009
From: foom at fuhm.net (James Y Knight)
Date: Sat, 3 Oct 2009 02:36:38 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><loom.20091001T010253-42@post.gmane.org><loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com><loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
Message-ID: <A2C65B8B-9FC2-41DE-A671-5D6307347058@fuhm.net>

On Oct 2, 2009, at 2:56 PM, Raymond Hettinger wrote:
> Do the users get any say in this?
> I imagine that some people are heavily invested in %-formatting.
>
> Because there has been limited uptake on {}-formatting (afaict),
> we still have limited experience with knowing that it is actually
> better, less error-prone, easier to learn/rember, etc.   Outside
> a handful of people on this list, I have yet to see anyone adopt
> it as the preferred syntax.

Well, I actually think it was a pretty bad idea to introduce {}  
formatting, because %-formatting is well-known in many other  
languages, and $-formatting is used by basically all the rest. So the  
introduction of {}-formatting has always seemed silly to me, and I  
wish it had not happened.

HOWEVER, much worse than having a new, different, and strange  
formatting convention is having *multiple* formatting conventions  
arbitrarily used in different places within the language, with no  
rhyme or reason.

So, given that brace-formatting was added, and that it's been declared  
the way forward, I'd *greatly* prefer it taking over everywhere in  
python, instead of having to use a mixture.

James

From g.brandl at gmx.net  Sat Oct  3 11:22:04 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 03 Oct 2009 11:22:04 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
Message-ID: <ha758a$ebc$1@ger.gmane.org>

Bugbee, Larry schrieb:
>>> Do the users get any say in this?
>> 
>> I'm a user! :-)
>> 
>> I hate calling methods on string literals, I think it looks very odd to
>> have code like this:
>> 
>> "Displaying {0} of {1} revisions".format(x, y)
> 
> Ugh!  Good point.
> 
> Is Python to be an easy-to-learn-and-remember language?  I submit we are
> losing that one.  To a user, this will be confusing.  To a C programmer
> coming over to Python, especially so.  Some of what makes %-formatting easy
> to remember is its parallel in C.

To a .NET programmer coming to Python (think IronPython), %-formatting will
be strange, while {}-style formatting will be familiar.  I don't know how many
programmers come to Python from C nowadays.

I also think that we are merely so accustomed to ``"..." % bar`` that we never
think about new users who go "Huh? String modulo string?" -- why should that
initial discomfort be less than that of calling a method on a literal?

Being able to call a method on literals is simply consistent; literals are
objects as well as an object referred to by a name.  There are languages where
you can't even call ``foo[bar].baz()`` because method calls only work on single
names.  I call that ugly.

> I'm conflicted.  Philosophically I like the idea of mnemonic names over
> positional variables and allowing variable values determined elsewhere to be
> inserted in print strings.  It is appealing.
> 
> Unless the benefit is at least 2x, a change should not be made,

I don't see how numbers like "2x" can be applied when measuring the
benefit of a language feature.

> and I don't
> think this benefit rises to where it is worth the confusion and problems.
> ...and converting the legacy base.  And forget pretty, not that %-formatting
> is pretty either.  Besides, according to the bench, it is slower too.  And it
> will take editors a while before the new syntax is supported and colorized,
> thus some errors for a while.

This is no different from all other language features.  Some editors I
encountered do not yet color the with statement.  Should I therefore refrain
from using it?

Highlighting formatting placeholders in string literals is questionable anyway,
since a highlighter has no way of knowing whether a given string literal will
be used for formatting.

> ....and if one wants a "{" or a "}" in the printed output, one has to escape
> it?  That is -2x over wanting a "%" in the output.

Again a number.  Where does it come from?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ncoghlan at gmail.com  Sat Oct  3 11:40:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 03 Oct 2009 19:40:19 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <878wfthwb0.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>	<4AC12AC6.80304@v.loewis.de>	<4AC2A9D7.4080603@gmail.com>	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>	<4AC33BA1.8050203@gmail.com>
	<878wfthwb0.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4AC71C03.5020904@gmail.com>

Stephen J. Turnbull wrote:
> Nick Coghlan writes:
> 
>  > However, while I'd still be a little happier if the .ip attribute
>  > went away all together and another means was found to conveniently
>  > associate an IPAddress and an IPNetwork, keeping it doesn't bother
>  > me anywhere near as much as having network equivalence defined in
>  > terms of something other than the network ID and the netmask.
> 
> s/equivalence/equality/
> 
> There are several notions of equivalence in play, and all can be
> implemented as methods.  The question is which one is going to be "==".

I have just enough pure math in my educational background to think of
'equality' as the 'default equivalence class', but yes, the meaning of
"==" is exactly what I was referring to.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ubershmekel at gmail.com  Sat Oct  3 12:45:09 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Sat, 3 Oct 2009 13:45:09 +0300
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC6DE09.3030003@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>
Message-ID: <9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>

I haven't checked if it's possible, but I suggest Argparse have it's
own exception class that inherits from SystemExit and that exception
would be thrown.

ParseError, or something similar.

I suggest this just because it would be more readable I guess and
would exactly explain why this code exits.

On Sat, Oct 3, 2009 at 8:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Toshio Kuratomi wrote:
>> About exit(), I agree with others about wanting to catch the exception
>> myself and then choosing to exit from the code. ?I'm not sure that it's
>> actually useful in practice, though...it might just feel cleaner but not
>> actually be that helpful.
>
> As others have pointed out, if you want to do that then you can just
> catch SystemExit.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>

From steven.bethard at gmail.com  Sat Oct  3 17:12:49 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 3 Oct 2009 08:12:49 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>
	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>
Message-ID: <d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>

On Sat, Oct 3, 2009 at 3:45 AM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> I haven't checked if it's possible, but I suggest Argparse have it's
> own exception class that inherits from SystemExit and that exception
> would be thrown.
>
> ParseError, or something similar.
>
> I suggest this just because it would be more readable I guess and
> would exactly explain why this code exits.

I've never seen such an idiom before (subclassing SystemExit) but it
would certainly be possible create an ArgumentParserExit exception
like that. Then you would have your choice of overriding .exit() or
catching the exception.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From fuzzyman at voidspace.org.uk  Sat Oct  3 17:17:36 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 03 Oct 2009 16:17:36 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	<4AC2846E.8060304@g.nevcal.com>	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>	<4AC29F71.3000307@gmail.com>
	<4AC6DE09.3030003@gmail.com>	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>
	<d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>
Message-ID: <4AC76B10.1000209@voidspace.org.uk>

Steven Bethard wrote:
> On Sat, Oct 3, 2009 at 3:45 AM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
>   
>> I haven't checked if it's possible, but I suggest Argparse have it's
>> own exception class that inherits from SystemExit and that exception
>> would be thrown.
>>
>> ParseError, or something similar.
>>
>> I suggest this just because it would be more readable I guess and
>> would exactly explain why this code exits.
>>     
>
> I've never seen such an idiom before (subclassing SystemExit) but it
> would certainly be possible create an ArgumentParserExit exception
> like that. Then you would have your choice of overriding .exit() or
> catching the exception.
>
> Steve
>   
Why not just catch SystemExit? If you want a custom exception the 
overriding .exit() should be sufficient.

I'd be much more interested in Guido's suggestion of auto-generated 
custom help messages for sub-commands.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From steven.bethard at gmail.com  Sat Oct  3 17:41:36 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 3 Oct 2009 08:41:36 -0700
Subject: [Python-Dev] summary of transitioning from % to {} formatting
Message-ID: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>

I thought it might be useful for those who don't have time to read a
million posts to have a summary of what's happened in the formatting
discussion.

The basic problem is that many APIs in the standard library and
elsewhere support only %-formatting and not {}-formatting, e.g.
logging.Formatter accepts::
  logging.Formatter(fmt="%(asctime)s - %(name)s")
but not::
  logging.Formatter(fmt="{asctime} - {name}")

There seems to be mostly agreement that these APIs should at least
support both formatting styles, and a sizable group (including Guido)
believe that %-formatting should eventually be phased out (e.g. by
Python 4). There are a number of competing proposals on how to allow
such APIs to start accepting {}-format strings:

* Add a parameter which declares the type of format string::
    logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
  The API code would then switch between %-format and {}-format
  based on the value of that parameter. If %-formatting is to be
  deprecated, this could be done by first deprecating
  format=PERCENTS and requiring format=BRACES, and then changing the
  default to format=BRACES.

* Create string subclasses which convert % use to .format calls::
    __ = brace_fmt
    logging.Formatter(fmt=__("{asctime} - {name}"))
  The API code wouldn't have to change at all at first, as applying
  % to brace_fmt objects would call .format() instead. If
  %-formatting is to be deprecated, this could be done by first
  deprecating plain strings and requiring brace_fmt strings, and
  then allowing plain strings again but assuming they are {}-format
  strings.

* Teach the API to accept callables as well as strings::
    logging.Formatter(fmt="{asctime} - {name}".format)
  The API code would just call the object with .format() style
  arguments if a callable was given instead of a string. If
  %-formatting is to be deprecated, this could be done by first
  deprecating plain strings and requiring callables, and then
  allowing plain strings again but assuming they are {}-format
  strings

* Create translators between %-format and {}-format::
    assert to_braces("%(asctime)s") == "{asctime}"
    assert to_percents("{asctime}") == "%(asctime)s"
  these could then either be used outside of the API::
    logging.Formatter(fmt=to_percents("{asctime} - {name}"))
  or they could be used within the API combined with some sort of
  heuristic for guessing whether a {}-format string or a %-format
  string was passed in::
    logging.Formatter(fmt="{asctime} - {name}")
  If %-formatting is to be deprecated, the transition strategy here
  is trivial. However, no one has yet written translators, and it is
  not clear what heuristics should be used, e.g. should the method
  just try %-formatting first and then {}-formatting if it fails?

I don't think there is consensus yet on which of these proposals
should be the "blessed" one.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Sat Oct  3 17:49:18 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 3 Oct 2009 08:49:18 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC76B10.1000209@voidspace.org.uk>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>
	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>
	<d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>
	<4AC76B10.1000209@voidspace.org.uk>
Message-ID: <d11dcfba0910030849v111160abge291d24ad980507a@mail.gmail.com>

On Sat, Oct 3, 2009 at 8:17 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Steven Bethard wrote:
>> On Sat, Oct 3, 2009 at 3:45 AM, Yuvgoog Greenle <ubershmekel at gmail.com>
>> wrote:
>>> I haven't checked if it's possible, but I suggest Argparse have it's
>>> own exception class that inherits from SystemExit and that exception
>>> would be thrown.
>>
>> I've never seen such an idiom before (subclassing SystemExit) but it
>> would certainly be possible create an ArgumentParserExit exception
>> like that. Then you would have your choice of overriding .exit() or
>> catching the exception.
>
> Why not just catch SystemExit? If you want a custom exception the overriding
> .exit() should be sufficient.

I'm certainly fine with that -- I'm just trying to make sure I've
addressed whatever needs addressed to get argparse in.

> I'd be much more interested in Guido's suggestion of auto-generated custom
> help messages for sub-commands.

Maybe I misunderstood, but I think this is already the default
argparse behavior, no?

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
subparsers = parser.add_subparsers()
parser1 = subparsers.add_parser('1')
parser1.add_argument('--bar')
parser2 = subparsers.add_parser('2')
parser2.add_argument('baz')
parser.parse_args(['--help'])

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> subparsers = parser.add_subparsers()
>>> parser1 = subparsers.add_parser('1')
>>> parser1.add_argument('--bar')
>>> parser2 = subparsers.add_parser('2')
>>> parser2.add_argument('baz')

>>> # top level argument help
>>> parser.parse_args(['--help'])
usage: [-h] [--foo FOO] {1,2} ...

positional arguments:
  {1,2}

optional arguments:
  -h, --help  show this help message and exit
  --foo FOO

>>> # help for subparser 1
>>> parser.parse_args(['1', '--help'])
usage:  1 [-h] [--bar BAR]

optional arguments:
  -h, --help  show this help message and exit
  --bar BAR

>>> # help for subparser 2
>>> parser.parse_args(['2', '--help'])
usage:  2 [-h] baz

positional arguments:
  baz

optional arguments:
  -h, --help  show this help message and exit

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From solipsis at pitrou.net  Sat Oct  3 18:00:48 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Oct 2009 16:00:48 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?summary_of_transitioning_from_=25_to_=7B?=
	=?utf-8?q?=7D_formatting?=
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
Message-ID: <loom.20091003T174748-914@post.gmane.org>

Steven Bethard <steven.bethard <at> gmail.com> writes:
> 
>   If %-formatting is to be deprecated, the transition strategy here
>   is trivial. However, no one has yet written translators, and it is
>   not clear what heuristics should be used, e.g. should the method
>   just try %-formatting first and then {}-formatting if it fails?

This would be a reasonable heuristic. It should be done only on the first call,
though, and then the result remembered (for both performance reasons and
consistency).

The cases where a format string would function in both formatting styles and
expect the same parameters must be very rare in the real world.

Regards

Antoine.



From p.f.moore at gmail.com  Sat Oct  3 18:08:13 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 3 Oct 2009 17:08:13 +0100
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <loom.20091003T174748-914@post.gmane.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<loom.20091003T174748-914@post.gmane.org>
Message-ID: <79990c6b0910030908ia71714cpd6f7b06afa85191@mail.gmail.com>

2009/10/3 Antoine Pitrou <solipsis at pitrou.net>:
> Steven Bethard <steven.bethard <at> gmail.com> writes:
>>
>> ? If %-formatting is to be deprecated, the transition strategy here
>> ? is trivial. However, no one has yet written translators, and it is
>> ? not clear what heuristics should be used, e.g. should the method
>> ? just try %-formatting first and then {}-formatting if it fails?
>
> This would be a reasonable heuristic. It should be done only on the first call,
> though, and then the result remembered (for both performance reasons and
> consistency).
>
> The cases where a format string would function in both formatting styles and
> expect the same parameters must be very rare in the real world.

Define "fails":

>>> "{a} {b} c" % {'a':12}
'{a} {b} c'

That didn't fail...

Paul.

From rdmurray at bitdance.com  Sat Oct  3 18:19:32 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 3 Oct 2009 12:19:32 -0400 (EDT)
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <79990c6b0910030908ia71714cpd6f7b06afa85191@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<loom.20091003T174748-914@post.gmane.org>
	<79990c6b0910030908ia71714cpd6f7b06afa85191@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0910031218100.18193@kimball.webabinitio.net>

On Sat, 3 Oct 2009 at 17:08, Paul Moore wrote:
> 2009/10/3 Antoine Pitrou <solipsis at pitrou.net>:
>> Steven Bethard <steven.bethard <at> gmail.com> writes:
>>>
>>> ? If %-formatting is to be deprecated, the transition strategy here
>>> ? is trivial. However, no one has yet written translators, and it is
>>> ? not clear what heuristics should be used, e.g. should the method
>>> ? just try %-formatting first and then {}-formatting if it fails?
>>
>> This would be a reasonable heuristic. It should be done only on the first call,
>> though, and then the result remembered (for both performance reasons and
>> consistency).
>>
>> The cases where a format string would function in both formatting styles and
>> expect the same parameters must be very rare in the real world.
>
> Define "fails":
>
>>>> "{a} {b} c" % {'a':12}
> '{a} {b} c'
>
> That didn't fail...

Also, what if both fail?  Which failure's error message gets printed?

--David (RDM)

From solipsis at pitrou.net  Sat Oct  3 18:23:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 03 Oct 2009 18:23:24 +0200
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <79990c6b0910030908ia71714cpd6f7b06afa85191@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<loom.20091003T174748-914@post.gmane.org>
	<79990c6b0910030908ia71714cpd6f7b06afa85191@mail.gmail.com>
Message-ID: <1254587004.5470.8.camel@localhost>


> Define "fails":
> 
> >>> "{a} {b} c" % {'a':12}
> '{a} {b} c'
> 
> That didn't fail...

Ah, my bad. I had completely overlooked that formatting was laxist when
faced with unused named parameters.

Then we need something smarter, like counting the number of unescaped
"%" characters, the number of pairs of braces, and try first the
formatting style expecting the greatest number of parameters. It can
either be done cheaply in Python, or more rigourously by providing an
extra API to the existing C parsing routines.

This might look fragile, but we should not forget that we are talking
about uses - logging, etc. - which will most of time involve very simple
format strings, such that making an automatic decision is not too
difficult. In the more undecideable cases, the heuristic might decide to
raise an exception and require the developer to specify the formatting
style explicitly (by adding e.g. style='%' or style='{}' to the method
call).

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Sat Oct  3 18:21:43 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 03 Oct 2009 17:21:43 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0910030849v111160abge291d24ad980507a@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	
	<4AC2846E.8060304@g.nevcal.com>	
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>	
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>	
	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>	
	<d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>	
	<4AC76B10.1000209@voidspace.org.uk>
	<d11dcfba0910030849v111160abge291d24ad980507a@mail.gmail.com>
Message-ID: <4AC77A17.7050101@voidspace.org.uk>

Steven Bethard wrote:
> [snip...]
>> I'd be much more interested in Guido's suggestion of auto-generated custom
>> help messages for sub-commands.
>>     
>
> Maybe I misunderstood, but I think this is already the default
> argparse behavior, no?
>
>   
Cool. I didn't realise that help for subcommands was already 
implemented. :-)

Michael


> import argparse
> parser = argparse.ArgumentParser()
> parser.add_argument('--foo')
> subparsers = parser.add_subparsers()
> parser1 = subparsers.add_parser('1')
> parser1.add_argument('--bar')
> parser2 = subparsers.add_parser('2')
> parser2.add_argument('baz')
> parser.parse_args(['--help'])
>
>   
>>>> import argparse
>>>> parser = argparse.ArgumentParser()
>>>> parser.add_argument('--foo')
>>>> subparsers = parser.add_subparsers()
>>>> parser1 = subparsers.add_parser('1')
>>>> parser1.add_argument('--bar')
>>>> parser2 = subparsers.add_parser('2')
>>>> parser2.add_argument('baz')
>>>>         
>
>   
>>>> # top level argument help
>>>> parser.parse_args(['--help'])
>>>>         
> usage: [-h] [--foo FOO] {1,2} ...
>
> positional arguments:
>   {1,2}
>
> optional arguments:
>   -h, --help  show this help message and exit
>   --foo FOO
>
>   
>>>> # help for subparser 1
>>>> parser.parse_args(['1', '--help'])
>>>>         
> usage:  1 [-h] [--bar BAR]
>
> optional arguments:
>   -h, --help  show this help message and exit
>   --bar BAR
>
>   
>>>> # help for subparser 2
>>>> parser.parse_args(['2', '--help'])
>>>>         
> usage:  2 [-h] baz
>
> positional arguments:
>   baz
>
> optional arguments:
>   -h, --help  show this help message and exit
>
> Steve
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From barry at python.org  Sat Oct  3 18:30:59 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 3 Oct 2009 12:30:59 -0400
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
Message-ID: <05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>

On Oct 3, 2009, at 11:41 AM, Steven Bethard wrote:

> I thought it might be useful for those who don't have time to read a
> million posts to have a summary of what's happened in the formatting
> discussion.
>
> The basic problem is that many APIs in the standard library and
> elsewhere support only %-formatting and not {}-formatting, e.g.
> logging.Formatter accepts::
>  logging.Formatter(fmt="%(asctime)s - %(name)s")
> but not::
>  logging.Formatter(fmt="{asctime} - {name}")

I'd like to at least keep in mind $-strings.  I don't have any  
experience with {}-strings w.r.t. i18n translators, but I'm very  
confident that $-strings are better for translators than %-strings.   
OT1H {}-strings don't suffer from the #1 problem of %-strings: leaving  
off the trailing 's' or whatever.  OTOH, I suspect that $-strings are  
still easier for the simple substitution case; for example, it's  
conceivable that translators may forget the trailing close brace.

Since we're likely to have multiple formatting styles at least for  
Python 3's lifetime, I'd like any solution we come up with to at least  
not preclude the use of $-strings.

I also don't think this is a case of anti-TOOWTDI.  For most  
situations {}-strings are great (IMO), but in the specific translation  
domain, I suspect $-strings are still better.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091003/11371b26/attachment.pgp>

From fuzzyman at voidspace.org.uk  Sat Oct  3 18:40:45 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 03 Oct 2009 17:40:45 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	
	<4AC2846E.8060304@g.nevcal.com>	
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>	
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>	
	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>	
	<d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>	
	<4AC76B10.1000209@voidspace.org.uk>	
	<d11dcfba0910030849v111160abge291d24ad980507a@mail.gmail.com>	
	<4AC77A17.7050101@voidspace.org.uk>
	<9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
Message-ID: <4AC77E8D.9070205@voidspace.org.uk>

Yuvgoog Greenle wrote:
> On Sat, Oct 3, 2009 at 7:21 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>   
>> [snip...]
>> Why not just catch SystemExit? If you want a custom exception the overriding .exit() should be sufficient.
>> I'd be much more interested in Guido's suggestion of auto-generated custom help messages for sub-commands.
>>     
>
> Check it out:
>
> def ParseAndRun():
>     crazy_external_function_that_might_exit()
>
>     # Argparse blah blah
>     parser.parse_args()
>
> if __name__ == "__main__":
>     try:
>         ParseAndRun()
>     except SystemExit:
>         # was it crazy_external_function_that_might_exit or an argparse error?
>
>
> I know this might come through as bike shedding but it's just
> customary python that every module have it's own exception types as to
> not mix them up with others.
>
>   

Then subclass and override .exit() as discussed - or put proper 
exception handling around the call to parse_args() (optionally 
rethrowing with whatever custom exception type you wish).

Michael
> --yuv
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From dickinsm at gmail.com  Sat Oct  3 18:34:41 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 3 Oct 2009 17:34:41 +0100
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
Message-ID: <5c6f2a5d0910030934u7399d871yfb71f3319ce6183e@mail.gmail.com>

On Sat, Oct 3, 2009 at 4:41 PM, Steven Bethard <steven.bethard at gmail.com> wrote:
> I thought it might be useful for those who don't have time to read a
> million posts to have a summary of what's happened in the formatting
> discussion.

Definitely useful.  Thanks for the summary!

[...]

> * Add a parameter which declares the type of format string::
> ? ?logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
> ?The API code would then switch between %-format and {}-format
> ?based on the value of that parameter. If %-formatting is to be
> ?deprecated, this could be done by first deprecating
> ?format=PERCENTS and requiring format=BRACES, and then changing the
> ?default to format=BRACES.

+1.

> * Create string subclasses which convert % use to .format calls::
> ? ?__ = brace_fmt
> ? ?logging.Formatter(fmt=__("{asctime} - {name}"))
> ?The API code wouldn't have to change at all at first, as applying
> ?% to brace_fmt objects would call .format() instead. If
> ?%-formatting is to be deprecated, this could be done by first
> ?deprecating plain strings and requiring brace_fmt strings, and
> ?then allowing plain strings again but assuming they are {}-format
> ?strings.

Uurgh.  This just feels... icky.  A badly-rationalized -1 from me.

> * Teach the API to accept callables as well as strings::
> ? ?logging.Formatter(fmt="{asctime} - {name}".format)
> ?The API code would just call the object with .format() style
> ?arguments if a callable was given instead of a string. If
> ?%-formatting is to be deprecated, this could be done by first
> ?deprecating plain strings and requiring callables, and then
> ?allowing plain strings again but assuming they are {}-format
> ?strings

+0.5.  Seems like it could work, but the first solution feels
cleaner.

> * Create translators between %-format and {}-format::
> ? ?assert to_braces("%(asctime)s") == "{asctime}"
> ? ?assert to_percents("{asctime}") == "%(asctime)s"
> ?these could then either be used outside of the API::
> ? ?logging.Formatter(fmt=to_percents("{asctime} - {name}"))
> ?or they could be used within the API combined with some sort of
> ?heuristic for guessing whether a {}-format string or a %-format
> ?string was passed in::
> ? ?logging.Formatter(fmt="{asctime} - {name}")
> ?If %-formatting is to be deprecated, the transition strategy here
> ?is trivial. However, no one has yet written translators, and it is
> ?not clear what heuristics should be used, e.g. should the method
> ?just try %-formatting first and then {}-formatting if it fails?

I'm reserving judgement on this one until it becomes clear how
feasible it is.  Without having thought about it too hard, this sounds
potentially tricky and bug-prone.

Mark

From ubershmekel at gmail.com  Sat Oct  3 18:38:31 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Sat, 3 Oct 2009 19:38:31 +0300
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC77A17.7050101@voidspace.org.uk>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>
	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>
	<d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>
	<4AC76B10.1000209@voidspace.org.uk>
	<d11dcfba0910030849v111160abge291d24ad980507a@mail.gmail.com>
	<4AC77A17.7050101@voidspace.org.uk>
Message-ID: <9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>

On Sat, Oct 3, 2009 at 7:21 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> [snip...]
> Why not just catch SystemExit? If you want a custom exception the overriding .exit() should be sufficient.
> I'd be much more interested in Guido's suggestion of auto-generated custom help messages for sub-commands.

Check it out:

def ParseAndRun():
    crazy_external_function_that_might_exit()

    # Argparse blah blah
    parser.parse_args()

if __name__ == "__main__":
    try:
        ParseAndRun()
    except SystemExit:
        # was it crazy_external_function_that_might_exit or an argparse error?


I know this might come through as bike shedding but it's just
customary python that every module have it's own exception types as to
not mix them up with others.

--yuv

From python at mrabarnett.plus.com  Sat Oct  3 19:02:52 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 03 Oct 2009 18:02:52 +0100
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
Message-ID: <4AC783BC.70404@mrabarnett.plus.com>

Steven Bethard wrote:
> I thought it might be useful for those who don't have time to read a
> million posts to have a summary of what's happened in the formatting
> discussion.
> 
> The basic problem is that many APIs in the standard library and
> elsewhere support only %-formatting and not {}-formatting, e.g.
> logging.Formatter accepts::
>   logging.Formatter(fmt="%(asctime)s - %(name)s")
> but not::
>   logging.Formatter(fmt="{asctime} - {name}")
> 
> There seems to be mostly agreement that these APIs should at least
> support both formatting styles, and a sizable group (including Guido)
> believe that %-formatting should eventually be phased out (e.g. by
> Python 4). There are a number of competing proposals on how to allow
> such APIs to start accepting {}-format strings:
> 
> * Add a parameter which declares the type of format string::
>     logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
>   The API code would then switch between %-format and {}-format
>   based on the value of that parameter. If %-formatting is to be
>   deprecated, this could be done by first deprecating
>   format=PERCENTS and requiring format=BRACES, and then changing the
>   default to format=BRACES.
> 
> * Create string subclasses which convert % use to .format calls::
>     __ = brace_fmt
>     logging.Formatter(fmt=__("{asctime} - {name}"))
>   The API code wouldn't have to change at all at first, as applying
>   % to brace_fmt objects would call .format() instead. If
>   %-formatting is to be deprecated, this could be done by first
>   deprecating plain strings and requiring brace_fmt strings, and
>   then allowing plain strings again but assuming they are {}-format
>   strings.
> 
> * Teach the API to accept callables as well as strings::
>     logging.Formatter(fmt="{asctime} - {name}".format)
>   The API code would just call the object with .format() style
>   arguments if a callable was given instead of a string. If
>   %-formatting is to be deprecated, this could be done by first
>   deprecating plain strings and requiring callables, and then
>   allowing plain strings again but assuming they are {}-format
>   strings
> 
I'm not keen on deprecating strings in favour of callables and then
callables in favour of strings.

> * Create translators between %-format and {}-format::
>     assert to_braces("%(asctime)s") == "{asctime}"
>     assert to_percents("{asctime}") == "%(asctime)s"
>   these could then either be used outside of the API::
>     logging.Formatter(fmt=to_percents("{asctime} - {name}"))
>   or they could be used within the API combined with some sort of
>   heuristic for guessing whether a {}-format string or a %-format
>   string was passed in::
>     logging.Formatter(fmt="{asctime} - {name}")
>   If %-formatting is to be deprecated, the transition strategy here
>   is trivial. However, no one has yet written translators, and it is
>   not clear what heuristics should be used, e.g. should the method
>   just try %-formatting first and then {}-formatting if it fails?
> 
Another possibility:

A StringFormat class with subclasses PercentStringFormat, 
BraceStringFormat, and perhaps DollarStringFormat.

Or:

A StringFormat class with methods parse_percent_format,
parse_brace_format, and parse_dollar_format. There could also be a
format-guesser method.

> I don't think there is consensus yet on which of these proposals
> should be the "blessed" one.
> 


From python at mrabarnett.plus.com  Sat Oct  3 19:14:03 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 03 Oct 2009 18:14:03 +0100
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <5c6f2a5d0910030934u7399d871yfb71f3319ce6183e@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<5c6f2a5d0910030934u7399d871yfb71f3319ce6183e@mail.gmail.com>
Message-ID: <4AC7865B.9020607@mrabarnett.plus.com>

Mark Dickinson wrote:
> On Sat, Oct 3, 2009 at 4:41 PM, Steven Bethard <steven.bethard at gmail.com> wrote:
>> I thought it might be useful for those who don't have time to read a
>> million posts to have a summary of what's happened in the formatting
>> discussion.
> 
> Definitely useful.  Thanks for the summary!
> 
> [...]
> 
>> * Add a parameter which declares the type of format string::
>>    logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
>>  The API code would then switch between %-format and {}-format
>>  based on the value of that parameter. If %-formatting is to be
>>  deprecated, this could be done by first deprecating
>>  format=PERCENTS and requiring format=BRACES, and then changing the
>>  default to format=BRACES.
> 
> +1.
[snip]
'fmt' (which is an abbreviation for 'format') and 'format'? Maybe 'fmt'
and 'style' instead?

From p.f.moore at gmail.com  Fri Oct  2 15:20:28 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 2 Oct 2009 14:20:28 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC53417.1040405@trueblade.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
	<4AC5250C.8030403@gmail.com> <loom.20091002T001232-270@post.gmane.org>
	<4AC53417.1040405@trueblade.com>
Message-ID: <79990c6b0910020620l5ff4e774wce36f6d48d1043ef@mail.gmail.com>

2009/10/1 Eric Smith <eric at trueblade.com>:
> It's tangential, but in the str.format case you don't want to check for just
> '{asctime}', because you might want '{asctime:%Y-%m-%d}', for example.
>
> But there are ways to delay computing the time until you're sure it's
> actually being used in the format string, without parsing the format string.
> Now that I think of it, the same technique could be used with %-formatting:

Still tangential, but it seems to me that this discussion has exposed
a couple of areas where the logging interface is less than ideal:

- The introspection of the format string to delay computing certain
items (Eric's suggestion may be an improvement here).
- The "call str() on any non-string object to get a format string" API
(which precludes string subclasses).

I suspect other APIs will exist with similar issues once the whole
question of supporting multiple format syntaxes gets wider
publicity...

Paul.

From steve at pearwood.info  Sat Oct  3 19:29:52 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 4 Oct 2009 03:29:52 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<4AC77A17.7050101@voidspace.org.uk>
	<9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
Message-ID: <200910040429.53011.steve@pearwood.info>

On Sun, 4 Oct 2009 03:38:31 am Yuvgoog Greenle wrote:
> Check it out:
>
> def ParseAndRun():
> ? ? crazy_external_function_that_might_exit()
>
> ? ? # Argparse blah blah
> ? ? parser.parse_args()
>
> if __name__ == "__main__":
> ? ? try:
> ? ? ? ? ParseAndRun()
> ? ? except SystemExit:
> ? ? ? ? # was it crazy_external_function_that_might_exit or an
> argparse error?

Does it matter? What difference does it make?


> I know this might come through as bike shedding but it's just
> customary python that every module have it's own exception types as
> to not mix them up with others.

You are mistaken.

>>> import os
>>> os.path.split(45)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.6/posixpath.py", line 82, in split
    i = p.rfind('/') + 1
AttributeError: 'int' object has no attribute 'rfind'

Plain, ordinary AttributeError, not OSModuleSubclassOfAttributeError.

I could show a thousand other examples. It simply isn't true that all, 
or even most, modules have their own exception types.

There's no reason to treat SystemExit as special in this regard. 
optparse, for example, calls sys.exit(), which operates by raising 
SystemExit. Ordinary SystemExit, not a special module-specific 
subclass.



-- 
Steven D'Aprano

From fuzzyman at voidspace.org.uk  Sat Oct  3 19:35:29 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 03 Oct 2009 18:35:29 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <79990c6b0910020620l5ff4e774wce36f6d48d1043ef@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T124105-392@post.gmane.org>	<loom.20091001T140503-467@post.gmane.org>	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>	<loom.20091001T151606-886@post.gmane.org>	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>	<69124.66598.qm@web25807.mail.ukl.yahoo.com>	<4AC5250C.8030403@gmail.com>
	<loom.20091002T001232-270@post.gmane.org>	<4AC53417.1040405@trueblade.com>
	<79990c6b0910020620l5ff4e774wce36f6d48d1043ef@mail.gmail.com>
Message-ID: <4AC78B61.2080501@voidspace.org.uk>

Paul Moore wrote:
> 2009/10/1 Eric Smith <eric at trueblade.com>:
>   
>> It's tangential, but in the str.format case you don't want to check for just
>> '{asctime}', because you might want '{asctime:%Y-%m-%d}', for example.
>>
>> But there are ways to delay computing the time until you're sure it's
>> actually being used in the format string, without parsing the format string.
>> Now that I think of it, the same technique could be used with %-formatting:
>>     
>
> Still tangential, but it seems to me that this discussion has exposed
> a couple of areas where the logging interface is less than ideal:
>
> - The introspection of the format string to delay computing certain
> items (Eric's suggestion may be an improvement here).
> - The "call str() on any non-string object to get a format string" API
> (which precludes string subclasses).
>   

Calling str on non-string objects to get a format string does not 
(prima-facie) preclude string subclasses:

 >>> class X(str): pass
...
 >>> class Y(object):
...  def __str__(self):
...   return X('foo')
...
 >>> a = Y()
 >>> type(str(a))
<class '__main__.X'>

Michael

> I suspect other APIs will exist with similar issues once the whole
> question of supporting multiple format syntaxes gets wider
> publicity...
>
> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From vinay_sajip at yahoo.co.uk  Sat Oct  3 19:38:36 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 3 Oct 2009 17:38:36 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T124105-392@post.gmane.org>
	<loom.20091001T140503-467@post.gmane.org>
	<79990c6b0910010611u6c772f11w7029c3154fa7be49@mail.gmail.com>
	<loom.20091001T151606-886@post.gmane.org>
	<bbaeab100910011103i4324e7bob86849ff275aa3ee@mail.gmail.com>
	<69124.66598.qm@web25807.mail.ukl.yahoo.com>
	<4AC5250C.8030403@gmail.com>
	<loom.20091002T001232-270@post.gmane.org>
	<4AC53417.1040405@trueblade.com>
	<79990c6b0910020620l5ff4e774wce36f6d48d1043ef@mail.gmail.com>
Message-ID: <loom.20091003T193333-551@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> Still tangential, but it seems to me that this discussion has exposed
> a couple of areas where the logging interface is less than ideal:
> 
> - The introspection of the format string to delay computing certain
> items (Eric's suggestion may be an improvement here).

Yes, but that's an implementation detail and not part of the logging interface.
It can be changed without any particular additional impact on user code - when
I say "additional" I mean apart from the need to change the format strings to
{} format, which they would have to do anyway at some point.

> - The "call str() on any non-string object to get a format string" API
> (which precludes string subclasses).

It doesn't preclude string subclasses: it just calls str() on an arbitrary
message object to get the string representation for that object. The return
value is used to interpolate into the formatted output, and that's all. So
I don't understand what's being precluded and how - please elaborate.

Thanks & regards,

Vinay Sajip


From solipsis at pitrou.net  Sat Oct  3 19:50:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Oct 2009 17:50:20 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?summary_of_transitioning_from_=25_to_=7B?=
	=?utf-8?q?=7D_formatting?=
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<4AC783BC.70404@mrabarnett.plus.com>
Message-ID: <loom.20091003T194219-114@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> Another possibility:
> 
> A StringFormat class with subclasses PercentStringFormat, 
> BraceStringFormat, and perhaps DollarStringFormat.
> 
> Or:
> 
> A StringFormat class with methods parse_percent_format,
> parse_brace_format, and parse_dollar_format. There could also be a
> format-guesser method.

I'm sorry to say this, but I think these suggestions are getting foolish. We
core developers might have an interest in transitioning users from one
formatting style to another, but the users mostly don't care and don't want to
bother. Imposing on users the explicit use of such wrapper classes, moreover
with such awfully long-winded names, is not helpful to them at all, and it will
earn Python the reputation of a language which imposes silly constructs in the
name of purity.

If we can't find a way to make things almost transparent, we should IMO abandon
the whole idea of a transition.

Regards

Antoine.



From steve at pearwood.info  Sat Oct  3 20:01:56 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 4 Oct 2009 04:01:56 +1000
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
Message-ID: <200910040501.57077.steve@pearwood.info>

On Sun, 4 Oct 2009 01:41:36 am Steven Bethard wrote:
> I thought it might be useful for those who don't have time to read a
> million posts to have a summary of what's happened in the formatting
> discussion.
>
> The basic problem is that many APIs in the standard library and
> elsewhere support only %-formatting and not {}-formatting, e.g.
> logging.Formatter accepts::
>   logging.Formatter(fmt="%(asctime)s - %(name)s")
> but not::
>   logging.Formatter(fmt="{asctime} - {name}")

Why is this a problem? Is it because the APIs require extra 
functionality that only {} formatting can provide? Possibly, but I 
doubt it -- I expect that the reason is:

(1) Some people would like to deprecate % formatting, and they can't 
while the std lib uses % internally.

(2) Some APIs in the std lib are tightly coupled to their internal 
implementation, and so you can't change their implementation without 
changing the API as well.

Remove either of these issues, and the problem becomes a non-problem, 
and no action is needed. Personally, I'd like to see no further talk 
about deprecating % until at least Python 3.2, that is, the *earliest* 
we would need to solve this issue would be 3.3. As the Zen 
says, "Although never is often better than *right* now."


> There seems to be mostly agreement that these APIs should at least
> support both formatting styles, and a sizable group (including Guido)
> believe that %-formatting should eventually be phased out (e.g. by
> Python 4). 

-1 on that. Time will tell if I change my mind in a couple of years, but 
I suspect not -- for simple formatting, I far prefer %. Judging by the 
reaction on comp.lang.python when this has been discussed in the past, 
I think a large (or at least loud) proportion of Python programmers 
agree with me.


> There are a number of competing proposals on how to allow 
> such APIs to start accepting {}-format strings:
>
> * Add a parameter which declares the type of format string::
>     logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
>   The API code would then switch between %-format and {}-format
>   based on the value of that parameter. If %-formatting is to be
>   deprecated, this could be done by first deprecating
>   format=PERCENTS and requiring format=BRACES, and then changing the
>   default to format=BRACES.

+0.5


> * Create string subclasses which convert % use to .format calls::
>     __ = brace_fmt
>     logging.Formatter(fmt=__("{asctime} - {name}"))

There are a few problems with this approach:

- Nobody has yet demonstrated that this brace_fmt class is even 
possible. It should be easy to handle a restricted set of simple 
templates (e.g. of the form "%(name)s" only), but what of the full 
range of behaviour supported by % formatting?

- Even if it is doable, it is a wrapper class, which means the module 
will suffer a performance hit on every call. At this time, we have no 
idea what the magnitude of that hit will be, but .format() is already 
slower than % so it will likely be significant.

- It strikes me as hideously ugly. I for one would delay using it as 
long as possible, no matter how many DepreciationWarnings I got. I'd 
drag my feet and avoid changing and complain loudly and then become 
sullen and resentful when I couldn't avoid making the change. I'd much 
rather go straight from %-based templates to {} in a single step than 
have this Frankenstein monster intermediate.


[...]
> * Teach the API to accept callables as well as strings::
>     logging.Formatter(fmt="{asctime} - {name}".format)
>   The API code would just call the object with .format() style
>   arguments if a callable was given instead of a string.

+0.5

> * Create translators between %-format and {}-format::
>     assert to_braces("%(asctime)s") == "{asctime}"
>     assert to_percents("{asctime}") == "%(asctime)s"

+1, assuming such translators are even possible. Being optimistic, such 
translators would have one additional benefit: they would enable 
modules to completely decouple the API they offer from their internal 
implementation, without paying a runtime cost on every call, just a 
single once-off translation at initialisation time.

In theory, this could mean that modules could, if they choose, continue 
to offer an API based on % long after str.__mod__ is removed from the 
language.


-- 
Steven D'Aprano

From steve at pearwood.info  Sat Oct  3 20:22:28 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 4 Oct 2009 04:22:28 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0910031046y6f25fc2fg9b80d96ca0b3e61b@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<200910040429.53011.steve@pearwood.info>
	<9d153b7c0910031046y6f25fc2fg9b80d96ca0b3e61b@mail.gmail.com>
Message-ID: <200910040522.28374.steve@pearwood.info>

On Sun, 4 Oct 2009 04:46:19 am Yuvgoog Greenle wrote:

> I just think that if a parser error is causing the SystemExit, I
> would rather catch a parser error than catching a SystemExit for the
> sake of readability. It saves me the comments:
>
> # Catching SystemExit because parse_args() throws SystemExit on
> parser errors.

But why are you catching the error? As a general rule, you *want* your 
command line app to exit if it can't understand the arguments you pass 
to it. What else can it do? Guess what you wanted?

Assuming you have a reason for catching the exception, I don't see that 
there's any difference in readability between these:

parser = argparse.ArgumentParser()
setup_args(parser)
try:
    ns = parser.parse_args()
except argparse.ParserError:
    process_error()
else:
    main(ns)

and:

parser = argparse.ArgumentParser()
setup_args(parser)
try:
    ns = parser.parse_args()
except SystemExit:
    process_error()
else:
    main(ns)


You don't need a comment warning that you are catching SystemExit 
because parse_args raises SystemExit, any more than you need a comment 
saying that you are catching ValueError because some function raises 
ValueError. The fact that you are catching an exception implies that 
the function might raise that exception. A comment like:

"Catching SystemExit because parse_args() throws SystemExit on parser 
errors."

is up them with comments like this:

x += 1  # Add 1 to x.



-- 
Steven D'Aprano

From brett at python.org  Sat Oct  3 20:27:20 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 3 Oct 2009 11:27:20 -0700
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <200910040501.57077.steve@pearwood.info>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com> 
	<200910040501.57077.steve@pearwood.info>
Message-ID: <bbaeab100910031127x5fa3655h55f41f41ae8c62b0@mail.gmail.com>

On Sat, Oct 3, 2009 at 11:01, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, 4 Oct 2009 01:41:36 am Steven Bethard wrote:
>> I thought it might be useful for those who don't have time to read a
>> million posts to have a summary of what's happened in the formatting
>> discussion.
>>
>> The basic problem is that many APIs in the standard library and
>> elsewhere support only %-formatting and not {}-formatting, e.g.
>> logging.Formatter accepts::
>> ? logging.Formatter(fmt="%(asctime)s - %(name)s")
>> but not::
>> ? logging.Formatter(fmt="{asctime} - {name}")
>
> Why is this a problem? Is it because the APIs require extra
> functionality that only {} formatting can provide? Possibly, but I
> doubt it -- I expect that the reason is:
>
> (1) Some people would like to deprecate % formatting, and they can't
> while the std lib uses % internally.
>
> (2) Some APIs in the std lib are tightly coupled to their internal
> implementation, and so you can't change their implementation without
> changing the API as well.
>
> Remove either of these issues, and the problem becomes a non-problem,
> and no action is needed. Personally, I'd like to see no further talk
> about deprecating % until at least Python 3.2, that is, the *earliest*
> we would need to solve this issue would be 3.3. As the Zen
> says, "Although never is often better than *right* now."

No one is saying we should deprecate % any time soon on strings
themselves or anywhere. This discussion is purely in regards to
argparse and logging to transition *their* APIs over to {} formatting
which would most likely involve some deprecation for *using* %
formatting in those APIs. But % formatting on strings themselves is
not directly being discussed here.

-Brett

From brett at python.org  Sat Oct  3 20:05:50 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 3 Oct 2009 11:05:50 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> 
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com> 
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
Message-ID: <bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>

On Fri, Oct 2, 2009 at 20:03, Raymond Hettinger <python at rcn.com> wrote:
>
> [Terry Reedy]
>>
>> I would agree, for instance, that an auto-translation tool is needed.
>
> We should get one written. ?ISTM, every %-formatting
> string is directly translatable to an equivalent {}-formatting string.

Why don't we start something in the sandbox and see how far we can
get. If no one beats me to it I will add the directory some time today
and we can start hashing out the solution there.

-Brett

From ubershmekel at gmail.com  Sat Oct  3 19:46:19 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Sat, 3 Oct 2009 20:46:19 +0300
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <200910040429.53011.steve@pearwood.info>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<4AC77A17.7050101@voidspace.org.uk>
	<9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
	<200910040429.53011.steve@pearwood.info>
Message-ID: <9d153b7c0910031046y6f25fc2fg9b80d96ca0b3e61b@mail.gmail.com>

On Sat, Oct 3, 2009 at 8:29 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> I could show a thousand other examples. It simply isn't true that all,
> or even most, modules have their own exception types.

I might be wrong on this. Your point is extra true for modules in the
standard library (which is what we're talking about for argparse).

I just think that if a parser error is causing the SystemExit, I would
rather catch a parser error than catching a SystemExit for the sake of
readability. It saves me the comments:

# Catching SystemExit because parse_args() throws SystemExit on parser errors.

# Subclassing ArgumentParser and overriding exit because I don't want
to exit() upon parser errors.

So I'm sorry if what I said was irrelevant. I've never written or
taken part of writing a std-lib module.

--yuv

From nd at perlig.de  Sat Oct  3 20:35:04 2009
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Sat, 3 Oct 2009 20:35:04 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <200910040522.28374.steve@pearwood.info>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<9d153b7c0910031046y6f25fc2fg9b80d96ca0b3e61b@mail.gmail.com>
	<200910040522.28374.steve@pearwood.info>
Message-ID: <200910032035.05117@news.perlig.de>

* Steven D'Aprano wrote:

> You don't need a comment warning that you are catching SystemExit
> because parse_args raises SystemExit, any more than you need a comment
> saying that you are catching ValueError because some function raises
> ValueError. The fact that you are catching an exception implies that
> the function might raise that exception. A comment like:
>
> "Catching SystemExit because parse_args() throws SystemExit on parser
> errors."
>
> is up them with comments like this:
>
> x += 1  # Add 1 to x.

It's semantically different. You usually don't catch SystemExit directly, 
because you want your programs to be stopped. Additionally, a library 
exiting your program is badly designed, as it's unexpected. Thatswhy such a 
comment is useful.

Here's what I'd do: I'd subclass SystemExit in this case and raise the 
subclass from argparse. That way all parties here should be satisifed. (I 
do the same all the time in my signal handlers - that's another reason I'd 
rather not catch SystemExit directly as well :-)

nd
-- 
"Umfassendes Werk (auch fuer Umsteiger vom Apache 1.3)"
                                          -- aus einer Rezension

<http://pub.perlig.de/books.html#apache2>

From python at mrabarnett.plus.com  Sat Oct  3 21:09:38 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 03 Oct 2009 20:09:38 +0100
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <loom.20091003T194219-114@post.gmane.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>	<4AC783BC.70404@mrabarnett.plus.com>
	<loom.20091003T194219-114@post.gmane.org>
Message-ID: <4AC7A172.2050808@mrabarnett.plus.com>

Antoine Pitrou wrote:
> MRAB <python <at> mrabarnett.plus.com> writes:
>> Another possibility:
>> 
>> A StringFormat class with subclasses PercentStringFormat, 
>> BraceStringFormat, and perhaps DollarStringFormat.
>> 
>> Or:
>> 
>> A StringFormat class with methods parse_percent_format, 
>> parse_brace_format, and parse_dollar_format. There could also be a
>>  format-guesser method.
> 
> I'm sorry to say this, but I think these suggestions are getting 
> foolish. We core developers might have an interest in transitioning 
> users from one formatting style to another, but the users mostly 
> don't care and don't want to bother. Imposing on users the explicit 
> use of such wrapper classes, moreover with such awfully long-winded 
> names, is not helpful to them at all, and it will earn Python the 
> reputation of a language which imposes silly constructs in the name 
> of purity.
> 
Fair enough.

The purpose of the post was really just so that we cover as many
possibilities as we can so that if at some time in the future someone
asks why we didn't consider such-and-such then we won't be saying "Oh,
never thought of that!". :-)

> If we can't find a way to make things almost transparent, we should 
> IMO abandon the whole idea of a transition.
> 


From robert.kern at gmail.com  Sat Oct  3 23:21:46 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 03 Oct 2009 16:21:46 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <200910040522.28374.steve@pearwood.info>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<200910040429.53011.steve@pearwood.info>	<9d153b7c0910031046y6f25fc2fg9b80d96ca0b3e61b@mail.gmail.com>
	<200910040522.28374.steve@pearwood.info>
Message-ID: <ha8f9a$u8i$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Sun, 4 Oct 2009 04:46:19 am Yuvgoog Greenle wrote:
> 
>> I just think that if a parser error is causing the SystemExit, I
>> would rather catch a parser error than catching a SystemExit for the
>> sake of readability. It saves me the comments:
>>
>> # Catching SystemExit because parse_args() throws SystemExit on
>> parser errors.
> 
> But why are you catching the error? As a general rule, you *want* your 
> command line app to exit if it can't understand the arguments you pass 
> to it. What else can it do? Guess what you wanted?

There are uses of argparse outside of command line apps. For example, I use it 
to parse --options for IPython %magic commands. Of course, I just subclass 
ArgumentParser and replace the .error() method. I require a particular IPython 
exception type in order for the error to be recognized correctly in the %magic 
subsystem.

The other use case, as mentioned earlier, was for debugging your parser on the 
interactive prompt. A custom subclass may also be able to hold more 
machine-readable information about the error than the formatted error message, 
but I don't have any particular use cases about what may be the most useful.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From vinay_sajip at yahoo.co.uk  Sun Oct  4 00:27:27 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 3 Oct 2009 22:27:27 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
Message-ID: <loom.20091004T001826-666@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> Why don't we start something in the sandbox and see how far we can
> get. If no one beats me to it I will add the directory some time today
> and we can start hashing out the solution there.
> 

I've done a first cut of a converter from %-format to {}-format strings. I'm not
sure where you want to put it in the sandbox, I've created a gist on GitHub:

http://gist.github.com/200936

Not thoroughly tested, but runs in interactive mode so you can try things out.
All feedback appreciated!

Regards,


Vinay


From vinay_sajip at yahoo.co.uk  Sun Oct  4 00:29:34 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 3 Oct 2009 22:29:34 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><loom.20091001T010253-42@post.gmane.org><loom.20091001T024614-990@post.gmane.org>	<ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com><loom.20091002T113352-479@post.gmane.org>	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<4AC6E019.4000701@gmail.com>
Message-ID: <loom.20091004T002759-476@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> I'm starting to think that a converter between the two format
> mini-languages may be the way to go though.
> 
> fmt_braces is meant to provide a superset of the capabilites of
> fmt_percent, so a forward converter shouldn't be too hard. A reverse
> converter may have to punt with ValueError when it finds things that
> cannot be expressed in the fmt_percent mini language though.

I've done a first cut of a forward (% -> {}) converter:

http://gist.github.com/200936

but I'm not sure there's a case for a converter in the reverse direction, if
we're encouraging movement in one particular direction.

Regards,

Vinay Sajip




From vinay_sajip at yahoo.co.uk  Sun Oct  4 00:33:33 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 3 Oct 2009 22:33:33 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
Message-ID: <loom.20091004T003159-317@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:

> We should get one written.  ISTM, every %-formatting
> string is directly translatable to an equivalent {}-formatting string.
> 

I've made a start, but I'm not sure how best to handle the '#' and ' '
conversion flags.

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Sun Oct  4 00:43:18 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 3 Oct 2009 22:43:18 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?summary_of_transitioning_from_=25_to_=7B?=
	=?utf-8?q?=7D_formatting?=
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<200910040501.57077.steve@pearwood.info>
	<bbaeab100910031127x5fa3655h55f41f41ae8c62b0@mail.gmail.com>
Message-ID: <loom.20091004T003523-410@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> No one is saying we should deprecate % any time soon on strings
> themselves or anywhere. This discussion is purely in regards to
> argparse and logging to transition *their* APIs over to {} formatting
> which would most likely involve some deprecation for *using* %
> formatting in those APIs. But % formatting on strings themselves is
> not directly being discussed here.

While I have no problem with supporting {}-formatting in logging if we find a
good way of doing it, one thing that bothers me about transitioning the logging
API to deprecate or otherwise de-emphasise %-formatting is the question of
performance. Now, str.format is more flexible than str.__mod__ and so some
performance loss may be tolerable in many scenarios. However, in the feedback I
regularly get about logging, people are concerned about performance. No-one ever
comes up with any hard numbers, but logging is often bashed as slow (see e.g.
Andrii Mishkovskyi's LoggingPackage page on the Python wiki). I don't especially
want to add fuel to the fire, as any performance degradation caused by
supporting {}-formatting will likely just result in more finger-pointing at
logging in general.

Regards,

Vinay Sajip



From brett at python.org  Sun Oct  4 00:32:48 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 3 Oct 2009 15:32:48 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091004T001826-666@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> 
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com> 
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1> 
	<bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com> 
	<loom.20091004T001826-666@post.gmane.org>
Message-ID: <bbaeab100910031532h5ea00081u490e2bd9d290d0f@mail.gmail.com>

On Sat, Oct 3, 2009 at 15:27, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> Why don't we start something in the sandbox and see how far we can
>> get. If no one beats me to it I will add the directory some time today
>> and we can start hashing out the solution there.
>>
>
> I've done a first cut of a converter from %-format to {}-format strings. I'm not
> sure where you want to put it in the sandbox, I've created a gist on GitHub:
>
> http://gist.github.com/200936
>
> Not thoroughly tested, but runs in interactive mode so you can try things out.
> All feedback appreciated!

I mentioned it in the README. We now have two separate attempts at
converting from % to {}. If people are serious about this we should
start to work on a single implementation.

-Brett

From benjamin at python.org  Sun Oct  4 00:38:55 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 3 Oct 2009 17:38:55 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091004T003159-317@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091004T003159-317@post.gmane.org>
Message-ID: <1afaf6160910031538g1d196ca4w61f3328a9fa8eb90@mail.gmail.com>

2009/10/3 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> Raymond Hettinger <python <at> rcn.com> writes:
>
>> We should get one written. ?ISTM, every %-formatting
>> string is directly translatable to an equivalent {}-formatting string.
>>
>
> I've made a start, but I'm not sure how best to handle the '#' and ' '
> conversion flags.

Mine handles it differently for each type specifier.



-- 
Regards,
Benjamin

From tjreedy at udel.edu  Sun Oct  4 01:31:01 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 03 Oct 2009 19:31:01 -0400
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
Message-ID: <ha8mri$gge$1@ger.gmane.org>

Steven Bethard wrote:
> I thought it might be useful for those who don't have time to read a
> million posts to have a summary of what's happened in the formatting
> discussion.

definitely

> 
> The basic problem is that many APIs in the standard library and
> elsewhere support only %-formatting and not {}-formatting, e.g.
> logging.Formatter accepts::
>   logging.Formatter(fmt="%(asctime)s - %(name)s")
> but not::
>   logging.Formatter(fmt="{asctime} - {name}")
> 
> There seems to be mostly agreement that these APIs should at least
> support both formatting styles, and a sizable group (including Guido)
> believe that %-formatting should eventually be phased out (e.g. by
> Python 4). There are a number of competing proposals on how to allow
> such APIs to start accepting {}-format strings:
> 
> * Add a parameter which declares the type of format string::
>     logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
>   The API code would then switch between %-format and {}-format
>   based on the value of that parameter. If %-formatting is to be
>   deprecated, this could be done by first deprecating
>   format=PERCENTS and requiring format=BRACES, and then changing the
>   default to format=BRACES.

...

> * Create translators between %-format and {}-format::
>     assert to_braces("%(asctime)s") == "{asctime}"
>     assert to_percents("{asctime}") == "%(asctime)s"
>   these could then either be used outside of the API::
>     logging.Formatter(fmt=to_percents("{asctime} - {name}"))
>   or they could be used within the API combined with some sort of
>   heuristic for guessing whether a {}-format string or a %-format
>   string was passed in::
>     logging.Formatter(fmt="{asctime} - {name}")

How about combining these two. Add an optional form or style=xxx 
parameter -- which could also allow DOLLARS for $-formats -- for 
resolving ambiguities. If not present, make a reasonable guess. IE, if 
string has no '%' and multiple {} pairs, or no {} pairs and multiple %s, 
the guess is at least .999 sure. which is to not, not hardly a guess. 
The extra arg can be supplied if and when needed.

The machinery for this should be not be logging specific, so it can be 
used through throughout the library, and exposed so that others can use it.

Terry Jan Reedy


From python at mrabarnett.plus.com  Sun Oct  4 01:38:50 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 04 Oct 2009 00:38:50 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091004T001826-666@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
	<loom.20091004T001826-666@post.gmane.org>
Message-ID: <4AC7E08A.9050601@mrabarnett.plus.com>

Vinay Sajip wrote:
> Brett Cannon <brett <at> python.org> writes:
> 
>> Why don't we start something in the sandbox and see how far we can
>> get. If no one beats me to it I will add the directory some time today
>> and we can start hashing out the solution there.
>>
> 
> I've done a first cut of a converter from %-format to {}-format strings. I'm not
> sure where you want to put it in the sandbox, I've created a gist on GitHub:
> 
> http://gist.github.com/200936
> 
> Not thoroughly tested, but runs in interactive mode so you can try things out.
> All feedback appreciated!
> 
Interesting that you're using the %-format to translate %-formats to
{}-formats! :-)

From vinay_sajip at yahoo.co.uk  Sun Oct  4 01:49:06 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 3 Oct 2009 23:49:06 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
	<loom.20091004T001826-666@post.gmane.org>
	<4AC7E08A.9050601@mrabarnett.plus.com>
Message-ID: <loom.20091004T014859-982@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:

> Interesting that you're using the %-format to translate %-formats to
> {}-formats! 

Yes, ironic, isn't it? ;-)

Regards,

Vinay Sajip



From python at mrabarnett.plus.com  Sun Oct  4 02:11:39 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 04 Oct 2009 01:11:39 +0100
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <ha8mri$gge$1@ger.gmane.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<ha8mri$gge$1@ger.gmane.org>
Message-ID: <4AC7E83B.2010102@mrabarnett.plus.com>

Terry Reedy wrote:
> Steven Bethard wrote:
>> I thought it might be useful for those who don't have time to read a
>> million posts to have a summary of what's happened in the formatting
>> discussion.
> 
> definitely
> 
>>
>> The basic problem is that many APIs in the standard library and
>> elsewhere support only %-formatting and not {}-formatting, e.g.
>> logging.Formatter accepts::
>>   logging.Formatter(fmt="%(asctime)s - %(name)s")
>> but not::
>>   logging.Formatter(fmt="{asctime} - {name}")
>>
>> There seems to be mostly agreement that these APIs should at least
>> support both formatting styles, and a sizable group (including Guido)
>> believe that %-formatting should eventually be phased out (e.g. by
>> Python 4). There are a number of competing proposals on how to allow
>> such APIs to start accepting {}-format strings:
>>
>> * Add a parameter which declares the type of format string::
>>     logging.Formatter(fmt="{asctime} - {name}", format=BRACES)
>>   The API code would then switch between %-format and {}-format
>>   based on the value of that parameter. If %-formatting is to be
>>   deprecated, this could be done by first deprecating
>>   format=PERCENTS and requiring format=BRACES, and then changing the
>>   default to format=BRACES.
> 
> ...
> 
>> * Create translators between %-format and {}-format::
>>     assert to_braces("%(asctime)s") == "{asctime}"
>>     assert to_percents("{asctime}") == "%(asctime)s"
>>   these could then either be used outside of the API::
>>     logging.Formatter(fmt=to_percents("{asctime} - {name}"))
>>   or they could be used within the API combined with some sort of
>>   heuristic for guessing whether a {}-format string or a %-format
>>   string was passed in::
>>     logging.Formatter(fmt="{asctime} - {name}")
> 
> How about combining these two. Add an optional form or style=xxx 
> parameter -- which could also allow DOLLARS for $-formats -- for 
> resolving ambiguities. If not present, make a reasonable guess. IE, if 
> string has no '%' and multiple {} pairs, or no {} pairs and multiple %s, 
> the guess is at least .999 sure. which is to not, not hardly a guess. 
> The extra arg can be supplied if and when needed.
> 
Maybe if the style is a callable then that's the formatting function?
(Or is that asking for trouble? :-))

In that case the style could default to a function which guesses which
style is being used and then the appropriate function.

> The machinery for this should be not be logging specific, so it can be 
> used through throughout the library, and exposed so that others can use it.
> 


From steve at pearwood.info  Sun Oct  4 03:26:51 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 4 Oct 2009 11:26:51 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <200910032035.05117@news.perlig.de>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<200910040522.28374.steve@pearwood.info>
	<200910032035.05117@news.perlig.de>
Message-ID: <200910041226.51573.steve@pearwood.info>

On Sun, 4 Oct 2009 05:35:04 am Andr? Malo wrote:
> * Steven D'Aprano wrote:
> > You don't need a comment warning that you are catching SystemExit
> > because parse_args raises SystemExit, any more than you need a
> > comment saying that you are catching ValueError because some
> > function raises ValueError. The fact that you are catching an
> > exception implies that the function might raise that exception. A
> > comment like:
> >
> > "Catching SystemExit because parse_args() throws SystemExit on
> > parser errors."
> >
> > is up them with comments like this:
> >
> > x += 1  # Add 1 to x.
>
> It's semantically different. You usually don't catch SystemExit
> directly, because you want your programs to be stopped. 

Exactly -- so why catch it at all? But even if there is a good reason to 
catch it, there's still no reason to subclass SystemExit unless 
argparse wants to distinguish different types of fatal error, and allow 
code to catch some but not all. But frankly, if I'm having a hard time 
thinking of a reason to catch SystemExit, I'm having an even harder 
time thinking why you'd want to (say) catch SystemExitTooManyArguments 
but not SystemExitMissingArgument.


> Additionally, 
> a library exiting your program is badly designed, as it's unexpected.

It's not unexpected for an argument parser. Do you know any applications 
that run when given invalid arguments? As a general rule, what can the 
application do? Guess what you wanted?


> Thatswhy such a comment is useful.

The comment doesn't tell you anything that wasn't obvious from the code. 
It is pointless.



> Here's what I'd do: I'd subclass SystemExit in this case and raise
> the subclass from argparse. 

In the following code snippet:

try:
    ns = argparse.parse_args()
except SystemExit:
    ...

is there any confusion between SystemExit raised by parse_args and 
SystemExit raised by other components? *What* other components? If 
SystemExit was raised in that try block, where could it have come from 
other than parse_args?

Do you write comments like these?

try:
    value = mydict[key]
except KeyError:
    # Catch KeyError raised by dict lookup
    ...


try:
    n = mylist.index(x)
except ValueError:
    # Catch ValueError raised by mylist.index
    ...



Useless comments are worse than no comments, because useless comments 
waste the readers' time and they risk becoming out of sync with the 
code and turning into misleading comments.


> That way all parties here should be satisifed.

No. It wastes the time of the argparse developer, it wastes the time of 
people learning argparse, it wastes the time of people who read the 
code and wonder what's the difference between SystemExit and 
ArgparseSystemExit. (Answer: there is no difference.)



-- 
Steven D'Aprano

From steve at pearwood.info  Sun Oct  4 03:29:23 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 4 Oct 2009 11:29:23 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <ha8f9a$u8i$1@ger.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<200910040522.28374.steve@pearwood.info>
	<ha8f9a$u8i$1@ger.gmane.org>
Message-ID: <200910041229.23279.steve@pearwood.info>

On Sun, 4 Oct 2009 08:21:46 am Robert Kern wrote:

> There are uses of argparse outside of command line apps. For example,
> I use it to parse --options for IPython %magic commands. Of course, I
> just subclass ArgumentParser and replace the .error() method.

Exactly. There are uses for catching SystemExit, which is why it's an 
exception and not an unconditional exit. But they're rare, and not 
difficult to deal with: in your case, having argparse raise a subclass 
of SystemExit won't help you, you would still need to subclass, and in 
other cases, you can just catch SystemExit.


> The other use case, as mentioned earlier, was for debugging your
> parser on the interactive prompt. A custom subclass may also be able
> to hold more machine-readable information about the error than the
> formatted error message, but I don't have any particular use cases
> about what may be the most useful.

Nobody has requested that the exception expose more information. They've 
requested that argparse paint the SystemExit a slightly different shade 
of yellow to the colour it already is -- this is pure bike-shedding.

Subclassing SystemExit just in case someday in the indefinite future 
there comes a need to add extra information to the exception falls foul 
of You Ain't Gonna Need It. Keep it simple -- if, someday, such a need 
becomes apparent, then subclass.



-- 
Steven D'Aprano

From kmtracey at gmail.com  Thu Oct  1 03:50:25 2009
From: kmtracey at gmail.com (Karen Tracey)
Date: Wed, 30 Sep 2009 21:50:25 -0400
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
References: <20090929202035.GA5492@gmail.com>
	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
	<9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
Message-ID: <af3536270909301850u64e0ddcbjba8693dc86c13e64@mail.gmail.com>

On Wed, Sep 30, 2009 at 5:15 PM, Yuvgoog Greenle <ubershmekel at gmail.com>wrote:

> I like how python has a minimalistic and powerful syntax (-1 for the break
> ___ PEP).
> Also, I really dislike the for/else ambiguity "butterflies".
> When is the else after a loop executed? 1. When the loop isn't entered at
> all.
> 2. When the loop terminates through exhaustion of the list (does this
> include when the list was empty?)
> 3. When the loop didn't exit because of a break statement.
>
> HINTS: The way django does it is opposite the way python does it and there
> may be more than one correct answer.
>
>
Django's template language does not have for/else, it has for/empty:
http://docs.djangoproject.com/en/dev/ref/templates/builtins/#for-empty

This construct did come from an external snippet that used 'else' instead of
'empty'.  However when it was moved into Django the 'else' name was
specifically rejected because it did the opposite of what for/else does in
Python.

Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/ea11e595/attachment.htm>

From benjamin at python.org  Sat Oct  3 20:37:42 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 3 Oct 2009 13:37:42 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
Message-ID: <1afaf6160910031137h2d298f75y99cde9666616bd22@mail.gmail.com>

2009/10/3 Brett Cannon <brett at python.org>:
> On Fri, Oct 2, 2009 at 20:03, Raymond Hettinger <python at rcn.com> wrote:
>>
>> [Terry Reedy]
>>>
>>> I would agree, for instance, that an auto-translation tool is needed.
>>
>> We should get one written. ?ISTM, every %-formatting
>> string is directly translatable to an equivalent {}-formatting string.
>
> Why don't we start something in the sandbox and see how far we can
> get. If no one beats me to it I will add the directory some time today
> and we can start hashing out the solution there.

I've already started a converter. It's here:
https://code.launchpad.net/~gutworth/+junk/mod2format



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sat Oct  3 11:37:00 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 03 Oct 2009 19:37:00 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
Message-ID: <4AC71B3C.10908@gmail.com>

Bugbee, Larry wrote:
> So until I see a *significant* benefit, my vote is *not* remove
> %-formatting.  Make both available and if {} is to win, it will.

Percent formatting isn't going anywhere (certainly not for the next
decade or so). However, its well documented limitations (especially the
lack of extensibility) and error-prone syntax (handling of dicts and
tuples, forgetting trailing type codes in namespace formatting, operator
precedence issues) are what led directly to PEP 3101 and the
introduction of brace formatting.

For the basic cases that percent formatting actually covers, it does a
reasonable job and is often more concise and currently executes faster
than the brace formatted equivalent (this was especially the case before
support for implicit positional argument numbering was added to brace
formatting).

The advantages of brace formatting don't really start to show up until
you actually start using the *new* features that the approach provides,
such as formatting for non-builtin types (e.g. Decimal and datetime
objects), referring to argument subscripts and attributes in field
definitions and the optional explicit numbering of references to
positional arguments (allowing the order of interpolation to be changed
in the format string without having to change the argument order in all
uses of that format string).

Also, don't forget that the percent formatting code has been around for
more than a decade and a half and has been optimised over that time. The
brace formatting code, on the other hand, is relatively new and probably
still offers plenty of opportunities for optimisation (although the
additionally flexibility in the brace formatting approach means that it
is unlikely to ever catch up completely to the raw speed of percent
formatting).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From benjamin at python.org  Thu Oct  1 00:21:12 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 30 Sep 2009 17:21:12 -0500
Subject: [Python-Dev] Python 2.6.3rc1 available
In-Reply-To: <87pr98yt2m.fsf@benfinney.id.au>
References: <B132EC23-B905-4BEC-B845-5E19BD4BE9A1@python.org>
	<87pr98yt2m.fsf@benfinney.id.au>
Message-ID: <1afaf6160909301521pfcc347fsa0f297c72945dabe@mail.gmail.com>

2009/9/30 Ben Finney <ben+python at benfinney.id.au>:
> Barry Warsaw <barry at python.org> writes:
>
>> The first (and hopefully last) release candidate for Python 2.6.3 is
>> now available
> [?]
>
> Thank you for this announcement, and the efforts that go into making
> this work available.
>
> *Especially* thank you for avoiding the oxymoronic ?Released: 2.6.3
> release candidate? or similar. I hope this signifies a trend toward
> using the more accurate term ?available? for announcing availability of
> anything that's not an actual release.

Alphas, betas, and release candidates are all releases, too.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sun Oct  4 10:13:59 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 04 Oct 2009 18:13:59 +1000
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <loom.20091003T194219-114@post.gmane.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>	<4AC783BC.70404@mrabarnett.plus.com>
	<loom.20091003T194219-114@post.gmane.org>
Message-ID: <4AC85947.3030702@gmail.com>

Antoine Pitrou wrote:
> If we can't find a way to make things almost transparent, we should IMO abandon
> the whole idea of a transition.

Yep - this is the reason some attempts at actual format translation
implementations started up as a result of the previous discussion.
Without that, I suspect any API transitions are going to be too much
hassle to be worthwhile.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sun Oct  4 10:23:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 04 Oct 2009 18:23:06 +1000
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <bbaeab100910031127x5fa3655h55f41f41ae8c62b0@mail.gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<200910040501.57077.steve@pearwood.info>
	<bbaeab100910031127x5fa3655h55f41f41ae8c62b0@mail.gmail.com>
Message-ID: <4AC85B6A.2040600@gmail.com>

Brett Cannon wrote:
> No one is saying we should deprecate % any time soon on strings
> themselves or anywhere. This discussion is purely in regards to
> argparse and logging to transition *their* APIs over to {} formatting
> which would most likely involve some deprecation for *using* %
> formatting in those APIs. But % formatting on strings themselves is
> not directly being discussed here.

Actually, I think percent formatting and brace formatting should remain
fully supported peers for the life of 3.x (including in APIs such as
argparse and logging).

No matter how much we might like the new formatting system (and I
certainly prefer it), there's a long period of evangelisation and
performance improvements that needs to happen before we can
realistically start to deprecate percent formatting based APIs in any
way shape or form.

Until that happens, then users do have the right to dig in their heels
and say "look, the limitations on percent formatting really don't bother
us, certainly not enough to switch to a new formatting system".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at barrys-emacs.org  Sun Oct  4 14:10:57 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 4 Oct 2009 13:10:57 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC47856.2090307@egenix.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<4AC2CD65.30905@v.loewis.de>
	<4AC33DE1.3040304@trueblade.com> <4AC47856.2090307@egenix.com>
Message-ID: <941A2586-F79E-4F5E-A77F-01FA0C183BBB@barrys-emacs.org>


On 1 Oct 2009, at 10:37, M.-A. Lemburg wrote:

> Eric Smith wrote:
>> Martin v. L?wis wrote:
>>> Steven Bethard wrote:
>>>> There's a lot of code already out there (in the standard library  
>>>> and
>>>> other places) that uses %-style formatting, when in Python 3.0 we
>>>> should be encouraging {}-style formatting.
>>>
>>> I don't agree that we should do that. I see nothing wrong with using
>>> % substitution.
>
> I agree with Martin.
>
> Both approaches have their ups and downs, but forcing users to move
> from %-formatting to .format()-formatting will just frustrate them:
> having to convert several thousand such (working) uses in their code
> with absolutely no benefit simply doesn't look like a good way to
> spend your time.

I agree you cannot force the move to {} format. There are programs that
expose the %(name)s in user interfaces for customisation.

>
> In addition to the code changes, such a move would also render
> existing translations of the %-formatted string templates useless.

Speaking of translation support has xgettext been updated to support {}?
It is a life saver to have xgettext report that "This %s and %s" is not
translatable.

Barry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091004/95c7fd51/attachment.htm>

From ncoghlan at gmail.com  Sun Oct  4 10:25:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 04 Oct 2009 18:25:50 +1000
Subject: [Python-Dev] Python 2.6.3rc1 available
In-Reply-To: <1afaf6160909301521pfcc347fsa0f297c72945dabe@mail.gmail.com>
References: <B132EC23-B905-4BEC-B845-5E19BD4BE9A1@python.org>	<87pr98yt2m.fsf@benfinney.id.au>
	<1afaf6160909301521pfcc347fsa0f297c72945dabe@mail.gmail.com>
Message-ID: <4AC85C0E.9000800@gmail.com>

Benjamin Peterson wrote:
> 2009/9/30 Ben Finney <ben+python at benfinney.id.au>:
>> Barry Warsaw <barry at python.org> writes:
>>
>>> The first (and hopefully last) release candidate for Python 2.6.3 is
>>> now available
>> [?]
>>
>> Thank you for this announcement, and the efforts that go into making
>> this work available.
>>
>> *Especially* thank you for avoiding the oxymoronic ?Released: 2.6.3
>> release candidate? or similar. I hope this signifies a trend toward
>> using the more accurate term ?available? for announcing availability of
>> anything that's not an actual release.
> 
> Alphas, betas, and release candidates are all releases, too.

Not really - they're just key builds that are made available in the lead
up to the actual release.

Using "<whatever> is available" for the pre-release builds and then
"RELEASED: <whatever>" for the actual release as Barry did this time
around makes a great deal of sense to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From brett at python.org  Sat Oct  3 22:09:36 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 3 Oct 2009 13:09:36 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> 
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com> 
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1> 
	<bbaeab100910031105ud738e2en762efe8e6b3ce0ea@mail.gmail.com>
Message-ID: <bbaeab100910031309k69d39b7bp6c7c20a6752e3324@mail.gmail.com>

On Sat, Oct 3, 2009 at 11:05, Brett Cannon <brett at python.org> wrote:
> On Fri, Oct 2, 2009 at 20:03, Raymond Hettinger <python at rcn.com> wrote:
>>
>> [Terry Reedy]
>>>
>>> I would agree, for instance, that an auto-translation tool is needed.
>>
>> We should get one written. ?ISTM, every %-formatting
>> string is directly translatable to an equivalent {}-formatting string.
>
> Why don't we start something in the sandbox and see how far we can
> get. If no one beats me to it I will add the directory some time today
> and we can start hashing out the solution there.

Created str_fmt_conversion in the sandbox to hold any work people want
to do on converting % formats to {} automatically.

-Brett

From ctrachte at gmail.com  Fri Oct  2 22:23:11 2009
From: ctrachte at gmail.com (Carl Trachte)
Date: Fri, 2 Oct 2009 14:23:11 -0600
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <87y6nt37kb.fsf@hbox.dyndns.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<87y6nt37kb.fsf@hbox.dyndns.org>
Message-ID: <426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>

>> Do the users get any say in this?
>
> I'm a user! :-)
>
> I hate calling methods on string literals, I think it looks very odd to
> have code like this:
>
>   "Displaying {0} of {1} revisions".format(x, y)
>
> Will we be able to write this as
>
>   "Displaying {0} of {1} revisions" % (x, y)
>
> too?
>
>> I imagine that some people are heavily invested in %-formatting.
>>
>> Because there has been limited uptake on {}-formatting (afaict), we
>> still have limited experience with knowing that it is actually better,
>> less error-prone, easier to learn/rember, etc. Outside a handful of
>> people on this list, I have yet to see anyone adopt it as the
>> preferred syntax.
>
> I've skimmed over the PEP, and the new {}-syntax seems to have some nice
> features. But I've not seen it used anywhere yet.
>

<delurk>

Rami Chowdhury posted this to a mailing list; I've been using it
(perhaps unintentionally promoting it) as part of non-English,
non-ASCII font outreach:

> def ?????_??(???):
>    ??? = "????? {0}. ???? ?????".format(???)
>    print(???)
>
> def say_greeting(name):
>   to_say = "Greetings, {0}. How are you?".format(name)
>   print(to_say)

As a user, my assumption was {} was going forward, rain or shine, and
everyone should be on board by Python 3.2.  (I thought once the Talin
PEP got approved, that was it).  I wrote Steven Bethard privately
about this.

Sorry for the intrusion.

</delurk>

From songofacandy at gmail.com  Mon Oct  5 03:52:31 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Mon, 5 Oct 2009 10:52:31 +0900
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<87y6nt37kb.fsf@hbox.dyndns.org>
	<426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
Message-ID: <ee9557410910041852i15f15471y36648c224f5ef482@mail.gmail.com>

What about using string prefix 'f'?

  f"{foo} and {bar}" % something == "{foo} and {bar}.format(something)

  s = f"{foo}"
  t = "%(bar)s"
  s + t  # raises Exception

Transition plan:
n: Just add F prefix. And adding "format_string" in future.
n+1: deprecate __mod__() without 'F'.
n+2: libraries use .format() and deprecate __mod__() with 'F'
n+3: remove __mod__()

-- 
Naoki INADA  <songofacandy at gmail.com>

From songofacandy at gmail.com  Mon Oct  5 04:07:07 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Mon, 5 Oct 2009 11:07:07 +0900
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <1afaf6160910041854v55edcc96we4fb2d6a7f2da948@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<87y6nt37kb.fsf@hbox.dyndns.org>
	<426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
	<ee9557410910041852i15f15471y36648c224f5ef482@mail.gmail.com>
	<1afaf6160910041854v55edcc96we4fb2d6a7f2da948@mail.gmail.com>
Message-ID: <ee9557410910041907g14322e24t934ff7f37b3aa95c@mail.gmail.com>

> -1 That requires keeping formatting information around in every string instance.
Adding new "format_string" class avoids it.
 unicode("foo") <=> u"foo"
 format_string("foo") <=> f"foo"

This way's pros:
* Many libraries can use one transition way.
* Transition stage syncs to Python version. "library A uses {} and
library B uses %" problem not happen in transition.
* We have experience same way on unicode.

-- 
Naoki INADA  <songofacandy at gmail.com>

From tjreedy at udel.edu  Mon Oct  5 06:43:05 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 05 Oct 2009 00:43:05 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<loom.20091002T113352-479@post.gmane.org>	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>	<87y6nt37kb.fsf@hbox.dyndns.org>
	<426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
Message-ID: <habtgm$v7u$1@ger.gmane.org>

Carl Trachte wrote:

>> I've skimmed over the PEP, and the new {}-syntax seems to have some nice
>> features. But I've not seen it used anywhere yet.

I am using it with 3.1 in an unreleased book I am still writing, and 
will in any code I publish.

> <delurk>
> 
> Rami Chowdhury posted this to a mailing list; I've been using it
> (perhaps unintentionally promoting it) as part of non-English,
> non-ASCII font outreach:
> 
>> def ?????_??(???):
>>    ??? = "????? {0}. ???? ?????".format(???)
>>    print(???)
>>
>> def say_greeting(name):
>>   to_say = "Greetings, {0}. How are you?".format(name)
>>   print(to_say)
> 
> As a user, my assumption was {} was going forward, rain or shine, and
> everyone should be on board by Python 3.2.

Autonumbering, added in 3.1, makes '{}' as easy to write for simple 
cases as '%s'. That was one complaint about the original 3.0 version. 
Another was and still is the lack of conversion, which is being worked on.

   (I thought once the Talin
> PEP got approved, that was it).  I wrote Steven Bethard privately
> about this.
> 
> Sorry for the intrusion.
> 
> </delurk>

tjr


From martin at v.loewis.de  Mon Oct  5 07:30:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 05 Oct 2009 07:30:45 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>
	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
Message-ID: <4AC98485.5050608@v.loewis.de>

> I agree with this 100% but I can't see it working unless we have some
> sort of transition plan. Just saying "ok, switch your format strings
> from % to {}" didn't work in Python 3.0 for various good reasons, and
> I can't imagine it will work in Python 4.0 unless we have a transition
> plan.

That doesn't mean we have to have a transition plan *now*. Creating one
after Python 3.5 is released (i.e. in 2015 or so) might be sufficient.

To create a transition plan, you first need *consensus* that you
actually do want to transition. I don't think such consensus is
available, and might not be available for a few more years. Pushing
the issue probably delays obtaining consensus.

Regards,
Martin

From ncoghlan at gmail.com  Sun Oct  4 10:03:24 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 04 Oct 2009 18:03:24 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091004T002759-476@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><loom.20091001T010253-42@post.gmane.org><loom.20091001T024614-990@post.gmane.org>	<ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com><loom.20091002T113352-479@post.gmane.org>	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>	<4AC6E019.4000701@gmail.com>
	<loom.20091004T002759-476@post.gmane.org>
Message-ID: <4AC856CC.4010806@gmail.com>

Vinay Sajip wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
>> I'm starting to think that a converter between the two format
>> mini-languages may be the way to go though.
>>
>> fmt_braces is meant to provide a superset of the capabilites of
>> fmt_percent, so a forward converter shouldn't be too hard. A reverse
>> converter may have to punt with ValueError when it finds things that
>> cannot be expressed in the fmt_percent mini language though.
> 
> I've done a first cut of a forward (% -> {}) converter:
> 
> http://gist.github.com/200936
> 
> but I'm not sure there's a case for a converter in the reverse direction, if
> we're encouraging movement in one particular direction.

It would allow an application to still use brace formatting throughout
even if one particularly library only accepted percent formatting.
Probably not worth the effort at this point though, as if we can get a
reliable forward converter happening then it may become possible for
APIs to correctly guess which kind of format string they have been passed.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From vinay_sajip at yahoo.co.uk  Mon Oct  5 10:27:49 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 5 Oct 2009 08:27:49 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
Message-ID: <loom.20091005T102425-605@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> We should get one written.  ISTM, every %-formatting
> string is directly translatable to an equivalent {}-formatting string.

I'm not sure you can always get equivalent output from the formatting, though.
For example:

>>> "%0#8x" % 0x1234
'0x001234'
>>> "{0:0>#8x}".format(0x1234)
'000x1234'
>>>

Someone please tell me if there's a better {}-format string which I've missed!

Regards,

Vinay Sajip


From fredrik at pythonware.com  Mon Oct  5 10:43:22 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 5 Oct 2009 10:43:22 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
Message-ID: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>

it's revews like this that makes me wonder if releasing open source is
a good idea:

   no egg - worst seen ever, remove it from pypi or provide an egg
(jensens, 2009-10-05, 0 points)

</F>

From p.f.moore at gmail.com  Mon Oct  5 10:41:08 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 5 Oct 2009 09:41:08 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091005T102425-605@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091005T102425-605@post.gmane.org>
Message-ID: <79990c6b0910050141r46338bdcjb7219a0658b01a5@mail.gmail.com>

2009/10/5 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> Raymond Hettinger <python <at> rcn.com> writes:
>> We should get one written. ?ISTM, every %-formatting
>> string is directly translatable to an equivalent {}-formatting string.
>
> I'm not sure you can always get equivalent output from the formatting, though.
> For example:
>
>>>> "%0#8x" % 0x1234
> '0x001234'
>>>> "{0:0>#8x}".format(0x1234)
> '000x1234'
>>>>
>
> Someone please tell me if there's a better {}-format string which I've missed!

Ironically,

>>> "0x{0:0>6x}".format(0x1234)
'0x001234'

ignoring the # modifier altogether...

Paul.

From python at mrabarnett.plus.com  Mon Oct  5 10:47:30 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 05 Oct 2009 09:47:30 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091005T102425-605@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091005T102425-605@post.gmane.org>
Message-ID: <4AC9B2A2.3010808@mrabarnett.plus.com>

Vinay Sajip wrote:
> Raymond Hettinger <python <at> rcn.com> writes:
>> We should get one written.  ISTM, every %-formatting
>> string is directly translatable to an equivalent {}-formatting string.
> 
> I'm not sure you can always get equivalent output from the formatting, though.
> For example:
> 
>>>> "%0#8x" % 0x1234
> '0x001234'
>>>> "{0:0>#8x}".format(0x1234)
> '000x1234'
> 
> Someone please tell me if there's a better {}-format string which I've missed!
> 
 >>> "{0:#08x}".format(0x1234)
'0x001234'

From ben+python at benfinney.id.au  Mon Oct  5 10:59:41 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 05 Oct 2009 19:59:41 +1100
Subject: [Python-Dev] eggs now mandatory for pypi?
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
Message-ID: <87hbuew70i.fsf@benfinney.id.au>

Fredrik Lundh <fredrik at pythonware.com> writes:

> it's revews like this that makes me wonder if releasing open source is
> a good idea:
>
>    no egg - worst seen ever, remove it from pypi or provide an egg
> (jensens, 2009-10-05, 0 points)

Heh. If harsh, uninformed responses make you wonder whether it's worth
doing free software, then you're in the wrong social philanthropy
movement.

Make the software good quality, release it in the standard format (in
this case, a Distutils ?sdist? on PyPI), respond in a timely manner to
genuine requests for help. Eggs are certainly not a requirement, nor is
responding to hostility like the above comment.

If you *want* to respond, you can politely direct them to
<URL:http://docs.python.org/install/> where they can learn how to
install Python distributions ? no mention of eggs at all.

-- 
 \         ?What is it that makes a complete stranger dive into an icy |
  `\   river to save a solid gold baby? Maybe we'll never know.? ?Jack |
_o__)                                                           Handey |
Ben Finney


From vinay_sajip at yahoo.co.uk  Mon Oct  5 12:16:53 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 5 Oct 2009 10:16:53 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091005T102425-605@post.gmane.org>
	<4AC9B2A2.3010808@mrabarnett.plus.com>
Message-ID: <loom.20091005T120251-827@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:

>  >>> "{0:#08x}".format(0x1234)
> '0x001234'

Good call, but here's another case:

>>> "%#o" % 0x1234
'011064'

I don't see how to automatically convert the "%#o" spec, though of course there
are ways of fudging it. The obvious conversion doesn't give the same value:

>>> "{0:#o}".format(0x1234)
'0o11064'

I couldn't see a backward-compatibility mode for str.format generating just a
leading "0" for octal alternative format (the C style) as opposed to "0o".

Regards,

Vinay Sajip


From orsenthil at gmail.com  Mon Oct  5 12:25:47 2009
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Mon, 5 Oct 2009 15:55:47 +0530
Subject: [Python-Dev] eggs now mandatory for pypi?
Message-ID: <20091005102547.GA23457@ubuntu.ubuntu-domain>

On Mon, Oct 05, 2009 at 10:43:22AM +0200, Fredrik Lundh wrote:
> it's revews like this that makes me wonder if releasing open source is
> a good idea:
>    no egg - worst seen ever, remove it from pypi or provide an egg
> (jensens, 2009-10-05, 0 points)
> 

Greetings effbot. :)
As you might already know, "eggs" are not mandatory for pypi. No where
in the documentation it mentions so. Also many good packages in pypi
are not eggs (MaL's egenix packages for e.g) and we have known his
dislike for that particular format too.

But still, sometimes people have found eggs to be convenient.

That was a particularly harsh comment from jensens, should be ignored
or he should be corrected that eggs are not mandatory.

But, i have a feeling that this is going to open up the well known,
can-of-worms on what is the best distribution format for python
packages debate.



-- 
Senthil
No one may kill a man.  Not for any purpose.  It cannot be condoned.
		-- Kirk, "Spock's Brain", stardate 5431.6

From fredrik at pythonware.com  Mon Oct  5 10:58:06 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 5 Oct 2009 10:58:06 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
Message-ID: <368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>

Oh, it was just yet another Zope developer behaving like an ass.  Why
am I not surprised?

</F>

On Mon, Oct 5, 2009 at 10:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
> it's reviews like this that makes me wonder if releasing open source is
> a good idea:
>
> ? no egg - worst seen ever, remove it from pypi or provide an egg
> (jensens, 2009-10-05, 0 points)
>
> </F>
>

From ncoghlan at gmail.com  Mon Oct  5 12:49:22 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 05 Oct 2009 20:49:22 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC98485.5050608@v.loewis.de>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org>	<ha449d$9n5$1@ger.gmane.org>	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<loom.20091002T113352-479@post.gmane.org>	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<4AC98485.5050608@v.loewis.de>
Message-ID: <4AC9CF32.4010404@gmail.com>

Martin v. L?wis wrote:
> That doesn't mean we have to have a transition plan *now*. Creating one
> after Python 3.5 is released (i.e. in 2015 or so) might be sufficient.
> 
> To create a transition plan, you first need *consensus* that you
> actually do want to transition. I don't think such consensus is
> available, and might not be available for a few more years. Pushing
> the issue probably delays obtaining consensus.

Agreed, but that doesn't rule out discussions of what can be done to
make such a transition easier. And just as 2to3 makes the overall Python
transition practical, a percent to brace format translator should make
an eventual formatting transition feasible.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fdrake at gmail.com  Mon Oct  5 12:47:29 2009
From: fdrake at gmail.com (Fred Drake)
Date: Mon, 5 Oct 2009 06:47:29 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com> 
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
Message-ID: <9cee7ab80910050347k6c5985b3m7290f64099526a2b@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:58 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Oh, it was just yet another Zope developer behaving like an ass. ?Why
> am I not surprised?

Actually, most of us Zope developers prefer sdist packages, not eggs.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From ncoghlan at gmail.com  Mon Oct  5 13:02:16 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 05 Oct 2009 21:02:16 +1000
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
Message-ID: <4AC9D238.5000501@gmail.com>

Fredrik Lundh wrote:
> Oh, it was just yet another Zope developer behaving like an ass.  Why
> am I not surprised?

Tarring an entire community for the actions of one twit is more than a
little unfair.

It's fine that you don't like eggs and it's fine that you don't want to
provide them. There's a reason egg-based packaging tools make it fairly
easy to create an egg from a normal sdist package.

As for people being twits... it's the internet. There isn't a lot to be
done other than to ignore them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From lists at cheimes.de  Mon Oct  5 13:07:02 2009
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 05 Oct 2009 13:07:02 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
Message-ID: <4AC9D356.5050202@cheimes.de>

Fredrik Lundh wrote:
> Oh, it was just yet another Zope developer behaving like an ass.  Why
> am I not surprised?
> 
> </F>
> 
> On Mon, Oct 5, 2009 at 10:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
>> it's reviews like this that makes me wonder if releasing open source is
>> a good idea:
>>
>>   no egg - worst seen ever, remove it from pypi or provide an egg
>> (jensens, 2009-10-05, 0 points)
>>
>> </F>

Actually Jens is a Plone developer ...

I'm including Jens in this discussion so he may shed some light on his
comment.

Christian

From fredrik at pythonware.com  Mon Oct  5 13:28:43 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 5 Oct 2009 13:28:43 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4AC9D238.5000501@gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com> 
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com> 
	<4AC9D238.5000501@gmail.com>
Message-ID: <368a5cd50910050428i30bded11v3d408b72b75e1e62@mail.gmail.com>

He's not the first one from the Zope community (whatever that is)
that's behaved this way on this specific topic.  The problem here is
that a certain (marginal) user community decides to standardize on a
certain distribution model, and then goes off attacking people who've
released stuff *before* they did that.  That *is* a community problem.

(Luckily, there are people helping out, and the "nice people
driven-development" rule overrides that other rule I mentioned, so
things will get tweaked sooner or later.)

</F>

On Mon, Oct 5, 2009 at 1:02 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Fredrik Lundh wrote:
>> Oh, it was just yet another Zope developer behaving like an ass. ?Why
>> am I not surprised?
>
> Tarring an entire community for the actions of one twit is more than a
> little unfair.
>
> It's fine that you don't like eggs and it's fine that you don't want to
> provide them. There's a reason egg-based packaging tools make it fairly
> easy to create an egg from a normal sdist package.
>
> As for people being twits... it's the internet. There isn't a lot to be
> done other than to ignore them.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
>

From jens at bluedynamics.com  Mon Oct  5 13:26:29 2009
From: jens at bluedynamics.com (Jens W. Klein)
Date: Mon, 05 Oct 2009 13:26:29 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4AC9D356.5050202@cheimes.de>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
	<4AC9D356.5050202@cheimes.de>
Message-ID: <1254741989.7368.22.camel@minime>

Am Montag, den 05.10.2009, 13:07 +0200 schrieb Christian Heimes:
> Fredrik Lundh wrote:
> > Oh, it was just yet another Zope developer behaving like an ass.  Why
> > am I not surprised?
> > 
> > </F>
> > 
> > On Mon, Oct 5, 2009 at 10:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
> >> it's reviews like this that makes me wonder if releasing open source is
> >> a good idea:
> >>
> >>   no egg - worst seen ever, remove it from pypi or provide an egg
> >> (jensens, 2009-10-05, 0 points)
> >>
> >> </F>
> 
> Actually Jens is a Plone developer ...
> 
> I'm including Jens in this discussion so he may shed some light on his
> comment.
> 
> Christian

I wrote a private message to Frederik (gmail account) and apologised for
my wrong comment. It was targeted wrong. Note-to-self: Never post when
angry about some $whatever. And as far as I understand PIL is not the
problem, but the packaging/ setuptools. For the records: PIL is a great
piece of software and I dont want to miss it. 

I hope in future we have a packaging-tool solving those problems. 

regards Jens


From nad at acm.org  Mon Oct  5 14:06:56 2009
From: nad at acm.org (Ned Deily)
Date: Mon, 05 Oct 2009 05:06:56 -0700
Subject: [Python-Dev] Package install failures in 2.6.3
Message-ID: <nad-6965DF.05065605102009@news.gmane.org>

There has been some discussion on the distutils-sig list over the past 
few days about a change to distutils released with 2.6.3 which 
inadvertently causes a number (all?) packages with C extensions that use 
setuptools/easy_install to fail during builds/installs with a rather 
cryptic error message.  A number of popular packages are affected 
directly or indirectly through dependencies.

The problem is summarized in the issue tracker here;
   http://bugs.python.org/issue7064
which includes links to the distutils list archive and the setuptools 
and Distribute project bug trackers.

Assuming that distutils is not changed in a forthcoming 2.6.4, users 
and, possibly package maintainers, will need to take some action: either 
migrate from setuptools to Distribute (an easy migration), or wait for a 
fix to setuptools or the package or to the distribution (if there is 
one).

I'm not proposing to debate the merits of all of the options here.  
However, if a 2.6.4 release is to be pushed out quickly for other 
reasons,  a one-time window of opportunity would be available and it 
would be prudent to at least consider the possibility of a distutils fix.

In any case, it seems like there should be some effort to make package 
maintainers aware of the problem so they can take whatever steps 
necessary to update their packages or help files or web sites etc.

-- 
 Ned Deily,
 nad at acm.org


From ziade.tarek at gmail.com  Mon Oct  5 14:22:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 14:22:06 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <nad-6965DF.05065605102009@news.gmane.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
Message-ID: <94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>

On Mon, Oct 5, 2009 at 2:06 PM, Ned Deily <nad at acm.org> wrote:
> Assuming that distutils is not changed in a forthcoming 2.6.4

Since the changes made in Distutils were bug fixes that kept all public API
backward compatible, I don't see why this should be changed.

Setuptools development has been discontinued for a year, and does
patches on Distutils code. Some of these patches are sensitive to any change
made on Distutils, wether those changes are internal or not.

It's up to the Setuptools project to provide a release that fixes this problem.
And this fix is one or two lines long.

Distutils will *not* check in its code if it was patched by Setuptools, to
make sure Setuptools 0.6c9 still works.

> I'm not proposing to debate the merits of all of the options here.
> However, if a 2.6.4 release is to be pushed out quickly for other
> reasons, ?a one-time window of opportunity would be available and it
> would be prudent to at least consider the possibility of a distutils fix.

What about a Setuptools release, with the proper fix ?

Tarek

From mal at egenix.com  Mon Oct  5 14:27:46 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 05 Oct 2009 14:27:46 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
Message-ID: <4AC9E642.3070508@egenix.com>

Tarek Ziad? wrote:
>> I'm not proposing to debate the merits of all of the options here.
>> However, if a 2.6.4 release is to be pushed out quickly for other
>> reasons,  a one-time window of opportunity would be available and it
>> would be prudent to at least consider the possibility of a distutils fix.
> 
> What about a Setuptools release, with the proper fix ?

+1

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From skip at pobox.com  Mon Oct  5 14:50:13 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 5 Oct 2009 07:50:13 -0500
Subject: [Python-Dev] [New-bugs-announce] [issue7064] Python 2.6.3 /
 setuptools 0.6c9: extension module builds fail with KeyError
In-Reply-To: <1254742115.57.0.346118025313.issue7064@psf.upfronthosting.co.za>
References: <1254742115.57.0.346118025313.issue7064@psf.upfronthosting.co.za>
Message-ID: <19145.60293.661555.312221@montanaro.dyndns.org>


    Ned> Due to a change in distutils released with Python 2.6.3, packages
    Ned> that use setuptools (version 0.6c9, as of this writing), or the
    Ned> easy_install command, to build C extension modules fail ...
    ...
    Ned> Among the packages known to be affected include lxml,
    Ned> zope-interface, jinja2, and, hence, packages dependent on these
    Ned> packages (e.g. sphinx, twisted, etc.).

Maybe the Python test suite should include tests with a small number of
widely used non-core packages like setuptools.  I realize the pybots project
exists to tackle this sort of stuff in greater detail.  I'm thinking more of
a smoke test than a comprehensive test suite covering all external packages.
Setuptools is particularly important because so many extension authors use
it.  If it breaks it implicitly breaks a lot of PyPI packages.

Skip

From ziade.tarek at gmail.com  Mon Oct  5 15:04:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 15:04:11 +0200
Subject: [Python-Dev] [New-bugs-announce] [issue7064] Python 2.6.3 /
	setuptools 0.6c9: extension module builds fail with KeyError
In-Reply-To: <19145.60293.661555.312221@montanaro.dyndns.org>
References: <1254742115.57.0.346118025313.issue7064@psf.upfronthosting.co.za>
	<19145.60293.661555.312221@montanaro.dyndns.org>
Message-ID: <94bdd2610910050604k5cc20cdblf24a65c307a1cfdf@mail.gmail.com>

On Mon, Oct 5, 2009 at 2:50 PM,  <skip at pobox.com> wrote:
>
> ? ?Ned> Due to a change in distutils released with Python 2.6.3, packages
> ? ?Ned> that use setuptools (version 0.6c9, as of this writing), or the
> ? ?Ned> easy_install command, to build C extension modules fail ...
> ? ?...
> ? ?Ned> Among the packages known to be affected include lxml,
> ? ?Ned> zope-interface, jinja2, and, hence, packages dependent on these
> ? ?Ned> packages (e.g. sphinx, twisted, etc.).
>
> Maybe the Python test suite should include tests with a small number of
> widely used non-core packages like setuptools. ?I realize the pybots project
> exists to tackle this sort of stuff in greater detail. ?I'm thinking more of
> a smoke test than a comprehensive test suite covering all external packages.
> Setuptools is particularly important because so many extension authors use
> it. ?If it breaks it implicitly breaks a lot of PyPI packages.
>

I have created 6 months ago such a buildbot that downloads tarballs
from the community,
and run a few distutils commands on them, and make sure the result is
similar in 2.6/2.7.
and for "sdist" that the resulting tarball is similar.

It was running over Twisted and Numpy, but has been discontinued because
it was on my own server, where it was hard to keep it up (cpu/bandwidth)

If the Snakebite project could host my buildbot (or at least slaves)
or if the PSF could pay for a dedicated
server for this, we would be able to trigger such warnings, and
provide an e-mail service to package maintainers for example.

The build could occur everytime Distutils *or* the project changes.

> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From jnoller at gmail.com  Mon Oct  5 15:13:44 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 5 Oct 2009 09:13:44 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
Message-ID: <4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
> it's revews like this that makes me wonder if releasing open source is
> a good idea:
>
> ? no egg - worst seen ever, remove it from pypi or provide an egg
> (jensens, 2009-10-05, 0 points)
>
> </F>

Unfortunately; we're now staring down the barrel of having
youtube-style comments on Python packages on the index.

From barry at python.org  Mon Oct  5 15:32:28 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Oct 2009 09:32:28 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
Message-ID: <BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>

On Oct 5, 2009, at 8:22 AM, Tarek Ziad? wrote:

>> I'm not proposing to debate the merits of all of the options here.
>> However, if a 2.6.4 release is to be pushed out quickly for other
>> reasons,  a one-time window of opportunity would be available and it
>> would be prudent to at least consider the possibility of a  
>> distutils fix.
>
> What about a Setuptools release, with the proper fix ?

If setuptools can be made to work with Python 2.6.3 /and/ earlier  
versions of Python 2.6.x, then it should be patched and an update  
released.  If not, then perhaps we should revert the change in a quick  
Python 2.6.4.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/573dc51b/attachment.pgp>

From olemis at gmail.com  Mon Oct  5 16:07:52 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 5 Oct 2009 09:07:52 -0500
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <1254741989.7368.22.camel@minime>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
	<4AC9D356.5050202@cheimes.de> <1254741989.7368.22.camel@minime>
Message-ID: <24ea26600910050707t3d5cbfccu9a3e9113503ef0d5@mail.gmail.com>

On Mon, Oct 5, 2009 at 6:26 AM, Jens W. Klein <jens at bluedynamics.com> wrote:
> Am Montag, den 05.10.2009, 13:07 +0200 schrieb Christian Heimes:
>> Fredrik Lundh wrote:
>> >
>> > Oh, it was just yet another Zope developer behaving like an ass. ?Why
>> > am I not surprised?
>> >
>> > On Mon, Oct 5, 2009 at 10:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
>> >> it's reviews like this that makes me wonder if releasing open source is
>> >> a good idea:
>> >>

Peace for the world brothers . Lots of yoga, meditation, tai-chi ...
and tons of FOSS

>> >> ? no egg - worst seen ever, remove it from pypi or provide an egg
>> >> (jensens, 2009-10-05, 0 points)
>> >>
[...]
>
> Note-to-self: Never post when
> angry about some $whatever.

e.g. I don't do it either when I'm drunk or thinking about Britney  ;o)

> And as far as I understand PIL is not the
> problem, but the packaging/ setuptools.

well I just mentioned few times before (and J. P. Eby said something
too, many times actually ;o) that setuptools solved a crucial problem
that was unsolved when it was released

> For the records: PIL is a great
> piece of software and I dont want to miss it.
>

Yes, and the fact is that without setuptools many other wonderful
(commands | libs | frameworks | apps | software) would be in the
darkness. I mention some of them :

  - setuptools `test` command
  - Trac
  - PasteDeploy
  - ... and here fuel is over ... there are a lot believe me ;o)

the "small" price to pay is that there are a few annoying
implementation details in there ...

> I hope in future we have a packaging-tool solving those problems.
>

... or OTOH that some time is needed to improve it ;o) but considering
the benefits ...

Besides you could use

{{{
$ easy_install -Z eggs_are_awesome.egg
}}}

and the package will be installed directly in the file system ( modify
.pth + PYTHONPATH if desired ;o).

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Nabble - Trac Users - Coupling trac and symfony framework  -
http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html

From ziade.tarek at gmail.com  Mon Oct  5 16:17:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 16:17:07 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
Message-ID: <94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>

On Mon, Oct 5, 2009 at 3:32 PM, Barry Warsaw <barry at python.org> wrote:
> On Oct 5, 2009, at 8:22 AM, Tarek Ziad? wrote:
>
>>> I'm not proposing to debate the merits of all of the options here.
>>> However, if a 2.6.4 release is to be pushed out quickly for other
>>> reasons, ?a one-time window of opportunity would be available and it
>>> would be prudent to at least consider the possibility of a distutils fix.
>>
>> What about a Setuptools release, with the proper fix ?
>
> If setuptools can be made to work with Python 2.6.3 /and/ earlier versions
> of Python 2.6.x, then it should be patched and an update released. ?If not,
> then perhaps we should revert the change in a quick Python 2.6.4.

It's technically possible to fix Setuptools. We did this fix on Distribute, and
the patch is 2 lines long.

But it's just a matter of having the maintainer doing it. A few months ago we
couldn't make him fix and release the bug that made setuptools fail
with svn 1.6, and the year before it took several months to get it
fixed for svn 1.5
(a one line, not risky change !!!)

That's why we have forked and created Distribute, to provide bug fixes.

If PJE is not concerned anymore by the maintenance, imho he should let someone
that is willing to do it take over the maintenance of his package to
fix this (and the other problems). That is not a new problem.

Beware that I don't want to run in any new vicious thread here: I had
my share of those.

So about taking over Setuptools maintenance :
1/ I am not saying it should be me, and I am not saying that I am
offended that PJE didn't open the maintenance of setuptools to me.  I
think he should trust the community and let the maintenance of
setuptools be done by all the people that are actively working on the topic.

2/ No, as someone told me in IRC, that's not an evil plan of mine to
make people switch to Distribute. This is not in our interest, it's a
loss-loss situation.

Now I am strongly opposed to revert any bug fix change in Distutils
just because it breaks Setuptools, which is unmaintained since a year.

We have been struggling over a year with this issue. And we are still
struggling because we have to work in a fork to try to provide solutions
for the community, with a lot of bootstrapping issues.

Now I am astonished that we are talking about reverting changes in
Distutils that were done for bugfixes, for a third party package that
does monkey
patches on Distutils.

If this choice wins here, it means that setuptools and the stdlib are
tied together, and that the setuptools package should be integrated to
the stdlib *immediatly*.

Tarek

From mal at egenix.com  Mon Oct  5 16:27:00 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 05 Oct 2009 16:27:00 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
Message-ID: <4ACA0234.1070700@egenix.com>

Tarek Ziad? wrote:
> Now I am astonished that we are talking about reverting changes in
> Distutils that were done for bugfixes, for a third party package that
> does monkey
> patches on Distutils.
> 
> If this choice wins here, it means that setuptools and the stdlib are
> tied together, and that the setuptools package should be integrated to
> the stdlib *immediatly*.

We've discussed ways of doing that some years ago and found that
it was not possible to make ends join.

I'd much rather like to see some of the features in setuptools
get implemented in distutils. eGenix could contribute a
bdist_egg implementation that doesn't rely on setuptools and its
monkey patches - along with some other new commands that people
might find useful such as the ability to build Unix libraries,
optional and self-configuring extensions, autoconf-style setups,
etc. (see mxSetup.py in egenix-mx-base for all the details).

We'd just need some help with integrating the things into
distutils, since we currently don't have the bandwidth for such
things.

Aside: After importing setuptools a lot of things in distutils no
longer work as advertised. This is due to the heavy monkey patching
being applied in setuptools. In some cases there's on other way
to adapt distutils to ones needs, but most of the times, a little
more thought put into the OO design of a tool makes it possible to
play nice with distutils without any monkey patching.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Mon Oct  5 16:32:13 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Oct 2009 10:32:13 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
Message-ID: <F0920F9C-849A-4761-91C3-118D7555C742@python.org>

On Oct 5, 2009, at 10:17 AM, Tarek Ziad? wrote:

>> If setuptools can be made to work with Python 2.6.3 /and/ earlier  
>> versions
>> of Python 2.6.x, then it should be patched and an update released.   
>> If not,
>> then perhaps we should revert the change in a quick Python 2.6.4.
>
> It's technically possible to fix Setuptools. We did this fix on  
> Distribute, and
> the patch is 2 lines long.

My question was less about the political aspects than the technical  
aspects.  I gather you're saying that the fix to setuptools will make  
it work in 2.6.3 without inadvertently breaking it for 2.6.2, 2.6.1,  
and 2.6.0, right?  Have you tried the fix in those older versions to  
be sure?

If, as I hope, the answer to that is "yes", then I strongly support  
releasing a fixed setuptools instead of reverting the change to Python.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/df1994a9/attachment-0001.pgp>

From mal at egenix.com  Mon Oct  5 16:32:23 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 05 Oct 2009 16:32:23 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <4ACA0234.1070700@egenix.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<4ACA0234.1070700@egenix.com>
Message-ID: <4ACA0377.3070903@egenix.com>

M.-A. Lemburg wrote:
> Tarek Ziad? wrote:
>> Now I am astonished that we are talking about reverting changes in
>> Distutils that were done for bugfixes, for a third party package that
>> does monkey
>> patches on Distutils.
>>
>> If this choice wins here, it means that setuptools and the stdlib are
>> tied together, and that the setuptools package should be integrated to
>> the stdlib *immediatly*.
> 
> We've discussed ways of doing that some years ago and found that
> it was not possible to make ends join.

[just realized that the above could be mistaken for "make ends
 meet" which has a completely different meaning in English than
 the German idiom; what I meant was that no consensus could be
 reached]

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From skip at pobox.com  Mon Oct  5 16:40:02 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 5 Oct 2009 09:40:02 -0500
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
Message-ID: <19146.1346.257750.667842@montanaro.dyndns.org>


    Tarek> That's why we have forked and created Distribute, to provide bug
    Tarek> fixes.

I suspect you might need to publicize this a bit better.  The first I heard
of Distribute or non-responsiveness of PJE regarding setuptools was this
thread.  (I don't read comp.lang.python anymore.  I do read python-dev and
comp.lang.python.announce.  Maybe I just missed it.)

    Tarek> Now I am astonished that we are talking about reverting changes
    Tarek> in Distutils that were done for bugfixes, for a third party
    Tarek> package that does monkey patches on Distutils.

As I said, I was completely unaware of the problems you're addressing with
Distribute.  My guess is that many extension writers and almost certainly
those people who install extensions will be similarly unaware of the issues.

Skip

From daniel at stutzbachenterprises.com  Mon Oct  5 16:50:43 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 5 Oct 2009 09:50:43 -0500
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <F0920F9C-849A-4761-91C3-118D7555C742@python.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
Message-ID: <eae285400910050750n18a281c6ub86759103725b821@mail.gmail.com>

On Mon, Oct 5, 2009 at 9:32 AM, Barry Warsaw <barry at python.org> wrote:

> If, as I hope, the answer to that is "yes", then I strongly support
> releasing a fixed setuptools instead of reverting the change to Python.
>

How do your propose to get the author of setuptools to release a new
version?

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/a6436b36/attachment.htm>

From ziade.tarek at gmail.com  Mon Oct  5 16:50:46 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 16:50:46 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <F0920F9C-849A-4761-91C3-118D7555C742@python.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
Message-ID: <94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:32 PM, Barry Warsaw <barry at python.org> wrote:
> On Oct 5, 2009, at 10:17 AM, Tarek Ziad? wrote:
>
>>> If setuptools can be made to work with Python 2.6.3 /and/ earlier
>>> versions
>>> of Python 2.6.x, then it should be patched and an update released. ?If
>>> not,
>>> then perhaps we should revert the change in a quick Python 2.6.4.
>>
>> It's technically possible to fix Setuptools. We did this fix on
>> Distribute, and
>> the patch is 2 lines long.
>
> My question was less about the political aspects than the technical aspects.
> ?I gather you're saying that the fix to setuptools will make it work in
> 2.6.3 without inadvertently breaking it for 2.6.2, 2.6.1, and 2.6.0, right?
> ?Have you tried the fix in those older versions to be sure?
>
> If, as I hope, the answer to that is "yes", then I strongly support
> releasing a fixed setuptools instead of reverting the change to Python.

Yes it does.

The fix makes sure build_ext.get_ext_filename still works as it is supposed
(just returning the extension of a filename depending on the
environment) under all versions,

*and* continues to do extra work for the Setuptools internal needs when
it is called by Setuptools itself on various places.

It's two lines in Setuptools.

But beware that a new Setuptools release might take a few months and
maybe will never happen.
That is why a suggested solution was to install Distribute, because it
addresses among other
bug fixes, this one.

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From ziade.tarek at gmail.com  Mon Oct  5 16:53:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 16:53:41 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <4ACA0234.1070700@egenix.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<4ACA0234.1070700@egenix.com>
Message-ID: <94bdd2610910050753x5f1eab38w3d24d3d8eacd09b4@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Tarek Ziad? wrote:
>> Now I am astonished that we are talking about reverting changes in
>> Distutils that were done for bugfixes, for a third party package that
>> does monkey
>> patches on Distutils.
>>
>> If this choice wins here, it means that setuptools and the stdlib are
>> tied together, and that the setuptools package should be integrated to
>> the stdlib *immediatly*.
>
> We've discussed ways of doing that some years ago and found that
> it was not possible to make ends join.
>
> I'd much rather like to see some of the features in setuptools
> get implemented in distutils. eGenix could contribute a
> bdist_egg implementation that doesn't rely on setuptools and its
> monkey patches - along with some other new commands that people
> might find useful such as the ability to build Unix libraries,
> optional and self-configuring extensions, autoconf-style setups,
> etc. (see mxSetup.py in egenix-mx-base for all the details).
>
> We'd just need some help with integrating the things into
> distutils, since we currently don't have the bandwidth for such
> things.

What about making the Distribute project the laboratory for this work ?
It's open to contributions.

We are also planning to implement early versions of the latest PEP proposals
concerning packaging in this project, once they are finished and accepted,
before it goes into Distutils itself.

>
> Aside: After importing setuptools a lot of things in distutils no
> longer work as advertised. This is due to the heavy monkey patching
> being applied in setuptools.

Yes the code there is very sensitive to any change.

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From ziade.tarek at gmail.com  Mon Oct  5 17:02:02 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 17:02:02 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <19146.1346.257750.667842@montanaro.dyndns.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<19146.1346.257750.667842@montanaro.dyndns.org>
Message-ID: <94bdd2610910050802v72a3c8aeq9698d78d3f5777fa@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:40 PM,  <skip at pobox.com> wrote:
>
> ? ?Tarek> That's why we have forked and created Distribute, to provide bug
> ? ?Tarek> fixes.
>
> I suspect you might need to publicize this a bit better. ?The first I heard
> of Distribute or non-responsiveness of PJE regarding setuptools was this
> thread. ?(I don't read comp.lang.python anymore. ?I do read python-dev and
> comp.lang.python.announce. ?Maybe I just missed it.)

No you didn't miss it. That's probably my fault because the only places
I publicize about it are my blog (indexed in planet python) and the
distutils-SIG.

Maybe I should blog a summary of the situation and post it to python annoucement
as well.

>
> ? ?Tarek> Now I am astonished that we are talking about reverting changes
> ? ?Tarek> in Distutils that were done for bugfixes, for a third party
> ? ?Tarek> package that does monkey patches on Distutils.
>
> As I said, I was completely unaware of the problems you're addressing with
> Distribute. ?My guess is that many extension writers and almost certainly
> those people who install extensions will be similarly unaware of the issues.

Right, and an end-user can't be aware of those. But until they depend on a
project that is not maintained and that we, active people, can't maintain;
we will face such problems.

Now I am dedicated to help all extension writers out there, and as a matter
of fact I've been doing it since a few weeks on #distutils on freenode,
and all the results of this work are contributed in Distribute.

I have written down two important things so far from this thread:

1/ the need to provide a builbdot service for packagers that wish to
try their package against the latest Distutils
2/ the need to publicize what is going on for a wider audience.

Tarek

From mal at egenix.com  Mon Oct  5 17:11:54 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 05 Oct 2009 17:11:54 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <20091005102547.GA23457@ubuntu.ubuntu-domain>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
Message-ID: <4ACA0CBA.2030200@egenix.com>

Senthil Kumaran wrote:
> On Mon, Oct 05, 2009 at 10:43:22AM +0200, Fredrik Lundh wrote:
>> it's revews like this that makes me wonder if releasing open source is
>> a good idea:
>>    no egg - worst seen ever, remove it from pypi or provide an egg
>> (jensens, 2009-10-05, 0 points)
>>
> 
> Greetings effbot. :)
> As you might already know, "eggs" are not mandatory for pypi. No where
> in the documentation it mentions so. Also many good packages in pypi
> are not eggs (MaL's egenix packages for e.g) and we have known his
> dislike for that particular format too.

I don't dislike the format itself, just the tools that are
available for using it.

We've implemented our own bdist_egg now which doesn't use setuptools
and will start to ship eggs in addition to our prebuilt format with
the next releases.

As time permits, we'll also write an egg installer/uninstaller
(one that does only that ;-).

> But still, sometimes people have found eggs to be convenient.
> 
> That was a particularly harsh comment from jensens, should be ignored
> or he should be corrected that eggs are not mandatory.
> 
> But, i have a feeling that this is going to open up the well known,
> can-of-worms on what is the best distribution format for python
> packages debate.

Let's hope not :-) Besides, the discussions should be about the
tools used for package distribution - the egg format itself
is fine.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Mon Oct  5 17:26:02 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 05 Oct 2009 17:26:02 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050753x5f1eab38w3d24d3d8eacd09b4@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>	<4ACA0234.1070700@egenix.com>
	<94bdd2610910050753x5f1eab38w3d24d3d8eacd09b4@mail.gmail.com>
Message-ID: <4ACA100A.8020602@egenix.com>

Tarek Ziad? wrote:
> On Mon, Oct 5, 2009 at 4:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Tarek Ziad? wrote:
>>> Now I am astonished that we are talking about reverting changes in
>>> Distutils that were done for bugfixes, for a third party package that
>>> does monkey
>>> patches on Distutils.
>>>
>>> If this choice wins here, it means that setuptools and the stdlib are
>>> tied together, and that the setuptools package should be integrated to
>>> the stdlib *immediatly*.
>>
>> We've discussed ways of doing that some years ago and found that
>> it was not possible to make ends join.
>>
>> I'd much rather like to see some of the features in setuptools
>> get implemented in distutils. eGenix could contribute a
>> bdist_egg implementation that doesn't rely on setuptools and its
>> monkey patches - along with some other new commands that people
>> might find useful such as the ability to build Unix libraries,
>> optional and self-configuring extensions, autoconf-style setups,
>> etc. (see mxSetup.py in egenix-mx-base for all the details).
>>
>> We'd just need some help with integrating the things into
>> distutils, since we currently don't have the bandwidth for such
>> things.
> 
> What about making the Distribute project the laboratory for this work ?
> It's open to contributions.

The way the distutils extensions are implemented (by sub-classing
distutils classes) make it easier to add them back to core
distutils, rather than to integrate them into another 3rd
party distutils extension.

I'm not sure how adding them to Distribute would help, since then
you'd only be able to access them using Distribute.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From guido at python.org  Mon Oct  5 17:28:27 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Oct 2009 08:28:27 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
Message-ID: <ca471dc20910050828o2ed717fbn6434a9c319ef5b15@mail.gmail.com>

\>> I hate calling methods on string literals, I think it looks very odd
>> to have code like this:
>>
>> ? "Displaying {0} of {1} revisions".format(x, y)
>
> Ugh! ?Good point.

This objection was made years ago when we introduced
"separator".join(list_of_things), and I don't think ignoring it has
caused any casualties. In fact, it serves as an early reminder to the
user that string literals are, in fact, objects like all others.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ziade.tarek at gmail.com  Mon Oct  5 17:42:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 17:42:50 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <4ACA100A.8020602@egenix.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<4ACA0234.1070700@egenix.com>
	<94bdd2610910050753x5f1eab38w3d24d3d8eacd09b4@mail.gmail.com>
	<4ACA100A.8020602@egenix.com>
Message-ID: <94bdd2610910050842p615a92a4je57560c04b2d076d@mail.gmail.com>

On Mon, Oct 5, 2009 at 5:26 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Tarek Ziad? wrote:
>> On Mon, Oct 5, 2009 at 4:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> Tarek Ziad? wrote:
>>>> Now I am astonished that we are talking about reverting changes in
>>>> Distutils that were done for bugfixes, for a third party package that
>>>> does monkey
>>>> patches on Distutils.
>>>>
>>>> If this choice wins here, it means that setuptools and the stdlib are
>>>> tied together, and that the setuptools package should be integrated to
>>>> the stdlib *immediatly*.
>>>
>>> We've discussed ways of doing that some years ago and found that
>>> it was not possible to make ends join.
>>>
>>> I'd much rather like to see some of the features in setuptools
>>> get implemented in distutils. eGenix could contribute a
>>> bdist_egg implementation that doesn't rely on setuptools and its
>>> monkey patches - along with some other new commands that people
>>> might find useful such as the ability to build Unix libraries,
>>> optional and self-configuring extensions, autoconf-style setups,
>>> etc. (see mxSetup.py in egenix-mx-base for all the details).
>>>
>>> We'd just need some help with integrating the things into
>>> distutils, since we currently don't have the bandwidth for such
>>> things.
>>
>> What about making the Distribute project the laboratory for this work ?
>> It's open to contributions.
>
> The way the distutils extensions are implemented (by sub-classing
> distutils classes) make it easier to add them back to core
> distutils, rather than to integrate them into another 3rd
> party distutils extension.
>
> I'm not sure how adding them to Distribute would help, since then
> you'd only be able to access them using Distribute.

I was thinking about the release cycles but this is maybe because I am
unaware of the way eGenix packages are developed, so I might be wrong.

Distribute has the bandwidth to work on this with many contributors
with a smaller release cycle than stdlib, allowing more feedback.

Notice that I am also doing nightly builds of Distutils that can be installed
and tried in released version of Python, and that can be used instead of the
Python's embed Distutils  (see the installable tarballs at nightly.ziade.org).
so maybe it's just a matter of continuous integration

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From barry at python.org  Mon Oct  5 17:44:40 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Oct 2009 11:44:40 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
Message-ID: <A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>

On Oct 5, 2009, at 10:50 AM, Tarek Ziad? wrote:

>> If, as I hope, the answer to that is "yes", then I strongly support
>> releasing a fixed setuptools instead of reverting the change to  
>> Python.
>
> Yes it does.

Excellent, thanks.

> The fix makes sure build_ext.get_ext_filename still works as it is  
> supposed
> (just returning the extension of a filename depending on the
> environment) under all versions,
>
> *and* continues to do extra work for the Setuptools internal needs  
> when
> it is called by Setuptools itself on various places.
>
> It's two lines in Setuptools.
>
> But beware that a new Setuptools release might take a few months and
> maybe will never happen.
> That is why a suggested solution was to install Distribute, because it
> addresses among other
> bug fixes, this one.

I think we have two other choices.

2) PJE releases a new version of setuptools that fixes this problem.

3) We (a.k.a. Tarek with our blessing) hijacks the setuptools name  
(e.g. on cheeseshop) and releases a new version

I still hope #2 happens, but let's have a deadline for when the more  
drastic measures will be taken.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/9caae0fd/attachment.pgp>

From barry at python.org  Mon Oct  5 17:45:57 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Oct 2009 11:45:57 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <eae285400910050750n18a281c6ub86759103725b821@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<eae285400910050750n18a281c6ub86759103725b821@mail.gmail.com>
Message-ID: <2B4FDAB5-7D76-42BD-9352-8A98CB82EC7F@python.org>

On Oct 5, 2009, at 10:50 AM, Daniel Stutzbach wrote:

> On Mon, Oct 5, 2009 at 9:32 AM, Barry Warsaw <barry at python.org> wrote:
> If, as I hope, the answer to that is "yes", then I strongly support  
> releasing a fixed setuptools instead of reverting the change to  
> Python.
>
> How do your propose to get the author of setuptools to release a new  
> version?

He either will or he won't.  Let's give a deadline for when it must  
happen or "we" will do what we need to do.  I think a week to 10 days  
is reasonable.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/81053bfb/attachment-0001.pgp>

From g.brandl at gmx.net  Mon Oct  5 18:00:22 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 05 Oct 2009 18:00:22 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
Message-ID: <had53u$rrp$1@ger.gmane.org>

Jesse Noller schrieb:
> On Mon, Oct 5, 2009 at 4:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
>> it's revews like this that makes me wonder if releasing open source is
>> a good idea:
>>
>>   no egg - worst seen ever, remove it from pypi or provide an egg
>> (jensens, 2009-10-05, 0 points)
>>
>> </F>
> 
> Unfortunately; we're now staring down the barrel of having
> youtube-style comments on Python packages on the index.

Yes, unfortunately. I originally thought that restricting the commenters
to those with a PyPI account would make them useful, but seeing this one
(even if it was not intended) and the comment on hgsvn that belongs into
a bug tracker instead, I'm not so sure anymore.

Georg


From asmodai at in-nomine.org  Mon Oct  5 18:29:06 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Mon, 5 Oct 2009 18:29:06 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
Message-ID: <20091005162905.GF7162@nexus.in-nomine.org>

-On [20091005 17:45], Barry Warsaw (barry at python.org) wrote:
>2) PJE releases a new version of setuptools that fixes this problem.

Will all due respect to PJE, but I sincerely doubt this will happen. There's
still a bunch of outstanding patches over at the setuptools tracker,
including SVN 1.x support, which have been open for more than a year.

Personally I consider setuptools unmaintained and have started moving my
projects over to Distribute.

So, if you decide to wait, make sure to put a deadline on it.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
The earth laughs in flowers...

From skip at pobox.com  Mon Oct  5 18:36:49 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 5 Oct 2009 11:36:49 -0500
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050802v72a3c8aeq9698d78d3f5777fa@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<19146.1346.257750.667842@montanaro.dyndns.org>
	<94bdd2610910050802v72a3c8aeq9698d78d3f5777fa@mail.gmail.com>
Message-ID: <19146.8353.817970.409627@montanaro.dyndns.org>


    Tarek> No you didn't miss it. That's probably my fault because the only
    Tarek> places I publicize about it are my blog (indexed in planet
    Tarek> python) and the distutils-SIG.

Bloggers beware!!!  Not everyone reads blogs.  (I don't unless someone calls
my attention to something of particular interest.)  Even if everyone did
read blogs, the risk of missing a particular post is extremely high
considering the number of planet.python.org subscriptions.  I don't know how
many blogs are aggregated on planet.python.org but a quick scan suggests
it's well over 100 at this point.

Moral of the story:  If you have something to announce, announce it in the
proper channel: python-announce-list at python.org.

Skip

From solipsis at pitrou.net  Mon Oct  5 18:44:02 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 5 Oct 2009 16:44:02 +0000 (UTC)
Subject: [Python-Dev] Package install failures in 2.6.3
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
Message-ID: <loom.20091005T182820-734@post.gmane.org>

Hello,

> Now I am astonished that we are talking about reverting changes in
> Distutils that were done for bugfixes, for a third party package that
> does monkey
> patches on Distutils.

I think we should avoid any questions of responsability here (besides, it is
quite clear that you, Tarek, are not responsible for the pre-existing mess, and
the lack of maintenance on certain popular software). Knowing who is responsible
doesn't make our users' life any better if nothing's fixed in time.

The only question is, given that 2.6.x is supposed to be a bug-fix branch, do we
want to fix that incompatibility with a widely deployed existing piece of
software? Whether or not the incompatibility is legitimate (i.e., whether
setuptools is badly written and relies on implementation details) is beyond the
point, IMO.

[ The issue, of course, is quite different if we come to talk about trunk. ]

> If this choice wins here, it means that setuptools and the stdlib are
> tied together, and that the setuptools package should be integrated to
> the stdlib *immediatly*.

Oh, certainly not. I don't think anybody would want that at this point, even the
most hardcore supporters of setuptools :-)

Regards

Antoine.



From ziade.tarek at gmail.com  Mon Oct  5 18:53:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 18:53:31 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <loom.20091005T182820-734@post.gmane.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<loom.20091005T182820-734@post.gmane.org>
Message-ID: <94bdd2610910050953p7158aabcx429f535ce7c674b8@mail.gmail.com>

On Mon, Oct 5, 2009 at 6:44 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> The only question is, given that 2.6.x is supposed to be a bug-fix branch, do we
> want to fix that incompatibility with a widely deployed existing piece of
> software? Whether or not the incompatibility is legitimate (i.e., whether
> setuptools is badly written and relies on implementation details) is beyond the
> point, IMO.

Right. As discussed with Barry, I will work on an internal special
case in Distutils in the 2.6 maint.
branch so it fixes the incompatibility with the Setuptools 0.6c9 release.

Tarek.

From stephen at xemacs.org  Mon Oct  5 19:13:04 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 Oct 2009 02:13:04 +0900
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050802v72a3c8aeq9698d78d3f5777fa@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<19146.1346.257750.667842@montanaro.dyndns.org>
	<94bdd2610910050802v72a3c8aeq9698d78d3f5777fa@mail.gmail.com>
Message-ID: <871vlhvk67.fsf@uwakimon.sk.tsukuba.ac.jp>

Tarek Ziad? writes:

 > Maybe I should blog a summary of the situation and post it to
 > python annoucement as well.

Please don't.  It's unlikely to do anything except incite a lot of
flames.


From mal at egenix.com  Mon Oct  5 19:06:22 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 05 Oct 2009 19:06:22 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050953p7158aabcx429f535ce7c674b8@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>	<loom.20091005T182820-734@post.gmane.org>
	<94bdd2610910050953p7158aabcx429f535ce7c674b8@mail.gmail.com>
Message-ID: <4ACA278E.9090505@egenix.com>

Tarek Ziad? wrote:
> On Mon, Oct 5, 2009 at 6:44 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> The only question is, given that 2.6.x is supposed to be a bug-fix branch, do we
>> want to fix that incompatibility with a widely deployed existing piece of
>> software? Whether or not the incompatibility is legitimate (i.e., whether
>> setuptools is badly written and relies on implementation details) is beyond the
>> point, IMO.
> 
> Right. As discussed with Barry, I will work on an internal special
> case in Distutils in the 2.6 maint.
> branch so it fixes the incompatibility with the Setuptools 0.6c9 release.

You could add a selective work-around that just triggers if setuptools
has been loaded on the 2.6 branch.

if sys.modules.has_key('setuptools'):
   # monkey-patch fix-up setuptools to work with the new code
   ...

or the other way around:

if sys.modules.has_key('setuptools'):
   # undo r1234 to make setuptools 0.6c9 happy again
   ...

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Mon Oct  5 19:08:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Oct 2009 19:08:07 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <4ACA278E.9090505@egenix.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<loom.20091005T182820-734@post.gmane.org>
	<94bdd2610910050953p7158aabcx429f535ce7c674b8@mail.gmail.com>
	<4ACA278E.9090505@egenix.com>
Message-ID: <94bdd2610910051008r2a4d5733ud3f5be4959c715a7@mail.gmail.com>

On Mon, Oct 5, 2009 at 7:06 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Tarek Ziad? wrote:
>> On Mon, Oct 5, 2009 at 6:44 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>
>>> The only question is, given that 2.6.x is supposed to be a bug-fix branch, do we
>>> want to fix that incompatibility with a widely deployed existing piece of
>>> software? Whether or not the incompatibility is legitimate (i.e., whether
>>> setuptools is badly written and relies on implementation details) is beyond the
>>> point, IMO.
>>
>> Right. As discussed with Barry, I will work on an internal special
>> case in Distutils in the 2.6 maint.
>> branch so it fixes the incompatibility with the Setuptools 0.6c9 release.
>
> You could add a selective work-around that just triggers if setuptools
> has been loaded on the 2.6 branch.
>
> if sys.modules.has_key('setuptools'):
> ? # monkey-patch fix-up setuptools to work with the new code
> ? ...
>
> or the other way around:
>
> if sys.modules.has_key('setuptools'):
> ? # undo r1234 to make setuptools 0.6c9 happy again

Yes I am working on this. I am putting setuptools specific build_ext
command into the distutils
tests to reproduce the issue, then will do a specific setuptools fix,


> ? ...
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source ?(#1, Oct 05 2009)
>>>> Python/Zope Consulting and Support ... ? ? ? ?http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ... ? ? ? ? ? ? http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ... ? ? ? ?http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
> ? eGenix.com Software, Skills and Services GmbH ?Pastor-Loeh-Str.48
> ? ?D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> ? ? ? ? ? Registered at Amtsgericht Duesseldorf: HRB 46611
> ? ? ? ? ? ? ? http://www.egenix.com/company/contact/
>



-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From jnoller at gmail.com  Mon Oct  5 19:38:06 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 5 Oct 2009 13:38:06 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <had53u$rrp$1@ger.gmane.org>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
Message-ID: <4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>

On Mon, Oct 5, 2009 at 12:00 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Jesse Noller schrieb:
>> On Mon, Oct 5, 2009 at 4:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
>>> it's revews like this that makes me wonder if releasing open source is
>>> a good idea:
>>>
>>> ? no egg - worst seen ever, remove it from pypi or provide an egg
>>> (jensens, 2009-10-05, 0 points)
>>>
>>> </F>
>>
>> Unfortunately; we're now staring down the barrel of having
>> youtube-style comments on Python packages on the index.
>
> Yes, unfortunately. I originally thought that restricting the commenters
> to those with a PyPI account would make them useful, but seeing this one
> (even if it was not intended) and the comment on hgsvn that belongs into
> a bug tracker instead, I'm not so sure anymore.
>
> Georg

There would need to be a fair amount of work to make the system useful
and almost self-policing. Not to mention people can make plenty of
fake pypi accounts for pure astroturfing reasons.

jesse

From jan.matejek at novell.com  Mon Oct  5 19:18:14 2009
From: jan.matejek at novell.com (Jan Matejek)
Date: Mon, 05 Oct 2009 19:18:14 +0200
Subject: [Python-Dev] please consider changing --enable-unicode	default
 to ucs4
In-Reply-To: <h95m1n$d57$2@ger.gmane.org>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200933t26ec258em21289fa8f22e81fc@mail.gmail.com>
	<h95m1n$d57$2@ger.gmane.org>
Message-ID: <4ACA2A56.7080501@novell.com>



Dne 20.9.2009 18:42, Antoine Pitrou napsal(a):
> Le Sun, 20 Sep 2009 10:33:23 -0600, Zooko O'Whielacronx a ?crit :
>>
>> By the way, I was investigating this, and discovered an issue on the
>> Mandriva tracker which suggests that they intend to switch to UCS4 in
>> the next release in order to avoid compatibility problems like these.
> 
> Trying to use a Fedora or Suse RPM under Mandriva (or the other way 
> round) isn't reasonable and is certainly not supported.
> I don't understand why this so-called "compatibility problem" should be 
> taken seriously by anyone.

You're not making sense. No distro is an island - plus, upstream
distributors have a nasty habit of providing RPMs only for Fedora. I
don't see what is bad about improving compatibility in a place where the
setting doesn't hurt one way or the other.
Besides, the more compatibility we achieve now, the easier time we'll
have once python makes it into LSB

regards
m.

> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jmatejek%40suse.cz

From guido at python.org  Mon Oct  5 19:54:22 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Oct 2009 10:54:22 -0700
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com> 
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com> 
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
Message-ID: <ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>

On Mon, Oct 5, 2009 at 10:38 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Mon, Oct 5, 2009 at 12:00 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Jesse Noller schrieb:
>>> On Mon, Oct 5, 2009 at 4:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
>>>> it's revews like this that makes me wonder if releasing open source is
>>>> a good idea:
>>>>
>>>> ? no egg - worst seen ever, remove it from pypi or provide an egg
>>>> (jensens, 2009-10-05, 0 points)
>>>>
>>>> </F>
>>>
>>> Unfortunately; we're now staring down the barrel of having
>>> youtube-style comments on Python packages on the index.
>>
>> Yes, unfortunately. I originally thought that restricting the commenters
>> to those with a PyPI account would make them useful, but seeing this one
>> (even if it was not intended) and the comment on hgsvn that belongs into
>> a bug tracker instead, I'm not so sure anymore.
>>
>> Georg
>
> There would need to be a fair amount of work to make the system useful
> and almost self-policing. Not to mention people can make plenty of
> fake pypi accounts for pure astroturfing reasons.

It seems like a worthy cause though. User ratings and comments are the
future for "app store" style sites such as PyPI, and spam
unfortunately comes with the terrain. There are plenty of things we
can learn about fighting spam and other forms of vandalism from other
areas of the social web, including our very own wiki, and other wikis
(WikiPedia survives despite spam).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake at gmail.com  Mon Oct  5 19:58:17 2009
From: fdrake at gmail.com (Fred Drake)
Date: Mon, 5 Oct 2009 13:58:17 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com> 
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com> 
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com> 
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
Message-ID: <9cee7ab80910051058t36b83bbck2ad366536d83063c@mail.gmail.com>

On Mon, Oct 5, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:
> User ratings and comments are the
> future for "app store" style sites such as PyPI

Interestingly, I consider sites like PyPI as developer resources
rather than the more end-user-centric "App Store" sites.

While I don't consider it bad to provide support for commenting and
rating, I find it much less valuable than for end-user-centric sites.
But then, some may find me a bit retro.  :-/


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From benjamin at python.org  Sat Oct  3 23:39:20 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 3 Oct 2009 16:39:20 -0500
Subject: [Python-Dev] % to .format() converter
Message-ID: <1afaf6160910031439q544ffc3bw82e3300ce0d03e66@mail.gmail.com>

Interested in the recent threads about % and str.format(), I wrote a
little Python 3 script that converts % format strings to format
strings compatible with str.format(). I invite you to try it out. The
code is at https://code.launchpad.net/~gutworth/+junk/mod2format (That
means you can do "bzr branch lp:~gutworth/+junk/mod2format".) It
should basically work except for some obscure cases like "%#.0f".

-- 
Regards,
Benjamin

From doug.hellmann at gmail.com  Mon Oct  5 13:58:27 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 5 Oct 2009 07:58:27 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <87hbuew70i.fsf@benfinney.id.au>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<87hbuew70i.fsf@benfinney.id.au>
Message-ID: <5EB6ECE5-8C3F-4C7E-8922-15EE2B9D4D56@gmail.com>


On Oct 5, 2009, at 4:59 AM, Ben Finney wrote:

> Fredrik Lundh <fredrik at pythonware.com> writes:
>
>> it's revews like this that makes me wonder if releasing open source  
>> is
>> a good idea:
>>
>>   no egg - worst seen ever, remove it from pypi or provide an egg
>> (jensens, 2009-10-05, 0 points)
>
> Heh. If harsh, uninformed responses make you wonder whether it's worth
> doing free software, then you're in the wrong social philanthropy
> movement.
>
> Make the software good quality, release it in the standard format (in
> this case, a Distutils ?sdist? on PyPI), respond in a timely manner to
> genuine requests for help. Eggs are certainly not a requirement, nor  
> is
> responding to hostility like the above comment.
>
> If you *want* to respond, you can politely direct them to
> <URL:http://docs.python.org/install/> where they can learn how to
> install Python distributions ? no mention of eggs at all.

Package owners are not allowed to comment on their own PyPI packages,  
so responding to comments requires tracking down the person who left  
them and contacting them in some other way.

Doug


From solipsis at pitrou.net  Mon Oct  5 20:09:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 05 Oct 2009 20:09:47 +0200
Subject: [Python-Dev] please consider changing --enable-unicode	default
 to ucs4
In-Reply-To: <4ACA2A56.7080501@novell.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200933t26ec258em21289fa8f22e81fc@mail.gmail.com>
	<h95m1n$d57$2@ger.gmane.org>  <4ACA2A56.7080501@novell.com>
Message-ID: <1254766187.6226.41.camel@localhost>

Le lundi 05 octobre 2009 ? 19:18 +0200, Jan Matejek a ?crit :
> 
> I
> don't see what is bad about improving compatibility in a place where the
> setting doesn't hurt one way or the other.

I can't speak for Mandriva, but I'm sure they care more about not
breaking user installs when they upgrade to Mandriva X + 1, than about
making it possible to use Fedora RMPs on Mandriva.

In any case, this is quite off-topic for python-dev. If you are
motivated about this, you should try to convinve the Mandriva developers
instead.

Regards

Antoine.



From jnoller at gmail.com  Mon Oct  5 20:21:41 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 5 Oct 2009 14:21:41 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
Message-ID: <4222a8490910051121g95538b8n4f24591dd819d9c0@mail.gmail.com>

On Mon, Oct 5, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Oct 5, 2009 at 10:38 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> On Mon, Oct 5, 2009 at 12:00 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>>> Jesse Noller schrieb:
>>>> On Mon, Oct 5, 2009 at 4:43 AM, Fredrik Lundh <fredrik at pythonware.com> wrote:
>>>>> it's revews like this that makes me wonder if releasing open source is
>>>>> a good idea:
>>>>>
>>>>> ? no egg - worst seen ever, remove it from pypi or provide an egg
>>>>> (jensens, 2009-10-05, 0 points)
>>>>>
>>>>> </F>
>>>>
>>>> Unfortunately; we're now staring down the barrel of having
>>>> youtube-style comments on Python packages on the index.
>>>
>>> Yes, unfortunately. I originally thought that restricting the commenters
>>> to those with a PyPI account would make them useful, but seeing this one
>>> (even if it was not intended) and the comment on hgsvn that belongs into
>>> a bug tracker instead, I'm not so sure anymore.
>>>
>>> Georg
>>
>> There would need to be a fair amount of work to make the system useful
>> and almost self-policing. Not to mention people can make plenty of
>> fake pypi accounts for pure astroturfing reasons.
>
> It seems like a worthy cause though. User ratings and comments are the
> future for "app store" style sites such as PyPI, and spam
> unfortunately comes with the terrain. There are plenty of things we
> can learn about fighting spam and other forms of vandalism from other
> areas of the social web, including our very own wiki, and other wikis
> (WikiPedia survives despite spam).
>

I agree that feedback, commentary/etc is a Good Thing; but doing it
right is not an easy thing, and typically implementing it poorly leads
to spam, people filing bugs in comments, astroturfing, etc. Just on
first glance, I could see immediate improvements around:

* Moderation
* Allowing authors to respond
* Flagging as spam
* Upvoting/downvoting
* Nested conversations

And so on. Sites like stackoverflow/reddit/hackernews/etc have spent a
lot of time "doing it right".

I know, I know - patches welcome. The problem here is that I would
make an argument that in the case of PyPI nothing is better than what
we have currently.

jesse

From solipsis at pitrou.net  Mon Oct  5 20:20:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 5 Oct 2009 18:20:58 +0000 (UTC)
Subject: [Python-Dev] eggs now mandatory for pypi?
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
Message-ID: <loom.20091005T202019-122@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> There are plenty of things we
> can learn about fighting spam and other forms of vandalism from other
> areas of the social web, including our very own wiki, and other wikis
> (WikiPedia survives despite spam).

Doesn't Wikipedia have a lot of human eyes watching, however?



From olemis at gmail.com  Mon Oct  5 20:38:14 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 5 Oct 2009 13:38:14 -0500
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4222a8490910051121g95538b8n4f24591dd819d9c0@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<4222a8490910051121g95538b8n4f24591dd819d9c0@mail.gmail.com>
Message-ID: <24ea26600910051138m5c014523gf48b61be7c1f8fc3@mail.gmail.com>

On Mon, Oct 5, 2009 at 1:21 PM, Jesse Noller <jnoller at gmail.com> wrote:
> On Mon, Oct 5, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:
>>
[...]
>>
>> User ratings and comments are the
>> future for "app store" style sites such as PyPI, and spam
>> unfortunately comes with the terrain. There are plenty of things we
>> can learn about fighting spam and other forms of vandalism from other
>> areas of the social web, including our very own wiki, and other wikis
>> (WikiPedia survives despite spam).
>>
>
> I agree that feedback, commentary/etc is a Good Thing; but doing it
> right is not an easy thing, and typically implementing it poorly leads
> to spam, people filing bugs in comments, astroturfing, etc. Just on
> first glance, I could see immediate improvements around:
>
> * Moderation
[...]
> * Flagging as spam

* Captcha ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Nabble - Trac Users - Coupling trac and symfony framework  -
http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html

From pje at telecommunity.com  Mon Oct  5 20:48:37 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 05 Oct 2009 14:48:37 -0400
Subject: [Python-Dev] "PIL" vs. "Imaging" (was Re: eggs now mandatory for
 pypi?)
In-Reply-To: <1254741989.7368.22.camel@minime>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
	<4AC9D356.5050202@cheimes.de> <1254741989.7368.22.camel@minime>
Message-ID: <20091005184844.647633A4045@sparrow.telecommunity.com>

At 01:26 PM 10/5/2009 +0200, Jens W. Klein wrote:
>And as far as I understand PIL is not the
>problem, but the packaging/ setuptools. For the records: PIL is a great
>piece of software and I dont want to miss it.

Actually, the problem is that the package is called "PIL" in Pypi, 
but the actual installed package is called "Imaging".  If you want to 
depend on PIL, you need to add "Imaging" to your dependencies, and 
add a dependency link to http://effbot.org/downloads/ in order to 
make it work...  at least on platforms where you have a compiler.  If 
you're working on Windows, you need to depend on PIL in order to get 
a download to work, but once installed, it's still called 
Imaging.  So, pretty much you're hosed either way.  ;-)

If the package had just *one* name, used consistently for source, 
binaries, and PyPI registration, it would work.  Unfortunately, 
setuptools is not a human being and can't figure out that "PIL" and 
"Imaging" are two different names for the same thing.  (I'm guessing 
that PIL was registered on PyPI manually, before the "setup.py 
register" command existed.  Heck, it was probably being distributed 
before the distutils even existed, and indeed before there were such 
things as "packages" in Python.)

When I was first writing setuptools, I tried to get it to work with 
effbot.org packages as best I could, but there were some things I 
just couldn't do without hardcoding special cases, and working around 
this problem was one of them.


From pje at telecommunity.com  Mon Oct  5 20:52:49 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 05 Oct 2009 14:52:49 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.co
 m>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
Message-ID: <20091005185255.D4EA53A408F@sparrow.telecommunity.com>

At 02:22 PM 10/5/2009 +0200, Tarek Ziad? wrote:
>Setuptools development has been discontinued for a year, and does
>patches on Distutils code. Some of these patches are sensitive to any change
>made on Distutils, wether those changes are internal or not.

Setuptools is  also not the only thing out there that subclasses 
distutils commands in general, or the build_ext command in 
particular.  Numpy, Twisted, the mx extensions...  there are plenty 
of things out there that subclass distutils commands, quite in 
adherence to the rules.  (Note too that subclassing != patching, and 
the ability to subclass and substitute distutils commands is documented.)

It's therefore not appropriate to treat the issue as if it were 
setuptools-specific; it could have broken any other major (or minor) 
package's subclassing of the build_ext command.


From pje at telecommunity.com  Mon Oct  5 20:55:21 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 05 Oct 2009 14:55:21 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.co
 m>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
Message-ID: <20091005185528.530E53A408F@sparrow.telecommunity.com>

At 04:17 PM 10/5/2009 +0200, Tarek Ziad? wrote:
>Now I am astonished that we are talking about reverting changes in
>Distutils that were done for bugfixes, for a third party package that
>does monkey patches on Distutils.

Subclassing is not monkeypatching, so if you consider the above to be 
a general policy, then the developers of at least Numpy and Twisted 
should consider themselves warned that bugfix releases may break 
their build processes.


From fredrik at pythonware.com  Mon Oct  5 21:03:38 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 5 Oct 2009 21:03:38 +0200
Subject: [Python-Dev] "PIL" vs. "Imaging" (was Re: eggs now mandatory
	for pypi?)
In-Reply-To: <20091005184844.647633A4045@sparrow.telecommunity.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com> 
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com> 
	<4AC9D356.5050202@cheimes.de> <1254741989.7368.22.camel@minime> 
	<20091005184844.647633A4045@sparrow.telecommunity.com>
Message-ID: <368a5cd50910051203g7b71140ep6b26e3e66e13a2d@mail.gmail.com>

On Mon, Oct 5, 2009 at 8:48 PM, P.J. Eby <pje at telecommunity.com> wrote:

> names for the same thing. ?(I'm guessing that PIL was registered on PyPI
> manually, before the "setup.py register" command existed. ?Heck, it was
> probably being distributed before the distutils even existed, and indeed
> before there were such things as "packages" in Python.)

$ more PIL/Image.py
...
# partial release history:
# 1995-09-09 fl   Created
# 1996-03-11 fl   PIL release 0.0
# 1996-04-30 fl   PIL release 0.1b1
...

Looks like the quickest fix is to distribute two source packages, one
built the traditional way to eliminate breakage for people using the
current tarball layout, and one built with "sdist".

</F>

From brett at python.org  Mon Oct  5 20:21:01 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 5 Oct 2009 11:21:01 -0700
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com> 
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com> 
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com> 
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
Message-ID: <bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>

On Mon, Oct 5, 2009 at 08:44, Barry Warsaw <barry at python.org> wrote:

> On Oct 5, 2009, at 10:50 AM, Tarek Ziad? wrote:
>
>  If, as I hope, the answer to that is "yes", then I strongly support
>>> releasing a fixed setuptools instead of reverting the change to Python.
>>>
>>
>> Yes it does.
>>
>
> Excellent, thanks.
>
>  The fix makes sure build_ext.get_ext_filename still works as it is
>> supposed
>> (just returning the extension of a filename depending on the
>> environment) under all versions,
>>
>> *and* continues to do extra work for the Setuptools internal needs when
>> it is called by Setuptools itself on various places.
>>
>> It's two lines in Setuptools.
>>
>> But beware that a new Setuptools release might take a few months and
>> maybe will never happen.
>> That is why a suggested solution was to install Distribute, because it
>> addresses among other
>> bug fixes, this one.
>>
>
> I think we have two other choices.
>
> 2) PJE releases a new version of setuptools that fixes this problem.
>
> 3) We (a.k.a. Tarek with our blessing) hijacks the setuptools name (e.g. on
> cheeseshop) and releases a new version
>
> I still hope #2 happens, but let's have a deadline for when the more
> drastic measures will be taken.


Or we VERY clearly and loudly state in the release notes and anywhere else
we can that Distribute has replaced setuptools and everyone should upgrade
immediately because of critical bugs like the one under discussion.

I should also mention this bug was not unknown. I discovered it after
Distribute 0.6 was released as I always run cutting edge interpreters. Never
bothered to report it until Distribute 0.6.1 was released which Tarek fixed
in less than a week. I never bothered to report it for setuptools as I know
it isn't maintained.

It's probably in our best interest to just get people over to Distribute,
let it continue to hijack setuptools, and slowly let that name fade out if
it is going to continue to be unmaintained. I have to admit I find it really
disheartening that we are letting an unmaintained project dictate how we fix
a bug. I really hope this is a one-time deal and from this point forward we
all move the community towards Distribute so we never feel pressured like
this again.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/f6070328/attachment.htm>

From fuzzyman at voidspace.org.uk  Mon Oct  5 21:22:40 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 05 Oct 2009 20:22:40 +0100
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
	<bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>
Message-ID: <4ACA4780.2050404@voidspace.org.uk>

Brett Cannon wrote:
>
>
> On Mon, Oct 5, 2009 at 08:44, Barry Warsaw <barry at python.org 
> <mailto:barry at python.org>> wrote:
>
>     On Oct 5, 2009, at 10:50 AM, Tarek Ziad? wrote:
>
>             If, as I hope, the answer to that is "yes", then I
>             strongly support
>             releasing a fixed setuptools instead of reverting the
>             change to Python.
>
>
>         Yes it does.
>
>
>     Excellent, thanks.
>
>
>         The fix makes sure build_ext.get_ext_filename still works as
>         it is supposed
>         (just returning the extension of a filename depending on the
>         environment) under all versions,
>
>         *and* continues to do extra work for the Setuptools internal
>         needs when
>         it is called by Setuptools itself on various places.
>
>         It's two lines in Setuptools.
>
>         But beware that a new Setuptools release might take a few
>         months and
>         maybe will never happen.
>         That is why a suggested solution was to install Distribute,
>         because it
>         addresses among other
>         bug fixes, this one.
>
>
>     I think we have two other choices.
>
>     2) PJE releases a new version of setuptools that fixes this problem.
>
>     3) We (a.k.a. Tarek with our blessing) hijacks the setuptools name
>     (e.g. on cheeseshop) and releases a new version
>
>     I still hope #2 happens, but let's have a deadline for when the
>     more drastic measures will be taken.
>
>
> Or we VERY clearly and loudly state in the release notes and anywhere 
> else we can that Distribute has replaced setuptools and everyone 
> should upgrade immediately because of critical bugs like the one under 
> discussion.

+1 - this sounds like a good solution.

I'm intrigued as to whether setuptools does monkey patch distutils or 
subclass though, they are fundamentally different concepts.

Michael

>
> I should also mention this bug was not unknown. I discovered it after 
> Distribute 0.6 was released as I always run cutting edge interpreters. 
> Never bothered to report it until Distribute 0.6.1 was released which 
> Tarek fixed in less than a week. I never bothered to report it for 
> setuptools as I know it isn't maintained.
>
> It's probably in our best interest to just get people over to 
> Distribute, let it continue to hijack setuptools, and slowly let that 
> name fade out if it is going to continue to be unmaintained. I have to 
> admit I find it really disheartening that we are letting an 
> unmaintained project dictate how we fix a bug. I really hope this is a 
> one-time deal and from this point forward we all move the community 
> towards Distribute so we never feel pressured like this again.
>
> -Brett
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From barry at python.org  Mon Oct  5 22:19:50 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Oct 2009 16:19:50 -0400
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <4AC858B9.605@gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>
	<4AC858B9.605@gmail.com>
Message-ID: <2F47507B-B9FB-4556-8755-8B561ECB293E@python.org>

On Oct 4, 2009, at 4:11 AM, Nick Coghlan wrote:

> Barry Warsaw wrote:
>> I also don't think this is a case of anti-TOOWTDI.  For most  
>> situations
>> {}-strings are great (IMO), but in the specific translation domain, I
>> suspect $-strings are still better.
>
> I agree that keeping string.Template around is valid due to its  
> focus on
> being very simple to use (especially for non-coders producing
> translation templates)
>
> However, string.Template is already set up to work for the relevant  
> APIs
> in that domain so I don't think it will really be affected by any
> changes to the underlying language and library support for brace  
> formatting.

What if you wanted to translate log messages?  Printing to a StringIO  
first really isn't a very good solution.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/b2b04cc7/attachment.pgp>

From pje at telecommunity.com  Mon Oct  5 22:29:52 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 05 Oct 2009 16:29:52 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.co
 m>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
	<bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>
Message-ID: <20091005202959.757973A4045@sparrow.telecommunity.com>

At 11:21 AM 10/5/2009 -0700, Brett Cannon wrote:
>I have to admit I find it really disheartening that we are letting 
>an unmaintained project dictate how we fix a bug.

Setuptools was not the only project broken by this change.  See:

   http://bugs.python.org/issue7020#msg93355

Apparently, I'm not the only person who read the get_ext_filename() 
docstring that way, and subclassed build_ext in a way that broke 
it.  (In this case, it was pywin32.)

IMO, the fact that it broke Pywin32 in 2.6.3rc1 should have been 
considered evidence that the change was not suitable for a bugfix release.


From pje at telecommunity.com  Mon Oct  5 22:31:52 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 05 Oct 2009 16:31:52 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <4ACA4780.2050404@voidspace.org.uk>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
	<bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>
	<4ACA4780.2050404@voidspace.org.uk>
Message-ID: <20091005203159.AB8C43A4045@sparrow.telecommunity.com>

At 08:22 PM 10/5/2009 +0100, Michael Foord wrote:
>I'm intrigued as to whether setuptools does monkey patch distutils 
>or subclass though, they are fundamentally different concepts.

It does both; however, the specific issue under discussion is purely 
a subclassing problem, and one which pywin32 ran afoul of as well.


From ncoghlan at gmail.com  Mon Oct  5 22:26:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 06 Oct 2009 06:26:20 +1000
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <5EB6ECE5-8C3F-4C7E-8922-15EE2B9D4D56@gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>	<87hbuew70i.fsf@benfinney.id.au>
	<5EB6ECE5-8C3F-4C7E-8922-15EE2B9D4D56@gmail.com>
Message-ID: <4ACA566C.4070902@gmail.com>

Doug Hellmann wrote:
> On Oct 5, 2009, at 4:59 AM, Ben Finney wrote:
>> If you *want* to respond, you can politely direct them to
>> <URL:http://docs.python.org/install/> where they can learn how to
>> install Python distributions ? no mention of eggs at all.
> 
> Package owners are not allowed to comment on their own PyPI packages, so
> responding to comments requires tracking down the person who left them
> and contacting them in some other way.

Really? I can understand package owners not being able to add
recommendations for their own packages, but if they can't add comments
how are they meant to correct misunderstandings or redirect
inappropriate messages to the correct forums?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Oct  5 22:24:39 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 06 Oct 2009 06:24:39 +1000
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <9cee7ab80910051058t36b83bbck2ad366536d83063c@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<9cee7ab80910051058t36b83bbck2ad366536d83063c@mail.gmail.com>
Message-ID: <4ACA5607.8060100@gmail.com>

Fred Drake wrote:
> On Mon, Oct 5, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:
>> User ratings and comments are the
>> future for "app store" style sites such as PyPI
> 
> Interestingly, I consider sites like PyPI as developer resources
> rather than the more end-user-centric "App Store" sites.
> 
> While I don't consider it bad to provide support for commenting and
> rating, I find it much less valuable than for end-user-centric sites.
> But then, some may find me a bit retro.  :-/

When it comes to comments and recommendations for selecting software
packages, developers *are* the end users :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Oct  5 22:41:25 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 06 Oct 2009 06:41:25 +1000
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <2F47507B-B9FB-4556-8755-8B561ECB293E@python.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>
	<4AC858B9.605@gmail.com>
	<2F47507B-B9FB-4556-8755-8B561ECB293E@python.org>
Message-ID: <4ACA59F5.1080103@gmail.com>

Barry Warsaw wrote:
> On Oct 4, 2009, at 4:11 AM, Nick Coghlan wrote:
> 
>> Barry Warsaw wrote:
>>> I also don't think this is a case of anti-TOOWTDI.  For most situations
>>> {}-strings are great (IMO), but in the specific translation domain, I
>>> suspect $-strings are still better.
>>
>> I agree that keeping string.Template around is valid due to its focus on
>> being very simple to use (especially for non-coders producing
>> translation templates)
>>
>> However, string.Template is already set up to work for the relevant APIs
>> in that domain so I don't think it will really be affected by any
>> changes to the underlying language and library support for brace
>> formatting.
> 
> What if you wanted to translate log messages?  Printing to a StringIO
> first really isn't a very good solution.

Oh, I see what you meant now - you were pointing out that lazy
formatting APIs (such as logging) already don't work properly for
alternative formatting mechanisms (such as string.Template).

(Although printing to a String IO doesn't seem necessary - simply
pre-formatting the message seems like it should work, just as
preformatting brace formatted messages already works today).

The idea of adding an optional "fmt_style" callable to these APIs
(defaulting to None, which would imply the use of the mod operator) is
becoming more appealing...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Oct  5 22:20:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 06 Oct 2009 06:20:47 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ca471dc20910050828o2ed717fbn6434a9c319ef5b15@mail.gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
	<ca471dc20910050828o2ed717fbn6434a9c319ef5b15@mail.gmail.com>
Message-ID: <4ACA551F.9090309@gmail.com>

Guido van Rossum wrote:
> \>> I hate calling methods on string literals, I think it looks very odd
>>> to have code like this:
>>>
>>>   "Displaying {0} of {1} revisions".format(x, y)
>> Ugh!  Good point.
> 
> This objection was made years ago when we introduced
> "separator".join(list_of_things), and I don't think ignoring it has
> caused any casualties. In fact, it serves as an early reminder to the
> user that string literals are, in fact, objects like all others.

The other string literal method I personally use reasonably often is to
create lists of strings by using "a list of strings".split() instead of
['a', 'list', 'of', 'strings'] (while the character count isn't all that
different, I find the former is easier to both write and read without
all those quotes and commas). It's a trick I've seen plenty of other
people use as well.

So I would agree that method invocation on literals (particularly string
literals) is an already established language idiom.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From foom at fuhm.net  Mon Oct  5 22:56:16 2009
From: foom at fuhm.net (James Y Knight)
Date: Mon, 5 Oct 2009 16:56:16 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>
	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
	<bbaeab100910051121s4ca4fe8fy9af9211bbff05a0c@mail.gmail.com>
Message-ID: <8EFA8DDA-BC90-4248-9CF3-42F7CF33301B@fuhm.net>

On Oct 5, 2009, at 2:21 PM, Brett Cannon wrote:
> I should also mention this bug was not unknown. I discovered it  
> after Distribute 0.6 was released as I always run cutting edge  
> interpreters. Never bothered to report it until Distribute 0.6.1 was  
> released which Tarek fixed in less than a week. I never bothered to  
> report it for setuptools as I know it isn't maintained.
>
> It's probably in our best interest to just get people over to  
> Distribute, let it continue to hijack setuptools, and slowly let  
> that name fade out if it is going to continue to be unmaintained. I  
> have to admit I find it really disheartening that we are letting an  
> unmaintained project dictate how we fix a bug. I really hope this is  
> a one-time deal and from this point forward we all move the  
> community towards Distribute so we never feel pressured like this  
> again.

Even though the bug was noticed, nobody thought that, just perhaps,  
breaking other software in a minor point release might be a bad idea,  
no matter whether it was updated in less-than-a-week, or mostly- 
unmaintained?

Once you have an API that you encourage people to subclass, *of  
course* it dictates how you can fix a bug.

James

From barry at python.org  Mon Oct  5 23:01:15 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Oct 2009 17:01:15 -0400
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <4ACA59F5.1080103@gmail.com>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>
	<4AC858B9.605@gmail.com>
	<2F47507B-B9FB-4556-8755-8B561ECB293E@python.org>
	<4ACA59F5.1080103@gmail.com>
Message-ID: <0B84287E-F693-4399-9C6D-BDFE7F1AA4C5@python.org>

On Oct 5, 2009, at 4:41 PM, Nick Coghlan wrote:

> Oh, I see what you meant now - you were pointing out that lazy
> formatting APIs (such as logging) already don't work properly for
> alternative formatting mechanisms (such as string.Template).

Yep.

> (Although printing to a String IO doesn't seem necessary - simply
> pre-formatting the message seems like it should work, just as
> preformatting brace formatted messages already works today).

Yep. :)

> The idea of adding an optional "fmt_style" callable to these APIs
> (defaulting to None, which would imply the use of the mod operator) is
> becoming more appealing...

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091005/7fbff8bb/attachment.pgp>

From fdrake at gmail.com  Mon Oct  5 23:06:50 2009
From: fdrake at gmail.com (Fred Drake)
Date: Mon, 5 Oct 2009 17:06:50 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACA5607.8060100@gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com> 
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com> 
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com> 
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com> 
	<9cee7ab80910051058t36b83bbck2ad366536d83063c@mail.gmail.com> 
	<4ACA5607.8060100@gmail.com>
Message-ID: <9cee7ab80910051406j7397138du834084704e52fdfb@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> When it comes to comments and recommendations for selecting software
> packages, developers *are* the end users :)

Yes, most certainly.  But developers as consumers are very different
from application users as consumers, which is what I was getting at.

The convenience interfaces for commenting on a library are far less
valuable for developers, IMO, since developers are expected to better
understand how their context impacts their perception.  Useful
feedback from a developer just doesn't fit will into the
giant-pile-of-comments UIs conventional for non-developers.

If I'm wrong about that, then I'm saddened by the state of the profession.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From martin at v.loewis.de  Mon Oct  5 23:22:22 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 05 Oct 2009 23:22:22 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>
	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
Message-ID: <4ACA638E.30609@v.loewis.de>

>> My question was less about the political aspects than the technical aspects.
>>  I gather you're saying that the fix to setuptools will make it work in
>> 2.6.3 without inadvertently breaking it for 2.6.2, 2.6.1, and 2.6.0, right?
>>  Have you tried the fix in those older versions to be sure?
>>
>> If, as I hope, the answer to that is "yes", then I strongly support
>> releasing a fixed setuptools instead of reverting the change to Python.
> 
> Yes it does.
> 
> The fix makes sure build_ext.get_ext_filename still works as it is supposed
> (just returning the extension of a filename depending on the
> environment) under all versions,

I don't think this is factually correct. I assume you talk about
distribute change e07e5309cd2a, right?

While that change arranges to always return a result when called with an
unqualified module name, it doesn't do the additional computation that
setuptools does in that case, which may give something different in case
the extension is a Library or in case _links_to_dynamic is true.

So I would rather expect that distribute is now broken in these cases
(which are seemingly rare, though).

> It's two lines in Setuptools.

But that change may be incorrect.

Regards,
Martin

From olemis at gmail.com  Mon Oct  5 23:23:50 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 5 Oct 2009 16:23:50 -0500
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <9cee7ab80910051406j7397138du834084704e52fdfb@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<9cee7ab80910051058t36b83bbck2ad366536d83063c@mail.gmail.com>
	<4ACA5607.8060100@gmail.com>
	<9cee7ab80910051406j7397138du834084704e52fdfb@mail.gmail.com>
Message-ID: <24ea26600910051423y570d91drabcbe7b58364f141@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:06 PM, Fred Drake <fdrake at gmail.com> wrote:
> On Mon, Oct 5, 2009 at 4:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> When it comes to comments and recommendations for selecting software
>> packages, developers *are* the end users :)
>
> Yes, most certainly. ?But developers as consumers are very different
> from application users as consumers, which is what I was getting at.
>
> The convenience interfaces for commenting on a library are far less
> valuable for developers, IMO, since developers are expected to better
> understand how their context impacts their perception. ?Useful
> feedback from a developer just doesn't fit will into the
> giant-pile-of-comments UIs conventional for non-developers.
>

+1

IMO :

  - decision matrix are useful to decide which lib to use (i.e. which
    one supports the features I need ;o). BTW that's something cool about
    wikipedia ;o)
  - project metrics and build results are useful to have a idea
    of project dev status (e.g. coverage, test results, ...).
  - the rest goes to issue tracker + project (sites | wikis). that's what
    they are for ;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Nabble - Trac Users - Coupling trac and symfony framework  -
http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html

From olemis at gmail.com  Mon Oct  5 23:30:53 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 5 Oct 2009 16:30:53 -0500
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <24ea26600910051423y570d91drabcbe7b58364f141@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<9cee7ab80910051058t36b83bbck2ad366536d83063c@mail.gmail.com>
	<4ACA5607.8060100@gmail.com>
	<9cee7ab80910051406j7397138du834084704e52fdfb@mail.gmail.com>
	<24ea26600910051423y570d91drabcbe7b58364f141@mail.gmail.com>
Message-ID: <24ea26600910051430h3b2d3185lb93401b0a702276f@mail.gmail.com>

On Mon, Oct 5, 2009 at 4:23 PM, Olemis Lang <olemis at gmail.com> wrote:
> On Mon, Oct 5, 2009 at 4:06 PM, Fred Drake <fdrake at gmail.com> wrote:
>> On Mon, Oct 5, 2009 at 4:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> When it comes to comments and recommendations for selecting software
>>> packages, developers *are* the end users :)
>>
>> Yes, most certainly. ?But developers as consumers are very different
>> from application users as consumers, which is what I was getting at.
>>
>> The convenience interfaces for commenting on a library are far less
>> valuable for developers, IMO, since developers are expected to better
>> understand how their context impacts their perception. ?Useful
>> feedback from a developer just doesn't fit will into the
>> giant-pile-of-comments UIs conventional for non-developers.
>>
>
> +1
>
> IMO :
>
> ?- decision matrix are useful to decide which lib to use (i.e. which
> ? ?one supports the features I need ;o). BTW that's something cool about
> ? ?wikipedia ;o)

I mean feature matrix

> ?- project metrics and build results are useful to have a idea
> ? ?of project dev status (e.g. coverage, test results, ...).
> ?- the rest goes to issue tracker + project (sites | wikis). that's what
> ? ?they are for ;o)
>
> --
> Regards,
>
> Olemis.
>
> Blog ES: http://simelo-es.blogspot.com/
> Blog EN: http://simelo-en.blogspot.com/
>
> Featured article:
> Nabble - Trac Users - Coupling trac and symfony framework ?-
> http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html
>



-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Nabble - Trac Users - Coupling trac and symfony framework  -
http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html

From vinay_sajip at yahoo.co.uk  Mon Oct  5 23:46:52 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 5 Oct 2009 21:46:52 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?summary_of_transitioning_from_=25_to_=7B?=
	=?utf-8?q?=7D_formatting?=
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>
	<4AC858B9.605@gmail.com>
	<2F47507B-B9FB-4556-8755-8B561ECB293E@python.org>
	<4ACA59F5.1080103@gmail.com>
Message-ID: <loom.20091005T233721-264@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Oh, I see what you meant now - you were pointing out that lazy
> formatting APIs (such as logging) already don't work properly for
> alternative formatting mechanisms (such as string.Template).
> 

Logging doesn't work automatically with string.Template as it pre-dates
string.Template, but it can be made to work without too much trouble:

(a) Subclass Formatter to use $-formatting instead of %-formatting for e.g. the
lines written to log files, and
(b) Use a class like DollarMessage which I mentioned elsewhere in this thread,
to format logging call format strings and arguments.

Both of these approaches will also work for {}-formatting. The present thread
really started out with a view to suggesting that the stdlib start adopting
{}-format as "native", rather than %-format.

Would it be helpful if I added a section to the Python docs about how to use
alternative formatting systems with logging?

Regards,

Vinay Sajip



From benjamin at python.org  Mon Oct  5 04:07:52 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 4 Oct 2009 21:07:52 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ee9557410910041907g14322e24t934ff7f37b3aa95c@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<87y6nt37kb.fsf@hbox.dyndns.org>
	<426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
	<ee9557410910041852i15f15471y36648c224f5ef482@mail.gmail.com>
	<1afaf6160910041854v55edcc96we4fb2d6a7f2da948@mail.gmail.com>
	<ee9557410910041907g14322e24t934ff7f37b3aa95c@mail.gmail.com>
Message-ID: <1afaf6160910041907m7dc6b82ftaa08506cad9c67c2@mail.gmail.com>

2009/10/4 INADA Naoki <songofacandy at gmail.com>:
>> -1 That requires keeping formatting information around in every string instance.
> Adding new "format_string" class avoids it.
> ?unicode("foo") <=> u"foo"
> ?format_string("foo") <=> f"foo"
>
> This way's pros:
> * Many libraries can use one transition way.
> * Transition stage syncs to Python version. "library A uses {} and
> library B uses %" problem not happen in transition.
> * We have experience same way on unicode.

And all are outweighted by the introduction of a new string class.



-- 
Regards,
Benjamin

From martin at v.loewis.de  Tue Oct  6 00:10:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 06 Oct 2009 00:10:25 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <24ea26600910051138m5c014523gf48b61be7c1f8fc3@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>	<had53u$rrp$1@ger.gmane.org>	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>	<4222a8490910051121g95538b8n4f24591dd819d9c0@mail.gmail.com>
	<24ea26600910051138m5c014523gf48b61be7c1f8fc3@mail.gmail.com>
Message-ID: <4ACA6ED1.5020302@v.loewis.de>

>> * Moderation
> [...]
>> * Flagging as spam
> 
> * Captcha ?

In the specific case, neither would have helped.

a) the user making the comment that the package author felt to be
impolite was a real user, with an established, well-known identity,
and long-time member of the community (IIUC). That didn't stop him
from making comments that the package author considered as impolite.

b) the comment itself was not spam, by any convention that I'm familiar
with. It was negative, but it was on-topic (in that it commented on
the very package that it was attached to).

I have said this already in various other places: I feel that the
freedom of speech is of high value, and that PyPI is not only (and
perhaps not primarily) made for the package authors - but for the
package users. If we get spam, we certainly need (and have)
moderation. But in the specific case, I probably would not have
moderated the comment away since it was not spam.

Regards,
Martin

From benjamin at python.org  Mon Oct  5 22:57:15 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 5 Oct 2009 15:57:15 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACA551F.9090309@gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>
	<ca471dc20910050828o2ed717fbn6434a9c319ef5b15@mail.gmail.com>
	<4ACA551F.9090309@gmail.com>
Message-ID: <1afaf6160910051357t32301c3aq593ac1745c807290@mail.gmail.com>

2009/10/5 Nick Coghlan <ncoghlan at gmail.com>:
> So I would agree that method invocation on literals (particularly string
> literals) is an already established language idiom.

And who hasn't ever used 4.56.as_integer_ratio()? :)



-- 
Regards,
Benjamin

From fuzzyman at voidspace.org.uk  Tue Oct  6 00:23:39 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 05 Oct 2009 23:23:39 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <1afaf6160910051357t32301c3aq593ac1745c807290@mail.gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>	<ca471dc20910050828o2ed717fbn6434a9c319ef5b15@mail.gmail.com>	<4ACA551F.9090309@gmail.com>
	<1afaf6160910051357t32301c3aq593ac1745c807290@mail.gmail.com>
Message-ID: <4ACA71EB.4050003@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/10/5 Nick Coghlan <ncoghlan at gmail.com>:
>   
>> So I would agree that method invocation on literals (particularly string
>> literals) is an already established language idiom.
>>     
>
> And who hasn't ever used 4.56.as_integer_ratio()? :)
>
>
>
>   
I've tried 4.__add__ a few times (not for a while now though).

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ncoghlan at gmail.com  Sun Oct  4 10:11:37 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 04 Oct 2009 18:11:37 +1000
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>
	<05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>
Message-ID: <4AC858B9.605@gmail.com>

Barry Warsaw wrote:
> I also don't think this is a case of anti-TOOWTDI.  For most situations
> {}-strings are great (IMO), but in the specific translation domain, I
> suspect $-strings are still better.

I agree that keeping string.Template around is valid due to its focus on
being very simple to use (especially for non-coders producing
translation templates)

However, string.Template is already set up to work for the relevant APIs
in that domain so I don't think it will really be affected by any
changes to the underlying language and library support for brace formatting.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sun Oct  4 09:56:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 04 Oct 2009 17:56:20 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	
	<4AC2846E.8060304@g.nevcal.com>	
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>	
	<4AC29F71.3000307@gmail.com> <4AC6DE09.3030003@gmail.com>	
	<9d153b7c0910030345v5a062b08h5092fe0f2ca3e8b7@mail.gmail.com>	
	<d11dcfba0910030812s3d8a24c7l2fa04d272170b399@mail.gmail.com>	
	<4AC76B10.1000209@voidspace.org.uk>	
	<d11dcfba0910030849v111160abge291d24ad980507a@mail.gmail.com>	
	<4AC77A17.7050101@voidspace.org.uk>
	<9d153b7c0910030938k66310859h510f67237e319ee5@mail.gmail.com>
Message-ID: <4AC85524.1030401@gmail.com>

Yuvgoog Greenle wrote:
> I know this might come through as bike shedding but it's just
> customary python that every module have it's own exception types as to
> not mix them up with others.

Not in my Python world it isn't. While that is sometimes the right
answer, more often the right answer is to properly scope your try/except
statement in order to avoid catching exceptions from code you aren't
interested in. If you end up stuck with a library call that lacks both
granularity and specificity of exceptions then you live with it (and
maybe file a bug report/feature request with the library author).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From stephen at xemacs.org  Tue Oct  6 06:43:55 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 Oct 2009 13:43:55 +0900
Subject: [Python-Dev] comments vs spam in PyPI [was: eggs now mandatory for
	pypi?]
In-Reply-To: <loom.20091005T202019-122@post.gmane.org>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<loom.20091005T202019-122@post.gmane.org>
Message-ID: <87ws39t9mc.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:
 > Guido van Rossum <guido <at> python.org> writes:
 > > 
 > > There are plenty of things we
 > > can learn about fighting spam and other forms of vandalism from other
 > > areas of the social web, including our very own wiki, and other wikis
 > > (WikiPedia survives despite spam).
 > 
 > Doesn't Wikipedia have a lot of human eyes watching, however?

Yes.  In fact Wikipedia's real issue is not spam, but edit wars.  It
just happens that the same people who are willing to watch to make
sure that nobody "corrects" the facts consider spam damage, too, and
they get rid of it as part of their mission.

What this means is that the most active parts of the Wikipedia are
also quickly policed.  What's left is much less attractive for
spammers, and there are a number of volunteers willing to respond
fairly promptly to reports of spam in articles that nobody currently
considers "theirs".

I think it could probably be adapted to Python community scale, but it
would probably require new mechanisms (spam reporting and possibly
cleaning -- in Wikipedia you hit the revert button, choose a known
good version, and you're done) in PyPI, and recruitment of volunteers
to take care of spam to products currently not "owned".

IMO it would be better to design developer-specific mechanisms rather
than a generic commenting vehicle, cf. Fred Drake's thinking.

From pje at telecommunity.com  Tue Oct  6 06:47:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 00:47:41 -0400
Subject: [Python-Dev] comments vs spam in PyPI [was: eggs now mandatory
 for pypi?]
In-Reply-To: <87ws39t9mc.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<loom.20091005T202019-122@post.gmane.org>
	<87ws39t9mc.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20091006044752.A4C4C3A4045@sparrow.telecommunity.com>

At 01:43 PM 10/6/2009 +0900, Stephen J. Turnbull wrote:
>IMO it would be better to design developer-specific mechanisms rather
>than a generic commenting vehicle, cf. Fred Drake's thinking.

For example, having a packages reddit (nb: open source, written in 
Python), where people can upvote or downvote packages and leave 
comments.  That's probably the minimum amount of checks and balances 
required to avoid problems of the sort the PyPI commenting feature is 
already having, since others will be able to reply to the comments, 
and downvote nonsense into oblivion.

(Alternatively, shutting off the comment system would also 
work.  Nothing stops people from using Google to search for "foo 
sucks" or "problems using foo" if they want to research what's been 
said about a package.)


From vinay_sajip at yahoo.co.uk  Tue Oct  6 08:09:55 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 6 Oct 2009 06:09:55 +0000 (UTC)
Subject: [Python-Dev] comments vs spam in PyPI [was: eggs now mandatory
	for pypi?]
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<4222a8490910050613w6f631a54m2cecd2a6c7d9d889@mail.gmail.com>
	<had53u$rrp$1@ger.gmane.org>
	<4222a8490910051038u779e8f71s177c7f17ba92abf1@mail.gmail.com>
	<ca471dc20910051054x78d6b3cbk24131d2a5c9933c@mail.gmail.com>
	<loom.20091005T202019-122@post.gmane.org>
	<87ws39t9mc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20091006044752.A4C4C3A4045@sparrow.telecommunity.com>
Message-ID: <loom.20091006T075325-612@post.gmane.org>

P.J. Eby <pje <at> telecommunity.com> writes:

> For example, having a packages reddit (nb: open source, written in 
> Python), where people can upvote or downvote packages and leave 
> comments.  That's probably the minimum amount of checks and balances 
> required to avoid problems of the sort the PyPI commenting feature is 
> already having, since others will be able to reply to the comments, 
> and downvote nonsense into oblivion.

Seems like a reasonable idea, given that there's already a Python reddit which
seems popular. There was also www.cheeserater.com (which appears down at the
moment) which was a Django demo site (source available at
http://github.com/jacobian/cheeserater) for rating PyPI packages.

I just created a "cheeseshop" sub-reddit to play around with.

> (Alternatively, shutting off the comment system would also 
> work.  Nothing stops people from using Google to search for "foo 
> sucks" or "problems using foo" if they want to research what's been 
> said about a package.)

Yes, but it's good to have all the feedback in one place, if possible.

Regards,

Vinay Sajip


From ncoghlan at gmail.com  Tue Oct  6 12:32:10 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 06 Oct 2009 20:32:10 +1000
Subject: [Python-Dev] summary of transitioning from % to {} formatting
In-Reply-To: <loom.20091005T233721-264@post.gmane.org>
References: <d11dcfba0910030841k6363ae5kf29e841489fe4c4a@mail.gmail.com>	<05DB5F3D-A69B-4F4A-A24C-5216666A50B3@python.org>	<4AC858B9.605@gmail.com>	<2F47507B-B9FB-4556-8755-8B561ECB293E@python.org>	<4ACA59F5.1080103@gmail.com>
	<loom.20091005T233721-264@post.gmane.org>
Message-ID: <4ACB1CAA.2000009@gmail.com>

Vinay Sajip wrote:
> Both of these approaches will also work for {}-formatting. The present thread
> really started out with a view to suggesting that the stdlib start adopting
> {}-format as "native", rather than %-format.
> 
> Would it be helpful if I added a section to the Python docs about how to use
> alternative formatting systems with logging?

That would probably be a good idea.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From bheemesh at gmail.com  Tue Oct  6 12:19:32 2009
From: bheemesh at gmail.com (bheemesh v)
Date: Tue, 6 Oct 2009 15:49:32 +0530
Subject: [Python-Dev] Help needed for tar archive creation with changed
	UID/GID
Message-ID: <670534840910060319t4af51e38r5fca7e11f4f4d8f8@mail.gmail.com>

Hello,

I am a newbie to this mailing list, i have seen in some mail discussions
about a tar archive creation by forcibly setting the UID/GID to a specific
user (say root).

Te code mentioned goes like this:

tar = tarfile.open("foo.tar.gz", "w:gz")
for filename in filenames:
  tarinfo = tar.gettarinfo(filename)
  if tarinfo.isreg():
    fobj = open(filename)
  else:
    fobj = None

  tarinfo.uid = 0
  tarinfo.gid = 0
  tar.addfile(tarinfo, fobj)
tar.close()

But this is not really working for me as when i am trying to un-tar or
un-compress contents to a new directory and contents still have diff GID &
UID in the long listing of directory.

Can somebody suggest an alternative or a working case if already changes are
done here.

Thanks in advance.

Best regards,
Bheemesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091006/d399c927/attachment.htm>

From stefan_ml at behnel.de  Tue Oct  6 12:41:30 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 06 Oct 2009 12:41:30 +0200
Subject: [Python-Dev] Help needed for tar archive creation with changed
	UID/GID
In-Reply-To: <670534840910060319t4af51e38r5fca7e11f4f4d8f8@mail.gmail.com>
References: <670534840910060319t4af51e38r5fca7e11f4f4d8f8@mail.gmail.com>
Message-ID: <haf6sq$p2h$1@ger.gmane.org>

bheemesh v wrote:
> I am a newbie to this mailing list, i have seen in some mail discussions
> about a tar archive creation by forcibly setting the UID/GID to a specific
> user (say root).

Note that this is the mailing list about core development of the CPython
interpreter, not about general Python development.

Please refer your request to python-list at python.org or tutor at python.org
instead.

Stefan


From fuzzyman at voidspace.org.uk  Tue Oct  6 12:43:34 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 06 Oct 2009 11:43:34 +0100
Subject: [Python-Dev] Help needed for tar archive creation with changed
 UID/GID
In-Reply-To: <670534840910060319t4af51e38r5fca7e11f4f4d8f8@mail.gmail.com>
References: <670534840910060319t4af51e38r5fca7e11f4f4d8f8@mail.gmail.com>
Message-ID: <4ACB1F56.8020008@voidspace.org.uk>

bheemesh v wrote:
> Hello,
>
> I am a newbie to this mailing list, i have seen in some mail 
> discussions about a tar archive creation by forcibly setting the 
> UID/GID to a specific user (say root).

Hello Bheemesh,

This mailing list is for the development of Python, not for developing 
*with* Python. There are other more appropriate mailing lists where you 
will hopefully get helpful answers:


The Python tutor list
http://mail.python.org/mailman/listinfo/tutor

The comp.lang.python newsgroup
The email gateway: http://mail.python.org/mailman/listinfo/python-list
A google groups interface: 
http://groups.google.com/group/comp.lang.python/topics?lnk=li&hl=en&pli=1

All the best,

Michael Foord


>
> Te code mentioned goes like this:
> tar = tarfile.open("foo.tar.gz", "w:gz")
> for filename in filenames:
>   tarinfo = tar.gettarinfo(filename)
>   if tarinfo.isreg():
>     fobj = open(filename)
>   else:
>     fobj = None
>
>
>   tarinfo.uid = 0
>   tarinfo.gid = 0
>   tar.addfile(tarinfo, fobj)
> tar.close()
>   
> But this is not really working for me as when i am trying to un-tar or 
> un-compress contents to a new directory and contents still have diff 
> GID & UID in the long listing of directory.
>
> Can somebody suggest an alternative or a working case if already 
> changes are done here.
>
> Thanks in advance.
>
> Best regards,
> Bheemesh
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From cournape at gmail.com  Tue Oct  6 12:55:56 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 6 Oct 2009 19:55:56 +0900
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <20091005185255.D4EA53A408F@sparrow.telecommunity.com>
References: <nad-6965DF.05065605102009@news.gmane.org>
	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>
	<20091005185255.D4EA53A408F@sparrow.telecommunity.com>
Message-ID: <5b8d13220910060355u1c3081ag1c80874b1aba715@mail.gmail.com>

2009/10/6 P.J. Eby <pje at telecommunity.com>:
> At 02:22 PM 10/5/2009 +0200, Tarek Ziad? wrote:
>>
>> Setuptools development has been discontinued for a year, and does
>> patches on Distutils code. Some of these patches are sensitive to any
>> change
>> made on Distutils, wether those changes are internal or not.
>
> Setuptools is ?also not the only thing out there that subclasses distutils
> commands in general, or the build_ext command in particular. ?Numpy,
> Twisted, the mx extensions... ?there are plenty of things out there that
> subclass distutils commands, quite in adherence to the rules. ?(Note too
> that subclassing != patching, and the ability to subclass and substitute
> distutils commands is documented.)
>
> It's therefore not appropriate to treat the issue as if it were
> setuptools-specific; it could have broken any other major (or minor)
> package's subclassing of the build_ext command.

The internal vs published API difference does not make much sense in
distutils case anyway, since a lot of implementation details are
necessary to make non trivial extension work.

When working on numpy.distutils, I almost always have to look at
distutils sources since the docs are vastly insufficient, and even
then, the code is so bad that quite often the only way to interact
with distutils is to "reverse engineer" its behavior by trial and
error.

cheers,

David

From chris at simplistix.co.uk  Tue Oct  6 15:00:24 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:00:24 +0100
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACA566C.4070902@gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>	<87hbuew70i.fsf@benfinney.id.au>	<5EB6ECE5-8C3F-4C7E-8922-15EE2B9D4D56@gmail.com>
	<4ACA566C.4070902@gmail.com>
Message-ID: <4ACB3F68.4030900@simplistix.co.uk>

Nick Coghlan wrote:
> Really? I can understand package owners not being able to add
> recommendations for their own packages, but if they can't add comments
> how are they meant to correct misunderstandings or redirect
> inappropriate messages to the correct forums?

Indeed, yet another reason I would like to have the option to disable 
comments for my packages. I make sure there are advertised mailing lists 
available for all my packages, and they are all much for featureful than 
PyPI should ever hope to be...

https://sourceforge.net/tracker/?func=detail&aid=2872293&group_id=66150&atid=513503

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct  6 15:01:52 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:01:52 +0100
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACA0CBA.2030200@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com>
Message-ID: <4ACB3FC0.1010906@simplistix.co.uk>

M.-A. Lemburg wrote:
> We've implemented our own bdist_egg now which doesn't use setuptools
> and will start to ship eggs in addition to our prebuilt format with
> the next releases.

Egg-cellent ;-)

Any chance this tool is open source? Better yet, could it make its way 
into distutils asap?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct  6 15:02:38 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:02:38 +0100
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
Message-ID: <4ACB3FEE.9060708@simplistix.co.uk>

Fredrik Lundh wrote:
> Oh, it was just yet another Zope developer behaving like an ass.  Why
> am I not surprised?

Actually Plohn, there aren't that many Zope developers left ;-)

Chris - looking mournfully at his sig...

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct  6 15:06:24 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:06:24 +0100
Subject: [Python-Dev] "PIL" vs. "Imaging" (was Re: eggs now mandatory
 for pypi?)
In-Reply-To: <368a5cd50910051203g7b71140ep6b26e3e66e13a2d@mail.gmail.com>
References: <368a5cd50910050143n7735652ftcc202363d9878e2c@mail.gmail.com>
	<368a5cd50910050158gca67163i5ba7e993269c7948@mail.gmail.com>
	<4AC9D356.5050202@cheimes.de> <1254741989.7368.22.camel@minime>
	<20091005184844.647633A4045@sparrow.telecommunity.com>
	<368a5cd50910051203g7b71140ep6b26e3e66e13a2d@mail.gmail.com>
Message-ID: <4ACB40D0.3000006@simplistix.co.uk>

Fredrik Lundh wrote:
> Looks like the quickest fix is to distribute two source packages, one
> built the traditional way to eliminate breakage for people using the
> current tarball layout, and one built with "sdist".

Why the need for two?

Or, asked differently, who would be hurt if the distribution name became 
PIL? (distribution name != package names in distribution)

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct  6 15:17:51 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:17:51 +0100
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <94bdd2610910050842p615a92a4je57560c04b2d076d@mail.gmail.com>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>	<4ACA0234.1070700@egenix.com>	<94bdd2610910050753x5f1eab38w3d24d3d8eacd09b4@mail.gmail.com>	<4ACA100A.8020602@egenix.com>
	<94bdd2610910050842p615a92a4je57560c04b2d076d@mail.gmail.com>
Message-ID: <4ACB437F.9060602@simplistix.co.uk>

Tarek Ziad? wrote:
> Notice that I am also doing nightly builds of Distutils that can be installed
> and tried in released version of Python, and that can be used instead of the
> Python's embed Distutils  (see the installable tarballs at nightly.ziade.org).
> so maybe it's just a matter of continuous integration

Could these be released as normal sdist packages on PyPI?
That would seem the best way to get them used...

(bonus points for working with buildout,virtualenv,pip,etc)

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From mal at egenix.com  Tue Oct  6 15:18:22 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 06 Oct 2009 15:18:22 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB3FC0.1010906@simplistix.co.uk>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>
	<4ACB3FC0.1010906@simplistix.co.uk>
Message-ID: <4ACB439E.6050102@egenix.com>

Chris Withers wrote:
> M.-A. Lemburg wrote:
>> We've implemented our own bdist_egg now which doesn't use setuptools
>> and will start to ship eggs in addition to our prebuilt format with
>> the next releases.
> 
> Egg-cellent ;-)
> 
> Any chance this tool is open source? Better yet, could it make its way
> into distutils asap?

mxSetup.py, the module implementing all our distutils extensions,
is available in egenix-mx-base which is open source:

	http://www.egenix.com/products/python/mxBase/

Here's current snapshot:

http://downloads.egenix.com/python/egenix-mx-base-3.2.0_20091006.zip

or as egg

http://downloads.egenix.com/python/egenix_mx_base-3.2.0_20091006-py2.6-linux-x86_64.egg

Note how the package name is mangled... easy_install requires this.
The file name also doesn't tell you that the above is for
a UCS2 Python build. Again, easy_install fails with that information
added to the "py2.6" version marker.

We're still tweaking the implementation a bit, but it's stabilizing
rather quickly.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From chris at simplistix.co.uk  Tue Oct  6 15:19:33 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:19:33 +0100
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
References: <nad-6965DF.05065605102009@news.gmane.org>	<94bdd2610910050522u5e7e7dfawcf44f23b4b786b69@mail.gmail.com>	<BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>	<94bdd2610910050717m388d8e86y868f2126781046fc@mail.gmail.com>	<F0920F9C-849A-4761-91C3-118D7555C742@python.org>	<94bdd2610910050750g38613d83jb3978f7b23e2aec5@mail.gmail.com>
	<A8990C32-20ED-4BAC-A723-40DA9909A2F2@python.org>
Message-ID: <4ACB43E5.6030003@simplistix.co.uk>

Barry Warsaw wrote:
> 2) PJE releases a new version of setuptools that fixes this problem.
> 
> 3) We (a.k.a. Tarek with our blessing) hijacks the setuptools name (e.g. 
> on cheeseshop) and releases a new version

It's a shame you didn't suggest this sooner. It would have avoided the 
need for the seperate Distribute project and all the horrible hacks it 
entails for every package that uses setuptools...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct  6 15:28:26 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:28:26 +0100
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB439E.6050102@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com>
Message-ID: <4ACB45FA.3080707@simplistix.co.uk>

M.-A. Lemburg wrote:
> mxSetup.py, the module implementing all our distutils extensions,
> is available in egenix-mx-base which is open source:
> 
> 	http://www.egenix.com/products/python/mxBase/

I have memories of mxBase having a load of other stuff in it too?

Would it be possible to split just the distutils extensions into their 
own package and get that package as an sdist on PyPI?

That said, I still think getting the bdist_egg code, at least, into 
Distutils in the core as soon as possible would be a very good thing. 
It's kinda odd having the python equivalent of jars or gems only being 
created by third party software that's no longer maintained...

 > Note how the package name is mangled... easy_install requires this.
 > The file name also doesn't tell you that the above is for
 > a UCS2 Python build. Again, easy_install fails with that information
 > added to the "py2.6" version marker.

Surely this stuff all belongs in the static metadata file that keeps on 
overcomplicating itself in discussions?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct  6 15:45:31 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 14:45:31 +0100
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4AC371D7.9000904@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
Message-ID: <4ACB49FB.3000407@simplistix.co.uk>

Hi Phil,

It's almost a week since I made this offer. I haven't heard anything 
from you. If I've missed anything please let me know and I'll track it 
down, otherwise I hope you can have a look at this some time soon.

cheers,

Chris

Chris Withers wrote:
> P.J. Eby wrote:
>> Here's what actually happened, if anyone cares.  Tarek and friends 
>> announced a fork of setuptools.  I reviewed the work and saw that -- 
>> for the most part -- I was happy with it, and opined as how I might be 
>> willing to bless the the "package inquisition" team as official 
>> maintainers of the 0.6 branch of setuptools, so that I could work on 
>> the fun bits I've long planned for 0.7, but never felt free to start 
>> on while there was so much still needing to be done on 0.6.
> 
> If this offer is still available, I'd lake to take you up on it.
> I'd be more than willing to merge changes on the 0.6 distribute branch 
> back into the setuptools codebase and do whatever is needed to get a new 
> setuptools release out.
> 
> Why? Because there are a *lot* of copies of ez_setup.py and buildout's 
> bootstrap.py that will need replacing if it doesn't happen. I think it'd 
> be better for the python community all round if setuptools just 
> continued in maintenance mode until whatever-ends-up-in-the-core exists 
> and people want to use...
> 
> I'm happy to submit to whatever supervision is needed for you to trust 
> me to do this, and I promise to be as careful as I can with this. I 
> *know* how important this is and want to make it work...
> 
>> All I want is for good stuff to happen for setuptools users and Python 
>> users in general, so I don't think all the suspicion and backbiting is 
>> merited. 
> 
> Fine, if that's true, I apologize (even spelled correctly!) for any 
> previous involvement in this, but please help me help you achieve your 
> aims...
> 
> To put this into a way that makes sense to me: I'm volunteering to keep 
> distribute 0.6 and setuptools 0.6 in sync, no more, no less, and try and 
> keep that as uncontroversial as possible, and get setuptools 0.6 
> releases out to match distribute 0.6 releases as soon as I can.
> 
> Again, I feel I need to stress that the *only* reason I want to do this 
> is to bring the benefits of the distribute work to the existing 
> setuptools codebase, with appropriate attribution if that makes a 
> difference.
> 
> Apologies if any of this is offensive to anyone. For once (really!) I 
> really mean that :-)
> 
> cheers,
> 
> Chris
> 

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ronaldoussoren at mac.com  Tue Oct  6 15:49:23 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 06 Oct 2009 15:49:23 +0200
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on
 Mac OS X?
In-Reply-To: <ee9557410910021640o2e2ae1ceyf48cf6736fc11c8@mail.gmail.com>
References: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
	<92839421634652657618189239787554831247-Webmail@me.com>
	<ee9557410910020129w1ed7be0fie2c45d099209189e@mail.gmail.com>
	<2ECCFBA0-1A70-4D12-925C-CE620D430AAD@gmail.com>
	<ee9557410910021640o2e2ae1ceyf48cf6736fc11c8@mail.gmail.com>
Message-ID: <CE8F0089-16A7-4693-BCA9-309C25133DEA@mac.com>


On 3 Oct, 2009, at 1:40, INADA Naoki wrote:

>> Confirmed on 10.6 for 2.6.3 and 2.7a0. For 3.2a0 both asserts fail.
>
> OK.
> `s = '\xa0'` should be `s = b'\xa0'`.
>
> Should I file a bug?

Please do, that helps us remember that the issue exists.

Ronald

>
> -- 
> Naoki INADA  <songofacandy at gmail.com>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091006/dcc7f6e8/attachment.bin>

From arcriley at gmail.com  Tue Oct  6 15:54:07 2009
From: arcriley at gmail.com (Arc Riley)
Date: Tue, 6 Oct 2009 09:54:07 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB45FA.3080707@simplistix.co.uk>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com> <4ACB45FA.3080707@simplistix.co.uk>
Message-ID: <bad82a80910060654t51c35bf1lb3235d39b936fd22@mail.gmail.com>

I'll make the argument that feedback is useful, comments are much less so
and a lot more work.

It would be more useful to allow package users post feedback, visible only
to the package maintainer, and also add support for bugtracker links/etc.

Is the intention of Pypi really to turn it into a social networking site?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091006/c772105c/attachment.htm>

From ziade.tarek at gmail.com  Tue Oct  6 16:00:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 6 Oct 2009 16:00:29 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACA0CBA.2030200@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com>
Message-ID: <94bdd2610910060700n4d948c3boc716a7979ac22bd5@mail.gmail.com>

On Mon, Oct 5, 2009 at 5:11 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Senthil Kumaran wrote:
>> On Mon, Oct 05, 2009 at 10:43:22AM +0200, Fredrik Lundh wrote:
>>> it's revews like this that makes me wonder if releasing open source is
>>> a good idea:
>>> ? ?no egg - worst seen ever, remove it from pypi or provide an egg
>>> (jensens, 2009-10-05, 0 points)
>>>
>>
>> Greetings effbot. :)
>> As you might already know, "eggs" are not mandatory for pypi. No where
>> in the documentation it mentions so. Also many good packages in pypi
>> are not eggs (MaL's egenix packages for e.g) and we have known his
>> dislike for that particular format too.
>
> I don't dislike the format itself, just the tools that are
> available for using it.
>
> We've implemented our own bdist_egg now which doesn't use setuptools
> and will start to ship eggs in addition to our prebuilt format with
> the next releases.
>
> As time permits, we'll also write an egg installer/uninstaller
> (one that does only that ;-).
>
>> But still, sometimes people have found eggs to be convenient.

The idea of PEP 376 we are working on is to have a unique unified
standard for distributions
installed in site-packages, instead of the current situation where we
have many installation formats.
(Distutils pre-Python 2.5, Distutils post-Python2.5, Setuptools
self-contained Eggs, ..)
because it makes it simple to provide APIs to query what's installed,
and ways to uninstall things.

That's what I would call the "Missing APIs" to make Distutils a
package manager, or at least a basis for package
managers.

Having an egg installer/uninstaller that works with Setuptools-like
self-contained eggs sounds like
a package manager system on its own and yet another format.

But as far as we went in the discussions for PEP 376, it seemed like a
first step would be to work with a single
installation format and provides all APIs for it, then to think about
having other formats and ways to query them
in a second move.

That's why I don't think its the right time to add a bdist_egg command
in Distutils, until PEP 376 is
fully defined (and maybe extended to support several format).

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From songofacandy at gmail.com  Tue Oct  6 16:32:32 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Tue, 6 Oct 2009 23:32:32 +0900
Subject: [Python-Dev] Old libc's isspace() bug on FreeBSD brings back on
	Mac OS X?
In-Reply-To: <CE8F0089-16A7-4693-BCA9-309C25133DEA@mac.com>
References: <ee9557410910011907w73e7685cofeb65a4e4e5f748a@mail.gmail.com>
	<92839421634652657618189239787554831247-Webmail@me.com>
	<ee9557410910020129w1ed7be0fie2c45d099209189e@mail.gmail.com>
	<2ECCFBA0-1A70-4D12-925C-CE620D430AAD@gmail.com>
	<ee9557410910021640o2e2ae1ceyf48cf6736fc11c8@mail.gmail.com>
	<CE8F0089-16A7-4693-BCA9-309C25133DEA@mac.com>
Message-ID: <ee9557410910060732l30828638k8fe53c18824b428e@mail.gmail.com>

I filed as issue7072.

On Tue, Oct 6, 2009 at 10:49 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 3 Oct, 2009, at 1:40, INADA Naoki wrote:
>
>>> Confirmed on 10.6 for 2.6.3 and 2.7a0. For 3.2a0 both asserts fail.
>>
>> OK.
>> `s = '\xa0'` should be `s = b'\xa0'`.
>>
>> Should I file a bug?
>
> Please do, that helps us remember that the issue exists.
>
> Ronald
>
>>
>> --
>> Naoki INADA ?<songofacandy at gmail.com>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com
>
>



-- 
Naoki INADA  <songofacandy at gmail.com>

From mal at egenix.com  Tue Oct  6 16:39:12 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 06 Oct 2009 16:39:12 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB45FA.3080707@simplistix.co.uk>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk>
Message-ID: <4ACB5690.1080208@egenix.com>

Chris Withers wrote:
> M.-A. Lemburg wrote:
>> mxSetup.py, the module implementing all our distutils extensions,
>> is available in egenix-mx-base which is open source:
>>
>>     http://www.egenix.com/products/python/mxBase/
> 
> I have memories of mxBase having a load of other stuff in it too?

Yep, lots of it.

> Would it be possible to split just the distutils extensions into their
> own package and get that package as an sdist on PyPI?

Nope. The main reason for having switched to distributions is
to make things easier to manage - both for us and our users.

> That said, I still think getting the bdist_egg code, at least, into
> Distutils in the core as soon as possible would be a very good thing.
> It's kinda odd having the python equivalent of jars or gems only being
> created by third party software that's no longer maintained...
> 
>> Note how the package name is mangled... easy_install requires this.
>> The file name also doesn't tell you that the above is for
>> a UCS2 Python build. Again, easy_install fails with that information
>> added to the "py2.6" version marker.
> 
> Surely this stuff all belongs in the static metadata file that keeps on
> overcomplicating itself in discussions?

The complicated stuff does belong somewhere else, but the
basic things need to go into the filename - simply because
you have to build separate files for each Python build
dimension and want to be able to put everything into
a single directory or index:

 * Python version
 * Architecture/Platform
 * UCS2/UCS4

E.g. to solve the UCS2/4 part, we had to setup separate
PyPI-style index sites. We'd really like to avoid that if
possible.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From chris at simplistix.co.uk  Tue Oct  6 17:45:29 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Oct 2009 16:45:29 +0100
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB5690.1080208@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk> <4ACB5690.1080208@egenix.com>
Message-ID: <4ACB6619.4080807@simplistix.co.uk>

M.-A. Lemburg wrote:
>> Would it be possible to split just the distutils extensions into their
>> own package and get that package as an sdist on PyPI?
> 
> Nope. 

shame :-(

> The complicated stuff does belong somewhere else, but the
> basic things need to go into the filename 

But everyone's basic things are different.


> - simply because
> you have to build separate files for each Python build
> dimension and want to be able to put everything into
> a single directory or index:
> 
>  * Python version
>  * Architecture/Platform
>  * UCS2/UCS4
> 
> E.g. to solve the UCS2/4 part, we had to setup separate
> PyPI-style index sites. We'd really like to avoid that if
> possible.

Well yeah, and the only sane way I can think to handle this is to have a 
metadata file that gets uploaded with each distribution that covers all 
these things (and the other things that other people need) and then have 
the index (which would hopefully be PyPI on the whole) be queryable 
along the lines of "give me a download url for a distribution named X 
that is for Python 2.6, Win32, UCS4, PostGreSQL 8.5, BlagBlah"...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From mal at egenix.com  Tue Oct  6 18:03:08 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 06 Oct 2009 18:03:08 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB6619.4080807@simplistix.co.uk>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com> <4ACB6619.4080807@simplistix.co.uk>
Message-ID: <4ACB6A3C.6000909@egenix.com>

Chris Withers wrote:
> M.-A. Lemburg wrote:
>> The complicated stuff does belong somewhere else, but the
>> basic things need to go into the filename 
> 
> But everyone's basic things are different.

The really basic stuff is the same for everyone since it's
dictated by the way Python works on computers :-)

>> - simply because
>> you have to build separate files for each Python build
>> dimension and want to be able to put everything into
>> a single directory or index:
>>
>>  * Python version
>>  * Architecture/Platform
>>  * UCS2/UCS4
>>
>> E.g. to solve the UCS2/4 part, we had to setup separate
>> PyPI-style index sites. We'd really like to avoid that if
>> possible.
> 
> Well yeah, and the only sane way I can think to handle this is to have a
> metadata file that gets uploaded with each distribution that covers all
> these things (and the other things that other people need) and then have
> the index (which would hopefully be PyPI on the whole) be queryable
> along the lines of "give me a download url for a distribution named X
> that is for Python 2.6, Win32, UCS4, PostGreSQL 8.5, BlagBlah"...

So you want the indirection to happen even before fetching
the download package ?

That's a good idea - and also a lot better than the web page
link scanning that setuptools is doing.

It goes a bit in the direction of what we had in mind with writing
for our clients: a tool that looks at the Python installation and
automatically finds/downloads/installs the right package from
our website. Only that we wanted to put up a single index file
for that tool to use, not one per package.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From pje at telecommunity.com  Tue Oct  6 19:52:34 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 13:52:34 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB439E.6050102@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com>
Message-ID: <20091006175244.D1FF03A4116@sparrow.telecommunity.com>

At 03:18 PM 10/6/2009 +0200, M.-A. Lemburg wrote:
>Note how the package name is mangled... easy_install requires this.
>The file name also doesn't tell you that the above is for
>a UCS2 Python build. Again, easy_install fails with that information
>added to the "py2.6" version marker.

Btw, every couple years or so I've sent out a call on the 
distutils-SIG to try to get consensus on a format for the platform 
tag information used by setuptools.  (The first time was before 
easy_install even existed.)  Unfortunately, it's rare that anybody 
contributes more than criticism of the existing scheme.  The last 
updates to the scheme were by Mac folks, who at least had a clear 
vision of what was needed.  There's been some win32/win64 discussion 
in the past, but no proposals or patches of comparable 
specificity.  Linux and other unices haven't had any concrete 
suggestions at all, AFAICR.

Anyway...  just saying that if you *have* a concretely implementable 
proposal about what to do with platform tags, please do share on the 
distutils-sig.  Platform tags are one area that's reasonably easy to 
upgrade in a backward-compatible-ish way; newer versions of 
easy_install can still recognize older tags, although new eggs won't 
be recognized by older installers (forcing people to upgrade 
setuptools in that case).


From pje at telecommunity.com  Tue Oct  6 20:03:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 14:03:51 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4ACB49FB.3000407@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
Message-ID: <20091006180400.BA69D3A4116@sparrow.telecommunity.com>

At 02:45 PM 10/6/2009 +0100, Chris Withers wrote:
>>To put this into a way that makes sense to me: I'm volunteering to 
>>keep distribute 0.6 and setuptools 0.6 in sync, no more, no less, 
>>and try and keep that as uncontroversial as possible, and get 
>>setuptools 0.6 releases out to match distribute 0.6 releases as soon as I can.

That may not be as easy as it sounds; Distribute deleted various 
things from the setuptools tree (e.g. the release.sh script, used for 
issuing releases) and of course it adds other stuff (e.g. stuff to 
overwrite setuptools).  So you'd need to screen the diffs.

Second, I still intend to move setuptools 0.7 forward at some point, 
which means the patches also need to go to the trunk.

If I had the time to co-ordinate and supervise all this, I'd have the 
time to just do it myself.

If you want to help with that, the most helpful thing would be for 
you to consolidate all the changes into a pair of patches: one for 
the 0.6 branch and one for the 0.7 trunk.

These patches would also need to exclude the SVN 1.6 changes (as I 
already have a change for that in my working copy, not yet checked 
in).  They would also need to include appropriate changelog and 
documentation updates.

If you can get those to me by Friday, I'll have them reviewed, 
applied, and released by Monday.  I was already planning to spend a 
little time on bug closing and patch application this coming weekend, 
so if you can do this for me first, it will maximize the number I can get done.


From pje at telecommunity.com  Tue Oct  6 20:11:40 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 14:11:40 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACB6A3C.6000909@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com> <4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com> <4ACB6619.4080807@simplistix.co.uk>
	<4ACB6A3C.6000909@egenix.com>
Message-ID: <20091006181149.133E83A4116@sparrow.telecommunity.com>

At 06:03 PM 10/6/2009 +0200, M.-A. Lemburg wrote:
>Chris Withers wrote:
> > Well yeah, and the only sane way I can think to handle this is to have a
> > metadata file that gets uploaded with each distribution that covers all
> > these things (and the other things that other people need) and then have
> > the index (which would hopefully be PyPI on the whole) be queryable
> > along the lines of "give me a download url for a distribution named X
> > that is for Python 2.6, Win32, UCS4, PostGreSQL 8.5, BlagBlah"...
>
>So you want the indirection to happen even before fetching
>the download package ?
>
>That's a good idea - and also a lot better than the web page
>link scanning that setuptools is doing.

They're not mutually exclusive, btw.  More information could be added 
to the PyPI "Simple" API to include this sort of thing, visible to 
newer installers and invisible to older ones.

The main reason why easy_install has a relatively simple, REST 
outgoing API is that it allows people to make their own compatible 
indexes just by throwing some files in a directory and turning on 
Apache directory indexing.  By using a... what do they call it, 
"microformat"? to mark up other information in the file, it'd still 
be possible for someone to slap up a web page and use it as a package index.


>It goes a bit in the direction of what we had in mind with writing
>for our clients: a tool that looks at the Python installation and
>automatically finds/downloads/installs the right package from
>our website. Only that we wanted to put up a single index file
>for that tool to use, not one per package.

That'd get a bit big for PyPI, though, with thousands of listed 
packages.  It's already pretty slow whenever somebody miscapitalizes 
or mis-punctuates a package name and easy_install has to read the 
"file" that just lists all the package names.  That, of course, is 
more a function of the fact that PyPI's "simple" API doesn't support 
canonicalization, although I believe certain other PyPI-mirroring and 
package indexing apps do.


From mal at egenix.com  Tue Oct  6 20:17:44 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 06 Oct 2009 20:17:44 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <20091006175244.D1FF03A4116@sparrow.telecommunity.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>
	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>
	<20091006175244.D1FF03A4116@sparrow.telecommunity.com>
Message-ID: <4ACB89C8.3030405@egenix.com>

P.J. Eby wrote:
> At 03:18 PM 10/6/2009 +0200, M.-A. Lemburg wrote:
>> Note how the package name is mangled... easy_install requires this.
>> The file name also doesn't tell you that the above is for
>> a UCS2 Python build. Again, easy_install fails with that information
>> added to the "py2.6" version marker.
> 
> Btw, every couple years or so I've sent out a call on the distutils-SIG
> to try to get consensus on a format for the platform tag information
> used by setuptools.  (The first time was before easy_install even
> existed.)  Unfortunately, it's rare that anybody contributes more than
> criticism of the existing scheme.  The last updates to the scheme were
> by Mac folks, who at least had a clear vision of what was needed. 
> There's been some win32/win64 discussion in the past, but no proposals
> or patches of comparable specificity.  Linux and other unices haven't
> had any concrete suggestions at all, AFAICR.
> 
> Anyway...  just saying that if you *have* a concretely implementable
> proposal about what to do with platform tags, please do share on the
> distutils-sig.  Platform tags are one area that's reasonably easy to
> upgrade in a backward-compatible-ish way; newer versions of easy_install
> can still recognize older tags, although new eggs won't be recognized by
> older installers (forcing people to upgrade setuptools in that case).

Here's a very simple Python version scheme we've been using
for years:

pyX.X_ucsY

with X.X being the major.minor Python version and Y being
either 2 or 4 depending on the UCS-build that was used to
build the egg.

The "_ucsY" part is not needed for pure-Python archives (but it
doesn't hurt using it there as well) and also not needed on
Windows, since Python is always UCS2 on Windows.

easy_install should first look for the full version string and
then fall back to the non-UCS one.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From guido at python.org  Tue Oct  6 21:16:07 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Oct 2009 12:16:07 -0700
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091006180400.BA69D3A4116@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com> 
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> 
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> 
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk> 
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
Message-ID: <ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>

On Tue, Oct 6, 2009 at 11:03 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 02:45 PM 10/6/2009 +0100, Chris Withers wrote:
>>>
>>> To put this into a way that makes sense to me: I'm volunteering to keep
>>> distribute 0.6 and setuptools 0.6 in sync, no more, no less, and try and
>>> keep that as uncontroversial as possible, and get setuptools 0.6 releases
>>> out to match distribute 0.6 releases as soon as I can.
>
> That may not be as easy as it sounds; Distribute deleted various things from
> the setuptools tree (e.g. the release.sh script, used for issuing releases)
> and of course it adds other stuff (e.g. stuff to overwrite setuptools). ?So
> you'd need to screen the diffs.
>
> Second, I still intend to move setuptools 0.7 forward at some point, which
> means the patches also need to go to the trunk.

Dream on.

> If I had the time to co-ordinate and supervise all this, I'd have the time
> to just do it myself.

I think at this point the community should not be forced wait for you
to get a new supply of round tuits. The wait has been too long
already. You can stay on in an advisory role, but I don't think it's
reasonable to block development or decisions until you have time.

> If you want to help with that, the most helpful thing would be for you to
> consolidate all the changes into a pair of patches: one for the 0.6 branch
> and one for the 0.7 trunk.
>
> These patches would also need to exclude the SVN 1.6 changes (as I already
> have a change for that in my working copy, not yet checked in). ?They would
> also need to include appropriate changelog and documentation updates.
>
> If you can get those to me by Friday, I'll have them reviewed, applied, and
> released by Monday. ?I was already planning to spend a little time on bug
> closing and patch application this coming weekend, so if you can do this for
> me first, it will maximize the number I can get done.

That's great, but I don't think it solves the structural problem,
which is that you don't have enough time to devote to the project.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From olemis at gmail.com  Tue Oct  6 21:56:04 2009
From: olemis at gmail.com (Olemis Lang)
Date: Tue, 6 Oct 2009 14:56:04 -0500
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
Message-ID: <24ea26600910061256n748ecd05o94ea921a3b2327e8@mail.gmail.com>

On Tue, Oct 6, 2009 at 2:16 PM, Guido van Rossum <guido at python.org> wrote:
> On Tue, Oct 6, 2009 at 11:03 AM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 02:45 PM 10/6/2009 +0100, Chris Withers wrote:
>>>>
>>>> To put this into a way that makes sense to me: I'm volunteering to keep
>>>> distribute 0.6 and setuptools 0.6 in sync, no more, no less, and try and
>>>> keep that as uncontroversial as possible, and get setuptools 0.6 releases
>>>> out to match distribute 0.6 releases as soon as I can.
>>
>> That may not be as easy as it sounds; Distribute deleted various things from
>> the setuptools tree (e.g. the release.sh script, used for issuing releases)
>> and of course it adds other stuff (e.g. stuff to overwrite setuptools). ?So
>> you'd need to screen the diffs.
>>
>> Second, I still intend to move setuptools 0.7 forward at some point, which
>> means the patches also need to go to the trunk.
>
> Dream on.
>
>> If I had the time to co-ordinate and supervise all this, I'd have the time
>> to just do it myself.
>
> I think at this point the community should not be forced wait for you
> to get a new supply of round tuits. The wait has been too long
> already. You can stay on in an advisory role, but I don't think it's
> reasonable to block development or decisions until you have time.
>

Is it possible to fork `setuptools` somehow ? This way :

  - patches may be built for setuptools 0.7 if it evolves
  - community can continue development in the ?new? branch.

all this easy if a DVCS is used ;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Nabble - Trac Users - Coupling trac and symfony framework  -
http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html

From ronaldoussoren at mac.com  Tue Oct  6 22:06:41 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 06 Oct 2009 22:06:41 +0200
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <24ea26600910061256n748ecd05o94ea921a3b2327e8@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk> <4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<24ea26600910061256n748ecd05o94ea921a3b2327e8@mail.gmail.com>
Message-ID: <94A5D218-CC9D-46DD-8DB8-FCECEF8D3F93@mac.com>


On 6 Oct, 2009, at 21:56, Olemis Lang wrote:

>
> Is it possible to fork `setuptools` somehow ? This way :

Yes, see <http://pypi.python.org/pypi/distribute>

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091006/fc7a9a2c/attachment.bin>

From pje at telecommunity.com  Tue Oct  6 22:08:42 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 16:08:42 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.co
 m>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
Message-ID: <20091006200852.551BC3A4116@sparrow.telecommunity.com>

At 12:16 PM 10/6/2009 -0700, Guido van Rossum wrote:
>I think at this point the community should not be forced wait for you
>to get a new supply of round tuits. The wait has been too long
>already. You can stay on in an advisory role, but I don't think it's
>reasonable to block development or decisions until you have time.

Tarek didn't think so either, which is why he created a fork, called 
Distribute.  So I'm not really clear on what you're trying to say 
here (or in the rest of the email, for that matter).


From guido at python.org  Tue Oct  6 22:14:18 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Oct 2009 13:14:18 -0700
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091006200852.551BC3A4116@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> 
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> 
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk> 
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com> 
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com> 
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
Message-ID: <ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>

I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
Distribute is mature and in widespread use. IOW I support Tarek's fork
and suggest nobody hold their breath waiting for setuptools 0.7.

On Tue, Oct 6, 2009 at 1:08 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 12:16 PM 10/6/2009 -0700, Guido van Rossum wrote:
>>
>> I think at this point the community should not be forced wait for you
>> to get a new supply of round tuits. The wait has been too long
>> already. You can stay on in an advisory role, but I don't think it's
>> reasonable to block development or decisions until you have time.
>
> Tarek didn't think so either, which is why he created a fork, called
> Distribute. ?So I'm not really clear on what you're trying to say here (or
> in the rest of the email, for that matter).
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Tue Oct  6 22:17:14 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 06 Oct 2009 22:17:14 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <20091006181149.133E83A4116@sparrow.telecommunity.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>
	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk>	<4ACB5690.1080208@egenix.com>
	<4ACB6619.4080807@simplistix.co.uk>	<4ACB6A3C.6000909@egenix.com>
	<20091006181149.133E83A4116@sparrow.telecommunity.com>
Message-ID: <4ACBA5CA.2020902@egenix.com>

P.J. Eby wrote:
> At 06:03 PM 10/6/2009 +0200, M.-A. Lemburg wrote:
>> It goes a bit in the direction of what we had in mind with writing
>> for our clients: a tool that looks at the Python installation and
>> automatically finds/downloads/installs the right package from
>> our website. Only that we wanted to put up a single index file
>> for that tool to use, not one per package.
> 
> That'd get a bit big for PyPI, though, with thousands of listed
> packages.  It's already pretty slow whenever somebody miscapitalizes or
> mis-punctuates a package name and easy_install has to read the "file"
> that just lists all the package names.  That, of course, is more a
> function of the fact that PyPI's "simple" API doesn't support
> canonicalization, although I believe certain other PyPI-mirroring and
> package indexing apps do.

The above was only meant for our eGenix packages, not PyPI.

For PyPI the natural answer to this is to use the already existing
PyPI XML-RPC interface:

    http://wiki.python.org/moin/PyPiXmlRpc

This would only have to grow a few more fields for the release
downloads in order to be able to match the user's Python configuration
to the available release downloads.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From zookog at gmail.com  Tue Oct  6 22:51:32 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Tue, 6 Oct 2009 14:51:32 -0600
Subject: [Python-Dev] [New-bugs-announce] [issue7064] Python 2.6.3 /
	setuptools 0.6c9: extension module builds fail with KeyError
In-Reply-To: <94bdd2610910050604k5cc20cdblf24a65c307a1cfdf@mail.gmail.com>
References: <1254742115.57.0.346118025313.issue7064@psf.upfronthosting.co.za>
	<19145.60293.661555.312221@montanaro.dyndns.org>
	<94bdd2610910050604k5cc20cdblf24a65c307a1cfdf@mail.gmail.com>
Message-ID: <cd6401a0910061351i1d84cea3h624608496d35cef6@mail.gmail.com>

Here are three buildbot farms for three different projects that
exercise various features of setuptools: build, install, sdist_dsc,
bdist_egg, sdist, and various specific requirements that our projects
have, such as the "Desert Island Build" in which setuptools is not
allowed to download anything from the Internet at build time or else
it flunks the test.

http://allmydata.org/buildbot/waterfall
http://allmydata.org/buildbot-pycryptopp/waterfall
http://allmydata.org/buildbot-zfec/waterfall

I would love it if new versions of setuptools/Distribute would make
more of these tests pass on more platforms or at least avoid causing
any regressions in these tests on these platforms.

Unfortunately, we can't really deploy new versions of
setuptools/Distribute to this buildbot farm in order to re-run all the
tests, because the only way that I know of to trigger all the tests is
to make a commit to our central darcs repository for Tahoe-LAFS,
pycryptopp, or zfec, and I don't want to do that to experiment with
new versions of setuptools/Distribute.  Does anyone know how to use a
buildbot farm like this one to run tests without committing patches to
the central repository?

Regards,

Zooko

From pje at telecommunity.com  Tue Oct  6 23:17:33 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 17:17:33 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.co
 m>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
Message-ID: <20091006211742.9E43E3A4116@sparrow.telecommunity.com>

At 01:14 PM 10/6/2009 -0700, Guido van Rossum wrote:
>suggest nobody hold their breath waiting for setuptools 0.7.

I've never suggested or implied otherwise.

But, if you like Distribute so much, why not just add it directly to 
the stdlib?  ;-)

AFAIK, the only reason they've had multiple releases of it is to 
address the issues with its hijacking of setuptools; in a stdlib 
version all that stuff could be dropped.  Otherwise, it'd already be "mature".

There are many wins to be had from such a move, not the least of 
which is that it allows something to go into the stdlib that isn't 
(branding-wise) associated with me or setuptools, and lets it become 
owned/supported by an even wider community.

Less political bickering, and the some of the technical results I 
hoped for all along are achieved.  Yay, open source.


From pje at telecommunity.com  Tue Oct  6 23:19:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 06 Oct 2009 17:19:51 -0400
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACBA5CA.2020902@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com> <4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com> <4ACB6619.4080807@simplistix.co.uk>
	<4ACB6A3C.6000909@egenix.com>
	<20091006181149.133E83A4116@sparrow.telecommunity.com>
	<4ACBA5CA.2020902@egenix.com>
Message-ID: <20091006212000.34B423A4116@sparrow.telecommunity.com>

At 10:17 PM 10/6/2009 +0200, M.-A. Lemburg wrote:
>P.J. Eby wrote:
> > At 06:03 PM 10/6/2009 +0200, M.-A. Lemburg wrote:
> >> It goes a bit in the direction of what we had in mind with writing
> >> for our clients: a tool that looks at the Python installation and
> >> automatically finds/downloads/installs the right package from
> >> our website. Only that we wanted to put up a single index file
> >> for that tool to use, not one per package.
> >
> > That'd get a bit big for PyPI, though, with thousands of listed
> > packages.  It's already pretty slow whenever somebody miscapitalizes or
> > mis-punctuates a package name and easy_install has to read the "file"
> > that just lists all the package names.  That, of course, is more a
> > function of the fact that PyPI's "simple" API doesn't support
> > canonicalization, although I believe certain other PyPI-mirroring and
> > package indexing apps do.
>
>The above was only meant for our eGenix packages, not PyPI.
>
>For PyPI the natural answer to this is to use the already existing
>PyPI XML-RPC interface:
>
>     http://wiki.python.org/moin/PyPiXmlRpc
>
>This would only have to grow a few more fields for the release
>downloads in order to be able to match the user's Python configuration
>to the available release downloads.

Yes, but it's not easily duplicated by somebody who's just got a 
plain ol' web page, nor is it amenable to static page mirroring.  A 
GET-only REST API, on the other hand, is compatible with both. 


From greg.ewing at canterbury.ac.nz  Sat Oct  3 03:29:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 03 Oct 2009 13:29:06 +1200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ECC08A6A-846E-4334-899C-F42D9D8BC811@python.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<ECC08A6A-846E-4334-899C-F42D9D8BC811@python.org>
Message-ID: <4AC6A8E2.4030304@canterbury.ac.nz>

Has anyone considered the idea of having the string % operator
behave intelligently according to the contents of the format
string?

If it contains one or more valid %-formats, use old-style
formatting; if it contains one or more valid {}-formats,
use new-style formatting.

Ambiguous cases could arise, of course, but hopefully they
will be fairly rare, and raising an exception would point
out the problem and allow it to be fixed.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Thu Oct  1 02:16:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 01 Oct 2009 13:16:06 +1300
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909301448v4a7f891q3e12cec01ebde44a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
	<4AC3CED3.80104@v.loewis.de>
	<8517e9350909301448v4a7f891q3e12cec01ebde44a@mail.gmail.com>
Message-ID: <4AC3F4C6.9080107@canterbury.ac.nz>

Peter Moody wrote:
> it's useful to take an
> address like 192.168.1.100/24 and derive a bunch of information from
> it (like the network address, broadcast address, containing supernets,
> etc), but still remember that the original address was 192.168.1.100.
> having a separate class or two for this is overly burdensome in my
> mind.

Seems to me what you want isn't so much an IPNetwork that
can optionally have an address, as an IPAddress that can
optionally have a mask, and methods for deriving those
other things from it.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Oct  1 01:35:01 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 01 Oct 2009 12:35:01 +1300
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC38BD0.9070608@v.loewis.de>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de> <4AC33DE1.3040304@trueblade.com>
	<4AC38BD0.9070608@v.loewis.de>
Message-ID: <4AC3EB25.1000307@canterbury.ac.nz>

Martin v. L?wis wrote:

> So if a decision was made to eventually remove % formatting, it would
> be reasonable to start migrating code to PEP 3101. However, no such
> decision has been made (and hopefully won't be throughout 3.x)

If it's not done during 3.x, then by the time 4.x comes
around, there will still be heaps of code around that
uses %-formatting, so it won't be possible to remove
it in 4.x either.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Oct  1 02:05:49 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 01 Oct 2009 13:05:49 +1300
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
References: <20090929202035.GA5492@gmail.com>
	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
	<9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
Message-ID: <4AC3F25D.8000907@canterbury.ac.nz>

Yuvgoog Greenle wrote:

> When is the else after a loop executed?
 > 1. When the loop isn't entered at all.
> 2. When the loop terminates through exhaustion of the list (does this
> include when the list was empty?)
> 3. When the loop didn't exit because of a break statement.

1 and 3 are just special cases of 2, when you include
the empty-list case. So you're making it more complicated
in your mind than it really is.

> Any chances of cleaning this one up for python 4?

I doubt that many other people will see anything
here that needs cleaning up.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Oct  1 00:41:42 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 01 Oct 2009 10:41:42 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
Message-ID: <4AC3DEA6.2050804@canterbury.ac.nz>

Paul Moore wrote:
> linus and snoopy are hosts not networks, so
> making them IPv4Network classes seems wrong. I'd instinctively make
> them IPv4Address objects (which, I believe, would work).

However, by allowing IPNetwork objects to also contain
a host address, we seem to be officially sanctioning
such abuse, or at least turning a blind eye to it.

It's possible that the user is thinking of linus and
snoopy as being *both* networks and host addresses,
in which case this kind of mistake will be easy to
make. Not to mention the potential confusion in the
mind of someone trying to read or maintain the code.

My feeling is that we shouldn't accept this kind of
compromise as being "good enough". Once a module is
in the stdlib, we're going to be stuck with it for
quite a while, so I think it's worth taking some
time to make sure that what we adopt is based on a
conceptually sound design.

Peter is of course within his rights not to change
his module in this way. But this just means that
ipaddr is not the right module to adopt, and we
should design another one and give it a different
name.

-- 
Greg

From guido at python.org  Wed Oct  7 05:44:36 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Oct 2009 20:44:36 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC6A8E2.4030304@canterbury.ac.nz>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org> 
	<ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com> 
	<loom.20091002T113352-479@post.gmane.org>
	<ECC08A6A-846E-4334-899C-F42D9D8BC811@python.org> 
	<4AC6A8E2.4030304@canterbury.ac.nz>
Message-ID: <ca471dc20910062044k74bf97bcv37d0ea8c426ac97b@mail.gmail.com>

On Fri, Oct 2, 2009 at 6:29 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Has anyone considered the idea of having the string % operator
> behave intelligently according to the contents of the format
> string?
>
> If it contains one or more valid %-formats, use old-style
> formatting; if it contains one or more valid {}-formats,
> use new-style formatting.
>
> Ambiguous cases could arise, of course, but hopefully they
> will be fairly rare, and raising an exception would point
> out the problem and allow it to be fixed.

Hm... The % operator already does too much guessing: if the string
contains exactly one %-format, the argument may be either a size-1
tuple or a non-tuple, otherwise it has to be a size-N tuple, except if
the %-formats use the %(name)X form, then the argument must always be
a dict. It doesn't sound to me as if adding more guesswork is going to
improve its reliability.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From david.lyon at preisshare.net  Wed Oct  7 07:04:17 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 07 Oct 2009 01:04:17 -0400
Subject: [Python-Dev] =?utf-8?q?eggs_now_mandatory_for_pypi=3F?=
In-Reply-To: <20091006175244.D1FF03A4116@sparrow.telecommunity.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com>
	<20091006175244.D1FF03A4116@sparrow.telecommunity.com>
Message-ID: <75c06ee14c3ef81164d5b4a9f3177799@preisshare.net>

On Tue, 06 Oct 2009 13:52:34 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> Btw, every couple years or so I've sent out a call on the 
> distutils-SIG to try to get consensus on a format for the platform 
> tag information used by setuptools.  (The first time was before 
> easy_install even existed.)  Unfortunately, it's rare that anybody 
> contributes more than criticism of the existing scheme.  The last 
> updates to the scheme were by Mac folks, who at least had a clear 
> vision of what was needed.  There's been some win32/win64 discussion 
> in the past, but no proposals or patches of comparable 
> specificity.  

Consider the fact that Mac users won't answer or discuss posts by some 
windows users..

Why do this ? Why stir up platform bias?

Fact is Phil, windows users get *turned-away* at the door in
distutils... or ignored. Issues can hang around for years.

And your post is now asking "why no windows users?"

Go figure.. nobody answers them or takes issues seriously.

Distutils for windows is very, very dead.. grave-ware in-fact.

It should be quite obvious that windows users are forked off.. 

Why would windows people make proposals when they know it 
won't go anywhere...

David





From ubershmekel at gmail.com  Wed Oct  7 09:08:37 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Wed, 7 Oct 2009 09:08:37 +0200
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <4AC3F25D.8000907@canterbury.ac.nz>
References: <20090929202035.GA5492@gmail.com>
	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
	<9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
	<4AC3F25D.8000907@canterbury.ac.nz>
Message-ID: <9d153b7c0910070008l70b115cfma791e892ea25e54e@mail.gmail.com>

This thread moved to python-ideas so please post only there.

http://mail.python.org/pipermail/python-ideas/2009-October/005924.html

--yuv

From mal at egenix.com  Wed Oct  7 09:45:13 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 09:45:13 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <75c06ee14c3ef81164d5b4a9f3177799@preisshare.net>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>
	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<20091006175244.D1FF03A4116@sparrow.telecommunity.com>
	<75c06ee14c3ef81164d5b4a9f3177799@preisshare.net>
Message-ID: <4ACC4709.4090405@egenix.com>

David Lyon wrote:
> Distutils for windows is very, very dead.. grave-ware in-fact.

Now that is not true at all. We have a native Windows installer
(bdist_wininst) and an MSI builder (bdist_msi) that both work
great on Windows.

Plus there are add-ons for other installers such as NSIS and
InnoSetup.

Please don't forget that setuptools is not distutils, it's just
a package extending distutils. If you find bugs related to Windows,
please file a bug report. If you have specific proposals for
distutils on Windows, you can use the mailing list or send
patches to the tracker.

The ham/egg ratio on the mailing list tends to be very much
on the egg side, so the tracker can help get more attention from
folks doing distutils without setuptools.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From benjamin at python.org  Mon Oct  5 03:54:34 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 4 Oct 2009 20:54:34 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ee9557410910041852i15f15471y36648c224f5ef482@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T024614-990@post.gmane.org> <ha449d$9n5$1@ger.gmane.org>
	<d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<loom.20091002T113352-479@post.gmane.org>
	<d11dcfba0910021023t67409839oee07e2e64b9aeb5d@mail.gmail.com>
	<ECBF427B08D9436E9E5CD4E2002BD8B4@RaymondLaptop1>
	<87y6nt37kb.fsf@hbox.dyndns.org>
	<426ada670910021323g3ab0d206t46a00d6210260dd0@mail.gmail.com>
	<ee9557410910041852i15f15471y36648c224f5ef482@mail.gmail.com>
Message-ID: <1afaf6160910041854v55edcc96we4fb2d6a7f2da948@mail.gmail.com>

2009/10/4 INADA Naoki <songofacandy at gmail.com>:
> What about using string prefix 'f'?
>
> ?f"{foo} and {bar}" % something == "{foo} and {bar}.format(something)
>
> ?s = f"{foo}"
> ?t = "%(bar)s"
> ?s + t ?# raises Exception
>
> Transition plan:
> n: Just add F prefix. And adding "format_string" in future.
> n+1: deprecate __mod__() without 'F'.
> n+2: libraries use .format() and deprecate __mod__() with 'F'
> n+3: remove __mod__()

-1 That requires keeping formatting information around in every string instance.


-- 
Regards,
Benjamin

From hrvoje.niksic at avl.com  Wed Oct  7 13:26:59 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 07 Oct 2009 13:26:59 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	<4AC2846E.8060304@g.nevcal.com>	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>	<ha01t7$e2j$1@ger.gmane.org>
	<79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
Message-ID: <4ACC7B03.3040700@avl.com>

Paul Moore wrote:
> Traceback (most recent call last):
>   File "hello.py", line 13, in <module>
>     main()
>   File "hello.py", line 7, in main
>     sys.stdout.flush()
> IOError: [Errno 9] Bad file descriptor
> 
> (Question - is it *ever* possible for a Unix program to have invalid
> file descriptors 0,1 and 2? At startup - I'm assuming anyone who does
> os.close(1) knows what they are doing!)

Of course; simply use the >&- pseudo-redirection, which has been a 
standard sh feature (later inherited by ksh and bash, but not csh) for 
~30 years.  The error message is amusing, too:

$ python -c 'print "foo"' >&-
close failed in file object destructor:
Error in sys.excepthook:

Original exception was:


Adding an explicit flush results in a more understandable error message:

Traceback (most recent call last):
   File "<string>", line 1, in <module>
IOError: [Errno 9] Bad file descriptor

From swapnil.st at gmail.com  Wed Oct  7 16:34:48 2009
From: swapnil.st at gmail.com (Swapnil Talekar)
Date: Wed, 7 Oct 2009 20:04:48 +0530
Subject: [Python-Dev] Python byte-compiled and optimized code
Message-ID: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>

I am working on deploying Python on VxWorks platform as a part of project
for my company. Accordingly, I would like to know couple of things from
python's core-developers. Although I think I already know the answers for
most of the questions, we need a confirmation from the community
1) Is the byte-compiled .pyc file and optimized .pyo file
platform-independent?(including python versions 3.x) *If yes, is it
guaranteed to stay that way in future?*
2) If the the generation of .pyc file fails (say, due to write protected
area), does that stop the script execution?
3) Is it possible to redirect the location of the generation of .pyc files
to other than that of the corresponding .py files?



Regards,
Swapnil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/03b6169f/attachment.htm>

From vinay_sajip at yahoo.co.uk  Wed Oct  7 16:49:44 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 7 Oct 2009 14:49:44 +0000 (UTC)
Subject: [Python-Dev] A new way to configure logging
Message-ID: <loom.20091007T164835-226@post.gmane.org>

At present, configuration of Python's logging package can be done in one of two
ways:

1. Create a ConfigParser-readable configuration file and use
logging.config.fileConfig() to read and implement the configuration therein.
2. Use the logging API to programmatically configure logging using getLogger(),
addHandler() etc.

The first of these works for simple cases but does not cover all of the logging
API (e.g. Filters). The second of these provides maximal control, but besides
requiring users to write the configuration code, it fixes the configuration in
Python code and does not facilitate changing it easily at runtime.

In addition, the ConfigParser format appears to engender dislike (sometimes
strong dislike) in some quarters. Though it was chosen because it was the only
configuration format supported in the stdlib at that time, many people regard it
(or perhaps just the particular schema chosen for logging's configuration) as
'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent
versions of Python of course support an additional battery-included format which
can be used for configuration - namely, JSON. Other options, such as YAML, are
also possible ways of configuring systems, Google App Engine-style, and PyYAML
has matured nicely.

There has also been talk on the django-dev mailing list about providing better
support for using Python logging in Django. When it happens (as of course I hope
it does) this has the consequence that many new users who use Django but are
relatively inexperienced in Python (e.g. in PHP shops which are moving to
Django) will become exposed to Python logging. As Django is configured using a
Python module and use of ConfigParser-style files is not a common approach in
that ecosystem, users will find either of the two approaches outlined above a
particular pain point when configuring logging for their Django applications and
websites, unless something is done to avoid it.

All three of the contenders for the title of "commonly found configuration
mechanism" - JSON, YAML and Python code - will be expressible, in Python, as
Python dicts. So it seems to make sense to add, to logging.config, a new
callable bound to "dictConfig" which will take a single dictionary argument and
configure logging from that dictionary.

An important facet of implementing such a scheme will be the format or schema
which the dictionary has to adhere to. I have started working on what such a
schema would look like, and if people here think it's a good idea to go ahead
with this, I'll provide the details of the schema on a separate post which I'll
also cross-post on comp.lang.python so that I can get feedback from there, too.

In outline, the scheme I have in mind will look like this, in terms of the new
public API:

class DictConfigurator:
    def __init__(self, config): #config is a dict-like object (duck-typed)
        import copy
        self.config = copy.deepcopy(config)
        
    def configure(self):
        #  actually do the configuration here using self.config

dictConfigClass = DictConfigurator

def dictConfig(config):
    dictConfigClass(config).configure()

This allows easy replacement of DictConfigurator with a suitable subclass where
needed.

What's the general feeling here about this proposal? All comments and
suggestions will be gratefully received.

Regards,


Vinay Sajip



From fdrake at gmail.com  Wed Oct  7 17:12:09 2009
From: fdrake at gmail.com (Fred Drake)
Date: Wed, 7 Oct 2009 11:12:09 -0400
Subject: [Python-Dev] Python byte-compiled and optimized code
In-Reply-To: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>
References: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>
Message-ID: <9cee7ab80910070812y7d9e2504qca040f981adc0f0d@mail.gmail.com>

On Wed, Oct 7, 2009 at 10:34 AM, Swapnil Talekar <swapnil.st at gmail.com> wrote:
> 1) Is the byte-compiled .pyc file and optimized .pyo file
> platform-independent?(including python versions 3.x)

Yes.

> If yes, is it
> guaranteed to stay that way in future?

Yes.

> 2)?If the the generation of .pyc file fails (say, due to write protected
> area), does that stop the script execution?

No.  It only means the bytecode won't be cached.

> 3) Is it possible to redirect the location of the generation of .pyc files
> to other than that of the corresponding .py files?

I think some support for this has been developed, at least
experimentally, but I'm not sure if it's part of a stable release or
not.  It's more likely in Python 3.x, which I'm significantly less
familiar with.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From olemis at gmail.com  Wed Oct  7 17:19:27 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 7 Oct 2009 10:19:27 -0500
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <loom.20091007T164835-226@post.gmane.org>
References: <loom.20091007T164835-226@post.gmane.org>
Message-ID: <24ea26600910070819r364f6715l9164a754bc20b8f4@mail.gmail.com>

On Wed, Oct 7, 2009 at 9:49 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> At present, configuration of Python's logging package can be done in one of two
> ways:
>
> 1. Create a ConfigParser-readable configuration file and use
> logging.config.fileConfig() to read and implement the configuration therein.
> 2. Use the logging API to programmatically configure logging using getLogger(),
> addHandler() etc.
>
[...]

Wow ! Great explanation !
;o)

>
> All three of the contenders for the title of "commonly found configuration
> mechanism" - JSON, YAML and Python code - will be expressible, in Python, as
> Python dicts.

.INI files too

{
  'section1' :
    {'opt11' : 'val1', 'opt12' : 32323},
  'section1' :
    {'opt21' : 'val2', 'opt22' : 32323},
  'section1' :
    {'opt31' : 'val3', 'opt32' : 32323},
...
}

In fact it seems that this is a typical characteristic of config formats (CMIIW)

> So it seems to make sense to add, to logging.config, a new
> callable bound to "dictConfig" which will take a single dictionary argument and
> configure logging from that dictionary.
>

This kind of problems is similar to the one mentioned in another
thread about modifying config options after executing commands. In
that case I mentioned that the same dict-like interface also holds for
WinReg and so on ...

So thinking big (yes ! I have a big head ! ) I think that the best
approach in this case is to build an adaptation (simple) layer on top
of ConfigParser, JSON, WinReg, PyCode, YAML, ... and build specific
extensions for these formats . Perhaps the proper interfaces are
already there (e.g. `dict`, `shelve` ) and I'm just blind and looking
somewhere else ;o)

If `dict` (possibly nested ;o) is enough to represent config files
then the new method should be ok ... IMHO

[...]
>
> In outline, the scheme I have in mind will look like this, in terms of the new
> public API:
>
> class DictConfigurator:
> ? ?def __init__(self, config): #config is a dict-like object (duck-typed)

========

> ? ? ? ?import copy
> ? ? ? ?self.config = copy.deepcopy(config)

Why ?

>
> ? ?def configure(self):
> ? ? ? ?# ?actually do the configuration here using self.config
>
> dictConfigClass = DictConfigurator
>
> def dictConfig(config):
> ? ?dictConfigClass(config).configure()
>
> This allows easy replacement of DictConfigurator with a suitable subclass where
> needed.
>

extension is cool ... what's the point about adding the new method
instead of using `DictConfigurator` directly ?

> What's the general feeling here about this proposal?
>

I'm feeling things ... :)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Nabble - Trac Users - Coupling trac and symfony framework  -
http://feedproxy.google.com/~r/TracGViz-full/~3/hlNmupEonF0/Coupling-trac-and-symfony-framework-td25431579.html

From solipsis at pitrou.net  Wed Oct  7 17:24:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 7 Oct 2009 15:24:27 +0000 (UTC)
Subject: [Python-Dev] Python byte-compiled and optimized code
References: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>
	<9cee7ab80910070812y7d9e2504qca040f981adc0f0d@mail.gmail.com>
Message-ID: <loom.20091007T172334-197@post.gmane.org>

Fred Drake <fdrake <at> gmail.com> writes:
> > 3) Is it possible to redirect the location of the generation of .pyc files
> > to other than that of the corresponding .py files?
> 
> I think some support for this has been developed, at least
> experimentally, but I'm not sure if it's part of a stable release or
> not.  It's more likely in Python 3.x, which I'm significantly less
> familiar with.

I haven't seen any such thing.
If you need pyc files to reside in read-only directories, the best solution
would be to generate them at install time (when you have write access to the
directories).

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Wed Oct  7 17:49:56 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 7 Oct 2009 15:49:56 +0000 (UTC)
Subject: [Python-Dev] A new way to configure logging
References: <loom.20091007T164835-226@post.gmane.org>
	<24ea26600910070819r364f6715l9164a754bc20b8f4@mail.gmail.com>
Message-ID: <loom.20091007T174509-394@post.gmane.org>

Olemis Lang <olemis <at> gmail.com> writes:

> This kind of problems is similar to the one mentioned in another
> thread about modifying config options after executing commands. In
> that case I mentioned that the same dict-like interface also holds for
> WinReg and so on ...
> 
> So thinking big (yes ! I have a big head ! ) I think that the best
> approach in this case is to build an adaptation (simple) layer on top
> of ConfigParser, JSON, WinReg, PyCode, YAML, ... and build specific
> extensions for these formats . Perhaps the proper interfaces are
> already there (e.g. `dict`, `shelve` ) and I'm just blind and looking
> somewhere else ;o)

Sorry, you've lost me :-)

> > ? ? ? ?import copy
> > ? ? ? ?self.config = copy.deepcopy(config)
> 
> Why ?

So I'm free to mutate self.config as I see fit.
 
> extension is cool ... what's the point about adding the new method
> instead of using `DictConfigurator` directly ?

When you say "the new method", if you mean "configure" - the pattern is so that
a subclass can override __init__ and do additional setup before configure() is
called.

Regards,

Vinay Sajip



From ksande at uva.nl  Wed Oct  7 16:54:58 2009
From: ksande at uva.nl (Koen van de Sande)
Date: Wed, 07 Oct 2009 16:54:58 +0200
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
References: <BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
Message-ID: <4ACCABC2.6030402@uva.nl>

 > If setuptools can be made to work with Python 2.6.3 /and/ earlier 
 > versions of Python 2.6.x, then it should be patched and an update 
 > released.  If not, then perhaps we should revert the change in a quick 
 > Python 2.6.4.

If there is going to be any quick 2.6.4 release, can you consider a fix 
to the building of extensions under Windows ( 
http://bugs.python.org/issue4120 )?
Extensions built without this patch applied will not work if the MSVC9 
runtimes are installed locally (in the Python folder) instead of 
system-wide (WinSxS).
PIL and Matplotlib now use this patch when building extensions. But many 
others (e.g. PyGame) still create problematic extensions.

-Koen

From paul at rudin.co.uk  Wed Oct  7 18:06:56 2009
From: paul at rudin.co.uk (Paul Rudin)
Date: Wed, 07 Oct 2009 17:06:56 +0100
Subject: [Python-Dev] A new way to configure logging
References: <loom.20091007T164835-226@post.gmane.org>
Message-ID: <87ws37dw7z.fsf@rudin.co.uk>

Vinay Sajip <vinay_sajip at yahoo.co.uk> writes:


> What's the general feeling here about this proposal? All comments and
> suggestions will be gratefully received.
>

How about the global logging configuration being available as an object
supporting the usual dictionary interface? So you could, for example, do
something like: "logging.dictconfig.update(partialconfig)"


From olemis at gmail.com  Wed Oct  7 18:26:42 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 7 Oct 2009 11:26:42 -0500
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <loom.20091007T174509-394@post.gmane.org>
References: <loom.20091007T164835-226@post.gmane.org>
	<24ea26600910070819r364f6715l9164a754bc20b8f4@mail.gmail.com>
	<loom.20091007T174509-394@post.gmane.org>
Message-ID: <24ea26600910070926g6b742492xd6be4fa07a932a0c@mail.gmail.com>

On Wed, Oct 7, 2009 at 10:49 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Olemis Lang <olemis <at> gmail.com> writes:
>
>> This kind of problems is similar to the one mentioned in another
>> thread about modifying config options after executing commands. In
>> that case I mentioned that the same dict-like interface also holds for
>> WinReg and so on ...
>>
>> So thinking big (yes ! I have a big head ! ) I think that the best
>> approach in this case is to build an adaptation (simple) layer on top
>> of ConfigParser, JSON, WinReg, PyCode, YAML, ... and build specific
>> extensions for these formats . Perhaps the proper interfaces are
>> already there (e.g. `dict`, `shelve` ) and I'm just blind and looking
>> somewhere else ;o)
>
> Sorry, you've lost me :-)
>

Never mind . I was just trying to say that using `dict`, an adapter
could be implemented (if not already there ;o) for multiple formats
like the ones I mentioned above and the solution would cover many
config formats ... and also I was saying that I was not sure about
whether this is correct , I mean for *ANY* config formats, but
definitely will work for many ;o)

>> > ? ? ? ?import copy
>> > ? ? ? ?self.config = copy.deepcopy(config)
>>
>> Why ?
>
> So I'm free to mutate self.config as I see fit.
>

why not to let the user do it if he | she thinks so. I mean if
somebody supplies in a temporary mapping that can be written then why
to spend that time cloning the dict ? If the function has
side-effects, well just document it ;o)

>> extension is cool ... what's the point about adding the new method
>> instead of using `DictConfigurator` directly ?
>
> When you say "the new method", if you mean "configure" - the pattern is so that
> a subclass can override __init__ and do additional setup before configure() is
> called.
>

Sorry I was confused. I was talking about `dictConfig` *FUNCTION* (to
be added to logging.config ... isn't it ? ;o). I mean if dictConfig is
sematically equivalent to a simple `dc.configure` why to add such
function ?

PS: Not a big problem anyway ;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Looking for a technique to create flexible, graphical dashboards ...
- http://feedproxy.google.com/~r/TracGViz-full/~3/71kRhT34BgU/43789

From olemis at gmail.com  Wed Oct  7 18:29:26 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 7 Oct 2009 11:29:26 -0500
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <87ws37dw7z.fsf@rudin.co.uk>
References: <loom.20091007T164835-226@post.gmane.org>
	<87ws37dw7z.fsf@rudin.co.uk>
Message-ID: <24ea26600910070929u383b3718yc5b7d573149677a@mail.gmail.com>

On Wed, Oct 7, 2009 at 11:06 AM, Paul Rudin <paul at rudin.co.uk> wrote:
> Vinay Sajip <vinay_sajip at yahoo.co.uk> writes:
>
>
>> What's the general feeling here about this proposal? All comments and
>> suggestions will be gratefully received.
>>
>
> How about the global logging configuration being available as an object
> supporting the usual dictionary interface? So you could, for example, do
> something like: "logging.dictconfig.update(partialconfig)"
>

Seems interesting ... yes ! ;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Search for milestone change in change history - Trac Users ...  -
http://feedproxy.google.com/~r/TracGViz-full/~3/Gsj4xLjuCtk/578d40b734e5e753

From zookog at gmail.com  Wed Oct  7 18:56:49 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Wed, 7 Oct 2009 10:56:49 -0600
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091006211742.9E43E3A4116@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
Message-ID: <cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>

+1

For a large number of people [1, 2, 3], setuptools is already a
critical part of Python.  Make it official.  Let everyone know that
future releases of Python will not break setuptools/Distribute, and
that they can rely on backwards-compatibility with the myriad existing
packages.  Make the next release of the distutils standard lib module
be Distribute.

(Perhaps some people will complain, but they can channel their energy
into improving the new distutils.)

Regards,

Zooko

[1] The majority of professional developers using Python rely on
setuptools to distribute and to use packages:
     http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first-results/
[2] setuptools is one of the most downloaded packages on PyPI:
     http://pypi.python.org/pypi/setuptools
     http://blog.delaguardia.com.mx/tags/pypi
[3] about one fifth of Debian users who install python2.5 also install
python-pkg-resources:
     http://qa.debian.org/popcon.php?package=python-setuptools
     http://qa.debian.org/popcon.php?package=python2.5

From zookog at gmail.com  Wed Oct  7 19:07:37 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Wed, 7 Oct 2009 11:07:37 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default to
	ucs4
In-Reply-To: <cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
Message-ID: <cd6401a0910071007x30ab6e91wdd8098c5ca8489d4@mail.gmail.com>

Folks:

I accidentally sent this letter just to MAL when I intended it to
python-dev.  Please read it, as it explains why the issue I'm raising
is not just the "we should switch to ucs4 because it is better" issue
that was previously settled by GvR.  This is a current, practical
problem that is preventing people from distributing and using Python
packages with binary extension modules on Linux.

Regards,

Zooko


---------- Forwarded message ----------
From: Zooko O'Whielacronx <zookog at gmail.com>
Date: Sun, Sep 27, 2009 at 11:43 AM
Subject: Re: [Python-Dev] please consider changing --enable-unicode
default to ucs4
To: "M.-A. Lemburg" <mal at egenix.com>


Folks:

I'm sorry, I think I didn't make my concern clear. ?My users, and lots
of other users, are having a problem with incompatibility between
Python binary extension modules. ?One way to improve the situation
would be if the Python devs would use their "bully pulpit" -- their
unique position as a source respected by all Linux distributions --
and say "We recommend that Linux distributions use UCS4 for
compatibility with one another". ?This would not abrogate anyone's
ability to choose their preferred setting nor, as far as I can tell,
would it interfere with the ongoing development of Python.

Here are the details:

I'm the maintainer of several Python packages. ?I work hard to make it
easy for users, even users who don't know anything about Python, to
use my software. ?There have been many pain points in this process and
I've spent a lot of time on it for about three years now working on
packaging, including the tools such as setuptools and distutils and
the new "distribute" tool. ?Python packaging has been improving during
these years -- things are looking up.

One of the remaining pain points is that I can distribute binaries of
my Python extension modules for Windows or Mac, but if I distribute a
binary Python extension module on Linux, then if the user has a
different UCS2/UCS4 setting then they won't be able to use the
extension module. ?The current de facto standard for Linux is UCS4 --
it is used by Debian, Ubuntu, Fedora, RHEL, OpenSuSE, etc. etc.. ?The
vast majority of Linux users in practice have UCS4, and most binary
Python modules are compiled for UCS4.

That means that a few folks will get left out. ?Those folks, from my
experience, are people who built their python executable themselves
without specifying an override for the default, and the smaller Linux
distributions who insist on doing whatever upstream Python devs
recommend instead of doing whatever the other Linux distros are doing.
?One of the data points that I reported was a Python interpreter that
was built locally on an Ubuntu server. ?Since the person building it
didn't know to override the default setting of --enable-unicode, he
ended up with a Python interpreter built for UCS2, even though all the
Python extension modules shipped by Ubuntu were built with UCS4.

These are not isolated incidents. ?The following google searches
suggest that a number of people spend time trying to figure out why
Python extension modules fail on their linux systems:

http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol
http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol
http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol

Another data point is the Mandriva Linux distribution. ?It is probably
much smaller than Debian, Ubuntu, or RedHat, but it is still one of
the major, well-known distributions. ?I requested of the Python
maintainer for Mandriva, Michael Scherer, that they switch from UCS2
to UCS4 in order to reduce compatibility problems like these. ?His
answer as I understood it was that it is best to follow the
recommendations of the upstream Python devs by using the default
setting instead of choosing a setting for himself.

(Now we could implement a protocol which would show whether a given
Python package was compiled for UCS2 or UCS4. ?That would be good.
Hopefully it would make incompatibility more explicit and
understandable to users. ?Here is a ticket for that -- which project I
am contributing to: http://bugs.python.org/setuptools/issue78 .
However, even if we implement that feature in the distribute tool (the
successor to setuptools), users who build their own python or who use
a Linux distribution that follows upstream configuration defaults will
still be unable to use most Python packages with compiled extension
modules.)

In a message on this thread, MvL wrote:

> "For that reason I think it's also better that the configure script
> continues to default to UTF-16 -- this will give the UTF-16 support
> code the necessary exercise."
>
> This is effectively a BDFL pronouncement. Nothing has changed the
> validity of the premise of the statement, so the conclusion remains
> valid, as well.

My understand of the earlier thread was that someone suggested that
UCS4 would be technically better and GvR decided that there were
technical reasons to continue actively maintaining the UCS2 code.
This thread is different: I'm saying that users are suffering
packaging problems and asking for help with that. ?The way that
python-dev can help is to make it so that people who choose "Whatever
the upstream devs prefer" (--enable-unicde) on Linux get the de facto
standard setting.

In the earlier thread, GvR wrote: "I think we should continue to leave
this up to the distribution. AFAIK many Linux distros already use UCS4
for everything anyway.". ?But at least some distributions are asking
the upstream Python devs to choose for them, by leaving the setting at
the default.

Hm, pondering GvR's words: "I think we should continue to leave this
up to the distribution", I have a new proposal: make it so that *on
Linux only* "--enable-unicode" errors out with an error message saying
"please choose either --enable-unicode=ucs2 or --enable-unicode=ucs4.
ucs4 is the most widely used setting on Linux. See
http://python.org/wiki/UCS2_vs_UCS4 for details.". ?This would force
those Linux distributions who are *not* currently deciding to decide.

Thank you for your attention.

Regards,

Zooko

From tjreedy at udel.edu  Wed Oct  7 18:54:32 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 07 Oct 2009 12:54:32 -0400
Subject: [Python-Dev] Python byte-compiled and optimized code
In-Reply-To: <9cee7ab80910070812y7d9e2504qca040f981adc0f0d@mail.gmail.com>
References: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>
	<9cee7ab80910070812y7d9e2504qca040f981adc0f0d@mail.gmail.com>
Message-ID: <haih46$ef0$1@ger.gmane.org>

Fred Drake wrote:
> On Wed, Oct 7, 2009 at 10:34 AM, Swapnil Talekar <swapnil.st at gmail.com> wrote:
>> 1) Is the byte-compiled .pyc file and optimized .pyo file
>> platform-independent?(including python versions 3.x)
> 
> Yes.

To be clear, CPython's .pyc is platform independent for a particular x.y 
version, and should, I believe, stay the same for all x.y.z releases,
but the format of .pyc may change slightly from x.y to x.(y+1)


From vinay_sajip at yahoo.co.uk  Wed Oct  7 18:55:48 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 7 Oct 2009 16:55:48 +0000 (UTC)
Subject: [Python-Dev] A new way to configure logging
References: <loom.20091007T164835-226@post.gmane.org>
	<87ws37dw7z.fsf@rudin.co.uk>
Message-ID: <loom.20091007T183353-798@post.gmane.org>

Paul Rudin <paul at ...> writes:

> How about the global logging configuration being available as an object
> supporting the usual dictionary interface? So you could, for example, do
> something like: "logging.dictconfig.update(partialconfig)"

A "partial configuration" only makes sense under certain limited conditions. For
example, a dict used for configuration will map to a graph of objects - filters,
formatters, handlers and loggers - which by necessity need to be referred to via
a string key in the dict - because you can't easily encode a graph in a dict
which comes from, say, a YAML or JSON file. These keys are temporary and just
used to indicate e.g. which handlers to attach to which loggers for that
particular configuration call:

{
  "handlers": {
    "h1": {
       configuration for a handler
    },
    "h2": {
       configuration for another handler
    },
    "h3": {
       configuration for yet another handler
    },
  },
  "loggers": {
    "foo": {
      "level": "DEBUG",
      "handlers": ["h1", "h3"],
    },
    "bar.baz": {
      "level": "INFO",
      "handlers": ["h2", "h3"],
    },
    "spam.eggs": {
      "level": "ERROR",
      "handlers": ["h1", "h3"],
    },
  }
}

Here, the keys "h1", "h2" etc. are just used to encode connections in the graph,
and are useless outside the context of the specific configuration call. If you
pass a partial configuration, to be implemented incrementally, there's no way of
knowing if (in general) it's going to be consistent with the existing
configuration. Nor can you refer to e.g. handlers, formatters or filters you
passed in earlier configuration calls.

Since logger names such as "foo", "bar.baz" in the above example *are*
meaningful across multiple configuration calls, it *would be* possible to make
certain changes in the configuration - such as changing levels of loggers - by
passing in a partial configuration. However, if this "partial" functionality is
provided without restriction, it's hard to ensure that a user doesn't
inadvertently misconfigure logging. And while that is equally true with a
one-shot configuration, it's at least easier to debug. Things can get pretty
murky from a diagnostic point of view if code can make arbitrary changes to the
logging configuration during application execution, so that's not a good
practice to encourage :-)

Also, providing a readable dict-like interface to the internal configuration
will almost certainly lead to backward-compatibility headaches in the future.

Regards,


Vinay Sajip



From phd at phd.pp.ru  Wed Oct  7 19:12:48 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 7 Oct 2009 21:12:48 +0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
References: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
Message-ID: <20091007171248.GB1901@phd.pp.ru>

On Wed, Oct 07, 2009 at 10:56:49AM -0600, Zooko O'Whielacronx wrote:
> For a large number of people [1, 2, 3], setuptools is already a
> critical part of Python.  Make it official.  Let everyone know that
> future releases of Python will not break setuptools/Distribute, and
> that they can rely on backwards-compatibility

   Who will pay the price of maintaining that part of stdlib and the price
of compatibility? Because there is always a price; should the core
developers be burdened?

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From barry at python.org  Wed Oct  7 19:18:04 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 7 Oct 2009 13:18:04 -0400
Subject: [Python-Dev] Python 2.6.4rc1
Message-ID: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>

Hello everyone.

The source tarballs and Windows installers for Python 2.6.4rc1 are now  
available:

http://www.python.org/download/releases/2.6.4/

Please download them, install them, and try to use them with your  
projects and environments.  Let us know if you encounter any problems  
with them.  Hopefully we can avoid the situation with 2.6.3 having  
such critical bugs.

2.6.4 final is planned for 18-October.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/4ddcec76/attachment-0001.pgp>

From scott+python-dev at scottdial.com  Wed Oct  7 19:22:12 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 07 Oct 2009 13:22:12 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091006211742.9E43E3A4116@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	<4AC371D7.9000904@simplistix.co.uk>	<4ACB49FB.3000407@simplistix.co.uk>	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
Message-ID: <4ACCCE44.8050905@scottdial.com>

P.J. Eby wrote:
> At 01:14 PM 10/6/2009 -0700, Guido van Rossum wrote:
>> suggest nobody hold their breath waiting for setuptools 0.7.
> 
> I've never suggested or implied otherwise.
> 
> But, if you like Distribute so much, why not just add it directly to the
> stdlib?  ;-)
> 
> There are many wins to be had from such a move, not the least of which
> is that it allows something to go into the stdlib that isn't
> (branding-wise) associated with me or setuptools, and lets it become
> owned/supported by an even wider community.

I think the biggest problem here is that the brand ("setuptools") is so
ingrained in the world that someone releasing something
setuptools-but-not-setuptools ("Distribute") is at a serious
disadvantage. Your insistence on retaining the name "setuptools" for
yourself and your vapor-ware only harms the community at-large.

Even experienced developers are unaware of Distribute's existence.. I
was entirely unaware until yourself and PJE got in a bickering exchange
on Python-Dev this past month. The extremely high visibility of your
stale package compared to their non-stale package is quite unfortunate.
You are free to say, "that is their problem," but you are not free to
say, "it is not my fault."

> AFAIK, the only reason they've had multiple releases of it is to
> address the issues with its hijacking of setuptools; in a stdlib
> version all that stuff could be dropped.  Otherwise, it'd already be
> "mature".

IOW, you acknowledge that your own position and requiring them to
tolerate the parallel existence (in the world) of setuptools harms
Distribute. I fail to see how this relates to being in the stdlib. As
long as it is not called "setuptools" and packages in the world say
"import setuptools", then there are conflicts they will be forced to
managed. And moreover, as long as a stale, perhaps buggy version of
setuptools is the compatibility model they must emulate, they will have
a hard time coexisting.

> Less political bickering, and the some of the technical results I
> hoped for all along are achieved.  Yay, open source.

And yet, political bickering seems to be all you are good for in this case.

</flame>

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From solipsis at pitrou.net  Wed Oct  7 19:21:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 7 Oct 2009 17:21:53 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091005T102425-605@post.gmane.org>
Message-ID: <loom.20091007T191603-331@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
> 
> >>> "%0#8x" % 0x1234
> '0x001234'
> >>> "{0:0>#8x}".format(0x1234)
> '000x1234'

Apart from the sheer unreadability of the {}-style format string, the result 
looks rather unexpected from a human being's point of view.

(in those situations, I would output the 0x manually anyway, such as:
>>> "0x%06x" % 0x1234
'0x001234'
>>> "0x{:06x}".format(0x1234)
'0x001234'
)

Regards

Antoine.



From scott+python-dev at scottdial.com  Wed Oct  7 19:26:01 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 07 Oct 2009 13:26:01 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
Message-ID: <4ACCCF29.7030003@scottdial.com>

Barry Warsaw wrote:
> 2.6.4 final is planned for 18-October.

Barry,

I suspect this release is primarily to quench the problems with
distutils, but..

http://bugs.python.org/issue5949

doesn't seem to have been addressed by you. And this seems like it would
be another unfortunate loss of an opportunity.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From mal at egenix.com  Wed Oct  7 19:27:55 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 19:27:55 +0200
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	<4AC371D7.9000904@simplistix.co.uk>	<4ACB49FB.3000407@simplistix.co.uk>	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
Message-ID: <4ACCCF9B.5060303@egenix.com>

Zooko O'Whielacronx wrote:
> +1
> 
> For a large number of people [1, 2, 3], setuptools is already a
> critical part of Python.  Make it official.  Let everyone know that
> future releases of Python will not break setuptools/Distribute, and
> that they can rely on backwards-compatibility with the myriad existing
> packages.  Make the next release of the distutils standard lib module
> be Distribute.
> 
> (Perhaps some people will complain, but they can channel their energy
> into improving the new distutils.)

We've had that discussion in past and a few times. setuptools/Distribute
is not distutils, just an add-on (and there are others as well).

There are good concepts in setuptools, but the implementation is just
not suitable for stdlib integration.

Hopefully, Distribute can fix the hacks, trim down the number of
features to what people really use, add a proper uninstall command and
then we can have the add-to-the-stdlib discussion again.

Having more competition will also help, e.g. ActiveState's PyPM looks
promising (provided they choose to open-source it) and then there's
pip.

Things look promising, indeed.

> Regards,
> 
> Zooko
> 
> [1] The majority of professional developers using Python rely on
> setuptools to distribute and to use packages:
>      http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first-results/
> [2] setuptools is one of the most downloaded packages on PyPI:
>      http://pypi.python.org/pypi/setuptools
>      http://blog.delaguardia.com.mx/tags/pypi
> [3] about one fifth of Debian users who install python2.5 also install
> python-pkg-resources:
>      http://qa.debian.org/popcon.php?package=python-setuptools
>      http://qa.debian.org/popcon.php?package=python2.5
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From vinay_sajip at yahoo.co.uk  Wed Oct  7 19:34:02 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 7 Oct 2009 17:34:02 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091005T102425-605@post.gmane.org>
	<loom.20091007T191603-331@post.gmane.org>
Message-ID: <loom.20091007T193137-190@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> 
> Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
> > 
> > >>> "%0#8x" % 0x1234
> > '0x001234'
> > >>> "{0:0>#8x}".format(0x1234)
> > '000x1234'
> 
> Apart from the sheer unreadability of the {}-style format string, the result 
> looks rather unexpected from a human being's point of view.
> 
> (in those situations, I would output the 0x manually anyway, such as:
> >>> "0x%06x" % 0x1234
> '0x001234'
> >>> "0x{:06x}".format(0x1234)
> '0x001234'
> )
> 

Well of course, but I asked the question in the context of providing an
*automatic* converter from %-format strings to {}-format. At the moment, it
doesn't seem like a 100%-faithful automatic conversion is feasible.

Regards,

Vinay Sajip


From ziade.tarek at gmail.com  Wed Oct  7 19:37:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 7 Oct 2009 19:37:07 +0200
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4ACCCF9B.5060303@egenix.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
	<4ACCCF9B.5060303@egenix.com>
Message-ID: <94bdd2610910071037h19b014c3i67bf402169eada9f@mail.gmail.com>

On Wed, Oct 7, 2009 at 7:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Zooko O'Whielacronx wrote:
>> +1
>>
>> For a large number of people [1, 2, 3], setuptools is already a
>> critical part of Python. ?Make it official. ?Let everyone know that
>> future releases of Python will not break setuptools/Distribute, and
>> that they can rely on backwards-compatibility with the myriad existing
>> packages. ?Make the next release of the distutils standard lib module
>> be Distribute.
>>
>> (Perhaps some people will complain, but they can channel their energy
>> into improving the new distutils.)
>
> We've had that discussion in past and a few times. setuptools/Distribute
> is not distutils, just an add-on (and there are others as well).
>
> There are good concepts in setuptools, but the implementation is just
> not suitable for stdlib integration.
>
> Hopefully, Distribute can fix the hacks, trim down the number of
> features to what people really use, add a proper uninstall command and
> then we can have the add-to-the-stdlib discussion again.
>
> Having more competition will also help, e.g. ActiveState's PyPM looks
> promising (provided they choose to open-source it) and then there's
> pip.
>
> Things look promising, indeed.
>

100% Agreed !

And before considering adding "bits" of Distribute (or Setuptools) in Distutils,
we need to finish our work on PEP 376 and PEP 386, and the changes
we've planned for PEP 341.

The work we've done there so far in these PEPs is basically trying to
have a consensus
on the missing standard that is lacking in Distutils, and that was
partially solved by Setuptools.

I have good hopes that these PEP will be finished before 2.7 is out.

Notice that PEP 382 (namespaced packages) is already a great step forward.
(by the way, I thought this one was accepted, shouldn't its status updated ?)

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From zookog at gmail.com  Wed Oct  7 19:37:14 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Wed, 7 Oct 2009 11:37:14 -0600
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4ACCCF9B.5060303@egenix.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
	<4ACCCF9B.5060303@egenix.com>
Message-ID: <cd6401a0910071037j5b0226dfl91113b27dc6d518b@mail.gmail.com>

Thanks for the reply, MAL.

How would we judge whether Distribute is ready for inclusion in the
Python standard lib?  Maybe if it has a few more releases, leaving a
trail of "closed: fixed" issue tickets behind it?

Regards,

Zooko

From zookog at gmail.com  Wed Oct  7 19:41:06 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Wed, 7 Oct 2009 11:41:06 -0600
Subject: [Python-Dev] Python byte-compiled and optimized code
In-Reply-To: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>
References: <ffeba7620910070734l3f605fffm84cb7aea434d6b63@mail.gmail.com>
Message-ID: <cd6401a0910071041s52306a94ke14b0f506e12b135@mail.gmail.com>

You might be interested in the new PYTHONDONTWRITEBYTECODE environment
variable supported as of Python 2.6.  I personally think it is a great
improvement.  :-)

Regards,

Zooko

From barry at python.org  Wed Oct  7 19:46:04 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 7 Oct 2009 13:46:04 -0400
Subject: [Python-Dev] Package install failures in 2.6.3
In-Reply-To: <4ACCABC2.6030402@uva.nl>
References: <BD66C502-E5DA-4E5C-92CE-23D8E9F1B3DB@python.org>
	<4ACCABC2.6030402@uva.nl>
Message-ID: <15834B11-EA17-4D9B-B7BB-EBFF25BD5965@python.org>

On Oct 7, 2009, at 10:54 AM, Koen van de Sande wrote:

> If there is going to be any quick 2.6.4 release, can you consider a  
> fix to the building of extensions under Windows ( http://bugs.python.org/issue4120 
>  )?
> Extensions built without this patch applied will not work if the  
> MSVC9 runtimes are installed locally (in the Python folder) instead  
> of system-wide (WinSxS).
> PIL and Matplotlib now use this patch when building extensions. But  
> many others (e.g. PyGame) still create problematic extensions.

I'm not in favor of including this in 2.6.4 since this wasn't a  
regression in 2.6.3.  However if Martin von Loewis approves of the  
patch and feels strongly that it should go in 2.6.4, I'll allow it.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/94126fee/attachment.pgp>

From mal at egenix.com  Wed Oct  7 20:05:27 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 20:05:27 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to	ucs4
In-Reply-To: <cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>	<4AC07309.1010608@egenix.com>
	<cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>
Message-ID: <4ACCD867.1080501@egenix.com>

Zooko O'Whielacronx wrote:
> Dear MAL and python-dev:
> 
> I failed to explain the problem that users are having.  I will try
> again, and this time I will omit my ideas about how to improve things
> and just focus on describing the problem.
> 
> Some users are having trouble using Python packages containing binary
> extensions on Linux.  I want to provide such binary Python packages
> for Linux for the pycryptopp project
> (http://allmydata.org/trac/pycryptopp ) and the zfec project
> (http://allmydata.org/trac/zfec ).  I also want to make it possible
> for users to install the Tahoe-LAFS project (http://allmydata.org )
> without having a compiler or Python header files.  (You'd be surprised
> at how often Tahoe-LAFS users try to do this on Linux.  Linux is no
> longer only for people who have the knowledge and patience to compile
> software themselves.)  Tahoe-LAFS also depends on many packages that
> are maintained by other people and are not packaged or distributed by
> me -- pyOpenSSL, simplejson, etc..
> 
> There have been several hurdles in the way that we've overcome, and no
> doubt there will be more, but the current hurdle is that there are two
> "formats" for Python extension modules that are used on Linux -- UCS2
> and UCS4.  If a user gets a Python package containing a compiled
> extension module which was built for the wrong UCS2/4 setting, he will
> get mysterious (to him) "undefined symbol" errors at import time.

Zooko, I really fail to see the reasoning here:

Why would people who know how to build their own Python interpreter
on Linux and expect it to work like the distribution-provided one,
have a problem looking up the distribution-used configuration
settings ?

This is like compiling your own Linux kernel without using
the same configuration as the distribution kernel and still
expecting the distribution kernel modules to load without
problems.

Note that this has nothing to do with compiling your own
Python extensions. Python's distutils will automatically
use the right settings for compiling those, based on the
configuration of the Python interpreter used for running
the compilation - which will usually be the distribution
interpreter.

Your argument doesn't really live up to the consequences
of switching to UCS4.

Just as data-point: eGenix has been shipping binaries for
Python packages for several years and while we do occasionally
get reports about UCS2/UCS4 mismatches, those are really
in the minority.

I'd also question using the UCS4 default only on Linux.

If we do go for a change, we should use sizeof(wchar_t)
as basis for the new default - on all platforms that
provide a wchar_t type.

However, before we can make such a decision, we need more
data about the consequences. That is:

 * memory footprint changes

 * performance changes

For both Python 2.x and 3.x. After all, UCS4 uses twice
as much memory for all Unicode objects as UCS2.

Since Python 3.x uses Unicode for all strings, I'd expect
such a change to have more impact there.

We'd also need to look into possible problems with different
compilers using different wchar_t sizes on the same platform
(I doubt that there are any).

On Windows, the default is fixed since Windows uses
UTF-16 for everything Unicode, so UCS2 will for a long
time be the only option on that platform.

That said, it'll take a while for distributions to
upgrade, so you're always better off getting the tools
you're using to deal with the problem for you and your
users, since those are easier to upgrade.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Wed Oct  7 20:09:39 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 7 Oct 2009 14:09:39 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <4ACCCF29.7030003@scottdial.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
Message-ID: <8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>

On Oct 7, 2009, at 1:26 PM, Scott Dial wrote:

> I suspect this release is primarily to quench the problems with
> distutils, but..
>
> http://bugs.python.org/issue5949
>
> doesn't seem to have been addressed by you. And this seems like it  
> would
> be another unfortunate loss of an opportunity.

I want us to be very careful about 2.6.4.  This isn't a normal bug fix  
release, it's specifically there to remove the brown paper bag of  
2.6.3 from our heads.  So let's be conservative and fix this one in  
2.6.5.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/f2f48a65/attachment.pgp>

From mal at egenix.com  Wed Oct  7 20:13:27 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 20:13:27 +0200
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <cd6401a0910071037j5b0226dfl91113b27dc6d518b@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AC371D7.9000904@simplistix.co.uk>	<4ACB49FB.3000407@simplistix.co.uk>	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>	<4ACCCF9B.5060303@egenix.com>
	<cd6401a0910071037j5b0226dfl91113b27dc6d518b@mail.gmail.com>
Message-ID: <4ACCDA47.6010605@egenix.com>

Zooko O'Whielacronx wrote:
> Thanks for the reply, MAL.
> 
> How would we judge whether Distribute is ready for inclusion in the
> Python standard lib?  Maybe if it has a few more releases, leaving a
> trail of "closed: fixed" issue tickets behind it?

I guess it'll just have to take the usual path of new modules/packages
for the stdlib...

	http://www.python.org/dev/peps/pep-0002/

(the PEP is a bit outdated, but it still provides a good basis for
 the process)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From eric at trueblade.com  Wed Oct  7 20:21:01 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 07 Oct 2009 14:21:01 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091007T191603-331@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>
	<loom.20091007T191603-331@post.gmane.org>
Message-ID: <4ACCDC0D.2080800@trueblade.com>

Antoine Pitrou wrote:
> Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
>>>>> "%0#8x" % 0x1234
>> '0x001234'
>>>>> "{0:0>#8x}".format(0x1234)
>> '000x1234'
> 
> Apart from the sheer unreadability of the {}-style format string, the result 
> looks rather unexpected from a human being's point of view.
> 
> (in those situations, I would output the 0x manually anyway, such as:
>>>> "0x%06x" % 0x1234
> '0x001234'
>>>> "0x{:06x}".format(0x1234)
> '0x001234'
> )

"#" formatting was added to int.__format__ in order to support this case:

 >>> format(10, '#6x')
'   0xa'

Without '#', there's no way to specify a field width but still have the 
'0x' up against the digits (at least not without generating an 
intermediate result and figuring out the width manually).

The fact that it works in combination with '0' or '>' (not sure which 
one makes it unreadable to you) wasn't really the point of the feature.

Eric.


From solipsis at pitrou.net  Wed Oct  7 20:25:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 7 Oct 2009 18:25:19 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?please_consider_changing_--enable-unicode_?=
	=?utf-8?q?default_to=09ucs4?=
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<cd6401a0910071007x30ab6e91wdd8098c5ca8489d4@mail.gmail.com>
Message-ID: <loom.20091007T202351-653@post.gmane.org>


Zooko O'Whielacronx <zookog <at> gmail.com> writes:
> 
> I accidentally sent this letter just to MAL when I intended it to
> python-dev.  Please read it, as it explains why the issue I'm raising
> is not just the "we should switch to ucs4 because it is better" issue
> that was previously settled by GvR.

For what it's worth, with stringbench under py3k, an UCS2 build is roughly 8%
faster than an UCS4 build (190 s. total against 206 s.).




From pje at telecommunity.com  Wed Oct  7 20:57:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 07 Oct 2009 14:57:10 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4ACCCE44.8050905@scottdial.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<4ACCCE44.8050905@scottdial.com>
Message-ID: <20091007185722.D35B23A407A@sparrow.telecommunity.com>

I'm not really sure how to reply to your email, since it seems to be 
based on several major misunderstandings.  So, just a few key points:

* Distribute 0.6.x is a stable maintenance branch, much like setuptools 0.6
* Distribute 0.7 is vaporware, very much like setuptools 0.7
* Packages using distribute 0.6.x still say "import setuptools"
* Suggesting that the stdlib include the *other guys' code* is not 
"political bickering" - it's a positive proposal for moving forward 
that eliminates both technical and political obstacles.  If you 
thought I was being sarcastic or ironic, you are mistaken.

At 01:22 PM 10/7/2009 -0400, Scott Dial wrote:
>P.J. Eby wrote:
> > At 01:14 PM 10/6/2009 -0700, Guido van Rossum wrote:
> >> suggest nobody hold their breath waiting for setuptools 0.7.
> >
> > I've never suggested or implied otherwise.
> >
> > But, if you like Distribute so much, why not just add it directly to the
> > stdlib?  ;-)
> >
> > There are many wins to be had from such a move, not the least of which
> > is that it allows something to go into the stdlib that isn't
> > (branding-wise) associated with me or setuptools, and lets it become
> > owned/supported by an even wider community.
>
>I think the biggest problem here is that the brand ("setuptools") is so
>ingrained in the world that someone releasing something
>setuptools-but-not-setuptools ("Distribute") is at a serious
>disadvantage. Your insistence on retaining the name "setuptools" for
>yourself and your vapor-ware only harms the community at-large.
>
>Even experienced developers are unaware of Distribute's existence.. I
>was entirely unaware until yourself and PJE got in a bickering exchange
>on Python-Dev this past month. The extremely high visibility of your
>stale package compared to their non-stale package is quite unfortunate.
>You are free to say, "that is their problem," but you are not free to
>say, "it is not my fault."
>
> > AFAIK, the only reason they've had multiple releases of it is to
> > address the issues with its hijacking of setuptools; in a stdlib
> > version all that stuff could be dropped.  Otherwise, it'd already be
> > "mature".
>
>IOW, you acknowledge that your own position and requiring them to
>tolerate the parallel existence (in the world) of setuptools harms
>Distribute. I fail to see how this relates to being in the stdlib. As
>long as it is not called "setuptools" and packages in the world say
>"import setuptools", then there are conflicts they will be forced to
>managed. And moreover, as long as a stale, perhaps buggy version of
>setuptools is the compatibility model they must emulate, they will have
>a hard time coexisting.
>
> > Less political bickering, and the some of the technical results I
> > hoped for all along are achieved.  Yay, open source.
>
>And yet, political bickering seems to be all you are good for in this case.
>
></flame>
>
>--
>Scott Dial
>scott at scottdial.com
>scodial at cs.indiana.edu
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com


From ronaldoussoren at mac.com  Wed Oct  7 21:13:58 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 07 Oct 2009 21:13:58 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <4ACCD867.1080501@egenix.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
	<cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>
	<4ACCD867.1080501@egenix.com>
Message-ID: <EC454F06-5A4C-415F-96B9-4FE119F373F0@mac.com>


On 7 Oct, 2009, at 20:05, M.-A. Lemburg wrote:
>
>
> If we do go for a change, we should use sizeof(wchar_t)
> as basis for the new default - on all platforms that
> provide a wchar_t type.

I'd be -1 on that. Sizeof(wchar_t) is 4 on OSX, but all non-Unix API's  
that deal with Unicode text use ucs16.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/df46ef9a/attachment.bin>

From pje at telecommunity.com  Wed Oct  7 21:35:18 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 07 Oct 2009 15:35:18 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4ACCCF9B.5060303@egenix.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
	<4ACCCF9B.5060303@egenix.com>
Message-ID: <20091007193533.8A4AE3A407A@sparrow.telecommunity.com>

At 07:27 PM 10/7/2009 +0200, M.-A. Lemburg wrote:
>Having more competition will also help, e.g. ActiveState's PyPM looks
>promising (provided they choose to open-source it) and then there's
>pip.

Note that both PyPM and pip use setuptools as an important piece of 
their implementation (as does buildout), so they are technically the 
competition of easy_install, rather than setuptools per se.

IOW, putting setuptools in the stdlib wouldn't be declaring a victor 
in the installation tools competition, it'd simply be providing 
infrastructure for (present and future) tools to build on.


From fuzzyman at voidspace.org.uk  Wed Oct  7 21:38:20 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 07 Oct 2009 20:38:20 +0100
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	<4AC371D7.9000904@simplistix.co.uk>	<4ACB49FB.3000407@simplistix.co.uk>	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>	<4ACCCF9B.5060303@egenix.com>
	<20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
Message-ID: <4ACCEE2C.4050308@voidspace.org.uk>

P.J. Eby wrote:
> At 07:27 PM 10/7/2009 +0200, M.-A. Lemburg wrote:
>> Having more competition will also help, e.g. ActiveState's PyPM looks
>> promising (provided they choose to open-source it) and then there's
>> pip.
>
> Note that both PyPM and pip use setuptools as an important piece of 
> their implementation (as does buildout), so they are technically the 
> competition of easy_install, rather than setuptools per se.
>
> IOW, putting setuptools in the stdlib wouldn't be declaring a victor 
> in the installation tools competition, it'd simply be providing 
> infrastructure for (present and future) tools to build on.
>

I'd like to see PEP 370 (user site-packages folders) compatibility as a 
pre-condition of moving Distribute (or components of it) into the 
standard library.

There are some issues around PEP 370 for alternative implementations 
that I'd like to address as a compatibility fix for Python 2.6 as well. :-)

Michael

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ronaldoussoren at mac.com  Wed Oct  7 21:42:08 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 07 Oct 2009 21:42:08 +0200
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <bbaeab100910071153w77f52957h329b83bd69d66ca9@mail.gmail.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<bbaeab100910071153w77f52957h329b83bd69d66ca9@mail.gmail.com>
Message-ID: <D570FDA3-7F80-48B5-ADF9-923F5E2744C5@mac.com>


On 7 Oct, 2009, at 20:53, Brett Cannon wrote:

> I just tried building out of svn and a ton of tests that rely on  
> urllib failed because the _scproxy module wasn't built and it  
> unconditionally imports it under darwin. Turns out that it requires  
> the Mac toolbox glue to be built which I always skip since I don't  
> care about it.
>
> I am fairly certain this toolbox glue dependency for urllib is a  
> regression as I used to always be able to get networking working w/o  
> the Mac-specific modules.

Bummer. _scproxy was introduced in 2.6.3 to fix a crash in the ctypes  
based code it replaces, as well as very vague failures in the same  
code on OSX 10.5 Server on PPC machines.

Luckily the fix is easy enough: move some code in setup.py from the  
block that checks for "not --disable-toolbox-glue", the _scproxy  
module has no dependencies on the toolbox glue.

The attached patch should fix the issue,

Ronald



>
> -Brett
>
> On Wed, Oct 7, 2009 at 10:18, Barry Warsaw <barry at python.org> wrote:
> Hello everyone.
>
> The source tarballs and Windows installers for Python 2.6.4rc1 are  
> now available:
>
> http://www.python.org/download/releases/2.6.4/
>
> Please download them, install them, and try to use them with your  
> projects and environments.  Let us know if you encounter any  
> problems with them.  Hopefully we can avoid the situation with 2.6.3  
> having such critical bugs.
>
> 2.6.4 final is planned for 18-October.
>
> Cheers,
> -Barry
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/5e75af2f/attachment.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: compile-scproxy-unconditional.txt
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/5e75af2f/attachment.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/5e75af2f/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/5e75af2f/attachment.bin>

From brett at python.org  Wed Oct  7 20:12:15 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 7 Oct 2009 11:12:15 -0700
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <cd6401a0910071037j5b0226dfl91113b27dc6d518b@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com> 
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com> 
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com> 
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com> 
	<4ACCCF9B.5060303@egenix.com>
	<cd6401a0910071037j5b0226dfl91113b27dc6d518b@mail.gmail.com>
Message-ID: <bbaeab100910071112q1d967168ne237bab9f0587fb7@mail.gmail.com>

On Wed, Oct 7, 2009 at 10:37, Zooko O'Whielacronx <zookog at gmail.com> wrote:

> Thanks for the reply, MAL.
>
> How would we judge whether Distribute is ready for inclusion in the
> Python standard lib?  Maybe if it has a few more releases, leaving a
> trail of "closed: fixed" issue tickets behind it?
>

When Tarek says it's ready.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/52e17889/attachment.htm>

From mal at egenix.com  Wed Oct  7 22:13:45 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 22:13:45 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <EC454F06-5A4C-415F-96B9-4FE119F373F0@mac.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>	<4AC07309.1010608@egenix.com>	<cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>	<4ACCD867.1080501@egenix.com>
	<EC454F06-5A4C-415F-96B9-4FE119F373F0@mac.com>
Message-ID: <4ACCF679.3020109@egenix.com>

Ronald Oussoren wrote:
> 
> On 7 Oct, 2009, at 20:05, M.-A. Lemburg wrote:
>>
>>
>> If we do go for a change, we should use sizeof(wchar_t)
>> as basis for the new default - on all platforms that
>> provide a wchar_t type.
> 
> I'd be -1 on that. Sizeof(wchar_t) is 4 on OSX, but all non-Unix API's
> that deal with Unicode text use ucs16.

Is that true for non-Carbon APIs as well ?

This is what I found on the web (in summary):

Apple chose to go with UTF-16 at about the same time as Microsoft did
and used sizeof(wchar_t) == 2 for Mac OS. When they moved to Mac OS X,
they switched wchar_t to sizeof(wchar_t) == 4.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Wed Oct  7 22:21:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 7 Oct 2009 20:21:59 +0000 (UTC)
Subject: [Python-Dev] eggs now mandatory for pypi?
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com> <4ACB45FA.3080707@simplistix.co.uk>
	<bad82a80910060654t51c35bf1lb3235d39b936fd22@mail.gmail.com>
Message-ID: <loom.20091007T222110-805@post.gmane.org>

Arc Riley <arcriley <at> gmail.com> writes:
> 
> Is the intention of Pypi really to turn it into a social networking site?? 

Sure, why not?
It's not like there are enough social networking sites nowadays, are there? :)

Regards

Antoine.



From ronaldoussoren at mac.com  Wed Oct  7 22:24:10 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 07 Oct 2009 22:24:10 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <4ACCF679.3020109@egenix.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
	<cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>
	<4ACCD867.1080501@egenix.com>
	<EC454F06-5A4C-415F-96B9-4FE119F373F0@mac.com>
	<4ACCF679.3020109@egenix.com>
Message-ID: <8D09A004-31C0-445F-BCEF-6FCFFC5774AA@mac.com>


On 7 Oct, 2009, at 22:13, M.-A. Lemburg wrote:

> Ronald Oussoren wrote:
>>
>> On 7 Oct, 2009, at 20:05, M.-A. Lemburg wrote:
>>>
>>>
>>> If we do go for a change, we should use sizeof(wchar_t)
>>> as basis for the new default - on all platforms that
>>> provide a wchar_t type.
>>
>> I'd be -1 on that. Sizeof(wchar_t) is 4 on OSX, but all non-Unix  
>> API's
>> that deal with Unicode text use ucs16.
>
> Is that true for non-Carbon APIs as well ?
>
> This is what I found on the web (in summary):
>
> Apple chose to go with UTF-16 at about the same time as Microsoft did
> and used sizeof(wchar_t) == 2 for Mac OS. When they moved to Mac OS X,
> they switched wchar_t to sizeof(wchar_t) == 4.
>

Both Carbon and the modern APIs use UTF-16.

What I don't quite get in the UTF-16 vs. UTF-32 discussion is why  
UTF-32 would be useful, because if you want to do generic Unicode  
processing you have to look at sequences of composed characters (base  
characters + composing marks) anyway instead of separate code points.   
Not that I'm a unicode expert in any way...

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/99fa789a/attachment-0001.bin>

From vinay_sajip at yahoo.co.uk  Wed Oct  7 22:23:57 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 7 Oct 2009 20:23:57 +0000 (UTC)
Subject: [Python-Dev] a new setuptools release?
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<4ACCCE44.8050905@scottdial.com>
	<20091007185722.D35B23A407A@sparrow.telecommunity.com>
Message-ID: <loom.20091007T221930-247@post.gmane.org>

P.J. Eby <pje <at> telecommunity.com> writes:

> * Distribute 0.6.x is a stable maintenance branch, much like setuptools 0.6

I'm new to this particular discussion so feel free to correct me if I'm wrong,
but ISTM that Distribute 0.6.x differs markedly from setuptools 0.6 in that the
former has an active maintainer and the latter does not, or am I wrong about
that?

Regards,

Vinay Sajip


From sridharr at activestate.com  Wed Oct  7 22:43:20 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 07 Oct 2009 13:43:20 -0700
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk> <4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
	<4ACCCF9B.5060303@egenix.com>
	<20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
Message-ID: <op.u1f4poet1q4jlt@whymac>

On Wed, 07 Oct 2009 12:35:18 -0700, P.J. Eby <pje at telecommunity.com> wrote:

> At 07:27 PM 10/7/2009 +0200, M.-A. Lemburg wrote:
> Having more competition will also help, e.g. ActiveState's PyPM looks
> promising (provided they choose to open-source it) and then there's
> pip.
>  Note that both PyPM and pip use setuptools as an important piece of  
> their implementation (as does buildout), so they are technically the  
> competition of easy_install, rather than setuptools per se.
>  IOW, putting setuptools in the stdlib wouldn't be declaring a victor in  
> the installation tools competition, it'd simply be providing  
> infrastructure for (present and future) tools to build on.

PyPM client relies on pkg_resources *only*[1]. Specifically for

1) the version comparison algorithm:

   $ grep pkg_resources pypm/client/version.py
   import pkg_resources
       return cmp(pkg_resources.parse_version(pkg1.version),
                  pkg_resources.parse_version(pkg2.version))

2) parsing the "install_requires" string:

   $ grep parse pypm/client/dependency.py
       return [pkg_resources.Requirement.parse(reqstring)

Both these features are definitely worthy of addition to stdlib but only  
after *standardizing* them (like PEP 376 does with .egg-info structure and  
files list). Now that Distribute is getting some visibility, it will be  
extremely good for the community to add distribute-0.7 (which would  
include the above two features apart from others) to the stdlib.

-srid

***
[1] The backend code (our mirroring component) also uses  
setuptools.package_index


From mal at egenix.com  Wed Oct  7 22:46:19 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 22:46:19 +0200
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	<4AC371D7.9000904@simplistix.co.uk>	<4ACB49FB.3000407@simplistix.co.uk>	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>	<4ACCCF9B.5060303@egenix.com>
	<20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
Message-ID: <4ACCFE1B.5020408@egenix.com>

P.J. Eby wrote:
> At 07:27 PM 10/7/2009 +0200, M.-A. Lemburg wrote:
>> Having more competition will also help, e.g. ActiveState's PyPM looks
>> promising (provided they choose to open-source it) and then there's
>> pip.
> 
> Note that both PyPM and pip use setuptools as an important piece of
> their implementation (as does buildout), so they are technically the
> competition of easy_install, rather than setuptools per se.
> 
> IOW, putting setuptools in the stdlib wouldn't be declaring a victor in
> the installation tools competition, it'd simply be providing
> infrastructure for (present and future) tools to build on.

I'm sure that some implementation of some of the concepts of
setuptools will end up in the stdlib - in a well-integrated and
distutils compatible form.

Perhaps we can even find a way to remove the need for .pth files
and long sys.path lists :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From brett at python.org  Wed Oct  7 21:46:32 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 7 Oct 2009 12:46:32 -0700
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <D570FDA3-7F80-48B5-ADF9-923F5E2744C5@mac.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org> 
	<bbaeab100910071153w77f52957h329b83bd69d66ca9@mail.gmail.com> 
	<D570FDA3-7F80-48B5-ADF9-923F5E2744C5@mac.com>
Message-ID: <bbaeab100910071246g58f5bae0sf4c292884a4ed839@mail.gmail.com>

On Wed, Oct 7, 2009 at 12:42, Ronald Oussoren <ronaldoussoren at mac.com>wrote:

>
> On 7 Oct, 2009, at 20:53, Brett Cannon wrote:
>
> I just tried building out of svn and a ton of tests that rely on urllib
> failed because the _scproxy module wasn't built and it unconditionally
> imports it under darwin. Turns out that it requires the Mac toolbox glue to
> be built which I always skip since I don't care about it.
> I am fairly certain this toolbox glue dependency for urllib is a regression
> as I used to always be able to get networking working w/o the Mac-specific
> modules.
>
>
> Bummer. _scproxy was introduced in 2.6.3 to fix a crash in the ctypes based
> code it replaces, as well as very vague failures in the same code on OSX
> 10.5 Server on PPC machines.
>
> Luckily the fix is easy enough: move some code in setup.py from the block
> that checks for "not --disable-toolbox-glue", the _scproxy module has no
> dependencies on the toolbox glue.
>
> The attached patch should fix the issue,
>
>
Patch fixed it. Barry, can Ronald apply the patch?

-Brett



> Ronald
>
>
>
>
> -Brett
>
> On Wed, Oct 7, 2009 at 10:18, Barry Warsaw <barry at python.org> wrote:
>
>> Hello everyone.
>>
>> The source tarballs and Windows installers for Python 2.6.4rc1 are now
>> available:
>>
>> http://www.python.org/download/releases/2.6.4/
>>
>> Please download them, install them, and try to use them with your projects
>> and environments.  Let us know if you encounter any problems with them.
>>  Hopefully we can avoid the situation with 2.6.3 having such critical bugs.
>>
>> 2.6.4 final is planned for 18-October.
>>
>> Cheers,
>> -Barry
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/9f1f77c4/attachment.htm>

From pje at telecommunity.com  Wed Oct  7 23:01:12 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 07 Oct 2009 17:01:12 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <loom.20091007T221930-247@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<4ACCCE44.8050905@scottdial.com>
	<20091007185722.D35B23A407A@sparrow.telecommunity.com>
	<loom.20091007T221930-247@post.gmane.org>
Message-ID: <20091007210124.544F33A407A@sparrow.telecommunity.com>

At 08:23 PM 10/7/2009 +0000, Vinay Sajip wrote:
>P.J. Eby writes:
>
> > * Distribute 0.6.x is a stable maintenance branch, much like setuptools 0.6
>
>I'm new to this particular discussion so feel free to correct me if I'm wrong,
>but ISTM that Distribute 0.6.x differs markedly from setuptools 0.6 
>in that the
>former has an active maintainer and the latter does not, or am I wrong about
>that?

That depends on how you define "active".  I haven't resigned, nor do 
I plan to.  There's simply a long mean time between releases at the 
moment, and some people feel that it's too long.  That's certainly 
their prerogative, but it doesn't mean I have to agree with such 
characterizations.


From ziade.tarek at gmail.com  Wed Oct  7 23:02:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 7 Oct 2009 23:02:22 +0200
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <op.u1f4poet1q4jlt@whymac>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
	<4ACCCF9B.5060303@egenix.com>
	<20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
	<op.u1f4poet1q4jlt@whymac>
Message-ID: <94bdd2610910071402p4399ec36y20e966bbe28d70bf@mail.gmail.com>

On Wed, Oct 7, 2009 at 10:43 PM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
> PyPM client relies on pkg_resources *only*[1]. Specifically for
>
> 1) the version comparison algorithm:
[...]
>
> 2) parsing the "install_requires" string:
>
[...]
>
> Both these features are definitely worthy of addition to stdlib but only
> after *standardizing* them (like PEP 376 does with .egg-info structure and
> files list). Now that Distribute is getting some visibility, it will be
> extremely good for the community to add distribute-0.7 (which would include
> the above two features apart from others) to the stdlib.

Three remarks :
 - Distutils contains already a version comparison algorithm, and
   the version comparison algorithm is being reworked in PEP 386. The
goal is to provide
   a version scheme that allows adding a field like install_requires
in PEP 341, and would
  allow package manager to compare versions.

 - The roadmap of Distribute includes the splitting you are mentioning

 - I don't think that adding Distribute-0.7 to the stdlib it the best
plan : I'd rather see bits of them
   included in Distutils, with Distribute being an incubator because
its release cycle is shorter.

I will push on python-dev a detailed roadmap of Distutils we had in
mind and what it means for
Python 2.7. Give me a day or so to prepare it.

Regards
Tarek

From pje at telecommunity.com  Wed Oct  7 23:06:48 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 07 Oct 2009 17:06:48 -0400
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <4ACCFE1B.5020408@egenix.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<20091006211742.9E43E3A4116@sparrow.telecommunity.com>
	<cd6401a0910070956nb1cea1ft72ce6c1ac1a6ccd7@mail.gmail.com>
	<4ACCCF9B.5060303@egenix.com>
	<20091007193533.8A4AE3A407A@sparrow.telecommunity.com>
	<4ACCFE1B.5020408@egenix.com>
Message-ID: <20091007210700.BB3403A408F@sparrow.telecommunity.com>

At 10:46 PM 10/7/2009 +0200, M.-A. Lemburg wrote:
>P.J. Eby wrote:
> > At 07:27 PM 10/7/2009 +0200, M.-A. Lemburg wrote:
> >> Having more competition will also help, e.g. ActiveState's PyPM looks
> >> promising (provided they choose to open-source it) and then there's
> >> pip.
> >
> > Note that both PyPM and pip use setuptools as an important piece of
> > their implementation (as does buildout), so they are technically the
> > competition of easy_install, rather than setuptools per se.
> >
> > IOW, putting setuptools in the stdlib wouldn't be declaring a victor in
> > the installation tools competition, it'd simply be providing
> > infrastructure for (present and future) tools to build on.
>
>I'm sure that some implementation of some of the concepts of
>setuptools will end up in the stdlib - in a well-integrated and
>distutils compatible form.
>
>Perhaps we can even find a way to remove the need for .pth files
>and long sys.path lists :-)

Setuptools doesn't *require* either of those now.  Installing in 
flat, distutils-compatible format has been supported for years, and 
.pth files are only needed for good namespace package support.  But 
PEP 382 makes namespace packages possible without .pth files, so even 
that could go away.


From barry at python.org  Wed Oct  7 23:09:21 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 7 Oct 2009 17:09:21 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <bbaeab100910071246g58f5bae0sf4c292884a4ed839@mail.gmail.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<bbaeab100910071153w77f52957h329b83bd69d66ca9@mail.gmail.com>
	<D570FDA3-7F80-48B5-ADF9-923F5E2744C5@mac.com>
	<bbaeab100910071246g58f5bae0sf4c292884a4ed839@mail.gmail.com>
Message-ID: <65922EB3-DD7A-49E1-B482-462E35D1F890@python.org>

On Oct 7, 2009, at 3:46 PM, Brett Cannon wrote:

>
>
> On Wed, Oct 7, 2009 at 12:42, Ronald Oussoren  
> <ronaldoussoren at mac.com> wrote:
>
> On 7 Oct, 2009, at 20:53, Brett Cannon wrote:
>
>> I just tried building out of svn and a ton of tests that rely on  
>> urllib failed because the _scproxy module wasn't built and it  
>> unconditionally imports it under darwin. Turns out that it requires  
>> the Mac toolbox glue to be built which I always skip since I don't  
>> care about it.
>>
>> I am fairly certain this toolbox glue dependency for urllib is a  
>> regression as I used to always be able to get networking working w/ 
>> o the Mac-specific modules.
>
> Bummer. _scproxy was introduced in 2.6.3 to fix a crash in the  
> ctypes based code it replaces, as well as very vague failures in the  
> same code on OSX 10.5 Server on PPC machines.
>
> Luckily the fix is easy enough: move some code in setup.py from the  
> block that checks for "not --disable-toolbox-glue", the _scproxy  
> module has no dependencies on the toolbox glue.
>
> The attached patch should fix the issue,
>
>
> Patch fixed it. Barry, can Ronald apply the patch?

Since this is a 2.6.3 regression, yes.

Thanks,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/a29f7b1d/attachment.pgp>

From mal at egenix.com  Wed Oct  7 23:21:09 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Oct 2009 23:21:09 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to	ucs4
In-Reply-To: <8D09A004-31C0-445F-BCEF-6FCFFC5774AA@mac.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>	<4AC07309.1010608@egenix.com>	<cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>	<4ACCD867.1080501@egenix.com>	<EC454F06-5A4C-415F-96B9-4FE119F373F0@mac.com>	<4ACCF679.3020109@egenix.com>
	<8D09A004-31C0-445F-BCEF-6FCFFC5774AA@mac.com>
Message-ID: <4ACD0645.8060207@egenix.com>

Ronald Oussoren wrote:
> 
> On 7 Oct, 2009, at 22:13, M.-A. Lemburg wrote:
> 
>> Ronald Oussoren wrote:
>>>
>>> On 7 Oct, 2009, at 20:05, M.-A. Lemburg wrote:
>>>>
>>>>
>>>> If we do go for a change, we should use sizeof(wchar_t)
>>>> as basis for the new default - on all platforms that
>>>> provide a wchar_t type.
>>>
>>> I'd be -1 on that. Sizeof(wchar_t) is 4 on OSX, but all non-Unix API's
>>> that deal with Unicode text use ucs16.
>>
>> Is that true for non-Carbon APIs as well ?
>>
>> This is what I found on the web (in summary):
>>
>> Apple chose to go with UTF-16 at about the same time as Microsoft did
>> and used sizeof(wchar_t) == 2 for Mac OS. When they moved to Mac OS X,
>> they switched wchar_t to sizeof(wchar_t) == 4.
>>
> 
> Both Carbon and the modern APIs use UTF-16.

Thanks for that data point. So UTF-16 would be the more
natural choice on Mac OS X, despite the choice of sizeof(wchar_t).

> What I don't quite get in the UTF-16 vs. UTF-32 discussion is why UTF-32
> would be useful, because if you want to do generic Unicode processing
> you have to look at sequences of composed characters (base characters +
> composing marks) anyway instead of separate code points.  Not that I'm a
> unicode expert in any way...

Very true.

It's one of the reasons why I'm not much of a UCS4-fan - it only
helps with surrogates and that's about it.

Combining characters, various types of control code points
(e.g. joiners, bidirectional marks, breaks, non-breaks, annotations)
context sensitive casing, bidirectional marks and other such
features found in scripts cause very similar problems - often
much harder to solve, since they are not as easily identifiable
as surrogate high and low code points.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 07 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Thu Oct  8 00:24:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 7 Oct 2009 22:24:08 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	<4AC2846E.8060304@g.nevcal.com>	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>	<ha01t7$e2j$1@ger.gmane.org>
	<79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
	<4ACC7B03.3040700@avl.com>
Message-ID: <loom.20091008T002130-593@post.gmane.org>

Hrvoje Niksic <hrvoje.niksic <at> avl.com> writes:
> 
> Of course; simply use the >&- pseudo-redirection, which has been a 
> standard sh feature (later inherited by ksh and bash, but not csh) for 
> ~30 years.  The error message is amusing, too:
> 
> $ python -c 'print "foo"' >&-
> close failed in file object destructor:
> Error in sys.excepthook:
> 
> Original exception was:

Python 3 complains at startup and is a bit more explicit:

$ ./python -c '1' >&-
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Abandon

Of course, if it is stderr that you explicitly close, you lose all reporting:

$ ./python -c '1' 2>&-
Abandon


Regards

Antoine.



From nyamatongwe at gmail.com  Thu Oct  8 00:55:35 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Thu, 8 Oct 2009 09:55:35 +1100
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <8D09A004-31C0-445F-BCEF-6FCFFC5774AA@mac.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
	<cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>
	<4ACCD867.1080501@egenix.com>
	<EC454F06-5A4C-415F-96B9-4FE119F373F0@mac.com>
	<4ACCF679.3020109@egenix.com>
	<8D09A004-31C0-445F-BCEF-6FCFFC5774AA@mac.com>
Message-ID: <50862ebd0910071555m10cf794xd1949fe45dc48312@mail.gmail.com>

Ronald Oussoren:

> Both Carbon and the modern APIs use UTF-16.

   If Unicode size standardization is seen as sufficiently beneficial
then UTF-16 would be more widely applicable than UTF-32. Unix mostly
uses 8-bit APIs which are either explicitly UTF-8 (such as GTK+) or
can accept UTF-8 when the locale is set to UTF-8. They don't accept
UTF-32. It is possible that Unix could move towards UTF-32 but that
hasn't been the case up to now and with both OS X and Windows being
UTF-16, it is more likely that UTF-16 APIs will become more popular on
Unix.

   Neil

From p.f.moore at gmail.com  Wed Oct  7 17:05:43 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 7 Oct 2009 16:05:43 +0100
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <loom.20091007T164835-226@post.gmane.org>
References: <loom.20091007T164835-226@post.gmane.org>
Message-ID: <79990c6b0910070805t5feacb34ice669ca5d0bc0546@mail.gmail.com>

2009/10/7 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> What's the general feeling here about this proposal? All comments and
> suggestions will be gratefully received.

+1

One option I would have found useful in some code I wrote would be to
extend the configuration -

class DictConfigurator:
   ...
   def extend(self, moreconfig):
      import copy
      more = copy.deepcopy(moreconfig) # Not sure if this is needed?
      self.config.update(more)

Paul.

From rhamph at gmail.com  Thu Oct  8 02:10:25 2009
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 7 Oct 2009 18:10:25 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
Message-ID: <aac2c7cb0910071710v35d96e79xe8a37dbf55ba2fb0@mail.gmail.com>

On Sun, Sep 20, 2009 at 10:17, Zooko O'Whielacronx <zookog at gmail.com> wrote:
> On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting.
>
> That would be an improvement! ?Unfortunately we instead get mysterious
> misbehavior of the module, e.g.:
>
> http://bugs.python.org/setuptools/msg309
> http://allmydata.org/trac/tahoe/ticket/704#comment:5

The real issue here is getting confused because python's option is
misnamed.  We support UTF-16 and UTF-32, not UCS-2 and UCS-4.  This
means that when decoding UTF-8, any scalar value outside the BMP will
be split into a pair of surrogates on UTF-16 builds; if we were using
UCS-2 that'd be an error instead (and *nothing* would understand
surrogates.)

Yet we are getting an error here.  However, if you look at the details
you'll notice it's on a 6-byte UTF-8 code unit sequence, corresponding
in the second link to U+6E657770.  Although the originally UTF-8 left
open the possibility of including up to 31 bits (or U+7FFFFFFF), this
was removed in RFC 3629 and is now strictly prohibited.  The modern
unicode character set itself also imposes that restriction.  There is
nothing beyond U+10FFFF.  Nothing should create a such a high code
point, and even if it happened internally a RFC 3629-conformant UTF-8
encoder must refuse to pass it through.

Something more subtle must be going on.  Possibly several bugs (such
as a non-conformant encoder or garbage being misinterpreted as UTF-8).


-- 
Adam Olsen, aka Rhamphoryncus

From david.lyon at preisshare.net  Thu Oct  8 02:17:34 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 07 Oct 2009 20:17:34 -0400
Subject: [Python-Dev] =?utf-8?q?eggs_now_mandatory_for_pypi=3F?=
In-Reply-To: <4ACB6619.4080807@simplistix.co.uk>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk> <4ACB5690.1080208@egenix.com>
	<4ACB6619.4080807@simplistix.co.uk>
Message-ID: <f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>

On Tue, 06 Oct 2009 16:45:29 +0100, Chris Withers <chris at simplistix.co.uk>
wrote:

> Well yeah, and the only sane way I can think to handle this is to have a 
> metadata file that gets uploaded with each distribution that covers all 
> these things (and the other things that other people need) and then have 
> the index (which would hopefully be PyPI on the whole) be queryable 
> along the lines of "give me a download url for a distribution named X 
> that is for Python 2.6, Win32, UCS4, PostGreSQL 8.5, BlagBlah"...

Exactly.

I'd like to see that and it sounds like a very simple and reasonable
proposal.

One could say that much of the setuptools cloud came about because of
the lack of the queryable download url. Setuptools does a lot of work
here to 'work-around' the ommission on pypi of a package download url.

I'm just with you 100% on this...

David





From v+python at g.nevcal.com  Thu Oct  8 03:46:44 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 07 Oct 2009 18:46:44 -0700
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <loom.20091007T164835-226@post.gmane.org>
References: <loom.20091007T164835-226@post.gmane.org>
Message-ID: <4ACD4484.2080409@g.nevcal.com>

On approximately 10/7/2009 7:49 AM, came the following characters from 
the keyboard of Vinay Sajip:

> In outline, the scheme I have in mind will look like this, in terms of the new
> public API:
> 
> class DictConfigurator:
>     def __init__(self, config): #config is a dict-like object (duck-typed)
>         import copy
>         self.config = copy.deepcopy(config)
>         
>     def configure(self):
>         #  actually do the configuration here using self.config
> 
> dictConfigClass = DictConfigurator
> 
> def dictConfig(config):
>     dictConfigClass(config).configure()
> 
> This allows easy replacement of DictConfigurator with a suitable subclass where
> needed.


Concept sounds good, and the idea of separating the syntax of the 
configuration file from the action of configuring is a clever way of 
avoiding the "syntax of the (day|platform|environment)" since everyone 
seems to invent new formats.  So people that want to expose a text file 
to their users have the choice of syntaxes to expose, and then they do

logCfg = readMyFavoriteSyntax( logCfgFile )
     # or extract a subset of a larger config file for their project
DictConfigurator( logCfg )

But DictConfigurator the name seems misleading... like you are 
configuring how dicts work, rather than how logs work.  Maybe with more 
context this is not a problem, but if it is a standalone class, it is 
confusing what it does, by name alone.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From vinay_sajip at yahoo.co.uk  Thu Oct  8 07:43:20 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 05:43:20 +0000 (UTC)
Subject: [Python-Dev] A new way to configure logging
References: <loom.20091007T164835-226@post.gmane.org>
	<79990c6b0910070805t5feacb34ice669ca5d0bc0546@mail.gmail.com>
Message-ID: <loom.20091008T074306-675@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> One option I would have found useful in some code I wrote would be to
> extend the configuration -
> 
> class DictConfigurator:
>    ...
>    def extend(self, moreconfig):
>       import copy
>       more = copy.deepcopy(moreconfig) # Not sure if this is needed?
>       self.config.update(more)

See my response to Paul Rudin's post about incremental configuration.

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Thu Oct  8 07:45:02 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 05:45:02 +0000 (UTC)
Subject: [Python-Dev] A new way to configure logging
References: <loom.20091007T164835-226@post.gmane.org>
	<4ACD4484.2080409@g.nevcal.com>
Message-ID: <loom.20091008T074346-184@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

> But DictConfigurator the name seems misleading... like you are 
> configuring how dicts work, rather than how logs work.  Maybe with more 
> context this is not a problem, but if it is a standalone class, it is 
> confusing what it does, by name alone.

Of course the fully qualified name would be logging.config.DictConfigurator
which does provide the requisite context, but if I think of/someone suggests a
better name I'll certainly use it.





From brett at python.org  Wed Oct  7 20:53:04 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 7 Oct 2009 11:53:04 -0700
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
Message-ID: <bbaeab100910071153w77f52957h329b83bd69d66ca9@mail.gmail.com>

I just tried building out of svn and a ton of tests that rely on urllib
failed because the _scproxy module wasn't built and it unconditionally
imports it under darwin. Turns out that it requires the Mac toolbox glue to
be built which I always skip since I don't care about it.
I am fairly certain this toolbox glue dependency for urllib is a regression
as I used to always be able to get networking working w/o the Mac-specific
modules.

-Brett

On Wed, Oct 7, 2009 at 10:18, Barry Warsaw <barry at python.org> wrote:

> Hello everyone.
>
> The source tarballs and Windows installers for Python 2.6.4rc1 are now
> available:
>
> http://www.python.org/download/releases/2.6.4/
>
> Please download them, install them, and try to use them with your projects
> and environments.  Let us know if you encounter any problems with them.
>  Hopefully we can avoid the situation with 2.6.3 having such critical bugs.
>
> 2.6.4 final is planned for 18-October.
>
> Cheers,
> -Barry
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091007/5ffc1f61/attachment-0001.htm>

From vinay_sajip at yahoo.co.uk  Thu Oct  8 08:11:23 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 06:11:23 +0000 (UTC)
Subject: [Python-Dev] Logging, Unicode and sockets
Message-ID: <loom.20091008T075520-844@post.gmane.org>

>
Thanks to 

http://bugs.python.org/issue7077

I've noticed that the socket-based logging handlers - SocketHandler,
DatagramHandler and SysLogHandler - aren't Unicode-aware and can break in the
presence of Unicode messages. I'd like to fix this by giving these handlers an
optional (encoding=None) parameter in their __init__, and then using this to
encode on output. If no encoding is specified, is it best to use
locale.getpreferredencoding(), sys.getdefaultencoding(),
sys.getfilesystemencoding(), 'utf-8' or something else? On my system:

>>> sys.getdefaultencoding()
'ascii'
>>> sys.getfilesystemencoding()
'mbcs'
>>> locale.getpreferredencoding()
'cp1252'

which suggests to me that the locale.getpreferredencoding() should be the
default. However, as I'm not a Unicode maven, any suggestions would be welcome.

Regards,


Vinay Sajip 


From mal at egenix.com  Thu Oct  8 09:35:57 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 09:35:57 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>
	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>
Message-ID: <4ACD965D.5080701@egenix.com>

David Lyon wrote:
> On Tue, 06 Oct 2009 16:45:29 +0100, Chris Withers <chris at simplistix.co.uk>
> wrote:
> 
>> Well yeah, and the only sane way I can think to handle this is to have a 
>> metadata file that gets uploaded with each distribution that covers all 
>> these things (and the other things that other people need) and then have 
>> the index (which would hopefully be PyPI on the whole) be queryable 
>> along the lines of "give me a download url for a distribution named X 
>> that is for Python 2.6, Win32, UCS4, PostGreSQL 8.5, BlagBlah"...
> 
> Exactly.
> 
> I'd like to see that and it sounds like a very simple and reasonable
> proposal.
> 
> One could say that much of the setuptools cloud came about because of
> the lack of the queryable download url. Setuptools does a lot of work
> here to 'work-around' the ommission on pypi of a package download url.

I think you two have missed my email to PJE.

In summary:

PyPI already does have mechanism for querying download URLs.

	http://wiki.python.org/moin/PyPiXmlRpc

It only lacks some more meta-data that would be required to
match up the user's configuration with the one used in the
uploaded URLs.

And it would be handy to have a mechanism to just upload the
URL data rather than the whole package. Looking at distutils'
upload command, PyPI currently appears to only support uploads
of the package file itself.

Basically, most of the mechnisms are already there, they just need
some extra care.

BTW: Who would I need to contact for the PyPI side to work out
an upload_url distutils command ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Thu Oct  8 09:47:03 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 09:47:03 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <aac2c7cb0910071710v35d96e79xe8a37dbf55ba2fb0@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<aac2c7cb0910071710v35d96e79xe8a37dbf55ba2fb0@mail.gmail.com>
Message-ID: <4ACD98F7.6060700@egenix.com>

Adam Olsen wrote:
> On Sun, Sep 20, 2009 at 10:17, Zooko O'Whielacronx <zookog at gmail.com> wrote:
>> On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting.
>>
>> That would be an improvement!  Unfortunately we instead get mysterious
>> misbehavior of the module, e.g.:
>>
>> http://bugs.python.org/setuptools/msg309
>> http://allmydata.org/trac/tahoe/ticket/704#comment:5

I agree that a better error message would help. I'm just not sure
how to achieve that.

The error message you currently see gets generated by the dynamic
linker trying to resolve a Python Unicode API symbol: the API names
are mangled to assure that you cannot mix UCS2 interpreters and UCS4
extensions (and vice-versa).

We could try to scan the linker error message for 'Py.*UCS.'
and then replace the message with a more helpful one (in importdl.c),
but I'm not sure how portable that is.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Thu Oct  8 10:31:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 10:31:40 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words on
	Virtualenv, Pip)
Message-ID: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>

Here's a quick summary of the main things that are going to happen in
Distutils, and Distribute, and a few words on virtualenv and pip.
(there is much much more work going on, but I don't want to drown
people with details)

= Distutils =

Distutils is a package manager and competes with OS package managers.
This is a good thing because, unless you are developing a library or
an application that will only run one specific system that has its own
packaging system like Debian, you will be able to reach much more
people. Of course the goal is to avoid making the work of a Debian
packager (or any other OS that has a package manager) too hard. In
other words, re-packaging a Distutils-based project should be easy and
Distutils should not get in their way (or as less as possible).

But right now Distutils is incomplete in many ways and we are trying to fix'em.

== What's installed ? what's the installation format ? how to uninstall ? ==

First, it's an incomplete package manager : you can install a
distribution using it, but there's no way to list installed
distributions. Worst, you can't uninstall a distribution.

PEP 376 resolves this, and once it's finished, the goal is to include
the APIs described there into Distutils itself and into the pkgutil
module in stdlib. Notice that there's an implementation at
http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376
so people can see what we are talking about.

Another problem that popped during the last years is the fact that, in
the same site-packages, depending on the tool that was used to install
a Distribution, and depending if this distribution uses Distutils or
Setuptools, you can have different installation formats.

End-users end up with zipped eggs (one file), unzipped eggs (one
self-contained format in a directory) and regular Distutils (packages
and modules in site-packages). And the Metadata are also located in
many different places depending on the installation format used.

That can't be. there's no point to keep various installation format in
the *same* site-packages directory.

PEP 376 also resolves this by describing a *unique* format that works
for all. Once this is finished, Distutils will implement it by
changing the install command accordingly.

- Work left to do in PEP 376 : restrict its scope to a disk-based,
file-based site-packages.
- Goal: 2.7 / 3.2

== Dependencies ==

The other feature that makes a packaging system nice is dependencies.
e.g. a way to list in a distribution, the distributions it requires to
run. As a matter of fact, PEP 314 has introduced in the Metadata new
fields for this purpose ("Requires", "Provides and "Obsoletes"). So,
you can write things like "Requires: lxml >= 2.2.1", meaning that your
distribution requires lxml 2.2.1 or a newer version to run. But this
was just description fields and Distutils was not providing any
feature based on these new fields.

In fact, no third-party tool either provided a feature based on those
fields. Setuptools provided "easy_install" a script that looks for the
dependencies and install them, by querying the Python Package Index
(PyPI). But this feature was implemented with its own metadata: you
can add an "install_requires" option in the setup() call in setup.py,
and it will end up in a "requires.txt" file at installation time that
is located alongside the Metadata for you distribution.

So the goal is to review PEP 314 and update the Metadata w.r.t. the
setuptools feedback and community usage. Once it's done, Distutils
will implement this new metadata version and promote its usage.
Promoting its usage means that Distutils will provide some APIs to
work with these APIs, like a version comparison algorithm.

And while we're at it, we need to work out some inconsistency with the
"Author" and "Maintainer" fields. (The latter doesn't exists in the
Metadata but exists on setup.py side).

- Work left to do in PEP 314 : finish PEP 386, finish the discussion
on the "maintainer" field.
- Goal: 2.7 / 3.2

== Version comparison ==

Once you provide dependency fields in the metadata, you need to
provide a version scheme: a way to compare two versions. Distutils has
two version comparison algorithms that are not used in its code and in
only one place in the stdlib where it could be removed with no pain.
One version scheme is "strict" and one is "loose". And Setuptools has
another one, which is more heuristic (it will deal with any version
string and compare it, wether it's wrong or not).

PEP 386 goal is to describe a version scheme that can be used by all
and if we can meet a consensus there, we can move on and add
it as a reference in the update done in  PEP 314, besides the
dependencies fields. Then, in Distutils we can deprecate the existing
version
comparison algorithms and provide a new one based on PEP 386 and
promote its usage.

One very important point: we will not force the community to use the
scheme described in PEP 386, but *there is* already a
de-facto convention on version schemes at PyPI if you use Pip or
easy_install, so let's have a documented standard for this,
and a reference implementation in Distutils.

There's an implementation at
http://bitbucket.org/tarek/distutilsversion that is kept up-to-date
with PEP 386.

- Work left to do in PEP 386 : another round with the community
- Goal: 2.7 / 3.2

== The fate of setup.py, and static metadata ==

Setup.py is a CLI to create distribution, install them etc. You can
also use it to retrieve the metadata of a distribution. For
example you can call "python setup.py --name" and the name will be
displayed. That's fine. That's great for developers.

But there's a major flaw: it's Python code. It's a problem because,
depending on the complexity of this file, an OS packager that just
wants to get the metadata for the platform he's working on, will run
arbitrary code that mught do unwanted things (or even that light not
work)

So we are going to separate the metadata description from setup.py, in
a static configuration file, that can be open and read by anyone
without
running any code. The only problem with this is the fact that some
metadata fields might depend on the execution environment. For
instance, once "Requires" is re-defined and re-introduced via PEP 314,
we will have cases where "pywin32" will be a dependency to have only
on win32 systems.

So we've worked on that lately in Distutils-SIG and came up with a
micro-language, based on a ConfigParser file, that allows
writing metadata fields that depends on sys.platform etc. I won't
detail the syntax here but the idea is that the interpretation
of this file can be done with a vanilla Python without running arbitrary code.

In other words : we will be able to get the metadata for a
distribution without having to install it or to run any setup.py
command.
One use case is the ability to list all dependencies a distribution
requires for a given platform, just by querying PyPI.

So I am adding this in Distutils for 2.7.

Of course setup.py stays, and this is backward compatible.

- Work left to do : publish the final syntax, and do the implementation
- Goal: 2.7 / 3.2

== The fate of bdist_* commands ==

During last Pycon summit we said that we would remove commands like
bdist_rpm because Python is unable, due to its release cycle,
to do a good work there. Here's an example: I have from time to time
cryptic issues in the issue tracker from people from Fedora (or any
rpm-based system), and I have all the pain in the world for these very
specific problems to do the proper fix unless some RPM expert helps
around. And by the time it's detected then fixed, it can be year(s)
before it's available on their side. That's why, depending on the
communities, commands like bdist_rpm are just totally ignored, and OS
packager have their own tools.

So the best way to handle this is to ask these communities to build
their own tool and to encourage them to use Distutils as a basis for
that.

This does not concern bdist_* commands for win32 because those are
very stable and don't change too much: Windows doesn't have a package
manager that would require these commands to evolve with it.

Anyways, when we said that we would remove bdist_rpm, this was very
controversial because some people use it and love it.

So what is going to happen is a status-quo: no bdist_* command will be
removed but no new bdist_* command wil be added. That's why I've
encouraged Andrew and Garry, that are working on a bdist_deb command,
to keep it in the "stdeb" project, and eventually we will
refer to it in the Distutils documentation if this bdist_deb comply
with Distutils standard. It doesn't right now because it uses a
custom version of the Distribution class (through Setuptools) that
doesn't behave like Distutils' one anymore.

For Distutils, I'll add some documentation explaining this, and a
section that will list community-driven commands.

- Work left to do : write the documentation
- Goal: 2.7 / 3.2

= Distribute =

I won't explain here again why we have forked, I think it's obvious to
anyone here now. I'll rather explain what
we are planning in Distribute and how it will interact with Distutils.

Distribute has two branches:

- 0.6.x : provides a Setuptools-0.6c9 compatible version
- 0.7.x : will provide a refactoring

== 0.6.x ==

Not "much" is going to happen here, we want this branch to be helpful
to the community *today* by addressing the 40-or-so bugs
that were found in Setuptools and never fixed. This is eventually
happen soon because its development is
fast : there are up to 5 commiters that are working on it very often
(and the number grows weekly.)

The biggest issue with this branch is that it is providing the same
packages and modules setuptools does, and this
requires some bootstrapping work where we make sure once Distribute is
installed, all Distribution that requires Setuptools
will continue to work. This is done by faking the metadata of
Setuptools 0.6c9. That's the only way we found to do this.

There's one major thing though: thanks to the work of Lennart, Alex,
Martin, this branch supports Python 3,
which is great to have to speed up Py3 adoption.

The goal of the 0.6.x is to remove as much bugs as we can, and try if
possible to remove the patches done
on Distutils. We will support 0.6.x maintenance for years and we will
promote its usage everywhere instead of
Setuptools.

Some new commands are added there, when they are helpful and don't
interact with the rest. I am thinking
about "upload_docs" that let you upload documentation to PyPI. The
goal is to move it to Distutils
at some point, if the documentation feature of PyPI stays and starts to be used.

== 0.7.x ==

We've started to refactor Distribute with this roadmap in mind (and
no, as someone said, it's not vaporware,
we've done a lot already)

- 0.7.x can be installed and used with 0.6.x

- easy_install is going to be deprecated ! use Pip !

- the version system will be deprecated, in favor of the one in Distutils

- no more Distutils monkey-patch that happens once you use the code
 (things like 'from distutils import cmd; cmd.Command = CustomCommand')

- no more custom site.py (that is: if something misses in Python's
site.py we'll add it there instead of patching it)

- no more namespaced packages system, if PEP 381 (namespaces package
support) makes it to 2.7

- The code is splitted in many packages and might be distributed under
several distributions.

   - distribute.resources: that's the old pkg_resources, but
reorganized in clean, pep-8 modules. This package will
     only contain the query APIs and will focus on being PEP 376
compatible. We will promote its usage and see if Pip wants
     to use it as a basis. And maybe PyPM once it's open source ?
(<hint> <hint>).
	It will probably shrink a lot though, once the stdlib provides PEP 376 support.

   - distribute.entrypoints: that's the old pkg_resources entry points
system, but on its own. it uses distribute.resources

   - distribute.index: that's package_index and a few other things.
everything required to interact with PyPI. We will promote
     its usage and see if Pip wants to use it as a basis.

   - distribute.core (might be renamed to main): that's everything
else, and uses the other packages.


Goal: A first release before (or when) Python 2.7 / 3.2 is out.


= Virtualenv and the multiple version support in Distribute =

(I am not saying "We" here because this part was not discussed yet
with everyone)

Virtualenv allows you to create an isolated environment to install
some distribution without polluting the
main site-packages, a bit like a user site-packages.

My opinion is that this tool exists only because Python doesn't
support the installation of multiple versions for the same
distributions.
But if PEP 376 and PEP 386 support are added in Python, we're not far
from being able to provide multiple version support with
the help of importlib.

Setuptools provided a multiple version support but I don't like its
implementation and the way its works.
I would like to create a new site-packages format that can contains
several versions of the same distribution, and :

- a special import system using importlib that would automatically
pick the latest version, thanks to PEP 376.
- an API to force at runtime a specific version (that would be located
at the beginning of all imports, like __future__)
- a layout that is compatible with the way OS packagers works with
python packages

Goal: a prototype asap (one was started under the "VSP" name (virtual
site-packages) but not finished yet)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From mal at egenix.com  Thu Oct  8 10:38:06 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 10:38:06 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <94bdd2610910080133y788cb0d6xd67aaf3b58ac48a0@mail.gmail.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>
	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk>	<4ACB5690.1080208@egenix.com>
	<4ACB6619.4080807@simplistix.co.uk>	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>	<4ACD965D.5080701@egenix.com>
	<94bdd2610910080133y788cb0d6xd67aaf3b58ac48a0@mail.gmail.com>
Message-ID: <4ACDA4EE.4050005@egenix.com>

Tarek Ziad? wrote:
> On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> BTW: Who would I need to contact for the PyPI side to work out
>> an upload_url distutils command ?
> 
> I am not sure to understand. How upload_url will differ from the
> register command
> that let you upload metadata at PyPI ?

The register command only uploads the package's meta-data.

The upload command uploads a file and registers the file's
details with PyPI as well.

I'd like to add an upload_url command which only registers a
file's details (including the URL, but without the actual file)
with PyPI and perhaps extend the number of attributes sent to
PyPI with more file configuration information (upload currently
only sends the Python version).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Thu Oct  8 10:33:01 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 10:33:01 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACD965D.5080701@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACA0CBA.2030200@egenix.com> <4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com> <4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com> <4ACB6619.4080807@simplistix.co.uk>
	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>
	<4ACD965D.5080701@egenix.com>
Message-ID: <94bdd2610910080133y788cb0d6xd67aaf3b58ac48a0@mail.gmail.com>

On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> BTW: Who would I need to contact for the PyPI side to work out
> an upload_url distutils command ?

I am not sure to understand. How upload_url will differ from the
register command
that let you upload metadata at PyPI ?

Tarek

From ziade.tarek at gmail.com  Thu Oct  8 10:59:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 10:59:48 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACDA4EE.4050005@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACB3FC0.1010906@simplistix.co.uk> <4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk> <4ACB5690.1080208@egenix.com>
	<4ACB6619.4080807@simplistix.co.uk>
	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>
	<4ACD965D.5080701@egenix.com>
	<94bdd2610910080133y788cb0d6xd67aaf3b58ac48a0@mail.gmail.com>
	<4ACDA4EE.4050005@egenix.com>
Message-ID: <94bdd2610910080159r319b0fdtd02b779bba893e18@mail.gmail.com>

On Thu, Oct 8, 2009 at 10:38 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> Tarek Ziad? wrote:
>> On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> BTW: Who would I need to contact for the PyPI side to work out
>>> an upload_url distutils command ?
>>
>> I am not sure to understand. How upload_url will differ from the
>> register command
>> that let you upload metadata at PyPI ?
>
> The register command only uploads the package's meta-data.
>
> The upload command uploads a file and registers the file's
> details with PyPI as well.
>
> I'd like to add an upload_url command which only registers a
> file's details (including the URL, but without the actual file)
> with PyPI and perhaps extend the number of attributes sent to
> PyPI with more file configuration information (upload currently
> only sends the Python version).

Sounds like an option to be added in upload.

>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source ?(#1, Oct 08 2009)
>>>> Python/Zope Consulting and Support ... ? ? ? ?http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ... ? ? ? ? ? ? http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ... ? ? ? ?http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
> ? eGenix.com Software, Skills and Services GmbH ?Pastor-Loeh-Str.48
> ? ?D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> ? ? ? ? ? Registered at Amtsgericht Duesseldorf: HRB 46611
> ? ? ? ? ? ? ? http://www.egenix.com/company/contact/
>



-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From p.f.moore at gmail.com  Thu Oct  8 10:56:58 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 8 Oct 2009 09:56:58 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <79990c6b0910080156s2e540184r255624d2166cbae2@mail.gmail.com>

2009/10/8 Tarek Ziad? <ziade.tarek at gmail.com>:
> Here's a quick summary of the main things that are going to happen in
> Distutils, and Distribute, and a few words on virtualenv and pip.
> (there is much much more work going on, but I don't want to drown
> people with details)

Thanks for this summary. The overview was getting lost in all the details.

> == What's installed ? what's the installation format ? how to uninstall ? ==
>
> First, it's an incomplete package manager : you can install a
> distribution using it, but there's no way to list installed
> distributions. Worst, you can't uninstall a distribution.
>
> PEP 376 resolves this, and once it's finished, the goal is to include
> the APIs described there into Distutils itself and into the pkgutil
> module in stdlib. Notice that there's an implementation at
> http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376
> so people can see what we are talking about.
>
> Another problem that popped during the last years is the fact that, in
> the same site-packages, depending on the tool that was used to install
> a Distribution, and depending if this distribution uses Distutils or
> Setuptools, you can have different installation formats.
>
> End-users end up with zipped eggs (one file), unzipped eggs (one
> self-contained format in a directory) and regular Distutils (packages
> and modules in site-packages). And the Metadata are also located in
> many different places depending on the installation format used.
>
> That can't be. there's no point to keep various installation format in
> the *same* site-packages directory.
>
> PEP 376 also resolves this by describing a *unique* format that works
> for all. Once this is finished, Distutils will implement it by
> changing the install command accordingly.
>
> - Work left to do in PEP 376 : restrict its scope to a disk-based,
> file-based site-packages.
> - Goal: 2.7 / 3.2

+1

This addresses my biggest concern with the way PEP 376 was going
(largely prompted by my suggestions, so I'll take the blame for this!)
which was that PEP 376 was trying to define list/uninstall methods for
package formats (i.e. zip files/eggs) that it couldn't install itself.

I presume that when you refer to the restriction in scope, you do mean
that PEP 375 will lose all support for zip files, as well as the
generalised PEP 302 support I proposed? I still believe that
special-casing zip files is wrong - PEP 302 deliberately abstracted
the import mechanisms and built zipfile support on top of that, and I
still believe that's a good thing. But expanding PEP 302 to cover the
requirements of PEP 376 looked more and more inappropriate the more I
wrote code to do it - what would be needed would be a *new* protocol
on top of which zip-based formats could be layered. I'm quite happy
for that to be outside the scope of PEP 376, though, as distutils
itself doesn't support any such formats.

The egg format

One thing missing from your roadmap (unless I missed it) is the fate
of the egg (zipfile) format. If it's to remain a valid option
(bdist_egg and the like) then I'd assume that Distribute needs to be
the place to develop it. One thing it will need is an equivalent to
PEP 376 introspection and uninstall features. And *that* is where I
believe a standard protocol belongs - developed based on the real use
requirements for egg files, with a suitable file-based layer which can
be added into distutils (somewhat like importlib does for PEP 302)
once the protocol is agreed and standardised.

Paul.

From ziade.tarek at gmail.com  Thu Oct  8 11:11:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 11:11:10 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <79990c6b0910080156s2e540184r255624d2166cbae2@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<79990c6b0910080156s2e540184r255624d2166cbae2@mail.gmail.com>
Message-ID: <94bdd2610910080211q5ff4fd09wf62c44dd4a680a8f@mail.gmail.com>

On Thu, Oct 8, 2009 at 10:56 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>
> +1
>
> This addresses my biggest concern with the way PEP 376 was going
> (largely prompted by my suggestions, so I'll take the blame for this!)
> which was that PEP 376 was trying to define list/uninstall methods for
> package formats (i.e. zip files/eggs) that it couldn't install itself.
>
> I presume that when you refer to the restriction in scope, you do mean
> that PEP 375 will lose all support for zip files, as well as the
> generalised PEP 302 support I proposed? I still believe that
> special-casing zip files is wrong - PEP 302 deliberately abstracted
> the import mechanisms and built zipfile support on top of that, and I
> still believe that's a good thing. But expanding PEP 302 to cover the
> requirements of PEP 376 looked more and more inappropriate the more I
> wrote code to do it - what would be needed would be a *new* protocol
> on top of which zip-based formats could be layered. I'm quite happy
> for that to be outside the scope of PEP 376, though, as distutils
> itself doesn't support any such formats.

Yes exactly so.

>
> The egg format
>
> One thing missing from your roadmap (unless I missed it) is the fate
> of the egg (zipfile) format. If it's to remain a valid option
> (bdist_egg and the like) then I'd assume that Distribute needs to be
> the place to develop it.
> One thing it will need is an equivalent to
> PEP 376 introspection and uninstall features. And *that* is where I
> believe a standard protocol belongs - developed based on the real use
> requirements for egg files, with a suitable file-based layer which can
> be added into distutils (somewhat like importlib does for PEP 302)
> once the protocol is agreed and standardised.

Distribute is planning to continue supporting the egg format, but will
probably drop the zipped version.
This is still under discussions though so don't take this as the final word.

In any case, distribute.resources will probably add the missing bits
on the top of a PEP 376 -compliant Distutils
to provide support for any file querying Distutils will not provide
(for installation and uninstallation)

Tarek

From cournape at gmail.com  Thu Oct  8 11:42:28 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 8 Oct 2009 18:42:28 +0900
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <5b8d13220910080242q19efb5dbre3fae15462253bfb@mail.gmail.com>

On Thu, Oct 8, 2009 at 5:31 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> = Virtualenv and the multiple version support in Distribute =
>
> (I am not saying "We" here because this part was not discussed yet
> with everyone)
>
> Virtualenv allows you to create an isolated environment to install
> some distribution without polluting the
> main site-packages, a bit like a user site-packages.
>
> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.

I am really worried about this, because it may encourage people to use
multiple versions as a bandaid to maintaining backward compatibility.
At least with virtual-env, the problem is restricted to the user.

Generalized multiple, side by side installation  has been tried in
many different contexts, and I have never seen a single one working
and not bringing more problems that it solved. One core problem being
the exponential number of combination (package A depends on B and C, B
depends on one version of D, C on another version of D). Being able to
install *some* libraries in multiple versions is OK, but generalizing
is very dangerous IMHO.

David

From ronaldoussoren at mac.com  Thu Oct  8 11:49:38 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 08 Oct 2009 11:49:38 +0200
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <65922EB3-DD7A-49E1-B482-462E35D1F890@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<bbaeab100910071153w77f52957h329b83bd69d66ca9@mail.gmail.com>
	<D570FDA3-7F80-48B5-ADF9-923F5E2744C5@mac.com>
	<bbaeab100910071246g58f5bae0sf4c292884a4ed839@mail.gmail.com>
	<65922EB3-DD7A-49E1-B482-462E35D1F890@python.org>
Message-ID: <8374F9DB-487C-454E-AB31-453E388329AA@mac.com>


On 7 Oct, 2009, at 23:09, Barry Warsaw wrote:

> On Oct 7, 2009, at 3:46 PM, Brett Cannon wrote:
>
>>
>>
>> On Wed, Oct 7, 2009 at 12:42, Ronald Oussoren  
>> <ronaldoussoren at mac.com> wrote:
>>
>> On 7 Oct, 2009, at 20:53, Brett Cannon wrote:
>>
>>> I just tried building out of svn and a ton of tests that rely on  
>>> urllib failed because the _scproxy module wasn't built and it  
>>> unconditionally imports it under darwin. Turns out that it  
>>> requires the Mac toolbox glue to be built which I always skip  
>>> since I don't care about it.
>>>
>>> I am fairly certain this toolbox glue dependency for urllib is a  
>>> regression as I used to always be able to get networking working w/ 
>>> o the Mac-specific modules.
>>
>> Bummer. _scproxy was introduced in 2.6.3 to fix a crash in the  
>> ctypes based code it replaces, as well as very vague failures in  
>> the same code on OSX 10.5 Server on PPC machines.
>>
>> Luckily the fix is easy enough: move some code in setup.py from the  
>> block that checks for "not --disable-toolbox-glue", the _scproxy  
>> module has no dependencies on the toolbox glue.
>>
>> The attached patch should fix the issue,
>>
>>
>> Patch fixed it. Barry, can Ronald apply the patch?
>
> Since this is a 2.6.3 regression, yes.

I've commited the patch (r75282)

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/72f30a67/attachment.bin>

From ziade.tarek at gmail.com  Thu Oct  8 11:59:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 11:59:12 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <5b8d13220910080242q19efb5dbre3fae15462253bfb@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<5b8d13220910080242q19efb5dbre3fae15462253bfb@mail.gmail.com>
Message-ID: <94bdd2610910080259q31cf830fsea147f612a05ece9@mail.gmail.com>

On Thu, Oct 8, 2009 at 11:42 AM, David Cournapeau <cournape at gmail.com> wrote:
> On Thu, Oct 8, 2009 at 5:31 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>
>> = Virtualenv and the multiple version support in Distribute =
>>
>> (I am not saying "We" here because this part was not discussed yet
>> with everyone)
>>
>> Virtualenv allows you to create an isolated environment to install
>> some distribution without polluting the
>> main site-packages, a bit like a user site-packages.
>>
>> My opinion is that this tool exists only because Python doesn't
>> support the installation of multiple versions for the same
>> distributions.
>
> I am really worried about this, because it may encourage people to use
> multiple versions as a bandaid to maintaining backward compatibility.
> At least with virtual-env, the problem is restricted to the user.
>
> Generalized multiple, side by side installation ?has been tried in
> many different contexts, and I have never seen a single one working
> and not bringing more problems that it solved. One core problem being
> the exponential number of combination (package A depends on B and C, B
> depends on one version of D, C on another version of D). Being able to
> install *some* libraries in multiple versions is OK, but generalizing
> is very dangerous IMHO.

The goal here is to be able to have a *single* instance of "Foo 1.2" on your
system, managed by the admin sys. that can be shared and used for
several python
apps. And yes, I don't expect to have so many different versions of
many distributions
on a system, but just to provide a mechanism to allow it.

But that's just an idea we had with some people, and it still requires
a lot of work and brainstroming (even if the prototype kinda work)

I should've mentioned that I've added it at the end of the roadmap
because I had the feeling that something has to be done to reunite
zc.buildout, virtualenv and Python one day.

But consider this as the topic for the "fun sprint" we would have at
Pycon, after we will have issued other points. If you are interested
in this particular topic, I propose that we continue this discussion
on distutils-SIG.

Regards
Tarek

From kiorky at cryptelium.net  Thu Oct  8 12:00:15 2009
From: kiorky at cryptelium.net (kiorky)
Date: Thu, 08 Oct 2009 12:00:15 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on	Virtualenv, Pip)
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <4ACDB82F.7010006@cryptelium.net>



Tarek Ziad? a ?crit :
> Here's a quick summary of the main things that are going to happen in
> Distutils, and Distribute, and a few words on virtualenv and pip.
> (there is much much more work going on, but I don't want to drown
> people with details)
> 
> = Distutils =
> 
> Distutils is a package manager and competes with OS package managers.
> This is a good thing because, unless you are developing a library or
> an application that will only run one specific system that has its own
> packaging system like Debian, you will be able to reach much more
> people. Of course the goal is to avoid making the work of a Debian
> packager (or any other OS that has a package manager) too hard. In
> other words, re-packaging a Distutils-based project should be easy and
> Distutils should not get in their way (or as less as possible).
> 
> But right now Distutils is incomplete in many ways and we are trying to fix'em.
> 
> == What's installed ? what's the installation format ? how to uninstall ? ==
> 
> First, it's an incomplete package manager : you can install a
> distribution using it, but there's no way to list installed
> distributions. Worst, you can't uninstall a distribution.
> 
> PEP 376 resolves this, and once it's finished, the goal is to include
> the APIs described there into Distutils itself and into the pkgutil
> module in stdlib. Notice that there's an implementation at
> http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376
> so people can see what we are talking about.
> 
> Another problem that popped during the last years is the fact that, in
> the same site-packages, depending on the tool that was used to install
> a Distribution, and depending if this distribution uses Distutils or
> Setuptools, you can have different installation formats.
> 
> End-users end up with zipped eggs (one file), unzipped eggs (one
> self-contained format in a directory) and regular Distutils (packages
> and modules in site-packages). And the Metadata are also located in
> many different places depending on the installation format used.
> 
> That can't be. there's no point to keep various installation format in
> the *same* site-packages directory.
> 
> PEP 376 also resolves this by describing a *unique* format that works
> for all. Once this is finished, Distutils will implement it by
> changing the install command accordingly.
> 
> - Work left to do in PEP 376 : restrict its scope to a disk-based,
> file-based site-packages.
> - Goal: 2.7 / 3.2
> 
> == Dependencies ==
> 
> The other feature that makes a packaging system nice is dependencies.
> e.g. a way to list in a distribution, the distributions it requires to
> run. As a matter of fact, PEP 314 has introduced in the Metadata new
> fields for this purpose ("Requires", "Provides and "Obsoletes"). So,
> you can write things like "Requires: lxml >= 2.2.1", meaning that your
> distribution requires lxml 2.2.1 or a newer version to run. But this
> was just description fields and Distutils was not providing any
> feature based on these new fields.
> 
> In fact, no third-party tool either provided a feature based on those
> fields. Setuptools provided "easy_install" a script that looks for the
> dependencies and install them, by querying the Python Package Index
> (PyPI). But this feature was implemented with its own metadata: you
> can add an "install_requires" option in the setup() call in setup.py,
> and it will end up in a "requires.txt" file at installation time that
> is located alongside the Metadata for you distribution.
> 
> So the goal is to review PEP 314 and update the Metadata w.r.t. the
> setuptools feedback and community usage. Once it's done, Distutils
> will implement this new metadata version and promote its usage.
> Promoting its usage means that Distutils will provide some APIs to
> work with these APIs, like a version comparison algorithm.
> 
> And while we're at it, we need to work out some inconsistency with the
> "Author" and "Maintainer" fields. (The latter doesn't exists in the
> Metadata but exists on setup.py side).
> 
> - Work left to do in PEP 314 : finish PEP 386, finish the discussion
> on the "maintainer" field.
> - Goal: 2.7 / 3.2
> 
> == Version comparison ==
> 
> Once you provide dependency fields in the metadata, you need to
> provide a version scheme: a way to compare two versions. Distutils has
> two version comparison algorithms that are not used in its code and in
> only one place in the stdlib where it could be removed with no pain.
> One version scheme is "strict" and one is "loose". And Setuptools has
> another one, which is more heuristic (it will deal with any version
> string and compare it, wether it's wrong or not).
> 
> PEP 386 goal is to describe a version scheme that can be used by all
> and if we can meet a consensus there, we can move on and add
> it as a reference in the update done in  PEP 314, besides the
> dependencies fields. Then, in Distutils we can deprecate the existing
> version
> comparison algorithms and provide a new one based on PEP 386 and
> promote its usage.
> 
> One very important point: we will not force the community to use the
> scheme described in PEP 386, but *there is* already a
> de-facto convention on version schemes at PyPI if you use Pip or
> easy_install, so let's have a documented standard for this,
> and a reference implementation in Distutils.
> 
> There's an implementation at
> http://bitbucket.org/tarek/distutilsversion that is kept up-to-date
> with PEP 386.
> 
> - Work left to do in PEP 386 : another round with the community
> - Goal: 2.7 / 3.2
> 
> == The fate of setup.py, and static metadata ==
> 
> Setup.py is a CLI to create distribution, install them etc. You can
> also use it to retrieve the metadata of a distribution. For
> example you can call "python setup.py --name" and the name will be
> displayed. That's fine. That's great for developers.
> 
> But there's a major flaw: it's Python code. It's a problem because,
> depending on the complexity of this file, an OS packager that just
> wants to get the metadata for the platform he's working on, will run
> arbitrary code that mught do unwanted things (or even that light not
> work)
> 
> So we are going to separate the metadata description from setup.py, in
> a static configuration file, that can be open and read by anyone
> without
> running any code. The only problem with this is the fact that some
> metadata fields might depend on the execution environment. For
> instance, once "Requires" is re-defined and re-introduced via PEP 314,
> we will have cases where "pywin32" will be a dependency to have only
> on win32 systems.
> 
> So we've worked on that lately in Distutils-SIG and came up with a
> micro-language, based on a ConfigParser file, that allows
> writing metadata fields that depends on sys.platform etc. I won't
> detail the syntax here but the idea is that the interpretation
> of this file can be done with a vanilla Python without running arbitrary code.
> 
> In other words : we will be able to get the metadata for a
> distribution without having to install it or to run any setup.py
> command.
> One use case is the ability to list all dependencies a distribution
> requires for a given platform, just by querying PyPI.
> 
> So I am adding this in Distutils for 2.7.
> 
> Of course setup.py stays, and this is backward compatible.
> 
> - Work left to do : publish the final syntax, and do the implementation
> - Goal: 2.7 / 3.2
> 
> == The fate of bdist_* commands ==
> 
> During last Pycon summit we said that we would remove commands like
> bdist_rpm because Python is unable, due to its release cycle,
> to do a good work there. Here's an example: I have from time to time
> cryptic issues in the issue tracker from people from Fedora (or any
> rpm-based system), and I have all the pain in the world for these very
> specific problems to do the proper fix unless some RPM expert helps
> around. And by the time it's detected then fixed, it can be year(s)
> before it's available on their side. That's why, depending on the
> communities, commands like bdist_rpm are just totally ignored, and OS
> packager have their own tools.
> 
> So the best way to handle this is to ask these communities to build
> their own tool and to encourage them to use Distutils as a basis for
> that.
> 
> This does not concern bdist_* commands for win32 because those are
> very stable and don't change too much: Windows doesn't have a package
> manager that would require these commands to evolve with it.
> 
> Anyways, when we said that we would remove bdist_rpm, this was very
> controversial because some people use it and love it.
> 
> So what is going to happen is a status-quo: no bdist_* command will be
> removed but no new bdist_* command wil be added. That's why I've
> encouraged Andrew and Garry, that are working on a bdist_deb command,
> to keep it in the "stdeb" project, and eventually we will
> refer to it in the Distutils documentation if this bdist_deb comply
> with Distutils standard. It doesn't right now because it uses a
> custom version of the Distribution class (through Setuptools) that
> doesn't behave like Distutils' one anymore.
> 
> For Distutils, I'll add some documentation explaining this, and a
> section that will list community-driven commands.
> 
> - Work left to do : write the documentation
> - Goal: 2.7 / 3.2
> 
> = Distribute =
> 
> I won't explain here again why we have forked, I think it's obvious to
> anyone here now. I'll rather explain what
> we are planning in Distribute and how it will interact with Distutils.
> 
> Distribute has two branches:
> 
> - 0.6.x : provides a Setuptools-0.6c9 compatible version
> - 0.7.x : will provide a refactoring
> 
> == 0.6.x ==
> 
> Not "much" is going to happen here, we want this branch to be helpful
> to the community *today* by addressing the 40-or-so bugs
> that were found in Setuptools and never fixed. This is eventually
> happen soon because its development is
> fast : there are up to 5 commiters that are working on it very often
> (and the number grows weekly.)
> 
> The biggest issue with this branch is that it is providing the same
> packages and modules setuptools does, and this
> requires some bootstrapping work where we make sure once Distribute is
> installed, all Distribution that requires Setuptools
> will continue to work. This is done by faking the metadata of
> Setuptools 0.6c9. That's the only way we found to do this.
> 
> There's one major thing though: thanks to the work of Lennart, Alex,
> Martin, this branch supports Python 3,
> which is great to have to speed up Py3 adoption.
> 
> The goal of the 0.6.x is to remove as much bugs as we can, and try if
> possible to remove the patches done
> on Distutils. We will support 0.6.x maintenance for years and we will
> promote its usage everywhere instead of
> Setuptools.
> 
> Some new commands are added there, when they are helpful and don't
> interact with the rest. I am thinking
> about "upload_docs" that let you upload documentation to PyPI. The
> goal is to move it to Distutils
> at some point, if the documentation feature of PyPI stays and starts to be used.
> 
> == 0.7.x ==
> 
> We've started to refactor Distribute with this roadmap in mind (and
> no, as someone said, it's not vaporware,
> we've done a lot already)
> 
> - 0.7.x can be installed and used with 0.6.x
> 
> - easy_install is going to be deprecated ! use Pip !
> 
> - the version system will be deprecated, in favor of the one in Distutils
> 
> - no more Distutils monkey-patch that happens once you use the code
>  (things like 'from distutils import cmd; cmd.Command = CustomCommand')
> 
> - no more custom site.py (that is: if something misses in Python's
> site.py we'll add it there instead of patching it)
> 
> - no more namespaced packages system, if PEP 381 (namespaces package
> support) makes it to 2.7
> 
> - The code is splitted in many packages and might be distributed under
> several distributions.
> 
>    - distribute.resources: that's the old pkg_resources, but
> reorganized in clean, pep-8 modules. This package will
>      only contain the query APIs and will focus on being PEP 376
> compatible. We will promote its usage and see if Pip wants
>      to use it as a basis. And maybe PyPM once it's open source ?
> (<hint> <hint>).
> 	It will probably shrink a lot though, once the stdlib provides PEP 376 support.
> 
>    - distribute.entrypoints: that's the old pkg_resources entry points
> system, but on its own. it uses distribute.resources
> 
>    - distribute.index: that's package_index and a few other things.
> everything required to interact with PyPI. We will promote
>      its usage and see if Pip wants to use it as a basis.
> 
>    - distribute.core (might be renamed to main): that's everything
> else, and uses the other packages.
> 
> 
> Goal: A first release before (or when) Python 2.7 / 3.2 is out.
> 
> 
> = Virtualenv and the multiple version support in Distribute =
> 
> (I am not saying "We" here because this part was not discussed yet
> with everyone)
> 
> Virtualenv allows you to create an isolated environment to install
> some distribution without polluting the
> main site-packages, a bit like a user site-packages.
> 
> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.

Not only, for me it exists also to provide a way to isolate/chroot an
application from the host system.


> But if PEP 376 and PEP 386 support are added in Python, we're not far
> from being able to provide multiple version support with
> the help of importlib.
> 
> Setuptools provided a multiple version support but I don't like its
> implementation and the way its works.
> I would like to create a new site-packages format that can contains
> several versions of the same distribution, and :
> 
> - a special import system using importlib that would automatically
> pick the latest version, thanks to PEP 376.
> - an API to force at runtime a specific version (that would be located
> at the beginning of all imports, like __future__)
> - a layout that is compatible with the way OS packagers works with
> python packages
> 
> Goal: a prototype asap (one was started under the "VSP" name (virtual
> site-packages) but not finished yet)
> 
> Regards
> Tarek
> 

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/61cda4f3/attachment.pgp>

From lists at cheimes.de  Thu Oct  8 13:17:46 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 08 Oct 2009 13:17:46 +0200
Subject: [Python-Dev] PEP 370 and IronPython
Message-ID: <4ACDCA5A.4090609@cheimes.de>

Dear Pythonistas!

Michael Foord has written a blog posting [1] regarding IronPython, site
packages and my PEP 370. I'm referring to the section that talks about
an issue with PEP 370.

---
If you install CPython 2.6 on Windows it creates a folder in the
location: %APPDATA%\Python\Python26\site-packages. On my machine the
full path is:
C:\Users\michael\AppData\Roaming\Python\Python26\site-packages.

This is put on the path by site.py when you run Python and as it is a
user directory it doesn't require elevation to install there. You
install into the user directory rather than the 'usual' location with:

    python setup.py --user install

Herein lies another problem though. IronPython uses the standard Python
site.py, meaning that currently it shares the user site-packages folder
with CPython. This is not ideal as typically you will want separate
libraries install for IronPython. When Jython ports to Python 2.6 it
will have the same issue.
---

I didn't think about IronPython during the design phase of the PEP. I
didn't come into my mind that another implementation may require a
different site packages directory. After spinning my head around the
issue a simple solution came to me.

The solution requires a new attribute in the sys module that contains
the name of the implementation. As an alternative we could use the first
field of sys.subversion but I prefer an explicit attribute. I'm
proposing "sys.name" with a value of "CPython", "IronPython", "Jython"
or "PyPy". The site module uses the information of the attribute to
modify the path to the user site directory.

The altered user site directories are:

CPython:
  ~/.local/lib/python2.6/site-packages
  %APPDATA%/Python/Python26

IronPython:
  ~/.local/lib/ironpython2.6/site-packages
  %APPDATA%/Python/IronPython26

Jython:
  ~/.local/lib/jython2.6/site-packages
  %APPDATA%/Python/Jython26


Proposed patch:

Index: Lib/site.py
===================================================================
--- Lib/site.py (revision 75282)
+++ Lib/site.py (working copy)
@@ -74,7 +74,12 @@
 USER_SITE = None
 USER_BASE = None

+interpreter_name = getattr(sys, "name", "CPython")
+if interpreter_name == "CPython":
+    # b/w compatible name
+    interpreter_name = "python"

+
 def makepath(*paths):
     dir = os.path.abspath(os.path.join(*paths))
     return dir, os.path.normcase(dir)
@@ -253,10 +258,12 @@
         return USER_SITE

     if os.name == "nt":
-        USER_SITE = os.path.join(user_base, "Python" + sys.version[0] +
-                                 sys.version[2], "site-packages")
+        USER_SITE = os.path.join(user_base, interpreter_name.capitalize() +
+                                 sys.version[0] + sys.version[2],
+                                 "site-packages")
     else:
-        USER_SITE = os.path.join(user_base, "lib", "python" +
sys.version[:3],
+        USER_SITE = os.path.join(user_base, "lib",
+                                 interpreter_name.lower() +
sys.version[:3],
                                  "site-packages")

     return USER_SITE


The patch doesn't modify the search path for global directories. These
can be changed by altering sys.prefix and sys.exec_prefix.

Christian


[1]
http://ironpython-urls.blogspot.com/2009/10/distributing-ironpython-packages.html

From mal at egenix.com  Thu Oct  8 13:27:57 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 13:27:57 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on	Virtualenv, Pip)
In-Reply-To: <4ACDB82F.7010006@cryptelium.net>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACDB82F.7010006@cryptelium.net>
Message-ID: <4ACDCCBD.4040908@egenix.com>

I have just a few comments to make, so I'm trimming the long quote
a bit...

> Tarek Ziad? a ?crit :
>> == Version comparison ==
>>
>> One very important point: we will not force the community to use the
>> scheme described in PEP 386, but *there is* already a
>> de-facto convention on version schemes at PyPI if you use Pip or
>> easy_install, so let's have a documented standard for this,
>> and a reference implementation in Distutils.
>>
>> There's an implementation at
>> http://bitbucket.org/tarek/distutilsversion that is kept up-to-date
>> with PEP 386.
>>
>> - Work left to do in PEP 386 : another round with the community
>> - Goal: 2.7 / 3.2

Please make sure that the version scheme is compatible with
other tools we use in the distutils chain: RPM and the MSI
compiler.

Some data points: RPM doesn't like hyphens in version strings.
MSI requires a strict N.N.N format.

>> == The fate of setup.py, and static metadata ==
>>
>> So we are going to separate the metadata description from setup.py, in
>> a static configuration file, that can be open and read by anyone
>> without
>> running any code. The only problem with this is the fact that some
>> metadata fields might depend on the execution environment. For
>> instance, once "Requires" is re-defined and re-introduced via PEP 314,
>> we will have cases where "pywin32" will be a dependency to have only
>> on win32 systems.

How are you planning to deal with dynamic version strings, e.g.
say you want to build a snapshot release the current date and
revision number included in the version string ?

I suppose this file is going to be uploaded to PyPI, so the
file would have to be formatted at build time and include
markers to be replaced with values taken from the distribution
meta-data.

>> - Work left to do : publish the final syntax, and do the implementation

Is there a PEP for this ?

>> == The fate of bdist_* commands ==
>>
>> So the best way to handle this is to ask these communities to build
>> their own tool and to encourage them to use Distutils as a basis for
>> that.

Rather than having external communities build their own
tools and then basically fork distutils in order to get
these implemented, I think it'd be better to make the
set of distutils commands extensible via namespace packages.

>> So what is going to happen is a status-quo: no bdist_* command will be
>> removed but no new bdist_* command will be added. 

I hope that comment means "no new bdist_* command will be added
*right now*". It's well possible to create new bdist_* commands
that don't rely on external tool chains or only on ones that
don't change often.

I'm thinking of bdist_inno (Windows installer using InnoSetup)
and bdist_nsis (Windows installer using NSIS) here.

>> = Virtualenv and the multiple version support in Distribute =
>>
>> (I am not saying "We" here because this part was not discussed yet
>> with everyone)
>>
>> Virtualenv allows you to create an isolated environment to install
>> some distribution without polluting the
>> main site-packages, a bit like a user site-packages.
>>
>> My opinion is that this tool exists only because Python doesn't
>> support the installation of multiple versions for the same
>> distributions.
>
> Not only, for me it exists also to provide a way to isolate/chroot an
> application from the host system.

I agree: IMHO, that's the main use case for things like virtualenv.

Note that Python has had a PYTHONHOME environment variable for ages.
This pretty much serves the same purpose and together with setting
PYTHONPATH can easily be used to create your own private Python
universe.

>> But if PEP 376 and PEP 386 support are added in Python, we're not far
>> from being able to provide multiple version support with
>> the help of importlib.

Before putting much work into this: do you really think that having
multiple versions of the same package in the same Python installation
is a good idea ?

Examples:
What if you have an application that uses two modules which each
require two different versions of the same package ? Would that
load the same package twice, causing globals used by these package
to no work (e.g. isinstance(x, class_name) or global caches) ?

This sounds a lot like DLL- or RPM-hell to me.

I think it's much better to keep things simple and under user
control, e.g. by encouraging use of virtualenv-like setups
and perhaps adding better native support for these to Python.

If the user finds that there's a version conflict this can
then be resolved during environment setup instead of hoping
for the best and waiting for application failures at run-time.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Thu Oct  8 13:28:48 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 13:28:48 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <94bdd2610910080159r319b0fdtd02b779bba893e18@mail.gmail.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>	<4ACD965D.5080701@egenix.com>	<94bdd2610910080133y788cb0d6xd67aaf3b58ac48a0@mail.gmail.com>	<4ACDA4EE.4050005@egenix.com>
	<94bdd2610910080159r319b0fdtd02b779bba893e18@mail.gmail.com>
Message-ID: <4ACDCCF0.8070107@egenix.com>

Tarek Ziad? wrote:
> On Thu, Oct 8, 2009 at 10:38 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Tarek Ziad? wrote:
>>> On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>> BTW: Who would I need to contact for the PyPI side to work out
>>>> an upload_url distutils command ?
>>>
>>> I am not sure to understand. How upload_url will differ from the
>>> register command
>>> that let you upload metadata at PyPI ?
>>
>> The register command only uploads the package's meta-data.
>>
>> The upload command uploads a file and registers the file's
>> details with PyPI as well.
>>
>> I'd like to add an upload_url command which only registers a
>> file's details (including the URL, but without the actual file)
>> with PyPI and perhaps extend the number of attributes sent to
>> PyPI with more file configuration information (upload currently
>> only sends the Python version).
> 
> Sounds like an option to be added in upload.

With a little code refactoring that would be possible as well, yes.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Thu Oct  8 13:43:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 8 Oct 2009 11:43:24 +0000 (UTC)
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on	Virtualenv, Pip)
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <hakj8s$7a9$1@ger.gmane.org>


Hello,

Thanks for the summary :)

[Distribute 0.7]
> - easy_install is going to be deprecated ! use Pip !

Apparently there are a couple of things Pip doesn't support:

? It cannot install from eggs. It only installs from source. (Maybe this 
will be changed sometime, but it?s low priority.) ?
? It is incompatible with some packages that customize distutils or 
setuptools in their setup.py files. ?
? Maybe it doesn?t work on Windows. At least, the author doesn?t test on 
Windows often. ?

(from http://pip.openplans.org/#differences-from-easy-install)

> Virtualenv allows you to create an isolated environment to install
> some distribution without polluting the
> main site-packages, a bit like a user site-packages.
> 
> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.

virtualenv has the virtue of keeping your main installation clean and
manageable. Besides, as someone said, it also allows you to create 
completely isolated environments if you want full control over your 
deployment 
(especially when you're not the sysadmin).

Regards

Antoine.



From solipsis at pitrou.net  Thu Oct  8 13:47:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 8 Oct 2009 11:47:25 +0000 (UTC)
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on	Virtualenv, Pip)
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACDB82F.7010006@cryptelium.net> <4ACDCCBD.4040908@egenix.com>
Message-ID: <hakjgd$7a9$2@ger.gmane.org>

Le Thu, 08 Oct 2009 13:27:57 +0200, M.-A. Lemburg a ?crit?:
> 
> This sounds a lot like DLL- or RPM-hell to me.
> 
> I think it's much better to keep things simple and under user control,
> e.g. by encouraging use of virtualenv-like setups and perhaps adding
> better native support for these to Python.

Agreed.

virtualenv enables a feature while keeping things simple and clear.
The proposed scheme (multi-versioned stuff) is supposed to enable the 
same feature, but by making things more complicated. ;)


From ncoghlan at gmail.com  Thu Oct  8 13:53:39 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 08 Oct 2009 21:53:39 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091007T191603-331@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>
	<loom.20091007T191603-331@post.gmane.org>
Message-ID: <4ACDD2C3.4090608@gmail.com>

Antoine Pitrou wrote:
> Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
>>>>> "%0#8x" % 0x1234
>> '0x001234'
>>>>> "{0:0>#8x}".format(0x1234)
>> '000x1234'
> 
> Apart from the sheer unreadability of the {}-style format string, the result 
> looks rather unexpected from a human being's point of view.

The percent format string is pretty unreadable too - you're just more
used to it, so it doesn't look as weird :)

Vinay's problem above is due to using the wrong alignment flag: ">",
which says to right align everything, instead of "=", which says to left
align the sign character and the numeric prefix with the fill character
inserted in the middle. In this particular case he could also use the
zero-padding shortcut which leaves out the alignment flag altogether
(and implies a "0=" alignment format).

That is (using 2.6/3.1):

>>> "%#08x" % 0x1234
'0x001234'
>>> "%0#8x" % 0x1234
'0x001234'
>>> "{0:#08x}".format(0x1234)
'0x001234'
>>> "{0:0>#8x}".format(0x1234)
'000x1234'
>>> "{0:0=#8x}".format(0x1234)
'0x001234'

Adding in the sign bit gives the following translations:

>>> "%+#08x" % 0x1234
'+0x01234'
>>> "%+0#8x" % 0x1234
'+0x01234'
>>> "{0:+#08x}".format(0x1234)
'+0x01234'
>>> "{0:0>+#8x}".format(0x1234)
'0+0x1234'
>>> "{0:0=+#8x}".format(0x1234)
'+0x01234'

Note that ">" alignment is actually now *broken* on trunk and py3k,
since ">" and "=" are now behaving exactly the same instead of the
former right aligning the entire number including the sign bit and prefix:

>>> "{:0>+8x}".format(0x1234)
'+0001234'
>>> "{:0=+8x}".format(0x1234)
'+0001234'

(bug assigned to Eric: http://bugs.python.org/issue7081)

Note that, since percent formatting doesn't allow specification of the
fill characters or the field alignment, translations should probably
rely on the simple field width specifier, optionally selecting zero
padding by preceding it with a zero. It should never be necessary to use
the full alignment spec for translated formats.

The other thing to keep in mind is that brace formatting is fussier
about the order of things - items *must* appear in the order they are
listed in PEP 3101 (i.e. if wanting a zero padded field with leading
sign and numeric prefix, you must write "+#0"). Percent format, on the
other hand, allows the "#", "+" and "0" to be placed in any order you
like (although they must appear before the field width definition,
precision specifier and type code).

As far as I can see, that leaves the prefixing of octal numbers ("0o" vs
"0") as the only true incompatibility between percent formatting and
brace formatting, and even for those the incompatibility is limited to
cases where a field width is specified without leading zeroes or a sign
character is specified. In other cases, the translation can just stick a
leading literal "0" in front of the field in the brace formatting string.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Oct  8 14:22:00 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 08 Oct 2009 22:22:00 +1000
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACDCA5A.4090609@cheimes.de>
References: <4ACDCA5A.4090609@cheimes.de>
Message-ID: <4ACDD968.1070205@gmail.com>

Christian Heimes wrote:
> The solution requires a new attribute in the sys module that contains
> the name of the implementation. As an alternative we could use the first
> field of sys.subversion but I prefer an explicit attribute. I'm
> proposing "sys.name" with a value of "CPython", "IronPython", "Jython"
> or "PyPy".

My Google skills almost failed me, but searching for "sys.vm" found me
what I was after: http://bugs.python.org/issue4242 (a discussion
relating to a similar need in the context of flagging implementation
specific tests).

As mentioned in that discussion, as of Python 2.6, you can do the following:
>>> import platform
>>> platform.python_implementation()
'CPython'

(Although according to the function docstring, PyPy is currently missing
from the list of known implementations)

Importing yet-another-module for use in site.py doesn't sound like a
great idea, so it may make sense to migrate that information into the
sys module is this approach is taken. "sys.name" is a little generic
though - something more explicit like "sys.vm" would be better.

> The site module uses the information of the attribute to
> modify the path to the user site directory.

Implementation specific user directories sound like a good idea indeed.

An alternative to a lookup table approach might be to be a little more
direct and just retrieve the final part of the user specific directory
name directly from a new function in the sys module. Then different VM
authors can choose any directory name they want without CPython's
site.py needing to know anything else about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From zvezdan at zope.com  Thu Oct  8 14:45:54 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Thu, 8 Oct 2009 08:45:54 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
Message-ID: <FDE6DC02-40CC-49DB-BDEA-B9C47ABCF359@zope.com>

On Oct 7, 2009, at 2:09 PM, Barry Warsaw wrote:
> I want us to be very careful about 2.6.4.  This isn't a normal bug  
> fix release, it's specifically there to remove the brown paper bag  
> of 2.6.3 from our heads.  So let's be conservative and fix this one  
> in 2.6.5.

Can we get the readline patch reviewed as well.  It was applied to  
trunk already: http://svn.python.org/view?view=rev&revision=74970

It's marked for backport to 2.6 as "needs review":
http://bugs.python.org/issue6877

It fixes a BusError crash that was just swiped under a rug by  
disabling the build in setup.py.  :-)

FWIW, the part of the patch that changes setup.py will be useful for  
another comparison of Mac OS X release numbers.  The line 1362 on the  
trunk (1364 on the 2.6 branch) compares strings and, unfortunately,   
'10.' < '8.' for Darwin 10 (Mac OS X 10.6 Snow Leopard):

	if os.uname()[2] > '8.':

So, it should be changed to:

	if os_release > 8:

where ``os_release`` is introduced by the patch mentioned above as:

	os_release = int(os.uname()[2].split('.')[0])

on the line 550 on the trunk.

Regards,

	Zvezdan


From hodgestar+pythondev at gmail.com  Thu Oct  8 14:52:24 2009
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 8 Oct 2009 14:52:24 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
Message-ID: <fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>

 On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> = Virtualenv and the multiple version support in Distribute =
...
> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.

This is not at all how I use virtualenv. For me virtualenv is a
sandbox so that I don't have to become root whenever I need to install
a Python package for testing purposes and to allow me to hop between
sets of installed Python packages while developing on multiple Python
projects. I also use it as a sandbox for build bots so that multiple
bots on the same machine can each build their own projects with just
the known dependencies installed.

Schiavo
Simon

From ziade.tarek at gmail.com  Thu Oct  8 14:54:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 14:54:44 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACDCCBD.4040908@egenix.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACDB82F.7010006@cryptelium.net> <4ACDCCBD.4040908@egenix.com>
Message-ID: <94bdd2610910080554m530bfdbeg2bf010c813c71873@mail.gmail.com>

On Thu, Oct 8, 2009 at 1:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>
>>> == The fate of setup.py, and static metadata ==
>>>
>>> So we are going to separate the metadata description from setup.py, in
>>> a static configuration file, that can be open and read by anyone
>>> without
>>> running any code. The only problem with this is the fact that some
>>> metadata fields might depend on the execution environment. For
>>> instance, once "Requires" is re-defined and re-introduced via PEP 314,
>>> we will have cases where "pywin32" will be a dependency to have only
>>> on win32 systems.
>
> How are you planning to deal with dynamic version strings, e.g.
> say you want to build a snapshot release the current date and
> revision number included in the version string ?
>
> I suppose this file is going to be uploaded to PyPI, so the
> file would have to be formatted at build time and include
> markers to be replaced with values taken from the distribution
> meta-data.

The metadata will always be customizable from setup.py somehow.

We won't treat those cases : if the metadata is provided in the static file,
you can get it without any further processing, if not you will get an
"UNKOWN" value
for it and you will have to grab the whole archive and run setup.py to get it.

Now the example you've shown is in my opinion not a problem : you can
create the static value for "version" that you put in your static file
with some pre-processing that
occurs when you build your release. (ala autoconf)


>
>>> - Work left to do : publish the final syntax, and do the implementation
>
> Is there a PEP for this ?

No, but if you ask for it I can write it.

>
>>> == The fate of bdist_* commands ==
>>>
>>> So the best way to handle this is to ask these communities to build
>>> their own tool and to encourage them to use Distutils as a basis for
>>> that.
>
> Rather than having external communities build their own
> tools and then basically fork distutils in order to get
> these implemented, I think it'd be better to make the
> set of distutils commands extensible via namespace packages.

I am not sure, why they would fork distutils.

Distutils is already extensible, you can configure in distutils.cfg a
list of namespace packages,
that contains commands and they'll get loaded by dist.py.


>
>>> So what is going to happen is a status-quo: no bdist_* command will be
>>> removed but no new bdist_* command will be added.
>
> I hope that comment means "no new bdist_* command will be added
> *right now*". It's well possible to create new bdist_* commands
> that don't rely on external tool chains or only on ones that
> don't change often.
>
> I'm thinking of bdist_inno (Windows installer using InnoSetup)
> and bdist_nsis (Windows installer using NSIS) here.

As I said, windows don't have any packaging system so I don't think
it's a problem to have more installers for this platform.

Regards

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From ziade.tarek at gmail.com  Thu Oct  8 15:16:27 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 15:16:27 +0200
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACDCA5A.4090609@cheimes.de>
References: <4ACDCA5A.4090609@cheimes.de>
Message-ID: <94bdd2610910080616w27b43020sa52670269e01b7ff@mail.gmail.com>

On Thu, Oct 8, 2009 at 1:17 PM, Christian Heimes <lists at cheimes.de> wrote:
[...]
> The altered user site directories are:
>
> CPython:
> ?~/.local/lib/python2.6/site-packages
> ?%APPDATA%/Python/Python26
>
> IronPython:
> ?~/.local/lib/ironpython2.6/site-packages
> ?%APPDATA%/Python/IronPython26
>
> Jython:
> ?~/.local/lib/jython2.6/site-packages
> ?%APPDATA%/Python/Jython26
>

Hi Christian,

+1

I have a suggestion though, that completes Nick's answer.

distutils/command/install.py already contains install schemes and
imports USER_SITE and USER_BASE to builds user-specific install
schemes. I think it wouldn't hurt for clarity to reunite all these
schemes in a single place in the stdlib (so in sys maybe?)

Tarek

From fuzzyman at voidspace.org.uk  Thu Oct  8 15:52:57 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 08 Oct 2009 14:52:57 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
Message-ID: <4ACDEEB9.2000308@voidspace.org.uk>

Simon Cross wrote:
>  On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>   
>> = Virtualenv and the multiple version support in Distribute =
>>     
> ...
>   
>> My opinion is that this tool exists only because Python doesn't
>> support the installation of multiple versions for the same
>> distributions.
>>     
>
> This is not at all how I use virtualenv. For me virtualenv is a
> sandbox so that I don't have to become root whenever I need to install
> a Python package for testing purposes and to allow me to hop between
> sets of installed Python packages while developing on multiple Python
> projects. I also use it as a sandbox for build bots so that multiple
> bots on the same machine can each build their own projects with just
> the known dependencies installed.
>
>   

This is exactly why I use virtualenv as well. I don't recall ever having 
wanted / needed to install multiple versions of the same library - 
whilst I can appreciate that it *can* be a real issue it has never been 
a problem for me.

Michael

> Schiavo
> Simon
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From anders at 0x63.nu  Thu Oct  8 15:43:44 2009
From: anders at 0x63.nu (Anders Waldenborg)
Date: Thu, 08 Oct 2009 15:43:44 +0200
Subject: [Python-Dev] locals() different behaviour when tracing?
Message-ID: <hakqag$vt6$1@ger.gmane.org>


Is it intentional that locals() returns a copy/snapshot of the local 
variables sometimes, and sometimes a reference? Like when enabling 
tracing, such as in the code pasted below. The documentation ("Update 
and return a dictionary containing the current scope's local 
variables.") is pretty unclear.

import sys

def X():
     l = locals()
     i = "foo"
     print "Is 'i' in stored locals()? ", ('i' in l)

print "Running normally"
X()

print "Enabling tracing"
def t(*a):
     return t
sys.settrace(t)
X()


From v+python at g.nevcal.com  Thu Oct  8 09:44:22 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 08 Oct 2009 00:44:22 -0700
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <loom.20091008T074346-184@post.gmane.org>
References: <loom.20091007T164835-226@post.gmane.org>	<4ACD4484.2080409@g.nevcal.com>
	<loom.20091008T074346-184@post.gmane.org>
Message-ID: <4ACD9856.1080206@g.nevcal.com>

On approximately 10/7/2009 10:45 PM, came the following characters from 
the keyboard of Vinay Sajip:
> Glenn Linderman <v+python <at> g.nevcal.com> writes:
> 
>> But DictConfigurator the name seems misleading... like you are 
>> configuring how dicts work, rather than how logs work.  Maybe with more 
>> context this is not a problem, but if it is a standalone class, it is 
>> confusing what it does, by name alone.
> 
> Of course the fully qualified name would be logging.config.DictConfigurator
> which does provide the requisite context, but if I think of/someone suggests a
> better name I'll certainly use it.


Yes, I suspected it might be something like that, and that does provide 
the context, so I'm content.  Thanks for verifying it, and the rest is 
bikeshedding...

In thinking more about this, I guess I would have the same comment about 
the existing fileConfig() API also... it isn't file that is being 
configured, but the logs.  readConfigFromFile would have been more 
descriptive (and lots longer :( ).

Had fileConfig been called "bulkConfig" or "configAll" (these names 
attempt to contrast the function of this API versus the individual APIs 
that configure one thing at a time) taking a file parameter, then it 
could just simply/suddenly also allow a dict parameter, and no new API 
would have to be created.  Too late for this, however, and with the new 
API, it would be possible to deprecate the fileConfig API, and tell the 
user to use the ConfigParser (or JSON or YAML or whatever) to create the 
dict, and then pass that to DictConfigurator (or bulkConfig or whatever).


-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From vinay_sajip at yahoo.co.uk  Thu Oct  8 16:16:18 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 14:16:18 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>
	<loom.20091007T191603-331@post.gmane.org>
	<4ACDD2C3.4090608@gmail.com>
Message-ID: <loom.20091008T160529-262@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Vinay's problem above is due to using the wrong alignment flag: ">",
> which says to right align everything, instead of "=", which says to left
> align the sign character and the numeric prefix with the fill character
> inserted in the middle. In this particular case he could also use the
> zero-padding shortcut which leaves out the alignment flag altogether
> (and implies a "0=" alignment format).
> 
[snip]
> Note that, since percent formatting doesn't allow specification of the
> fill characters or the field alignment, translations should probably
> rely on the simple field width specifier, optionally selecting zero
> padding by preceding it with a zero. It should never be necessary to use
> the full alignment spec for translated formats.
> 
> The other thing to keep in mind is that brace formatting is fussier
> about the order of things - items *must* appear in the order they are
> listed in PEP 3101 (i.e. if wanting a zero padded field with leading
> sign and numeric prefix, you must write "+#0"). Percent format, on the
> other hand, allows the "#", "+" and "0" to be placed in any order you
> like (although they must appear before the field width definition,
> precision specifier and type code).
> 
> As far as I can see, that leaves the prefixing of octal numbers ("0o" vs
> "0") as the only true incompatibility between percent formatting and
> brace formatting, and even for those the incompatibility is limited to
> cases where a field width is specified without leading zeroes or a sign
> character is specified. In other cases, the translation can just stick a
> leading literal "0" in front of the field in the brace formatting string.

Helpful analysis there, Nick, thanks. Bonzer ;-)

There's also the corner case of things like

%#.0f

which, when asked to format 3e100, will print

3.e+100

whereas the translated format {0:.0f}, will print

3e+100

for the same value.

BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
worth raising an enhancement request on the bug tracker using "O" to generate
compatible rendering for octals.

Regards,

Vinay Sajip


From eric at trueblade.com  Thu Oct  8 16:24:33 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 08 Oct 2009 10:24:33 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20091008T160529-262@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>	<loom.20091007T191603-331@post.gmane.org>	<4ACDD2C3.4090608@gmail.com>
	<loom.20091008T160529-262@post.gmane.org>
Message-ID: <4ACDF621.4070603@trueblade.com>

Vinay Sajip wrote:
> BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
> worth raising an enhancement request on the bug tracker using "O" to generate
> compatible rendering for octals.

I didn't get your message, could you resend?.

I was thinking the same thing, but it seems like a transition step. I'd 
rather not keep such backward compatibility hacks (if you will) around 
for the long haul.  How about a flag (maybe '*') at the start of the 
format specification which says "operate in backward compatibility 
mode"? We could document it as being only useful for the % to {} 
translator, and promise to remove it at some point in the future. Either 
actually deprecate it or just promise to deprecate it in the future.

Eric.


From olemis at gmail.com  Thu Oct  8 16:26:17 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 8 Oct 2009 09:26:17 -0500
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <4ACD9856.1080206@g.nevcal.com>
References: <loom.20091007T164835-226@post.gmane.org>
	<4ACD4484.2080409@g.nevcal.com>
	<loom.20091008T074346-184@post.gmane.org>
	<4ACD9856.1080206@g.nevcal.com>
Message-ID: <24ea26600910080726h715dbc4ex23bce17740a3c390@mail.gmail.com>

On Thu, Oct 8, 2009 at 2:44 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On approximately 10/7/2009 10:45 PM, came the following characters from the
> keyboard of Vinay Sajip:
>>
>> Glenn Linderman <v+python <at> g.nevcal.com> writes:
>>
>>> But DictConfigurator the name seems misleading... like you are
>>> configuring how dicts work, rather than how logs work. ?Maybe with more
>>> context this is not a problem, but if it is a standalone class, it is
>>> confusing what it does, by name alone.
>>
>> Of course the fully qualified name would be
>> logging.config.DictConfigurator
>> which does provide the requisite context, but if I think of/someone
>> suggests a
>> better name I'll certainly use it.
>
[...]
>
> In thinking more about this, I guess I would have the same comment about the
> existing fileConfig() API also... it isn't file that is being configured,
> but the logs. ?readConfigFromFile would have been more descriptive (and lots
> longer :( ).
>

cfgFromIni | cfgFromFile ? I prefer the former because YAML, shelves,
JSON & Co can also be stored in a file , so at least I get a better
understanding of the format in use. Otherwise I'd had to read the docs
or figure out which one

> Had fileConfig been called "bulkConfig" or "configAll" (these names attempt
> to contrast the function of this API versus the individual APIs that
> configure one thing at a time) taking a file parameter, then it could just
> simply/suddenly also allow a dict parameter, and no new API would have to be
> created. ?Too late for this, however, and with the new API, it would be
> possible to deprecate the fileConfig API, and tell the user to use the
> ConfigParser (or JSON or YAML or whatever) to create the dict, and then pass
> that to DictConfigurator (or bulkConfig or whatever).
>

similar and consistent with what I mentioned before but IMO fileConfig
should remain for backwards compatibility

Objections ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Looking for a technique to create flexible, graphical dashboards ...
- http://feedproxy.google.com/~r/TracGViz-full/~3/71kRhT34BgU/43789

From barry at python.org  Thu Oct  8 16:30:11 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 8 Oct 2009 10:30:11 -0400
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <EF9B4029-235D-49FD-86A1-897C2E6B5564@python.org>

On Oct 8, 2009, at 4:31 AM, Tarek Ziad? wrote:

> - no more namespaced packages system, if PEP 381 (namespaces package
> support) makes it to 2.7

Make that PEP 382:

http://www.python.org/dev/peps/pep-0382/

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/406af4ce/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Thu Oct  8 16:47:59 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 14:47:59 +0000 (GMT)
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <09B60FD3-D227-43E3-B901-5418C7FA950C@gmail.com>
References: <loom.20091007T164835-226@post.gmane.org>
	<09B60FD3-D227-43E3-B901-5418C7FA950C@gmail.com>
Message-ID: <137556.63116.qm@web25801.mail.ukl.yahoo.com>

> I've had bad experiences in the past with dictionary-based APIs.  They seem 

> "simpler" in the short run, because the user "only needs to create some 
> dictionaries".  Once the complexity of that nested dictionary grows to a certain 
> point, though, one has to refer back to documentation constantly to make sure

Fair point, and agreed that the schema needs some care, here's roughly what I'm thinking of as an example configuration (YAML):

formatters:
  brief:
    format: '%(levelname)-8s: %(name)-15s: %(message)s'
  precise:
    format: '%(asctime)s %(name)-15s %(levelname)-8s %(message)s'
filters:
  allow_foo:
    name: foo
handlers:
  console:
    class : logging.StreamHandler
    formatter: brief
    level   : INFO
    stream  : sys.stdout
    filters: [allow_foo]
  file:
    class : logging.handlers.RotatingFileHandler
    formatter: precise
    filename: logconfig.log
    maxBytes: 1024
    backupCount: 3
  debugfile:
    class : logging.FileHandler
    formatter: precise
    filename: logconfig-detail.log
    mode: a
loggers:
  foo:
    level : ERROR
    handlers: [debugfile]
  spam:
    level : CRITICAL
    handlers: [debugfile]
    propagate: no
  bar.baz:
    level: WARNING

root:
  level     : DEBUG
  handlers  : [console, file]

It's not too deeply nested, and I can't see any need for it being more deeply nested. It more or less mirrors the way you'd have to write the corresponding ConfigParser-compatible configuration, but if you use a dict as a common format, you have the benefits which I mentioned of supporting JSON, YAML or Python code in a Django settings.py.
 
> the structure conforms to the "schema".  Building a simple config tree using 
> light-weight classes with documented APIs tends to be more sustainable in the 
> long run.

When you say 'config tree using light-weight classes', I presume you mean a parallel set of classes to the actual logging classes which will be instantiated? ISTM logging classes are already reasonably usable for programmatic configuration, but this doesn't address e.g. updating configurations easily without program code changes, or the ability to configure from say a YAML node, where YAML may be being used for application configuration as a whole. (Unless of course I've misunderstood what you're getting at.)

Regards,

Vinay Sajip



      

From vinay_sajip at yahoo.co.uk  Thu Oct  8 16:54:53 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 14:54:53 +0000 (GMT)
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACDF621.4070603@trueblade.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<loom.20091001T010253-42@post.gmane.org>
	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>
	<ha69vt$qso$1@ger.gmane.org>
	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>
	<loom.20091005T102425-605@post.gmane.org>
	<loom.20091007T191603-331@post.gmane.org>
	<4ACDD2C3.4090608@gmail.com>
	<loom.20091008T160529-262@post.gmane.org>
	<4ACDF621.4070603@trueblade.com>
Message-ID: <67182.50646.qm@web25806.mail.ukl.yahoo.com>

> I didn't get your message, could you resend?.


Resent, it may have been stopped by your spam filters since it came from my vinay-underscore-sajip-at-red-hyphen-dove-dot-com address. The subject was "Python str.format() and octal formatting compatibility".

> I was thinking the same thing, but it seems like a transition step. I'd rather 
> not keep such backward compatibility hacks (if you will) around for the long 
> haul.  How about a flag (maybe '*') at the start of the format specification 
> which says "operate in backward compatibility mode"? We could document it as 
> being only useful for the % to {} translator, and promise to remove it at some 
> point in the future. Either actually deprecate it or just promise to deprecate 
> it in the future.

I don't much mind matter exactly which mechanism we use to distinguish between 0o and 0 prefixes, as long as it's one most people are happy with :-)

Regards,

Vinay Sajip



      

From mal at egenix.com  Thu Oct  8 16:55:16 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 16:55:16 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080554m530bfdbeg2bf010c813c71873@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<4ACDB82F.7010006@cryptelium.net>
	<4ACDCCBD.4040908@egenix.com>
	<94bdd2610910080554m530bfdbeg2bf010c813c71873@mail.gmail.com>
Message-ID: <4ACDFD54.3080202@egenix.com>

Tarek Ziad? wrote:
> On Thu, Oct 8, 2009 at 1:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>>>> == The fate of setup.py, and static metadata ==
>>>>
>>>> So we are going to separate the metadata description from setup.py, in
>>>> a static configuration file, that can be open and read by anyone
>>>> without
>>>> running any code. The only problem with this is the fact that some
>>>> metadata fields might depend on the execution environment. For
>>>> instance, once "Requires" is re-defined and re-introduced via PEP 314,
>>>> we will have cases where "pywin32" will be a dependency to have only
>>>> on win32 systems.
>>
>> How are you planning to deal with dynamic version strings, e.g.
>> say you want to build a snapshot release the current date and
>> revision number included in the version string ?
>>
>> I suppose this file is going to be uploaded to PyPI, so the
>> file would have to be formatted at build time and include
>> markers to be replaced with values taken from the distribution
>> meta-data.
> 
> The metadata will always be customizable from setup.py somehow.
> 
> We won't treat those cases : if the metadata is provided in the static file,
> you can get it without any further processing, if not you will get an
> "UNKOWN" value
> for it and you will have to grab the whole archive and run setup.py to get it.
> 
> Now the example you've shown is in my opinion not a problem : you can
> create the static value for "version" that you put in your static file
> with some pre-processing that
> occurs when you build your release. (ala autoconf)

Hmm, then I'm not really sure I understand the proposal for a
new static meta-data file...

distutils already creates the static meta-data file PKG-INFO
when building sdist distributions and that's done based on the
data read from the setup() call. It would be easy to use that
file as static meta-data file in other distribution formats
as well (e.g. like you proposed in PEP 376).

OTOH, the register command sends the meta-data directly to
the PyPI database, so it doesn't even need another file
for storing away the meta data.

Unless I'm missing something important (which I probably am),
the only added value of the new format over PKG-INFO is the
micro-language.

Couldn't we just stick that into the values of the various
meta-data fields, e.g. to have the Requires field be different
depending on platform ?

We'd then only have one meta-data format in distutils -
the PKG-INFO one.

>>>> - Work left to do : publish the final syntax, and do the implementation
>>
>> Is there a PEP for this ?
> 
> No, but if you ask for it I can write it.

Please do. The format will need documentation anyway and the
motivation should be made clear as well.

>>>> == The fate of bdist_* commands ==
>>>>
>>>> So the best way to handle this is to ask these communities to build
>>>> their own tool and to encourage them to use Distutils as a basis for
>>>> that.
>>
>> Rather than having external communities build their own
>> tools and then basically fork distutils in order to get
>> these implemented, I think it'd be better to make the
>> set of distutils commands extensible via namespace packages.
> 
> I am not sure, why they would fork distutils.

Not really a fork, but a copy of distutils in order to give
the user the complete tool set in one download, rather than
having to configure a system installed distutils to load
the new command.

> Distutils is already extensible, you can configure in distutils.cfg a
> list of namespace packages,
> that contains commands and they'll get loaded by dist.py.

Right, but dist.py could load all distutils.command modules
it can find to make the installation of extensions more user
friendly (like in a plugin system).

Just a minor nit, though.

>>>> So what is going to happen is a status-quo: no bdist_* command will be
>>>> removed but no new bdist_* command will be added.
>>
>> I hope that comment means "no new bdist_* command will be added
>> *right now*". It's well possible to create new bdist_* commands
>> that don't rely on external tool chains or only on ones that
>> don't change often.
>>
>> I'm thinking of bdist_inno (Windows installer using InnoSetup)
>> and bdist_nsis (Windows installer using NSIS) here.
> 
> As I said, windows don't have any packaging system so I don't think
> it's a problem to have more installers for this platform.

Ok.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From eric at trueblade.com  Thu Oct  8 16:55:21 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 08 Oct 2009 10:55:21 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <E1MvuKa-0004Sm-36@swing.co.at>
References: <4ACDF621.4070603@trueblade.com> <E1MvuKa-0004Sm-36@swing.co.at>
Message-ID: <4ACDFD59.4060200@trueblade.com>

Christian Tanzer wrote:
> Eric Smith wrote at Thu, 08 Oct 2009 10:24:33 -0400:
> 
>> Vinay Sajip wrote:
>>> BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
>>> worth raising an enhancement request on the bug tracker using "O" to generate
>>> compatible rendering for octals.
>> I didn't get your message, could you resend?.
>>
>> I was thinking the same thing, but it seems like a transition step. I'd
>> rather not keep such backward compatibility hacks (if you will) around
>> for the long haul.  How about a flag (maybe '*') at the start of the
>> format specification which says "operate in backward compatibility
>> mode"? We could document it as being only useful for the % to {}
>> translator, and promise to remove it at some point in the future. Either
>> actually deprecate it or just promise to deprecate it in the future.
> 
> That doesn't seem very useful to me. IIUC, the point of the translator
> is to allow porting of the millions of existing %-formating operations
> to the new-style .format.
> 
> If the result of that is deprecated or removed a few years from now,
> all maintainers of long existing code have exactly the same problem.

I was thinking of it as a transition step until all application code 
switched to {} formatting. In which case the application has to deal 
with it.

> IMHO, either the translation is done once and gives identical output or
> it isn't worth doing at all.

I disagree. I doubt even 0.001% of all format strings involve octal 
formatting. Is it really worth not providing a transition path if it 
can't cover this case?

Eric.


From benjamin at python.org  Thu Oct  8 16:25:10 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 8 Oct 2009 09:25:10 -0500
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACDD968.1070205@gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACDD968.1070205@gmail.com>
Message-ID: <1afaf6160910080725q888d72r5da38ad9baa0696e@mail.gmail.com>

2009/10/8 Nick Coghlan <ncoghlan at gmail.com>:
> As mentioned in that discussion, as of Python 2.6, you can do the following:
>>>> import platform
>>>> platform.python_implementation()
> 'CPython'
>
> (Although according to the function docstring, PyPy is currently missing
> from the list of known implementations)

That docstring should be updated then because I added support for PyPy
in 2.7 and 3.1.



-- 
Regards,
Benjamin

From tanzer at swing.co.at  Thu Oct  8 17:08:29 2009
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 08 Oct 2009 15:08:29 -0000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: Your message of "Thu, 08 Oct 2009 10:55:21 -0400"
	<4ACDFD59.4060200@trueblade.com>
References: <4ACDFD59.4060200@trueblade.com>
Message-ID: <E1MvubV-0004VK-R0@swing.co.at>

Eric Smith wrote at Thu, 08 Oct 2009 10:55:21 -0400:

> >>> BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
> >>> worth raising an enhancement request on the bug tracker using "O" to generate
> >>> compatible rendering for octals.
> >> I didn't get your message, could you resend?.
> >>
> >> I was thinking the same thing, but it seems like a transition step. I'd
> >> rather not keep such backward compatibility hacks (if you will) around
> >> for the long haul.  How about a flag (maybe '*') at the start of the
> >> format specification which says "operate in backward compatibility
> >> mode"? We could document it as being only useful for the % to {}
> >> translator, and promise to remove it at some point in the future. Either
> >> actually deprecate it or just promise to deprecate it in the future.
> >
> > That doesn't seem very useful to me. IIUC, the point of the translator
> > is to allow porting of the millions of existing %-formating operations
> > to the new-style .format.
> >
> > If the result of that is deprecated or removed a few years from now,
> > all maintainers of long existing code have exactly the same problem.
>
> I was thinking of it as a transition step until all application code
> switched to {} formatting. In which case the application has to deal
> with it.

You lost me here.

All that talk of deprecating %-formatting makes me really nervous.
%-formatting is pervasive in all existing Python code.

Without an automatic translator that is 100% accurate, porting all
that code to {}-formatting is not possible. Heck, it's not even
possible to grep for all instances of %-formatting.

How do you suppose that maintainers could ever do the transition from
%- to {}-formatting manually?

> > IMHO, either the translation is done once and gives identical output or
> > it isn't worth doing at all.
>
> I disagree. I doubt even 0.001% of all format strings involve octal
> formatting. Is it really worth not providing a transition path if it
> can't cover this case?

If %-formatting is first deprecated then removed from Python and there
is no automatic transition path that effectively means that existing
code using %-formatting is forced to stay at whatever Python version
was the last one supporting %-formatting.

I surely hope nobody is seriously considering such a scenario. Perl 6
seems harmless in comparison.

--
Christian Tanzer                                    http://www.c-tanzer.at/

From tanzer at swing.co.at  Thu Oct  8 16:51:00 2009
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 08 Oct 2009 14:51:00 -0000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: Your message of "Thu, 08 Oct 2009 10:24:33 -0400"
	<4ACDF621.4070603@trueblade.com>
References: <4ACDF621.4070603@trueblade.com>
Message-ID: <E1MvuKa-0004Sm-36@swing.co.at>

Eric Smith wrote at Thu, 08 Oct 2009 10:24:33 -0400:

> Vinay Sajip wrote:
> > BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
> > worth raising an enhancement request on the bug tracker using "O" to generate
> > compatible rendering for octals.
>
> I didn't get your message, could you resend?.
>
> I was thinking the same thing, but it seems like a transition step. I'd
> rather not keep such backward compatibility hacks (if you will) around
> for the long haul.  How about a flag (maybe '*') at the start of the
> format specification which says "operate in backward compatibility
> mode"? We could document it as being only useful for the % to {}
> translator, and promise to remove it at some point in the future. Either
> actually deprecate it or just promise to deprecate it in the future.

That doesn't seem very useful to me. IIUC, the point of the translator
is to allow porting of the millions of existing %-formating operations
to the new-style .format.

If the result of that is deprecated or removed a few years from now,
all maintainers of long existing code have exactly the same problem.

IMHO, either the translation is done once and gives identical output or
it isn't worth doing at all.

--
Christian Tanzer                                    http://www.c-tanzer.at/

From eric at trueblade.com  Thu Oct  8 17:27:14 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 08 Oct 2009 11:27:14 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>
References: <4ACDF621.4070603@trueblade.com> <E1MvuKa-0004Sm-36@swing.co.at>	
	<4ACDFD59.4060200@trueblade.com>
	<1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>
Message-ID: <4ACE04D2.1010100@trueblade.com>

Benjamin Peterson wrote:
> 2009/10/8 Eric Smith <eric at trueblade.com>:
>> Christian Tanzer wrote:
>>> IMHO, either the translation is done once and gives identical output or
>>> it isn't worth doing at all.
>> I disagree. I doubt even 0.001% of all format strings involve octal
>> formatting. Is it really worth not providing a transition path if it can't
>> cover this case?
> 
> It's also really easy to just write 0{:o} like my translator does.
> 
> 

I apologize for not having looked at anyone's implementation yet. But I 
suspect you'll have a problem with this case unless int.__format__ has 
special logic for backward compatible octal formatting:

 >>> '%# 5o' % 8
'  010'

Eric.


From solipsis at pitrou.net  Thu Oct  8 17:38:48 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 8 Oct 2009 15:38:48 +0000 (UTC)
Subject: [Python-Dev] locals() different behaviour when tracing?
References: <hakqag$vt6$1@ger.gmane.org>
Message-ID: <loom.20091008T173728-46@post.gmane.org>

Anders Waldenborg <anders <at> 0x63.nu> writes:
> 
> Is it intentional that locals() returns a copy/snapshot of the local 
> variables sometimes, and sometimes a reference? Like when enabling 
> tracing, such as in the code pasted below.

Since someone else opened a bug, I answered there. Anyone, feel free to correct
me if my answer is wrong.
http://bugs.python.org/issue7083

Regards

Antoine.



From python at mrabarnett.plus.com  Thu Oct  8 17:52:20 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 08 Oct 2009 16:52:20 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACDF621.4070603@trueblade.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>	<loom.20091007T191603-331@post.gmane.org>	<4ACDD2C3.4090608@gmail.com>	<loom.20091008T160529-262@post.gmane.org>
	<4ACDF621.4070603@trueblade.com>
Message-ID: <4ACE0AB4.8030707@mrabarnett.plus.com>

Eric Smith wrote:
> Vinay Sajip wrote:
>> BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see 
>> if it was
>> worth raising an enhancement request on the bug tracker using "O" to 
>> generate
>> compatible rendering for octals.
> 
> I didn't get your message, could you resend?.
> 
> I was thinking the same thing, but it seems like a transition step. I'd 
> rather not keep such backward compatibility hacks (if you will) around 
> for the long haul.  How about a flag (maybe '*') at the start of the 
> format specification which says "operate in backward compatibility 
> mode"? We could document it as being only useful for the % to {} 
> translator, and promise to remove it at some point in the future. Either 
> actually deprecate it or just promise to deprecate it in the future.
> 
I think the flag should be '%'.

From fuzzyman at voidspace.org.uk  Thu Oct  8 17:56:35 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 08 Oct 2009 16:56:35 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <E1MvubV-0004VK-R0@swing.co.at>
References: <4ACDFD59.4060200@trueblade.com> <E1MvubV-0004VK-R0@swing.co.at>
Message-ID: <4ACE0BB3.90206@voidspace.org.uk>

Christian Tanzer wrote:
> [snip...]
>>> IMHO, either the translation is done once and gives identical output or
>>> it isn't worth doing at all.
>>>       
>> I disagree. I doubt even 0.001% of all format strings involve octal
>> formatting. Is it really worth not providing a transition path if it
>> can't cover this case?
>>     
>
> If %-formatting is first deprecated then removed from Python and there
> is no automatic transition path that effectively means that existing
> code using %-formatting is forced to stay at whatever Python version
> was the last one supporting %-formatting.
>
> I surely hope nobody is seriously considering such a scenario. Perl 6
> seems harmless in comparison.
>   
That is vastly overstating it. Making 'with' and 'as' keywords and 
removing string exceptions (which have already happened) will affect far 
more programs than a minor incompatibility in transitioning string 
formatting.

Michael


> --
> Christian Tanzer                                    http://www.c-tanzer.at/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From python at mrabarnett.plus.com  Thu Oct  8 17:57:35 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 08 Oct 2009 16:57:35 +0100
Subject: [Python-Dev] Logging, Unicode and sockets
In-Reply-To: <loom.20091008T075520-844@post.gmane.org>
References: <loom.20091008T075520-844@post.gmane.org>
Message-ID: <4ACE0BEF.6070609@mrabarnett.plus.com>

Vinay Sajip wrote:
> Thanks to 
> 
> http://bugs.python.org/issue7077
> 
> I've noticed that the socket-based logging handlers - SocketHandler,
> DatagramHandler and SysLogHandler - aren't Unicode-aware and can break in the
> presence of Unicode messages. I'd like to fix this by giving these handlers an
> optional (encoding=None) parameter in their __init__, and then using this to
> encode on output. If no encoding is specified, is it best to use
> locale.getpreferredencoding(), sys.getdefaultencoding(),
> sys.getfilesystemencoding(), 'utf-8' or something else? On my system:
> 
>>>> sys.getdefaultencoding()
> 'ascii'
>>>> sys.getfilesystemencoding()
> 'mbcs'
>>>> locale.getpreferredencoding()
> 'cp1252'
> 
> which suggests to me that the locale.getpreferredencoding() should be the
> default. However, as I'm not a Unicode maven, any suggestions would be welcome.
> 
Well, encodings can vary from machine to machine, and if the encoding
doesn't cover all the Unicode codepoints then you could get an encoding
exception. For these reasons I'd vote for UTF-8.

From doug.hellmann at gmail.com  Thu Oct  8 18:00:35 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 8 Oct 2009 12:00:35 -0400
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <137556.63116.qm@web25801.mail.ukl.yahoo.com>
References: <loom.20091007T164835-226@post.gmane.org>
	<09B60FD3-D227-43E3-B901-5418C7FA950C@gmail.com>
	<137556.63116.qm@web25801.mail.ukl.yahoo.com>
Message-ID: <98209F12-9EA9-462C-8330-025B8254B6DA@gmail.com>


On Oct 8, 2009, at 10:47 AM, Vinay Sajip wrote:

>> I've had bad experiences in the past with dictionary-based APIs.   
>> They seem
>
>> "simpler" in the short run, because the user "only needs to create  
>> some
>> dictionaries".  Once the complexity of that nested dictionary grows  
>> to a certain
>> point, though, one has to refer back to documentation constantly to  
>> make sure
>
> Fair point, and agreed that the schema needs some care, here's  
> roughly what I'm thinking of as an example configuration (YAML):
>
> formatters:
>  brief:
>    format: '%(levelname)-8s: %(name)-15s: %(message)s'
>  precise:
>    format: '%(asctime)s %(name)-15s %(levelname)-8s %(message)s'
> filters:
>  allow_foo:
>    name: foo
> handlers:
>  console:
>    class : logging.StreamHandler
>    formatter: brief
>    level   : INFO
>    stream  : sys.stdout
>    filters: [allow_foo]
>  file:
>    class : logging.handlers.RotatingFileHandler
>    formatter: precise
>    filename: logconfig.log
>    maxBytes: 1024
>    backupCount: 3
>  debugfile:
>    class : logging.FileHandler
>    formatter: precise
>    filename: logconfig-detail.log
>    mode: a
> loggers:
>  foo:
>    level : ERROR
>    handlers: [debugfile]
>  spam:
>    level : CRITICAL
>    handlers: [debugfile]
>    propagate: no
>  bar.baz:
>    level: WARNING
>
> root:
>  level     : DEBUG
>  handlers  : [console, file]
>
> It's not too deeply nested, and I can't see any need for it being  
> more deeply nested. It more or less mirrors the way you'd have to  
> write the corresponding ConfigParser-compatible configuration, but  
> if you use a dict as a common format, you have the benefits which I  
> mentioned of supporting JSON, YAML or Python code in a Django  
> settings.py.

Yes, that makes sense.

>> the structure conforms to the "schema".  Building a simple config  
>> tree using
>> light-weight classes with documented APIs tends to be more  
>> sustainable in the
>> long run.
>
> When you say 'config tree using light-weight classes', I presume you  
> mean a parallel set of classes to the actual logging classes which  
> will be instantiated? ISTM logging classes are already reasonably  
> usable for programmatic configuration, but this doesn't address e.g.  
> updating configurations easily without program code changes, or the  
> ability to configure from say a YAML node, where YAML may be being  
> used for application configuration as a whole. (Unless of course  
> I've misunderstood what you're getting at.)

The probably are light-weight enough, but I didn't want to assume  
those classes would be used directly.  I suspect they could be, though.

I guess there aren't *so* many levels or keys that the data structure  
will become unwieldy.

Doug


From jnoller at gmail.com  Thu Oct  8 18:04:38 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 8 Oct 2009 12:04:38 -0400
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACDEEB9.2000308@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<4ACDEEB9.2000308@voidspace.org.uk>
Message-ID: <4222a8490910080904m62947983w6900b816623565a5@mail.gmail.com>

On Thu, Oct 8, 2009 at 9:52 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Simon Cross wrote:
>>
>> ?On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziad? <ziade.tarek at gmail.com>
>> wrote:
>>
>>>
>>> = Virtualenv and the multiple version support in Distribute =
>>>
>>
>> ...
>>
>>>
>>> My opinion is that this tool exists only because Python doesn't
>>> support the installation of multiple versions for the same
>>> distributions.
>>>
>>
>> This is not at all how I use virtualenv. For me virtualenv is a
>> sandbox so that I don't have to become root whenever I need to install
>> a Python package for testing purposes and to allow me to hop between
>> sets of installed Python packages while developing on multiple Python
>> projects. I also use it as a sandbox for build bots so that multiple
>> bots on the same machine can each build their own projects with just
>> the known dependencies installed.
>>
>>
>
> This is exactly why I use virtualenv as well. I don't recall ever having
> wanted / needed to install multiple versions of the same library - whilst I
> can appreciate that it *can* be a real issue it has never been a problem for
> me.
>
> Michael

+1 - virtualenv, AFAIK is used for sandboxing/isolation of various
environments, not dealing with multiple versions within the *same*
environment. Of course, it does solve the "being dependent on a
specific version" of a dependency because it *is* sandboxed from
everything else.

Adding multiple version support doesn't remove the need for sandboxing.

From tanzer at swing.co.at  Thu Oct  8 18:08:41 2009
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 08 Oct 2009 16:08:41 -0000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: Your message of "Thu, 08 Oct 2009 16:56:35 +0100"
	<4ACE0BB3.90206@voidspace.org.uk>
References: <4ACE0BB3.90206@voidspace.org.uk>
Message-ID: <E1MvvXl-0004Zv-D7@swing.co.at>

Michael Foord wrote at Thu, 08 Oct 2009 16:56:35 +0100:

> > If %-formatting is first deprecated then removed from Python and there
> > is no automatic transition path that effectively means that existing
> > code using %-formatting is forced to stay at whatever Python version
> > was the last one supporting %-formatting.
> >
> > I surely hope nobody is seriously considering such a scenario. Perl 6
> > seems harmless in comparison.
> >
> That is vastly overstating it. Making 'with' and 'as' keywords and
> removing string exceptions (which have already happened) will affect far
> more programs than a minor incompatibility in transitioning string
> formatting.

`with` and `as` are trivial to fix and certainly not pervasive in
existing code. String exceptions have been deprecated for years.

--
Christian Tanzer                                    http://www.c-tanzer.at/

From fuzzyman at voidspace.org.uk  Thu Oct  8 18:08:57 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 08 Oct 2009 17:08:57 +0100
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACDCA5A.4090609@cheimes.de>
References: <4ACDCA5A.4090609@cheimes.de>
Message-ID: <4ACE0E99.7050102@voidspace.org.uk>

Christian Heimes wrote:
> [snip...]
> The altered user site directories are:
>
> CPython:
>   ~/.local/lib/python2.6/site-packages
>   %APPDATA%/Python/Python26
>
> IronPython:
>   ~/.local/lib/ironpython2.6/site-packages
>   %APPDATA%/Python/IronPython26
>
> Jython:
>   ~/.local/lib/jython2.6/site-packages
>   %APPDATA%/Python/Jython26
>
>   
+1

I really like this scheme. The important thing for IronPython is that we 
can get it into Python 2.6 (along with other fixes to make distutils 
compatible with IronPython - like not attempting to bytecode-compile 
when sys.dont_write_bytecode is True).

All the best,

Michael Foord

-- 
http://www.ironpythoninaction.com/


From benjamin at python.org  Thu Oct  8 17:07:53 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 8 Oct 2009 10:07:53 -0500
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACDFD59.4060200@trueblade.com>
References: <4ACDF621.4070603@trueblade.com> <E1MvuKa-0004Sm-36@swing.co.at>
	<4ACDFD59.4060200@trueblade.com>
Message-ID: <1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>

2009/10/8 Eric Smith <eric at trueblade.com>:
> Christian Tanzer wrote:
>> IMHO, either the translation is done once and gives identical output or
>> it isn't worth doing at all.
>
> I disagree. I doubt even 0.001% of all format strings involve octal
> formatting. Is it really worth not providing a transition path if it can't
> cover this case?

It's also really easy to just write 0{:o} like my translator does.


-- 
Regards,
Benjamin

From a.badger at gmail.com  Thu Oct  8 18:11:49 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 8 Oct 2009 09:11:49 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
In-Reply-To: <4ACDCCBD.4040908@egenix.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACDB82F.7010006@cryptelium.net> <4ACDCCBD.4040908@egenix.com>
Message-ID: <20091008161148.GG28231@clingman.lan>

On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote:
> > Tarek Ziad? a ?crit :
> >> But if PEP 376 and PEP 386 support are added in Python, we're not far
> >> from being able to provide multiple version support with
> >> the help of importlib.
> 
> Before putting much work into this: do you really think that having
> multiple versions of the same package in the same Python installation
> is a good idea ?
> 
I think it is a good idea.

> Examples:
> What if you have an application that uses two modules which each
> require two different versions of the same package ? Would that
> load the same package twice, causing globals used by these package
> to no work (e.g. isinstance(x, class_name) or global caches) ?
> 
That's not how it should work.  Look at other systems that allow for
installing multiple versions of a library -- for instance, loading dynamic
shared objects in C
* You can install multiple versions of a library in parallel
* The dynamic loader will pick the version of the library that is
  appropriate from the list of available options (the program specifies the
  SONAME it needs -- library name plus API version.  The loader then
  chooses the most recent revision that matches that API version.)
* When one binary needs multiple API versions of the library, the
  application cannot start.

The last point addresses your concern -- depending on multiple, incompatible
versions of a library is prohibited.  The programmer of the application
needs to make the code run with a single version of the code.

> This sounds a lot like DLL- or RPM-hell to me.
> 
RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression
that it is the lack of enough version specification rather than too much but
I don't know for sure). is similar but it takes place on the end-user's
system.  This should take place on the programmer's system instead.
End-users are not in a position to fix things like RPM-hell.  Programmers
are.

Example RPM-hell:
Application Foo requires libbar-1.x
Application Baz requires libbar-2.x

The user may either have Foo or Baz installed on their system with the
appropriate libbar but not both.  They depend on the packagers and
developers of Foo and Bar to do one of the following to resolve the
situation:

* Port Foo and Baz to use the same version of libbar.
* Package libbar in such a way that libbar-1 and libbar-2 are parallel
  installable on the system.  Then they can install two separate packages,
  libbar1-1.0 and libbar2-2.0.

Example of similar Distutils multiple version problem:
The programmer creates an application Foo that depends on python-bar-1.x. He
has recently started work on a file that imports python-baz-1.0.  python-baz
depends on python-bar-2.x.  The first time he tries to run his new code,
python gives him an error message that it is impossible to satisfy the
version requirements for python-bar.  Depending on how the versions are
specified, the error message could be very specific and helpful:

  Impossible version requirements:
    bar Requires: python-baz>=2.0, < 3.0
    foo.py Requires: python-baz >=1.0, < 2.0

The programmer can then discard their new code, port foo.py to
python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to
the upstream of that module.  Note two things about this scenario:

1) The programmer is the person who is responsible for creating the conflict
and for resolving it.  They are the proper authority for making the decision
to port to python-baz-2.x or not using python-bar.  The end-user who is not
responsible is not impacted by this at all.
2) The programmer would have had to deal with this issue whether we allow
multiple versions to be installed or not.  With multiple version support we
may be able to get them better error messages (depending on how the
dependency information is formatted and how completely it was specified in
the app and modules).

> I think it's much better to keep things simple and under user
> control, e.g. by encouraging use of virtualenv-like setups
> and perhaps adding better native support for these to Python.
> 
> If the user finds that there's a version conflict this can
> then be resolved during environment setup instead of hoping
> for the best and waiting for application failures at run-time.
> 
For the class of user that is actually a developer, it might be somewhat
true that version conflicts should be resolved by them.  But for the class
of user that is an end-user, version conflicts are a totally foreign
concept.  They should be dealt with by the person who is coding the
application for them.

Also note, the ability to have multiple versions makes things easier for
system packagers and provides an easy alternative to a virtualenv for
end-users.

* System packagers: virtualenv does not provide a method suitable for system
  packagers.  The nearest adaptation would be for the system packager to
  install python packages into their own hierarchy not in the PYTHONPATH.
  Then they would need to create a virtualenv-like directory that symlinks
  to the packages in that directory.  Then they would need to write a
  wrapper script for the application that put that virtualenv-like directory
  into the PYTHONPATH before any other site package directories and have
  that wrapper call the real binary.  This is needless complication for the
  typical virtualenv install so the work is not likely to be done there and
  it's incredibly hacky for the system packager so the work is not likely to
  be done there.
* End users: virtualenv creates a whole environment for the application to
  live in.  If python understood how to use multiple versions then we'd only
  need to install the versions of packages that didn't match up with what's
  already on the system into the user's personal site-package directory.
  Then the module loader would take care of loading the module from the
  personal site-packages since it met the version requirements instead of
  the system copy.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/8567bf94/attachment.pgp>

From a.badger at gmail.com  Thu Oct  8 18:17:32 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 8 Oct 2009 09:17:32 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
In-Reply-To: <fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
Message-ID: <20091008161732.GH28231@clingman.lan>

On Thu, Oct 08, 2009 at 02:52:24PM +0200, Simon Cross wrote:
>  On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> > = Virtualenv and the multiple version support in Distribute =
> ...
> > My opinion is that this tool exists only because Python doesn't
> > support the installation of multiple versions for the same
> > distributions.
> 
Let's actually look at these reasons:

> This is not at all how I use virtualenv. For me virtualenv is a
> sandbox so that I don't have to become root whenever I need to install
> a Python package for testing purposes

This is needing to install multiple versions and use the newly installed
version for testing.

> and to allow me to hop between
> sets of installed Python packages while developing on multiple Python
> projects.

This is the ability to install multiple versions and specify different
versions for different projects you're working on.

> I also use it as a sandbox for build bots so that multiple
> bots on the same machine can each build their own projects with just
> the known dependencies installed.
> 
This is the only use in the list that is virtualenv specific.  The rest are
cases of needing to install multiple versions on the system.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/f5a4f27f/attachment.pgp>

From doug.hellmann at gmail.com  Thu Oct  8 14:21:15 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 8 Oct 2009 08:21:15 -0400
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <loom.20091007T164835-226@post.gmane.org>
References: <loom.20091007T164835-226@post.gmane.org>
Message-ID: <09B60FD3-D227-43E3-B901-5418C7FA950C@gmail.com>


On Oct 7, 2009, at 10:49 AM, Vinay Sajip wrote:

> All three of the contenders for the title of "commonly found  
> configuration
> mechanism" - JSON, YAML and Python code - will be expressible, in  
> Python, as
> Python dicts. So it seems to make sense to add, to logging.config, a  
> new
> callable bound to "dictConfig" which will take a single dictionary  
> argument and
> configure logging from that dictionary.

I've had bad experiences in the past with dictionary-based APIs.  They  
seem "simpler" in the short run, because the user "only needs to  
create some dictionaries".  Once the complexity of that nested  
dictionary grows to a certain point, though, one has to refer back to  
documentation constantly to make sure the structure conforms to the  
"schema".  Building a simple config tree using light-weight classes  
with documented APIs tends to be more sustainable in the long run.

Doug


From benjamin at python.org  Thu Oct  8 16:21:56 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 8 Oct 2009 09:21:56 -0500
Subject: [Python-Dev] locals() different behaviour when tracing?
In-Reply-To: <hakqag$vt6$1@ger.gmane.org>
References: <hakqag$vt6$1@ger.gmane.org>
Message-ID: <1afaf6160910080721u459320c3va63baffe2b7137d0@mail.gmail.com>

2009/10/8 Anders Waldenborg <anders at 0x63.nu>:
>
> Is it intentional that locals() returns a copy/snapshot of the local
> variables sometimes, and sometimes a reference? Like when enabling tracing,
> such as in the code pasted below. The documentation ("Update and return a
> dictionary containing the current scope's local variables.") is pretty
> unclear.

Yes, it does, and that's why the documentation says changing it is undefined. :)


-- 
Regards,
Benjamin

From solipsis at pitrou.net  Thu Oct  8 18:28:52 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 8 Oct 2009 16:28:52 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Distutils_and_Distribute_roadmap_=28and_so?=
	=?utf-8?q?me=09words=09on_Virtualenv=2C_Pip=29?=
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
Message-ID: <loom.20091008T182710-265@post.gmane.org>

Toshio Kuratomi <a.badger <at> gmail.com> writes:
> 
> This is needing to install multiple versions and use the newly installed
> version for testing.
[...]

What you're missing is that having separate environments has a virtue of
cleanliness, understandability and robustness that a multiple-versioned solution
doesn't have. While the technical merits are debatable I'm sure some people
definitely prefer to manage a virtualenv-based version.

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Thu Oct  8 18:30:00 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 08 Oct 2009 17:30:00 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some	words
 on Virtualenv, Pip)
In-Reply-To: <20091008161732.GH28231@clingman.lan>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
Message-ID: <4ACE1388.5020602@voidspace.org.uk>

Toshio Kuratomi wrote:
> On Thu, Oct 08, 2009 at 02:52:24PM +0200, Simon Cross wrote:
>   
>>  On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>     
>>> = Virtualenv and the multiple version support in Distribute =
>>>       
>> ...
>>     
>>> My opinion is that this tool exists only because Python doesn't
>>> support the installation of multiple versions for the same
>>> distributions.
>>>       
> Let's actually look at these reasons:
>
>   
>> This is not at all how I use virtualenv. For me virtualenv is a
>> sandbox so that I don't have to become root whenever I need to install
>> a Python package for testing purposes
>>     
>
> This is needing to install multiple versions and use the newly installed
> version for testing.
>
>   

Not really - it is wanting to install a single version of a library that 
you don't want to install into your 'main' (whether that be user or 
system) Python install. It is sandboxing and orthogonal to multiple 
versions.

>> and to allow me to hop between
>> sets of installed Python packages while developing on multiple Python
>> projects.
>>     
>
> This is the ability to install multiple versions and specify different
> versions for different projects you're working on.
>   

No, it is working on multiple projects that have *different* 
dependencies and being able to work in an environment that *only* has 
direct dependencies installed - again sandboxed from your main Python 
environment.

The fact that virtualenv allows you to have multiple different versions 
of the same library installed in the different environments you create 
is completely separate from the motivation that causes many people to 
use it.

What virtualenv *doesn't* do (I believe) is allow you to have multiple 
versions of libraries installed within a single environment and switch 
between them (at least it doesn't offer anything beyond what setuptools 
or pip provides).

Michael
>   
>> I also use it as a sandbox for build bots so that multiple
>> bots on the same machine can each build their own projects with just
>> the known dependencies installed.
>>
>>     
> This is the only use in the list that is virtualenv specific.  The rest are
> cases of needing to install multiple versions on the system.
>
> -Toshio
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From masklinn at masklinn.net  Thu Oct  8 18:35:45 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 8 Oct 2009 18:35:45 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
In-Reply-To: <20091008161732.GH28231@clingman.lan>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
Message-ID: <7DD3FAEA-88FE-48EE-9B92-972F5C8EFB3E@masklinn.net>

On 8 Oct 2009, at 18:17 , Toshio Kuratomi wrote:
>
>> This is not at all how I use virtualenv. For me virtualenv is a
>> sandbox so that I don't have to become root whenever I need to  
>> install
>> a Python package for testing purposes
>
> This is needing to install multiple versions and use the newly  
> installed
> version for testing.
>
No it's not. It's keeping the python package *being tested* out of the  
system's or user's site-package because it's potentially unstable or  
unneeded. It provides a trivial way of *removing* the package to get  
rid of it: delete the virtualenv. No trace anywhere that the package  
was ever installed, no impact on the system  (apart from the potential  
side-effects of executing the system).

The issue here isn't "multiple installed packages", it will more than  
likely be the only version of itself: note that it's a package being  
tested, not an *upgrade* being tested.

The issues solved are:
* not having to become root (solved by PEP 370 if it ever lands)
* minimizing as much as possible the impact of testing the package on  
the system (not solved by any other solution)

>> and to allow me to hop between
>> sets of installed Python packages while developing on multiple Python
>> projects.
>
> This is the ability to install multiple versions and specify different
> versions for different projects you're working on.
>
No, this is the ability to not needlessly clutter site-packages, keep  
them clean, tight, focused; and the ability to keep a project's  
environment (such as its dependencies) clear and repeatable. Nowhere  
was it indicated that multiple versions were involved.

Both of those *might* imply issues of multiple versions concurrently  
installed on the system, and virtualenv would incidentally solve these  
issues, but these issues are *not* the core of either use case.  
They're at best peripheral and potential.

From masklinn at masklinn.net  Thu Oct  8 18:37:40 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 8 Oct 2009 18:37:40 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
In-Reply-To: <20091008161732.GH28231@clingman.lan>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
Message-ID: <7FC2E2D2-16AF-490B-9E9B-FB465C02D357@masklinn.net>

On 8 Oct 2009, at 18:17 , Toshio Kuratomi wrote:
>> This is not at all how I use virtualenv. For me virtualenv is a
>> sandbox so that I don't have to become root whenever I need to  
>> install
>> a Python package for testing purposes
>
> This is needing to install multiple versions and use the newly  
> installed
> version for testing.
>
No it's not. It's keeping the python package *being tested* out of the  
system's or user's site-package because it's potentially unstable or  
unneeded. It provides a trivial way of *removing* the package to get  
rid of it: delete the virtualenv. No trace anywhere that the package  
was ever installed, no impact on the system  (apart from the potential  
side-effects of executing the system).

The issue here isn't "multiple installed packages", it will more than  
likely be the only version of itself: note that it's a package being  
tested, not an *upgrade* being tested.

The issues solved are:
* not having to become root (solved by PEP 370 if it ever lands)
* minimizing as much as possible the impact of testing the package on  
the system (not solved by any other solution)

>> and to allow me to hop between
>> sets of installed Python packages while developing on multiple Python
>> projects.
>
> This is the ability to install multiple versions and specify different
> versions for different projects you're working on.
>
No, this is the ability to not needlessly clutter site-packages, keep  
them clean, tight, focused; and the ability to keep a project's  
environment (such as its dependencies) clear and repeatable. Nowhere  
was it indicated that multiple versions were involved.

Both of those *might* imply issues of multiple versions concurrently  
installed on the system, and virtualenv would incidentally solve these  
issues, but these issues are *not* the core of either use case.  
They're at best peripheral and potential

From ziade.tarek at gmail.com  Thu Oct  8 18:44:08 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 8 Oct 2009 18:44:08 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACDFD54.3080202@egenix.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACDB82F.7010006@cryptelium.net> <4ACDCCBD.4040908@egenix.com>
	<94bdd2610910080554m530bfdbeg2bf010c813c71873@mail.gmail.com>
	<4ACDFD54.3080202@egenix.com>
Message-ID: <94bdd2610910080944q1785aad0nc15937610701190f@mail.gmail.com>

On Thu, Oct 8, 2009 at 4:55 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> OTOH, the register command sends the meta-data directly to
> the PyPI database, so it doesn't even need another file
> for storing away the meta data.
>
> Unless I'm missing something important (which I probably am),
> the only added value of the new format over PKG-INFO is the
> micro-language.

That's one. The other one is to avoid to deal with yet another
configuration file
alongside setup.py, and just deal with a [metadata]?section in setup.cfg.

And PKG-INFO stays a static, platform-specific file that exists only when the
distribution is installed on the target platform.


>> No, but if you ask for it I can write it.
>
> Please do. The format will need documentation anyway and the
> motivation should be made clear as well.

Ok I will

Tarek

From p.f.moore at gmail.com  Thu Oct  8 00:30:26 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 7 Oct 2009 23:30:26 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <loom.20091008T002130-593@post.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<ha01t7$e2j$1@ger.gmane.org>
	<79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
	<4ACC7B03.3040700@avl.com> <loom.20091008T002130-593@post.gmane.org>
Message-ID: <79990c6b0910071530x2bd08dc7vc90c66d6f48c3a01@mail.gmail.com>

2009/10/7 Antoine Pitrou <solipsis at pitrou.net>:
> Python 3 complains at startup and is a bit more explicit:
>
> $ ./python -c '1' >&-
> Fatal Python error: Py_Initialize: can't initialize sys standard streams
> OSError: [Errno 9] Bad file descriptor
> Abandon
>
> Of course, if it is stderr that you explicitly close, you lose all reporting:
>
> $ ./python -c '1' 2>&-
> Abandon

Hmm, how does Python 3's pythonw work on Windows? (I don't have a
Windows installation of Python 3 to hand at the moment)

Paul

From vinay_sajip at yahoo.co.uk  Thu Oct  8 18:51:53 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 16:51:53 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <4ACDF621.4070603@trueblade.com> <E1MvuKa-0004Sm-36@swing.co.at>
	<4ACDFD59.4060200@trueblade.com>
	<1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>
Message-ID: <loom.20091008T185131-288@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:

> It's also really easy to just write 0{:o} like my translator does.

How does that cope when handed a negative number to format?

>>> "%#0o" % -1234
'-02322'
>>> "0{0:o}".format(-1234)
'0-2322'

Regards,

Vinay Sajip




From guido at python.org  Thu Oct  8 18:55:11 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 8 Oct 2009 09:55:11 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <E1MvubV-0004VK-R0@swing.co.at>
References: <4ACDFD59.4060200@trueblade.com> <E1MvubV-0004VK-R0@swing.co.at>
Message-ID: <ca471dc20910080955o1c55fc53gf26402e9ec30232b@mail.gmail.com>

On Thu, Oct 8, 2009 at 8:08 AM, Christian Tanzer <tanzer at swing.co.at> wrote:
> All that talk of deprecating %-formatting makes me really nervous.
> %-formatting is pervasive in all existing Python code.
>
> Without an automatic translator that is 100% accurate, porting all
> that code to {}-formatting is not possible. Heck, it's not even
> possible to grep for all instances of %-formatting.
>
> How do you suppose that maintainers could ever do the transition from
> %- to {}-formatting manually?

This is pretty much the situation with integer division (you can only
recognize it by running the code), and yet we figured a way to change
that in 3.x. Or take classic classes vs. new-style classes. They
cannot be translated 100% automatically either. The solution is to
support the old and new style in parallel for a really long time -- we
did this with int division (read PEP 238), we did it with classes, and
we can do it again with formatting.

Unless I missed something, we're not planning to remove %-formatting
until Python 4.0 comes along, which we won't even start until a long
time after everyone has switched to some version of 3.x. So the same
approach will apply: support both forms, nudge people to start using
the new form, wait, nudge some more, etc.

So, yes, we will continue to make noise about this. And yes you should
opportunistically migrate your code to {}-formatting, like when you're
rewriting some code anyway. One of the nice things about {}-formatting
is that in most cases (things like the logging API excluded) you can
change it one format string at a time.

And no, the sky isn't falling.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From kiorky at cryptelium.net  Thu Oct  8 18:56:19 2009
From: kiorky at cryptelium.net (kiorky)
Date: Thu, 08 Oct 2009 18:56:19 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some	words
 on Virtualenv, Pip)
In-Reply-To: <20091008161148.GG28231@clingman.lan>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<4ACDB82F.7010006@cryptelium.net>
	<4ACDCCBD.4040908@egenix.com> <20091008161148.GG28231@clingman.lan>
Message-ID: <4ACE19B3.4060106@cryptelium.net>



Toshio Kuratomi a ?crit :
> On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote:
>>> Tarek Ziad? a ?crit :
>>>> But if PEP 376 and PEP 386 support are added in Python, we're not far
>>>> from being able to provide multiple version support with
>>>> the help of importlib.
>> Before putting much work into this: do you really think that having
>> multiple versions of the same package in the same Python installation
>> is a good idea ?
>>
> I think it is a good idea.
> 
>> Examples:
>> What if you have an application that uses two modules which each
>> require two different versions of the same package ? Would that
>> load the same package twice, causing globals used by these package
>> to no work (e.g. isinstance(x, class_name) or global caches) ?
>>
> That's not how it should work.  Look at other systems that allow for
> installing multiple versions of a library -- for instance, loading dynamic
> shared objects in C
> * You can install multiple versions of a library in parallel
> * The dynamic loader will pick the version of the library that is
>   appropriate from the list of available options (the program specifies the
>   SONAME it needs -- library name plus API version.  The loader then
>   chooses the most recent revision that matches that API version.)
> * When one binary needs multiple API versions of the library, the
>   application cannot start.
> 
> The last point addresses your concern -- depending on multiple, incompatible
> versions of a library is prohibited.  The programmer of the application
> needs to make the code run with a single version of the code.
> 
>> This sounds a lot like DLL- or RPM-hell to me.
>>
> RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression
> that it is the lack of enough version specification rather than too much but
> I don't know for sure). is similar but it takes place on the end-user's
> system.  This should take place on the programmer's system instead.
> End-users are not in a position to fix things like RPM-hell.  Programmers
> are.
> 
> Example RPM-hell:
> Application Foo requires libbar-1.x
> Application Baz requires libbar-2.x
> 
> The user may either have Foo or Baz installed on their system with the
> appropriate libbar but not both.  They depend on the packagers and
> developers of Foo and Bar to do one of the following to resolve the
> situation:
> 
> * Port Foo and Baz to use the same version of libbar.
> * Package libbar in such a way that libbar-1 and libbar-2 are parallel
>   installable on the system.  Then they can install two separate packages,
>   libbar1-1.0 and libbar2-2.0.
> 
> Example of similar Distutils multiple version problem:
> The programmer creates an application Foo that depends on python-bar-1.x. He
> has recently started work on a file that imports python-baz-1.0.  python-baz
> depends on python-bar-2.x.  The first time he tries to run his new code,
> python gives him an error message that it is impossible to satisfy the
> version requirements for python-bar.  Depending on how the versions are
> specified, the error message could be very specific and helpful:
> 
>   Impossible version requirements:
>     bar Requires: python-baz>=2.0, < 3.0
>     foo.py Requires: python-baz >=1.0, < 2.0
> 
> The programmer can then discard their new code, port foo.py to
> python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to
> the upstream of that module.  Note two things about this scenario:
> 
> 1) The programmer is the person who is responsible for creating the conflict
> and for resolving it.  They are the proper authority for making the decision
> to port to python-baz-2.x or not using python-bar.  The end-user who is not
> responsible is not impacted by this at all.
> 2) The programmer would have had to deal with this issue whether we allow
> multiple versions to be installed or not.  With multiple version support we
> may be able to get them better error messages (depending on how the
> dependency information is formatted and how completely it was specified in
> the app and modules).
> 
>> I think it's much better to keep things simple and under user
>> control, e.g. by encouraging use of virtualenv-like setups
>> and perhaps adding better native support for these to Python.
>>
>> If the user finds that there's a version conflict this can
>> then be resolved during environment setup instead of hoping
>> for the best and waiting for application failures at run-time.
>>
> For the class of user that is actually a developer, it might be somewhat
> true that version conflicts should be resolved by them.  But for the class
> of user that is an end-user, version conflicts are a totally foreign
> concept.  They should be dealt with by the person who is coding the
> application for them.
> 
> Also note, the ability to have multiple versions makes things easier for
> system packagers and provides an easy alternative to a virtualenv for
> end-users.
> 
> * System packagers: virtualenv does not provide a method suitable for system

Yes, there is no doubt virtualenv is useless for system packagers but:

* System and applications deployment have not to be tied.
It s up to the user to install things system wide or to use locally isolation
technics. Virtualenv is one of those.
As a conclusion, there are not very much problem for system packagers as if
their users have specific needs, they will do something Outside the system.
If not, they use their global python with packages installed in that global one.

>   packagers.  The nearest adaptation would be for the system packager to
>   install python packages into their own hierarchy not in the PYTHONPATH.
>   Then they would need to create a virtualenv-like directory that symlinks
>   to the packages in that directory.  Then they would need to write a
>   wrapper script for the application that put that virtualenv-like directory
>   into the PYTHONPATH before any other site package directories and have
>   that wrapper call the real binary.  This is needless complication for the
>   typical virtualenv install so the work is not likely to be done there and
>   it's incredibly hacky for the system packager so the work is not likely to
>   be done there.
> * End users: virtualenv creates a whole environment for the application to
>   live in.  If python understood how to use multiple versions then we'd only
>   need to install the versions of packages that didn't match up with what's
>   already on the system into the user's personal site-package directory.
>   Then the module loader would take care of loading the module from the
>   personal site-packages since it met the version requirements instead of
>   the system copy.
> 

I dont want to have interaction between my different hosted projects, isolation
is the only thing i want. As many other people there already.
I do not want at all to have to deal with multiple version picking system and
debug an application at 2:00am because of that.

> -Toshio
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/690c1359/attachment.pgp>

From kiorky at cryptelium.net  Thu Oct  8 19:05:09 2009
From: kiorky at cryptelium.net (kiorky)
Date: Thu, 08 Oct 2009 19:05:09 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some	words
 on Virtualenv, Pip)
In-Reply-To: <20091008161148.GG28231@clingman.lan>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<4ACDB82F.7010006@cryptelium.net>
	<4ACDCCBD.4040908@egenix.com> <20091008161148.GG28231@clingman.lan>
Message-ID: <4ACE1BC5.2070706@cryptelium.net>



Toshio Kuratomi a ?crit :
> On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote:
>>> Tarek Ziad? a ?crit :
>>>> But if PEP 376 and PEP 386 support are added in Python, we're not far
>>>> from being able to provide multiple version support with
>>>> the help of importlib.
>> Before putting much work into this: do you really think that having
>> multiple versions of the same package in the same Python installation
>> is a good idea ?
>>
> I think it is a good idea.
> 
>> Examples:
>> What if you have an application that uses two modules which each
>> require two different versions of the same package ? Would that
>> load the same package twice, causing globals used by these package
>> to no work (e.g. isinstance(x, class_name) or global caches) ?
>>
> That's not how it should work.  Look at other systems that allow for
> installing multiple versions of a library -- for instance, loading dynamic
> shared objects in C
> * You can install multiple versions of a library in parallel
> * The dynamic loader will pick the version of the library that is
>   appropriate from the list of available options (the program specifies the
>   SONAME it needs -- library name plus API version.  The loader then
>   chooses the most recent revision that matches that API version.)
> * When one binary needs multiple API versions of the library, the
>   application cannot start.
> 
> The last point addresses your concern -- depending on multiple, incompatible
> versions of a library is prohibited.  The programmer of the application
> needs to make the code run with a single version of the code.
> 
>> This sounds a lot like DLL- or RPM-hell to me.
>>
> RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression
> that it is the lack of enough version specification rather than too much but
> I don't know for sure). is similar but it takes place on the end-user's
> system.  This should take place on the programmer's system instead.
> End-users are not in a position to fix things like RPM-hell.  Programmers
> are.
> 
> Example RPM-hell:
> Application Foo requires libbar-1.x
> Application Baz requires libbar-2.x
> 
> The user may either have Foo or Baz installed on their system with the
> appropriate libbar but not both.  They depend on the packagers and
> developers of Foo and Bar to do one of the following to resolve the
> situation:
> 
> * Port Foo and Baz to use the same version of libbar.
> * Package libbar in such a way that libbar-1 and libbar-2 are parallel
>   installable on the system.  Then they can install two separate packages,
>   libbar1-1.0 and libbar2-2.0.
> 
> Example of similar Distutils multiple version problem:
> The programmer creates an application Foo that depends on python-bar-1.x. He
> has recently started work on a file that imports python-baz-1.0.  python-baz
> depends on python-bar-2.x.  The first time he tries to run his new code,
> python gives him an error message that it is impossible to satisfy the
> version requirements for python-bar.  Depending on how the versions are
> specified, the error message could be very specific and helpful:
> 
>   Impossible version requirements:
>     bar Requires: python-baz>=2.0, < 3.0
>     foo.py Requires: python-baz >=1.0, < 2.0
> 
> The programmer can then discard their new code, port foo.py to
> python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to
> the upstream of that module.  Note two things about this scenario:
> 
> 1) The programmer is the person who is responsible for creating the conflict
> and for resolving it.  They are the proper authority for making the decision
> to port to python-baz-2.x or not using python-bar.  The end-user who is not
> responsible is not impacted by this at all.
> 2) The programmer would have had to deal with this issue whether we allow
> multiple versions to be installed or not.  With multiple version support we
> may be able to get them better error messages (depending on how the
> dependency information is formatted and how completely it was specified in
> the app and modules).
> 
>> I think it's much better to keep things simple and under user
>> control, e.g. by encouraging use of virtualenv-like setups
>> and perhaps adding better native support for these to Python.
>>
>> If the user finds that there's a version conflict this can
>> then be resolved during environment setup instead of hoping
>> for the best and waiting for application failures at run-time.
>>
> For the class of user that is actually a developer, it might be somewhat
> true that version conflicts should be resolved by them.  But for the class
> of user that is an end-user, version conflicts are a totally foreign
> concept.  They should be dealt with by the person who is coding the
> application for them.
> 
> Also note, the ability to have multiple versions makes things easier for
> system packagers and provides an easy alternative to a virtualenv for
> end-users.
> 
> * System packagers: virtualenv does not provide a method suitable for system

Yes, there is no doubt virtualenv is useless for system packagers but:

* System and applications deployment have not to be tied.
It s up to the user to install things system wide or to use locally isolation
technics. Virtualenv is one of those.
As a conclusion, there are not very much problem for system packagers as if
their users have specific needs, they will do something Outside the system.
If not, they use their global python with packages installed in that global one.

>   packagers.  The nearest adaptation would be for the system packager to
>   install python packages into their own hierarchy not in the PYTHONPATH.
>   Then they would need to create a virtualenv-like directory that symlinks
>   to the packages in that directory.  Then they would need to write a
>   wrapper script for the application that put that virtualenv-like directory
>   into the PYTHONPATH before any other site package directories and have
>   that wrapper call the real binary.  This is needless complication for the
>   typical virtualenv install so the work is not likely to be done there and
>   it's incredibly hacky for the system packager so the work is not likely to
>   be done there.
> * End users: virtualenv creates a whole environment for the application to
>   live in.  If python understood how to use multiple versions then we'd only
>   need to install the versions of packages that didn't match up with what's
>   already on the system into the user's personal site-package directory.
>   Then the module loader would take care of loading the module from the
>   personal site-packages since it met the version requirements instead of
>   the system copy.
> 

I dont want to have interaction between my different hosted projects, isolation
is the only thing i want. As many other people there already.
I do not want at all to have to deal with multiple version picking system and
debug an application at 2:00am because of that.

> -Toshio
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/78426e48/attachment-0002.pgp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/78426e48/attachment-0003.pgp>

From python at rcn.com  Thu Oct  8 19:14:19 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 8 Oct 2009 10:14:19 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
References: <4ACDFD59.4060200@trueblade.com> <E1MvubV-0004VK-R0@swing.co.at>
	<ca471dc20910080955o1c55fc53gf26402e9ec30232b@mail.gmail.com>
Message-ID: <4B9C0675D45347A887DBC074153641EA@RaymondLaptop1>


[Christian Tanzer]
>> How do you suppose that maintainers could ever do the transition from
>> %- to {}-formatting manually?

[Guido van Rossum]
> This is pretty much the situation with integer division (you can only
> recognize it by running the code),

Do you think there may be some possible parallel to the -3 option
to flag cases of %-formatting?  If so, that could be helpful.



> So, yes, we will continue to make noise about this. And yes you should
> opportunistically migrate your code to {}-formatting, like when you're
> rewriting some code anyway. One of the nice things about {}-formatting
> is that in most cases (things like the logging API excluded) you can
> change it one format string at a time.

I've already have some code that mixes the styles (using {} for new stuff).


Raymond

From amauryfa at gmail.com  Thu Oct  8 19:19:59 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 8 Oct 2009 19:19:59 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <79990c6b0910071530x2bd08dc7vc90c66d6f48c3a01@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
	<ha01t7$e2j$1@ger.gmane.org>
	<79990c6b0910010158k2edf20ceh9d6b4f5c47a37995@mail.gmail.com>
	<4ACC7B03.3040700@avl.com> <loom.20091008T002130-593@post.gmane.org>
	<79990c6b0910071530x2bd08dc7vc90c66d6f48c3a01@mail.gmail.com>
Message-ID: <e27efe130910081019l5c8efb75k88ec0c873b57f1ac@mail.gmail.com>

2009/10/8 Paul Moore <p.f.moore at gmail.com>:
> 2009/10/7 Antoine Pitrou <solipsis at pitrou.net>:
>> Python 3 complains at startup and is a bit more explicit:
>>
>> $ ./python -c '1' >&-
>> Fatal Python error: Py_Initialize: can't initialize sys standard streams
>> OSError: [Errno 9] Bad file descriptor
>> Abandon
>>
>> Of course, if it is stderr that you explicitly close, you lose all reporting:
>>
>> $ ./python -c '1' 2>&-
>> Abandon
>
> Hmm, how does Python 3's pythonw work on Windows? (I don't have a
> Windows installation of Python 3 to hand at the moment)

When running pythonw, fileno(stdout) is negative, so sys.stdout is set to None,
and print() silently returns.
But the situation is not perfect, see http://bugs.python.org/issue6501
where Apache provides a stdout, but python cannot determine the
encoding because there is no console.

-- 
Amaury Forgeot d'Arc

From cournape at gmail.com  Thu Oct  8 19:22:32 2009
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 9 Oct 2009 02:22:32 +0900
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <7DD3FAEA-88FE-48EE-9B92-972F5C8EFB3E@masklinn.net>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
	<7DD3FAEA-88FE-48EE-9B92-972F5C8EFB3E@masklinn.net>
Message-ID: <5b8d13220910081022h19835524sea287e3fb256b0c6@mail.gmail.com>

On Fri, Oct 9, 2009 at 1:35 AM, Masklinn <masklinn at masklinn.net> wrote:
> On 8 Oct 2009, at 18:17 , Toshio Kuratomi wrote:
>>
>>> This is not at all how I use virtualenv. For me virtualenv is a
>>> sandbox so that I don't have to become root whenever I need to install
>>> a Python package for testing purposes
>>
>> This is needing to install multiple versions and use the newly installed
>> version for testing.
>>
> No it's not. It's keeping the python package *being tested* out of the
> system's or user's site-package because it's potentially unstable or
> unneeded. It provides a trivial way of *removing* the package to get rid of
> it: delete the virtualenv. No trace anywhere that the package was ever
> installed, no impact on the system ?(apart from the potential side-effects
> of executing the system).
>
> The issue here isn't "multiple installed packages", it will more than likely
> be the only version of itself: note that it's a package being tested, not an
> *upgrade* being tested.
>
> The issues solved are:
> * not having to become root (solved by PEP 370 if it ever lands)
> * minimizing as much as possible the impact of testing the package on the
> system (not solved by any other solution)

This is not true - stow solves the problem in a more general way (in
the sense that it is not restricted to python), at least on platforms
which support softlink. The only inconvenience of stow compared to
virtual env is namespace packages, but that's because of a design flaw
in namespace package (as implemented in setuptools, and hopefully
fixed in the upcoming namespace package PEP).

Virtualenv provides a possible solution to some deployment problems,
and is useful in those cases, but it is too specialized to be included
in python itself IMO.

cheers,

David

From guido at python.org  Thu Oct  8 19:33:39 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 8 Oct 2009 10:33:39 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4B9C0675D45347A887DBC074153641EA@RaymondLaptop1>
References: <4ACDFD59.4060200@trueblade.com> <E1MvubV-0004VK-R0@swing.co.at> 
	<ca471dc20910080955o1c55fc53gf26402e9ec30232b@mail.gmail.com> 
	<4B9C0675D45347A887DBC074153641EA@RaymondLaptop1>
Message-ID: <ca471dc20910081033x415f1a6fmb895c832344d6685@mail.gmail.com>

On Thu, Oct 8, 2009 at 10:14 AM, Raymond Hettinger <python at rcn.com> wrote:
> Do you think there may be some possible parallel to the -3 option
> to flag cases of %-formatting? ?If so, that could be helpful.

Absolutely. This should be simple, since there's just one or two
places where to place the warning. We might also automatically turn it
on when Python 2.7 is run with -3.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From masklinn at masklinn.net  Thu Oct  8 19:41:57 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 8 Oct 2009 19:41:57 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <5b8d13220910081022h19835524sea287e3fb256b0c6@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
	<7DD3FAEA-88FE-48EE-9B92-972F5C8EFB3E@masklinn.net>
	<5b8d13220910081022h19835524sea287e3fb256b0c6@mail.gmail.com>
Message-ID: <E1288F08-4C41-4FA0-8BA2-B07F1B56A4C2@masklinn.net>

On 8 Oct 2009, at 19:22 , David Cournapeau wrote:
>
> This is not true - stow solves the problem in a more general way (in
> the sense that it is not restricted to python), at least on platforms
> which support softlink.
I was, of course, talking about "pure" Python solutions (but I should  
indeed have qualified my statement thus), in the general cases there  
are several solutions to handle that including but not limited to stow  
you already mentioned, BSD jails or full-blown virtual machines.


From mal at egenix.com  Thu Oct  8 20:02:13 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Oct 2009 20:02:13 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080944q1785aad0nc15937610701190f@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<4ACDB82F.7010006@cryptelium.net>
	<4ACDCCBD.4040908@egenix.com>	<94bdd2610910080554m530bfdbeg2bf010c813c71873@mail.gmail.com>	<4ACDFD54.3080202@egenix.com>
	<94bdd2610910080944q1785aad0nc15937610701190f@mail.gmail.com>
Message-ID: <4ACE2925.6090900@egenix.com>

Tarek Ziad? wrote:
> On Thu, Oct 8, 2009 at 4:55 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> OTOH, the register command sends the meta-data directly to
>> the PyPI database, so it doesn't even need another file
>> for storing away the meta data.
>>
>> Unless I'm missing something important (which I probably am),
>> the only added value of the new format over PKG-INFO is the
>> micro-language.
> 
> That's one. The other one is to avoid to deal with yet another
> configuration file
> alongside setup.py, and just deal with a [metadata] section in setup.cfg.
>
> And PKG-INFO stays a static, platform-specific file that exists only when the
> distribution is installed on the target platform.

Hmm, PKG-INFO is generated from the parameters you pass to setup()
in setup.py, so the package author will not have to deal
with another config file and only has to write down the meta-data
in setup.py.

Being able to add some meta-data via setup.cfg would probably be an
extra feature which some may find handy.

However, your summary made also made it look like the meta-data would
be required in setup.cfg and that the whole file would get uploaded
(including all the options and settings for other distutils commands).

Was that intended ?

>>> No, but if you ask for it I can write it.
>>
>> Please do. The format will need documentation anyway and the
>> motivation should be made clear as well.
> 
> Ok I will

Thanks, I'll wait for the PEP.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 08 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From a.badger at gmail.com  Thu Oct  8 20:35:08 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 8 Oct 2009 11:35:08 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
In-Reply-To: <4ACE19B3.4060106@cryptelium.net>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACDB82F.7010006@cryptelium.net> <4ACDCCBD.4040908@egenix.com>
	<20091008161148.GG28231@clingman.lan>
	<4ACE19B3.4060106@cryptelium.net>
Message-ID: <20091008183508.GA513@clingman.lan>

On Thu, Oct 08, 2009 at 06:56:19PM +0200, kiorky wrote:
> 
> 
> Toshio Kuratomi a ?crit :
> > 
> > Also note, the ability to have multiple versions makes things easier for
> > system packagers and provides an easy alternative to a virtualenv for
> > end-users.
> > 
> > * System packagers: virtualenv does not provide a method suitable for system
> 
> Yes, there is no doubt virtualenv is useless for system packagers but:
> 
> * System and applications deployment have not to be tied.
> It s up to the user to install things system wide or to use locally isolation
> technics. Virtualenv is one of those.
> As a conclusion, there are not very much problem for system packagers as if
> their users have specific needs, they will do something Outside the system.
> If not, they use their global python with packages installed in that global one.
> 
This misses the point.  If there's two pieces of software to be deployed
via system packages and they use two different versions of a module, it's
currently not very easy to do this.  I do it with setuptools curently even
with all its warts.  Having a way to do this from within the stdlib that
tied directly into the import mechanism would make for a much cleaner
situation.

In other words, the suggestion that there is no need for a method to install
multiple versions of a module because virtualenv is a better solution is
bogus.  virtualenv is a better solution for creating isolated environments.
It is not a solution for all of the cases that being able to install
multiple versions of a library would be.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/5380a169/attachment-0001.pgp>

From a.badger at gmail.com  Thu Oct  8 20:40:03 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 8 Oct 2009 11:40:03 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap
	(and	some?words?on Virtualenv, Pip)
In-Reply-To: <loom.20091008T182710-265@post.gmane.org>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
	<loom.20091008T182710-265@post.gmane.org>
Message-ID: <20091008184003.GB513@clingman.lan>

On Thu, Oct 08, 2009 at 04:28:52PM +0000, Antoine Pitrou wrote:
> Toshio Kuratomi <a.badger <at> gmail.com> writes:
> > 
> > This is needing to install multiple versions and use the newly installed
> > version for testing.
> [...]
> 
> What you're missing is that having separate environments has a virtue of
> cleanliness, understandability and robustness that a multiple-versioned solution
> doesn't have. While the technical merits are debatable I'm sure some people
> definitely prefer to manage a virtualenv-based version.
> 
I'm not missing it.  I'm only saying that the precise requirement that is
being stated is not sandboxing (that was listed later).  It's being able to
use a newly installed library for testing.  The essential element of that is
being able to install a new version of the library and use that instead of
the sytem installed version.  sandboxing may be how someone wants to do this
but it isn't essential to be able to do this.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/0aa8c455/attachment.pgp>

From a.badger at gmail.com  Thu Oct  8 20:52:53 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 8 Oct 2009 11:52:53 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
In-Reply-To: <4ACE1388.5020602@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<20091008161732.GH28231@clingman.lan>
	<4ACE1388.5020602@voidspace.org.uk>
Message-ID: <20091008185253.GC513@clingman.lan>

On Thu, Oct 08, 2009 at 05:30:00PM +0100, Michael Foord wrote:
> Toshio Kuratomi wrote:
>> On Thu, Oct 08, 2009 at 02:52:24PM +0200, Simon Cross wrote:
>>   
>>>  On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>>     
>>>> = Virtualenv and the multiple version support in Distribute =
>>>>       
>>> ...
>>>     
>>>> My opinion is that this tool exists only because Python doesn't
>>>> support the installation of multiple versions for the same
>>>> distributions.
>>>>       
>> Let's actually look at these reasons:
>>
>>   
>>> This is not at all how I use virtualenv. For me virtualenv is a
>>> sandbox so that I don't have to become root whenever I need to install
>>> a Python package for testing purposes
>>>     
>>
>> This is needing to install multiple versions and use the newly installed
>> version for testing.
>>
>>   
>
> Not really - it is wanting to install a single version of a library that 
> you don't want to install into your 'main' (whether that be user or  
> system) Python install. It is sandboxing and orthogonal to multiple  
> versions.
>
What I'm replying to is specifically the need to: "become root whenever I
need to install a Python package for testing purposes" That doesn't require
sandboxing at all.  Can you use sandboxing to do this?  Yes.  But that is
separate to being able to install a non-system version of a library and be
able to test it.

My quoting of that phrase could have been better -- I missed the reference
to sandboxing since it is in a separate clause of the sentence from what I
was responding to.

>>>
>>> and to allow me to hop between
>>> sets of installed Python packages while developing on multiple Python
>>> projects.
>>>     
>>
>> This is the ability to install multiple versions and specify different
>> versions for different projects you're working on.
>>   
>
> No, it is working on multiple projects that have *different*  
> dependencies and being able to work in an environment that *only* has  
> direct dependencies installed - again sandboxed from your main Python  
> environment.
>
No. Here what is written is: "allow me to hop between sets of installed Python
packages while developing on multiple Python projects."

There's nothing about *only* having direct dependencies installed.  That's a
separate requirement than what was written.

> The fact that virtualenv allows you to have multiple different versions  
> of the same library installed in the different environments you create  
> is completely separate from the motivation that causes many people to  
> use it.
>
Precisely!  We see 100% eye-to-eye here.  My reply is just trying to say
that the ideas of
* testing a locally installed, conflicting version of a library
* running multiple projects with different, conflicting version requirements

are completely satisfiable without sandboxing.  Virtualenv is a sandboxing
tool.  It can be used to perform these tasks.  But it isn't necessary.
Having sandboxing is an additional feature on top of the base requirements
to perform the task.

> What virtualenv *doesn't* do (I believe) is allow you to have multiple  
> versions of libraries installed within a single environment and switch  
> between them (at least it doesn't offer anything beyond what setuptools  
> or pip provides).
>
Yep.  Which makes virtualenv unsuitable for certain other problem spaces
where sandboxing is inappropriate.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/41651a0c/attachment.pgp>

From martin at v.loewis.de  Thu Oct  8 21:00:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 08 Oct 2009 21:00:34 +0200
Subject: [Python-Dev] Logging, Unicode and sockets
In-Reply-To: <loom.20091008T075520-844@post.gmane.org>
References: <loom.20091008T075520-844@post.gmane.org>
Message-ID: <4ACE36D2.3060308@v.loewis.de>

> I've noticed that the socket-based logging handlers - SocketHandler,
> DatagramHandler and SysLogHandler - aren't Unicode-aware and can break in the
> presence of Unicode messages.

I can't understand what the problem with SocketHandler/DatagramHandler
is. As they use pickle, they should surely be able to send records with
Unicode strings in them, no?

OTOH, why is SMTPHandler not in your list?

> I'd like to fix this by giving these handlers an
> optional (encoding=None) parameter in their __init__, and then using this to
> encode on output.

For syslog, I don't think that's appropriate. I presume this is meant to
follow RFC 5424? If so, it SHOULD send the data in UTF-8, in which case
it MUST include a BOM also. A.8 then says that if you are not certain
that it is UTF-8 (which you wouldn't be if the application passes a byte
string), you MAY omit the BOM.

Regards,
Martin

From sridharr at activestate.com  Thu Oct  8 21:35:22 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Thu, 08 Oct 2009 12:35:22 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <4ACDEEB9.2000308@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
	<4ACDEEB9.2000308@voidspace.org.uk>
Message-ID: <op.u1hx08km1q4jlt@whymac.activestate.com>

On Thu, 08 Oct 2009 06:52:57 -0700, Michael Foord  
<fuzzyman at voidspace.org.uk> wrote:

> I don't recall ever having wanted / needed to install multiple versions  
> of the same library - whilst I can appreciate that it *can* be a real  
> issue it has never been a problem for me.

Multiple versions is going to be a mess. It is a pathetic workaround for
packages that do not care about backward compatibility (eg: CherryPy-2.x
vs CherryPy-3.x).  Drop support for multiple versions to force package
authors to deal with it.

I applaud the Jinja team for doing this:

     ARMIN: The final version of the Jinja2 Django-inspired template
     engine was just released. It comes with tons of improvements over
     the older Jinja1 engine and breaks backwards compatibility to some
     point over the old Jinja1 engine. It's packaged as a separate
     package so that you can use both right next to each other for an
     easier transition.

http://lucumr.pocoo.org/2008/7/17/jinja2-final-aka-jinjavitus-released

This is something the Python community can also learn from the Perl
world.

-srid


From p.f.moore at gmail.com  Thu Oct  8 14:56:05 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 8 Oct 2009 13:56:05 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080211q5ff4fd09wf62c44dd4a680a8f@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<79990c6b0910080156s2e540184r255624d2166cbae2@mail.gmail.com>
	<94bdd2610910080211q5ff4fd09wf62c44dd4a680a8f@mail.gmail.com>
Message-ID: <79990c6b0910080556m1f2d4156v791193da56db40fb@mail.gmail.com>

2009/10/8 Tarek Ziad? <ziade.tarek at gmail.com>:
>> The egg format
>>
>> One thing missing from your roadmap (unless I missed it) is the fate
>> of the egg (zipfile) format. If it's to remain a valid option
>> (bdist_egg and the like) then I'd assume that Distribute needs to be
>> the place to develop it.
>> One thing it will need is an equivalent to
>> PEP 376 introspection and uninstall features. And *that* is where I
>> believe a standard protocol belongs - developed based on the real use
>> requirements for egg files, with a suitable file-based layer which can
>> be added into distutils (somewhat like importlib does for PEP 302)
>> once the protocol is agreed and standardised.
>
> Distribute is planning to continue supporting the egg format, but will
> probably drop the zipped version.
> This is still under discussions though so don't take this as the final word.
>
> In any case, distribute.resources will probably add the missing bits
> on the top of a PEP 376 -compliant Distutils
> to provide support for any file querying Distutils will not provide
> (for installation and uninstallation)

[Note: For the purposes of below I use "egg format" to only refer to
zipped eggs. I see no value AT ALL in having an unzipped "egg format"
which is almost the same as, but subtly different from, the
distutils-supported standard filesystem format. If the distutils
standard doesn't provide for all the requirements, fix that, don't add
another variation!]

I'm not sure I follow this. I think it's important to provide for
non-filesystem based packages (that's the whole point of things like
PEP 302 and importlib!). The fact that core distutils doesn't support
installing, querying or removing such packages is fine. They can be
supplied by 3rd party systems like Distribute.

However, it is *not* fine (in my view) for each independent format to
have to implement its own set of interfaces. PEP 302 is precisely the
solution for this in terms of import - every importer has a
standardised set of protocols to follow, and if it does so, then it
will (should :-)) work seamlessly with any Python code. PEP 302 is not
perfect, in part because it aims to be minimal rather than
comprehensive, hence the fact that not all packages are "zip safe".
But that's a failing in PEP 302 rather than in the concept.

Multiple package formats should be supported in the same way - with a
set of well-defined (via a PEP) interfaces that all package formats
must provide, which enable core and user code to be written in a
format-independent way. Once such a PEP is written, distutils' PEP 376
support should be rewritten to use those interfaces, and to implement
the necessary filesystem-format support layer. (In other words,
whereas I support reducing PEP 376's scope to filesystem only for now,
I still strongly advocate that the longer-term solution should be
based on a user extensible format-independent approach).

In this context, eggs are "merely" the first (and most important)
example of a format extension, and so should drive the development of
a standard.

To summarise:

I believe that we need a statement of direction on the (zipped) egg
format. I see a number of options:

1. Retain it as-is in Distribute, but deprecated and no further development.
2. Deprecate egg support in Distribute, and ultimately drop it.
3. Develop into a reference example of an extensible architecture for
adding to PEP 376 support.
4. Extend PEP 376 support to eggs by means of egg-specific approaches
hooking into distutils.

For the record, I dislike (1), I see (2) as a lost opportunity, I see
(3) as the right way to go, but appreciate that it's the most work,
and I strongly oppose (4) (not least because it effectively makes a
latrge part of PEP 302 and importlib wasted effort).

It looks like you're proposing (2), but aren't entirely clear because
you're still allowing for non-zipped "egg" formats (which I see no
value in, as I noted above).

I'm willing to help work on option (3). Once it's decoupled from
questions around OS package managers (which are more the province of
PEP 376) then I'd hope that my lack of knowledge about package
managers is less of a problem.

Paul.

From vinay_sajip at yahoo.co.uk  Thu Oct  8 21:55:15 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 8 Oct 2009 19:55:15 +0000 (UTC)
Subject: [Python-Dev] Logging, Unicode and sockets
References: <loom.20091008T075520-844@post.gmane.org>
	<4ACE36D2.3060308@v.loewis.de>
Message-ID: <loom.20091008T213558-141@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:

> I can't understand what the problem with SocketHandler/DatagramHandler
> is. As they use pickle, they should surely be able to send records with
> Unicode strings in them, no?

Of course you are right. When I posted that it was a knee-jerk reaction to the
issue that was raised for SysLogHandler configured to use UDP. I did realise a
bit later that the issue didn't apply to the other two handlers but I was hoping
nobody would notice ;-)

> OTOH, why is SMTPHandler not in your list?

I assumed smtp.sendmail() would deal with it, as it deals with the wire
protocol, but perhaps I was wrong to do so. I noticed that Issue 521270 (SMTP
does not handle Unicode) was closed, but I didn't look at it closely. I now see
it was perhaps only a partial solution. I did a bit of searching and found this
post by Marius Gedminas:

http://mg.pov.lt/blog/unicode-emails-in-python.html

Now if that's the right approach, shouldn't it be catered for in a more general
part of the stdlib than logging - perhaps in smtplib itself? Or, seeing that
Marius' post is five years old, is there a better way of doing it using the
stdlib as it is now?

> For syslog, I don't think that's appropriate. I presume this is meant to
> follow RFC 5424? If so, it SHOULD send the data in UTF-8, in which case
> it MUST include a BOM also. A.8 then says that if you are not certain
> that it is UTF-8 (which you wouldn't be if the application passes a byte
> string), you MAY omit the BOM.

So ISTM that the right thing to do on 2.x would be: if str to be sent, send as
is; if unicode to be sent, encode using utf-8 and send with a BOM. For 3.x, just
encode using utf-8 and send with a BOM.

Does that seem right?

Thanks and regards,

Vinay Sajip


From nad at acm.org  Thu Oct  8 23:28:47 2009
From: nad at acm.org (Ned Deily)
Date: Thu, 08 Oct 2009 14:28:47 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <nad-B71566.14284708102009@ger.gmane.org>

In article 
<94bdd2610910080131j323b98d9i871bce43465f237a at mail.gmail.com>,
 Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Here's a quick summary of the main things that are going to happen in
> Distutils, and Distribute, and a few words on virtualenv and pip.
> (there is much much more work going on, but I don't want to drown
> people with details)

Thanks for the summary!   Unfortunately, as the saying goes, the devil 
is in the details.  Here are a few comments.

> = Distribute =
> 
> I won't explain here again why we have forked, I think it's obvious to
> anyone here now. I'll rather explain what
> we are planning in Distribute and how it will interact with Distutils.
> 
> Distribute has two branches:
> 
> - 0.6.x : provides a Setuptools-0.6c9 compatible version
> - 0.7.x : will provide a refactoring
> 
> == 0.6.x ==
> 
> Not "much" is going to happen here, we want this branch to be helpful
> to the community *today* by addressing the 40-or-so bugs
> that were found in Setuptools and never fixed. This is eventually
> happen soon because its development is
> fast : there are up to 5 commiters that are working on it very often
> (and the number grows weekly.)
> 
> The biggest issue with this branch is that it is providing the same
> packages and modules setuptools does, and this
> requires some bootstrapping work where we make sure once Distribute is
> installed, all Distribution that requires Setuptools
> will continue to work. This is done by faking the metadata of
> Setuptools 0.6c9. That's the only way we found to do this.
> 
> There's one major thing though: thanks to the work of Lennart, Alex,
> Martin, this branch supports Python 3,
> which is great to have to speed up Py3 adoption.
> 
> The goal of the 0.6.x is to remove as much bugs as we can, and try if
> possible to remove the patches done
> on Distutils. We will support 0.6.x maintenance for years and we will
> promote its usage everywhere instead of
> Setuptools.

This is a key and, I think, difficult issue that's going to need a lot 
of thought and work to be successful.  setuptools / easy_install filled 
several unmet needs of both package developers and end users and became 
a de-facto standard because there was essentially no competition.  So, 
in a sense, there was never a migration to setuptools from something.  
But now, especially given GvR's blessing of Distribute, there needs to 
be a migration from setuptools to Distribute, although neither are part 
of the standard library.  There are no references to setuptools or 
easy_install in any of the standard Python documentation.

So how do end users know about easy_install?  Their first encounter 
is/was probably through some recipe on a web site, most likely the 
installation instructions for some package they want to use.  And I 
suspect pretty much all of those installation instructions direct the 
user to one of two places, either the PyPi page for setuptools or the 
peakcommunity wiki page:

   http://pypi.python.org/pypi/setuptools
   http://peak.telecommunity.com/DevCenter/EasyInstall

Right now those are essentially the ultimate contact points for users.  
A general Google seach for easy_install gets 357,000 hits; one 
restricted to pypi.python.org gets 2,500.   I don't know how many of the 
latter are active packages on PyPi but certainly there must be hundreds 
that currently use setuptools and document the use of easy_install for 
their users.  Those packages' documentation often directs users to 
download the ez_install script to bootstrap setuptools; I believe there 
are some packages that download it automatically, if necessary.

So, without some other intervention, users will continue to follow the 
instructions with those hundreds of packages (or on those thousands of 
websites) and manually or automatically try to install setuptools.  
Which may or may not work - as witnessed with the release of 2.6.3.

Brett and others suggested including some sort of warnings in the python 
release documentation.  Is there consensus that should be done?  
Starting with 2.6.4?   Seems like that would be up to Barry and Guido to 
pronounce.

Ultimately, to successfully migrate to Distribute is going to require a 
good understanding of the use cases for easy_install (primarily those of 
end users) and for the setuptools API (those of package developers).  
Those use cases should be documented somehow, either in the roadmap or a 
reference elsewhere.  This is especially true if your intentions are to 
deprecate the easy_install command in Distribute 0.7 and if APIs are 
going to change for package developers.  As Antoine points out, pip is 
not equivalent to easy_install.  BTW, keep in mind that today people 
also use easy_install to manage the installation of packages that do not 
use setuptools.

It will also be important to understand and address the major 
distribution channels for Python installations, many of which already 
include setuptools in their distributions.  There are the multi-platform 
distributors, like python.org (which does not currently distribute 
setuptools), ActiveState, and Enthought.  You've already mentioned some 
of the Unix-y O/S ones: Debian, Fedora, et al.  On the OS X side, 
there's Apple and the major third-party providers of open-source 
packages, MacPorts and Fink - they all distribute setuptools.  I don't 
have personal experience with the Windows world to know what all the 
channels are there.  Are there other operating systems who package 
Python and possibly setuptools?  They, one way or another, need to know 
and have some sort of plan going forward for their distributions 
regarding setuptools and Distribute.

And, of course, the stickiest issues in all this are not technical, but 
rather social.  For most of this, the most important part will likely be 
giving a clear direction to all of the major groups involved: end users, 
package developers, distributors.  How to decide on and then effectively 
communicate that direction is not at all trivial, I think.

Thanks again for your hard work on this, Tarek!

-- 
 Ned Deily,
 nad at acm.org


From solipsis at pitrou.net  Thu Oct  8 23:41:21 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 8 Oct 2009 21:41:21 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Distutils_and_Distribute_roadmap_=28and_so?=
	=?utf-8?q?me_words=09on_Virtualenv=2C_Pip=29?=
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
Message-ID: <loom.20091008T233831-780@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:
> 
> How to decide on and then effectively 
> communicate that direction is not at all trivial, I think.

I think it's quite trivial actually. Since everybody agrees (except perhaps PJE)
that Distribute should replace setuptools, the word will spread and trickle
quite quickly in the various layers of the community.

If you look at the XFree86 -> XOrg transition, I don't think there have been a
lot of problems to bring people to understand the change.

Regards

Antoine.



From nad at acm.org  Fri Oct  9 00:01:22 2009
From: nad at acm.org (Ned Deily)
Date: Thu, 08 Oct 2009 15:01:22 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some
	words	on Virtualenv, Pip)
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<loom.20091008T233831-780@post.gmane.org>
Message-ID: <nad-15D2AD.15012208102009@ger.gmane.org>

In article <loom.20091008T233831-780 at post.gmane.org>,
 Antoine Pitrou <solipsis at pitrou.net> wrote:
> Ned Deily <nad <at> acm.org> writes:
> > How to decide on and then effectively 
> > communicate that direction is not at all trivial, I think.
> 
> I think it's quite trivial actually. Since everybody agrees (except perhaps 
> PJE)
> that Distribute should replace setuptools, the word will spread and trickle
> quite quickly in the various layers of the community.
> 
> If you look at the XFree86 -> XOrg transition, I don't think there have been 
> a
> lot of problems to bring people to understand the change.

I'm not sure that's a such a good example: some of the impacts of that 
transition are still affecting users.

The unspoken assumption here is that it is a good idea to make this 
transition as painless as possible within some unspecified limits of 
practicality.  Sure, at one end of the spectrum, you can just throw 
things out in the world and hope for the best.  I think the Python 
community has a well-deserved reputation for doing a lot better than 
that.

-- 
 Ned Deily,
 nad at acm.org


From ziade.tarek at gmail.com  Fri Oct  9 00:55:56 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 00:55:56 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <nad-B71566.14284708102009@ger.gmane.org>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
Message-ID: <94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>

On Thu, Oct 8, 2009 at 11:28 PM, Ned Deily <nad at acm.org> wrote:
[..]
> So, without some other intervention, users will continue to follow the
> instructions with those hundreds of packages (or on those thousands of
> websites) and manually or automatically try to install setuptools.

I think this will be in the hands of the projects developers : if they start to
use Distribute, they will start to document the way to install and use it,
and the word will spread eventually.

[..]
> Ultimately, to successfully migrate to Distribute is going to require a
> good understanding of the use cases for easy_install (primarily those of
> end users) and for the setuptools API (those of package developers).
> Those use cases should be documented somehow, either in the roadmap or a
> reference elsewhere.  This is especially true if your intentions are to
> deprecate the easy_install command in Distribute 0.7 and if APIs are
> going to change for package developers.  As Antoine points out, pip is
> not equivalent to easy_install.  BTW, keep in mind that today people
> also use easy_install to manage the installation of packages that do not
> use setuptools.

The choice to deprecate easy_install in 0.7 is done because the Pip project
is not far to meet all uses cases easy_install users have, and we're betting
on the fact that Pip is active and will be much more advanced that what
we could do with a 'new' refactored easy_install by the time 0.7 is ready.

meaning : why doing this work twice ?

> Thanks again for your hard work on this, Tarek!

Thanks for the advices

Tarek

From kiorky at cryptelium.net  Fri Oct  9 01:11:09 2009
From: kiorky at cryptelium.net (kiorky)
Date: Fri, 09 Oct 2009 01:11:09 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
Message-ID: <4ACE718D.8090400@cryptelium.net>



Tarek Ziad? a ?crit :

> The choice to deprecate easy_install in 0.7 is done because the Pip project
> is not far to meet all uses cases easy_install users have, and we're betting
> on the fact that Pip is active and will be much more advanced that what
> we could do with a 'new' refactored easy_install by the time 0.7 is ready.

But how about retro-compatibility?
Like with all those buildout recipes which relies on setuptool APIs
(Requirement, PackageIndex and so on), it's a risk to break same at some point.



> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/63f9a742/attachment.pgp>

From v+python at g.nevcal.com  Fri Oct  9 01:13:13 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 08 Oct 2009 16:13:13 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACDF621.4070603@trueblade.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>	<loom.20091007T191603-331@post.gmane.org>	<4ACDD2C3.4090608@gmail.com>	<loom.20091008T160529-262@post.gmane.org>
	<4ACDF621.4070603@trueblade.com>
Message-ID: <4ACE7209.9060208@g.nevcal.com>

On approximately 10/8/2009 7:24 AM, came the following characters from 
the keyboard of Eric Smith:
> Vinay Sajip wrote:
>> BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see 
>> if it was
>> worth raising an enhancement request on the bug tracker using "O" to 
>> generate
>> compatible rendering for octals.
> 
> I didn't get your message, could you resend?.
> 
> I was thinking the same thing, but it seems like a transition step. I'd 
> rather not keep such backward compatibility hacks (if you will) around 
> for the long haul.  How about a flag (maybe '*') at the start of the 
> format specification which says "operate in backward compatibility 
> mode"? We could document it as being only useful for the % to {} 
> translator, and promise to remove it at some point in the future. Either 
> actually deprecate it or just promise to deprecate it in the future.


Seems like the ability for Python {} formatting to be able to match not 
only old Python % formatting output, but also output created by C's 
sprintf, and other numeric formatting systems, make this particular 
feature useful in more scenarios than a "backward compatibility hack".

If you want to replace a C program that produces parsed output in a 
given format, and that given format includes leading-0-octal numbers, 
then it would be good to have the capability in Python .format, even 
though Python itself uses 0o prefix.

Similar arguments may apply anywhere else that sprintf produces 
something that .format cannot currently produce.


-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From ziade.tarek at gmail.com  Fri Oct  9 01:21:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 01:21:20 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACE718D.8090400@cryptelium.net>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
Message-ID: <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>

On Fri, Oct 9, 2009 at 1:11 AM, kiorky <kiorky at cryptelium.net> wrote:
>
>
> Tarek Ziad? a ?crit :
>
>> The choice to deprecate easy_install in 0.7 is done because the Pip project
>> is not far to meet all uses cases easy_install users have, and we're betting
>> on the fact that Pip is active and will be much more advanced that what
>> we could do with a 'new' refactored easy_install by the time 0.7 is ready.
>
> But how about retro-compatibility?
> Like with all those buildout recipes which relies on setuptool APIs
> (Requirement, PackageIndex and so on), it's a risk to break same at some point.

Keep in mind that Distribute 0.6.x will still be maintained and will
still provide easy_install and al,
and that you will be able to rely on it for your projects.

Distribute 0.7.x is to be considered as a new project in some ways :
all namespaces
will be different and the code will not interfer with a 0.6 installation.

So a buildout recipe will have the choice to use the old packages you
are mentioning and
at some point evolve and use the new tools 0.7 will provide.

Tarek

From david.lyon at preisshare.net  Fri Oct  9 01:50:13 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 08 Oct 2009 19:50:13 -0400
Subject: [Python-Dev] Distutils and Distribute roadmap
In-Reply-To: <loom.20091008T233831-780@post.gmane.org>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<loom.20091008T233831-780@post.gmane.org>
Message-ID: <599f3554ce7cf10c18de71c7c5702c33@preisshare.net>

On Thu, 8 Oct 2009 21:41:21 +0000 (UTC), Antoine Pitrou
<solipsis at pitrou.net> wrote:
> I think it's quite trivial actually. Since everybody agrees (except
perhaps
> PJE)
> that Distribute should replace setuptools, the word will spread and
trickle
> quite quickly in the various layers of the community.

I don't think that it's that black and white.

PJE put a lot of work (and creativity) into setuptools. There's some degree
of tragedy if his project gets forked off simply because of failures in
the testing and quality assurance department.

Personally, I find the setuptools code somewhat hard to follow. The style
goes back to early programming by 'masters' where the only person that
could follow it was the master himself.

Sadly, that is not what we need in open source. The newer trend is in 
highly readily readable and easy to understand code.

Tarek is excellent at producing high quality code of the type that
is required. But his programming world (on the mac) isn't full of the 
challenges that we poor other unfortunates face on other platforms face.

Coming back to the point, setuptools is a pretty poor package
to have to fork. It's very difficult to follow and the structure
and implementation is cryptic. Perphaps it's a thankless job
that Tarek has taken on in that regard - but he seems to be handling
the distribute fork quite well.

Being new to python, it's taken me a little while to figure out
the process. But I've decided to write a static metadata pep which
is underway now as an alternate proposal.

Being primarily a hacker, writing nice design documents doesn't come
easy. 

If there is any distutils roadmap, I'd like to see improvements
that work on Windows. Yes.. python does work on windows and we
can't all get our bosses to change to os-x just yet. 

There's many things in distutils that I could best describe as 
being unfinished or partially done. 

Let me make a few rambling direction suggestions...

 - command line package installation for distutils

   We need a simple script, not based on setuptools to
   allow installation, listing and deinstallation of packages.

   Currently *nothing* exists in distutils to do this.

   It's a **major** failing.

 - Improvements to operation on platform Windows

   Package/Application installation on windows sucks...

   For example, there's no documented way to get your
   programs installed to "\Program Files\myApplication"

   For non windows users, that's where programs are
   supposed to go. Any other place is "evil" - like
   where they are going now.

   If it is only a documentation issue, well can we
   fix that.

   All I'm asking for is a matching "elif sys.platform = 'win32'"
   for every "if sys.platform = 'darwin'".... or less. It's
   not unreasonable...

It's probably a sensible idea for Tarek to keep going with
Distribute. My user experience with setuptools was that it
wasted a lot of my time unnecessarily. 

I'm planning to put Distribute support in the Python Package
Manager project (pythonpkgmgr) for it's next release.

David


   








From david.lyon at preisshare.net  Fri Oct  9 02:50:16 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 08 Oct 2009 20:50:16 -0400
Subject: [Python-Dev] =?utf-8?q?eggs_now_mandatory_for_pypi=3F?=
In-Reply-To: <4ACD965D.5080701@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>
	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>
	<4ACD965D.5080701@egenix.com>
Message-ID: <30910449ef0aad44ac8d419c4e36896f@preisshare.net>

On Thu, 08 Oct 2009 09:35:57 +0200, "M.-A. Lemburg" <mal at egenix.com> wrote:
>> One could say that much of the setuptools cloud came about because of
>> the lack of the queryable download url. Setuptools does a lot of work
>> here to 'work-around' the ommission on pypi of a package download url.
> 
> I think you two have missed my email to PJE.
> 
> In summary:
> 
> PyPI already does have mechanism for querying download URLs.
> 
> 	http://wiki.python.org/moin/PyPiXmlRpc
> 
> It only lacks some more meta-data that would be required to
> match up the user's configuration with the one used in the
> uploaded URLs.
> 
> And it would be handy to have a mechanism to just upload the
> URL data rather than the whole package. Looking at distutils'
> upload command, PyPI currently appears to only support uploads
> of the package file itself.
> 
> Basically, most of the mechnisms are already there, they just need
> some extra care.

Maybe you're right. I'll look into it.

> BTW: Who would I need to contact for the PyPI side to work out
> an upload_url distutils command ?

pypi is on sourceforge... ask to become a developer I guess..

David




From ianb at colorstudy.com  Fri Oct  9 03:22:13 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 8 Oct 2009 20:22:13 -0500
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> 
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> 
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
Message-ID: <b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>

I'm coming in late and breaking threading, but wanted to reply to
Tarek's original email:

> - easy_install is going to be deprecated ! use Pip !

Cool!  I wouldn't have written pip if I didn't think it would improve
substantially on easy_install.

Incidentally (because I know people get really enthused about this)
Carl Meyer just contributed a feature to pip to do atomic
uninstallation.

Someone mentioned that easy_install provided some things pip didn't;
outside of multi-versioned installs (which I'm not very enthusiastic
about) I'm not sure what this is?

>   - distribute.resources: that's the old pkg_resources, but reorganized in clean, pep-8 modules. This package will
>      only contain the query APIs and will focus on being PEP 376 compatible. We will promote its usage and see if Pip wants
>      to use it as a basis. And maybe PyPM once it's open source ? (<hint> <hint>).
> 	It will probably shrink a lot though, once the stdlib provides PEP 376 support.

This seems easy enough to use in pip.

>    - distribute.index: that's package_index and a few other things. everything required to interact with PyPI. We will promote
>      its usage and see if Pip wants to use it as a basis.

This is a little tricky.  Primarily because there's a fair amount of
logic involved in the indexing (going around to different URLs,
parsing links, finding stuff).  So long as there is logic, something
can go wrong -- often not in the package itself, but simple user error
(e.g., it doesn't look where the user thinks it should, or a link is
malformed, etc).  Because of this, and as a general design goal of
pip, I want to show as much as I can about what it is doing and why.
This is primarily tied into pip's logging system (which is oriented
towards command-line output, and isn't the standard logging system).
Also it tracks *why* it got to a certain links.  These are the two
things I can think of where the index code in pip is tied to pip, and
why it would be hard to use an external system.


> = Virtualenv and the multiple version support in Distribute =
>
> (I am not saying "We" here because this part was not discussed yet
> with everyone)
>
> Virtualenv allows you to create an isolated environment to install some distribution without polluting the
> main site-packages, a bit like a user site-packages.
>
> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.
> But if PEP 376 and PEP 386 support are added in Python, we're not far
> from being able to provide multiple version support with
> the help of importlib.

Before making workingenv (virtualenv's predecessor) I actively tried
to use Setuptools' multi-version support.  I found it very
unsuccessful.  I don't think it was due to any problems with
Setuptools -- maybe a slight problem was the conflict between "active"
eggs and "multiversion" eggs (where active eggs would be more likely
to cause conflicts, while multiversion eggs aren't available until you
specifically require them).  But that was just awkward, I don't think
it was the real problem.

The real problem is that a set of packages that makes up a working
application is something separate from any library.  And you can only
know that an application works with an exact set of libraries.  Every
update can break a working application (and with fairly high
probability).  You can't know what updates are safe.  And it's really
a bit tricky to even be sure you know what libraries a package really
requires -- lots of libraries might be incidentally available but no
longer formally required.  (Someone mentioned a coworker that only
installed packages with easy_install -m, because he said it kept him
honest -- only packages that are explicitly required would be
available.  But most people don't do this, and it really only solves
the one problem of undeclared dependencies)

The way both virtualenv and buildout handle this is that libraries
will have a single, static version until you explicitly do something
to update that version.  Both are somewhat focused on a functional
unit -- like one virtualenv environment for one task, or one buildout
config for one application.  Buildout allows for a globally shared set
of versioned eggs, but that's really just a little optimization (for
disk space or installation speed) -- each egg is brought in only
explicitly, at build time, and not as an option during the program's
runtime.

This is verifiable, stable, and to varying degrees concrete
(virtualenv being more concrete than buildout, which tends more
towards the declarative).

What virtualenv does could certainly be in the Python interpreter (and
much more compact as a result, I am sure).  PYTHONHOME does it to a
degree, though binding a script to a environment through the
interpreter listed in #! is more stable than the implicit environment
of PYTHONHOME.  workingenv used an environmental variable (PYTHONPATH,
before PYTHONHOME existed) and it caused problems.  Also virtualenv
offers more system isolation.

If I had my way, buildout would use virtualenv and throw away its
funny script generation.  If virtualenv had existed before buildout
began development, probably things would have gone this way.  I think
it would make the environment more pleasant for buildout users.  Also
I wish it used pip instead of its own installation procedure (based on
easy_install).  I don't think the philosophical differences are that
great, and that it's more a matter of history -- because the code is
written, there's not much incentive for buildout to remove that code
and rely on other libraries (virtualenv and pip).

--
Ian Bicking ?| ?http://blog.ianbicking.org ?| ?http://topplabs.org/civichacker

From mal at egenix.com  Fri Oct  9 10:20:54 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 09 Oct 2009 10:20:54 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <30910449ef0aad44ac8d419c4e36896f@preisshare.net>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>	<4ACD965D.5080701@egenix.com>
	<30910449ef0aad44ac8d419c4e36896f@preisshare.net>
Message-ID: <4ACEF266.3020309@egenix.com>

David Lyon wrote:
> On Thu, 08 Oct 2009 09:35:57 +0200, "M.-A. Lemburg" <mal at egenix.com> wrote:
>>> One could say that much of the setuptools cloud came about because of
>>> the lack of the queryable download url. Setuptools does a lot of work
>>> here to 'work-around' the ommission on pypi of a package download url.
>>
>> I think you two have missed my email to PJE.
>>
>> In summary:
>>
>> PyPI already does have mechanism for querying download URLs.
>>
>> 	http://wiki.python.org/moin/PyPiXmlRpc
>>
>> It only lacks some more meta-data that would be required to
>> match up the user's configuration with the one used in the
>> uploaded URLs.
>>
>> And it would be handy to have a mechanism to just upload the
>> URL data rather than the whole package. Looking at distutils'
>> upload command, PyPI currently appears to only support uploads
>> of the package file itself.
>>
>> Basically, most of the mechnisms are already there, they just need
>> some extra care.
> 
> Maybe you're right. I'll look into it.
> 
>> BTW: Who would I need to contact for the PyPI side to work out
>> an upload_url distutils command ?
> 
> pypi is on sourceforge... ask to become a developer I guess..

I know that the issue tracker for PyPI is (still) on SF, but
development appear to happen elsewhere and I can't find any
contact information on the PyPI web pages.

Is Richard Jones still working on it ? They I could contact him
directly to hash this out.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 09 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From kiorky at cryptelium.net  Fri Oct  9 10:54:23 2009
From: kiorky at cryptelium.net (kiorky)
Date: Fri, 09 Oct 2009 10:54:23 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
Message-ID: <4ACEFA3F.2020706@cryptelium.net>

Ian Bicking a ?crit :
> I'm coming in late and breaking threading, but wanted to reply to
> Tarek's original email:
> 

> This is verifiable, stable, and to varying degrees concrete
> (virtualenv being more concrete than buildout, which tends more
> towards the declarative).

Is that a friday troll ? declarative over control ?

> 
> What virtualenv does could certainly be in the Python interpreter (and
> much more compact as a result, I am sure).  PYTHONHOME does it to a
> degree, though binding a script to a environment through the
> interpreter listed in #! is more stable than the implicit environment
> of PYTHONHOME.  workingenv used an environmental variable (PYTHONPATH,
> before PYTHONHOME existed) and it caused problems.  Also virtualenv
> offers more system isolation.
> 
> If I had my way, buildout would use virtualenv and throw away its
> funny script generation.  If virtualenv had existed before buildout

Which one, the one provided to generate scripts from entry points with the *.egg
recipes or the bin/buildout auto regeneration?

> began development, probably things would have gone this way.  I think
> it would make the environment more pleasant for buildout users.  Also

* I don't think so, buildout is the only tool atm that permit to have really
reproducible and isolated environments. Even, if you use the pip freezing
machinery, it is not equivalent to buildout, Control!
* Buildout can have single part to construct required eggs, at a specific
version and let you control that. Pip will just search for this version, see
that it's not available and fail. You have even recipes (like
minitage.recipe.egg that permit to construct eggs with special version when you
apply patches onto, thus, you can have the same egg in different flavors in the
same eggs cache available for different projects. Those projects will just have
to pin the right version to use, Control!.
* Another thing is the "funny script generation", you have not one global
site-packages for your project, but one global cache. But from this global
cache, your scripts will only have available the eggs you declared, see Control!
* Moreover buildout is not only a python packages manager, it's some of its
recipes that permit to use it as. Buildout is just a great deployment tool that
allow to script and manage your project in a "funny" and "flexible" way, Control!

> I wish it used pip instead of its own installation procedure (based on
> easy_install).  I don't think the philosophical differences are that
> great, and that it's more a matter of history -- because the code is

And retro-compatibility.

> written, there's not much incentive for buildout to remove that code
> and rely on other libraries (virtualenv and pip).

* Virtualenv is not really needed as buildout uses "eggs caches" and do the
isolation from there. Scripts generate the adequate sys.path relying on the
application requirements (which, once more, you are in total control). I prefer
throught to use a virtualenv for each project (or a bunch of projects), and to
bootstrap my buildouts from there. For me, they are complementary to achieve
your deployment and isolation needs.

> 
> --
> Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/4d1df0b4/attachment.pgp>

From ziade.tarek at gmail.com  Fri Oct  9 12:03:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 12:03:34 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
Message-ID: <94bdd2610910090303v4b3b9b46q3f77266e34c33197@mail.gmail.com>

On Fri, Oct 9, 2009 at 3:22 AM, Ian Bicking <ianb at colorstudy.com> wrote:
> I'm coming in late and breaking threading, but wanted to reply to
> Tarek's original email:
>
>> - easy_install is going to be deprecated ! use Pip !
>
> Cool! ?I wouldn't have written pip if I didn't think it would improve
> substantially on easy_install.
>
> Incidentally (because I know people get really enthused about this)
> Carl Meyer just contributed a feature to pip to do atomic
> uninstallation.

Yes I saw that, it's great. And he is now involved in PEP 376 work so
Pip could eventually become the first PEP 376 compliant installer/uninstaller.

>
> Someone mentioned that easy_install provided some things pip didn't;
> outside of multi-versioned installs (which I'm not very enthusiastic
> about) I'm not sure what this is?

Basically what you've listed on Pip front page I think, like 'not
tested under windows'
But I don't see any blocking point besides some testing, to move from
easy_install to pip,
and the deprecation of multi-versioned feature seem to go in the
direction of the community.

>> ? ?- distribute.index: that's package_index and a few other things. everything required to interact with PyPI. We will promote
>> ? ? ?its usage and see if Pip wants to use it as a basis.
>
> This is a little tricky. ?Primarily because there's a fair amount of
> logic involved in the indexing (going around to different URLs,
> parsing links, finding stuff). ?So long as there is logic, something
> can go wrong -- often not in the package itself, but simple user error
> (e.g., it doesn't look where the user thinks it should, or a link is
> malformed, etc). ?Because of this, and as a general design goal of
> pip, I want to show as much as I can about what it is doing and why.
> This is primarily tied into pip's logging system (which is oriented
> towards command-line output, and isn't the standard logging system).
> Also it tracks *why* it got to a certain links. ?These are the two
> things I can think of where the index code in pip is tied to pip, and
> why it would be hard to use an external system.

OK. Maybe this particular package could be used by another tool
that needs to work with PyPI. It will also include a set of APIs that
corresponds to PyPI XMLPRC services.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From ziade.tarek at gmail.com  Fri Oct  9 12:06:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 12:06:44 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACEF266.3020309@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>
	<4ACB3FC0.1010906@simplistix.co.uk> <4ACB439E.6050102@egenix.com>
	<4ACB45FA.3080707@simplistix.co.uk> <4ACB5690.1080208@egenix.com>
	<4ACB6619.4080807@simplistix.co.uk>
	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>
	<4ACD965D.5080701@egenix.com>
	<30910449ef0aad44ac8d419c4e36896f@preisshare.net>
	<4ACEF266.3020309@egenix.com>
Message-ID: <94bdd2610910090306o63daf0e9w970687a508bc5d55@mail.gmail.com>

On Fri, Oct 9, 2009 at 10:20 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>
> I know that the issue tracker for PyPI is (still) on SF, but
> development appear to happen elsewhere and I can't find any
> contact information on the PyPI web pages.

Everything is provided here:

http://wiki.python.org/moin/CheeseShopDev

>
> Is Richard Jones still working on it ? They I could contact him
> directly to hash this out.

I think Martin vL is the most active developer at this time.

Regards
Tarek

From mal at egenix.com  Fri Oct  9 12:12:44 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 09 Oct 2009 12:12:44 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <94bdd2610910090306o63daf0e9w970687a508bc5d55@mail.gmail.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACB3FC0.1010906@simplistix.co.uk>
	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>
	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>	<4ACD965D.5080701@egenix.com>	<30910449ef0aad44ac8d419c4e36896f@preisshare.net>	<4ACEF266.3020309@egenix.com>
	<94bdd2610910090306o63daf0e9w970687a508bc5d55@mail.gmail.com>
Message-ID: <4ACF0C9C.8030702@egenix.com>

Tarek Ziad? wrote:
> On Fri, Oct 9, 2009 at 10:20 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>> I know that the issue tracker for PyPI is (still) on SF, but
>> development appear to happen elsewhere and I can't find any
>> contact information on the PyPI web pages.
> 
> Everything is provided here:
> 
> http://wiki.python.org/moin/CheeseShopDev

Thanks for the URL.

>>
>> Is Richard Jones still working on it ? They I could contact him
>> directly to hash this out.
> 
> I think Martin vL is the most active developer at this time.

Ok, then I'll coordinate with Martin.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 09 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Fri Oct  9 12:43:50 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 9 Oct 2009 10:43:50 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Distutils_and_Distribute_roadmap_=28and_so?=
	=?utf-8?q?me_words=09on_Virtualenv=2C_Pip=29?=
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
Message-ID: <loom.20091009T124309-659@post.gmane.org>

Ian Bicking <ianb <at> colorstudy.com> writes:
> 
> Someone mentioned that easy_install provided some things pip didn't;
> outside of multi-versioned installs (which I'm not very enthusiastic
> about) I'm not sure what this is?

http://pip.openplans.org/#differences-from-easy-install

If it's obsolete the website should be updated...



From lists at cheimes.de  Fri Oct  9 13:16:37 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 09 Oct 2009 13:16:37 +0200
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACDD968.1070205@gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACDD968.1070205@gmail.com>
Message-ID: <4ACF1B95.50600@cheimes.de>

Nick Coghlan wrote:
> Christian Heimes wrote:
>> The solution requires a new attribute in the sys module that contains
>> the name of the implementation. As an alternative we could use the first
>> field of sys.subversion but I prefer an explicit attribute. I'm
>> proposing "sys.name" with a value of "CPython", "IronPython", "Jython"
>> or "PyPy".
> 
> My Google skills almost failed me, but searching for "sys.vm" found me
> what I was after: http://bugs.python.org/issue4242 (a discussion
> relating to a similar need in the context of flagging implementation
> specific tests).

Thanks, I knew about the discussion but I didn't know that somebody has
already suggested a sys.vm attribute.

> As mentioned in that discussion, as of Python 2.6, you can do the following:
>>>> import platform
>>>> platform.python_implementation()
> 'CPython'
> 
> (Although according to the function docstring, PyPy is currently missing
> from the list of known implementations)
> 
> Importing yet-another-module for use in site.py doesn't sound like a
> great idea, so it may make sense to migrate that information into the
> sys module is this approach is taken. "sys.name" is a little generic
> though - something more explicit like "sys.vm" would be better.

The platform modules uses additional modules like the re module. On my
system "import platform" loads 7 additional module. The platform modul
also uses multiple function calls and regular expression to detect the
vm. I'm not going to blame the author of the function for the way the
feature is implemented. It's the best way and I'd probably done it the
same way, too. But it's an unnecessary slow down of every interpreter
startup. 'sys.vm' it is.

> Implementation specific user directories sound like a good idea indeed.
> 
> An alternative to a lookup table approach might be to be a little more
> direct and just retrieve the final part of the user specific directory
> name directly from a new function in the sys module. Then different VM
> authors can choose any directory name they want without CPython's
> site.py needing to know anything else about them.


I like your idea. It might be a better idea to turn sys.vm into a struct
like sys.float_info. The struct can contains a set of required
attributes like an id, a capitalized name, the platform (C, .NET, Java)
and the suffix for the user site directory.

>>> sys.vm
sys.vm(id='cpython', name='CPython', platform='C',
usersitesuffix='python26') # on win32: Python2.6

>>> sys.vm
sys.vm(id='ironpython', name='IronPython', platform='.NET',
usersitesuffix='ironpython26) # on win32: IronPython2.6

Christian

From solipsis at pitrou.net  Fri Oct  9 13:23:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 9 Oct 2009 11:23:26 +0000 (UTC)
Subject: [Python-Dev] PEP 370 and IronPython
References: <4ACDCA5A.4090609@cheimes.de> <4ACDD968.1070205@gmail.com>
	<4ACF1B95.50600@cheimes.de>
Message-ID: <loom.20091009T132233-741@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> >>> sys.vm
> sys.vm(id='cpython', name='CPython', platform='C',
> usersitesuffix='python26') # on win32: Python2.6
> 
> >>> sys.vm
> sys.vm(id='ironpython', name='IronPython', platform='.NET',
> usersitesuffix='ironpython26) # on win32: IronPython2.6

`usersitesuffix` should probably be a separate sys attribute, since it doesn't
depend on the VM only, but also on the platform and version.

Regards

Antoine.



From lists at cheimes.de  Fri Oct  9 13:27:28 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 09 Oct 2009 13:27:28 +0200
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <94bdd2610910080616w27b43020sa52670269e01b7ff@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de>
	<94bdd2610910080616w27b43020sa52670269e01b7ff@mail.gmail.com>
Message-ID: <4ACF1E20.8060809@cheimes.de>

Tarek Ziad? wrote:
> I have a suggestion though, that completes Nick's answer.
> 
> distutils/command/install.py already contains install schemes and
> imports USER_SITE and USER_BASE to builds user-specific install
> schemes. I think it wouldn't hurt for clarity to reunite all these
> schemes in a single place in the stdlib (so in sys maybe?)

Hello Tarek!

I'm feeling uncomfortable with sticking so much information into the sys
module. Another module with installation schemata wouldn't hurt, though.
In my opinion it's usually wise to separate configuration from logic.

Christian

PS: Thanks for your job with distribute! You are a life safer.

From lists at cheimes.de  Fri Oct  9 13:32:13 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 09 Oct 2009 13:32:13 +0200
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <loom.20091009T132233-741@post.gmane.org>
References: <4ACDCA5A.4090609@cheimes.de>
	<4ACDD968.1070205@gmail.com>	<4ACF1B95.50600@cheimes.de>
	<loom.20091009T132233-741@post.gmane.org>
Message-ID: <4ACF1F3D.7010707@cheimes.de>

Antoine Pitrou wrote:

> `usersitesuffix` should probably be a separate sys attribute, since it doesn't
> depend on the VM only, but also on the platform and version.


I don't really care and I'm happy to follow the herd in this matter. :)

Christian

From lists at cheimes.de  Fri Oct  9 13:34:46 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 09 Oct 2009 13:34:46 +0200
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACE0E99.7050102@voidspace.org.uk>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
Message-ID: <han74m$v9c$1@ger.gmane.org>

Michael Foord wrote:
> I really like this scheme. The important thing for IronPython is that we 
> can get it into Python 2.6 (along with other fixes to make distutils 
> compatible with IronPython - like not attempting to bytecode-compile 
> when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe
for a bug fix release.

Christian


From mal at egenix.com  Fri Oct  9 13:47:53 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 09 Oct 2009 13:47:53 +0200
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACF1B95.50600@cheimes.de>
References: <4ACDCA5A.4090609@cheimes.de> <4ACDD968.1070205@gmail.com>
	<4ACF1B95.50600@cheimes.de>
Message-ID: <4ACF22E9.4020000@egenix.com>

Christian Heimes wrote:
> Nick Coghlan wrote:
>> Importing yet-another-module for use in site.py doesn't sound like a
>> great idea, so it may make sense to migrate that information into the
>> sys module is this approach is taken. "sys.name" is a little generic
>> though - something more explicit like "sys.vm" would be better.
> 
> The platform modules uses additional modules like the re module. On my
> system "import platform" loads 7 additional module. The platform modul
> also uses multiple function calls and regular expression to detect the
> vm. I'm not going to blame the author of the function for the way the
> feature is implemented. It's the best way and I'd probably done it the
> same way, too. But it's an unnecessary slow down of every interpreter
> startup. 'sys.vm' it is.
> 
>> Implementation specific user directories sound like a good idea indeed.
>>
>> An alternative to a lookup table approach might be to be a little more
>> direct and just retrieve the final part of the user specific directory
>> name directly from a new function in the sys module. Then different VM
>> authors can choose any directory name they want without CPython's
>> site.py needing to know anything else about them.
> 
> 
> I like your idea. It might be a better idea to turn sys.vm into a struct
> like sys.float_info. The struct can contains a set of required
> attributes like an id, a capitalized name, the platform (C, .NET, Java)
> and the suffix for the user site directory.
> 
>>>> sys.vm
> sys.vm(id='cpython', name='CPython', platform='C',
> usersitesuffix='python26') # on win32: Python2.6
> 
>>>> sys.vm
> sys.vm(id='ironpython', name='IronPython', platform='.NET',
> usersitesuffix='ironpython26) # on win32: IronPython2.6

+1 on adding something like this, though, I'd use
sys.python_implementation or just sys.implementation
since not all Python implementations are virtual machines -
many compile to machine code or some intermediate format.

-1 on importing platform in site.py. Python's startup time is not
great to begin with and this would cause the load time to increase
even more.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 09 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fuzzyman at voidspace.org.uk  Fri Oct  9 13:53:35 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 09 Oct 2009 12:53:35 +0100
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <han74m$v9c$1@ger.gmane.org>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org>
Message-ID: <4ACF243F.4080408@voidspace.org.uk>

Christian Heimes wrote:
> Michael Foord wrote:
>   
>> I really like this scheme. The important thing for IronPython is that we 
>> can get it into Python 2.6 (along with other fixes to make distutils 
>> compatible with IronPython - like not attempting to bytecode-compile 
>> when sys.dont_write_bytecode is True).
>>     
>
> I don't think my proposal will land into 2.6. The changes are too severe
> for a bug fix release.
>   

Right, certainly not adding umpteen new sys attributes. :-)

The problem is that the alternative implementations run well behind 
Python-trunk, indeed it doesn't really make sense for them to put a lot 
of effort into implementing a version that is still in development. The 
result is that they discover incompatibilites after a version has gone 
into 'bugfix only' mode.

Whilst the fix you have described (add information to sys that is used 
by site.py and distutils) is ideal it can only go into 2.7. I would 
*still* like to see a fix in 2.6 - even if it is simple logic in site.py 
using sys.platform (if sys.platform == 'cli'; elif sys.platform == 
'java' etc). That way IronPython 2.6 is able to be compatible with 
Python 2.6. This logic might need duplicating in distutils (I haven't 
looked at how distutils works out where the user site-packages folder 
is), but it is a 'maintenance only' fix.

All the best,

Michael

> Christian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From fuzzyman at voidspace.org.uk  Fri Oct  9 13:53:57 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 09 Oct 2009 12:53:57 +0100
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACF22E9.4020000@egenix.com>
References: <4ACDCA5A.4090609@cheimes.de>
	<4ACDD968.1070205@gmail.com>	<4ACF1B95.50600@cheimes.de>
	<4ACF22E9.4020000@egenix.com>
Message-ID: <4ACF2455.8030507@voidspace.org.uk>

M.-A. Lemburg wrote:
> Christian Heimes wrote:
>   
>> Nick Coghlan wrote:
>>     
>>> Importing yet-another-module for use in site.py doesn't sound like a
>>> great idea, so it may make sense to migrate that information into the
>>> sys module is this approach is taken. "sys.name" is a little generic
>>> though - something more explicit like "sys.vm" would be better.
>>>       
>> The platform modules uses additional modules like the re module. On my
>> system "import platform" loads 7 additional module. The platform modul
>> also uses multiple function calls and regular expression to detect the
>> vm. I'm not going to blame the author of the function for the way the
>> feature is implemented. It's the best way and I'd probably done it the
>> same way, too. But it's an unnecessary slow down of every interpreter
>> startup. 'sys.vm' it is.
>>
>>     
>>> Implementation specific user directories sound like a good idea indeed.
>>>
>>> An alternative to a lookup table approach might be to be a little more
>>> direct and just retrieve the final part of the user specific directory
>>> name directly from a new function in the sys module. Then different VM
>>> authors can choose any directory name they want without CPython's
>>> site.py needing to know anything else about them.
>>>       
>> I like your idea. It might be a better idea to turn sys.vm into a struct
>> like sys.float_info. The struct can contains a set of required
>> attributes like an id, a capitalized name, the platform (C, .NET, Java)
>> and the suffix for the user site directory.
>>
>>     
>>>>> sys.vm
>>>>>           
>> sys.vm(id='cpython', name='CPython', platform='C',
>> usersitesuffix='python26') # on win32: Python2.6
>>
>>     
>>>>> sys.vm
>>>>>           
>> sys.vm(id='ironpython', name='IronPython', platform='.NET',
>> usersitesuffix='ironpython26) # on win32: IronPython2.6
>>     
>
> +1 on adding something like this, though, I'd use
> sys.python_implementation or just sys.implementation
> since not all Python implementations are virtual machines -
> many compile to machine code or some intermediate format.
>   

Agrred.
> -1 on importing platform in site.py. Python's startup time is not
> great to begin with and this would cause the load time to increase
> even more.
>
>   

Agrred. (And import time is even worse on IronPython.)

Michael



-- 
http://www.ironpythoninaction.com/



From p.f.moore at gmail.com  Fri Oct  9 14:32:43 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 9 Oct 2009 13:32:43 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <loom.20091009T124309-659@post.gmane.org>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
Message-ID: <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>

2009/10/9 Antoine Pitrou <solipsis at pitrou.net>:
> Ian Bicking <ianb <at> colorstudy.com> writes:
>>
>> Someone mentioned that easy_install provided some things pip didn't;
>> outside of multi-versioned installs (which I'm not very enthusiastic
>> about) I'm not sure what this is?
>
> http://pip.openplans.org/#differences-from-easy-install
>
> If it's obsolete the website should be updated...

Specifically, combine "only installs from source" with "might not work
on Windows" and the result is pretty certainly unusable for C
extensions on Windows. You can pretty much guarantee that the average
user on Windows won't have a C compiler[1], and even if they do, they
won't be able to carefully line up all the 3rd party C libraries
needed to build some extensions.

Binary packages are essential on Windows.

Paul.

[1] Heck, some extensions only build with mingw, others only build
with MSVC. You need *two* compilers :-(

From fuzzyman at voidspace.org.uk  Fri Oct  9 14:38:42 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 09 Oct 2009 13:38:42 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<nad-B71566.14284708102009@ger.gmane.org>	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>	<4ACE718D.8090400@cryptelium.net>	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
Message-ID: <4ACF2ED2.8010209@voidspace.org.uk>

Paul Moore wrote:
> 2009/10/9 Antoine Pitrou <solipsis at pitrou.net>:
>   
>> Ian Bicking <ianb <at> colorstudy.com> writes:
>>     
>>> Someone mentioned that easy_install provided some things pip didn't;
>>> outside of multi-versioned installs (which I'm not very enthusiastic
>>> about) I'm not sure what this is?
>>>       
>> http://pip.openplans.org/#differences-from-easy-install
>>
>> If it's obsolete the website should be updated...
>>     
>
> Specifically, combine "only installs from source" with "might not work
> on Windows" and the result is pretty certainly unusable for C
> extensions on Windows. You can pretty much guarantee that the average
> user on Windows won't have a C compiler[1], and even if they do, they
> won't be able to carefully line up all the 3rd party C libraries
> needed to build some extensions.
>
> Binary packages are essential on Windows.
>   
Definitely. Most Windows users won't have any compilers let alone 2.

Michael

> Paul.
>
> [1] Heck, some extensions only build with mingw, others only build
> with MSVC. You need *two* compilers :-(
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From exarkun at twistedmatrix.com  Fri Oct  9 14:52:05 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 09 Oct 2009 12:52:05 -0000
Subject: [Python-Dev] [New-bugs-announce] [issue7064] Python 2.6.3
	/	setuptools 0.6c9: extension module builds fail with KeyError
In-Reply-To: <94bdd2610910050604k5cc20cdblf24a65c307a1cfdf@mail.gmail.com>
References: <1254742115.57.0.346118025313.issue7064@psf.upfronthosting.co.za>
	<19145.60293.661555.312221@montanaro.dyndns.org>
	<94bdd2610910050604k5cc20cdblf24a65c307a1cfdf@mail.gmail.com>
Message-ID: <20091009125205.24460.527857507.divmod.xquotient.228@boson>

On 5 Oct, 01:04 pm, ziade.tarek at gmail.com wrote:
>On Mon, Oct 5, 2009 at 2:50 PM,  <skip at pobox.com> wrote:
>>
>>? ?Ned> Due to a change in distutils released with Python 2.6.3, 
>>packages
>>? ?Ned> that use setuptools (version 0.6c9, as of this writing), or 
>>the
>>? ?Ned> easy_install command, to build C extension modules fail ...
>>? ?...
>>? ?Ned> Among the packages known to be affected include lxml,
>>? ?Ned> zope-interface, jinja2, and, hence, packages dependent on 
>>these
>>? ?Ned> packages (e.g. sphinx, twisted, etc.).
>>
>>Maybe the Python test suite should include tests with a small number 
>>of
>>widely used non-core packages like setuptools. ?I realize the pybots 
>>project
>>exists to tackle this sort of stuff in greater detail. ?I'm thinking 
>>more of
>>a smoke test than a comprehensive test suite covering all external 
>>packages.
>>Setuptools is particularly important because so many extension authors 
>>use
>>it. ?If it breaks it implicitly breaks a lot of PyPI packages.
>
>I have created 6 months ago such a buildbot that downloads tarballs
>from the community,
>and run a few distutils commands on them, and make sure the result is
>similar in 2.6/2.7.
>and for "sdist" that the resulting tarball is similar.
>
>It was running over Twisted and Numpy, but has been discontinued 
>because
>it was on my own server, where it was hard to keep it up 
>(cpu/bandwidth)
>
>If the Snakebite project could host my buildbot (or at least slaves)
>or if the PSF could pay for a dedicated
>server for this, we would be able to trigger such warnings, and
>provide an e-mail service to package maintainers for example.
>
>The build could occur everytime Distutils *or* the project changes.

If you want, until Snakebite is up, I can probably provide a slave which 
can at least do this testing for Twisted and perhaps some other 
projects.

Jean-Paul

From chris at simplistix.co.uk  Fri Oct  9 17:01:36 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 16:01:36 +0100
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
Message-ID: <4ACF5050.1060701@simplistix.co.uk>

Guido van Rossum wrote:
> I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
> Distribute is mature and in widespread use. IOW I support Tarek's fork
> and suggest nobody hold their breath waiting for setuptools 0.7.

Well, if this was the BDFL pronouncement that a lot of people have taken 
it to be, does that allow Martin von Lewis to give the keys to 
http://pypi.python.org/pypi/setuptools to the "distribute" developers, 
so we can get on and use the original "setuptools" name without all the 
confusion and hackery needed to make "distribute" work?

cheers,

Chris

PS: I had thought that MvL couldn't help because of ez_setup.py, but 
it's only buildout that appears foolish enough to have a hardcoded 
download of:

exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
                      ).read() in ez

..and even that will keep working provided PJE doesn't take it down...

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Fri Oct  9 17:23:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 17:23:31 +0200
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <4ACF5050.1060701@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<4ACF5050.1060701@simplistix.co.uk>
Message-ID: <94bdd2610910090823x77df06cdm5ed484c30feccced@mail.gmail.com>

On Fri, Oct 9, 2009 at 5:01 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Guido van Rossum wrote:
>>
>> I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
>> Distribute is mature and in widespread use. IOW I support Tarek's fork
>> and suggest nobody hold their breath waiting for setuptools 0.7.
>
> Well, if this was the BDFL pronouncement that a lot of people have taken it
> to be, does that allow Martin von Lewis to give the keys to
> http://pypi.python.org/pypi/setuptools to the "distribute" developers, so we
> can get on and use the original "setuptools" name without all the confusion
> and hackery needed to make "distribute" work?
>

I am -1 on this.

At Python and PyPI level, 'Setuptools' is PJE's project and he's the
one that manages the list of users that have
extended rights on his project.

In Distribute, our setup.py script makes sure setuptools is not in the
way. It is explicit and documented,
and there's a documentation explaining how to switch back to Setuptools.

This exists only because there's no way yet in Python to say that a
distribution can't run when another distribution is installed.

But it's up to every project and developer to choose to switch to
Distribute. And it's up to OS packagers to choose
the best strategy at their levels when they package tools for their
packaging system (so they pick one or the other
and they are able to make a distinction between PJE python
distribution and ours)

Distribute developers will of course promote Distutils usage
everywhere, but hijacking setuptools in PyPI
would be a very bad thing imho.

Tarek

From chris at simplistix.co.uk  Fri Oct  9 17:32:05 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 16:32:05 +0100
Subject: [Python-Dev]  no consensus on static metadata
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <4ACF5775.7080507@simplistix.co.uk>

Tarek Ziad? wrote:
> == The fate of setup.py, and static metadata ==
> 
> So we are going to separate the metadata description from setup.py, in
> a static configuration file, that can be open and read by anyone
> without
> running any code.

<snip>

> So we've worked on that lately in Distutils-SIG and came up with a
> micro-language, based on a ConfigParser file, that allows
> writing metadata fields that depends on sys.platform etc. I won't
> detail the syntax here but the idea is that the interpretation
> of this file can be done with a vanilla Python without running arbitrary code.

I don't understand how the above two paragraphs are compatible.

> So I am adding this in Distutils for 2.7.

NB: There was no consensus on this "micro-language" on distutils-sig.
While I suspect I don't care as none of my packages rely on anything 
other than other python packages, others did care, and I found the 
syntax Tarek was proposing pretty clumsy.

Does the PEP386 or the changes you're proposing for PEP314 cater for the 
extra_requires stuff in setuptools? I know I use that quite heavily and 
I'm not the only one...

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From solipsis at pitrou.net  Fri Oct  9 17:33:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 9 Oct 2009 15:33:35 +0000 (UTC)
Subject: [Python-Dev] BDFL pronouncement?
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<4ACF5050.1060701@simplistix.co.uk>
Message-ID: <loom.20091009T172719-354@post.gmane.org>

Chris Withers <chris <at> simplistix.co.uk> writes:
> 
> Well, if this was the BDFL pronouncement that a lot of people have taken 
> it to be, does that allow Martin von Lewis to give the keys to 
> http://pypi.python.org/pypi/setuptools to the "distribute" developers, 
> so we can get on and use the original "setuptools" name without all the 
> confusion and hackery needed to make "distribute" work?

Er, perhaps this is a bit brutal? I don't think the BDFL has a right of life and
death over third-party packages.

I mean, we're an anarcho-syndicalist commune, aren't we?

  WOMAN:  Who are the Britons?
  ARTHUR:  Well, we all are. we're all Britons and I am your king.
  WOMAN:  I didn't know we had a king.  I thought we were an autonomous
      collective.
  DENNIS:  You're fooling yourself.  We're living in a dictatorship.
      A self-perpetuating autocracy in which the working classes--
  [snip]


Regards

Antoine.


PS :

> PS: I had thought that MvL couldn't help because of ez_setup.py, but 
> it's only buildout that appears foolish enough to have a hardcoded 
> download of:

Some projects also have an SVN "externals" pointing to the setuptools SVN
repository, as described here:
http://peak.telecommunity.com/doc/ez_setup/index.html



From chris at simplistix.co.uk  Fri Oct  9 17:37:56 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 16:37:56 +0100
Subject: [Python-Dev] Distutils and Distribute "upload_docs" command
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <4ACF58D4.70205@simplistix.co.uk>

Tarek Ziad? wrote:
> Some new commands are added there, when they are helpful and don't
> interact with the rest. I am thinking
> about "upload_docs" that let you upload documentation to PyPI. The
> goal is to move it to Distutils
> at some point, if the documentation feature of PyPI stays and starts to be used.

This is already available:

http://pypi.python.org/pypi/Sphinx-PyPI-upload

How about helping the author of that extension make it work with 
Distribute or just get that implementation into Distutils core instead?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Fri Oct  9 17:42:04 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 16:42:04 +0100
Subject: [Python-Dev] Bits-of-Distribute naming
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <4ACF59CC.5050504@simplistix.co.uk>

Tarek Ziad? wrote:
> - The code is splitted in many packages and might be distributed under
> several distributions.
> 
>    - distribute.resources: that's the old pkg_resources, but
> reorganized in clean, pep-8 modules. This package will
>      only contain the query APIs and will focus on being PEP 376
> compatible. We will promote its usage and see if Pip wants
>      to use it as a basis. And maybe PyPM once it's open source ?
> (<hint> <hint>).
> 	It will probably shrink a lot though, once the stdlib provides PEP 376 support.

Why not just call it pkg_resources and/or merge it with pkgutil to get 
it into the python stdlib?

>    - distribute.entrypoints: that's the old pkg_resources entry points
> system, but on its own. it uses distribute.resources

Why not get it into the core as distutils.entrypoints? That's where it 
belongs...

>    - distribute.index: that's package_index and a few other things.
> everything required to interact with PyPI. We will promote
>      its usage and see if Pip wants to use it as a basis.

Why not call in "pypi.client" and "pypi.server" or, better yet, get it 
into the stdlib as disutils.index.client distutils.index.server?

>    - distribute.core (might be renamed to main): that's everything
> else, and uses the other packages.

...which would be left, and could just be called "distribute".

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Fri Oct  9 17:45:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 17:45:31 +0200
Subject: [Python-Dev] no consensus on static metadata
In-Reply-To: <4ACF5775.7080507@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF5775.7080507@simplistix.co.uk>
Message-ID: <94bdd2610910090845k56f89f7fk2fe0a834f3172a99@mail.gmail.com>

On Fri, Oct 9, 2009 at 5:32 PM, Chris Withers <chris at simplistix.co.uk> wrote:
>
>> So I am adding this in Distutils for 2.7.
>
> NB: There was no consensus on this "micro-language" on distutils-sig.
> While I suspect I don't care as none of my packages rely on anything other
> than other python packages, others did care, and I found the syntax Tarek
> was proposing pretty clumsy.

Please no bikeshedding again on the syntax. "consensus" doesn't
mean that you agreed on the syntax.

As I said to MAL, a PEP will be written with the precise syntax, and you will
be able to make your voice heard on **Distutils-SIG**, even participate to it.

But at the end, a syntax will be added.

> Does the PEP386 or the changes you're proposing for PEP314 cater for the
> extra_requires stuff in setuptools? I know I use that quite heavily and I'm
> not the only one...

Let's open a thread on Distutils-SIG about PEP 314 planned changes.

Tarek.

From chris at simplistix.co.uk  Fri Oct  9 17:51:00 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 16:51:00 +0100
Subject: [Python-Dev] supporting multiple versions of one package in one
 environment is insane
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <4ACF5BE4.70907@simplistix.co.uk>

Tarek Ziad? wrote:
> = Virtualenv and the multiple version support in Distribute =
> 
> (I am not saying "We" here because this part was not discussed yet
> with everyone)

Good, so maybe take this discussion to distutils-sig first?

> Virtualenv allows you to create an isolated environment to install
> some distribution without polluting the
> main site-packages, a bit like a user site-packages.

...as does buildout, and these are the right type of solution to this 
problem.

> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.

No, it exists because per-project specification of requirements is a 
really good thing. Having multiple versions of the same package in the 
same python environment it totally batshit crazy...

> Setuptools provided a multiple version support but I don't like its
> implementation and the way its works.

See the "crazy" comment above...

> - a special import system using importlib that would automatically
> pick the latest version, thanks to PEP 376.

As long as I never have to use it or see it, I don't really care, but 
this seems like creating more work for yourself and another gun pointed 
at the face, nevermind the foot, of the unwary newbie...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Fri Oct  9 17:56:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 17:56:31 +0200
Subject: [Python-Dev] Distutils and Distribute "upload_docs" command
In-Reply-To: <4ACF58D4.70205@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF58D4.70205@simplistix.co.uk>
Message-ID: <94bdd2610910090856x17808758g2896b80cc0024b9f@mail.gmail.com>

On Fri, Oct 9, 2009 at 5:37 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Some new commands are added there, when they are helpful and don't
>> interact with the rest. I am thinking
>> about "upload_docs" that let you upload documentation to PyPI. The
>> goal is to move it to Distutils
>> at some point, if the documentation feature of PyPI stays and starts to be
>> used.
>
> This is already available:
>
> http://pypi.python.org/pypi/Sphinx-PyPI-upload
>
> How about helping the author of that extension make it work with Distribute
> or just get that implementation into Distutils core instead?

It's the case. The author is a Distribute commiter and *he* has added
it in Distribute.

PS: its' weird, it looks like your answers to the original thread are
not belonging to
the thread anymore. Is that my gmail or something on your mail client.

Tarek

From chris at simplistix.co.uk  Fri Oct  9 18:03:41 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 17:03:41 +0100
Subject: [Python-Dev] Distutils and Distribute "upload_docs" command
In-Reply-To: <94bdd2610910090856x17808758g2896b80cc0024b9f@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<4ACF58D4.70205@simplistix.co.uk>
	<94bdd2610910090856x17808758g2896b80cc0024b9f@mail.gmail.com>
Message-ID: <4ACF5EDD.1030707@simplistix.co.uk>

Tarek Ziad? wrote:
>> How about helping the author of that extension make it work with Distribute
>> or just get that implementation into Distutils core instead?
> 
> It's the case. The author is a Distribute commiter and *he* has added
> it in Distribute.

Great, but about just getting it into Distutils? I can't imagine it's 
that contentious...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Fri Oct  9 18:05:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 18:05:50 +0200
Subject: [Python-Dev] Bits-of-Distribute naming
In-Reply-To: <4ACF59CC.5050504@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF59CC.5050504@simplistix.co.uk>
Message-ID: <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com>

On Fri, Oct 9, 2009 at 5:42 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> - The code is splitted in many packages and might be distributed under
>> several distributions.
>>
>> ? - distribute.resources: that's the old pkg_resources...

> Why not just call it pkg_resources and/or merge it with pkgutil to get it
> into the python stdlib?

Because "pkg_resources" belongs to 0.6 and we want 0.7 to be
installable alongside 0.6 if needed.

Now as I said earlier, this package might shrink once PEP 376 is
finished, and have bits included in pkgutil.

>
>> ? - distribute.entrypoints: that's the old pkg_resources entry points
>> system, but on its own. it uses distribute.resources
>
> Why not get it into the core as distutils.entrypoints? That's where it
> belongs...

What do you call 'core' ? distutils.core ?

If yes, why does it belong to core ? entrypoint is a standalon plugin system .

>
>> ? - distribute.index: that's package_index and a few other things.
>> everything required to interact with PyPI. We will promote
>> ? ? its usage and see if Pip wants to use it as a basis.
>
> Why not call in "pypi.client" and "pypi.server" or, better yet, get it into
> the stdlib as disutils.index.client distutils.index.server?

We are keeping a "distribute." namespace for all bits.

Now maybe one day it'll make it into Distutils, yes. But not now.

Also, have a look at PEP 381 (mirroring infratructure for PyPI)

In any case, the splitting is occuring *now* and is not written in the stone,
so we can talk about this in Distutils-SIG, and improve it with your feedback.


Regards,
Tarek

From status at bugs.python.org  Fri Oct  9 18:07:11 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  9 Oct 2009 18:07:11 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091009160711.6370E7865B@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/02/09 - 10/09/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2444 open (+44) / 16471 closed (+14) / 18915 total (+58)

Open issues with patches:   976

Average duration of open issues: 670 days.
Median duration of open issues: 420 days.

Open Issues Breakdown
   open  2410 (+44)
pending    33 ( +0)

Issues Created Or Reopened (59)
_______________________________

traceback.format_exception_only does not return SyntaxError caro 10/07/09
       http://bugs.python.org/issue6962    reopened thewtex                       
       patch                                                                   

Doc/reference/datamodel: Slots description needs update          10/02/09
       http://bugs.python.org/issue7036    created  Byron                         
       patch                                                                   

test_asynchat fails on os x 10.6                                 10/02/09
       http://bugs.python.org/issue7037    created  chuck                         
                                                                               

test_curses fails on os x 10.6                                   10/02/09
       http://bugs.python.org/issue7038    created  chuck                         
                                                                               

test_distutils fails on os x 10.6                                10/02/09
CLOSED http://bugs.python.org/issue7039    created  chuck                         
                                                                               

test_smtplib fails on os x 10.6                                  10/02/09
       http://bugs.python.org/issue7040    created  chuck                         
                                                                               

test_macostools fails on os x 10.6                               10/02/09
       http://bugs.python.org/issue7041    created  chuck                         
                                                                               

test_signal fails on os x 10.6                                   10/02/09
       http://bugs.python.org/issue7042    created  chuck                         
       patch                                                                   

test_urllib: constructLocalFileUrl returns invalid URLs on Windo 10/03/09
       http://bugs.python.org/issue7043    created  srid                          
                                                                               

urllib.urlopen crashes when used on Mac OS 10.6.1 through a prox 10/03/09
       http://bugs.python.org/issue7044    created  jweber                        
                                                                               

utf-8 encoding error                                             10/03/09
       http://bugs.python.org/issue7045    created  ArcRiley                      
                                                                               

decimal.py: use DivisionImpossible and DivisionUndefined         10/03/09
       http://bugs.python.org/issue7046    created  skrah                         
                                                                               

decimal.py: NaN payload conversion                               10/03/09
CLOSED http://bugs.python.org/issue7047    created  skrah                         
                                                                               

decimal.py: logb: round the result if it is greater than prec    10/03/09
       http://bugs.python.org/issue7048    created  skrah                         
       patch                                                                   

decimal.py: NaN result in pow(x, y, z) with prec 1               10/03/09
       http://bugs.python.org/issue7049    created  skrah                         
       patch                                                                   

Augmented assignment of tuple crashes IDLE                       10/03/09
CLOSED http://bugs.python.org/issue7050    created  mfitz                         
                                                                               

'g'/'G' format docs need a little more explanation               10/03/09
       http://bugs.python.org/issue7051    created  benjamin.peterson             
                                                                               

"from logging import *" causes an error under Ubuntu Karmic      10/04/09
CLOSED http://bugs.python.org/issue7052    created  vak                           
                                                                               

wrong overload of slot wrapper                                   10/04/09
CLOSED http://bugs.python.org/issue7053    created  omatt                         
                                                                               

Python25.chm seems to be broken                                  10/04/09
CLOSED http://bugs.python.org/issue7054    created  Mark77                        
                                                                               

Automatic test___all__                                           10/04/09
       http://bugs.python.org/issue7055    created  pitrou                        
       patch                                                                   

regrtest runtest_inner calls findtestdir unnecessarily           10/04/09
       http://bugs.python.org/issue7056    created  r.david.murray                
       patch                                                                   

tkinter doc: more 3.x updates                                    10/04/09
       http://bugs.python.org/issue7057    created  tjreedy                       
                                                                               

Add some test execution environment protection to regrtest       10/04/09
       http://bugs.python.org/issue7058    created  r.david.murray                
       patch                                                                   

'checking getaddrinfo bug' doesn't output the result during ./co 10/05/09
       http://bugs.python.org/issue7059    created  ezio.melotti                  
                                                                               

test_multiprocessing dictionary changed size errors and hang     10/05/09
       http://bugs.python.org/issue7060    created  r.david.murray                
                                                                               

Improve 24.5. turtle doc                                         10/05/09
       http://bugs.python.org/issue7061    created  tjreedy                       
                                                                               

No docs for module 'IN'                                          10/05/09
       http://bugs.python.org/issue7062    created  RSokolov                      
                                                                               

Memory errors in array.array                                     10/05/09
       http://bugs.python.org/issue7063    created  chuck                         
                                                                               

Python 2.6.3 / setuptools 0.6c9: extension module builds fail wi 10/05/09
CLOSED http://bugs.python.org/issue7064    reopened tarek                         
                                                                               

bytes.maketrans segfaults                                        10/05/09
       http://bugs.python.org/issue7065    created  egreen                        
       patch                                                                   

archive_util.make_archive doesn't restore the cwd if an error is 10/05/09
       http://bugs.python.org/issue7066    created  ezio.melotti                  
                                                                               

Suggest better documentation of 2 versus 3 on 3.1.1 site.        10/05/09
       http://bugs.python.org/issue7067    created  srl                           
                                                                               

2.6.3 does not use specified compiler                            10/06/09
CLOSED http://bugs.python.org/issue7068    created  gotoh                         
                                                                               

inspect.isabstract to return boolean values only                 10/06/09
       http://bugs.python.org/issue7069    created  gagenellina                   
       patch                                                                   

round(x) gives wrong result for large odd integers               10/06/09
       http://bugs.python.org/issue7070    created  mark.dickinson                
       easy                                                                    

distutils and IronPython compatibility                           10/06/09
       http://bugs.python.org/issue7071    created  michael.foord                 
       26backport                                                              

isspace(0xa0) is true on Mac OS X                                10/06/09
       http://bugs.python.org/issue7072    created  naoki                         
                                                                               

Python 2.6.3 final  windows installer installs a release candida 10/06/09
CLOSED http://bugs.python.org/issue7073    created  jamartinh                     
                                                                               

Turtle module crashes python                                     10/06/09
       http://bugs.python.org/issue7074    created  tjreedy                       
                                                                               

Bug while put a set on a dict                                    10/07/09
CLOSED http://bugs.python.org/issue7075    created  chucheng                      
                                                                               

Documentation add note about SystemRandom                        10/07/09
       http://bugs.python.org/issue7076    created  sligocki                      
       patch                                                                   

SysLogHandler can't handle Unicode                               10/07/09
       http://bugs.python.org/issue7077    created  rszefler                      
                                                                               

struct help in the interpreter does not explain about the fmt op 10/07/09
       http://bugs.python.org/issue7078    created  orsenthil                     
       patch, easy                                                             

file_close() ignores return value of close_the_file              10/07/09
       http://bugs.python.org/issue7079    created  stutzbach                     
       patch                                                                   

locale.strxfrm raises MemoryError                                10/07/09
       http://bugs.python.org/issue7080    created  egreen                        
       patch                                                                   

">" formatting behaving like "=" formatting                      10/08/09
CLOSED http://bugs.python.org/issue7081    created  ncoghlan                      
                                                                               

Patch for get_filename in email.message when content-disposition 10/08/09
       http://bugs.python.org/issue7082    created  DazWorrall                    
       patch                                                                   

locals() behaviour differs when tracing is in effect             10/08/09
CLOSED http://bugs.python.org/issue7083    created  andbj                         
                                                                               

printing a list releases the GIL carelessly                      10/08/09
       http://bugs.python.org/issue7084    created  pitrou                        
                                                                               

strptime problem                                                 10/08/09
       http://bugs.python.org/issue7085    created  kakacek                       
                                                                               

logging.handlers.SysLogHandler with TCP support                  10/08/09
       http://bugs.python.org/issue7086    created  enigma                        
       patch                                                                   

use keyword 'as' in _xmlplus                                     10/09/09
CLOSED http://bugs.python.org/issue7087    created  s7v7nislands                  
                                                                               

New (in 2.6) functions in signal module are not documented as un 10/09/09
       http://bugs.python.org/issue7088    created  stutzbach                     
                                                                               

shlex behaves unexpected if newlines are not whitespace          10/09/09
       http://bugs.python.org/issue7089    created  jjdmol2                       
       patch                                                                   

encoding uncode objects greater than FFFF                        10/09/09
CLOSED http://bugs.python.org/issue7090    created  msaghaei                      
                                                                               

Distutils build ignores the --compiler command line option       10/09/09
       http://bugs.python.org/issue7091    created  jmb                           
                                                                               

Test suite emits many DeprecationWarnings when -3 is enabled     10/09/09
       http://bugs.python.org/issue7092    created  exarkun                       
                                                                               

xmlrpclib.ServerProxy() doesn't support unicode uri              10/09/09
       http://bugs.python.org/issue7093    created  haypo                         
       patch                                                                   



Issues Now Closed (30)
______________________

fromfd() and dup() for _socket on WIndows                         701 days
       http://bugs.python.org/issue1378    planders                      
       patch                                                                   

unittest.py sys.exit error                                        510 days
       http://bugs.python.org/issue2821    mfitz                         
                                                                               

fix_idioms.py generates bad code                                  418 days
       http://bugs.python.org/issue3563    benjamin.peterson             
       patch                                                                   

distutils.util.get_platform() is wrong for universal builds on m  366 days
       http://bugs.python.org/issue4064    ronaldoussoren                
       patch, needs review                                                     

make io.BufferedWriter observe max_buffer_size limits             318 days
       http://bugs.python.org/issue4428    pitrou                        
       patch, needs review                                                     

PEP 383 implementation                                            157 days
       http://bugs.python.org/issue5915    exarkun                       
       patch                                                                   

reset owner/group to root for distutils tarballs                   80 days
       http://bugs.python.org/issue6516    bheemesh                      
                                                                               

subprocess issue on Win 7 x64                                      44 days
       http://bugs.python.org/issue6773    tesla                         
                                                                               

unmarshaling of artificial strings can produce funny longs.         4 days
       http://bugs.python.org/issue7019    mark.dickinson                
       patch                                                                   

Doc update for io module                                            5 days
       http://bugs.python.org/issue7022    pakal                         
                                                                               

webbrowser : Could not open ftp server using webbrowser.open()      5 days
       http://bugs.python.org/issue7024    orsenthil                     
                                                                               

test_urllib: unsetting missing 'env' variable                       4 days
       http://bugs.python.org/issue7026    orsenthil                     
                                                                               

Add int.hex for symmetry with float.hex                             1 days
       http://bugs.python.org/issue7028    mark.dickinson                
                                                                               

Improve pybench                                                     8 days
       http://bugs.python.org/issue7029    krisvale                      
       patch                                                                   

test_distutils fails on os x 10.6                                   0 days
       http://bugs.python.org/issue7039    ned.deily                     
                                                                               

decimal.py: NaN payload conversion                                  0 days
       http://bugs.python.org/issue7047    mark.dickinson                
                                                                               

Augmented assignment of tuple crashes IDLE                          0 days
       http://bugs.python.org/issue7050    benjamin.peterson             
                                                                               

"from logging import *" causes an error under Ubuntu Karmic         1 days
       http://bugs.python.org/issue7052    srid                          
                                                                               

wrong overload of slot wrapper                                      0 days
       http://bugs.python.org/issue7053    georg.brandl                  
                                                                               

Python25.chm seems to be broken                                     0 days
       http://bugs.python.org/issue7054    Mark77                        
                                                                               

Python 2.6.3 / setuptools 0.6c9: extension module builds fail wi    1 days
       http://bugs.python.org/issue7064    schmir                        
                                                                               

2.6.3 does not use specified compiler                               0 days
       http://bugs.python.org/issue7068    tarek                         
                                                                               

Python 2.6.3 final  windows installer installs a release candida    0 days
       http://bugs.python.org/issue7073    ezio.melotti                  
                                                                               

Bug while put a set on a dict                                       0 days
       http://bugs.python.org/issue7075    benjamin.peterson             
                                                                               

">" formatting behaving like "=" formatting                         0 days
       http://bugs.python.org/issue7081    ncoghlan                      
                                                                               

locals() behaviour differs when tracing is in effect                0 days
       http://bugs.python.org/issue7083    andbj                         
                                                                               

use keyword 'as' in _xmlplus                                        0 days
       http://bugs.python.org/issue7087    benjamin.peterson             
                                                                               

encoding uncode objects greater than FFFF                           0 days
       http://bugs.python.org/issue7090    ezio.melotti                  
                                                                               

urllib doesn't correct server returned urls                      2030 days
       http://bugs.python.org/issue918368  AdamN                         
       patch                                                                   

Generate numeric/space/linebreak from Unicode database.          1098 days
       http://bugs.python.org/issue1571184 amaury.forgeotdarc            
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 16 UnicodeEncodeError - I can't even see license                    250 days
open    http://bugs.python.org/issue5127   

 13 'g'/'G' format docs need a little more explanation                 6 days
open    http://bugs.python.org/issue7051   

 13 test_signal fails on os x 10.6                                     7 days
open    http://bugs.python.org/issue7042   

 11 "from logging import *" causes an error under Ubuntu Karmic        1 days
closed  http://bugs.python.org/issue7052   

 11 decimal.py: NaN result in pow(x, y, z) with prec 1                 6 days
open    http://bugs.python.org/issue7049   

 11 decimal.py: logb: round the result if it is greater than prec      6 days
open    http://bugs.python.org/issue7048   

  9 Python 2.6.3 / setuptools 0.6c9: extension module builds fail w    1 days
closed  http://bugs.python.org/issue7064   

  9 expat parser throws Memory Error when parsing multiple files      60 days
open    http://bugs.python.org/issue6676   

  9 array.fromfile not checking I/O errors                           223 days
open    http://bugs.python.org/issue5395   

  8 utf-8 encoding error                                               7 days
open    http://bugs.python.org/issue7045   




From chris at simplistix.co.uk  Fri Oct  9 18:08:40 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 17:08:40 +0100
Subject: [Python-Dev] Bits-of-Distribute naming
In-Reply-To: <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<4ACF59CC.5050504@simplistix.co.uk>
	<94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com>
Message-ID: <4ACF6008.8000201@simplistix.co.uk>

Tarek Ziad? wrote:
>>>   - distribute.entrypoints: that's the old pkg_resources entry points
>>> system, but on its own. it uses distribute.resources
>> Why not get it into the core as distutils.entrypoints? That's where it
>> belongs...
> 
> What do you call 'core' ? distutils.core ?

Sorry, mean stdlib. distutils.entrypoints would seem to be the sensible 
place.

>>>   - distribute.index: that's package_index and a few other things.
>>> everything required to interact with PyPI. We will promote
>>>     its usage and see if Pip wants to use it as a basis.
>> Why not call in "pypi.client" and "pypi.server" or, better yet, get it into
>> the stdlib as disutils.index.client distutils.index.server?
> 
> We are keeping a "distribute." namespace for all bits.

Why?

> In any case, the splitting is occuring *now* and is not written in the stone,
> so we can talk about this in Distutils-SIG, and improve it with your feedback.

Shame there hasn't been any discussion of this recently on distutils-sig...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Fri Oct  9 18:12:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 18:12:50 +0200
Subject: [Python-Dev] Distutils and Distribute "upload_docs" command
In-Reply-To: <4ACF5EDD.1030707@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF58D4.70205@simplistix.co.uk>
	<94bdd2610910090856x17808758g2896b80cc0024b9f@mail.gmail.com>
	<4ACF5EDD.1030707@simplistix.co.uk>
Message-ID: <94bdd2610910090912w4018e244jcaa08830b0555669@mail.gmail.com>

On Fri, Oct 9, 2009 at 6:03 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>>
>>> How about helping the author of that extension make it work with
>>> Distribute
>>> or just get that implementation into Distutils core instead?
>>
>> It's the case. The author is a Distribute commiter and *he* has added
>> it in Distribute.
>
> Great, but about just getting it into Distutils? I can't imagine it's that
> contentious...

This PyPI feature is pretty new. It's great to have the client command
on Distribute.
If its usage spreads it'll make it to Distutils.

If you want to see it included Distutils (I do too) help us promoting its usage

Tarek.

From chris at simplistix.co.uk  Fri Oct  9 18:15:08 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 17:15:08 +0100
Subject: [Python-Dev] Distutils and Distribute "upload_docs" command
In-Reply-To: <94bdd2610910090912w4018e244jcaa08830b0555669@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	
	<4ACF58D4.70205@simplistix.co.uk>	
	<94bdd2610910090856x17808758g2896b80cc0024b9f@mail.gmail.com>	
	<4ACF5EDD.1030707@simplistix.co.uk>
	<94bdd2610910090912w4018e244jcaa08830b0555669@mail.gmail.com>
Message-ID: <4ACF618C.7000202@simplistix.co.uk>

Tarek Ziad? wrote:
> This PyPI feature is pretty new. It's great to have the client command
> on Distribute.
> If its usage spreads it'll make it to Distutils.
> 
> If you want to see it included Distutils (I do too) help us promoting its usage

The only thing that discourages me from using the PyPI docs feature is 
that it only allows documentation per-distribution, not 
per-version-per-distribution.

While you can work around this by building docs for all versions and 
putting them in one .zip file, that seems like a hack.

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Fri Oct  9 18:14:54 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Oct 2009 18:14:54 +0200
Subject: [Python-Dev] Bits-of-Distribute naming
In-Reply-To: <4ACF6008.8000201@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF59CC.5050504@simplistix.co.uk>
	<94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com>
	<4ACF6008.8000201@simplistix.co.uk>
Message-ID: <94bdd2610910090914v3eb66ea8p8c954841260edc80@mail.gmail.com>

On Fri, Oct 9, 2009 at 6:08 PM, Chris Withers <chris at simplistix.co.uk> wrote:
>
> Shame there hasn't been any discussion of this recently on distutils-sig...

We had many discussion on this already. I don't mind to have another
round, on the contrary,
but let's do it in Distutils-SIG

We are making a lot of noise now in Python-dev, which is not required

Tarek

From chris at simplistix.co.uk  Fri Oct  9 18:17:12 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 17:17:12 +0100
Subject: [Python-Dev] no consensus on static metadata
In-Reply-To: <94bdd2610910090845k56f89f7fk2fe0a834f3172a99@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	
	<4ACF5775.7080507@simplistix.co.uk>
	<94bdd2610910090845k56f89f7fk2fe0a834f3172a99@mail.gmail.com>
Message-ID: <4ACF6208.4030506@simplistix.co.uk>

Tarek Ziad? wrote:
>> NB: There was no consensus on this "micro-language" on distutils-sig.
>> While I suspect I don't care as none of my packages rely on anything other
>> than other python packages, others did care, and I found the syntax Tarek
>> was proposing pretty clumsy.
> 
> Please no bikeshedding again on the syntax. 

Tarek, throwing "bikeshedding" at an important syntactical discussion 
like this is verging on rude. Please stop it and accept that the best 
solution has not been found yet.

> As I said to MAL, a PEP will be written with the precise syntax, and you will
> be able to make your voice heard on **Distutils-SIG**, even participate to it.

Great.

>> Does the PEP386 or the changes you're proposing for PEP314 cater for the
>> extra_requires stuff in setuptools? I know I use that quite heavily and I'm
>> not the only one...
> 
> Let's open a thread on Distutils-SIG about PEP 314 planned changes.

OK, look forward to reading it and commenting.

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ianb at colorstudy.com  Fri Oct  9 18:23:29 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 9 Oct 2009 11:23:29 -0500
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACEFA3F.2020706@cryptelium.net>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> 
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> 
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> 
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com> 
	<4ACEFA3F.2020706@cryptelium.net>
Message-ID: <b654cd2e0910090923l20bc4c1dpc63c976b7fcbb55a@mail.gmail.com>

On Fri, Oct 9, 2009 at 3:54 AM, kiorky <kiorky at cryptelium.net> wrote:
>> If I had my way, buildout would use virtualenv and throw away its
>> funny script generation. ?If virtualenv had existed before buildout
>
> Which one, the one provided to generate scripts from entry points with the *.egg
> recipes or the bin/buildout auto regeneration?

Well, if multi-versioned installs were deprecated, it would not be
necessary to use Setuptools' style of script generation.  Instead you
could simply dereference the entry point, calling the underlying
function directly in the script.  This detail is probably more of a
distutils-sig question, and I don't have a strong opinion.

But I was thinking specifically of the egg activation buildout puts at
the top of scripts.

>> began development, probably things would have gone this way. ?I think
>> it would make the environment more pleasant for buildout users. ?Also
>
> * I don't think so, buildout is the only tool atm that permit to have really
> reproducible and isolated environments. Even, if you use the pip freezing
> machinery, it is not equivalent to buildout, Control!

I believe that to fully insulate buildout you need still virtualenv
--no-site-packages.  But I'm not arguing that virtualenv/pip makes
buildout obsolete, only that they have overlapping functionality, and
I think buildout would benefit from making use of that overlap.

> * Buildout can have single part to construct required eggs, at a specific
> version and let you control that. Pip will just search for this version, see
> that it's not available and fail. You have even recipes (like
> minitage.recipe.egg that permit to construct eggs with special version when you
> apply patches onto, thus, you can have the same egg in different flavors in the
> same eggs cache available for different projects. Those projects will just have
> to pin the right version to use, Control!.

In my own work I use multiple virtualenv environments for this use
case, to similar effect.  pip of course is not a generalized build
tool, but then minitage.recipe.egg is not the main egg installer
either.

> * Another thing is the "funny script generation", you have not one global
> site-packages for your project, but one global cache. But from this global
> cache, your scripts will only have available the eggs you declared, see Control!
> * Moreover buildout is not only a python packages manager, it's some of its
> recipes that permit to use it as. Buildout is just a great deployment tool that
> allow to script and manage your project in a "funny" and "flexible" way, Control!

Sure; I'm just advocating that buildout more explicitly use some of
the functionality of virtualenv/pip (which may require some more
features in those tools, but I'm open to that).  But specific
discussion of this would probably be more appropriate on
distutils-sig.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker

From chris at simplistix.co.uk  Fri Oct  9 18:25:18 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 17:25:18 +0100
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <94bdd2610910090823x77df06cdm5ed484c30feccced@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	
	<4AC371D7.9000904@simplistix.co.uk>	
	<4ACB49FB.3000407@simplistix.co.uk>	
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>	
	<4ACF5050.1060701@simplistix.co.uk>
	<94bdd2610910090823x77df06cdm5ed484c30feccced@mail.gmail.com>
Message-ID: <4ACF63EE.9040001@simplistix.co.uk>

Tarek Ziad? wrote:
> At Python and PyPI level, 'Setuptools' is PJE's project and he's the
> one that manages the list of users that have
> extended rights on his project.

Indeed, if only he would see sense :-(

> In Distribute, our setup.py script makes sure setuptools is not in the
> way. It is explicit and documented,
> and there's a documentation explaining how to switch back to Setuptools.

It's a hack though...

> This exists only because there's no way yet in Python to say that a
> distribution can't run when another distribution is installed.

I don't this should ever be needed, except in this edge case caused by 
one man's stubbornness...

> Distribute developers will of course promote Distutils usage
> everywhere, but hijacking setuptools in PyPI
> would be a very bad thing imho.

*sigh* I don't see it as hijacking, provided Guido is making a BDFL 
pronouncement that you're maintaining this software and PJE finally sees 
  sense and accepts that.

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From a.badger at gmail.com  Fri Oct  9 18:22:26 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Fri, 9 Oct 2009 09:22:26 -0700
Subject: [Python-Dev] supporting multiple versions of one package in	one
	environment is insane
In-Reply-To: <4ACF5BE4.70907@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF5BE4.70907@simplistix.co.uk>
Message-ID: <20091009162226.GA5033@clingman.lan>

On Fri, Oct 09, 2009 at 04:51:00PM +0100, Chris Withers wrote:
> Tarek Ziad? wrote:
>> Virtualenv allows you to create an isolated environment to install
>> some distribution without polluting the
>> main site-packages, a bit like a user site-packages.
>
> ...as does buildout, and these are the right type of solution to this  
> problem.
>
where type of problem == sandboxed environment, sure.  How do you solve the
problem for system packagers?

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/e517e443/attachment-0001.pgp>

From chris at simplistix.co.uk  Fri Oct  9 18:29:28 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Oct 2009 17:29:28 +0100
Subject: [Python-Dev] supporting multiple versions of one package in	one
 environment is insane
In-Reply-To: <20091009162226.GA5033@clingman.lan>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF5BE4.70907@simplistix.co.uk>
	<20091009162226.GA5033@clingman.lan>
Message-ID: <4ACF64E8.8010501@simplistix.co.uk>

Toshio Kuratomi wrote:
> On Fri, Oct 09, 2009 at 04:51:00PM +0100, Chris Withers wrote:
>> Tarek Ziad? wrote:
>>> Virtualenv allows you to create an isolated environment to install
>>> some distribution without polluting the
>>> main site-packages, a bit like a user site-packages.
>> ...as does buildout, and these are the right type of solution to this  
>> problem.
>>
> where type of problem == sandboxed environment, sure.  How do you solve the
> problem for system packagers?

What's to stop a system packager either just running the buildout on 
install, or running the buildout at package build time and then just 
dropping the resulting environment wherever they want to install 
applications? Such a package would only be dependent on the right python 
version at runtime...

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ianb at colorstudy.com  Fri Oct  9 18:34:10 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 9 Oct 2009 11:34:10 -0500
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> 
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> 
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> 
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com> 
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
Message-ID: <b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>

On Fri, Oct 9, 2009 at 7:32 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/10/9 Antoine Pitrou <solipsis at pitrou.net>:
>> Ian Bicking <ianb <at> colorstudy.com> writes:
>>>
>>> Someone mentioned that easy_install provided some things pip didn't;
>>> outside of multi-versioned installs (which I'm not very enthusiastic
>>> about) I'm not sure what this is?
>>
>> http://pip.openplans.org/#differences-from-easy-install
>>
>> If it's obsolete the website should be updated...
>
> Specifically, combine "only installs from source" with "might not work
> on Windows" and the result is pretty certainly unusable for C
> extensions on Windows. You can pretty much guarantee that the average
> user on Windows won't have a C compiler[1], and even if they do, they
> won't be able to carefully line up all the 3rd party C libraries
> needed to build some extensions.
>
> Binary packages are essential on Windows.

I'll admit I have some blindness when it comes to Windows.  I agree
binary installation on Windows is important.  (I don't think it's very
important on other platforms, or at least not very effective in
easy_install so it wouldn't be a regression.)

I note some other differences in that document:

> It cannot install from eggs. It only installs from source. (Maybe this will be changed sometime, but it?s low priority.)

Outside of binaries on Windows, I'm still unsure if installing eggs
serves a useful purpose.  I'm not sure if eggs are any better than
wininst binaries either...?

> It doesn?t understand Setuptools extras (like package[test]). This should be added eventually.

I haven't really seen Setuptools' extras used effectively, so I'm
unsure if it's a useful feature.  I understand the motivation for
extras, but motivated features aren't necessarily useful features.

> It is incompatible with some packages that customize distutils or setuptools in their setup.py files.

I don't have a solution for this, and generally easy_install does not
perform much better than pip in these cases.  Work in Distribute
hopefully will apply to this issue.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker

From kiorky at cryptelium.net  Fri Oct  9 18:39:11 2009
From: kiorky at cryptelium.net (kiorky)
Date: Fri, 09 Oct 2009 18:39:11 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <b654cd2e0910090923l20bc4c1dpc63c976b7fcbb55a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<4ACEFA3F.2020706@cryptelium.net>
	<b654cd2e0910090923l20bc4c1dpc63c976b7fcbb55a@mail.gmail.com>
Message-ID: <4ACF672F.9070701@cryptelium.net>

I'm crossposting to continue on distutils.

Ian Bicking a ?crit :
> On Fri, Oct 9, 2009 at 3:54 AM, kiorky <kiorky at cryptelium.net> wrote:
> Well, if multi-versioned installs were deprecated, it would not be
> necessary to use Setuptools' style of script generation.  Instead you
> could simply dereference the entry point, calling the underlying
> function directly in the script.  This detail is probably more of a
> distutils-sig question, and I don't have a strong opinion.
> 
> But I was thinking specifically of the egg activation buildout puts at
> the top of scripts.

The egg activation is just the result of the constructed working set, which
aggregates all the python packages that fulfill the options inputed by the user.
The only needed packages and nothing else or more.

>>> began development, probably things would have gone this way.  I think
>>> it would make the environment more pleasant for buildout users.  Also
>> * I don't think so, buildout is the only tool atm that permit to have really
>> reproducible and isolated environments. Even, if you use the pip freezing
>> machinery, it is not equivalent to buildout, Control!
> 
> I believe that to fully insulate buildout you need still virtualenv
> --no-site-packages.  But I'm not arguing that virtualenv/pip makes

Right, I do so, but i also drop nothing in that virtualenv site-packages.

> buildout obsolete, only that they have overlapping functionality, and
> I think buildout would benefit from making use of that overlap.

recipes have more features and are another way to do, complementary or not.
(patching, compiling options, pinning version). You have also more control on
what is installed/used/shared with buildout than with the others.

>> * Buildout can have single part to construct required eggs, at a specific
>> version and let you control that. Pip will just search for this version, see
>> that it's not available and fail. You have even recipes (like
>> minitage.recipe.egg that permit to construct eggs with special version when you
>> apply patches onto, thus, you can have the same egg in different flavors in the
>> same eggs cache available for different projects. Those projects will just have
>> to pin the right version to use, Control!.
> 
> In my own work I use multiple virtualenv environments for this use
> case, to similar effect.  pip of course is not a generalized build
> tool, but then minitage.recipe.egg is not the main egg installer
> either.

*.recipe.egg can use caches to avoid duplications among many projects
environments. You setup seems to duplicate stuff around.

>> * Another thing is the "funny script generation", you have not one global
>> site-packages for your project, but one global cache. But from this global
>> cache, your scripts will only have available the eggs you declared, see Control!
>> * Moreover buildout is not only a python packages manager, it's some of its
>> recipes that permit to use it as. Buildout is just a great deployment tool that
>> allow to script and manage your project in a "funny" and "flexible" way, Control!
> 
> Sure; I'm just advocating that buildout more explicitly use some of
> the functionality of virtualenv/pip (which may require some more
> features in those tools, but I'm open to that).  But specific
> discussion of this would probably be more appropriate on
> distutils-sig.
> 


-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/c37551e9/attachment.pgp>

From fuzzyman at voidspace.org.uk  Fri Oct  9 18:49:23 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 09 Oct 2009 17:49:23 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
Message-ID: <4ACF6993.5040704@voidspace.org.uk>

Ian Bicking wrote:
> On Fri, Oct 9, 2009 at 7:32 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>   
>> 2009/10/9 Antoine Pitrou <solipsis at pitrou.net>:
>>     
>>> Ian Bicking <ianb <at> colorstudy.com> writes:
>>>       
>>>> Someone mentioned that easy_install provided some things pip didn't;
>>>> outside of multi-versioned installs (which I'm not very enthusiastic
>>>> about) I'm not sure what this is?
>>>>         
>>> http://pip.openplans.org/#differences-from-easy-install
>>>
>>> If it's obsolete the website should be updated...
>>>       
>> Specifically, combine "only installs from source" with "might not work
>> on Windows" and the result is pretty certainly unusable for C
>> extensions on Windows. You can pretty much guarantee that the average
>> user on Windows won't have a C compiler[1], and even if they do, they
>> won't be able to carefully line up all the 3rd party C libraries
>> needed to build some extensions.
>>
>> Binary packages are essential on Windows.
>>     
>
> I'll admit I have some blindness when it comes to Windows.  I agree
> binary installation on Windows is important.  (I don't think it's very
> important on other platforms, or at least not very effective in
> easy_install so it wouldn't be a regression.)
>
> I note some other differences in that document:
>
>   
>> It cannot install from eggs. It only installs from source. (Maybe this will be changed sometime, but it?s low priority.)
>>     
>
> Outside of binaries on Windows, I'm still unsure if installing eggs
> serves a useful purpose.  I'm not sure if eggs are any better than
> wininst binaries either...?
>
>   

Many Windows users would be quite happy if the standard mechanism for 
installing non-source distributions on Windows was via the wininst binaries.

I wonder if it is going to be possible to make this compatible with the 
upcoming distutils package management 'stuff' (querying for installed 
packages, uninstallation etc) since installation/uninstallation goes 
through the Windows system package management feature.  I guess it would 
be eminently possible but require some reasonably high level Windows-fu 
to do.

Michael

>> It doesn?t understand Setuptools extras (like package[test]). This should be added eventually.
>>     
>
> I haven't really seen Setuptools' extras used effectively, so I'm
> unsure if it's a useful feature.  I understand the motivation for
> extras, but motivated features aren't necessarily useful features.
>
>   
>> It is incompatible with some packages that customize distutils or setuptools in their setup.py files.
>>     
>
> I don't have a solution for this, and generally easy_install does not
> perform much better than pip in these cases.  Work in Distribute
> hopefully will apply to this issue.
>
>   


-- 
http://www.ironpythoninaction.com/


From ianb at colorstudy.com  Fri Oct  9 19:02:07 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 9 Oct 2009 12:02:07 -0500
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACF6993.5040704@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> 
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> 
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> 
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com> 
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> 
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com> 
	<4ACF6993.5040704@voidspace.org.uk>
Message-ID: <b654cd2e0910091002g5afdf26eoa2432bcd718c2339@mail.gmail.com>

Probably all these discussions are better on distutils-sig (just
copying python-dev to note the movement of the discussion)

On Fri, Oct 9, 2009 at 11:49 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>> Outside of binaries on Windows, I'm still unsure if installing eggs
>> serves a useful purpose. ?I'm not sure if eggs are any better than
>> wininst binaries either...?
>
> Many Windows users would be quite happy if the standard mechanism for
> installing non-source distributions on Windows was via the wininst binaries.
>
> I wonder if it is going to be possible to make this compatible with the
> upcoming distutils package management 'stuff' (querying for installed
> packages, uninstallation etc) since installation/uninstallation goes through
> the Windows system package management feature. ?I guess it would be
> eminently possible but require some reasonably high level Windows-fu to do.

As far as pip works, it unpacks a package and runs "python setup.py
install (and some options that aren't that interesting, but are
provided specifically by setuptools)".  Well, it's slightly more
complicated, but more to the point it doesn't install in-process or
dictate how setup.py works, except that it takes some specific
options.  Running a Windows installer in the same way would be fine,
in that sense.  Alternately pip could unpack the wininst zip file and
install it directly; I'm not sure if that would be better or worse?
If wininst uses the central package manager of the OS then certain
features (like virtualenv, PYTHONHOME, --prefix, etc) would not be
possible.

For Distribute (or Setuptools or by association pip) to see that a
package is installed, it must have the appropriate metadata.  For
Setuptools (and Distribute 0.6) this is a directory or file, on
sys.path, "Package.egg-info" (or in Package-X.Y.egg/EGG-INFO).  If a
file, it should be a PKG-INFO file, if a directory it should contain a
PKG-INFO file.  So however the package gets installed, if that
metadata is installed then it can be queried.  I don't think querying
the Windows system package management would be necessary or desirable.
 Nobody is trying that with deb/rpm either.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker

From solipsis at pitrou.net  Fri Oct  9 19:57:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 9 Oct 2009 17:57:57 +0000 (UTC)
Subject: [Python-Dev] BDFL pronouncement?
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	
	<4AC371D7.9000904@simplistix.co.uk>	
	<4ACB49FB.3000407@simplistix.co.uk>	
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>	
	<4ACF5050.1060701@simplistix.co.uk>
	<94bdd2610910090823x77df06cdm5ed484c30feccced@mail.gmail.com>
	<4ACF63EE.9040001@simplistix.co.uk>
Message-ID: <loom.20091009T193443-928@post.gmane.org>

Chris Withers <chris <at> simplistix.co.uk> writes:
> 
> *sigh* I don't see it as hijacking, provided Guido is making a BDFL 
> pronouncement that you're maintaining this software

Well, what you are proposing *is* hijacking. PJE is not paid or employed by
Guido, he is the full author of setuptools. Forking is of course fine (this is
free software), but saying "I overthrow you as the maintainer of this software"
is not.

While you (and I) may be unsatisfied with PJE's maintenance style, personal
(dis)affection should not be what motivates our decisions. A piece of software
has an author who has some moral rights on it, and should be treated with
respect. After all, when an author decides to release code under a free license,
it shows a fair amount of trust and respect towards the user; it is only fair
that the user shows the same amount of trust and respect (*).


(*) it has happened that some users don't, and in that case the author's
reaction can be violent. For example, Nessus became proprietary:
http://news.cnet.com/Nessus-security-tool-closes-its-source/2100-7344_3-5890093.html

Regards

Antoine.



From brett at python.org  Fri Oct  9 20:00:53 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 9 Oct 2009 11:00:53 -0700
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACF243F.4080408@voidspace.org.uk>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk> 
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
Message-ID: <bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>

On Fri, Oct 9, 2009 at 04:53, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> Christian Heimes wrote:
>
>> Michael Foord wrote:
>>
>>
>>> I really like this scheme. The important thing for IronPython is that we
>>> can get it into Python 2.6 (along with other fixes to make distutils
>>> compatible with IronPython - like not attempting to bytecode-compile when
>>> sys.dont_write_bytecode is True).
>>>
>>>
>>
>> I don't think my proposal will land into 2.6. The changes are too severe
>> for a bug fix release.
>>
>>
>
> Right, certainly not adding umpteen new sys attributes. :-)
>
> The problem is that the alternative implementations run well behind
> Python-trunk, indeed it doesn't really make sense for them to put a lot of
> effort into implementing a version that is still in development. The result
> is that they discover incompatibilites after a version has gone into 'bugfix
> only' mode.
>
> Whilst the fix you have described (add information to sys that is used by
> site.py and distutils) is ideal it can only go into 2.7. I would *still*
> like to see a fix in 2.6 - even if it is simple logic in site.py using
> sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc).
> That way IronPython 2.6 is able to be compatible with Python 2.6. This logic
> might need duplicating in distutils (I haven't looked at how distutils works
> out where the user site-packages folder is), but it is a 'maintenance only'
> fix.
>

But it's still a change in semantics. Tossing this into 2.6 would mean that
anyone who has worked around the current behaviour is going to have a busted
install.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/f5353c73/attachment.htm>

From brett at python.org  Fri Oct  9 20:13:16 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 9 Oct 2009 11:13:16 -0700
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <loom.20091009T193443-928@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com> 
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com> 
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com> 
	<4ACF5050.1060701@simplistix.co.uk>
	<94bdd2610910090823x77df06cdm5ed484c30feccced@mail.gmail.com> 
	<4ACF63EE.9040001@simplistix.co.uk>
	<loom.20091009T193443-928@post.gmane.org>
Message-ID: <bbaeab100910091113n50a8aab4i4be7e45a230f1fb4@mail.gmail.com>

On Fri, Oct 9, 2009 at 10:57, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Chris Withers <chris <at> simplistix.co.uk> writes:
> >
> > *sigh* I don't see it as hijacking, provided Guido is making a BDFL
> > pronouncement that you're maintaining this software
>
> Well, what you are proposing *is* hijacking. PJE is not paid or employed by
> Guido, he is the full author of setuptools. Forking is of course fine (this
> is
> free software), but saying "I overthrow you as the maintainer of this
> software"
> is not.
>
> While you (and I) may be unsatisfied with PJE's maintenance style, personal
> (dis)affection should not be what motivates our decisions. A piece of
> software
> has an author who has some moral rights on it, and should be treated with
> respect. After all, when an author decides to release code under a free
> license,
> it shows a fair amount of trust and respect towards the user; it is only
> fair
> that the user shows the same amount of trust and respect (*).
>

FWIW I agree with Antoine.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/74d5727a/attachment.htm>

From regebro at gmail.com  Fri Oct  9 20:23:22 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Fri, 9 Oct 2009 20:23:22 +0200
Subject: [Python-Dev] Bits-of-Distribute naming
In-Reply-To: <4ACF59CC.5050504@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF59CC.5050504@simplistix.co.uk>
Message-ID: <319e029f0910091123n95002cama014f9720510100a@mail.gmail.com>

2009/10/9 Chris Withers <chris at simplistix.co.uk>:
> Why not get it into the core as distutils.entrypoints? That's where it
> belongs...

Well, one reason is that it can't be distributed separately under that
name, and hence not be tested separately or bugfixed separately.

Rule #1 for inclusion in the standard library is that the module
pretty much is dead, and that you don't need new releases or new
features. This isn't true for any part of a refactored setuptools. We
will need to distribute most of this separately for a while first.

That means that distutils.anything doesn't work.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From fuzzyman at voidspace.org.uk  Fri Oct  9 20:32:05 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 9 Oct 2009 19:32:05 +0100
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
Message-ID: <1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>

The *only* change in semantics I'm proposing is for users of  
IronPython 2.6 which is not even at final release yet. CPython users  
would be unaffected.

Sorry for top-posting, mobile device.

Michael


--
http://www.ironpythoninaction.com

On 9 Oct 2009, at 19:00, Brett Cannon <brett at python.org> wrote:

>
>
> On Fri, Oct 9, 2009 at 04:53, Michael Foord  
> <fuzzyman at voidspace.org.uk> wrote:
> Christian Heimes wrote:
> Michael Foord wrote:
>
> I really like this scheme. The important thing for IronPython is  
> that we can get it into Python 2.6 (along with other fixes to make  
> distutils compatible with IronPython - like not attempting to  
> bytecode-compile when sys.dont_write_bytecode is True).
>
>
> I don't think my proposal will land into 2.6. The changes are too  
> severe
> for a bug fix release.
>
>
> Right, certainly not adding umpteen new sys attributes. :-)
>
> The problem is that the alternative implementations run well behind  
> Python-trunk, indeed it doesn't really make sense for them to put a  
> lot of effort into implementing a version that is still in  
> development. The result is that they discover incompatibilites after  
> a version has gone into 'bugfix only' mode.
>
> Whilst the fix you have described (add information to sys that is  
> used by site.py and distutils) is ideal it can only go into 2.7. I  
> would *still* like to see a fix in 2.6 - even if it is simple logic  
> in site.py using sys.platform (if sys.platform == 'cli'; elif  
> sys.platform == 'java' etc). That way IronPython 2.6 is able to be  
> compatible with Python 2.6. This logic might need duplicating in  
> distutils (I haven't looked at how distutils works out where the  
> user site-packages folder is), but it is a 'maintenance only' fix.
>
> But it's still a change in semantics. Tossing this into 2.6 would  
> mean that anyone who has worked around the current behaviour is  
> going to have a busted install.
>
> -Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/27a472aa/attachment-0001.htm>

From brett at python.org  Fri Oct  9 20:43:06 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 9 Oct 2009 11:43:06 -0700
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk> 
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk> 
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com> 
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
Message-ID: <bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>

On Fri, Oct 9, 2009 at 11:32, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> The *only* change in semantics I'm proposing is for users of IronPython 2.6
> which is not even at final release yet. CPython users would be unaffected.
>
>
Then why can't IronPython patch site.py to do what they want? I still feel
uncomfortable changing site.py for this in a micro release.


> Sorry for top-posting, mobile device.
>
>
Aahz was the most adamant hater of top-posting and he isn't subscribed to
python-dev anymore, so whatever.

-Brett




> Michael
>
>
> --
> http://www.ironpythoninaction.com
>
> On 9 Oct 2009, at 19:00, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Fri, Oct 9, 2009 at 04:53, Michael Foord < <fuzzyman at voidspace.org.uk>
> fuzzyman at voidspace.org.uk> wrote:
>
>> Christian Heimes wrote:
>>
>>> Michael Foord wrote:
>>>
>>>
>>>> I really like this scheme. The important thing for IronPython is that we
>>>> can get it into Python 2.6 (along with other fixes to make distutils
>>>> compatible with IronPython - like not attempting to bytecode-compile when
>>>> sys.dont_write_bytecode is True).
>>>>
>>>>
>>>
>>> I don't think my proposal will land into 2.6. The changes are too severe
>>> for a bug fix release.
>>>
>>>
>>
>> Right, certainly not adding umpteen new sys attributes. :-)
>>
>> The problem is that the alternative implementations run well behind
>> Python-trunk, indeed it doesn't really make sense for them to put a lot of
>> effort into implementing a version that is still in development. The result
>> is that they discover incompatibilites after a version has gone into 'bugfix
>> only' mode.
>>
>> Whilst the fix you have described (add information to sys that is used by
>> site.py and distutils) is ideal it can only go into 2.7. I would *still*
>> like to see a fix in 2.6 - even if it is simple logic in site.py using
>> sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc).
>> That way IronPython 2.6 is able to be compatible with Python 2.6. This logic
>> might need duplicating in distutils (I haven't looked at how distutils works
>> out where the user site-packages folder is), but it is a 'maintenance only'
>> fix.
>>
>
> But it's still a change in semantics. Tossing this into 2.6 would mean that
> anyone who has worked around the current behaviour is going to have a busted
> install.
>
> -Brett
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/10eba5b7/attachment.htm>

From fwierzbicki at gmail.com  Fri Oct  9 20:48:55 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 9 Oct 2009 14:48:55 -0400
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4ACDCA5A.4090609@cheimes.de>
References: <4ACDCA5A.4090609@cheimes.de>
Message-ID: <4dab5f760910091148t33d732ccr2c529996bc3d07c7@mail.gmail.com>

On Thu, Oct 8, 2009 at 7:17 AM, Christian Heimes <lists at cheimes.de> wrote:
> CPython:
> ?~/.local/lib/python2.6/site-packages
> ?%APPDATA%/Python/Python26
>
> IronPython:
> ?~/.local/lib/ironpython2.6/site-packages
> ?%APPDATA%/Python/IronPython26
>
> Jython:
> ?~/.local/lib/jython2.6/site-packages
> ?%APPDATA%/Python/Jython26
I like this too!  Jython has yet to seriously start on 2.6, but when
we do, would it help if Jython's sys.version grew a "Jython" in a
similar place as IronPython?  It would look something like:

>>>sys.version
'2.6.0 (Jython trunk:6849M, Oct 9 2009, 14:19:04) \n[Java HotSpot(TM)
Client VM (Apple Inc.)]'

-Frank

From a.badger at gmail.com  Fri Oct  9 21:20:01 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Fri, 9 Oct 2009 12:20:01 -0700
Subject: [Python-Dev] supporting multiple versions of one package in	one
	environment is insane
In-Reply-To: <4ACF64E8.8010501@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACF5BE4.70907@simplistix.co.uk>
	<20091009162226.GA5033@clingman.lan>
	<4ACF64E8.8010501@simplistix.co.uk>
Message-ID: <20091009192001.GC5033@clingman.lan>

On Fri, Oct 09, 2009 at 05:29:28PM +0100, Chris Withers wrote:
> Toshio Kuratomi wrote:
>> On Fri, Oct 09, 2009 at 04:51:00PM +0100, Chris Withers wrote:
>>> Tarek Ziad? wrote:
>>>> Virtualenv allows you to create an isolated environment to install
>>>> some distribution without polluting the
>>>> main site-packages, a bit like a user site-packages.
>>> ...as does buildout, and these are the right type of solution to 
>>> this  problem.
>>>
>> where type of problem == sandboxed environment, sure.  How do you solve the
>> problem for system packagers?
>
> What's to stop a system packager either just running the buildout on  
> install, or running the buildout at package build time and then just  
> dropping the resulting environment wherever they want to install  
> applications? Such a package would only be dependent on the right python 
> version at runtime...
>
If buildout creates sandboxed environments like virtualenv then everything
here applies:

https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries

You can also listen/watch the talk I gave at PyCon which is linked from that
page:
http://pycon.blip.tv/file/2072580/

If it doesn't create sandboxed environments, then you'll need to give me
about a paragraph explaining what it does do.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/b02a14fa/attachment.pgp>

From g.brandl at gmx.net  Fri Oct  9 22:36:48 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 09 Oct 2009 22:36:48 +0200
Subject: [Python-Dev] no consensus on static metadata
In-Reply-To: <4ACF6208.4030506@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>		<4ACF5775.7080507@simplistix.co.uk>	<94bdd2610910090845k56f89f7fk2fe0a834f3172a99@mail.gmail.com>
	<4ACF6208.4030506@simplistix.co.uk>
Message-ID: <hao71i$btg$1@ger.gmane.org>

Chris Withers schrieb:
> Tarek Ziad? wrote:
>>> NB: There was no consensus on this "micro-language" on distutils-sig.
>>> While I suspect I don't care as none of my packages rely on anything other
>>> than other python packages, others did care, and I found the syntax Tarek
>>> was proposing pretty clumsy.
>> 
>> Please no bikeshedding again on the syntax. 
> 
> Tarek, throwing "bikeshedding" at an important syntactical discussion 
> like this is verging on rude. Please stop it and accept that the best 
> solution has not been found yet.

At some point a decision must be made, and it will be made by Tarek as the
"main packaging guy" (don't want to throw "dictator" around again).  For
syntax, there often is no such thing as "the best solution".

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Fri Oct  9 22:50:06 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 09 Oct 2009 22:50:06 +0200
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <4ACF5050.1060701@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>	<4AC371D7.9000904@simplistix.co.uk>	<4ACB49FB.3000407@simplistix.co.uk>	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>	<20091006200852.551BC3A4116@sparrow.telecommunity.com>	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<4ACF5050.1060701@simplistix.co.uk>
Message-ID: <hao7qf$btg$2@ger.gmane.org>

Chris Withers schrieb:
> Guido van Rossum wrote:
>> I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
>> Distribute is mature and in widespread use. IOW I support Tarek's fork
>> and suggest nobody hold their breath waiting for setuptools 0.7.
> 
> Well, if this was the BDFL pronouncement that a lot of people have taken 
> it to be, does that allow Martin von Lewis to give the keys to 
> http://pypi.python.org/pypi/setuptools to the "distribute" developers, 
> so we can get on and use the original "setuptools" name without all the 
> confusion and hackery needed to make "distribute" work?

That's absurd.  There's a certain area where Guido can make pronouncements,
but third-party packages is not it.  Even if they're hosted on python.org
infrastructure.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From guido at python.org  Fri Oct  9 22:56:42 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 9 Oct 2009 13:56:42 -0700
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <hao7qf$btg$2@ger.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk> 
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com> 
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com> 
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com> 
	<4ACF5050.1060701@simplistix.co.uk> <hao7qf$btg$2@ger.gmane.org>
Message-ID: <ca471dc20910091356l11e0dfbcna10184c0e5d9c098@mail.gmail.com>

On Fri, Oct 9, 2009 at 1:50 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Chris Withers schrieb:
>> Guido van Rossum wrote:
>>> I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
>>> Distribute is mature and in widespread use. IOW I support Tarek's fork
>>> and suggest nobody hold their breath waiting for setuptools 0.7.
>>
>> Well, if this was the BDFL pronouncement that a lot of people have taken
>> it to be, does that allow Martin von Lewis to give the keys to
>> http://pypi.python.org/pypi/setuptools to the "distribute" developers,
>> so we can get on and use the original "setuptools" name without all the
>> confusion and hackery needed to make "distribute" work?
>
> That's absurd. ?There's a certain area where Guido can make pronouncements,
> but third-party packages is not it. ?Even if they're hosted on python.org
> infrastructure.

Right.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ctb at msu.edu  Fri Oct  9 23:00:09 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 9 Oct 2009 14:00:09 -0700
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <ca471dc20910091356l11e0dfbcna10184c0e5d9c098@mail.gmail.com>
References: <20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk>
	<4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<4ACF5050.1060701@simplistix.co.uk> <hao7qf$btg$2@ger.gmane.org>
	<ca471dc20910091356l11e0dfbcna10184c0e5d9c098@mail.gmail.com>
Message-ID: <20091009210009.GA15674@idyll.org>

On Fri, Oct 09, 2009 at 01:56:42PM -0700, Guido van Rossum wrote:
> On Fri, Oct 9, 2009 at 1:50 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> > Chris Withers schrieb:
> >> Guido van Rossum wrote:
> >>> I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
> >>> Distribute is mature and in widespread use. IOW I support Tarek's fork
> >>> and suggest nobody hold their breath waiting for setuptools 0.7.
> >>
> >> Well, if this was the BDFL pronouncement that a lot of people have taken
> >> it to be, does that allow Martin von Lewis to give the keys to
> >> http://pypi.python.org/pypi/setuptools to the "distribute" developers,
> >> so we can get on and use the original "setuptools" name without all the
> >> confusion and hackery needed to make "distribute" work?
> >
> > That's absurd. ?There's a certain area where Guido can make pronouncements,
> > but third-party packages is not it. ?Even if they're hosted on python.org
> > infrastructure.
> 
> Right.

Is that a pronouncement?

:)

"GvR, the self-limiting BDFL."

--titus
-- 
C. Titus Brown, ctb at msu.edu

From ndbecker2 at gmail.com  Sat Oct 10 00:45:34 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 09 Oct 2009 18:45:34 -0400
Subject: [Python-Dev] Howto handle multilib conflict?
Message-ID: <haoeee$2rf$1@ger.gmane.org>

Just received:
https://bugzilla.redhat.com/show_bug.cgi?id=528237

yum install libotf-devel.i586 libotf-devel.x86_64

yields:

Transaction Check Error:
  file /usr/bin/libotf-config from install of libotf-devel-0.9.8-2.fc11.i586
conflicts with file from package libotf-devel-0.9.8-2.fc11.x86_64
  file /usr/share/doc/libotf-devel-0.9.8/example/Makefile from install of
libotf-devel-0.9.8-2.fc11.i586 conflicts with file from package
libotf-devel-0.9.8-2.fc11.x86_64

What is the recommended way to resolve this?


From ndbecker2 at gmail.com  Sat Oct 10 00:56:08 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 09 Oct 2009 18:56:08 -0400
Subject: [Python-Dev] Howto handle multilib conflict?
References: <haoeee$2rf$1@ger.gmane.org>
Message-ID: <haof28$2rf$3@ger.gmane.org>

Sorry, sent to wrong list!  Please ignore.


From skip at pobox.com  Sat Oct 10 01:07:12 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 9 Oct 2009 18:07:12 -0500
Subject: [Python-Dev] BDFL pronouncement?
In-Reply-To: <ca471dc20910091356l11e0dfbcna10184c0e5d9c098@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<4AC371D7.9000904@simplistix.co.uk> <4ACB49FB.3000407@simplistix.co.uk>
	<20091006180400.BA69D3A4116@sparrow.telecommunity.com>
	<ca471dc20910061216h4187bea6t786c781fc5304666@mail.gmail.com>
	<20091006200852.551BC3A4116@sparrow.telecommunity.com>
	<ca471dc20910061314h15f4a7f8l18ecc0a61211c420@mail.gmail.com>
	<4ACF5050.1060701@simplistix.co.uk> <hao7qf$btg$2@ger.gmane.org>
	<ca471dc20910091356l11e0dfbcna10184c0e5d9c098@mail.gmail.com>
Message-ID: <19151.49696.479121.865212@montanaro.dyndns.org>


    >> That's absurd.  There's a certain area where Guido can make
    >> pronouncements, but third-party packages is not it.  Even if they're
    >> hosted on python.org infrastructure.

    Guido> Right.

Now if you were the un-BDFL you could step in here. ;-)

This whole topic seems to have a lot of people fairly agitated, so clearly
it's important to a significant subset of the Python development community.
Might I suggest taking this off python-dev for awhile though?  I seem to
recall a similar suggestion a couple days ago to take it to distutils-sig or
python-ideas, but that seems to have been ignored.)  I mostly tuned the
entire thread out until I saw that Guido had joined in the fray (sort of).
Maybe since I don't distribute a lot of Python packages it's not as
important to me.

Let me know when you've solved the problem.  I trust you.

Skip

From lists at cheimes.de  Sat Oct 10 01:29:55 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 10 Oct 2009 01:29:55 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
Message-ID: <haoh1j$9c7$1@ger.gmane.org>

Fellow Python developers!

In the light of our recent discussion about implementation specific
information and user site directory I like to start a new PEP. Much to
my regret I wasn't able to contribute to Python core development over
the past months. I hope I can find some free time over the next weeks.

Before I start with the PEP I like to reach consensus over the goal.
Alternative implementations of our beloved programming languages are
becoming more important every day. Although IronPython, Jython and PyPy
are trying hard to get as compatible to CPython as possible there are
numerous reasons for conditional code. General Python unit tests vs.
CPython specific unit tests are one reasons, implementation specific
module search paths are another. At the moment Python has no easy way to
inspect the implementation besides some code in the platform module.

I'm proposing two new attributes in the sys module: sys.implementation
and sys.userdirsuffix.

sys.implementation
------------------

(Proposed by Nick, MAL and others.)

sys.implementation is a PyStructSequence that contains various
information about the current implementation. Some fields are required
to be present on every implementation. Implementations may choose to add
additional fields as they see fit. Some fields like compiler are useful
for introspection and are already part of sys.version. I like to include
them for the sake of completeness.

id (required):
  lower case identifier, for example "cpython", "ironpython", "jython",
"pypy"

name (required):
  mixed case name of the implementation, for example "CPython",
"IronPython", "Jython", "PyPy"

platform (required):
  platform or language of the implementation, for example "C", ".NET",
"Java"

runtime (required):
  lower case runtime library of the implementation, for example "libc",
"msvcrt90", "java6", ".net"

compiler (required):
  verbose name of the compiler, for example "GCC 4.3.3", "Java
HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"

...


sys.userdirsuffix
-----------------

sys.userdirsuffix is an implementation and platform specific string that
is used to construct the path for the user site directory as explained
in PEP 370. The string contains the implementation name as well as the
version number of Python.

Examples:
  python2.6 (CPython, Unix)
  Python26 (CPython, Windows)
  ironpython2.6 (IronPython, Unix)
  IronPython26 (IronPython, Windows)
  ...


Comments?

Christian


From dinov at microsoft.com  Sat Oct 10 01:48:04 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Fri, 9 Oct 2009 23:48:04 +0000
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>

Christian wrote:
> sys.implementation is a PyStructSequence that contains various
> information about the current implementation. Some fields are required
> to be present on every implementation. Implementations may choose to
> add
> additional fields as they see fit. Some fields like compiler are useful
> for introspection and are already part of sys.version. I like to
> include
> them for the sake of completeness.

Given that this is all about establishing some common ground between
implementations I would propose not using a CPython specific term
here such as PyStructSequence :)  The Python docs seem to use structseq
for sys.float_info.

> compiler (required):
>   verbose name of the compiler, for example "GCC 4.3.3", "Java
> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"

What's the value of this attribute?  The main reason I ask is there's
no way that I know of to determine the JIT being used in .NET.  We could
of course fill in some dummy value here for IronPython but I'm also not 
sure why anyone would use this.

Otherwise it looks good to me.


From benjamin at python.org  Sat Oct 10 01:51:33 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 9 Oct 2009 18:51:33 -0500
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>

2009/10/9 Christian Heimes <lists at cheimes.de>:
> Fellow Python developers!
>
> In the light of our recent discussion about implementation specific
> information and user site directory I like to start a new PEP. Much to
> my regret I wasn't able to contribute to Python core development over
> the past months. I hope I can find some free time over the next weeks.

Glad to have you back!

>
> Before I start with the PEP I like to reach consensus over the goal.
> Alternative implementations of our beloved programming languages are
> becoming more important every day. Although IronPython, Jython and PyPy
> are trying hard to get as compatible to CPython as possible there are
> numerous reasons for conditional code. General Python unit tests vs.
> CPython specific unit tests are one reasons, implementation specific
> module search paths are another. At the moment Python has no easy way to
> inspect the implementation besides some code in the platform module.
>
> I'm proposing two new attributes in the sys module: sys.implementation
> and sys.userdirsuffix.

Overall, +1.

>
> sys.implementation
> ------------------
>
> (Proposed by Nick, MAL and others.)
>
> sys.implementation is a PyStructSequence that contains various
> information about the current implementation. Some fields are required
> to be present on every implementation. Implementations may choose to add
> additional fields as they see fit. Some fields like compiler are useful
> for introspection and are already part of sys.version. I like to include
> them for the sake of completeness.

Can we generate sys.version from sys.implementation then?

>
> id (required):
> ?lower case identifier, for example "cpython", "ironpython", "jython",
> "pypy"
>
> name (required):
> ?mixed case name of the implementation, for example "CPython",
> "IronPython", "Jython", "PyPy"
>
> platform (required):
> ?platform or language of the implementation, for example "C", ".NET",
> "Java"
>
> runtime (required):
> ?lower case runtime library of the implementation, for example "libc",
> "msvcrt90", "java6", ".net"
>
> compiler (required):
> ?verbose name of the compiler, for example "GCC 4.3.3", "Java
> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"
>
> ...
>
>
> sys.userdirsuffix
> -----------------

Why not site.userdirsuffix?


-- 
Regards,
Benjamin

From lists at cheimes.de  Sat Oct 10 02:24:44 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 10 Oct 2009 02:24:44 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>
Message-ID: <4ACFD44C.7040009@cheimes.de>

Benjamin Peterson wrote:
>> sys.implementation is a PyStructSequence that contains various
>> information about the current implementation. Some fields are required
>> to be present on every implementation. Implementations may choose to add
>> additional fields as they see fit. Some fields like compiler are useful
>> for introspection and are already part of sys.version. I like to include
>> them for the sake of completeness.
> 
> Can we generate sys.version from sys.implementation then?

sys.version is created by Py_GetVersion() which uses Py_GetCompiler()
for the compiler information:

PyOS_snprintf(version, sizeof(version), "%.80s (%.80s) %.80s",
              PY_VERSION, Py_GetBuildInfo(), Py_GetCompiler());

For CPython the compiler attribute should be set to the return value of
Py_GetCompiler().

>> sys.userdirsuffix
>> -----------------
> 
> Why not site.userdirsuffix?

Because all implementations of Python like to use the same, unpatched
site module. The sys module is different for every implementation. It's
more convenient to have an attribute on the sys module that can be
filled by each implementation. I could also add a lookup table for all
known implementations to the site module. But what about unknown or new
implementations? They would have to wait until we add a new entry for
them. The sys.userdirsuffix is more flexible and future prove.

Christian

From martin at v.loewis.de  Sat Oct 10 02:29:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 10 Oct 2009 02:29:18 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACEF266.3020309@egenix.com>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>	<4ACD965D.5080701@egenix.com>	<30910449ef0aad44ac8d419c4e36896f@preisshare.net>
	<4ACEF266.3020309@egenix.com>
Message-ID: <4ACFD55E.1090608@v.loewis.de>

> I know that the issue tracker for PyPI is (still) on SF, but
> development appear to happen elsewhere and I can't find any
> contact information on the PyPI web pages.

PyPI discussion takes place mostly on catalog-sig.

Regards,
Martin

From benjamin at python.org  Sat Oct 10 02:29:48 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 9 Oct 2009 19:29:48 -0500
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <4ACFD44C.7040009@cheimes.de>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>
	<4ACFD44C.7040009@cheimes.de>
Message-ID: <1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>

2009/10/9 Christian Heimes <lists at cheimes.de>:
> Benjamin Peterson wrote:
>>> sys.userdirsuffix
>>> -----------------
>>
>> Why not site.userdirsuffix?
>
> Because all implementations of Python like to use the same, unpatched
> site module. The sys module is different for every implementation. It's
> more convenient to have an attribute on the sys module that can be
> filled by each implementation. I could also add a lookup table for all
> known implementations to the site module. But what about unknown or new
> implementations? They would have to wait until we add a new entry for
> them. The sys.userdirsuffix is more flexible and future prove.

I think we should make a semi-private (public to the stdlib) module
like _sys or _implementation part of the Python VM API. Then, instead
of stuffing everything into sys, we can provide this information in
modules where it belongs.



-- 
Regards,
Benjamin

From martin at v.loewis.de  Sat Oct 10 02:34:08 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 10 Oct 2009 02:34:08 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
Message-ID: <4ACFD680.1080001@v.loewis.de>

> Given that this is all about establishing some common ground between
> implementations I would propose not using a CPython specific term
> here such as PyStructSequence :)  The Python docs seem to use structseq
> for sys.float_info.

Also, why does it have to be a sequence in the first place? Wouldn't
a plain object with named attributes be good enough?

>> compiler (required):
>>   verbose name of the compiler, for example "GCC 4.3.3", "Java
>> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"
> 
> What's the value of this attribute?  The main reason I ask is there's
> no way that I know of to determine the JIT being used in .NET.  We could
> of course fill in some dummy value here for IronPython but I'm also not 
> sure why anyone would use this.

Also, why is it the name of the JIT compiler, and not the name of the
source language compiler?

Regards,
Martin

From martin at v.loewis.de  Sat Oct 10 02:42:23 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 10 Oct 2009 02:42:23 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <4ACFD86F.3010305@v.loewis.de>

> sys.implementation is a PyStructSequence that contains various
> information about the current implementation. Some fields are required
> to be present on every implementation.

I think it is important to confirm in advance that all the
implementations listed below agree to implement the PEP "soonish" after
it's adopted. "Required" sounds like a strong term - however, if an
implementation chooses not to implement the PEP, it can do whatever it
wants, including omission of required fields.

> id (required):
>   lower case identifier, for example "cpython", "ironpython", "jython",
> "pypy"

Doing some bike-shedding: I'd like to not use "cpython" as the name of
the python.org implementation. This term, I believe, was coined around
JPython, somehow making the implementation language the primary means
of distinction. However, there may be alternative implementations
written in C (e.g. unladen swallow), or otherwise be "C based" (e.g.
pypy).

So I propose that the python.org version is identified as "python".

> name (required):
>   mixed case name of the implementation, for example "CPython",
> "IronPython", "Jython", "PyPy"

Likewise; alternatively "python.org".

Regards,
Martin

From lists at cheimes.de  Sat Oct 10 02:44:01 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 10 Oct 2009 02:44:01 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4ACFD680.1080001@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
	<4ACFD680.1080001@v.loewis.de>
Message-ID: <4ACFD8D1.6050302@cheimes.de>

Martin v. L?wis wrote:
>> Given that this is all about establishing some common ground between
>> implementations I would propose not using a CPython specific term
>> here such as PyStructSequence :)  The Python docs seem to use structseq
>> for sys.float_info.
> 
> Also, why does it have to be a sequence in the first place? Wouldn't
> a plain object with named attributes be good enough?

Any object with attributes is good enough. For CPython a structseq
provides the necessary feature of a read only object with attributes. I
figured out it's the easiest way to archive the goal without creating a
new class.

> 
>>> compiler (required):
>>>   verbose name of the compiler, for example "GCC 4.3.3", "Java
>>> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"
>> What's the value of this attribute?  The main reason I ask is there's
>> no way that I know of to determine the JIT being used in .NET.  We could
>> of course fill in some dummy value here for IronPython but I'm also not 
>> sure why anyone would use this.
> 
> Also, why is it the name of the JIT compiler, and not the name of the
> source language compiler?

The term "source language compiler" describes the intent of the field
perfectly. Thanks Martin! I was merely guessing what the compiler name
may look like on Jython.

Christian

From greg.ewing at canterbury.ac.nz  Sat Oct 10 02:57:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 10 Oct 2009 13:57:06 +1300
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4ACFD86F.3010305@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org> <4ACFD86F.3010305@v.loewis.de>
Message-ID: <4ACFDBE2.207@canterbury.ac.nz>

Martin v. L?wis wrote:

> So I propose that the python.org version is identified as "python".

But then we won't have a generic term for the language
itself, independent of any implementation.

The "c" in cpython doesn't necessarily have to refer
to the implementation language. Maybe it stands for
"classic python". :-)

-- 
Greg

From lists at cheimes.de  Sat Oct 10 02:58:42 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 10 Oct 2009 02:58:42 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4ACFD86F.3010305@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org> <4ACFD86F.3010305@v.loewis.de>
Message-ID: <4ACFDC42.7010306@cheimes.de>

Martin v. L?wis wrote:
>> sys.implementation is a PyStructSequence that contains various
>> information about the current implementation. Some fields are required
>> to be present on every implementation.
> 
> I think it is important to confirm in advance that all the
> implementations listed below agree to implement the PEP "soonish" after
> it's adopted. "Required" sounds like a strong term - however, if an
> implementation chooses not to implement the PEP, it can do whatever it
> wants, including omission of required fields.

Please let me rephrase my suggestion.

If an implementation wants implemented the PEP it is required to provide
a well defined set of attributes.

Of course I'm open for any suggestions regarding names, specification
and content of the attribues.

> Doing some bike-shedding: I'd like to not use "cpython" as the name of
> the python.org implementation. This term, I believe, was coined around
> JPython, somehow making the implementation language the primary means
> of distinction. However, there may be alternative implementations
> written in C (e.g. unladen swallow), or otherwise be "C based" (e.g.
> pypy).
> 
> So I propose that the python.org version is identified as "python".

CPython is a well established term that is widely used to distinguish
between multiple implementations of Python as a language. On the one
hand the python.org implementation was the first one. It founded the
language and made it popular. On the other hand it may look arrogant
from the perspective of the other implementors if we choose to use
"python" as the identifier for the python.org implementation. In my
opinion we should let Guido decide about the identifier before we have
another bike-shedding war. ;-)

>> name (required):
>>   mixed case name of the implementation, for example "CPython",
>> "IronPython", "Jython", "PyPy"
> 
> Likewise; alternatively "python.org".

Interesting suggestion. Guido?

Christian

From lists at cheimes.de  Sat Oct 10 03:01:12 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 10 Oct 2009 03:01:12 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
Message-ID: <4ACFDCD8.9050703@cheimes.de>

Dino Viehland wrote:
> Given that this is all about establishing some common ground between
> implementations I would propose not using a CPython specific term
> here such as PyStructSequence :)  The Python docs seem to use structseq
> for sys.float_info.

You got me! :)

>> compiler (required):
>>   verbose name of the compiler, for example "GCC 4.3.3", "Java
>> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"
> 
> What's the value of this attribute?  The main reason I ask is there's
> no way that I know of to determine the JIT being used in .NET.  We could
> of course fill in some dummy value here for IronPython but I'm also not 
> sure why anyone would use this.

Martin has already answered both points to my satisfaction. Do you agree
with him?

Christian

From dinov at microsoft.com  Sat Oct 10 03:05:16 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Sat, 10 Oct 2009 01:05:16 +0000
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4ACFDCD8.9050703@cheimes.de>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
	<4ACFDCD8.9050703@cheimes.de>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04A9E9AF@TK5EX14MBXC116.redmond.corp.microsoft.com>

Christian wrote:
> 
> Martin has already answered both points to my satisfaction. Do you
> agree with him?

Sure, source level makes more sense - so we'd have "csc" or "gmcs" if
compiled with Mono (assuming there's some way to do that...).


From lists at cheimes.de  Sat Oct 10 03:07:40 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 10 Oct 2009 03:07:40 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>	
	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>	
	<4ACFD44C.7040009@cheimes.de>
	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>
Message-ID: <4ACFDE5C.7080008@cheimes.de>

Benjamin Peterson wrote:
> I think we should make a semi-private (public to the stdlib) module
> like _sys or _implementation part of the Python VM API. Then, instead
> of stuffing everything into sys, we can provide this information in
> modules where it belongs.


That's an interesting counter proposal. Your idea requires an additional
import that I try to avoid. Looking at memory and performance, an
additional module that is imported anyway isn't better. In my humble
opinion the implementation information belongs into the sys module
anyway. A new module just for the user site suffix seems unnecessary.

Christian

From benjamin at python.org  Sat Oct 10 03:14:48 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 9 Oct 2009 20:14:48 -0500
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <4ACFDE5C.7080008@cheimes.de>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>
	<4ACFD44C.7040009@cheimes.de>
	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>
	<4ACFDE5C.7080008@cheimes.de>
Message-ID: <1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>

2009/10/9 Christian Heimes <lists at cheimes.de>:
> Benjamin Peterson wrote:
>> I think we should make a semi-private (public to the stdlib) module
>> like _sys or _implementation part of the Python VM API. Then, instead
>> of stuffing everything into sys, we can provide this information in
>> modules where it belongs.
>
>
> That's an interesting counter proposal. Your idea requires an additional
> import that I try to avoid. Looking at memory and performance, an
> additional module that is imported anyway isn't better. In my humble
> opinion the implementation information belongs into the sys module
> anyway. A new module just for the user site suffix seems unnecessary.

But we want to hide that this is an implementation detail from the
user. Having a new module just for this attribute might seem like
overkill, but I hope that we could use it for more things in the
future. Besides, if _sys is a builtin module, importing it will not
add much overhead.

I forgot to ask before: Does this deprecate platform.python_implementation()?


-- 
Regards,
Benjamin

From brett at python.org  Sat Oct 10 04:35:48 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 9 Oct 2009 19:35:48 -0700
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com> 
	<4ACFD44C.7040009@cheimes.de>
	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com> 
	<4ACFDE5C.7080008@cheimes.de>
	<1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>
Message-ID: <bbaeab100910091935m56d10d23l9c5a98ad9c096239@mail.gmail.com>

On Fri, Oct 9, 2009 at 18:14, Benjamin Peterson <benjamin at python.org> wrote:

> 2009/10/9 Christian Heimes <lists at cheimes.de>:
> > Benjamin Peterson wrote:
> >> I think we should make a semi-private (public to the stdlib) module
> >> like _sys or _implementation part of the Python VM API. Then, instead
> >> of stuffing everything into sys, we can provide this information in
> >> modules where it belongs.
> >
> >
> > That's an interesting counter proposal. Your idea requires an additional
> > import that I try to avoid. Looking at memory and performance, an
> > additional module that is imported anyway isn't better. In my humble
> > opinion the implementation information belongs into the sys module
> > anyway. A new module just for the user site suffix seems unnecessary.
>
> But we want to hide that this is an implementation detail from the
> user. Having a new module just for this attribute might seem like
> overkill, but I hope that we could use it for more things in the
> future.


To also address Christian's import worry, this new module could contain
everything needed to start the interpreter, much like _warnings does
compared to warnings.

But I honestly don't see why this doesn't belong in sys; it has to do with
the system environment which is the interpreter. Yes, some things currently
in sys could possibly be put elsewhere (e.g. the exception stuff could be in
'exceptions'), but I don't think sys is that much of an odd collection of
code. Without having a desire to eventually clean up the sys module (which I
am not proposing as I know that will never fly) I don't see this as being
worth it.



> Besides, if _sys is a builtin module, importing it will not
> add much overhead.
>

Well, that's assuming the other interpreters makes it built-in. Otherwise
it's only a CPython/python.org perk.


>
> I forgot to ask before: Does this deprecate
> platform.python_implementation()?


 Probably should.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091009/a6fc4983/attachment-0001.htm>

From benjamin at python.org  Sat Oct 10 04:49:02 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 9 Oct 2009 21:49:02 -0500
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <bbaeab100910091935m56d10d23l9c5a98ad9c096239@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>
	<4ACFD44C.7040009@cheimes.de>
	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>
	<4ACFDE5C.7080008@cheimes.de>
	<1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>
	<bbaeab100910091935m56d10d23l9c5a98ad9c096239@mail.gmail.com>
Message-ID: <1afaf6160910091949y48da1289wcbdd186404b66853@mail.gmail.com>

2009/10/9 Brett Cannon <brett at python.org>:
> But I honestly don't see why this doesn't belong in sys; it has to do with
> the system environment which is the interpreter. Yes, some things currently
> in sys could possibly be put elsewhere (e.g. the exception stuff could be in
> 'exceptions'), but I don't think sys is that much of an odd collection of
> code. Without having a desire to eventually clean up the sys module (which I
> am not proposing as I know that will never fly) I don't see this as being
> worth it.

That doesn't mean we have to make it more dirty. usersiteprefix is
used exclusively by site.py to load user modules; it has no function
in the interpreter core.

>> Besides, if _sys is a builtin module, importing it will not
>> add much overhead.
>
> Well, that's assuming the other interpreters makes it built-in. Otherwise
> it's only a CPython/python.org perk.

As far as I am aware, all current Python implementations can build in modules.


-- 
Regards,
Benjamin

From metawilm at gmail.com  Sat Oct 10 09:32:42 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Sat, 10 Oct 2009 09:32:42 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <f6bc9b490910100032y64cc037bx11ba4217268fa8f8@mail.gmail.com>

On Sat, Oct 10, 2009 at 1:29 AM, Christian Heimes <lists at cheimes.de> wrote:
> I'm proposing two new attributes in the sys module: sys.implementation
> and sys.userdirsuffix.

This seems like a good idea.

I'm not sure this idea will easily be accepted, but I'd like to see
the sys module eventually split up in two parts, so it is very obvious
to both implementers and users which system-specific features are
portable and which are not:

a) implementation details of the C implementation (subversion, _*,
dllhandle, dont_write_bytecode, settscdump, ..) in one module,

b) portable functionality (interpreter name and version etc,
builtin_module_names, copyright, excepthook, settrace, ..) in the
other

- Willem (CLPython)

From turnbull at sk.tsukuba.ac.jp  Sat Oct 10 10:25:31 2009
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Sat, 10 Oct 2009 17:25:31 +0900
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
Message-ID: <87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>

Brett Cannon writes:

 > On Fri, Oct 9, 2009 at 11:32, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

 > > Sorry for top-posting, mobile device.

Michael, I for one *very much* appreciate your courtesy.

 > Aahz was the most adamant hater of top-posting and he isn't subscribed to
 > python-dev anymore, so whatever.

Please, let's encourage people to indicate if they top-post.  In the
modern environment bandwidth is not really a consideration, but I *do*
read to the bottom on lists like this one, where the *great* majority
of posters do a good job of trimming.


From mal at egenix.com  Sat Oct 10 12:27:26 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 10 Oct 2009 12:27:26 +0200
Subject: [Python-Dev] eggs now mandatory for pypi?
In-Reply-To: <4ACFD55E.1090608@v.loewis.de>
References: <20091005102547.GA23457@ubuntu.ubuntu-domain>	<4ACA0CBA.2030200@egenix.com>	<4ACB3FC0.1010906@simplistix.co.uk>	<4ACB439E.6050102@egenix.com>	<4ACB45FA.3080707@simplistix.co.uk>	<4ACB5690.1080208@egenix.com>	<4ACB6619.4080807@simplistix.co.uk>	<f1dcf1dfd3d6a67f1affa43825f9e096@preisshare.net>	<4ACD965D.5080701@egenix.com>	<30910449ef0aad44ac8d419c4e36896f@preisshare.net>	<4ACEF266.3020309@egenix.com>
	<4ACFD55E.1090608@v.loewis.de>
Message-ID: <4AD0618E.6050406@egenix.com>

"Martin v. L?wis" wrote:
>> I know that the issue tracker for PyPI is (still) on SF, but
>> development appear to happen elsewhere and I can't find any
>> contact information on the PyPI web pages.
> 
> PyPI discussion takes place mostly on catalog-sig.

Ok, then I'll sign up there.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 10 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Sat Oct 10 12:42:19 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 10 Oct 2009 12:42:19 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>	<4ACFD44C.7040009@cheimes.de>	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>	<4ACFDE5C.7080008@cheimes.de>
	<1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>
Message-ID: <4AD0650B.7080600@egenix.com>

Benjamin Peterson wrote:
> I forgot to ask before: Does this deprecate platform.python_implementation()?

No, platform.py is meant to be portable across multiple Python
versions and as such not really suitable for such deprecations.

It'll also take a long while before all Python implementations
expose a new sys module attribute along the proposed lines.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 10 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Sat Oct 10 13:36:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 10 Oct 2009 11:36:29 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_about_sys=2Eimplementation_and_impleme?=
	=?utf-8?q?ntation=09specific_user_site_directory?=
References: <haoh1j$9c7$1@ger.gmane.org>
	<f6bc9b490910100032y64cc037bx11ba4217268fa8f8@mail.gmail.com>
Message-ID: <loom.20091010T133407-316@post.gmane.org>

Willem Broekema <metawilm <at> gmail.com> writes:
> 
> a) implementation details of the C implementation (subversion, _*,
> dllhandle, dont_write_bytecode, settscdump, ..) in one module,
> 
> b) portable functionality (interpreter name and version etc,
> builtin_module_names, copyright, excepthook, settrace, ..) in the
> other

Where does _getframe() end? :)

Regards

Antoine.



From benjamin at python.org  Sat Oct 10 17:11:52 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 10 Oct 2009 10:11:52 -0500
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <f6bc9b490910100032y64cc037bx11ba4217268fa8f8@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>
	<f6bc9b490910100032y64cc037bx11ba4217268fa8f8@mail.gmail.com>
Message-ID: <1afaf6160910100811p3a79e8a6gf4d6d3a077eb3895@mail.gmail.com>

2009/10/10 Willem Broekema <metawilm at gmail.com>:
> On Sat, Oct 10, 2009 at 1:29 AM, Christian Heimes <lists at cheimes.de> wrote:
>> I'm proposing two new attributes in the sys module: sys.implementation
>> and sys.userdirsuffix.
>
> This seems like a good idea.
>
> I'm not sure this idea will easily be accepted, but I'd like to see
> the sys module eventually split up in two parts, so it is very obvious
> to both implementers and users which system-specific features are
> portable and which are not:

I'm afraid this has already been proposed and rejected. See
http://www.python.org/dev/peps/pep-3139/.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sat Oct 10 17:41:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Oct 2009 01:41:27 +1000
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4AD0650B.7080600@egenix.com>
References: <haoh1j$9c7$1@ger.gmane.org>	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>	<4ACFD44C.7040009@cheimes.de>	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>	<4ACFDE5C.7080008@cheimes.de>	<1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>
	<4AD0650B.7080600@egenix.com>
Message-ID: <4AD0AB27.1040109@gmail.com>

M.-A. Lemburg wrote:
> Benjamin Peterson wrote:
>> I forgot to ask before: Does this deprecate platform.python_implementation()?
> 
> No, platform.py is meant to be portable across multiple Python
> versions and as such not really suitable for such deprecations.
> 
> It'll also take a long while before all Python implementations
> expose a new sys module attribute along the proposed lines.

However, the function could be updated to take advantage of the new
sys.implementation attribute when it was available. If it didn't find
it, it would fallback to figuring out the way it does now.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Oct 10 17:46:55 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Oct 2009 01:46:55 +1000
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4ACFD86F.3010305@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org> <4ACFD86F.3010305@v.loewis.de>
Message-ID: <4AD0AC6F.4000908@gmail.com>

Martin v. L?wis wrote:
> Doing some bike-shedding: I'd like to not use "cpython" as the name of
> the python.org implementation. This term, I believe, was coined around
> JPython, somehow making the implementation language the primary means
> of distinction. However, there may be alternative implementations
> written in C (e.g. unladen swallow), or otherwise be "C based" (e.g.
> pypy).

However, CPython has been used for years to distinguish
CPython-the-reference-implementation from
Python-the-language-definition. Starting to call it something else now
would just confuse people for no good reason.

As Greg said, it may be better to think of it as standing for "Classic
Python" rather than merely "A Python interpreter written in C".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fuzzyman at voidspace.org.uk  Sat Oct 10 17:48:58 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 10 Oct 2009 16:48:58 +0100
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <4AD0ACEA.2050506@voidspace.org.uk>

Christian Heimes wrote:
> Fellow Python developers!
>
> In the light of our recent discussion about implementation specific
> information and user site directory I like to start a new PEP. Much to
> my regret I wasn't able to contribute to Python core development over
> the past months. I hope I can find some free time over the next weeks.
>
> Before I start with the PEP I like to reach consensus over the goal.
> Alternative implementations of our beloved programming languages are
> becoming more important every day. Although IronPython, Jython and PyPy
> are trying hard to get as compatible to CPython as possible there are
> numerous reasons for conditional code. General Python unit tests vs.
> CPython specific unit tests are one reasons, implementation specific
> module search paths are another. At the moment Python has no easy way to
> inspect the implementation besides some code in the platform module.
>
> I'm proposing two new attributes in the sys module: sys.implementation
> and sys.userdirsuffix.
>   

Why not just sys.implementation as a string? Everything else can 
trivially be deduced from that anyway. What is the use-case for the 
extra information?

Michael

> sys.implementation
> ------------------
>
> (Proposed by Nick, MAL and others.)
>
> sys.implementation is a PyStructSequence that contains various
> information about the current implementation. Some fields are required
> to be present on every implementation. Implementations may choose to add
> additional fields as they see fit. Some fields like compiler are useful
> for introspection and are already part of sys.version. I like to include
> them for the sake of completeness.
>
> id (required):
>   lower case identifier, for example "cpython", "ironpython", "jython",
> "pypy"
>
> name (required):
>   mixed case name of the implementation, for example "CPython",
> "IronPython", "Jython", "PyPy"
>
> platform (required):
>   platform or language of the implementation, for example "C", ".NET",
> "Java"
>
> runtime (required):
>   lower case runtime library of the implementation, for example "libc",
> "msvcrt90", "java6", ".net"
>
> compiler (required):
>   verbose name of the compiler, for example "GCC 4.3.3", "Java
> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"
>
> ...
>
>
> sys.userdirsuffix
> -----------------
>
> sys.userdirsuffix is an implementation and platform specific string that
> is used to construct the path for the user site directory as explained
> in PEP 370. The string contains the implementation name as well as the
> version number of Python.
>
> Examples:
>   python2.6 (CPython, Unix)
>   Python26 (CPython, Windows)
>   ironpython2.6 (IronPython, Unix)
>   IronPython26 (IronPython, Windows)
>   ...
>
>
> Comments?
>
> Christian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ncoghlan at gmail.com  Sat Oct 10 18:05:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Oct 2009 02:05:27 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>
References: <4ACDF621.4070603@trueblade.com>
	<E1MvuKa-0004Sm-36@swing.co.at>	<4ACDFD59.4060200@trueblade.com>
	<1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>
Message-ID: <4AD0B0C7.3040700@gmail.com>

Benjamin Peterson wrote:
> 2009/10/8 Eric Smith <eric at trueblade.com>:
>> Christian Tanzer wrote:
>>> IMHO, either the translation is done once and gives identical output or
>>> it isn't worth doing at all.
>> I disagree. I doubt even 0.001% of all format strings involve octal
>> formatting. Is it really worth not providing a transition path if it can't
>> cover this case?
> 
> It's also really easy to just write 0{:o} like my translator does.

That works so long as the original format string doesn't specify either
a space padded field width or else a sign character. For those the extra
zero needs to be inserted after the leading characters but before the
number, so the formatting engine really has to handle it.

I'm actually thinking that having the ability to specify a single 0 as
the leading character for octal output is a legitimate feature. There
are plenty of other tools out there that use a single leading zero to
denote octal numbers (e.g. think of a Python script that generated C
code), so having Python be able to produce such numbers makes a lot of
sense.

Vinay's suggestion of using 'O' instead of 'o' to denote C-style octal
formatting instead of Python-style sounds reasonable to me (similar in
degree to the upper vs lower case distinction for 'x' and 'X' hex
formatting).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Oct 10 18:06:59 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Oct 2009 02:06:59 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACE7209.9060208@g.nevcal.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<loom.20091001T010253-42@post.gmane.org>	<loom.20091001T024614-990@post.gmane.org><ha449d$9n5$1@ger.gmane.org><d11dcfba0910020012s79435c39t4e3df1a4fae8cb11@mail.gmail.com>	<ha69vt$qso$1@ger.gmane.org>	<837DB8E29A4E4FAEABBA764EC89F3692@RaymondLaptop1>	<loom.20091005T102425-605@post.gmane.org>	<loom.20091007T191603-331@post.gmane.org>	<4ACDD2C3.4090608@gmail.com>	<loom.20091008T160529-262@post.gmane.org>	<4ACDF621.4070603@trueblade.com>
	<4ACE7209.9060208@g.nevcal.com>
Message-ID: <4AD0B123.40802@gmail.com>

Glenn Linderman wrote:
> If you want to replace a C program that produces parsed output in a
> given format, and that given format includes leading-0-octal numbers,
> then it would be good to have the capability in Python .format, even
> though Python itself uses 0o prefix.
> 
> Similar arguments may apply anywhere else that sprintf produces
> something that .format cannot currently produce.

Heh, I should have read this before making my last comment - this is
exactly what I was trying to get at :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From eric at trueblade.com  Sat Oct 10 18:09:33 2009
From: eric at trueblade.com (Eric Smith)
Date: Sat, 10 Oct 2009 12:09:33 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AD0B0C7.3040700@gmail.com>
References: <4ACDF621.4070603@trueblade.com>
	<E1MvuKa-0004Sm-36@swing.co.at>	<4ACDFD59.4060200@trueblade.com>
	<1afaf6160910080807w354506e0ifa2e0df2dec89568@mail.gmail.com>
	<4AD0B0C7.3040700@gmail.com>
Message-ID: <4AD0B1BD.10002@trueblade.com>

Nick Coghlan wrote:
> Benjamin Peterson wrote:
>> 2009/10/8 Eric Smith <eric at trueblade.com>:
>>> Christian Tanzer wrote:
>>>> IMHO, either the translation is done once and gives identical output or
>>>> it isn't worth doing at all.
>>> I disagree. I doubt even 0.001% of all format strings involve octal
>>> formatting. Is it really worth not providing a transition path if it can't
>>> cover this case?
>> It's also really easy to just write 0{:o} like my translator does.
> 
> That works so long as the original format string doesn't specify either
> a space padded field width or else a sign character. For those the extra
> zero needs to be inserted after the leading characters but before the
> number, so the formatting engine really has to handle it.
> 
> I'm actually thinking that having the ability to specify a single 0 as
> the leading character for octal output is a legitimate feature. There
> are plenty of other tools out there that use a single leading zero to
> denote octal numbers (e.g. think of a Python script that generated C
> code), so having Python be able to produce such numbers makes a lot of
> sense.
> 
> Vinay's suggestion of using 'O' instead of 'o' to denote C-style octal
> formatting instead of Python-style sounds reasonable to me (similar in
> degree to the upper vs lower case distinction for 'x' and 'X' hex
> formatting).

Mark points out in http://bugs.python.org/issue7094 that we'd also need 
to add alternate float formatting for any automated translation facility 
to work flawlessly. There might be other float issues involving trailing 
decimals with no zeros that work differently, too.

Eric.

From ncoghlan at gmail.com  Sat Oct 10 18:23:43 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Oct 2009 02:23:43 +1000
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4AD0ACEA.2050506@voidspace.org.uk>
References: <haoh1j$9c7$1@ger.gmane.org> <4AD0ACEA.2050506@voidspace.org.uk>
Message-ID: <4AD0B50F.1080008@gmail.com>

Michael Foord wrote:
> Why not just sys.implementation as a string? Everything else can
> trivially be deduced from that anyway. What is the use-case for the
> extra information?

I think Christian's set of required attributes is much too large (I
would only have "name" as a required field), but I can understand the
desire to use a structure rather than a simple string.

As with sys.float_info, it gives a new namespace to store "info about
the implementation" without throwing more and more stuff into the main
sys namespace.

So it might start with just "name" being compulsory (since people can
use lower() to iron out any capitalisation issues), but allow
implementation specific attributes to migrate from the sys module to the
implementation namespace.

So "url" might become a common attribute, with each implementation
providing a pointer to their homepage. CPython might decide to put our
eventual Hg based replacement for sys.subversion under
sys.implementation instead of at the module level, etc.

A designated location for "put your implementation specific attributes
here" inside the sys module is a lot less hassle than creating a whole
new module for the purpose, but gives most of the same benefits.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fuzzyman at voidspace.org.uk  Sat Oct 10 19:31:19 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 10 Oct 2009 18:31:19 +0100
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
Message-ID: <4AD0C4E7.4030307@voidspace.org.uk>

Brett Cannon wrote:
>
>
> On Fri, Oct 9, 2009 at 11:32, Michael Foord <fuzzyman at voidspace.org.uk 
> <mailto:fuzzyman at voidspace.org.uk>> wrote:
>
>     The *only* change in semantics I'm proposing is for users of
>     IronPython 2.6 which is not even at final release yet. CPython
>     users would be unaffected.
>
>
> Then why can't IronPython patch site.py to do what they want? I still 
> feel uncomfortable changing site.py for this in a micro release.

The IronPython team currently have legal issues distributing modified 
versions of the standard library (Dino can correct me if I'm wrong here).

That aside, I think that were a Python feature is implemented in such a 
way as to preclude compatibility with other implementations then we 
should see it as a bugfix to correct it.

I can't see why you should feel uncomfortable (beyond general 
principle), it seems clear that this change can't affect users of 
ClassicPython. It is not only IronPython that will benefit but also 
Jython when they port to Python 2.6.

As a general rule the alternative implementations will only start a 
serious port once a version is released and into maintenance mode. If we 
aren't able to fix compatibility issues then the implementations will 
always have to maintain their own patch sets that may or may not get 
merged back in (or in the case of implementations with awkward legal 
departments they'll have to ship bugs).

All the best,

Michael



>
>     On 9 Oct 2009, at 19:00, Brett Cannon <brett at python.org
>     <mailto:brett at python.org>> wrote:
>
>>
>>
>>     On Fri, Oct 9, 2009 at 04:53, Michael Foord
>>     <fuzzyman at voidspace.org.uk <mailto:fuzzyman at voidspace.org.uk>> wrote:
>>
>>         Christian Heimes wrote:
>>
>>             Michael Foord wrote:
>>              
>>
>>                 I really like this scheme. The important thing for
>>                 IronPython is that we can get it into Python 2.6
>>                 (along with other fixes to make distutils compatible
>>                 with IronPython - like not attempting to
>>                 bytecode-compile when sys.dont_write_bytecode is True).
>>                    
>>
>>
>>             I don't think my proposal will land into 2.6. The changes
>>             are too severe
>>             for a bug fix release.
>>              
>>
>>
>>         Right, certainly not adding umpteen new sys attributes. :-)
>>
>>         The problem is that the alternative implementations run well
>>         behind Python-trunk, indeed it doesn't really make sense for
>>         them to put a lot of effort into implementing a version that
>>         is still in development. The result is that they discover
>>         incompatibilites after a version has gone into 'bugfix only'
>>         mode.
>>
>>         Whilst the fix you have described (add information to sys
>>         that is used by site.py and distutils) is ideal it can only
>>         go into 2.7. I would *still* like to see a fix in 2.6 - even
>>         if it is simple logic in site.py using sys.platform (if
>>         sys.platform == 'cli'; elif sys.platform == 'java' etc). That
>>         way IronPython 2.6 is able to be compatible with Python 2.6.
>>         This logic might need duplicating in distutils (I haven't
>>         looked at how distutils works out where the user
>>         site-packages folder is), but it is a 'maintenance only' fix.
>>
>>
>>     But it's still a change in semantics. Tossing this into 2.6 would
>>     mean that anyone who has worked around the current behaviour is
>>     going to have a busted install.
>>
>>     -Brett 
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From vinay_sajip at yahoo.co.uk  Sat Oct 10 21:31:07 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 10 Oct 2009 19:31:07 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Initialization_of_=5F=5Fbuiltins=5F=5F?=
Message-ID: <loom.20091010T212353-350@post.gmane.org>

>I'm not top-posting, but gmane is giving me a hard time :-(

In the py3k branch, logging has the line

_unicode = 'unicode' in dir(__builtins__)

to determine the existence of Unicode support. The code in trunk, being 1.5.2
compatible, used

hasattr(types, 'UnicodeType')

I wanted to eliminate the types import and modernise the trunk code a bit, so I
copied the py3k line to the trunk version of logging. It didn't work as
expected! So I added the line

print dir(__builtins__)

in logging fairly early on (though not the very first line - just after the

__date__ = ...

line. Here's what I got with 2.7a0 and 2.6.1:

Python 2.7a0 (trunk:75292M, Oct  9 2009, 09:21:05) [MSC v.1500 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
 '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
 '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__',
 '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
 '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__',
 '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items',
 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem',
 'setdefault', 'update', 'values']
>>>

ActivePython 2.6.1.1 (ActiveState Software Inc.) based on
Python 2.6.1 (r261:67515, Dec  5 2008, 13:58:38) [MSC v.1500 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
 '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
 '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__',
 '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
 '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__',
 '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items',
 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem',
 'setdefault', 'update', 'values']
>>>

However, if I just do 

dir(__builtins__)

in the interactive prompt, I get the whole shebang.

Excuse my ignorance, but how come?

Regards,

Vinay Sajip


From benjamin at python.org  Sat Oct 10 21:38:32 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 10 Oct 2009 14:38:32 -0500
Subject: [Python-Dev] Initialization of __builtins__
In-Reply-To: <loom.20091010T212353-350@post.gmane.org>
References: <loom.20091010T212353-350@post.gmane.org>
Message-ID: <1afaf6160910101238m7189342fib18e036aa63ca6b@mail.gmail.com>

2009/10/10 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> Excuse my ignorance, but how come?

Because __builtins__ is a module in __main__ and a dict in other
places. You should always check __builtin__; __builtins__ is an
implementation detail.


-- 
Regards,
Benjamin

From solipsis at pitrou.net  Sat Oct 10 21:40:34 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 10 Oct 2009 19:40:34 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Initialization_of_=5F=5Fbuiltins=5F=5F?=
References: <loom.20091010T212353-350@post.gmane.org>
Message-ID: <loom.20091010T213906-440@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
> 
> In the py3k branch, logging has the line
> 
> _unicode = 'unicode' in dir(__builtins__)

Why do you do this?
In py3k, unicode is always enabled but it's called "str" and the name "unicode"
doesn't exist.

> to determine the existence of Unicode support. The code in trunk, being 1.5.2
> compatible, used
> 
> hasattr(types, 'UnicodeType')

Why don't you simply write:

    unicode_support = True
    try:
        unicode
    except NameError:
        unicode_support = False

?



From vinay_sajip at yahoo.co.uk  Sat Oct 10 22:03:52 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 10 Oct 2009 20:03:52 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Initialization_of_=5F=5Fbuiltins=5F=5F?=
References: <loom.20091010T212353-350@post.gmane.org>
	<loom.20091010T213906-440@post.gmane.org>
Message-ID: <loom.20091010T215615-759@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> Why do you do this?
> In py3k, unicode is always enabled but it's called "str" and the name "unicode"
> doesn't exist.
> 

That wasn't done by me but by GvR (according to svn annotate) in r55818,
presumably during the stdlib porting to py3k. I just copied the line over, not
thinking it wouldn't work for 2.7.

> Why don't you simply write:
> 
>     unicode_support = True
>     try:
>         unicode
>     except NameError:
>         unicode_support = False
> 

That's just about what I've actually done: I was just curious about the
difference between py3k and trunk :-)

If __builtins__ is an implementation detail which can't be relied on, should the
py3k code be changed to the try: form? Or shall I just remove the checks
altogether, since Unicode should always be there in 3.x?

Regards,

Vinay Sajip


From eric at trueblade.com  Sat Oct 10 22:29:26 2009
From: eric at trueblade.com (Eric Smith)
Date: Sat, 10 Oct 2009 16:29:26 -0400
Subject: [Python-Dev] Initialization of __builtins__
In-Reply-To: <loom.20091010T215615-759@post.gmane.org>
References: <loom.20091010T212353-350@post.gmane.org>	<loom.20091010T213906-440@post.gmane.org>
	<loom.20091010T215615-759@post.gmane.org>
Message-ID: <4AD0EEA6.9090600@trueblade.com>

Vinay Sajip wrote:
> If __builtins__ is an implementation detail which can't be relied on, should the
> py3k code be changed to the try: form? Or shall I just remove the checks
> altogether, since Unicode should always be there in 3.x?

Remember that the identifier "unicode" isn't present in py3k. There it's 
"str" and it holds Unicode strings.

Unless you're trying to keep the code identical in both branches, I'd 
just remove the check in py3k and assume str is what you always want to use.

Eric.



From dinov at microsoft.com  Sat Oct 10 23:06:34 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Sat, 10 Oct 2009 21:06:34 +0000
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4AD0C4E7.4030307@voidspace.org.uk>
References: <4ACDCA5A.4090609@cheimes.de>
	<4ACE0E99.7050102@voidspace.org.uk>	<han74m$v9c$1@ger.gmane.org>
	<4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<4AD0C4E7.4030307@voidspace.org.uk>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04AA4809@TK5EX14MBXC116.redmond.corp.microsoft.com>


Michael wrote:
> The IronPython team currently have legal issues distributing modified
> versions of the standard library (Dino can correct me if I'm wrong
> here).

It's actually not due to legal issues although we'd have to check w/ the
lawyers if we wanted to do it.  It's mainly that we don't want to ship a 
modified version of the standard library because it seems like it should
be standard.  It's also a little more convenient for us to not have to 
maintain a separate copy (although we do already do that for the tests).



From brett at python.org  Sat Oct 10 23:17:21 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 10 Oct 2009 14:17:21 -0700
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <4AD0C4E7.4030307@voidspace.org.uk>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk> 
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk> 
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com> 
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com> 
	<4AD0C4E7.4030307@voidspace.org.uk>
Message-ID: <bbaeab100910101417v68deaa34v592f3e6aac69e140@mail.gmail.com>

On Sat, Oct 10, 2009 at 10:31, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> Brett Cannon wrote:
>
>>
>>
>> On Fri, Oct 9, 2009 at 11:32, Michael Foord <fuzzyman at voidspace.org.uk<mailto:
>> fuzzyman at voidspace.org.uk>> wrote:
>>
>>    The *only* change in semantics I'm proposing is for users of
>>    IronPython 2.6 which is not even at final release yet. CPython
>>    users would be unaffected.
>>
>>
>> Then why can't IronPython patch site.py to do what they want? I still feel
>> uncomfortable changing site.py for this in a micro release.
>>
>
> The IronPython team currently have legal issues distributing modified
> versions of the standard library (Dino can correct me if I'm wrong here).
>
> That aside, I think that were a Python feature is implemented in such a way
> as to preclude compatibility with other implementations then we should see
> it as a bugfix to correct it.
>
> I can't see why you should feel uncomfortable (beyond general principle),
> it seems clear that this change can't affect users of ClassicPython. It is
> not only IronPython that will benefit but also Jython when they port to
> Python 2.6.
>

It's just principle. I'm not going to fight this as no one else has spoken
up with my view and both Dino and Frank would like this to happen.

The one unfortunate thing about this proposal is how this is going to mean I
have to install a package potentially four times if I want it available to
all possible Python interpreters. Then again, the chances of anyone beyond
the people on this mailing list using more than a single interpreter
regularly is so small that bothering to add support for yet another user
directory that works on any Python interpreter will not be worth it.

-Brett




> As a general rule the alternative implementations will only start a serious
> port once a version is released and into maintenance mode. If we aren't able
> to fix compatibility issues then the implementations will always have to
> maintain their own patch sets that may or may not get merged back in (or in
> the case of implementations with awkward legal departments they'll have to
> ship bugs).
>
> All the best,
>
> Michael
>
>
>
>
>>    On 9 Oct 2009, at 19:00, Brett Cannon <brett at python.org
>>    <mailto:brett at python.org>> wrote:
>>
>>
>>>
>>>    On Fri, Oct 9, 2009 at 04:53, Michael Foord
>>>    <fuzzyman at voidspace.org.uk <mailto:fuzzyman at voidspace.org.uk>> wrote:
>>>
>>>        Christian Heimes wrote:
>>>
>>>            Michael Foord wrote:
>>>
>>>                I really like this scheme. The important thing for
>>>                IronPython is that we can get it into Python 2.6
>>>                (along with other fixes to make distutils compatible
>>>                with IronPython - like not attempting to
>>>                bytecode-compile when sys.dont_write_bytecode is True).
>>>
>>>
>>>            I don't think my proposal will land into 2.6. The changes
>>>            are too severe
>>>            for a bug fix release.
>>>
>>>
>>>        Right, certainly not adding umpteen new sys attributes. :-)
>>>
>>>        The problem is that the alternative implementations run well
>>>        behind Python-trunk, indeed it doesn't really make sense for
>>>        them to put a lot of effort into implementing a version that
>>>        is still in development. The result is that they discover
>>>        incompatibilites after a version has gone into 'bugfix only'
>>>        mode.
>>>
>>>        Whilst the fix you have described (add information to sys
>>>        that is used by site.py and distutils) is ideal it can only
>>>        go into 2.7. I would *still* like to see a fix in 2.6 - even
>>>        if it is simple logic in site.py using sys.platform (if
>>>        sys.platform == 'cli'; elif sys.platform == 'java' etc). That
>>>        way IronPython 2.6 is able to be compatible with Python 2.6.
>>>        This logic might need duplicating in distutils (I haven't
>>>        looked at how distutils works out where the user
>>>        site-packages folder is), but it is a 'maintenance only' fix.
>>>
>>>
>>>    But it's still a change in semantics. Tossing this into 2.6 would
>>>    mean that anyone who has worked around the current behaviour is
>>>    going to have a busted install.
>>>
>>>    -Brett
>>>
>>
>>
>>
>
> --
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091010/9828898b/attachment-0001.htm>

From kevin at bud.ca  Sun Oct 11 00:16:22 2009
From: kevin at bud.ca (Kevin Teague)
Date: Sat, 10 Oct 2009 15:16:22 -0700
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <bbaeab100910101417v68deaa34v592f3e6aac69e140@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<4AD0C4E7.4030307@voidspace.org.uk>
	<bbaeab100910101417v68deaa34v592f3e6aac69e140@mail.gmail.com>
Message-ID: <45F3668A-AB95-4C65-8C86-83980CAB6C75@bud.ca>


On Oct 10, 2009, at 2:17 PM, Brett Cannon wrote:
>
> The one unfortunate thing about this proposal is how this is going  
> to mean I have to install a package potentially four times if I want  
> it available to all possible Python interpreters. Then again, the  
> chances of anyone beyond the people on this mailing list using more  
> than a single interpreter regularly is so small that bothering to  
> add support for yet another user directory that works on any Python  
> interpreter will not be worth it.
>


Yeah, I imagine that the use-case of wanting to re-use a shared set of  
packages between multiple interpreter implementations is relatively  
small. But if you did want to do so, you could just export the  
location for your (PYTHONIMPLEMENTATIONNAME)PATH to the location you  
want to re-use and off you go ...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091010/a160cd64/attachment.htm>

From fuzzyman at voidspace.org.uk  Sun Oct 11 01:06:38 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 11 Oct 2009 00:06:38 +0100
Subject: [Python-Dev] PEP 370 and IronPython
In-Reply-To: <bbaeab100910101417v68deaa34v592f3e6aac69e140@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<4AD0C4E7.4030307@voidspace.org.uk>
	<bbaeab100910101417v68deaa34v592f3e6aac69e140@mail.gmail.com>
Message-ID: <4AD1137E.2030604@voidspace.org.uk>

Brett Cannon wrote:
>
>
> On Sat, Oct 10, 2009 at 10:31, Michael Foord 
> <fuzzyman at voidspace.org.uk <mailto:fuzzyman at voidspace.org.uk>> wrote:
>
>     Brett Cannon wrote:
>
>
>
>         On Fri, Oct 9, 2009 at 11:32, Michael Foord
>         <fuzzyman at voidspace.org.uk <mailto:fuzzyman at voidspace.org.uk>
>         <mailto:fuzzyman at voidspace.org.uk
>         <mailto:fuzzyman at voidspace.org.uk>>> wrote:
>
>            The *only* change in semantics I'm proposing is for users of
>            IronPython 2.6 which is not even at final release yet. CPython
>            users would be unaffected.
>
>
>         Then why can't IronPython patch site.py to do what they want?
>         I still feel uncomfortable changing site.py for this in a
>         micro release.
>
>
>     The IronPython team currently have legal issues distributing
>     modified versions of the standard library (Dino can correct me if
>     I'm wrong here).
>
>     That aside, I think that were a Python feature is implemented in
>     such a way as to preclude compatibility with other implementations
>     then we should see it as a bugfix to correct it.
>
>     I can't see why you should feel uncomfortable (beyond general
>     principle), it seems clear that this change can't affect users of
>     ClassicPython. It is not only IronPython that will benefit but
>     also Jython when they port to Python 2.6.
>
>
> It's just principle. I'm not going to fight this as no one else has 
> spoken up with my view and both Dino and Frank would like this to happen.
>
> The one unfortunate thing about this proposal is how this is going to 
> mean I have to install a package potentially four times if I want it 
> available to all possible Python interpreters. Then again, the chances 
> of anyone beyond the people on this mailing list using more than a 
> single interpreter regularly is so small that bothering to add support 
> for yet another user directory that works on any Python interpreter 
> will not be worth it.
>

It is unfortunate, there are quite a few modules that I use with both 
IronPython and ClassicPython. Unfortunately there are many others where 
I have to make modifications or replace components (especially C 
extensions). It is these modules that require changes that mean separate 
site-packages directories are needed.

To be honest, at the moment I tend to bundle all third party 
dependencies in with my IronPython projects. As distutils support for 
IronPython improves I hope that will change.

All the best,


Michael

> -Brett
>
>
>
>
>     As a general rule the alternative implementations will only start
>     a serious port once a version is released and into maintenance
>     mode. If we aren't able to fix compatibility issues then the
>     implementations will always have to maintain their own patch sets
>     that may or may not get merged back in (or in the case of
>     implementations with awkward legal departments they'll have to
>     ship bugs).
>
>     All the best,
>
>     Michael
>
>
>
>
>            On 9 Oct 2009, at 19:00, Brett Cannon <brett at python.org
>         <mailto:brett at python.org>
>            <mailto:brett at python.org <mailto:brett at python.org>>> wrote:
>
>
>
>                On Fri, Oct 9, 2009 at 04:53, Michael Foord
>                <fuzzyman at voidspace.org.uk
>             <mailto:fuzzyman at voidspace.org.uk>
>             <mailto:fuzzyman at voidspace.org.uk
>             <mailto:fuzzyman at voidspace.org.uk>>> wrote:
>
>                    Christian Heimes wrote:
>
>                        Michael Foord wrote:
>                        
>                            I really like this scheme. The important
>             thing for
>                            IronPython is that we can get it into
>             Python 2.6
>                            (along with other fixes to make distutils
>             compatible
>                            with IronPython - like not attempting to
>                            bytecode-compile when
>             sys.dont_write_bytecode is True).
>                              
>
>                        I don't think my proposal will land into 2.6.
>             The changes
>                        are too severe
>                        for a bug fix release.
>                        
>
>                    Right, certainly not adding umpteen new sys
>             attributes. :-)
>
>                    The problem is that the alternative implementations
>             run well
>                    behind Python-trunk, indeed it doesn't really make
>             sense for
>                    them to put a lot of effort into implementing a
>             version that
>                    is still in development. The result is that they
>             discover
>                    incompatibilites after a version has gone into
>             'bugfix only'
>                    mode.
>
>                    Whilst the fix you have described (add information
>             to sys
>                    that is used by site.py and distutils) is ideal it
>             can only
>                    go into 2.7. I would *still* like to see a fix in
>             2.6 - even
>                    if it is simple logic in site.py using sys.platform (if
>                    sys.platform == 'cli'; elif sys.platform == 'java'
>             etc). That
>                    way IronPython 2.6 is able to be compatible with
>             Python 2.6.
>                    This logic might need duplicating in distutils (I
>             haven't
>                    looked at how distutils works out where the user
>                    site-packages folder is), but it is a 'maintenance
>             only' fix.
>
>
>                But it's still a change in semantics. Tossing this into
>             2.6 would
>                mean that anyone who has worked around the current
>             behaviour is
>                going to have a busted install.
>
>                -Brett
>
>
>
>
>
>     -- 
>     http://www.ironpythoninaction.com/
>     http://www.voidspace.org.uk/blog
>
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From greg.ewing at canterbury.ac.nz  Sun Oct 11 02:02:56 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 11 Oct 2009 13:02:56 +1300
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4AD120B0.5080100@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> In the
> modern environment bandwidth is not really a consideration,

That's no reason to squander it, though. Quoting the entire
message every time makes the size of the thread grow as
O(n**2), and makes things harder to read as well. That's
just senseless.

The only possible benefit is to allow latecomers to catch
up on the thread without having to consult any archives.
But I've never seen any complaints that people who *do*
quote responsibly are making this hard, so I can only
conclude that it's not a significant problem.

-- 
Greg

From stephen at xemacs.org  Sun Oct 11 06:26:41 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 11 Oct 2009 13:26:41 +0900
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <4AD120B0.5080100@canterbury.ac.nz>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
Message-ID: <873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > That's no reason to squander it, though. Quoting the entire
 > message every time makes the size of the thread grow as
 > O(n**2),

Agreed, but let's not exaggerate.  Threads are finite, so big O is not
really relevant.  Spam OTOH is boundless, and that's where the
bandwidth is going.

 > and makes things harder to read as well.

Indeed, and that's why I thanked Michael.  Trimming can be a PITA if
you're using a crummy MUA, and for reasons I have no intention of even
trying to remember, let alone understand, a lot of people are very
attached to their crummmy MUAs.  But adding a few words to at least
make it a little readable (whether apologetic or factual) would make
me a lot happier ... given that they're going to top post anyway.


From fdrake at gmail.com  Sun Oct 11 06:59:47 2009
From: fdrake at gmail.com (Fred Drake)
Date: Sun, 11 Oct 2009 00:59:47 -0400
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <4AD120B0.5080100@canterbury.ac.nz>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk> 
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk> 
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com> 
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com> 
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
Message-ID: <9cee7ab80910102159x46ccbb39l5b147bb17d8c6bb9@mail.gmail.com>

On Sat, Oct 10, 2009 at 8:02 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> That's no reason to squander it, though. Quoting the entire
> message every time makes the size of the thread grow as
> O(n**2), and makes things harder to read as well. That's
> just senseless.

Most importantly, insufficient trimming makes many of us start to
ignore threads we'd otherwise want to read more carefully or
participate in, because the tedium of wading through all the quotes to
make sure we catch all the content.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From steve at pearwood.info  Sun Oct 11 12:01:36 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 11 Oct 2009 21:01:36 +1100
Subject: [Python-Dev] [OT] Re:  Top-posting on this list
In-Reply-To: <873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4ACDCA5A.4090609@cheimes.de> <4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <200910112101.36823.steve@pearwood.info>

On Sun, 11 Oct 2009 03:26:41 pm Stephen J. Turnbull wrote:

> Indeed, and that's why I thanked Michael.  Trimming can be a PITA if
> you're using a crummy MUA, and for reasons I have no intention of
> even trying to remember, let alone understand, a lot of people are
> very attached to their crummmy MUAs. 

Perhaps I've never used a sufficiently crummy MUA, but I don't get this. 
Particularly for GUI apps on a PC, trimming just means: select text, 
press delete, repeat until done. How hard can it be? Surely even the 
crummiest MUA lets you delete text? This might be hard on a PDA or 
mobile phone, but on a PC with a keyboard and mouse?

(I have a vague recollection that Lotus Notes inserts the quoted message 
*after* you've done editing your reply, so Notes users should be 
forgiven. Not to mention pitied.)


-- 
Steven D'Aprano

From ben+python at benfinney.id.au  Sun Oct 11 13:26:02 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 11 Oct 2009 22:26:02 +1100
Subject: [Python-Dev] Top-posting on this list
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87fx9qrx2t.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> Trimming can be a PITA if you're using a crummy MUA

How so? It merely requires the ability to navigate up and down by lines,
and to select and delete text. I've used some very crummy MUAs, but the
ability to trim quoted text has never been absent or difficult. Are
there MUAs so crummy that this is a PITA, yet still used by any
significant portion of email users?

-- 
 \            ?Simplicity is prerequisite for reliability.? ?Edsger W. |
  `\                                                          Dijkstra |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Sun Oct 11 13:27:56 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 11 Oct 2009 22:27:56 +1100
Subject: [Python-Dev] Top-posting on this list
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<9cee7ab80910102159x46ccbb39l5b147bb17d8c6bb9@mail.gmail.com>
Message-ID: <87bpkerwzn.fsf@benfinney.id.au>

Fred Drake <fdrake at gmail.com> writes:

> Most importantly, insufficient trimming makes many of us start to
> ignore threads we'd otherwise want to read more carefully or
> participate in, because the tedium of wading through all the quotes to
> make sure we catch all the content.

Absolutely. This is a significant reason to publicly, and frequently,
respond to request inline-replied, suitably-trimmed messages: to help
newcomers reading the list understand what constitutes an easy-to-read
message, and to maximise their opportunity to write messages that won't
simply be skipped as too much effort to read.

-- 
 \          ?The WWW is exciting because Microsoft doesn't own it, and |
  `\              therefore, there's a tremendous amount of innovation |
_o__)                                          happening.? ?Steve Jobs |
Ben Finney


From solipsis at pitrou.net  Sun Oct 11 13:34:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 11:34:53 +0000 (UTC)
Subject: [Python-Dev] Top-posting on this list
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
Message-ID: <loom.20091011T133335-723@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> That's no reason to squander it, though. Quoting the entire
> message every time makes the size of the thread grow as
> O(n**2), and makes things harder to read as well. That's
> just senseless.

+1. It's always annoying to skim through a three-page quoted message just to
read a two-line reply. The problem is not electronic bandwidth, but visual
bandwidth.

Regards

Antoine.



From ziade.tarek at gmail.com  Sun Oct 11 13:36:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 11 Oct 2009 13:36:44 +0200
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <87fx9qrx2t.fsf@benfinney.id.au>
References: <4ACDCA5A.4090609@cheimes.de> <han74m$v9c$1@ger.gmane.org>
	<4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au>
Message-ID: <94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>

On Sun, Oct 11, 2009 at 1:26 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
>
>> Trimming can be a PITA if you're using a crummy MUA
>
> How so? It merely requires the ability to navigate up and down by lines,
> and to select and delete text. I've used some very crummy MUAs, but the
> ability to trim quoted text has never been absent or difficult. Are
> there MUAs so crummy that this is a PITA, yet still used by any
> significant portion of email users?

You just can't do it on some mobile device mail clients. For instance
Gmail's client on Android.
It will just top-post and quote the whole mail for you AFAIK.

Tarek

From ben+python at benfinney.id.au  Sun Oct 11 14:31:05 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 11 Oct 2009 23:31:05 +1100
Subject: [Python-Dev] Top-posting on this list
References: <4ACDCA5A.4090609@cheimes.de> <han74m$v9c$1@ger.gmane.org>
	<4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au>
	<94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>
Message-ID: <877hv2ru2e.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> You just can't do it on some mobile device mail clients. For instance
> Gmail's client on Android.
> It will just top-post and quote the whole mail for you AFAIK.

Wow, that *is* crummy. Perhaps a posse of users of that application can
loudly request this basic feature from the vendor, to at least make it
easy to avoid sending obnoxious messages.

-- 
 \        ?Ice water? Get some onions ? that'll make your eyes water!? |
  `\                                                     ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From masklinn at masklinn.net  Sun Oct 11 15:04:34 2009
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 11 Oct 2009 15:04:34 +0200
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de> <han74m$v9c$1@ger.gmane.org>
	<4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au>
	<94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>
Message-ID: <565817BF-F768-4E81-A099-5DF7474970BC@masklinn.net>

On 11 Oct 2009, at 13:36 , Tarek Ziad? wrote:
> On Sun, Oct 11, 2009 at 1:26 PM, Ben Finney <ben+python at benfinney.id.au 
> > wrote:
>> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
>>
>>> Trimming can be a PITA if you're using a crummy MUA
>>
>> How so? It merely requires the ability to navigate up and down by  
>> lines,
>> and to select and delete text. I've used some very crummy MUAs, but  
>> the
>> ability to trim quoted text has never been absent or difficult. Are
>> there MUAs so crummy that this is a PITA, yet still used by any
>> significant portion of email users?
>
> You just can't do it on some mobile device mail clients. For instance
> Gmail's client on Android.
> It will just top-post and quote the whole mail for you AFAIK.

I'll add the iPhone pre 3.0 Mail (also holds for the Touch): while  
trimming was possible, it was done using the delete key and character  
per character (jumping to word per word after a few seconds) because  
there was no selection.

This made trimming long mails a fairly long-winded task already, but  
things were even worse due to the lack of undo: not stopping soon  
enough (and deleting part of the text you wanted to quote) meant you  
had to either re-type it manually or restart the whole operation from  
the start.

This has thankfully been fixed (with the addition of a pretty good  
selection mechanism and an undo).

Another iPhone mail "feature", (which hasn't been fixed and is  
actually just as broken in the desktop version of Mail) is that Mail  
is hardcoded to top-post: it's not possible to make the insertion  
point default to bottom the quote, it's always above it with a bunch  
of white-spaces for the message. Not much of an issue, but it still  
makes top-posting lower friction than bottom-posting or trimming.

From mal at egenix.com  Sun Oct 11 15:13:33 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun, 11 Oct 2009 15:13:33 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4AD0AB27.1040109@gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org>	<1afaf6160910091651i3fa55fc0qf3e786af462b8e02@mail.gmail.com>	<4ACFD44C.7040009@cheimes.de>	<1afaf6160910091729j20f63d4ai389f1952716c58f9@mail.gmail.com>	<4ACFDE5C.7080008@cheimes.de>	<1afaf6160910091814p5e14c5e6o16620219378bca3a@mail.gmail.com>	<4AD0650B.7080600@egenix.com>
	<4AD0AB27.1040109@gmail.com>
Message-ID: <4AD1D9FD.3090206@egenix.com>

Nick Coghlan wrote:
> M.-A. Lemburg wrote:
>> Benjamin Peterson wrote:
>>> I forgot to ask before: Does this deprecate platform.python_implementation()?
>>
>> No, platform.py is meant to be portable across multiple Python
>> versions and as such not really suitable for such deprecations.
>>
>> It'll also take a long while before all Python implementations
>> expose a new sys module attribute along the proposed lines.
> 
> However, the function could be updated to take advantage of the new
> sys.implementation attribute when it was available. If it didn't find
> it, it would fallback to figuring out the way it does now.

Sure. It already does that for a couple of other APIs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 11 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Sun Oct 11 17:50:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 15:50:54 +0000 (UTC)
Subject: [Python-Dev] Weak dict iterators are fragile
Message-ID: <hasusu$986$1@ger.gmane.org>


Hello,

In py3k, the weak dict methods keys(), values() and items() have been 
changed to return iterators (they returned lists in 2.x).
However, it turns out that it makes these methods quite fragile, because 
a GC collection can occur whenever during iterating, destroy one of the 
weakref'ed objects, and trigger a resizing of the underlying dict, which 
in turn raises an exception ("RuntimeError: dictionary changed size 
during iteration").

This has just triggered (assuming the diagnosis is correct) a hang in 
test_multiprocessing: http://bugs.python.org/issue7060

I would like to propose some possible solutions against this:

1. Add the safe methods listkeys(), listitems(), listvalues() which would 
behave as the keys(), etc. methods from 2.x

2. Make it so that keys(), items(), values() atomically build a list of 
items internally, which makes them more costly for large weak dicts, but 
robust.

What do you think?

Regards

Antoine.



From python at mrabarnett.plus.com  Sun Oct 11 18:07:29 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 11 Oct 2009 17:07:29 +0100
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <565817BF-F768-4E81-A099-5DF7474970BC@masklinn.net>
References: <4ACDCA5A.4090609@cheimes.de>
	<han74m$v9c$1@ger.gmane.org>	<4ACF243F.4080408@voidspace.org.uk>	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>	<4AD120B0.5080100@canterbury.ac.nz>	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fx9qrx2t.fsf@benfinney.id.au>	<94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>
	<565817BF-F768-4E81-A099-5DF7474970BC@masklinn.net>
Message-ID: <4AD202C1.8060604@mrabarnett.plus.com>

Masklinn wrote:
> On 11 Oct 2009, at 13:36 , Tarek Ziad? wrote:
>> On Sun, Oct 11, 2009 at 1:26 PM, Ben Finney 
>> <ben+python at benfinney.id.au> wrote:
>>> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
>>>
>>>> Trimming can be a PITA if you're using a crummy MUA
>>>
>>> How so? It merely requires the ability to navigate up and down by lines,
>>> and to select and delete text. I've used some very crummy MUAs, but the
>>> ability to trim quoted text has never been absent or difficult. Are
>>> there MUAs so crummy that this is a PITA, yet still used by any
>>> significant portion of email users?
>>
>> You just can't do it on some mobile device mail clients. For instance
>> Gmail's client on Android.
>> It will just top-post and quote the whole mail for you AFAIK.
> 
> I'll add the iPhone pre 3.0 Mail (also holds for the Touch): while 
> trimming was possible, it was done using the delete key and character 
> per character (jumping to word per word after a few seconds) because 
> there was no selection.
> 
> This made trimming long mails a fairly long-winded task already, but 
> things were even worse due to the lack of undo: not stopping soon enough 
> (and deleting part of the text you wanted to quote) meant you had to 
> either re-type it manually or restart the whole operation from the start.
> 
> This has thankfully been fixed (with the addition of a pretty good 
> selection mechanism and an undo).
> 
> Another iPhone mail "feature", (which hasn't been fixed and is actually 
> just as broken in the desktop version of Mail) is that Mail is hardcoded 
> to top-post: it's not possible to make the insertion point default to 
> bottom the quote, it's always above it with a bunch of white-spaces for 
> the message. Not much of an issue, but it still makes top-posting lower 
> friction than bottom-posting or trimming.
> 
Didn't the iPhone also lack cut-and-paste?

From solipsis at pitrou.net  Sun Oct 11 18:18:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 16:18:32 +0000 (UTC)
Subject: [Python-Dev] Top-posting on this list
References: <4ACDCA5A.4090609@cheimes.de>
	<han74m$v9c$1@ger.gmane.org>	<4ACF243F.4080408@voidspace.org.uk>	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>	<4AD120B0.5080100@canterbury.ac.nz>	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fx9qrx2t.fsf@benfinney.id.au>	<94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>
	<565817BF-F768-4E81-A099-5DF7474970BC@masklinn.net>
	<4AD202C1.8060604@mrabarnett.plus.com>
Message-ID: <loom.20091011T181111-854@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
[snipped three pages of quoted messages before a one-liner]
> Didn't the iPhone also lack cut-and-paste?

Not to sound harsh, but your quoting was a perfect example of wasted visual
bandwidth...
(are you posting from an iPhone ? ;-))

Antoine.



From daniel at stutzbachenterprises.com  Sun Oct 11 18:22:41 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 11 Oct 2009 11:22:41 -0500
Subject: [Python-Dev] Weak dict iterators are fragile
In-Reply-To: <hasusu$986$1@ger.gmane.org>
References: <hasusu$986$1@ger.gmane.org>
Message-ID: <eae285400910110922hee23144wcb1ff87beb9118cb@mail.gmail.com>

On Sun, Oct 11, 2009 at 10:50 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> In py3k, the weak dict methods keys(), values() and items() have been
> changed to return iterators (they returned lists in 2.x).
> However, it turns out that it makes these methods quite fragile, because
> a GC collection can occur whenever during iterating, destroy one of the
> weakref'ed objects, and trigger a resizing of the underlying dict, which
> in turn raises an exception ("RuntimeError: dictionary changed size
> during iteration").
>

Ouch!

The iterator from __iter__ is also affected.

1. Add the safe methods listkeys(), listitems(), listvalues() which would
> behave as the keys(), etc. methods from 2.x
>
> 2. Make it so that keys(), items(), values() atomically build a list of
> items internally, which makes them more costly for large weak dicts, but
> robust.
>

-1 on 1.
+0 on 2.

It'd be nice if we could postpone the resize if there are active iterators,
but I don't think there's a clean way to track the iterators.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/d674ae2a/attachment.htm>

From solipsis at pitrou.net  Sun Oct 11 18:26:21 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 16:26:21 +0000 (UTC)
Subject: [Python-Dev] Weak dict iterators are fragile
References: <hasusu$986$1@ger.gmane.org>
Message-ID: <loom.20091011T182518-418@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> 1. Add the safe methods listkeys(), listitems(), listvalues() which would 
> behave as the keys(), etc. methods from 2.x
> 
> 2. Make it so that keys(), items(), values() atomically build a list of 
> items internally, which makes them more costly for large weak dicts, but 
> robust.

And a third one (a bit more complicated implementation-wise):

3. Delay weak dict removals until any iteration has finished.

Regards

Antoine.



From catch-all at masklinn.net  Sun Oct 11 18:32:16 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Sun, 11 Oct 2009 18:32:16 +0200
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <4AD202C1.8060604@mrabarnett.plus.com>
References: <4ACDCA5A.4090609@cheimes.de>
	<han74m$v9c$1@ger.gmane.org>	<4ACF243F.4080408@voidspace.org.uk>	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>	<4AD120B0.5080100@canterbury.ac.nz>	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fx9qrx2t.fsf@benfinney.id.au>	<94bdd2610910110436y7b1b708ey6bdc1dbb7ac63492@mail.gmail.com>
	<565817BF-F768-4E81-A099-5DF7474970BC@masklinn.net>
	<4AD202C1.8060604@mrabarnett.plus.com>
Message-ID: <75544DF4-D9AA-4BD5-B9A1-39694CEB0E19@masklinn.net>

On 11 Oct 2009, at 18:07 , MRAB wrote:
>
> Didn't the iPhone also lack cut-and-paste?

It did, but given text selection is a near-mandatory requirement to  
cutting text (and pasting isn't very useful if you can't put anything  
into the clipboard) those were implied consequences of the lack of  
selection.


From solipsis at pitrou.net  Sun Oct 11 18:37:38 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 16:37:38 +0000 (UTC)
Subject: [Python-Dev] Weak dict iterators are fragile
References: <hasusu$986$1@ger.gmane.org>
	<eae285400910110922hee23144wcb1ff87beb9118cb@mail.gmail.com>
Message-ID: <loom.20091011T183640-725@post.gmane.org>

Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> 
> -1 on 1.+0 on 2.It'd be nice if we could postpone the resize if there are
active iterators, but I don't think there's a clean way to track the iterators.

I've started experimenting, and it seems reasonably possible using a simple
guard class and a set of weakrefs.

Regards

Antoine.



From stephen at xemacs.org  Sun Oct 11 18:38:15 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 12 Oct 2009 01:38:15 +0900
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <87fx9qrx2t.fsf@benfinney.id.au>
References: <4ACDCA5A.4090609@cheimes.de> <4ACE0E99.7050102@voidspace.org.uk>
	<han74m$v9c$1@ger.gmane.org> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au>
Message-ID: <87r5t9khs8.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:
 > "Stephen J. Turnbull" <stephen at xemacs.org> writes:
 > 
 > > Trimming can be a PITA if you're using a crummy MUA
 > 
 > How so? It merely requires the ability to navigate up and down by lines,
 > and to select and delete text. I've used some very crummy MUAs, but the
 > ability to trim quoted text has never been absent or difficult.

Not difficult, time-consuming.  It can take many seconds to scroll to
bottom in a 200-line quote with gmail and the webmail system used by
the University of California at Santa Cruz.  You bet I get down on my
knees and give thanks that my other MUA is an Emacs every time I have
to use webmail.

Please note that I'm not defending the practice of top-posting; I'm
trying to understand why anybody would do it, and address the
readability issue in that context.  A sentinel that says "the next 500
lines are whatever happened to be in the tail of the 16KiB block on
the file system" does help.

If others are willing to play bad cop, as Aahz did, I'd be very happy
to accept the benefit of a cleaned-up list.  But I'm not willing to do
it myself.

From p.f.moore at gmail.com  Sun Oct 11 18:52:24 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 11 Oct 2009 17:52:24 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ACF6993.5040704@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
Message-ID: <79990c6b0910110952s266d1d0bj35a9a991420e7de9@mail.gmail.com>

2009/10/9 Michael Foord <fuzzyman at voidspace.org.uk>:
> Many Windows users would be quite happy if the standard mechanism for
> installing non-source distributions on Windows was via the wininst binaries.

+1 I'm one of those people.

> I wonder if it is going to be possible to make this compatible with the
> upcoming distutils package management 'stuff' (querying for installed
> packages, uninstallation etc) since installation/uninstallation goes through
> the Windows system package management feature. ?I guess it would be
> eminently possible but require some reasonably high level Windows-fu to do.

I am working with Tarek to keep Windows issues (and in particular this
one) on the agenda. It's quite hard at times, as getting a
representative sample of Windows users' preferences/requirements is
difficult at best (Windows users seem as a group to be either very
quiet, or very easy to please. I'm an exception :-))

If any Windows users want to speak up and help, that would be
immensely useful! (And please, identify yourself as such - it's often
hard to determine who knows Windows, and who is a Unix user making
assumptions or best guesses).

Paul.

From jnoller at gmail.com  Sun Oct 11 18:55:47 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Sun, 11 Oct 2009 12:55:47 -0400
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <87r5t9khs8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4ACDCA5A.4090609@cheimes.de> <4ACF243F.4080408@voidspace.org.uk>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au>
	<87r5t9khs8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4222a8490910110955s5d9d504br428b0fa27b27c6f7@mail.gmail.com>

On Sun, Oct 11, 2009 at 12:38 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:

> If others are willing to play bad cop, as Aahz did, I'd be very happy
> to accept the benefit of a cleaned-up list. ?But I'm not willing to do
> it myself.

Is it really that big of an issue that we have to discuss it
ad-infinitum and potentially have a quoting cop? Sometimes top-posting
happens. Sometimes people don't trim messages. Sometimes people argue
about the color of a shed.

jesse

From steven.bethard at gmail.com  Sun Oct 11 19:27:16 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 11 Oct 2009 10:27:16 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <79990c6b0910110952s266d1d0bj35a9a991420e7de9@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
	<79990c6b0910110952s266d1d0bj35a9a991420e7de9@mail.gmail.com>
Message-ID: <d11dcfba0910111027m736be59fkac5ffe1ea40cc7ac@mail.gmail.com>

On Sun, Oct 11, 2009 at 9:52 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/10/9 Michael Foord <fuzzyman at voidspace.org.uk>:
>> Many Windows users would be quite happy if the standard mechanism for
>> installing non-source distributions on Windows was via the wininst binaries.
>
> +1 I'm one of those people.

+1 on installing packages on Windows through the native package
manager (Add/Remove Programs).

> I am working with Tarek to keep Windows issues (and in particular this
> one) on the agenda. It's quite hard at times, as getting a
> representative sample of Windows users' preferences/requirements is
> difficult at best (Windows users seem as a group to be either very
> quiet, or very easy to please. I'm an exception :-))
>
> If any Windows users want to speak up and help, that would be
> immensely useful! (And please, identify yourself as such - it's often
> hard to determine who knows Windows, and who is a Unix user making
> assumptions or best guesses).

I'm a Windows user, and I've also spent a fair bit of time improving
Python's bdist_msi support exactly so that we have better support for
Windows native package management. However, I don't have enough time
to keep up with the massive package management threads. If you want to
CC me occasionally to get my feedback on a particular comment though,
I'd be happy to speak up.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From solipsis at pitrou.net  Sun Oct 11 19:31:10 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 17:31:10 +0000 (UTC)
Subject: [Python-Dev] Weak dict iterators are fragile
References: <hasusu$986$1@ger.gmane.org>
	<eae285400910110922hee23144wcb1ff87beb9118cb@mail.gmail.com>
	<loom.20091011T183640-725@post.gmane.org>
Message-ID: <loom.20091011T193042-474@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> > 
> > -1 on 1.+0 on 2.It'd be nice if we could postpone the resize if there are
> active iterators, but I don't think there's a clean way to track the iterators.
> 
> I've started experimenting, and it seems reasonably possible using a simple
> guard class and a set of weakrefs.

Open issue and proposed patch at http://bugs.python.org/issue7105


Antoine.



From benjamin at python.org  Sun Oct 11 19:37:58 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 11 Oct 2009 12:37:58 -0500
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <4222a8490910110955s5d9d504br428b0fa27b27c6f7@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de>
	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>
	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz>
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au>
	<87r5t9khs8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4222a8490910110955s5d9d504br428b0fa27b27c6f7@mail.gmail.com>
Message-ID: <1afaf6160910111037u10a4e818w1679a24266adf1bd@mail.gmail.com>

2009/10/11 Jesse Noller <jnoller at gmail.com>:
> On Sun, Oct 11, 2009 at 12:38 PM, Stephen J. Turnbull
> <stephen at xemacs.org> wrote:
>
>> If others are willing to play bad cop, as Aahz did, I'd be very happy
>> to accept the benefit of a cleaned-up list. ?But I'm not willing to do
>> it myself.
>
> Is it really that big of an issue that we have to discuss it
> ad-infinitum and potentially have a quoting cop? Sometimes top-posting
> happens. Sometimes people don't trim messages. Sometimes people argue
> about the color of a shed.

+1 This is really getting so meta that it's off topic.


-- 
Regards,
Benjamin

From g.brandl at gmx.net  Sun Oct 11 20:23:17 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 11 Oct 2009 20:23:17 +0200
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <1afaf6160910111037u10a4e818w1679a24266adf1bd@mail.gmail.com>
References: <4ACDCA5A.4090609@cheimes.de>	<bbaeab100910091100r5fab2ad7yaf6b810b9f0c543a@mail.gmail.com>	<1A2CCCA5-E910-4AE9-9303-032BE939A9BE@voidspace.org.uk>	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com>	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>	<4AD120B0.5080100@canterbury.ac.nz>	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fx9qrx2t.fsf@benfinney.id.au>	<87r5t9khs8.fsf@uwakimon.sk.tsukuba.ac.jp>	<4222a8490910110955s5d9d504br428b0fa27b27c6f7@mail.gmail.com>
	<1afaf6160910111037u10a4e818w1679a24266adf1bd@mail.gmail.com>
Message-ID: <hat7v8$3sb$1@ger.gmane.org>

Benjamin Peterson schrieb:
> 2009/10/11 Jesse Noller <jnoller at gmail.com>:
>> On Sun, Oct 11, 2009 at 12:38 PM, Stephen J. Turnbull
>> <stephen at xemacs.org> wrote:
>>
>>> If others are willing to play bad cop, as Aahz did, I'd be very happy
>>> to accept the benefit of a cleaned-up list.  But I'm not willing to do
>>> it myself.
>>
>> Is it really that big of an issue that we have to discuss it
>> ad-infinitum and potentially have a quoting cop? Sometimes top-posting
>> happens. Sometimes people don't trim messages. Sometimes people argue
>> about the color of a shed.
> 
> +1 This is really getting so meta that it's off topic.

quoting-SIG anyone?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From dickinsm at gmail.com  Sun Oct 11 20:28:11 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 11 Oct 2009 19:28:11 +0100
Subject: [Python-Dev] Backport new float repr to Python 2.7?
Message-ID: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>

In a recent #python-dev IRC conversation, it was suggested that we
should consider backporting the new-style float repr from py3k to
trunk.  I'd like to get people's opinions on this idea.

To recap quickly, the algorithm for computing the repr of floats changed
between Python 2.x and Python 3.x (well, actually between 3.0 and 3.1,
but 3.0 is dead):

 - in Python 2.x, repr(x) computes 17 significant decimal digits, and
   then strips trailing zeros.  In other words, it's pretty much identical
   to doing '%.17g' % x.  The computation is done using the platform's
   *printf functions.

 - in Python 3.x, repr(x) returns the shortest decimal string that's
   guaranteed to evaluate back to the float x under correct rounding.
   The computation is done using David Gay's dtoa.c code, adapted
   for inclusion in Python (in file Python/dtoa.c).

There are (in my view) many benefits to the new approach.  Among
them:

 - fewer newbie complaints and questions (on c.l.p, IRC, Stack
   Overflow, etc.) about Python 'rounding incorrectly'. Whether this is a
   good thing or not is the matter of some debate (I'm tempted to
   borrow the time machine and simply say 'see the replies
   to this message'!)

 - string to float *and* float to string conversions are both guaranteed
   correctly rounded in 3.x: David Gay's code implements the conversion
   in both directions, and having correctly rounded string -> float
   conversions is essential to ensure that eval(repr(x)) recovers x exactly.

 - the repr of round(x, n) really does have at most n digits after the
   point, giving the semi-illusion that x really has been rounded exactly,
   and eliminating one of the most common user complaints about the
   round function.

 - round(x, n) agrees exactly with '{:.{}f}'.format(x, n)  (this isn't
   true in Python 2.x, and the difference is a cause of bug reports)

 - side effects like finding that float(x) rounds correctly for
   Decimal instances x.

 - the output from the new rule is more consistent: the 'strip trailing
   zeros' part of the old rule has some strange consequences:  e.g.,
   in 2.x right now (on a typical machine):

   >>> 0.02
   0.02
   >>> 0.03
   0.029999999999999999

   even though neither 0.02 nor 0.03 can be exactly represented
   in binary.  3.x gives '0.02' and '0.03'.

 - repr(x) is consistent across platforms (or at least across platforms
   with IEEE 754 doubles;  in practice this seems to account for
   virtually all platforms currently running Python).

 - the float <-> string conversions are under our control, so any bugs
   found can be fixed in the Python source.  There's no shortage of
   conversion bugs in the wild, and certainly bugs have been observed in
   OS X, Linux and Windows.  (The ones I found in OS X 10.5 have
   been fixed in OS X 10.6, though.)

Possible problems:

 - breaking docstrings in third party code.  Though Eric reminded me
   that when we implemented this for 3.1, there were essentially no
   standard library test breakages resulting from the changed repr
   format.

 - some might argue that the new repr (and round) just allows users
   to remain ignorant of floating-point difficulties for longer, and that
   this is a bad thing.  I don't really buy either of these points.

 - someone has to put in the work.  As mentioned below, I'm happy
   to do this (and Eric's offered to help, without which this probably
   wouldn't be feasible at all), but it'll use cycles that I could also
   usefully be spending elsewhere.

I'm mostly neutral on the backport idea:  I'm very happy that this is
in 3.x, but don't see any great need to backport it.  But if there's
majority (+BDFL) support, I'm willing to put the work in to do the
backport.

Masochists who are still reading by this point and who want more
information about the new repr implementation can see the issue
discussion:

http://bugs.python.org/issue1580

Thoughts?

Mark

From python at rcn.com  Sun Oct 11 21:44:01 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 11 Oct 2009 12:44:01 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
Message-ID: <BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>


[Mark Dickinson]
> - string to float *and* float to string conversions are both guaranteed
>   correctly rounded in 3.x: David Gay's code implements the conversion
>   in both directions, and having correctly rounded string -> float
>   conversions is essential to ensure that eval(repr(x)) recovers x exactly.

IMO, that is so important that it should be considered a bug fix.
Recently, I lost a lot of time in a discussion about a piece of mathematical
code returning a wrong answer.  The problem wasn't the Python code;
instead, it was the str to float conversion (that we inherit from libc) giving
the wrong answer.  The code worked correctly under Py3.1 but not
Py2.6.

IDLE 2.6.2

>>> float('-30000000000000000477086733292797541405082425691835844169343982673243516316511143182440409957570445312.00000000000000000009000000000000000258695827614150149965556917147323765827915398318737061345018446445465087890625')
-2.9999999999999999e+100


Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> float('-30000000000000000477086733292797541405082425691835844169343982673243516316511143182440409957570445312.00000000000000000009000000000000000258695827614150149965556917147323765827915398318737061345018446445465087890625')
-3.0000000000000002e+100


> - the repr of round(x, n) really does have at most n digits after the
>   point, giving the semi-illusion that x really has been rounded exactly,
>   and eliminating one of the most common user complaints about the
>   round function.

This is also an important improvement and makes round() do what
people expect.


> - side effects like finding that float(x) rounds correctly for
>   Decimal instances x.

This is also important because other decimal calculations can
often be done exactly and it's a bummer to have an accurate
result thrown off by an incorrect rounding to float.


> - the float <-> string conversions are under our control, so any bugs
>   found can be fixed in the Python source.  There's no shortage of
>   conversion bugs in the wild, and certainly bugs have been observed in
>   OS X, Linux and Windows.  (The ones I found in OS X 10.5 have
>   been fixed in OS X 10.6, though.)

Nice win.


> Thoughts?

+1

I've worked with the 3.1 float reprs for a while and have been delighted.
It was a great improvement.


Raymond 


From guido at python.org  Sun Oct 11 21:48:08 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2009 12:48:08 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
Message-ID: <ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>

On Sun, Oct 11, 2009 at 11:28 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> In a recent #python-dev IRC conversation, it was suggested that we
> should consider backporting the new-style float repr from py3k to
> trunk. ?I'd like to get people's opinions on this idea.
[...]
> Possible problems:
>
> ?- breaking docstrings in third party code. ?Though Eric reminded me
> ? that when we implemented this for 3.1, there were essentially no
> ? standard library test breakages resulting from the changed repr
> ? format.

I think you mean doctests? These are the primary reason I've always
been hesitant to change this in 2.x.

> ?- some might argue that the new repr (and round) just allows users
> ? to remain ignorant of floating-point difficulties for longer, and that
> ? this is a bad thing. ?I don't really buy either of these points.

If we bought that we wouldn't have fixed in 3.x. :-)

> ?- someone has to put in the work. ?As mentioned below, I'm happy
> ? to do this (and Eric's offered to help, without which this probably
> ? wouldn't be feasible at all), but it'll use cycles that I could also
> ? usefully be spending elsewhere.
>
> I'm mostly neutral on the backport idea: ?I'm very happy that this is
> in 3.x, but don't see any great need to backport it. ?But if there's
> majority (+BDFL) support, I'm willing to put the work in to do the
> backport.

I'm -0 -- mostly because of the 3rd party doctests and perhaps also
because I'd like 3.x to have some carrots. (I've heard from at least
one author who is very happy with 3.x for the next edition of his
"programming for beginners" book.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Sun Oct 11 21:55:58 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 11 Oct 2009 21:55:58 +0200
Subject: [Python-Dev] http://www.pythonlabs.com/logos.html is gone
Message-ID: <hatdd2$kpo$1@ger.gmane.org>

Which I noticed since it's cited in the BeOpen license we still refer
to in LICENSE.  Since pythonlabs.com itself is still up, it probably
isn't much work to make the logos.html URI work again, but I don't know
who maintains that page.

cheer,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From glyph at twistedmatrix.com  Sun Oct 11 22:00:41 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Sun, 11 Oct 2009 16:00:41 -0400
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
Message-ID: <d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>

On Sun, Oct 11, 2009 at 3:48 PM, Guido van Rossum <guido at python.org> wrote:


> I'm -0 -- mostly because of the 3rd party doctests and perhaps also
> because I'd like 3.x to have some carrots. (I've heard from at least
> one author who is very happy with 3.x for the next edition of his
> "programming for beginners" book.)
>

This reasoning definitely makes sense to me; with all the
dependency-migration issues 3.x could definitely use some carrots.  However,
I don't think I agree with it, because this doesn't feel like a big new
feature, just some behavior which has changed.  The carrots I'm interested
in as a user are new possibilties, like new standard library features, a
better debugger/profiler, or everybody's favorate bugaboo, multicore
parallelism.  (Although, to be fair, the removal of old-style classes
qualifies.)

I'd much rather have my doctests and float-repr'ing code break on 2.7 so I
can deal with it as part of a minor-version upgrade than have it break on
3.x and have to deal with this at the same time as the unicode->str
explosion.  It feels like a backport of this behavior would make the 2->3
transition itself a little easier.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/ab5dd578/attachment.htm>

From guido at python.org  Sun Oct 11 22:21:11 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2009 13:21:11 -0700
Subject: [Python-Dev] http://www.pythonlabs.com/logos.html is gone
In-Reply-To: <hatdd2$kpo$1@ger.gmane.org>
References: <hatdd2$kpo$1@ger.gmane.org>
Message-ID: <ca471dc20910111321ia683d29naae39a3192fffaed@mail.gmail.com>

On Sun, Oct 11, 2009 at 12:55 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Which I noticed since it's cited in the BeOpen license we still refer
> to in LICENSE. ?Since pythonlabs.com itself is still up, it probably
> isn't much work to make the logos.html URI work again, but I don't know
> who maintains that page.

I own the domain. I don't know what was on logos.html and
http://web.archive.org won't show it to me because of a robots.txt
file. I think it's fine to let it be a 404.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From robert.kern at gmail.com  Sun Oct 11 22:43:31 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 11 Oct 2009 15:43:31 -0500
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
Message-ID: <hatg1k$shn$1@ger.gmane.org>

Guido van Rossum wrote:
> On Sun, Oct 11, 2009 at 11:28 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> In a recent #python-dev IRC conversation, it was suggested that we
>> should consider backporting the new-style float repr from py3k to
>> trunk.  I'd like to get people's opinions on this idea.
> [...]
>> Possible problems:
>>
>>  - breaking docstrings in third party code.  Though Eric reminded me
>>   that when we implemented this for 3.1, there were essentially no
>>   standard library test breakages resulting from the changed repr
>>   format.
> 
> I think you mean doctests? These are the primary reason I've always
> been hesitant to change this in 2.x.

I think that doctests inherently unsuited to testing floating point algorithms. 
Leaving aside the platform differences in actual floating point arithmetic that 
cause minute real differences in the results, Python 2.x relies on string 
conversion routines which give different doctest results on different platforms. 
Using a consistent routine would actually improve the ability to use doctests in 
that one regard. It certainly would make writing examples much more consistent, 
particularly for those of us that use infs and nans frequently.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From g.brandl at gmx.net  Sun Oct 11 22:46:04 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 11 Oct 2009 22:46:04 +0200
Subject: [Python-Dev] http://www.pythonlabs.com/logos.html is gone
In-Reply-To: <ca471dc20910111321ia683d29naae39a3192fffaed@mail.gmail.com>
References: <hatdd2$kpo$1@ger.gmane.org>
	<ca471dc20910111321ia683d29naae39a3192fffaed@mail.gmail.com>
Message-ID: <hatgav$t8m$1@ger.gmane.org>

Guido van Rossum schrieb:
> On Sun, Oct 11, 2009 at 12:55 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Which I noticed since it's cited in the BeOpen license we still refer
>> to in LICENSE.  Since pythonlabs.com itself is still up, it probably
>> isn't much work to make the logos.html URI work again, but I don't know
>> who maintains that page.
> 
> I own the domain. I don't know what was on logos.html and
> http://web.archive.org won't show it to me because of a robots.txt
> file. I think it's fine to let it be a 404.

Okay.  Though I am fairly sure that Tim *would* remember ;)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ziade.tarek at gmail.com  Sun Oct 11 22:59:21 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 11 Oct 2009 22:59:21 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <79990c6b0910080556m1f2d4156v791193da56db40fb@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<79990c6b0910080156s2e540184r255624d2166cbae2@mail.gmail.com>
	<94bdd2610910080211q5ff4fd09wf62c44dd4a680a8f@mail.gmail.com>
	<79990c6b0910080556m1f2d4156v791193da56db40fb@mail.gmail.com>
Message-ID: <94bdd2610910111359j7bfe2f0n82d1d6d13cb20c08@mail.gmail.com>

On Thu, Oct 8, 2009 at 2:56 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> In this context, eggs are "merely" the first (and most important)
[..]
> example of a format extension, and so should drive the development of
> a standard.
>
> To summarise:
>
> I believe that we need a statement of direction on the (zipped) egg
> format. I see a number of options:
>
> 1. Retain it as-is in Distribute, but deprecated and no further development.
> 2. Deprecate egg support in Distribute, and ultimately drop it.
> 3. Develop into a reference example of an extensible architecture for
> adding to PEP 376 support.
> 4. Extend PEP 376 support to eggs by means of egg-specific approaches
> hooking into distutils.
[..]
> I'm willing to help work on option (3).

I think PEP 376 needs first to be finished with only one installation
format, without thinking about adding an extensible architecture for
various formats. That's what I meant when I said I wanted to reduce
its scope.

Besides, I think the same site-packages should stick with a single
installation format for the distributions it contains. Although, I can
picture that a site-packages could be stored in a totally different
way, possibly in a database, or with only eggs, but I am still fuzzy
about what would this mean for PEP 376, and the stdlib itself.
(pkgutil+distutils)

And as you said earlier, this could be done later in a second phase,
once PEP 376 APIs (that are not directly related to the format, but
rather to what people do with installed distributions metadata) are
stabilized and proven to work.

Tarek

From ziade.tarek at gmail.com  Sun Oct 11 23:02:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 11 Oct 2009 23:02:29 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <d11dcfba0910111027m736be59fkac5ffe1ea40cc7ac@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
	<79990c6b0910110952s266d1d0bj35a9a991420e7de9@mail.gmail.com>
	<d11dcfba0910111027m736be59fkac5ffe1ea40cc7ac@mail.gmail.com>
Message-ID: <94bdd2610910111402k129fb244m197f1b3ce8606849@mail.gmail.com>

On Sun, Oct 11, 2009 at 7:27 PM, Steven Bethard
<steven.bethard at gmail.com> wrote:
>> I am working with Tarek to keep Windows issues (and in particular this
>> one) on the agenda. It's quite hard at times, as getting a
>> representative sample of Windows users' preferences/requirements is
>> difficult at best (Windows users seem as a group to be either very
>> quiet, or very easy to please. I'm an exception :-))
>>
>> If any Windows users want to speak up and help, that would be
>> immensely useful! (And please, identify yourself as such - it's often
>> hard to determine who knows Windows, and who is a Unix user making
>> assumptions or best guesses).
>
> I'm a Windows user, and I've also spent a fair bit of time improving
> Python's bdist_msi support exactly so that we have better support for
> Windows native package management. However, I don't have enough time
> to keep up with the massive package management threads. If you want to
> CC me occasionally to get my feedback on a particular comment though,
> I'd be happy to speak up.

Thanks for the help guys !

Just to mention, these win32 discussions have now moved back to Distutils-SIG.

Tarek

From brett at python.org  Sun Oct 11 23:16:44 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 11 Oct 2009 14:16:44 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com> 
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com> 
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
Message-ID: <bbaeab100910111416u56eb1b41v9236f0a1f4bce51d@mail.gmail.com>

On Sun, Oct 11, 2009 at 13:00, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:

> On Sun, Oct 11, 2009 at 3:48 PM, Guido van Rossum <guido at python.org>wrote:
>
>
>> I'm -0 -- mostly because of the 3rd party doctests and perhaps also
>> because I'd like 3.x to have some carrots. (I've heard from at least
>> one author who is very happy with 3.x for the next edition of his
>> "programming for beginners" book.)
>>
>
> This reasoning definitely makes sense to me; with all the
> dependency-migration issues 3.x could definitely use some carrots.  However,
> I don't think I agree with it, because this doesn't feel like a big new
> feature, just some behavior which has changed.  The carrots I'm interested
> in as a user are new possibilties, like new standard library features, a
> better debugger/profiler, or everybody's favorate bugaboo, multicore
> parallelism.  (Although, to be fair, the removal of old-style classes
> qualifies.)
>
>
Sure, but if people like Mark are having to spend their time backporting
every bit of behaviour like this then we won't have the time and energy to
add the bigger carrots to 3.x to help entice people to switch.


> I'd much rather have my doctests and float-repr'ing code break on 2.7 so I
> can deal with it as part of a minor-version upgrade than have it break on
> 3.x and have to deal with this at the same time as the unicode->str
> explosion.  It feels like a backport of this behavior would make the 2->3
> transition itself a little easier.
>

Maybe, but as Mark pointed out, at least in the test suite for Python, there
was no breakage. This will only be an issues if someone does::

  >>> x
  0.029999999

instead of::

  >>> x == 0.03
  True

Plus it should be obvious when a doctest breaks with 0.03 != 0.0299999999999
what has happened.

I'm with Guido: -0 on the backport, especially with Mark feeling neutral on
wanting to put the effort in.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/7fd84f19/attachment.htm>

From glyph at twistedmatrix.com  Sun Oct 11 23:27:04 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Sun, 11 Oct 2009 17:27:04 -0400
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <bbaeab100910111416u56eb1b41v9236f0a1f4bce51d@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
	<bbaeab100910111416u56eb1b41v9236f0a1f4bce51d@mail.gmail.com>
Message-ID: <d9047f780910111427ga1576eeic6c8835704cca70e@mail.gmail.com>

On Sun, Oct 11, 2009 at 5:16 PM, Brett Cannon <brett at python.org> wrote:

> On Sun, Oct 11, 2009 at 13:00, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:
>
>> The carrots I'm interested in as a user are new possibilties, like new
>> standard library features, a better debugger/profiler, or everybody's
>> favorate bugaboo, multicore parallelism.  (Although, to be fair, the removal
>> of old-style classes qualifies.)
>>
> Sure, but if people like Mark are having to spend their time backporting
> every bit of behaviour like this then we won't have the time and energy to
> add the bigger carrots to 3.x to help entice people to switch.
>

Okay, call me +0 then.  Not one of the migration issues I'm really sweating
about :).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/8b901cf0/attachment.htm>

From sparks.m at gmail.com  Sun Oct 11 23:35:56 2009
From: sparks.m at gmail.com (Michael Sparks)
Date: Sun, 11 Oct 2009 22:35:56 +0100
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
Message-ID: <20e5c9660910111435o4d2c48e2he3504a7750201fad@mail.gmail.com>

On Sunday 11 October 2009 21:00:41 Glyph Lefkowitz wrote:
> with all the
> dependency-migration issues 3.x could definitely use some carrots.
..
> everybody's favorate bugaboo, multicore parallelism.

I know it's the upteen-thousandth time it's been discussed, but
removal of the GIL in 3.x would probably be pretty big carrots for
some. I know the arguments about performance hits on single core
systems etc, and the simplifications it brings, but given every entry
level machine these days is multicore, is it time to reconsider some
of those points ? Not here perhaps - python-ideas or c.l.p, but if
bigger carrots are wanted... Just saying. (As time goes on, lack of a
GIL in Ironpython makes it more attractive for multicore work)

Not suggesting this happens, but just noting it would probably be a big carrot.


Michael.
-- 
http://yeoldeclue.com/blog
http://twitter.com/kamaelian
http://www.kamaelia.org/Home

From benjamin at python.org  Sun Oct 11 23:40:45 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 11 Oct 2009 16:40:45 -0500
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <20e5c9660910111435o4d2c48e2he3504a7750201fad@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
	<20e5c9660910111435o4d2c48e2he3504a7750201fad@mail.gmail.com>
Message-ID: <1afaf6160910111440r660767e8o1d7745911cd95f2e@mail.gmail.com>

2009/10/11 Michael Sparks <sparks.m at gmail.com>:
> On Sunday 11 October 2009 21:00:41 Glyph Lefkowitz wrote:
>> with all the
>> dependency-migration issues 3.x could definitely use some carrots.
> ..
>> everybody's favorate bugaboo, multicore parallelism.
>
> I know it's the upteen-thousandth time it's been discussed, but
> removal of the GIL in 3.x would probably be pretty big carrots for
> some. I know the arguments about performance hits on single core
> systems etc, and the simplifications it brings, but given every entry
> level machine these days is multicore, is it time to reconsider some
> of those points ? Not here perhaps - python-ideas or c.l.p, but if
> bigger carrots are wanted... Just saying. (As time goes on, lack of a
> GIL in Ironpython makes it more attractive for multicore work)

Oh, I don't think the problem is that Python-dev is opposed to it in
principle, but that someone has yet to submit a patch that doesn't
affect single thread preformance and remains compatible with the
C-API.


-- 
Regards,
Benjamin

From solipsis at pitrou.net  Sun Oct 11 23:41:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 11 Oct 2009 21:41:13 +0000 (UTC)
Subject: [Python-Dev] Backport new float repr to Python 2.7?
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
	<20e5c9660910111435o4d2c48e2he3504a7750201fad@mail.gmail.com>
Message-ID: <loom.20091011T234000-241@post.gmane.org>

Michael Sparks <sparks.m <at> gmail.com> writes:
> 
> I know it's the upteen-thousandth time it's been discussed, but
> removal of the GIL in 3.x would probably be pretty big carrots for
> some. I know the arguments [...]

Not before someone produces a patch anyway. It is certainly not as easy as you
seem to think it is.

Regards

Antoine.



From sparks.m at gmail.com  Sun Oct 11 23:49:25 2009
From: sparks.m at gmail.com (Michael Sparks)
Date: Sun, 11 Oct 2009 22:49:25 +0100
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <loom.20091011T234000-241@post.gmane.org>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
	<20e5c9660910111435o4d2c48e2he3504a7750201fad@mail.gmail.com>
	<loom.20091011T234000-241@post.gmane.org>
Message-ID: <20e5c9660910111449w7fea72aw28e9948e5e05595b@mail.gmail.com>

On Sun, Oct 11, 2009 at 10:41 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Michael Sparks <sparks.m <at> gmail.com> writes:
>>
>> I know it's the upteen-thousandth time it's been discussed, but
>> removal of the GIL in 3.x would probably be pretty big carrots for
>> some. I know the arguments [...]
>
> Not before someone produces a patch anyway. It is certainly not as easy as you
> seem to think it is.

You misunderstand me I think. I don't think it's easy, based on all
the discussion I've seen over the years, and the various attempts that
have been made, I think it's a Hard problem. I was just mentioning
that if someone wanted a serious carrot, that would probably be it.
I'll be quiet about it now though, since I *do* understand how
contentious an issue it is.


Michael.

From fwierzbicki at gmail.com  Mon Oct 12 01:31:06 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Sun, 11 Oct 2009 19:31:06 -0400
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <4ACFD86F.3010305@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org> <4ACFD86F.3010305@v.loewis.de>
Message-ID: <4dab5f760910111631j28e8c3b6g6e344a49506654a8@mail.gmail.com>

On Fri, Oct 9, 2009 at 8:42 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I think it is important to confirm in advance that all the
> implementations listed below agree to implement the PEP "soonish" after
> it's adopted. "Required" sounds like a strong term - however, if an
> implementation chooses not to implement the PEP, it can do whatever it
> wants, including omission of required fields.
Speaking for Jython, so far it looks like something we would adopt
soonish after it was accepted (it looks pretty useful to me).

> So I propose that the python.org version is identified as "python".
I'll add my voice to the group that likes "cpython" and "CPython" as
the identifier of the python.org implementation.  This version has a
long history, and "Classic Python" has a nice sound to it.  :) -- also
I hope (but won't hold my breath) that Python becomes more associated
with the abstract language in time.

-Frank

From fwierzbicki at gmail.com  Mon Oct 12 01:36:45 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Sun, 11 Oct 2009 19:36:45 -0400
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <4ACFD680.1080001@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04A96AA3@TK5EX14MBXC116.redmond.corp.microsoft.com>
	<4ACFD680.1080001@v.loewis.de>
Message-ID: <4dab5f760910111636j72d916a0u7cc9aa5a4c7b007@mail.gmail.com>

On Fri, Oct 9, 2009 at 8:34 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Also, why is it the name of the JIT compiler, and not the name of the
> source language compiler?
>From the Jython side it is easier to get the VM name compared to the
source language compiler.  Although there is a property on
java.lang.System called "java.compiler" it is often empty.

-Frank

From guido at python.org  Mon Oct 12 02:40:56 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2009 17:40:56 -0700
Subject: [Python-Dev] PEP about sys.implementation and implementation
	specific user site directory
In-Reply-To: <4dab5f760910111631j28e8c3b6g6e344a49506654a8@mail.gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org> <4ACFD86F.3010305@v.loewis.de> 
	<4dab5f760910111631j28e8c3b6g6e344a49506654a8@mail.gmail.com>
Message-ID: <ca471dc20910111740h346f20d3w24d4a4242022ba10@mail.gmail.com>

On Sun, Oct 11, 2009 at 4:31 PM, Frank Wierzbicki <fwierzbicki at gmail.com> wrote:
> On Fri, Oct 9, 2009 at 8:42 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So I propose that the python.org version is identified as "python".

> I'll add my voice to the group that likes "cpython" and "CPython" as
> the identifier of the python.org implementation. ?This version has a
> long history, and "Classic Python" has a nice sound to it. ?:)

Regardless of the history, "CPython" is indeed how most people refer
to this implementation *whenever the distinction matters*, so in a
variable used to identify the implementation this makes sense to me,
even though most people most of the time don't make this distinction.
I'm not worried about other implementations written in C aspiring to
the name CPython.

> -- also
> I hope (but won't hold my breath) that Python becomes more associated
> with the abstract language in time.

Me too (on both accounts :-).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Mon Oct 12 06:40:21 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 11 Oct 2009 21:40:21 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com><ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
Message-ID: <9CE66A617E6740D48190BB16F0E0D49F@RaymondLaptop1>


[Glyph Lefkowitz ]
> This reasoning definitely makes sense to me; with all the dependency-migration 
> issues 3.x could definitely use some carrots.  However, I don't think I agree with it, 
> because this doesn't feel like a big new feature, just some behavior which has changed.  

The carrots/incentives idea also sounds specious to me. First of all, I consider it 
to be more of a bug fix than a feature -- we've had plenty of bug reports and confusion
surrounding the current implementation and at least one of my scripts is broken 
(occasionally giving wrong answers and the same also is true for decimal.__float__ 
method being similarly afflicted).  Our current dependency on a badly implemented 
libc strtod() function is not a good thing (and not consistent across various Python builds).   
Second, as Glyph points out, the change is too small of an improvement to be a real carrot.

One quick thought on the doctest issue.  If the doctests are being used as originally
intended (as part of validating examples in docstrings), then consider that the docstrings
themselves would actually be improved with the shorter repr.  IMO, it significantly
detracts from examples if they are afflicted with floating point repr issues:

    def average(seq):
         """ Return the arithmetic mean of a sequence.

         >>> average([0.1, 0.5])
         0.29999999999999999
 
         """
         return sum(seq) / float(len(seq))

Wouldn't this example be much nicer if it returned 0.3 ?


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/1cf7bd4d/attachment.htm>

From guido at python.org  Mon Oct 12 08:36:05 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2009 23:36:05 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com> 
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
Message-ID: <ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com>

On Sun, Oct 11, 2009 at 12:44 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Mark Dickinson]
>>
>> - string to float *and* float to string conversions are both guaranteed
>> ?correctly rounded in 3.x: David Gay's code implements the conversion
>> ?in both directions, and having correctly rounded string -> float
>> ?conversions is essential to ensure that eval(repr(x)) recovers x exactly.
>
> IMO, that is so important that it should be considered a bug fix.

It looks like the majority of posters so far is +0 to +1 on this, and
I don't want my -0 to stand in the way of progress.

But make no mistake, it *is* a feature, not a bug fix. It took a
multidisciplinary team a year to implement it -- from existing
portable code in C! Mark wrote easily a page in favor of introducing
it. Those are signs of a Feature. The original behavior was no more a
bug than the distinction between int and long, or the existence of
classic and new classes, or the GIL. These were all deliberate design
decisions made as compromises between different requirements.

And make no mistake, I like this feature in 3.1. Just as I like the
new int type, the new str type, the new classes, the new rich
comparisons. The new f.p. repr() may look like a small feature -- but
it's a feature that few other languages have!

If I had a doctest that depended on the vagaries of f.p. repr(), I
would probably change the doctest to use simpler numbers; getting
1./3.+1./3. to be exactly 2./3. is hard enough, but I might rewrite my
test to use 1/2. + 1/2. = 1. instead. So the doctest argument isn't
all too strong.

Carrot-wise the feature is quite strong (see Mark's page of arguments
in favor) -- but that works also in favor of inclusion on 2.7.

In the end somebody needs to do the work. If it can reuse much of the
work done for 3.1, then hopefully it will be done in much less time
than the original was done. It looks like someone wants to do this
work. I wish them well.

But doing this work will mean not doing other work on 2.7 that may be
more productive.

PS. str(x) still seems to be using %.12g -- shouldn't it be made equal
to repr() in 3.1 or 3.2? *That* I would call a bug, an oversight.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Mon Oct 12 10:25:35 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 12 Oct 2009 10:25:35 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4ACFD86F.3010305@v.loewis.de>
References: <haoh1j$9c7$1@ger.gmane.org> <4ACFD86F.3010305@v.loewis.de>
Message-ID: <4AD2E7FF.8070700@egenix.com>

"Martin v. L?wis" wrote:
>> id (required):
>>   lower case identifier, for example "cpython", "ironpython", "jython",
>> "pypy"
> 
> Doing some bike-shedding: I'd like to not use "cpython" as the name of
> the python.org implementation. This term, I believe, was coined around
> JPython, somehow making the implementation language the primary means
> of distinction. However, there may be alternative implementations
> written in C (e.g. unladen swallow), or otherwise be "C based" (e.g.
> pypy).
> 
> So I propose that the python.org version is identified as "python".

The CPython name is the traditional and commonly used name of the
python.org C implementation. Even though the name is inspired by
using C as implementation language, it doesn't refer to CPython
being the only implementation of Python in C.

I don't think there's a need to do any bike shedding on well
established names.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 12 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Mon Oct 12 10:55:12 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 12 Oct 2009 10:55:12 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <4AD2EEF0.1080506@egenix.com>

Christian Heimes wrote:
> sys.implementation
> ------------------
> 
> platform (required):
>   platform or language of the implementation, for example "C", ".NET",
> "Java"

I'd call this attribute "language". We already have sys.platform with
a different meaning.

Possible values would then be "C", "C#", "Java", "Python", etc.

> runtime (required):
>   lower case runtime library of the implementation, for example "libc",
> "msvcrt90", "java6", ".net"
> 
> compiler (required):
>   verbose name of the compiler, for example "GCC 4.3.3", "Java
> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"

If you include the compiler as well, it may make sense to
also add the other creation details that platform.py parses
from the sys.version string:

branch, revision, buildno, builddate

> ...
> 
> 
> sys.userdirsuffix
> -----------------
> 
> sys.userdirsuffix is an implementation and platform specific string that
> is used to construct the path for the user site directory as explained
> in PEP 370. The string contains the implementation name as well as the
> version number of Python.

Don't we already have this information available as site.getuserbase()/
site.getusersitepackages() ?

> Examples:
>   python2.6 (CPython, Unix)
>   Python26 (CPython, Windows)
>   ironpython2.6 (IronPython, Unix)
>   IronPython26 (IronPython, Windows)
>   ...
> 

Since the examples point to the name of the Python stdlib dir
and this form is also used in a lot of other places to distinguish
different Python versions (e.g. in distutils build dirs), perhaps
we should use a more generic name for it like e.g. sys.versiondir ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 12 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From scott+python-dev at scottdial.com  Mon Oct 12 11:38:56 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Mon, 12 Oct 2009 05:38:56 -0400
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <haoh1j$9c7$1@ger.gmane.org>
References: <haoh1j$9c7$1@ger.gmane.org>
Message-ID: <4AD2F930.9050104@scottdial.com>

Christian Heimes wrote:
> id (required):
>   lower case identifier, for example "cpython", "ironpython", "jython",
> "pypy"
> 
> name (required):
>   mixed case name of the implementation, for example "CPython",
> "IronPython", "Jython", "PyPy"

Why both? Is it not true that the following is guaranteed?

sys.implementation.id == sys.implementation.name.lower()

Furthermore, why is a lower-case-only constant superior to a mixed-case
identifier? I wonder this especially given that
platform.python_implementation() already returns mixed-case constants.

Why not include the version of the language?

> platform (required):
>   platform or language of the implementation, for example "C", ".NET",
> "Java"

What should someone like PyPy say here? Do you have a use-case in mind?
What is the relevance of this field?

> runtime (required):
>   lower case runtime library of the implementation, for example "libc",
> "msvcrt90", "java6", ".net"

Same questions as above. Is the java-runtime not dependent on libc?

$ ldd /opt/sun-jdk-1.6.0.15/jre/bin/java | grep libc
        libc.so.6 => /lib/libc.so.6 (0xb7d70000)

> compiler (required):
>   verbose name of the compiler, for example "GCC 4.3.3", "Java
> HotSpot(TM) Client VM (Apple Inc.)", "Mono JIT compiler version 2.0.1"

I think other commenters have already asked what exactly is considered
the "compiler"? This is especially interesting if you consider PyPy, as
well. What is the item being compiled that you are interested in? What
is the use-case for this info? It could vary based on module: some
modules could've been built with a different compiled or even compiled
to bytecode by a different implementation or etc.

It seems like (platform, runtime, compiler) are guesses at what *might*
be useful. But, in the absence of use-cases they are not useful, perhaps
misguided, and create a maintenance burden for the future. A lot of this
info is available via the platform module. And the only reason given to
not use the platform module was to avoid adding dependencies for
site.py. I don't see that as an open invitation to adding other fields
that are already available via the platform module. Until they are
critical to run-time performance, why not wait to add these extra things?

The only thing that has been indicated as needed is the identifier for
the python implementation. sys.vm or sys.implementation may very well
fully support the use cases given merely by being a string.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From david.lyon at preisshare.net  Mon Oct 12 14:08:12 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 12 Oct 2009 08:08:12 -0400
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <4ACF6993.5040704@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<nad-B71566.14284708102009@ger.gmane.org>	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
Message-ID: <af8457f8c5b391ed2a69d8a0f7c611af@preisshare.net>

On Fri, 09 Oct 2009 17:49:23 +0100, Michael Foord
> I wonder if it is going to be possible to make this compatible with the 
> upcoming distutils package management 'stuff' (querying for installed 
> packages, uninstallation etc) since installation/uninstallation goes 
> through the Windows system package management feature.  I guess it would 
> be eminently possible but require some reasonably high level Windows-fu 
> to do.

It's a good question.

Searching on it turned up no easy way of doing control panel applications
in python.

A much easier way would be to just mount a package manager application
in the "Programs" + "Python X.Y" menu. Users would surely find it there.

Regards

David


From fetchinson at googlemail.com  Mon Oct 12 14:52:12 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Mon, 12 Oct 2009 14:52:12 +0200
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<fb73205e0910080551v6a27caa3u3657978ae67aa977@mail.gmail.com>
	<fb73205e0910080552s13ab6a1q76d73aea419a3964@mail.gmail.com>
Message-ID: <fbe2e2100910120552k57454fbnff8a78b05e307558@mail.gmail.com>

>> My opinion is that this tool exists only because Python doesn't
>> support the installation of multiple versions for the same
>> distributions.
>
> This is not at all how I use virtualenv. For me virtualenv is a
> sandbox so that I don't have to become root whenever I need to install
> a Python package for testing purposes and to allow me to hop between
> sets of installed Python packages while developing on multiple Python
> projects. I also use it as a sandbox for build bots so that multiple
> bots on the same machine can each build their own projects with just
> the known dependencies installed.

+1

I also don't use virtualenv for supporting multiple versions, but
rather for sandboxing, testing and experimentation. To make sure that
an app works with the exact dependencies I think it should work with.
Also, it's important that the 'global' site-packages typically
requires root privileges while installing to a virtualenv doesn't.

Cheers,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From eric at trueblade.com  Mon Oct 12 17:00:53 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 12 Oct 2009 11:00:53 -0400
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>	<ca471dc20910111248h3067468dj44b389ff6329f6d5@mail.gmail.com>
	<d9047f780910111300n3650ab4dx40f26ccf60d9db1f@mail.gmail.com>
Message-ID: <4AD344A5.6060605@trueblade.com>

Glyph Lefkowitz wrote:

> I'd much rather have my doctests and float-repr'ing code break on 2.7 so 
> I can deal with it as part of a minor-version upgrade than have it break 
> on 3.x and have to deal with this at the same time as the unicode->str 
> explosion.  It feels like a backport of this behavior would make the 
> 2->3 transition itself a little easier.

I'm +0 on making the change, primarily for this reason.

We can reuse the vast majority of the code from 3.x.

Eric.

From dickinsm at gmail.com  Mon Oct 12 20:41:44 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 12 Oct 2009 19:41:44 +0100
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
	<ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com>
Message-ID: <5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>

[Guido]
> I think you mean doctests? These are the primary reason I've always
> been hesitant to change this in 2.x.

Yes, sorry. I did of course mean doctests.

It occurs to me that any doctests that depend on the precise form of
repr(x) are, in a sense, already broken, since 2.x makes no guarantees
about repr(x) being consistent across platforms.  It's just an accident
that repr(x) in 2.x pretty much *is* consistent across major platforms,
so long as you steer clear of IEEE 754 oddities like subnormals, nans
and infinities.

[Glyph]
> I'd much rather have my doctests and float-repr'ing code break on
> 2.7 so I can deal with it as part of a minor-version upgrade than
> have it break on 3.x and have to deal with this at the same time
> as the unicode->str explosion.  It feels like a backport of this
> behavior would make the 2->3 transition itself a little easier.

I hadn't really thought about this aspect.  I find this quite convincing.

[Guido]
> PS. str(x) still seems to be using %.12g -- shouldn't it be made equal
> to repr() in 3.1 or 3.2? *That* I would call a bug, an oversight.

But str still has some value in py3k:  it protects users from
accumulated rounded errors produced by arithmetic operations:

Python 3.2a0 (py3k:75216:75220, Oct  3 2009, 21:38:04)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 0.1 + 0.2
0.30000000000000004
>>> 1.23 * 4.64
5.707199999999999
>>> str(0.1 + 0.2)
'0.3'
>>> str(1.23 * 4.64)
'5.7072'

I don't know whether this makes it worth keeping str different from repr.

Mark

From guido at python.org  Mon Oct 12 20:48:54 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Oct 2009 11:48:54 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com> 
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
	<ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com> 
	<5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>
Message-ID: <ca471dc20910121148ve9d2775l48ecf8ca661a5685@mail.gmail.com>

On Mon, Oct 12, 2009 at 11:41 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> [Guido]
>> PS. str(x) still seems to be using %.12g -- shouldn't it be made equal
>> to repr() in 3.1 or 3.2? *That* I would call a bug, an oversight.
>
> But str still has some value in py3k: ?it protects users from
> accumulated rounded errors produced by arithmetic operations:
[...]

I know, but this is much more questionable now. Making str() and
repr() different was an intentional choice when repr() would often
show ugly values even if the input was pretty; but now str() just
hides real differences (1.3 != 1.2 + 1.1), and the difference between
str() and repr() is pretty subtle. (Show of hands for those who didn't
even know there was a difference? :-). I think if we had to start from
scratch with the 3.1 float repr() I would have made float's str()
equal to its repr(). But this is at most a +1 feeling.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Mon Oct 12 20:54:44 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 12 Oct 2009 19:54:44 +0100
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <ca471dc20910121148ve9d2775l48ecf8ca661a5685@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
	<ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com>
	<5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>
	<ca471dc20910121148ve9d2775l48ecf8ca661a5685@mail.gmail.com>
Message-ID: <5c6f2a5d0910121154w431b9f1aoa5b51dfdfabfec86@mail.gmail.com>

On Mon, Oct 12, 2009 at 7:48 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Oct 12, 2009 at 11:41 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> But str still has some value in py3k: ?it protects users from
>> accumulated rounded errors produced by arithmetic operations:
> [...]
>
> I know, but this is much more questionable now. [...]

Would it be out of the question to make str = repr in 3.2, do you think?

Mark

From guido at python.org  Mon Oct 12 21:09:36 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Oct 2009 12:09:36 -0700
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <5c6f2a5d0910121154w431b9f1aoa5b51dfdfabfec86@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com> 
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
	<ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com> 
	<5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com> 
	<ca471dc20910121148ve9d2775l48ecf8ca661a5685@mail.gmail.com> 
	<5c6f2a5d0910121154w431b9f1aoa5b51dfdfabfec86@mail.gmail.com>
Message-ID: <ca471dc20910121209o100f2874nbb79c9b9d468ea12@mail.gmail.com>

On Mon, Oct 12, 2009 at 11:54 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Mon, Oct 12, 2009 at 7:48 PM, Guido van Rossum <guido at python.org> wrote:
>> On Mon, Oct 12, 2009 at 11:41 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>> But str still has some value in py3k: ?it protects users from
>>> accumulated rounded errors produced by arithmetic operations:
>> [...]
>>
>> I know, but this is much more questionable now. [...]
>
> Would it be out of the question to make str = repr in 3.2, do you think?

I don't think it is out of the question. (Though it would probably
break a few more doctests.)

I'd like to hear other voices.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.peters at gmail.com  Mon Oct 12 21:23:33 2009
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 12 Oct 2009 15:23:33 -0400
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com> 
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
	<ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com> 
	<5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>
Message-ID: <1f7befae0910121223i25539a1aqce7492b851e40b49@mail.gmail.com>

[Mark Dickinson]
> It occurs to me that any doctests that depend on the precise form of
> repr(x) are, in a sense, already broken, since 2.x makes no guarantees
> about repr(x) being consistent across platforms.

The doctest documentation has warned about this forever (look near the
end of the "Warnings" section,

    http://docs.python.org/library/doctest.html#warnings

).


>?It's just an accident that repr(x) in 2.x pretty much *is* consistent across
> major platforms, so long as you steer clear of IEEE 754 oddities like subnormals,
> nans and infinities.

If you don't consider Windows to be a major platform ;-)  Besides that
there's just no guessing what the Microsoft double->string routines
will produce for the 17th digit, the MS routines always produce 3
digits for the exponent in scientific notation, while AFAIK all other
platforms produce 2 digits when 3 aren't necessary.  For example, on
Windows under Python 2.x:

>>> repr(1e47)
'1e+047'
>>> repr(1e-47)
'9.9999999999999997e-048'

and "everywhere else":

>>> repr(1e47)
'1e+47'
>>> repr(1e-47)
'9.9999999999999997e-48'

The leading 0 in the Window's exponents is enough to break a naive
doctest too -- and repr(x) does produce scientific notation for
"almost all" finite doubles.  Why people are obsessed with the
relative handful of doubles between 1 and 1000 is beyond me ;-)

As doctest's original author, I have no sympathy for users who burn
themselves with float output.  Of course it's open to question whether
I have sympathy for anyone, ever, but that's not the issue here ;-)

From eric at trueblade.com  Mon Oct 12 21:42:52 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 12 Oct 2009 15:42:52 -0400 (EDT)
Subject: [Python-Dev] Backport new float repr to Python 2.7?
In-Reply-To: <1f7befae0910121223i25539a1aqce7492b851e40b49@mail.gmail.com>
References: <5c6f2a5d0910111128j672381b3k507be922a7582a3f@mail.gmail.com>
	<BCA6EB4B3A2A460BB09AAB5CEE0D2874@RaymondLaptop1>
	<ca471dc20910112336m19bbf1a7x740101f485254eff@mail.gmail.com>
	<5c6f2a5d0910121141u65da9668r73529489fd3591d2@mail.gmail.com>
	<1f7befae0910121223i25539a1aqce7492b851e40b49@mail.gmail.com>
Message-ID: <24288.63.251.87.214.1255376572.squirrel@mail.trueblade.com>

[Tim:]
> If you don't consider Windows to be a major platform ;-)  Besides that
> there's just no guessing what the Microsoft double->string routines
> will produce for the 17th digit, the MS routines always produce 3
> digits for the exponent in scientific notation, while AFAIK all other
> platforms produce 2 digits when 3 aren't necessary.  For example, on
> Windows under Python 2.x:
>
>>>> repr(1e47)
> '1e+047'
>>>> repr(1e-47)
> '9.9999999999999997e-048'
>
> and "everywhere else":
>
>>>> repr(1e47)
> '1e+47'
>>>> repr(1e-47)
> '9.9999999999999997e-48'

Not that it's germane to the discussion, but this 3 digit exponent on
Windows was fixed in 2.6.

> The leading 0 in the Window's exponents is enough to break a naive
> doctest too -- and repr(x) does produce scientific notation for
> "almost all" finite doubles.  Why people are obsessed with the
> relative handful of doubles between 1 and 1000 is beyond me ;-)
>
> As doctest's original author, I have no sympathy for users who burn
> themselves with float output.  Of course it's open to question whether
> I have sympathy for anyone, ever, but that's not the issue here ;-)

I agree, we shouldn't base the decision to change this based on doctests.

From tseaver at palladion.com  Mon Oct 12 23:36:49 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 12 Oct 2009 17:36:49 -0400
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
Message-ID: <4AD3A171.6070906@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:

> = Virtualenv and the multiple version support in Distribute =
> 
> (I am not saying "We" here because this part was not discussed yet
> with everyone)
> 
> Virtualenv allows you to create an isolated environment to install
> some distribution without polluting the
> main site-packages, a bit like a user site-packages.
> 
> My opinion is that this tool exists only because Python doesn't
> support the installation of multiple versions for the same
> distributions.
> But if PEP 376 and PEP 386 support are added in Python, we're not far
> from being able to provide multiple version support with
> the help of importlib.

I couldn't disagree more strongly:  I don't *want* multiple version
support:  compared to the simplicity of a virtualenv, adding such a
complex feature would be a huge backward step.

The idea that the "system" python should be polluted with whatever-any-
random-app-decides-to-throw-into-it is a source of endless pain,
confusion, and hair loss.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrToW0ACgkQ+gerLs4ltQ7fKgCdEPxU6JxKOKCFWtpQSks606DQ
XeAAoICgFZ8RURvFH4p3dLTzqq51ROqP
=e71z
-----END PGP SIGNATURE-----


From greg at krypto.org  Tue Oct 13 07:55:28 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 12 Oct 2009 22:55:28 -0700
Subject: [Python-Dev] Top-posting on this list
In-Reply-To: <hat7v8$3sb$1@ger.gmane.org>
References: <4ACDCA5A.4090609@cheimes.de>
	<bbaeab100910091143o705d46fbn83628b8dea1573a8@mail.gmail.com> 
	<87r5tblkp0.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AD120B0.5080100@canterbury.ac.nz> 
	<873a5qlfni.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fx9qrx2t.fsf@benfinney.id.au> 
	<87r5t9khs8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4222a8490910110955s5d9d504br428b0fa27b27c6f7@mail.gmail.com> 
	<1afaf6160910111037u10a4e818w1679a24266adf1bd@mail.gmail.com> 
	<hat7v8$3sb$1@ger.gmane.org>
Message-ID: <52dc1c820910122255r6328f44ax58e0dac22f36e4d2@mail.gmail.com>

Those who feel diverse can top post.

On Sun, Oct 11, 2009 at 11:23 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>>> Is it really that big of an issue that we have to discuss it
>>> ad-infinitum and potentially have a quoting cop? Sometimes top-posting
>>> happens. Sometimes people don't trim messages. Sometimes people argue
>>> about the color of a shed.
>>
>> +1 This is really getting so meta that it's off topic.
>
> quoting-SIG anyone?

Those who don't can post below.

From ncoghlan at gmail.com  Tue Oct 13 11:49:44 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 13 Oct 2009 19:49:44 +1000
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4AD2EEF0.1080506@egenix.com>
References: <haoh1j$9c7$1@ger.gmane.org> <4AD2EEF0.1080506@egenix.com>
Message-ID: <4AD44D38.1020608@gmail.com>

M.-A. Lemburg wrote:
>> sys.userdirsuffix
>> -----------------
>>
>> sys.userdirsuffix is an implementation and platform specific string that
>> is used to construct the path for the user site directory as explained
>> in PEP 370. The string contains the implementation name as well as the
>> version number of Python.
> 
> Don't we already have this information available as site.getuserbase()/
> site.getusersitepackages() ?

No, the new attribute is intended to allow a VM implementation to
customise the way site.py calculates the user directories so that
CPython, Jython, IronPython, etc, don't have to share the same directory.

CPython will stick with the directory as named in PEP 370 (since we
already released a version with that behaviour), but the other
implementations will be able to choose a name that doesn't conflict with it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From mal at egenix.com  Tue Oct 13 12:27:57 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 12:27:57 +0200
Subject: [Python-Dev] PEP about sys.implementation and implementation
 specific user site directory
In-Reply-To: <4AD44D38.1020608@gmail.com>
References: <haoh1j$9c7$1@ger.gmane.org> <4AD2EEF0.1080506@egenix.com>
	<4AD44D38.1020608@gmail.com>
Message-ID: <4AD4562D.9030104@egenix.com>

Nick Coghlan wrote:
> M.-A. Lemburg wrote:
>>> sys.userdirsuffix
>>> -----------------
>>>
>>> sys.userdirsuffix is an implementation and platform specific string that
>>> is used to construct the path for the user site directory as explained
>>> in PEP 370. The string contains the implementation name as well as the
>>> version number of Python.
>>
>> Don't we already have this information available as site.getuserbase()/
>> site.getusersitepackages() ?
> 
> No, the new attribute is intended to allow a VM implementation to
> customise the way site.py calculates the user directories so that
> CPython, Jython, IronPython, etc, don't have to share the same directory.
> 
> CPython will stick with the directory as named in PEP 370 (since we
> already released a version with that behaviour), but the other
> implementations will be able to choose a name that doesn't conflict with it.

Ah, so it's intended to provide information to site.getuserbase()/
site.getusersitepackages() ?

Still, since that path name component is used in other contexts
as well, wouldn't it be better to have all those contexts use the
name ?!

Python has always used the (lib|include)/pythonX.Y/ path naming
scheme on Unix, but it's not codified anywhere. A sys attribute
or a set of such attributes would be good way to do so.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Tue Oct 13 14:23:35 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 08:23:35 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <FDE6DC02-40CC-49DB-BDEA-B9C47ABCF359@zope.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
	<FDE6DC02-40CC-49DB-BDEA-B9C47ABCF359@zope.com>
Message-ID: <CD63DF12-986A-4D12-89E9-BFF78E0AA786@python.org>

On Oct 8, 2009, at 8:45 AM, Zvezdan Petkovic wrote:

> On Oct 7, 2009, at 2:09 PM, Barry Warsaw wrote:
>> I want us to be very careful about 2.6.4.  This isn't a normal bug  
>> fix release, it's specifically there to remove the brown paper bag  
>> of 2.6.3 from our heads.  So let's be conservative and fix this one  
>> in 2.6.5.
>
> Can we get the readline patch reviewed as well.  It was applied to  
> trunk already: http://svn.python.org/view?view=rev&revision=74970
>
> It's marked for backport to 2.6 as "needs review":
> http://bugs.python.org/issue6877
>
> It fixes a BusError crash that was just swiped under a rug by  
> disabling the build in setup.py.  :-)

I'm disinclined to approve this for 2.6.4 since it is not a regression  
in 2.6.3.  It seems fine to me for 2.6.5 though.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/204c47c9/attachment.pgp>

From barry at python.org  Tue Oct 13 14:27:01 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 08:27:01 -0400
Subject: [Python-Dev] On track for Python 2.6.4 final this Sunday?
Message-ID: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>

Are we on track to release 2.6.4 final this Sunday or do we need  
another rc?

Yesterday, Tarek committed another setuptools related fix and said  
that he was going to run a bunch of build tests locally.  Tarek, how  
did that go?

Please note that issue 7064 is still open as a release blocker.  Can  
we close that (again) now?

http://bugs.python.org/issue7064

I am still being very conservative about what goes in 2.6.4.  Only  
regressions introduced in 2.6.3 are being accepted.  There's plenty of  
time after that for patches to go in for 2.6.5.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/fe1891d5/attachment.pgp>

From zvezdan at zope.com  Tue Oct 13 14:31:00 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Tue, 13 Oct 2009 08:31:00 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <CD63DF12-986A-4D12-89E9-BFF78E0AA786@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
	<FDE6DC02-40CC-49DB-BDEA-B9C47ABCF359@zope.com>
	<CD63DF12-986A-4D12-89E9-BFF78E0AA786@python.org>
Message-ID: <ABB15763-8A8B-441E-B383-3125DED84B05@zope.com>

On Oct 13, 2009, at 8:23 AM, Barry Warsaw wrote:

> On Oct 8, 2009, at 8:45 AM, Zvezdan Petkovic wrote:
>
>> On Oct 7, 2009, at 2:09 PM, Barry Warsaw wrote:
>>> I want us to be very careful about 2.6.4.  This isn't a normal bug  
>>> fix release, it's specifically there to remove the brown paper bag  
>>> of 2.6.3 from our heads.  So let's be conservative and fix this  
>>> one in 2.6.5.
>>
>> Can we get the readline patch reviewed as well.  It was applied to  
>> trunk already: http://svn.python.org/view?view=rev&revision=74970
>>
>> It's marked for backport to 2.6 as "needs review":
>> http://bugs.python.org/issue6877
>>
>> It fixes a BusError crash that was just swiped under a rug by  
>> disabling the build in setup.py.  :-)
>
> I'm disinclined to approve this for 2.6.4 since it is not a  
> regression in 2.6.3.  It seems fine to me for 2.6.5 though.

Excellent.  That's exactly what I meant.
I quoted the part of the previous message where you proposed to review  
another patch for 2.6.5.  I guess it was not very clear that I'm  
proposing to review this for 2.6.5 as well.  Well, at least we're on  
the same page now it seems.  :-)

Thanks,

	Zvezdan


From barry at python.org  Tue Oct 13 15:48:41 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 09:48:41 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <ABB15763-8A8B-441E-B383-3125DED84B05@zope.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
	<FDE6DC02-40CC-49DB-BDEA-B9C47ABCF359@zope.com>
	<CD63DF12-986A-4D12-89E9-BFF78E0AA786@python.org>
	<ABB15763-8A8B-441E-B383-3125DED84B05@zope.com>
Message-ID: <8F38C15F-3A1C-4825-A38D-6242CC04B56E@python.org>

On Oct 13, 2009, at 8:31 AM, Zvezdan Petkovic wrote:

> Excellent.  That's exactly what I meant.
> I quoted the part of the previous message where you proposed to  
> review another patch for 2.6.5.  I guess it was not very clear that  
> I'm proposing to review this for 2.6.5 as well.  Well, at least  
> we're on the same page now it seems.  :-)

:)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/eee9c27c/attachment.pgp>

From mal at egenix.com  Tue Oct 13 16:16:36 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 16:16:36 +0200
Subject: [Python-Dev] On track for Python 2.6.4 final this Sunday?
In-Reply-To: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
Message-ID: <4AD48BC4.2060809@egenix.com>

Barry Warsaw wrote:
> Are we on track to release 2.6.4 final this Sunday or do we need another
> rc?
> 
> Yesterday, Tarek committed another setuptools related fix and said that
> he was going to run a bunch of build tests locally.  Tarek, how did that
> go?
> 
> Please note that issue 7064 is still open as a release blocker.  Can we
> close that (again) now?
> 
> http://bugs.python.org/issue7064
> 
> I am still being very conservative about what goes in 2.6.4.  Only
> regressions introduced in 2.6.3 are being accepted.  There's plenty of
> time after that for patches to go in for 2.6.5.

It would be nice to get this issue resolved out for 2.6.4:

http://bugs.python.org/issue4120

The problem is that extensions built with 2.6.x will not work
when used with a User-only installation of Python on machines that
don't already have the MS VC90 CRT DLLs installed system-wide.

There's a patch available that appears to work, but someone with
more manifest-Windows-DLL-fu than me will need to have a look at it.

One possible approach would be to use the patch in 2.6.4 and then
continue digging deeper until 2.6.5 is released.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From martin at v.loewis.de  Tue Oct 13 16:41:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Oct 2009 22:41:01 +0800
Subject: [Python-Dev] [python-committers] On track for Python
 2.6.4	final this Sunday?
In-Reply-To: <4AD48BC4.2060809@egenix.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<4AD48BC4.2060809@egenix.com>
Message-ID: <4AD4917D.8090800@v.loewis.de>


> It would be nice to get this issue resolved out for 2.6.4:
> 
> http://bugs.python.org/issue4120
> 
> The problem is that extensions built with 2.6.x will not work
> when used with a User-only installation of Python on machines that
> don't already have the MS VC90 CRT DLLs installed system-wide.

As this bug already exists in 2.6.2, I don't think the change is
eligible for 2.6.4.

In addition, I want to review it, which I won't be able to until
Sunday.

> One possible approach would be to use the patch in 2.6.4 and then
> continue digging deeper until 2.6.5 is released.

This also speaks against the patch. Anything being changed in this area
ideally should be the final state of affairs for the rest of 2.6.x.

Regards,
Martin


From pje at telecommunity.com  Tue Oct 13 16:52:02 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 13 Oct 2009 10:52:02 -0400
Subject: [Python-Dev] On track for Python 2.6.4 final this Sunday?
In-Reply-To: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
Message-ID: <20091013145211.8DB243A4068@sparrow.telecommunity.com>

At 08:27 AM 10/13/2009 -0400, Barry Warsaw wrote:
>Are we on track to release 2.6.4 final this Sunday or do we need
>another rc?
>
>Yesterday, Tarek committed another setuptools related fix and said
>that he was going to run a bunch of build tests locally.  Tarek, how
>did that go?

FWIW, that change won't work to fix the problem on Windows if a 
package's setup.py is written with cross-platform paths (i.e., the usual case).

Also, for the record, it's not really a setuptools-related fix; it's 
just attempting to fix collateral damage caused by the attempt to fix 
the problems caused by the original API change...  which affected all 
build_ext customizations relying on the docstring of the changed 
method, not just the one in setuptools.


From mal at egenix.com  Tue Oct 13 17:01:05 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 17:01:05 +0200
Subject: [Python-Dev] [python-committers] On track for Python
 2.6.4	final this Sunday?
In-Reply-To: <4AD4917D.8090800@v.loewis.de>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>
	<4AD4917D.8090800@v.loewis.de>
Message-ID: <4AD49631.70607@egenix.com>

"Martin v. L?wis" wrote:
> 
>> It would be nice to get this issue resolved out for 2.6.4:
>>
>> http://bugs.python.org/issue4120
>>
>> The problem is that extensions built with 2.6.x will not work
>> when used with a User-only installation of Python on machines that
>> don't already have the MS VC90 CRT DLLs installed system-wide.
> 
> As this bug already exists in 2.6.2, I don't think the change is
> eligible for 2.6.4.
> 
> In addition, I want to review it, which I won't be able to until
> Sunday.

Then I'd suggest to wait another week with 2.6.4 to give you a
chance to look at the patch.

>> One possible approach would be to use the patch in 2.6.4 and then
>> continue digging deeper until 2.6.5 is released.
> 
> This also speaks against the patch. Anything being changed in this area
> ideally should be the final state of affairs for the rest of 2.6.x.

Of course, but we don't live in an ideal world, otherwise we would
have noticed before releasing 2.6 :-)

I guess the problem is that developer typically already have the
MS VC90 CRT DLLs installed system-wide, so they simply don't notice.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Tue Oct 13 17:16:19 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 13 Oct 2009 17:16:19 +0200
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <20091013145211.8DB243A4068@sparrow.telecommunity.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
Message-ID: <94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>

On Tue, Oct 13, 2009 at 4:52 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 08:27 AM 10/13/2009 -0400, Barry Warsaw wrote:
>>
>> Are we on track to release 2.6.4 final this Sunday or do we need
>> another rc?
>>
>> Yesterday, Tarek committed another setuptools related fix and said
>> that he was going to run a bunch of build tests locally. ?Tarek, how
>> did that go?
>

I still need to do some more tests, I didn't have time to try the
various projects under win32.
It's planned to night.

The tests are consisting of compiling and insatling a dozain of
projects on linux and win32 (and bith when possible)

> FWIW, that change won't work to fix the problem on Windows if a package's
> setup.py is written with cross-platform paths (i.e., the usual case).
>
> Also, for the record, it's not really a setuptools-related fix; it's just
> attempting to fix collateral damage caused by the attempt to fix the
> problems caused by the original API change... ?which affected all build_ext
> customizations relying on the docstring of the changed method, not just the
> one in setuptools.

Yes the doctest was pretty fuzzy about what is an extension name. It's
will be improved in 2.7.

The current code is doing os.path.join()'s to join the Extension name
with the build path,
leading to the collateral damage you are mentioning. To fix the
problem, the API has to be as permissive
as it was before.

I have already three or four tests to fix that problem in
test_build_ext. If you can take a look at them and
think of any other test that would be missing, that would be
appreciated. Another pair of eye is never enough.

Tarek

From martin at v.loewis.de  Tue Oct 13 17:20:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Oct 2009 23:20:21 +0800
Subject: [Python-Dev] [python-committers] On track for Python
 2.6.4	final this Sunday?
In-Reply-To: <4AD49631.70607@egenix.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>
	<4AD4917D.8090800@v.loewis.de> <4AD49631.70607@egenix.com>
Message-ID: <4AD49AB5.9040703@v.loewis.de>


>> As this bug already exists in 2.6.2, I don't think the change is
>> eligible for 2.6.4.
>>
>> In addition, I want to review it, which I won't be able to until
>> Sunday.
> 
> Then I'd suggest to wait another week with 2.6.4 to give you a
> chance to look at the patch.

That won't make the change more eligible.

> Of course, but we don't live in an ideal world, otherwise we would
> have noticed before releasing 2.6 :-)

Oh, I did notice. I had been cautioning for years that switching to
a newer VS version (2005 or later) would cause severe problems. We
are still trying to recover from the switch to VS 2008. That said,
staying with VS 2003 really hadn't been an option, either. It's
just said that Microsoft has created a new DLL hell in their attempt
to fix the old one, and that they fail to admit it (assuming that
everybody should be using .NET, anyway).

Regards,
Martin


From pje at telecommunity.com  Tue Oct 13 17:30:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 13 Oct 2009 11:30:26 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
 final this Sunday?
In-Reply-To: <94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.co
 m>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
Message-ID: <20091013153037.3EB2A3A4068@sparrow.telecommunity.com>

At 05:16 PM 10/13/2009 +0200, Tarek Ziad? wrote:
>Yes the doctest was pretty fuzzy about what is an extension name. It's
>will be improved in 2.7.
>
>The current code is doing os.path.join()'s to join the Extension name
>with the build path,
>leading to the collateral damage you are mentioning. To fix the
>problem, the API has to be as permissive
>as it was before.
>
>I have already three or four tests to fix that problem in
>test_build_ext. If you can take a look at them and
>think of any other test that would be missing, that would be
>appreciated. Another pair of eye is never enough.

One identical to test_build_ext_path_with_os_sep, but that explicitly 
uses a '/' (rather than os.sep) will identify the problem I'm 
referring to, when run on Windows.

It's common practice to use /-separated paths in setup scripts, 
regardless of platform.  So, your current fix (converting os.sep to 
'.') will work on Linux, Mac, etc., but fail on Windows when run with 
the same setup.py, since os.sep is a backslash there.

(Just as a side note, if when you split off issue 7115 you'd said 
what the new issue number was in 7064, or copied me to the nosy-list 
on the new issue, I'd have been able to review this change and 
comment on it yesterday instead of today, and I'd have done it in the 
bug tracker rather than via Python-Dev.)


From ziade.tarek at gmail.com  Tue Oct 13 17:39:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 13 Oct 2009 17:39:10 +0200
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <20091013153037.3EB2A3A4068@sparrow.telecommunity.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
	<20091013153037.3EB2A3A4068@sparrow.telecommunity.com>
Message-ID: <94bdd2610910130839jf0b653fg231d1ed4c5236c10@mail.gmail.com>

On Tue, Oct 13, 2009 at 5:30 PM, P.J. Eby <pje at telecommunity.com> wrote:
>
> One identical to test_build_ext_path_with_os_sep, but that explicitly uses a
> '/' (rather than os.sep) will identify the problem I'm referring to, when
> run on Windows.
>
> It's common practice to use /-separated paths in setup scripts, regardless
> of platform. ?So, your current fix (converting os.sep to '.') will work on
> Linux, Mac, etc., but fail on Windows when run with the same setup.py, since
> os.sep is a backslash there.

Ok I will have a look at  this tonite. Thanks. Notice that this changed I made
to fix another bug opened a can of worms because Extension is not enough/clearly
documented in this regards.

The goal for 2.7/3.2 is to come up with one single way to define extensions,
and I will probably just add deprecation warnings  for the other forms.

>
> (Just as a side note, if when you split off issue 7115 you'd said what the
> new issue number was in 7064, or copied me to the nosy-list on the new
> issue, I'd have been able to review this change and comment on it yesterday
> instead of today, and I'd have done it in the bug tracker rather than via
> Python-Dev.)

Sorry about that, I'll add a note on #7064.

From mal at egenix.com  Tue Oct 13 17:42:32 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 17:42:32 +0200
Subject: [Python-Dev] On track for Python 2.6.4 final this Sunday?
In-Reply-To: <20091013145211.8DB243A4068@sparrow.telecommunity.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
Message-ID: <4AD49FE8.8050002@egenix.com>

P.J. Eby wrote:
> At 08:27 AM 10/13/2009 -0400, Barry Warsaw wrote:
>> Are we on track to release 2.6.4 final this Sunday or do we need
>> another rc?
>>
>> Yesterday, Tarek committed another setuptools related fix and said
>> that he was going to run a bunch of build tests locally.  Tarek, how
>> did that go?
> 
> FWIW, that change won't work to fix the problem on Windows if a
> package's setup.py is written with cross-platform paths (i.e., the usual
> case).
> 
> Also, for the record, it's not really a setuptools-related fix; it's
> just attempting to fix collateral damage caused by the attempt to fix
> the problems caused by the original API change...  which affected all
> build_ext customizations relying on the docstring of the changed method,
> not just the one in setuptools.

Just to clarify a bit what all the fuzz is about:

The API in question (build_ext.get_ext_filename()) was not changed
at all. The doc-string also doesn't refer to *the* extension
currently being built, but to just to some arbitrary extension:

    def get_ext_filename (self, ext_name):

        r"""Convert the name of an extension (eg. "foo.bar") into the name
        Convert the name of an extension (eg. "foo.bar") into the name
        of the file from which it will be loaded (eg. "foo/bar.so", or
        "foo\bar.pyd")."""

What did change is the way this method is used by other methods
in build_ext: the method is no longer being called with the full
dotted name, but only with the last component.

Now, setuptools and pywin32 both override the method, with
the understanding that this method is the one and only
way to convert the extensions (full) dotted name into a file
name.

Up until the change (which was needed to get inplace installations
working), that assumption was true. However, neither the doc-string
nor the implementation of the method ever implied this.

distutils occasionally makes such assumptions necessary since
there's no clean OO-way to override the functionality. That's
not ideal, but works most of the time.

However, authors of distutils add-ons cannot really expect
such hacks to survive forever.

Rather than relying on these, it's better to get distutils
fixed to provide the necessary OO-hooks to properly implement
the modified functionality.

FWIW, I've had to change and tweak our mxSetup.py many many
times to get it to work again with new distutils releases and
I expect this to occur even more often now that distutils is
finally being maintained again - and that's good, since I'll
be able to get rid off the few remaining hacks !

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Tue Oct 13 17:59:17 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 17:59:17 +0200
Subject: [Python-Dev] [python-committers] On track for Python
 2.6.4	final this Sunday?
In-Reply-To: <4AD49AB5.9040703@v.loewis.de>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>	<4AD4917D.8090800@v.loewis.de>
	<4AD49631.70607@egenix.com> <4AD49AB5.9040703@v.loewis.de>
Message-ID: <4AD4A3D5.8030907@egenix.com>

"Martin v. L?wis" wrote:
> 
>>> As this bug already exists in 2.6.2, I don't think the change is
>>> eligible for 2.6.4.
>>>
>>> In addition, I want to review it, which I won't be able to until
>>> Sunday.
>>
>> Then I'd suggest to wait another week with 2.6.4 to give you a
>> chance to look at the patch.
> 
> That won't make the change more eligible.

No, but chances increase :-)

>> Of course, but we don't live in an ideal world, otherwise we would
>> have noticed before releasing 2.6 :-)
> 
> Oh, I did notice. I had been cautioning for years that switching to
> a newer VS version (2005 or later) would cause severe problems. We
> are still trying to recover from the switch to VS 2008. That said,
> staying with VS 2003 really hadn't been an option, either. It's
> just said that Microsoft has created a new DLL hell in their attempt
> to fix the old one, and that they fail to admit it (assuming that
> everybody should be using .NET, anyway).

Well, hopefully we can work out those problems and then stay with
VS 2008 a bit.

MS is already working on VS 2010:

http://www.microsoft.com/visualstudio/en-us/products/2010/default.mspx

List of C/C++ changes:

http://msdn.microsoft.com/en-us/library/dd465215(VS.100).aspx

Not sure what this means:

"""
The libraries deployment model is no longer based on the fusion mechanism, and manifests are no
longer used. Instead, the name of each dynamic link library contains its version number.
"""

Perhaps they finally realized that adding version number to DLLs is
better than trying to match manifests.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From pje at telecommunity.com  Tue Oct 13 18:23:34 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 13 Oct 2009 12:23:34 -0400
Subject: [Python-Dev] On track for Python 2.6.4 final this Sunday?
In-Reply-To: <4AD49FE8.8050002@egenix.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<4AD49FE8.8050002@egenix.com>
Message-ID: <20091013162342.BED653A4068@sparrow.telecommunity.com>

At 05:42 PM 10/13/2009 +0200, M.-A. Lemburg wrote:
>FWIW, I've had to change and tweak our mxSetup.py many many
>times to get it to work again with new distutils releases and
>I expect this to occur even more often now that distutils is
>finally being maintained again - and that's good, since I'll
>be able to get rid off the few remaining hacks !

Thanks for the nice (and neutral) explanation, for those not up to 
speed on the details.

Of course, had the change been made in 2.7 rather than 2.6.3, there 
wouldn't have been a problem here.  People understand that new major 
versions sometimes mean that existing packages will need to adapt and 
cope.  Under normal circumstances, however, a dot release shouldn't 
break packages that worked with previous dot releases.  (At least, 
that's the common expectation.)

Now, I do have to admit that I'm still ignorant of what bug this was 
trying to fix in the first place.  You say it "was needed to get 
inplace installations working", but I don't know what that means, 
since AFAIK inplace extension builds (build_ext -i) have been working 
since before the first versions of setuptools appeared in 2003.

So, if there was a bug in build_ext that wasn't *introduced* in 2.6, 
it seems like 2.6.3 would be an odd place to fix that bug.

This isn't intended as a criticism of anybody, mind you; I'm just 
saying that I still don't understand why the change was made in 2.6.3 
in the first place, and it would be kind of nice to know.  These 
kinds of situations can be useful for illustrating and improving 
Python policy, if the root causes and reasoning are understood by everyone.


From barry at python.org  Tue Oct 13 18:56:09 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 12:56:09 -0400
Subject: [Python-Dev] [python-committers] On track for Python
	2.6.4	final this Sunday?
In-Reply-To: <4AD49631.70607@egenix.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>
	<4AD4917D.8090800@v.loewis.de> <4AD49631.70607@egenix.com>
Message-ID: <E048EC3C-4640-424D-B6B1-223A6CB519D5@python.org>

On Oct 13, 2009, at 11:01 AM, M.-A. Lemburg wrote:

> Then I'd suggest to wait another week with 2.6.4 to give you a
> chance to look at the patch.

That's not a good option, IMO.  We have a known broken 2.6.3 out there  
and we owe it to our users to correct our mistake and given them the  
release they should have gotten in the first place.

We don't need to wait too long for 2.6.5 though.  A few months would  
be appropriate.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/646e6f94/attachment.pgp>

From barry at python.org  Tue Oct 13 18:57:21 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 12:57:21 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
Message-ID: <52E997AF-A050-41F4-9E2A-36282590B230@python.org>

On Oct 13, 2009, at 11:16 AM, Tarek Ziad? wrote:

> I still need to do some more tests, I didn't have time to try the
> various projects under win32.
> It's planned to night.
>
> The tests are consisting of compiling and insatling a dozain of
> projects on linux and win32 (and bith when possible)

Great thanks, let us know how it goes.  If there's any question at all  
about the fixes that have gone in since rc1, we should spin an rc2.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/29a389de/attachment.pgp>

From ssteinerx at gmail.com  Tue Oct 13 19:04:41 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 13 Oct 2009 13:04:41 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <52E997AF-A050-41F4-9E2A-36282590B230@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
Message-ID: <FE461493-B4EB-4C25-B63D-1348E60E1B75@gmail.com>


On Oct 13, 2009, at 12:57 PM, Barry Warsaw wrote:

> On Oct 13, 2009, at 11:16 AM, Tarek Ziad? wrote:
>
>> I still need to do some more tests, I didn't have time to try the
>> various projects under win32.
>> It's planned to night.
>>
>> The tests are consisting of compiling and insatling a dozain of
>> projects on linux and win32 (and bith when possible)
>
> Great thanks, let us know how it goes.  If there's any question at  
> all about the fixes that have gone in since rc1, we should spin an  
> rc2.

Just curious, how big a server does the linux portion of the test  
require memory-wise?  Would, e.g. an Ubuntu 9.0x with 256MB do the job?

I ask because those types of servers are available for short-run jobs  
for pennines per hour and I would be willing to work on the code to  
fire-up, load them with the test suite, do the run, and shut them  
down.  I happen to have a bunch of code lying around for doing such  
things...

S




From mal at egenix.com  Tue Oct 13 19:07:22 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 19:07:22 +0200
Subject: [Python-Dev] [python-committers] On track for Python
 2.6.4	final this Sunday?
In-Reply-To: <E048EC3C-4640-424D-B6B1-223A6CB519D5@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>	<4AD4917D.8090800@v.loewis.de>
	<4AD49631.70607@egenix.com>
	<E048EC3C-4640-424D-B6B1-223A6CB519D5@python.org>
Message-ID: <4AD4B3CA.4000409@egenix.com>

Barry Warsaw wrote:
> On Oct 13, 2009, at 11:01 AM, M.-A. Lemburg wrote:
> 
>> Then I'd suggest to wait another week with 2.6.4 to give you a
>> chance to look at the patch.
> 
> That's not a good option, IMO.  We have a known broken 2.6.3 out there
> and we owe it to our users to correct our mistake and given them the
> release they should have gotten in the first place.
> 
> We don't need to wait too long for 2.6.5 though.  A few months would be
> appropriate.

Would it be reasonable to shorten that period, if the fix for the
mentioned problem gets ready for prime time earlier ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Tue Oct 13 19:12:42 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 13:12:42 -0400
Subject: [Python-Dev] [python-committers] On track for Python
	2.6.4	final this Sunday?
In-Reply-To: <4AD4B3CA.4000409@egenix.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>	<4AD4917D.8090800@v.loewis.de>
	<4AD49631.70607@egenix.com>
	<E048EC3C-4640-424D-B6B1-223A6CB519D5@python.org>
	<4AD4B3CA.4000409@egenix.com>
Message-ID: <66AEE52E-608D-4A55-A85B-B95C570950D0@python.org>

On Oct 13, 2009, at 1:07 PM, M.-A. Lemburg wrote:

> Would it be reasonable to shorten that period, if the fix for the
> mentioned problem gets ready for prime time earlier ?

I think there are many 2.6.x bugs queued up for after 2.6.4 is  
released.  I'm not at all opposed to setting a date approximately 1  
month from now for the first 2.6.5 rc.  That should give people plenty  
of time to get their top fixes in.

i-like-timed-releases-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/b20175f7/attachment.pgp>

From skip at pobox.com  Tue Oct 13 19:33:03 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 13 Oct 2009 12:33:03 -0500
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
 final this Sunday?
In-Reply-To: <4AD4B3CA.4000409@egenix.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<4AD48BC4.2060809@egenix.com> <4AD4917D.8090800@v.loewis.de>
	<4AD49631.70607@egenix.com>
	<E048EC3C-4640-424D-B6B1-223A6CB519D5@python.org>
	<4AD4B3CA.4000409@egenix.com>
Message-ID: <19156.47567.39139.469228@montanaro.dyndns.org>

    >> We don't need to wait too long for 2.6.5 though.  A few months would be
    >> appropriate.

    MAL> Would it be reasonable to shorten that period, if the fix for the
    MAL> mentioned problem gets ready for prime time earlier ?

I think it would be worthwhile to prioritize all outstanding bugs which have
been mentioned in the context of 2.6.[345] and run a bug day with the top
priority being to fix those bugs.  If that task is completed, then move onto
other stuff.  Once those primary bugs are tackled schedule a 2.6.5 release.

Skip
 

From rdmurray at bitdance.com  Tue Oct 13 19:41:49 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 13 Oct 2009 13:41:49 -0400 (EDT)
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
 final this Sunday?
In-Reply-To: <52E997AF-A050-41F4-9E2A-36282590B230@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
Message-ID: <Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>

On Tue, 13 Oct 2009 at 12:57, Barry Warsaw wrote:
> On Oct 13, 2009, at 11:16 AM, Tarek Ziad? wrote:
>> I still need to do some more tests, I didn't have time to try the
>> various projects under win32.
>> It's planned to night.
>> 
>> The tests are consisting of compiling and insatling a dozain of
>> projects on linux and win32 (and bith when possible)
>
> Great thanks, let us know how it goes.  If there's any question at all about 
> the fixes that have gone in since rc1, we should spin an rc2.

I always thought that the idea of a release candidate was that if there
were any fixes _at all_ that there would be a new rc.  Only when no
bugs needing fixed are found does the rc turn into the actual release.
But I understand that this is an idealized rc/release policy :)

--David (RDM)

From barry at python.org  Tue Oct 13 19:45:39 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 13:45:39 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
Message-ID: <39C0375D-0A34-45F8-90AA-A4AA089CC9C5@python.org>

On Oct 13, 2009, at 1:41 PM, R. David Murray wrote:

> I always thought that the idea of a release candidate was that if  
> there
> were any fixes _at all_ that there would be a new rc.  Only when no
> bugs needing fixed are found does the rc turn into the actual release.
> But I understand that this is an idealized rc/release policy :)

I'll invoke the Umbrella Rule now.  If we don't do one we'll wish we  
had.  If we do, we won't have needed it.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/3576ca18/attachment.pgp>

From mal at egenix.com  Tue Oct 13 19:54:20 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Oct 2009 19:54:20 +0200
Subject: [Python-Dev] [python-committers] On track for Python
 2.6.4	final this Sunday?
In-Reply-To: <66AEE52E-608D-4A55-A85B-B95C570950D0@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<4AD48BC4.2060809@egenix.com>	<4AD4917D.8090800@v.loewis.de>	<4AD49631.70607@egenix.com>	<E048EC3C-4640-424D-B6B1-223A6CB519D5@python.org>	<4AD4B3CA.4000409@egenix.com>
	<66AEE52E-608D-4A55-A85B-B95C570950D0@python.org>
Message-ID: <4AD4BECC.6010300@egenix.com>

Barry Warsaw wrote:
> On Oct 13, 2009, at 1:07 PM, M.-A. Lemburg wrote:
> 
>> Would it be reasonable to shorten that period, if the fix for the
>> mentioned problem gets ready for prime time earlier ?
> 
> I think there are many 2.6.x bugs queued up for after 2.6.4 is
> released.  I'm not at all opposed to setting a date approximately 1
> month from now for the first 2.6.5 rc.  That should give people plenty
> of time to get their top fixes in.
> 
> i-like-timed-releases-ly y'rs,

Agreed. Adding some pressure that way helps :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 13 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From guido at python.org  Tue Oct 13 20:00:27 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 13 Oct 2009 11:00:27 -0700
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <39C0375D-0A34-45F8-90AA-A4AA089CC9C5@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org> 
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com> 
	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net> 
	<39C0375D-0A34-45F8-90AA-A4AA089CC9C5@python.org>
Message-ID: <ca471dc20910131100s2b1c6e3as9eaa8ef124917c44@mail.gmail.com>

On Tue, Oct 13, 2009 at 10:45 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 13, 2009, at 1:41 PM, R. David Murray wrote:
>
>> I always thought that the idea of a release candidate was that if there
>> were any fixes _at all_ that there would be a new rc. ?Only when no
>> bugs needing fixed are found does the rc turn into the actual release.
>> But I understand that this is an idealized rc/release policy :)
>
> I'll invoke the Umbrella Rule now. ?If we don't do one we'll wish we had.
> ?If we do, we won't have needed it.

Traceback (most recent call last):
  File "umbrella_rules.py", line 6, in <module>
    UmbrellaRule()
  File "unbrella_rules.py", line 4, in UmbrellaRule
    raise DuplicateRuleName('http://whatplanetisthis.com/2008/02/the-umbrella-rule-and-sharing-rules-in-general/')
DuplicateRuleName:
http://whatplanetisthis.com/2008/02/the-umbrella-rule-and-sharing-rules-in-general/

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Oct 13 20:06:29 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 14:06:29 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <ca471dc20910131100s2b1c6e3as9eaa8ef124917c44@mail.gmail.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<39C0375D-0A34-45F8-90AA-A4AA089CC9C5@python.org>
	<ca471dc20910131100s2b1c6e3as9eaa8ef124917c44@mail.gmail.com>
Message-ID: <B82471F1-DFB4-4A5A-B52E-F6CA56660A19@python.org>

On Oct 13, 2009, at 2:00 PM, Guido van Rossum wrote:

> Traceback (most recent call last):
>  File "umbrella_rules.py", line 6, in <module>
>    UmbrellaRule()
>  File "unbrella_rules.py", line 4, in UmbrellaRule
>    raise DuplicateRuleName('http://whatplanetisthis.com/2008/02/the-umbrella-rule-and-sharing-rules-in-general/')
> DuplicateRuleName:
> http://whatplanetisthis.com/2008/02/the-umbrella-rule-and-sharing-rules-in-general/

MisinformedMeteorologistRule (probably more PC than stupid weatherman  
rule :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/299606e5/attachment.pgp>

From martin at v.loewis.de  Wed Oct 14 00:10:24 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 14 Oct 2009 06:10:24 +0800
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
 final this Sunday?
In-Reply-To: <Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<20091013145211.8DB243A4068@sparrow.telecommunity.com>	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
Message-ID: <4AD4FAD0.1080807@v.loewis.de>


> I always thought that the idea of a release candidate was that if there
> were any fixes _at all_ that there would be a new rc.  Only when no
> bugs needing fixed are found does the rc turn into the actual release.

This was also my understanding; that's the point of calling it
"candidate". Since the code base of 2.6.4rc1 was not released as-is,
we would need to consider another candidate.

But then, Barry doesn't like release candidates in the first place.

Regards,
Martin



From ziade.tarek at gmail.com  Wed Oct 14 00:44:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 14 Oct 2009 00:44:29 +0200
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
Message-ID: <94bdd2610910131544h231d07cdqb07d3f6f65eee06d@mail.gmail.com>

On Tue, Oct 13, 2009 at 5:16 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Tue, Oct 13, 2009 at 4:52 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 08:27 AM 10/13/2009 -0400, Barry Warsaw wrote:
>>>
>>> Are we on track to release 2.6.4 final this Sunday or do we need
>>> another rc?
>>>
>>> Yesterday, Tarek committed another setuptools related fix and said
>>> that he was going to run a bunch of build tests locally. ?Tarek, how
>>> did that go?
>>
>
> I still need to do some more tests, I didn't have time to try the
> various projects under win32.
> It's planned to night.
>
> The tests are consisting of compiling and insatling a dozain of
> projects on linux and win32 (and bith when possible)

Barry,

I'll continue my win32 smoke tests tomorrow, it's not over. So far, so good.

windows XP (vmware) - VC9 - running setup.py build+install

- zope.interface 3.5.2 + setuptools 0.6c9
- Twisted-8.2.0 + setuptools 0.6c9
- pythonwin32-314 +plain distutils: works but I had a failure on
'mc.exe' on one extension which is unrelated obviously.
  pywin32 builds fine otherwise. Still investigating. on this one

Tarek

From barry at python.org  Wed Oct 14 01:06:38 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 19:06:38 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <4AD4FAD0.1080807@v.loewis.de>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<20091013145211.8DB243A4068@sparrow.telecommunity.com>	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<4AD4FAD0.1080807@v.loewis.de>
Message-ID: <79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>

On Oct 13, 2009, at 6:10 PM, Martin v. L?wis wrote:

>> I always thought that the idea of a release candidate was that if  
>> there
>> were any fixes _at all_ that there would be a new rc.  Only when no
>> bugs needing fixed are found does the rc turn into the actual  
>> release.
>
> This was also my understanding; that's the point of calling it
> "candidate". Since the code base of 2.6.4rc1 was not released as-is,
> we would need to consider another candidate.
>
> But then, Barry doesn't like release candidates in the first place.

No, but let's do one anyway!

So, we can either make Sunday's release rc2 and do the final release  
one week later, or I can try to get an rc2 out in the next day or two,  
with a final release mid-next week.

Thoughts?
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/7027cbcd/attachment.pgp>

From martin at v.loewis.de  Wed Oct 14 04:15:55 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 14 Oct 2009 10:15:55 +0800
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
 final this Sunday?
In-Reply-To: <79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<20091013145211.8DB243A4068@sparrow.telecommunity.com>	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<4AD4FAD0.1080807@v.loewis.de>
	<79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>
Message-ID: <4AD5345B.4080301@v.loewis.de>

> So, we can either make Sunday's release rc2 and do the final release one
> week later, or I can try to get an rc2 out in the next day or two, with
> a final release mid-next week.
> 
> Thoughts?

I won't be able to produce Windows binaries until Saturday. Now sure how
that would fit into the "within the next day or two" plan.

Regards,
Martin

From barry at python.org  Wed Oct 14 04:45:43 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Oct 2009 22:45:43 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <4AD5345B.4080301@v.loewis.de>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<20091013145211.8DB243A4068@sparrow.telecommunity.com>	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<4AD4FAD0.1080807@v.loewis.de>
	<79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>
	<4AD5345B.4080301@v.loewis.de>
Message-ID: <B093CC07-0455-4ABA-BB47-7DC46CEBEB6D@python.org>

On Oct 13, 2009, at 10:15 PM, Martin v. L?wis wrote:

>> So, we can either make Sunday's release rc2 and do the final  
>> release one
>> week later, or I can try to get an rc2 out in the next day or two,  
>> with
>> a final release mid-next week.
>>
>> Thoughts?
>
> I won't be able to produce Windows binaries until Saturday. Now sure  
> how
> that would fit into the "within the next day or two" plan.

That seems to argue for doing rc2 on Sunday the 18th.  If I tag the  
release some time Saturday, you could have the binaries by Sunday  
right?  Then we'll take one more week and release the final on the 25th.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091013/c97ee626/attachment-0001.pgp>

From anthonybaxter at gmail.com  Wed Oct 14 04:37:16 2009
From: anthonybaxter at gmail.com (Anthony Baxter)
Date: Wed, 14 Oct 2009 13:37:16 +1100
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <e69d3ed20910131936w4ef49fd7oa97dd432b9920919@mail.gmail.com>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>
	<20091013145211.8DB243A4068@sparrow.telecommunity.com>
	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>
	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<4AD4FAD0.1080807@v.loewis.de>
	<79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>
	<e69d3ed20910131936w4ef49fd7oa97dd432b9920919@mail.gmail.com>
Message-ID: <e69d3ed20910131937m56648924of767bd57c2aa578@mail.gmail.com>

I strongly urge another release candidate. But then, I am not doing the
work, so take that advice for what it is...

On Oct 14, 2009 10:18 AM, "Barry Warsaw" <barry at python.org> wrote:

On Oct 13, 2009, at 6:10 PM, Martin v. L?wis wrote: >> I always thought that
the idea of a release ...
No, but let's do one anyway!

So, we can either make Sunday's release rc2 and do the final release one
week later, or I can try to get an rc2 out in the next day or two, with a
final release mid-next week.

Thoughts?
-Barry


_______________________________________________
python-committers mailing list
python-committers at python.org
http://mail.python.org/mailman/listinfo/python-committers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091014/05578d19/attachment.htm>

From martin at v.loewis.de  Wed Oct 14 14:06:58 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 14 Oct 2009 20:06:58 +0800
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
 final this Sunday?
In-Reply-To: <B093CC07-0455-4ABA-BB47-7DC46CEBEB6D@python.org>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<20091013145211.8DB243A4068@sparrow.telecommunity.com>	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<4AD4FAD0.1080807@v.loewis.de>
	<79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>
	<4AD5345B.4080301@v.loewis.de>
	<B093CC07-0455-4ABA-BB47-7DC46CEBEB6D@python.org>
Message-ID: <4AD5BEE2.2000405@v.loewis.de>


> That seems to argue for doing rc2 on Sunday the 18th.  If I tag the
> release some time Saturday, you could have the binaries by Sunday
> right?

Correct.

Regards,
Martin


From barry at python.org  Wed Oct 14 14:48:01 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Oct 2009 08:48:01 -0400
Subject: [Python-Dev] [python-committers] On track for Python 2.6.4
	final this Sunday?
In-Reply-To: <4AD5BEE2.2000405@v.loewis.de>
References: <430E0910-C2A1-48F7-B036-743BA5BA91A3@python.org>	<20091013145211.8DB243A4068@sparrow.telecommunity.com>	<94bdd2610910130816g42826010w3d20901683678ed3@mail.gmail.com>	<52E997AF-A050-41F4-9E2A-36282590B230@python.org>
	<Pine.LNX.4.64.0910131339190.18193@kimball.webabinitio.net>
	<4AD4FAD0.1080807@v.loewis.de>
	<79DD5085-8D85-43F7-87B7-B378D76ECA64@python.org>
	<4AD5345B.4080301@v.loewis.de>
	<B093CC07-0455-4ABA-BB47-7DC46CEBEB6D@python.org>
	<4AD5BEE2.2000405@v.loewis.de>
Message-ID: <9185B478-1C1B-4F26-A20D-24834EBFD898@python.org>

On Oct 14, 2009, at 8:06 AM, Martin v. L?wis wrote:

>> That seems to argue for doing rc2 on Sunday the 18th.  If I tag the
>> release some time Saturday, you could have the binaries by Sunday
>> right?
>
> Correct.

Cool.  I've updated the Python Release Schedule calendar to reflect  
the new dates.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091014/914686ad/attachment.pgp>

From nas at arctrix.com  Wed Oct 14 22:16:35 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 14 Oct 2009 14:16:35 -0600
Subject: [Python-Dev] Better module shutdown procedure
Message-ID: <20091014201635.GA28053@arctrix.com>

The current shutdown code in pythonrun.c zaps module globals by
setting them to None (an attempt to break reference cycles). That
causes problems since __del__ methods can try to use the globals
after they have been set to None.

The procedure implemented by http://bugs.python.org/issue812369
seems to be a better idea. References to modules are replaced by
weak references and the GC is allowed to cleanup reference cycles.

I would like to commit this change but I'm not sure if it is a good
time in the development cycle or if anyone has objections to the
idea. Please speak up if you have input.

  Neil

From exarkun at twistedmatrix.com  Wed Oct 14 22:35:28 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 14 Oct 2009 20:35:28 -0000
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014201635.GA28053@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
Message-ID: <20091014203528.8004.1935283805.divmod.xquotient.44@localhost.localdomain>

On 08:16 pm, nas at arctrix.com wrote:
>The current shutdown code in pythonrun.c zaps module globals by
>setting them to None (an attempt to break reference cycles). That
>causes problems since __del__ methods can try to use the globals
>after they have been set to None.
>
>The procedure implemented by http://bugs.python.org/issue812369
>seems to be a better idea. References to modules are replaced by
>weak references and the GC is allowed to cleanup reference cycles.
>
>I would like to commit this change but I'm not sure if it is a good
>time in the development cycle or if anyone has objections to the
>idea. Please speak up if you have input.

I notice that the patch doesn't include any unit tests for the feature 
being provided (it does change test_sys.py - although I don't understand 
how that change is related to this feature).  I hope that regardless of 
whatever else is decided, if the change is to be made, it will be 
accompanied by new unit tests verify its proper operation.

Jean-Paul

From solipsis at pitrou.net  Wed Oct 14 22:49:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 14 Oct 2009 20:49:59 +0000 (UTC)
Subject: [Python-Dev] Better module shutdown procedure
References: <20091014201635.GA28053@arctrix.com>
Message-ID: <loom.20091014T224713-288@post.gmane.org>

Neil Schemenauer <nas <at> arctrix.com> writes:
> 
> I would like to commit this change but I'm not sure if it is a good
> time in the development cycle or if anyone has objections to the
> idea. Please speak up if you have input.

Have you tried to remove the various hacks/workarounds in the stdlib that
currently avoid module finalization problems?
(take a look at e.g. threading.py)
It would be a good validation that the patch improves on the current behaviour.

Regards

Antoine.



From daniel at stutzbachenterprises.com  Wed Oct 14 23:13:12 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 14 Oct 2009 16:13:12 -0500
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014201635.GA28053@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
Message-ID: <eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>

On Wed, Oct 14, 2009 at 3:16 PM, Neil Schemenauer <nas at arctrix.com> wrote:

> The current shutdown code in pythonrun.c zaps module globals by
> setting them to None (an attempt to break reference cycles). That
> causes problems since __del__ methods can try to use the globals
> after they have been set to None.
>
> The procedure implemented by http://bugs.python.org/issue812369
> seems to be a better idea. References to modules are replaced by
> weak references and the GC is allowed to cleanup reference cycles.
>

Based on the description, it still resorts to zapping module globals by
setting them to None.  It zaps them to weakrefs first, which means that
globals are more likely to be valid during __del__, but it still cannot make
any guarantees and referencing globals from __del__ is still a bad idea.  Is
that a correct synopsis?

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091014/a68e762b/attachment.htm>

From nas at arctrix.com  Wed Oct 14 23:42:30 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 14 Oct 2009 15:42:30 -0600
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>
References: <20091014201635.GA28053@arctrix.com>
	<eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>
Message-ID: <20091014214230.GA28424@arctrix.com>

On Wed, Oct 14, 2009 at 04:13:12PM -0500, Daniel Stutzbach wrote:
> Based on the description, it still resorts to zapping module globals by
> setting them to None.  It zaps them to weakrefs first, which means that
> globals are more likely to be valid during __del__, but it still cannot make
> any guarantees and referencing globals from __del__ is still a bad idea.  Is
> that a correct synopsis?

Yes, it does still resort to setting globals to None. However, the
weakref step makes it much more likely that __del__ methods run
before that happens. After this change, referencing global variables
from __del__ methods is okay. What is not a good idea is creating
__del__ methods that are part of a reference cycle (i.e. an island
of references) and also refer to that cycle somehow.  That has never
been a good idea and those __del__ methods will never get run,
before or after the proposed change.

HTH,

  Neil

From nas at arctrix.com  Wed Oct 14 23:45:06 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 14 Oct 2009 15:45:06 -0600
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014203528.8004.1935283805.divmod.xquotient.44@localhost.localdomain>
References: <20091014201635.GA28053@arctrix.com>
	<20091014203528.8004.1935283805.divmod.xquotient.44@localhost.localdomain>
Message-ID: <20091014214506.GB28424@arctrix.com>

On Wed, Oct 14, 2009 at 08:35:28PM -0000, exarkun at twistedmatrix.com wrote:
> I notice that the patch doesn't include any unit tests for the feature  
> being provided

That's hard to do although I suppose not impossible. We would be
trying to test the interpreter shutdown procedure. I suppose on
plaforms that support it we could fork a subprocess and then inspect
the output from __del__ methods.

> (it does change test_sys.py - although I don't understand  how
> that change is related to this feature).

That's a spurious change and will be left out of the real fix.

  Neil

From glyph at twistedmatrix.com  Thu Oct 15 00:25:35 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Wed, 14 Oct 2009 15:25:35 -0700
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014214506.GB28424@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
	<20091014203528.8004.1935283805.divmod.xquotient.44@localhost.localdomain>
	<20091014214506.GB28424@arctrix.com>
Message-ID: <d9047f780910141525i2f328429g6da493e5128356d2@mail.gmail.com>

On Wed, Oct 14, 2009 at 2:45 PM, Neil Schemenauer <nas at arctrix.com> wrote:

> On Wed, Oct 14, 2009 at 08:35:28PM -0000, exarkun at twistedmatrix.com wrote:
> > I notice that the patch doesn't include any unit tests for the feature
> > being provided
>
> That's hard to do although I suppose not impossible. We would be
> trying to test the interpreter shutdown procedure. I suppose on
> plaforms that support it we could fork a subprocess and then inspect
> the output from __del__ methods.


Why not just expose the module-teardown procedure and call it on a module,
then inspect the state there as it's being torn down?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091014/b66b44ed/attachment.htm>

From daniel at stutzbachenterprises.com  Thu Oct 15 00:35:38 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 14 Oct 2009 17:35:38 -0500
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014214230.GA28424@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
	<eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>
	<20091014214230.GA28424@arctrix.com>
Message-ID: <eae285400910141535t379ddc79k609a1fb267babd25@mail.gmail.com>

On Wed, Oct 14, 2009 at 4:42 PM, Neil Schemenauer <nas at arctrix.com> wrote:

> Yes, it does still resort to setting globals to None. However, the
> weakref step makes it much more likely that __del__ methods run
> before that happens. After this change, referencing global variables
> from __del__ methods is okay.
>

The first and last sentences seem like a contradiction to me.  If I cannot
guarantee that globals will be valid when __del__ is executed, then I must
not reference globals from __del__.

I think I'm missing something here.  Is there some way the programmer can
determine that referencing a global variable in __del__ will be 100% safe?
(not just "likely")

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091014/1451f724/attachment.htm>

From nas at arctrix.com  Thu Oct 15 01:05:56 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 14 Oct 2009 17:05:56 -0600
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <eae285400910141535t379ddc79k609a1fb267babd25@mail.gmail.com>
References: <20091014201635.GA28053@arctrix.com>
	<eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>
	<20091014214230.GA28424@arctrix.com>
	<eae285400910141535t379ddc79k609a1fb267babd25@mail.gmail.com>
Message-ID: <20091014230556.GA28661@arctrix.com>

On Wed, Oct 14, 2009 at 05:35:38PM -0500, Daniel Stutzbach wrote:
> The first and last sentences seem like a contradiction to me.  If I cannot
> guarantee that globals will be valid when __del__ is executed, then I must
> not reference globals from __del__.

I think there is confusion about the word "reference". In the
method:

    def __del__():
        print sys.version

the global variable reference to 'sys' is not a reference on the GC
referencing counting sense. IOW, it does not result in a a Py_INCREF
while the function is not being executed and therefore should be
safe after the proposed change. Currently, it could result in 'None'
being printed.

HTH,

  Neil

From daniel at stutzbachenterprises.com  Thu Oct 15 01:27:37 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 14 Oct 2009 18:27:37 -0500
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014230556.GA28661@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
	<eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>
	<20091014214230.GA28424@arctrix.com>
	<eae285400910141535t379ddc79k609a1fb267babd25@mail.gmail.com>
	<20091014230556.GA28661@arctrix.com>
Message-ID: <eae285400910141627k7b21ee4cy2619585ae4c6559d@mail.gmail.com>

On Wed, Oct 14, 2009 at 6:05 PM, Neil Schemenauer <nas at arctrix.com> wrote:

>    def __del__():
>        print sys.version
>
> the global variable reference to 'sys' is not a reference on the GC
> referencing counting sense. IOW, it does not result in a a Py_INCREF
> while the function is not being executed and therefore should be
> safe after the proposed change. Currently, it could result in 'None'
> being printed.
>

Currently it throws an exception since "sys" is None. :-)

Here is my understanding of the proposed procedure:

1. Replace modules in sys.modules with weakrefs
2. Run the garbage collector
3. Replace globals in any remaining modules with None
4. Run the garbage collector

Is it possible for a __del__ method to be called in step 4 or not?  I am
still unclear on this point. :-)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091014/9b3e8287/attachment.htm>

From nas at arctrix.com  Thu Oct 15 01:49:04 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 14 Oct 2009 23:49:04 +0000 (UTC)
Subject: [Python-Dev] Better module shutdown procedure
References: <20091014201635.GA28053@arctrix.com>
	<loom.20091014T224713-288@post.gmane.org>
Message-ID: <hb5o1g$imb$1@ger.gmane.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:
> Have you tried to remove the various hacks/workarounds in the
> stdlib that currently avoid module finalization problems?

Good idea. These hacks are the best argument for applying the
change, IMHO. I made a quick review of Lib modules using __del__:

    tarfile: looks not safe
    tempfile: goes through elaborate motions to avoid globals
    threading: yikes, I didn't attempt to decypher this one
    shelve: probably not safe
    dumbdb: avoids globals, easy to remove hacks
    _pyio: try/except can be removed (hopefully)
    urllib: avoids globals, easy to remove hacks
    
Will try to cook up a patch in my copious free time.

  Neil


From solipsis at pitrou.net  Thu Oct 15 14:42:51 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 15 Oct 2009 12:42:51 +0000 (UTC)
Subject: [Python-Dev] Better module shutdown procedure
References: <20091014201635.GA28053@arctrix.com>
	<eae285400910141413p63d710dbyf8a8fccdd6b533d7@mail.gmail.com>
	<20091014214230.GA28424@arctrix.com>
	<eae285400910141535t379ddc79k609a1fb267babd25@mail.gmail.com>
	<20091014230556.GA28661@arctrix.com>
	<eae285400910141627k7b21ee4cy2619585ae4c6559d@mail.gmail.com>
Message-ID: <hb75cb$6k7$1@ger.gmane.org>

Le Wed, 14 Oct 2009 18:27:37 -0500, Daniel Stutzbach a ?crit?:
> 
> Here is my understanding of the proposed procedure:
> 
> 1. Replace modules in sys.modules with weakrefs 2. Run the garbage
> collector
> 3. Replace globals in any remaining modules with None 4. Run the garbage
> collector
> 
> Is it possible for a __del__ method to be called in step 4 or not?  I am
> still unclear on this point. :-)

If an object is holding a reference to a module (e.g. self._sys = sys), 
then yes it should be possible because the reference has been creating a 
cycle.



From jnoller at gmail.com  Thu Oct 15 15:05:34 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 15 Oct 2009 09:05:34 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
Message-ID: <4222a8490910150605x395465csa68177492dd275bd@mail.gmail.com>

On Wed, Oct 7, 2009 at 2:09 PM, Barry Warsaw <barry at python.org> wrote:
> On Oct 7, 2009, at 1:26 PM, Scott Dial wrote:
>
>> I suspect this release is primarily to quench the problems with
>> distutils, but..
>>
>> http://bugs.python.org/issue5949
>>
>> doesn't seem to have been addressed by you. And this seems like it would
>> be another unfortunate loss of an opportunity.
>
> I want us to be very careful about 2.6.4. ?This isn't a normal bug fix
> release, it's specifically there to remove the brown paper bag of 2.6.3 from
> our heads. ?So let's be conservative and fix this one in 2.6.5.
>
> -Barry
>

Here's another one barry:

http://bugs.python.org/issue7120

We should get this in - it's a regression I introduced awhile ago for
environments without the multiprocessing module using logging.

jesse

From skip at pobox.com  Thu Oct 15 13:39:30 2009
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 15 Oct 2009 06:39:30 -0500 (CDT)
Subject: [Python-Dev] Can 3.1 still be built without complex?
Message-ID: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>

I notice that WITHOUT_COMPLEX still appears in Python.h and several .c files
but nowhere else in the 2.6, 2.7 or 3.1 source, most particularly not in
configure or pyconfig.h.in.  Are builds --without-complex still supported?
Has it been tested at any time in the recent past?

Skip

From barry at python.org  Thu Oct 15 15:40:43 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Oct 2009 09:40:43 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <4222a8490910150605x395465csa68177492dd275bd@mail.gmail.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
	<4222a8490910150605x395465csa68177492dd275bd@mail.gmail.com>
Message-ID: <D82FD8E2-5DD2-40DD-B693-63F71AB0890C@python.org>

On Oct 15, 2009, at 9:05 AM, Jesse Noller wrote:

> Here's another one barry:
>
> http://bugs.python.org/issue7120
>
> We should get this in - it's a regression I introduced awhile ago for
> environments without the multiprocessing module using logging.

Was this a regression in 2.6.2 or 2.6.3?  I think it might not matter,  
but let me know and I'll decide.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091015/8c085286/attachment.pgp>

From eric at trueblade.com  Thu Oct 15 15:48:34 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 15 Oct 2009 09:48:34 -0400
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
Message-ID: <4AD72832.90409@trueblade.com>

skip at pobox.com wrote:
> I notice that WITHOUT_COMPLEX still appears in Python.h and several .c files
> but nowhere else in the 2.6, 2.7 or 3.1 source, most particularly not in
> configure or pyconfig.h.in.  Are builds --without-complex still supported?
> Has it been tested at any time in the recent past?

I haven't tested it, but I still see WITHOUT_COMPLEX in trunk and py3k 
branches. In py3k, it's referenced in:

./Include/Python.h
./Misc/HISTORY
./Objects/complexobject.c
./Objects/object.c
./Parser/tokenizer.c
./Python/ast.c
./Python/bltinmodule.c
./Python/getargs.c
./Python/marshal.c
./Python/modsupport.c

I checked complexobject.c, and it looks like it's used correctly there.

Eric.

From jnoller at gmail.com  Thu Oct 15 16:00:40 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 15 Oct 2009 10:00:40 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <D82FD8E2-5DD2-40DD-B693-63F71AB0890C@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
	<4222a8490910150605x395465csa68177492dd275bd@mail.gmail.com>
	<D82FD8E2-5DD2-40DD-B693-63F71AB0890C@python.org>
Message-ID: <4222a8490910150700u358568b2l1cf8ba623f14c22f@mail.gmail.com>

On Thu, Oct 15, 2009 at 9:40 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 15, 2009, at 9:05 AM, Jesse Noller wrote:
>
>> Here's another one barry:
>>
>> http://bugs.python.org/issue7120
>>
>> We should get this in - it's a regression I introduced awhile ago for
>> environments without the multiprocessing module using logging.
>
> Was this a regression in 2.6.2 or 2.6.3? ?I think it might not matter, but
> let me know and I'll decide.
>
> -Barry
>
>

My original commit was on 2009-01-18, you tagged 2.6.2 on Tue Apr 14
13:17:30 2009, so it went into 2.6.2.

The fix is minor, the bug affects things such as GAE which may use
logging, but not have multiprocessing available.

jesse

From barry at python.org  Thu Oct 15 16:14:46 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Oct 2009 10:14:46 -0400
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <4222a8490910150700u358568b2l1cf8ba623f14c22f@mail.gmail.com>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org>
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org>
	<4222a8490910150605x395465csa68177492dd275bd@mail.gmail.com>
	<D82FD8E2-5DD2-40DD-B693-63F71AB0890C@python.org>
	<4222a8490910150700u358568b2l1cf8ba623f14c22f@mail.gmail.com>
Message-ID: <FB875E2D-6A3C-4CE3-A5E5-CE5215726DAC@python.org>

On Oct 15, 2009, at 10:00 AM, Jesse Noller wrote:

> On Thu, Oct 15, 2009 at 9:40 AM, Barry Warsaw <barry at python.org>  
> wrote:
>> On Oct 15, 2009, at 9:05 AM, Jesse Noller wrote:
>>
>>> Here's another one barry:
>>>
>>> http://bugs.python.org/issue7120
>>>
>>> We should get this in - it's a regression I introduced awhile ago  
>>> for
>>> environments without the multiprocessing module using logging.
>>
>> Was this a regression in 2.6.2 or 2.6.3?  I think it might not  
>> matter, but
>> let me know and I'll decide.
>>
>> -Barry
>>
>>
>
> My original commit was on 2009-01-18, you tagged 2.6.2 on Tue Apr 14
> 13:17:30 2009, so it went into 2.6.2.
>
> The fix is minor, the bug affects things such as GAE which may use
> logging, but not have multiprocessing available.

Approved for 2.6.4rc2

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091015/64a6a9c3/attachment-0001.pgp>

From skip at pobox.com  Thu Oct 15 16:28:47 2009
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 15 Oct 2009 09:28:47 -0500
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <4AD72832.90409@trueblade.com>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
	<4AD72832.90409@trueblade.com>
Message-ID: <19159.12703.850168.592693@montanaro.dyndns.org>


    Eric> I haven't tested it, but I still see WITHOUT_COMPLEX in trunk and py3k 
    Eric> branches. In py3k, it's referenced in:
    ...

Sure, but is it ever exercised?  A name like WITHOUT_COMPLEX suggests that
it should be flipped on/off by configure using --without-complex, but that
script doesn't know about it and it's not mentioned in pyconfig.h.in, where
the various --with-* flags work their magic.

Skip

From eric at trueblade.com  Thu Oct 15 16:35:30 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 15 Oct 2009 10:35:30 -0400
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <19159.12703.850168.592693@montanaro.dyndns.org>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>	<4AD72832.90409@trueblade.com>
	<19159.12703.850168.592693@montanaro.dyndns.org>
Message-ID: <4AD73332.50506@trueblade.com>

skip at pobox.com wrote:
>     Eric> I haven't tested it, but I still see WITHOUT_COMPLEX in trunk and py3k 
>     Eric> branches. In py3k, it's referenced in:
>     ...
> 
> Sure, but is it ever exercised?  A name like WITHOUT_COMPLEX suggests that
> it should be flipped on/off by configure using --without-complex, but that
> script doesn't know about it and it's not mentioned in pyconfig.h.in, where
> the various --with-* flags work their magic.

Ah, I misunderstood. I thought you were asking "does setting 
WITHOUT_COMPLEX work?"

No idea about any configure option, sorry.

Eric.

From dickinsm at gmail.com  Thu Oct 15 16:46:55 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 15 Oct 2009 15:46:55 +0100
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
Message-ID: <5c6f2a5d0910150746u41d163dby369a2f6d86dd8075@mail.gmail.com>

[I originally sent this reply to Skip instead of to the list;  apologies.]

On Thu, Oct 15, 2009 at 12:39 PM,  <skip at pobox.com> wrote:
> I notice that WITHOUT_COMPLEX still appears in Python.h and several .c files
> but nowhere else in the 2.6, 2.7 or 3.1 source, most particularly not in
> configure or pyconfig.h.in.  Are builds --without-complex still supported?
> Has it been tested at any time in the recent past?

Apparently not.  :)

I just tried the following with an svn checkout of trunk (r75433), on OS X 10.6:

dickinsm$ CC='gcc -DWITHOUT_COMPLEX' ./configure && make

The build fails with:

gcc -DWITHOUT_COMPLEX -c -fno-strict-aliasing -DNDEBUG -g  -O3 -Wall
-Wstrict-prototypes  -I. -IInclude -I./Include   -DPy_BUILD_CORE -o
Python/compile.o Python/compile.c
Python/compile.c: In function ?compiler_add_o?:
Python/compile.c:914: error: ?Py_complex? undeclared (first use in
this function)
Python/compile.c:914: error: (Each undeclared identifier is reported only once
Python/compile.c:914: error: for each function it appears in.)
Python/compile.c:914: error: expected ?;? before ?z?
Python/compile.c:931: warning: implicit declaration of function
?PyComplex_Check?
Python/compile.c:937: error: ?z? undeclared (first use in this function)
Python/compile.c:937: warning: implicit declaration of function
?PyComplex_AsCComplex?
make: *** [Python/compile.o] Error 1

Mark

Postscript:  the above compilation failure is easily fixed.  The next
failure is:

gcc -DWITHOUT_COMPLEX  -u _PyMac_Error -o python.exe \
			Modules/python.o \
			libpython2.7.a -ldl
Undefined symbols:
  "_PyComplex_RealAsDouble", referenced from:
      __PyComplex_FormatAdvanced in libpython2.7.a(formatter_string.o)
  "_PyComplex_ImagAsDouble", referenced from:
      __PyComplex_FormatAdvanced in libpython2.7.a(formatter_string.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [python.exe] Error 1

From solipsis at pitrou.net  Thu Oct 15 17:06:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 15 Oct 2009 15:06:27 +0000 (UTC)
Subject: [Python-Dev] Can 3.1 still be built without complex?
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
Message-ID: <loom.20091015T170259-856@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> I notice that WITHOUT_COMPLEX still appears in Python.h and several .c files
> but nowhere else in the 2.6, 2.7 or 3.1 source, most particularly not in
> configure or pyconfig.h.in.  Are builds --without-complex still supported?
> Has it been tested at any time in the recent past?

Is there any point in building without complex? Size reduction perhaps?
If nobody uses it, we could remove that option. We have trouble staying
compatible with lots of build options (see how --without-threads is little
exercised).

Regards

Antoine.



From fdrake at gmail.com  Thu Oct 15 17:25:29 2009
From: fdrake at gmail.com (Fred Drake)
Date: Thu, 15 Oct 2009 11:25:29 -0400
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <loom.20091015T170259-856@post.gmane.org>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org> 
	<loom.20091015T170259-856@post.gmane.org>
Message-ID: <9cee7ab80910150825g3118325fjc4c76293de86e1c6@mail.gmail.com>

On Thu, Oct 15, 2009 at 11:06 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> We have trouble staying
> compatible with lots of build options (see how --without-threads is little
> exercised).

I don't know if folks deploying on embedded platforms are using these
options (directly or indirectly), but expect that's the primary
audience.

Hopefully somebody who's doing that can tell us if they're using the
defines, and what versions of Python they're currently using.  Then we
can determine whether the should be supported.

If they should be, there should be buildbot's set up for relevant
configurations.  Otherwise removal is best.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From guido at python.org  Thu Oct 15 19:57:32 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 15 Oct 2009 10:57:32 -0700
Subject: [Python-Dev] Python 2.6.4rc1
In-Reply-To: <FB875E2D-6A3C-4CE3-A5E5-CE5215726DAC@python.org>
References: <AF6969D8-F3B2-4885-B086-37F3F26E4F7D@python.org> 
	<4ACCCF29.7030003@scottdial.com>
	<8F1F8793-79E0-459C-9D6A-55AB79CD9EEC@python.org> 
	<4222a8490910150605x395465csa68177492dd275bd@mail.gmail.com> 
	<D82FD8E2-5DD2-40DD-B693-63F71AB0890C@python.org>
	<4222a8490910150700u358568b2l1cf8ba623f14c22f@mail.gmail.com> 
	<FB875E2D-6A3C-4CE3-A5E5-CE5215726DAC@python.org>
Message-ID: <ca471dc20910151057lc65a1fdx64e9cc6d112504ff@mail.gmail.com>

Thanks all!

On Thu, Oct 15, 2009 at 7:14 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 15, 2009, at 10:00 AM, Jesse Noller wrote:
>
>> On Thu, Oct 15, 2009 at 9:40 AM, Barry Warsaw <barry at python.org> wrote:
>>>
>>> On Oct 15, 2009, at 9:05 AM, Jesse Noller wrote:
>>>
>>>> Here's another one barry:
>>>>
>>>> http://bugs.python.org/issue7120
>>>>
>>>> We should get this in - it's a regression I introduced awhile ago for
>>>> environments without the multiprocessing module using logging.
>>>
>>> Was this a regression in 2.6.2 or 2.6.3? ?I think it might not matter,
>>> but
>>> let me know and I'll decide.
>>>
>>> -Barry
>>>
>>>
>>
>> My original commit was on 2009-01-18, you tagged 2.6.2 on Tue Apr 14
>> 13:17:30 2009, so it went into 2.6.2.
>>
>> The fix is minor, the bug affects things such as GAE which may use
>> logging, but not have multiprocessing available.
>
> Approved for 2.6.4rc2
>
> -Barry
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Thu Oct 15 21:11:25 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 15 Oct 2009 20:11:25 +0100
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <loom.20091015T170259-856@post.gmane.org>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
	<loom.20091015T170259-856@post.gmane.org>
Message-ID: <5c6f2a5d0910151211g199f5efq7aa4ff8f92f07daf@mail.gmail.com>

On Thu, Oct 15, 2009 at 4:06 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> <skip <at> pobox.com> writes:
>>
>> I notice that WITHOUT_COMPLEX still appears in Python.h and several .c files
>> but nowhere else in the 2.6, 2.7 or 3.1 source, most particularly not in
>> configure or pyconfig.h.in. ?Are builds --without-complex still supported?
>> Has it been tested at any time in the recent past?
>
> Is there any point in building without complex? Size reduction perhaps?
> If nobody uses it, we could remove that option. We have trouble staying
> compatible with lots of build options (see how --without-threads is little
> exercised).

Size reduction is the only point I can think of.

There's one respect in which complex is slightly more tightly
integrated in py3k than in trunk:  raising a negative number to a
non-integer power (e.g., (-1)**0.5) gives a complex result in py3k.

In trunk this raises ValueError, which means that the only way to
get a complex number in trunk is if you explicitly ask for one
somehow (e.g., by invoking complex, or using the cmath module,
or using imaginary literals, ...), so it makes slightly more sense
to build without the complex type there.

+1 for removing the WITHOUT_COMPLEX define from py3k.
-0 for removing it from trunk.

Mark

From solipsis at pitrou.net  Thu Oct 15 21:17:42 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 15 Oct 2009 19:17:42 +0000 (UTC)
Subject: [Python-Dev] Can 3.1 still be built without complex?
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
	<loom.20091015T170259-856@post.gmane.org>
	<5c6f2a5d0910151211g199f5efq7aa4ff8f92f07daf@mail.gmail.com>
Message-ID: <loom.20091015T211650-552@post.gmane.org>

Mark Dickinson <dickinsm <at> gmail.com> writes:
> 
> There's one respect in which complex is slightly more tightly
> integrated in py3k than in trunk:  raising a negative number to a
> non-integer power (e.g., (-1)**0.5) gives a complex result in py3k.

>>> (-1)**.5
(6.123031769111886e-17+1j)

Don't we have a precision problem here? 0.5 is supposed to be represented
exactly, isn't it?




From dickinsm at gmail.com  Thu Oct 15 21:42:50 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 15 Oct 2009 20:42:50 +0100
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <loom.20091015T211650-552@post.gmane.org>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
	<loom.20091015T170259-856@post.gmane.org>
	<5c6f2a5d0910151211g199f5efq7aa4ff8f92f07daf@mail.gmail.com>
	<loom.20091015T211650-552@post.gmane.org>
Message-ID: <5c6f2a5d0910151242q2bc80fdbq3d71fb16ca6a3669@mail.gmail.com>

On Thu, Oct 15, 2009 at 8:17 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>> (-1)**.5
> (6.123031769111886e-17+1j)
>
> Don't we have a precision problem here? 0.5 is supposed to be represented
> exactly, isn't it?

0.5 is represented exactly, but complex.__pow__ makes no pretence of
being correctly rounded (and making it correctly rounded would likely
be prohibitively expensive in terms of code size and complexity).  It's
using something like x**y = exp(y*log(x)) behind the scenes, at least
for computing the argument of the result.

For square roots, cmath.sqrt produces better results.

Mark

From skip.montanaro at gmail.com  Fri Oct 16 01:47:14 2009
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Thu, 15 Oct 2009 18:47:14 -0500
Subject: [Python-Dev] Can 3.1 still be built without complex?
In-Reply-To: <5c6f2a5d0910151242q2bc80fdbq3d71fb16ca6a3669@mail.gmail.com>
References: <20091015113930.7B0D812C1BE3@montanaro.dyndns.org>
	<loom.20091015T170259-856@post.gmane.org>
	<5c6f2a5d0910151211g199f5efq7aa4ff8f92f07daf@mail.gmail.com>
	<loom.20091015T211650-552@post.gmane.org>
	<5c6f2a5d0910151242q2bc80fdbq3d71fb16ca6a3669@mail.gmail.com>
Message-ID: <60bb7ceb0910151647k2b311e30x70fe6355c09a9f9a@mail.gmail.com>

http://bugs.python.org/issue7147

Passes test (no big surprise there).  Doesn't yet include any changes
to documentation
or Misc/NEWS entry.  Should this be decided I will take a look at that.

Skip

From nas at arctrix.com  Fri Oct 16 03:06:59 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu, 15 Oct 2009 19:06:59 -0600
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091014201635.GA28053@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
Message-ID: <20091016010659.GA4189@arctrix.com>

On Wed, Oct 14, 2009 at 02:16:35PM -0600, Neil Schemenauer wrote:
> The procedure implemented by http://bugs.python.org/issue812369
> seems to be a better idea.

After some experimentation I realize this idea is not ready yet.
The main problem comes from references to Python objects that
modules keep but don't expose to the garbage collector. For example,
gcmodule.c has a static pointer "tmod" that is a reference to the
"time" module. This reference prevents the "time" module from being
freed during interpreter shutdown.

Ideally, I suppose modules should be treated like any other object
and have tp_traverse and tp_clear methods that deal with these sorts
of pointers. They would have to delegated to the instance since each
module would have its own implementation.

  Neil

From amauryfa at gmail.com  Fri Oct 16 10:01:07 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 16 Oct 2009 10:01:07 +0200
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <20091016010659.GA4189@arctrix.com>
References: <20091014201635.GA28053@arctrix.com>
	<20091016010659.GA4189@arctrix.com>
Message-ID: <e27efe130910160101o4bf58d8dk630d8b549f511cdd@mail.gmail.com>

2009/10/16 Neil Schemenauer <nas at arctrix.com>:
> After some experimentation I realize this idea is not ready yet.
> The main problem comes from references to Python objects that
> modules keep but don't expose to the garbage collector. For example,
> gcmodule.c has a static pointer "tmod" that is a reference to the
> "time" module. This reference prevents the "time" module from being
> freed during interpreter shutdown.
>
> Ideally, I suppose modules should be treated like any other object
> and have tp_traverse and tp_clear methods that deal with these sorts
> of pointers. They would have to delegated to the instance since each
> module would have its own implementation.

Note since python 3.0 (and PEP 3121), the PyModuleDef structure has
some members like m_traverse, m_clear and m_free for this very
purpose.
So far, nobody cared to implement these methods for any module. Maybe
one should start at least for static PyObject* that contain references
to modules.

-- 
Amaury Forgeot d'Arc

From benjamin at python.org  Fri Oct 16 16:11:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 16 Oct 2009 09:11:17 -0500
Subject: [Python-Dev] Better module shutdown procedure
In-Reply-To: <e27efe130910160101o4bf58d8dk630d8b549f511cdd@mail.gmail.com>
References: <20091014201635.GA28053@arctrix.com>
	<20091016010659.GA4189@arctrix.com>
	<e27efe130910160101o4bf58d8dk630d8b549f511cdd@mail.gmail.com>
Message-ID: <1afaf6160910160711jd83c02ah1498d13c087cc9a6@mail.gmail.com>

2009/10/16 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> 2009/10/16 Neil Schemenauer <nas at arctrix.com>:
>> After some experimentation I realize this idea is not ready yet.
>> The main problem comes from references to Python objects that
>> modules keep but don't expose to the garbage collector. For example,
>> gcmodule.c has a static pointer "tmod" that is a reference to the
>> "time" module. This reference prevents the "time" module from being
>> freed during interpreter shutdown.
>>
>> Ideally, I suppose modules should be treated like any other object
>> and have tp_traverse and tp_clear methods that deal with these sorts
>> of pointers. They would have to delegated to the instance since each
>> module would have its own implementation.
>
> Note since python 3.0 (and PEP 3121), the PyModuleDef structure has
> some members like m_traverse, m_clear and m_free for this very
> purpose.
> So far, nobody cared to implement these methods for any module. Maybe
> one should start at least for static PyObject* that contain references
> to modules.

I believe the implementation is buggy because modules (and their
states) can easily be finalized before the objects contained in them.
For example, when I tried to convert the _io module to use a state, it
resulted in segfaults when the dealloc methods of objects tried to use
objects in a state, which had already been deallocated.



-- 
Regards,
Benjamin

From status at bugs.python.org  Fri Oct 16 18:07:37 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 16 Oct 2009 18:07:37 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091016160737.B03477803E@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/09/09 - 10/16/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2468 open (+37) / 16504 closed (+20) / 18972 total (+57)

Open issues with patches:   982

Average duration of open issues: 669 days.
Median duration of open issues: 423 days.

Open Issues Breakdown
   open  2432 (+37)
pending    35 ( +0)

Issues Created Or Reopened (58)
_______________________________

Python 2.6.3 / setuptools 0.6c9: extension module builds fail wi 10/12/09
CLOSED http://bugs.python.org/issue7064    reopened barry                         
                                                                               

Add alternate float formatting styles to new-style formatting.   10/09/09
       http://bugs.python.org/issue7094    created  mark.dickinson                
                                                                               

Multiprocessing.Array(lock=False) fails                          10/09/09
CLOSED http://bugs.python.org/issue7095    created  schlesin                      
                                                                               

test_curses fails on 3.1 when run under regrtest                 10/10/09
       http://bugs.python.org/issue7096    created  r.david.murray                
                                                                               

urlparse.urljoin of simple "http://" and "somedomain.com" produc 10/10/09
CLOSED http://bugs.python.org/issue7097    created  ssteiner                      
                                                                               

g formatting for decimal types should always strip trailing zero 10/10/09
       http://bugs.python.org/issue7098    created  mark.dickinson                
       patch                                                                   

Decimal.is_normal should return True even for numbers with expon 10/10/09
       http://bugs.python.org/issue7099    created  mark.dickinson                
                                                                               

test_xmlrpc: global name 'stop_serving' is not defined           10/10/09
       http://bugs.python.org/issue7100    created  pitrou                        
                                                                               

tarfile: OSError with TarFile.add(..., recursive=True) about non 10/10/09
       http://bugs.python.org/issue7101    created  denis                         
                                                                               

Problems building pyhton from source on Snow Leopard (Mac OS X   10/11/09
       http://bugs.python.org/issue7102    created  FredrikHedman                 
                                                                               

Error in config parser example (w/ Patch)                        10/11/09
CLOSED http://bugs.python.org/issue7103    created  tcourbon                      
       patch                                                                   

test_descr uses __cmp__ which will never be called               10/11/09
CLOSED http://bugs.python.org/issue7104    created  stutzbach                     
                                                                               

weak dict iterators are fragile because of unpredictable GC runs 10/11/09
       http://bugs.python.org/issue7105    created  pitrou                        
       patch                                                                   

Python crashed in the middle of an argument between wx	and boa_c 10/11/09
CLOSED http://bugs.python.org/issue7106    created  bvn                           
                                                                               

Missing uninstallation instructions for mac                      10/11/09
       http://bugs.python.org/issue7107    created  gradha                        
                                                                               

test_commands.py failing on OS X 10.5.7 due to '@' in ls output  10/11/09
       http://bugs.python.org/issue7108    created  minge                         
       patch                                                                   

broken link on news page                                         10/12/09
CLOSED http://bugs.python.org/issue7109    created  mkiever                       
                                                                               

Output test failures on stderr in regrtest.py                    10/12/09
       http://bugs.python.org/issue7110    created  ezio.melotti                  
                                                                               

core dump when stderr is moved                                   10/12/09
       http://bugs.python.org/issue7111    created  petere                        
                                                                               

unicodetype_db.h warning: integer constant is too large	for 'lon 10/12/09
CLOSED http://bugs.python.org/issue7112    created  ocean-city                    
                                                                               

ConfigParser load speedup                                        10/12/09
       http://bugs.python.org/issue7113    created  aioryi                        
       patch                                                                   

HTMLParser doesn't handle <![CDATA[ ... ]]>                      10/12/09
       http://bugs.python.org/issue7114    created  ggbaker                       
                                                                               

extension module builds fail when using paths in the extension n 10/12/09
CLOSED http://bugs.python.org/issue7115    created  tarek                         
                                                                               

str.join() should be documented as taking an iterable            10/12/09
CLOSED http://bugs.python.org/issue7116    created  jess.austin                   
       patch                                                                   

Backport py3k float repr to trunk                                10/13/09
       http://bugs.python.org/issue7117    created  mark.dickinson                
                                                                               

urllib2.urlopen() timeout argument ignored after redirect        10/13/09
       http://bugs.python.org/issue7118    created  gthaler                       
                                                                               

email:  msg.items() returns different values before and after ms 10/13/09
       http://bugs.python.org/issue7119    created  jnelson                       
                                                                               

logging depends on multiprocessing                               10/14/09
CLOSED http://bugs.python.org/issue7120    created  gvanrossum                    
                                                                               

ImportError of urllib.request & http.client under PYTHONHOME/Lib 10/14/09
CLOSED http://bugs.python.org/issue7121    created  mikezp59                      
                                                                               

multiprocessing.Pool() problem in windows                        10/14/09
CLOSED http://bugs.python.org/issue7122    created  SirLalala                     
                                                                               

Multiprocess Process does not always exit when run from a thread 10/14/09
       http://bugs.python.org/issue7123    created  pajs at fodder.org.uk            
                                                                               

idle.py -n : help() doesn't work in a reopened shell window      10/14/09
       http://bugs.python.org/issue7124    created  gregorlingl                   
                                                                               

typo (English) in threading.py                                   10/14/09
CLOSED http://bugs.python.org/issue7125    created  Yinon                         
                                                                               

Contradictory documentation for os.putenv and os.system          10/14/09
CLOSED http://bugs.python.org/issue7126    created  stutzbach                     
                                                                               

regrtest -j fails when tests write to stderr                     10/14/09
       http://bugs.python.org/issue7127    created  r.david.murray                
       easy                                                                    

cPickle looking for non-existent package copyreg                 10/14/09
       http://bugs.python.org/issue7128    created  joequant                      
       patch, easy                                                             

'filling' missing in __all__ ot turtle.py                        10/14/09
CLOSED http://bugs.python.org/issue7129    created  gregorlingl                   
       patch                                                                   

json uses sre_* modules which may not work on alternate implemen 10/14/09
CLOSED http://bugs.python.org/issue7130    created  DinoV                         
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat    10/14/09
       http://bugs.python.org/issue7131    created  dieterv                       
                                                                               

Regexp: capturing groups in repetitions                          10/14/09
       http://bugs.python.org/issue7132    created  verdy_p                       
                                                                               

test_ssl failure                                                 10/14/09
       http://bugs.python.org/issue7133    created  pitrou                        
       patch                                                                   

Add looping capability to regrtest                               10/14/09
       http://bugs.python.org/issue7134    created  jnoller                       
                                                                               

AttributeError: 'module' object has no attribute 'HTTPSConnectio 10/14/09
CLOSED http://bugs.python.org/issue7135    created  awelch                        
                                                                               

Idle File Menu Option Improvement                                10/15/09
       http://bugs.python.org/issue7136    created  tjd                           
                                                                               

Socket documentation not updated                                 10/15/09
       http://bugs.python.org/issue7137    created  yang                          
                                                                               

elementtree segfaults on invalid xml declaration                 10/15/09
       http://bugs.python.org/issue7138    created  whichlinden                   
                                                                               

ElementTree: Incorrect serialization of end-of-line characters i 10/15/09
       http://bugs.python.org/issue7139    created  moriyoshi                     
                                                                               

imp.new_module does not function correctly if the module is retu 10/15/09
CLOSED http://bugs.python.org/issue7140    created  jkp                           
                                                                               

2to3 should add from __future__ import print_statement           10/15/09
       http://bugs.python.org/issue7141    created  stutzbach                     
                                                                               

__repr__ of memoryview object has type unicode; should be str.   10/15/09
CLOSED http://bugs.python.org/issue7142    created  mark.dickinson                
       patch                                                                   

get_payload(decode=True) eats last newline                       10/15/09
       http://bugs.python.org/issue7143    created  athomas                       
                                                                               

imp.load_module in thread causes core dump on OSX 10.6           10/15/09
       http://bugs.python.org/issue7144    created  peterhunt                     
                                                                               

UTF-16 BOM is not written by socket.makefile (but is expected by 10/15/09
       http://bugs.python.org/issue7145    created  yang                          
                                                                               

[PATCH] platform.uname()[4] returns 'amd64' on Windows and 'x86- 10/15/09
       http://bugs.python.org/issue7146    created  zooko                         
                                                                               

Remove WITHOUT_COMPLEX from 3.x trunk                            10/15/09
       http://bugs.python.org/issue7147    created  skip.montanaro                
       patch, patch, easy, needs review                                        

socket.py: r75412 broke build                                    10/16/09
CLOSED http://bugs.python.org/issue7148    created  kbk                           
                                                                               

2.6.4rc1 regression: test_urllib2 fails on OS X with UnboundLoca 10/16/09
       http://bugs.python.org/issue7149    created  ned.deily                     
                                                                               

datetime operations spanning MINYEAR give bad results            10/16/09
       http://bugs.python.org/issue7150    created  mark.leander                  
                                                                               



Issues Now Closed (37)
______________________

sys.exit() called from optparse - bad, bad, bad                   491 days
       http://bugs.python.org/issue3079    pfeldman at verizon.net          
       easy                                                                    

distutils.util.get_platform() is wrong for universal builds on m  369 days
       http://bugs.python.org/issue4064    ned.deily                     
       patch, needs review                                                     

ihooks module cannot handle absolute imports                      359 days
       http://bugs.python.org/issue4152    nascheme                      
                                                                               

PEP 383 implementation                                            163 days
       http://bugs.python.org/issue5915    loewis                        
       patch                                                                   

distutils builds extension modules to root package directory      103 days
       http://bugs.python.org/issue6403    srid                          
                                                                               

ihooks support for relative imports                                38 days
       http://bugs.python.org/issue6855    brett.cannon                  
                                                                               

urllib2 doesn't respect "no_proxy" environment  (python2.6.2)      29 days
       http://bugs.python.org/issue6894    orsenthil                     
       patch                                                                   

test_urllib: constructLocalFileUrl returns invalid URLs on Windo   11 days
       http://bugs.python.org/issue7043    orsenthil                     
                                                                               

Automatic test___all__                                              6 days
       http://bugs.python.org/issue7055    pitrou                        
       patch                                                                   

Python 2.6.3 / setuptools 0.6c9: extension module builds fail wi    1 days
       http://bugs.python.org/issue7064    tarek                         
                                                                               

bytes.maketrans segfaults                                           9 days
       http://bugs.python.org/issue7065    pitrou                        
       patch                                                                   

inspect.isabstract to return boolean values only                    9 days
       http://bugs.python.org/issue7069    benjamin.peterson             
       patch                                                                   

logging.handlers.SysLogHandler with TCP support                     2 days
       http://bugs.python.org/issue7086    vinay.sajip                   
       patch                                                                   

Distutils build ignores the --compiler command line option          5 days
       http://bugs.python.org/issue7091    tarek                         
                                                                               

Multiprocessing.Array(lock=False) fails                             0 days
       http://bugs.python.org/issue7095    jnoller                       
                                                                               

urlparse.urljoin of simple "http://" and "somedomain.com" produc    0 days
       http://bugs.python.org/issue7097    brett.cannon                  
                                                                               

Error in config parser example (w/ Patch)                           0 days
       http://bugs.python.org/issue7103    r.david.murray                
       patch                                                                   

test_descr uses __cmp__ which will never be called                  0 days
       http://bugs.python.org/issue7104    benjamin.peterson             
                                                                               

Python crashed in the middle of an argument between wx	and boa_c    0 days
       http://bugs.python.org/issue7106    bvn                           
                                                                               

broken link on news page                                            0 days
       http://bugs.python.org/issue7109    mkiever                       
                                                                               

unicodetype_db.h warning: integer constant is too large	for 'lon    2 days
       http://bugs.python.org/issue7112    amaury.forgeotdarc            
                                                                               

extension module builds fail when using paths in the extension n    1 days
       http://bugs.python.org/issue7115    tarek                         
                                                                               

str.join() should be documented as taking an iterable               2 days
       http://bugs.python.org/issue7116    georg.brandl                  
       patch                                                                   

logging depends on multiprocessing                                  3 days
       http://bugs.python.org/issue7120    vinay.sajip                   
                                                                               

ImportError of urllib.request & http.client under PYTHONHOME/Lib    0 days
       http://bugs.python.org/issue7121    ezio.melotti                  
                                                                               

multiprocessing.Pool() problem in windows                           0 days
       http://bugs.python.org/issue7122    amaury.forgeotdarc            
                                                                               

typo (English) in threading.py                                      0 days
       http://bugs.python.org/issue7125    georg.brandl                  
                                                                               

Contradictory documentation for os.putenv and os.system             0 days
       http://bugs.python.org/issue7126    georg.brandl                  
                                                                               

'filling' missing in __all__ ot turtle.py                           0 days
       http://bugs.python.org/issue7129    georg.brandl                  
       patch                                                                   

json uses sre_* modules which may not work on alternate implemen    0 days
       http://bugs.python.org/issue7130    pitrou                        
                                                                               

AttributeError: 'module' object has no attribute 'HTTPSConnectio    0 days
       http://bugs.python.org/issue7135    r.david.murray                
                                                                               

imp.new_module does not function correctly if the module is retu    0 days
       http://bugs.python.org/issue7140    benjamin.peterson             
                                                                               

__repr__ of memoryview object has type unicode; should be str.      0 days
       http://bugs.python.org/issue7142    mark.dickinson                
       patch                                                                   

socket.py: r75412 broke build                                       0 days
       http://bugs.python.org/issue7148    kbk                           
                                                                               

dict subclass breaks cPickle noload()                            1736 days
       http://bugs.python.org/issue1101399 nascheme                      
                                                                               

Digest Authentication not working in all cases                   1499 days
       http://bugs.python.org/issue1285440 orsenthil                     
       patch, easy                                                             

Tighter co_stacksize computation in stackdepth_walk               823 days
       http://bugs.python.org/issue1754094 nascheme                      
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 23 Regexp: capturing groups in repetitions                            2 days
open    http://bugs.python.org/issue7132   

 19 logging depends on multiprocessing                                 3 days
closed  http://bugs.python.org/issue7120   

 13 Compilation error if configuref --with-computed-gotos             78 days
open    http://bugs.python.org/issue6603   

 11 Multiprocess Process does not always exit when run from a threa    2 days
open    http://bugs.python.org/issue7123   

  9 elementtree segfaults on invalid xml declaration                   1 days
open    http://bugs.python.org/issue7138   

  7 Python 2.6.3 / setuptools 0.6c9: extension module builds fail w    1 days
closed  http://bugs.python.org/issue7064   

  7 test_multiprocessing dictionary changed size errors and hang      12 days
open    http://bugs.python.org/issue7060   

  7 Unicode case mappings are incorrect                              311 days
open    http://bugs.python.org/issue4610   

  6 core dump when stderr is moved                                     4 days
open    http://bugs.python.org/issue7111   

  6 Test suite emits many DeprecationWarnings when -3 is enabled       7 days
open    http://bugs.python.org/issue7092   




From gregor.lingl at aon.at  Fri Oct 16 23:14:05 2009
From: gregor.lingl at aon.at (Gregor Lingl)
Date: Fri, 16 Oct 2009 23:14:05 +0200
Subject: [Python-Dev] Tkinter script crashes Python 3.1
Message-ID: <4AD8E21D.5010307@aon.at>

I've written a small Tkinter-Script, that crashes Python 3.1 (but not 
Python 2.6) without any specific rrror message. When started from within 
IDLE, the failing of the script also closes IDLE. (This is the case 
under Windows, Mac OS X and Linux (teted with Ubuntu 9.04))

Bug-Tracker Issue 6717
The script is attached to this issue

I think, errors like this should definitely not occur. Instead a message 
like "recusion depth exceeded" should be displayed (and IDLE should 
remain functional, if used)

Since I do not have the means availble to track down this bug, I'd like 
to draw your attention to it and to ask if someone else has the means 
and time to do so.

I'd also suggest to increase the priority of this bug in the bugtracker.

Regards,
Gregor




From vinay_sajip at yahoo.co.uk  Sat Oct 17 09:23:50 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 17 Oct 2009 07:23:50 +0000 (UTC)
Subject: [Python-Dev] A proposal: configuring logging using dictionaries
Message-ID: <loom.20091017T091238-255@post.gmane.org>

A little while ago, I posted here a suggestion about a new way to configure
logging, using dictionaries. This received some positive and no negative
feedback, so I have thought some more about the details of how it might work. I
present below the results of that thinking, in a PEP-style format. I don't know
if an actual PEP is required for a change of this type, but I felt that it's
still worth going through the exercise to try and achieve a reasonable level of
rigour. (I hope I've succeeded.)

I would welcome all your feedback on this proposal. If I hear no negative
feedback, I propose to implement this feature as suggested.

I thought about posting this on comp.lang.python as well, but possibly it's a
little too much information for most of the folks there. I think it would be
useful to get feedback from the wider community, though, and welcome any
suggestions on how best to achieve this.

Thanks and regards,

Vinay Sajip
-----------
PEP: XXX
Title: Dictionary-Based Configuration For Logging
Version: $Revision$
Last-Modified: $Date$
Author: Vinay Sajip <vinay_sajip at red-dove.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 15-Oct-2009
Python-Version: 2.7 and 3.2
Post-History:


Abstract
========

This PEP describes a new way of configuring logging using a dictionary to hold
configuration information.

Rationale
=========

The present means for configuring Python's logging package is either by using
the logging API to configure logging programmatically, or else by means of
ConfigParser-based configuration files.

Programmatic configuration, while offering maximal control, fixes the
configuration in Python code.  This does not facilitate changing it easily at
runtime, and, as a result, the ability to flexibly turn the verbosity of
logging up and down for different parts of a using application is lost.  This
limits the usability of logging as an aid to diagnosing problems - and
sometimes, logging is the only diagnostic aid available in production
environments.

The ConfigParser-based configuration system is usable, but does not allow its
users to configure all aspects of the logging package.  For example, Filters
cannot be configured using this system.  Furthermore, the ConfigParser format
appears to engender dislike (sometimes strong dislike) in some quarters.
Though it was chosen because it was the only configuration format supported in
the Python standard at that time, many people regard it (or perhaps just the
particular schema chosen for logging's configuration) as 'crufty' or 'ugly',
in some cases apparently on purely aesthetic grounds.

Recent versions of Python inlude JSON support in the standard library, and
this is also usable as a configuration format.  In other environments, such as
Google App Engine, YAML is used to configure applications, and usually the
configuration of logging would be considered an integral part of the
application configuration.  Although the standard library does not contain
YAML support at present, support for both JSON and YAML can be provided in a
common way because both of these serialization formats allow deserialization
of Python dictionaries.

By providing a way to configure logging by passing the configuration in a
dictionary, logging will be easier to configure not only for users of JSON
and/or YAML, but also for users of bespoke configuration methods, by providing
a common format in which to describe the desired configuration.

Another drawback of the current ConfigParser-based configuration system is
that it does not support incremental configuration: a new configuration
completely replaces the existing configuration.  Although full flexibility for
incremental configuration is difficult to provide in a multi-threaded
environment, the new configuration mechanism will allow the provision of
limited support for incremental configuration.

Specification
=============

The specification consists of two parts: the API and the format of the
dictionary used to convey configuration information (i.e. the schema to which
it must conform).

Naming
------

Historically, the logging package has not been PEP-8 conformant.  At some
future time, this will be corrected by changing method and function names in
the package in order to conform with PEP-8.  However, in the interests of
uniformity, the proposed additions to the API use a naming scheme which is
consistent with the present scheme used by logging.

API
---

The logging.config module will have the following additions:

* A class, called ``DictConfigurator``, whose constructor is passed the
  dictionary used for configuration, and which has a ``configure()`` method.

* A callable, called ``dictConfigClass``, which will (by default) be set to
  ``DictConfigurator``.  This is provided so that if desired,
  ``DictConfigurator`` can be replaced with a suitable user-defined
  implementation.

* A function, called ``dictConfig()``, which takes a single argument - the
  dictionary holding the configuration.  This function will call
  ``dictConfigClass`` passing the specified dictionary, and then call the
  ``configure()`` method on the returned object to actually put the 
  configuration into effect::
  
    def dictConfig(config):
        dictConfigClass(config).configure()

Dictionary Schema - Overview
----------------------------

Before describing the schema in detail, it is worth saying a few words about
object connections, support for user-defined objects and access to external
objects.

Object connections
''''''''''''''''''

The schema is intended to describe a set of logging objects - loggers,
handlers, formatters, filters - which are connected to each other in an
object graph.  Thus, the schema needs to represent connections between the
objects.  For example, say that, once configured, a particular logger has an
attached to it a particular handler.  For the purposes of this discussion,
we can say that the logger represents the source, and the handler the
destination, of a connection between the two.  Of course in the configured
objects this is represented by the logger holding a reference to the
handler.  In the configuration dict, this is done by giving each destination
object an id which identifies it unambiguously, and then using the id in the
source object's configuration to indicate that a connection exists between
the source and the destination object with that id.
  
So, for example, consider the following YAML snippet::
  
  handers:
    h1: #This is an id
     # configuration of handler with id h1 goes here
    h2: #This is another id
     # configuration of handler with id h2 goes here
  loggers:
    foo.bar.baz:
      # other configuration for logger "foo.bar.baz"
      handlers: [h1, h2]

(Note: YAML will be used in this document as it is more readable than the
equivalent Python source form for the dictionary.)
  
The ids for loggers are the logger names which would be used
programmatically to obtain a reference to those loggers, e.g.
``foo.bar.baz``.  The ids for other objects can be any string value (such as
``h1``, ``h2`` above) and they are transient, in that they are only
meaningful for processing the configuration dictionary and used to
determine connections between objects, and are not persisted anywhere when
the configuration call is complete.
  
The above snippet indicates that logger named ``foo.bar.baz`` should have
two handlers attached to it, which are described by the handler ids ``h1``
and ``h2``.

User-defined objects
''''''''''''''''''''

The schema should support user-defined objects for handlers, filters and
formatters.  (Loggers do not need to have different types for different
instances, so there is no support - in the configuration - for user-defined
logger classes.)

Objects to be configured will typically be described by dictionaries which
detail their configuration.  In some places, the logging system will be able
to infer from the context how an object is to be instantiated, but when a
user-defined object is to be instantiated, the system will not know how to do
this.  In order to provide complete flexibility for user-defined object
instantiation, the user will need to provide a 'factory' - a callable which
is called with a configuration dictionary and which returns the instantiated
object.  This will be signalled by the factory being made available under
the special key ``'()'``.  Here's a concrete example::
  
  formatters:
    brief:
      format: '%(message)s'
    default:
      format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s'
      datefmt: '%Y-%m-%d %H:%M:%S'
    custom:
        (): my.package.customFormatterFactory
        bar: baz
        spam: 99.9
        answer: 42

The above YAML snippet defines three formatters.  The first, with id
``brief``, is a standard ``logging.Formatter`` instance with the
specified format string.  The second, with id ``default``, has a longer
format and also defines the time format explicitly, and will result in a
``logging.Formatter`` initialized with those two format strings.  Shown in
Python source form, the ``brief`` and ``default`` formatters have
have configuration sub-dictionaries::
  
  {
    'format' : '%(message)s'
  }
    
and::
  
  {
    'format' : '%(asctime)s %(levelname)-8s %(name)-15s %(message)s',
    'datefmt' : '%Y-%m-%d %H:%M:%S'
  }
  
respectively, and as these dictionaries do not contain the special key
``'()'``, the instantiation is inferred from the context: as a result,
standard ``logging.Formatter`` instances are created.  The configuration
sub-dictionary for the third formatter, with id ``custom``, is::

  {
    '()' : 'my.package.customFormatterFactory',
    'bar' : 'baz',
    'spam' : 99.9,
    'answer' : 42
  }
  
and this contains the special key ``'()'``, which means that user-defined
instantiation is wanted.  In this case, the specified factory callable will be
located using normal import mechanisms and called with the *remaining* items
in the configuration sub-dictionary as keyword arguments.  In the above
example, the formatter with id ``custom`` will be assumed to be returned by
the call::

  my.package.customFormatterFactory(bar="baz", spam=99.9, answer=42)  

The key ``'()'`` has been used as the special key because it is not a valid
keyword parameter name, and so will not clash with the names of the keyword
arguments used in the call.  The ``'()'`` also serves as a mnemonic that the
corresponding value is a callable.

Access to external objects
''''''''''''''''''''''''''

There are times where a configuration will need to refer to objects external
to the configuration, for example ``sys.stderr``.  If the configuration dict
is constructed using Python code then this is straightforward, but a problem
arises when the configuration is provided via a text file (e.g. JSON, YAML).
In a text file, there is no standard way to distinguish ``sys.stderr`` from
the literal string ``'sys.stderr'``.  To facilitate this distinction, the
configuration system will look for certain special prefixes in string values
and treat them specially. For example, if the literal string
``'ext://sys.stderr'`` is provided as a value in the configuration, then the
``ext://`` will be stripped off and the remainder of the value processed using
normal import mechanisms.

The handling of such prefixes will be done in a way analogous to protocol
handling: there will be a generic mechanism to look for prefixes which match
the regular expression ``^(?P<prefix>[a-z]+)://(?P<suffix>.*)$`` whereby, if
the ``prefix`` is recognised, the ``suffix`` is processed in a prefix-
dependent manner and the result of the processing replaces the string value.
If the prefix is not recognised, then the string value will be left as-is.

The implementation will provide for a set of standard prefixes such as
``ext://`` but it will be possible to disable the mechanism completely or
provide additional or different prefixes for special handling.

Dictionary Schema - Detail
--------------------------

The dictionary passed to ``dictConfig()`` must contain the following keys:

* `version` - to be set to an integer value representing the schema
  version.  The only valid value at present is 1, but having this key allows
  the schema to evolve while still preserving backwards compatibility.

All other keys are optional, but if present they will be interpreted as described
below.  In all cases below where a 'configuring dict' is mentioned, it will be
checked for the special ``'()'`` key to see if a custom instantiation is
required.  If so, the mechanism described above is used to instantiate;
otherwise, the context is used to determine how to instantiate.

* `formatters` - the corresponding value will be a dict in which each key is
  a formatter id and each value is a dict describing how to configure the
  corresponding Formatter instance.
  
  The configuring dict is searched for keys ``format`` and ``datefmt`` (with
  defaults of ``None``) and these are used to construct a
  ``logging.Formatter`` instance.

* `filters` - the corresponding value will be a dict in which each key is
  a filter id and each value is a dict describing how to configure the
  corresponding Filter instance.

  The configuring dict is searched for key ``name`` (defaulting to the empty
  string) and this is used to construct a ``logging.Filter`` instance.

* `handlers` - the corresponding value will be a dict in which each key is
  a handler id and each value is a dict describing how to configure the
  corresponding Handler instance.

  The configuring dict is searched for the following keys:
  
  * ``class`` (mandatory).  This is the fully qualified name of the handler
    class.
    
  * ``level`` (optional).  The level of the handler.
  
  * ``formatter`` (optional).  The id of the formatter for this handler.
  
  * ``filters`` (optional).  A list of ids of the filters for this handler.

  All *other* keys are passed through as keyword arguments to the handler's
  constructor.  For example, given the snippet::

    handlers:
      console:
        class : logging.StreamHandler
        formatter: brief
        level   : INFO
        filters: [allow_foo]
        stream  : ext://sys.stdout
      file:
        class : logging.handlers.RotatingFileHandler
        formatter: precise
        filename: logconfig.log
        maxBytes: 1024
        backupCount: 3

  the handler with id ``console`` is instantiated as a
  ``logging.StreamHandler``, using ``sys.stdout`` as the underlying stream.
  The handler with id ``file`` is instantiated as a
  ``logging.handlers.RotatingFileHandler`` with the keyword arguments
  ``filename="logconfig.log", maxBytes=1024, backupCount=3``.

* `loggers` - the corresponding value will be a dict in which each key is
  a logger name and each value is a dict describing how to configure the
  corresponding Logger instance.

  The configuring dict is searched for the following keys:
  
  * ``level`` (optional).  The level of the logger.
  
  * ``propagate`` (optional).  The propagation setting of the logger.
  
  * ``filters`` (optional).  A list of ids of the filters for this logger.

  * ``handlers`` (optional).  A list of ids of the handlers for this logger.

  The specified loggers will be configured according to the level,
  propagation, filters and handlers specified.

* `root` - this will be the configuration for the root logger. Processing of
  the configuration will be as for any logger, except that the ``propagate``
  setting will not be applicable.
  
* `incremental` - whether the configuration is to be interpreted as
  incremental to the existing configuration. This value defaults to False,
  which means that the specified configuration replaces the existing
  configuration with the same semantics as used by the existing
  ``fileConfig()`` API.
  
  If the specified value is True, the configuration is processed as described
  in the section on "Incremental Configuration", below.

A Working Example
-----------------

The following is an actual working configuration in YAML format (except that
the email addresses are bogus)::

    formatters:
      brief:
        format: '%(levelname)-8s: %(name)-15s: %(message)s'
      precise:
        format: '%(asctime)s %(name)-15s %(levelname)-8s %(message)s'
    filters:
      allow_foo:
        name: foo
    handlers:
      console:
        class : logging.StreamHandler
        formatter: brief
        level   : INFO
        stream  : ext://sys.stdout
        filters: [allow_foo]
      file:
        class : logging.handlers.RotatingFileHandler
        formatter: precise
        filename: logconfig.log
        maxBytes: 1024
        backupCount: 3
      debugfile:
        class : logging.FileHandler
        formatter: precise
        filename: logconfig-detail.log
        mode: a
      email:
        class: logging.handlers.SMTPHandler
        mailhost: localhost
        fromaddr: my_app at domain.tld
        toaddrs:
          - support_team at domain.tld
          - dev_team at domain.tld
        subject: Houston, we have a problem.
    loggers:
      foo:
        level : ERROR
        handlers: [debugfile]
      spam:
        level : CRITICAL
        handlers: [debugfile]
        propagate: no
      bar.baz:
        level: WARNING
    root:
      level     : DEBUG
      handlers  : [console, file]

Incremental Configuration
=========================

It is difficult to provide complete flexibility for incremental configuration.
For example, because objects such as handlers, filters and formatters are
anonymous, once a configuration is set up, it is not possible to refer to such
anonymous objects when augmenting a configuration. For example, if an initial
call is made to configure the system where logger ``foo`` has a handler with
id ``console`` attached, then a subsequent call to configure a logger ``bar``
with id ``console`` would create a new handler instance, as the id ``console``
from the first call isn't kept.

Furthermore, there is not a compelling case for arbitrarily altering the
object graph of loggers, handlers, filters, formatters at run-time, once a 
configuration is set up; the verbosity of loggers can be controlled just by
setting levels (and perhaps propagation flags).

Thus, when the ``incremental`` key of a configuration dict is present and
is ``True``, the system will ignore the ``formatters``, ``filters``,
``handlers`` entries completely, and process only the ``level`` and
``propagate`` settings in the ``loggers`` and ``root`` entries.

Configuration Errors
====================

If an error is encountered during configuration, the system will raise a
``ValueError`` or a ``TypeError`` with a suitably descriptive message. The
following is a (possibly incomplete) list of conditions which will raise an
error:

* A ``level`` which is not a string or which is a string not corresponding to
  an actual logging level

* A ``propagate`` value which is not a Boolean

* An id which does not have a corresponding destination

* An invalid logger name

Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:




From ncoghlan at gmail.com  Sat Oct 17 10:10:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 17 Oct 2009 18:10:49 +1000
Subject: [Python-Dev] A proposal: configuring logging using dictionaries
In-Reply-To: <loom.20091017T091238-255@post.gmane.org>
References: <loom.20091017T091238-255@post.gmane.org>
Message-ID: <4AD97C09.3070705@gmail.com>

Vinay Sajip wrote:
> A little while ago, I posted here a suggestion about a new way to configure
> logging, using dictionaries. This received some positive and no negative
> feedback, so I have thought some more about the details of how it might work. I
> present below the results of that thinking, in a PEP-style format. I don't know
> if an actual PEP is required for a change of this type, but I felt that it's
> still worth going through the exercise to try and achieve a reasonable level of
> rigour. (I hope I've succeeded.)

Since part of the purpose of this new configuration option is to allow
other projects to use it as a target for specifying a logging
configuration, maintaining it as a formal PEP seems like a good idea to
me. This is already done for a few other standard-ish interfaces
(distutils metadata, DB-API, WSGI).


> I would welcome all your feedback on this proposal. If I hear no negative
> feedback, I propose to implement this feature as suggested.

My only real objection is to the "incremental" flag in the dictionary. I
would suggest having two different API methods instead - initDictConfig
and updateDictConfig. Or is there some specific reason for having the
choice of incremental update vs replacement configuration in the hands
of the config author rather than the application developer?

My other question is whether or not it would be worth having the logging
module able to export it's *current* configuration in dictionary form. I
don't have a use case other than that it might be useful for debugging
logging configuration errors when attempting to combine multiple sources
of logging data, but it's worth considering.

> So, for example, consider the following YAML snippet::
>   
>   handers:
>     h1: #This is an id
>      # configuration of handler with id h1 goes here
>     h2: #This is another id
>      # configuration of handler with id h2 goes here

Typo in the header here: s/handers/handlers/


> Configuration Errors
> ====================
> 
> If an error is encountered during configuration, the system will raise a
> ``ValueError`` or a ``TypeError`` with a suitably descriptive message. The
> following is a (possibly incomplete) list of conditions which will raise an
> error:
> 
> * A ``level`` which is not a string or which is a string not corresponding to
>   an actual logging level
> 
> * A ``propagate`` value which is not a Boolean
> 
> * An id which does not have a corresponding destination
> 
> * An invalid logger name

You may want to allow yourself AttributeError and ImportError here.
Alternatively, you could commit to converting those to an appropriate
ValueError that provides more detail on the config setting that
contained the problematic reference.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Sat Oct 17 10:33:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 17 Oct 2009 08:33:53 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?PY=5FSSIZE=5FT=5FCLEAN_in_py3k?=
Message-ID: <loom.20091017T103215-482@post.gmane.org>


Hello,

It turns out (*) that even in py3k, not all modules are PY_SSIZE_T_CLEAN.
Should we try to remedy that, and make PY_SSIZE_T_CLEAN the default in future
3.x versions?
As far I know, there's no reason not to be PY_SSIZE_T_CLEAN, except for having
to convert old code.

(*) http://bugs.python.org/issue7080

Regards

Antoine.

PS : no, I'm not volunteering to do it all myself :)



From vinay_sajip at yahoo.co.uk  Sat Oct 17 10:53:59 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 17 Oct 2009 08:53:59 +0000 (UTC)
Subject: [Python-Dev] A proposal: configuring logging using dictionaries
References: <loom.20091017T091238-255@post.gmane.org>
	<4AD97C09.3070705@gmail.com>
Message-ID: <loom.20091017T102943-823@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Since part of the purpose of this new configuration option is to allow
> other projects to use it as a target for specifying a logging
> configuration, maintaining it as a formal PEP seems like a good idea to
> me. This is already done for a few other standard-ish interfaces
> (distutils metadata, DB-API, WSGI).

I'm happy with this. I've subscribed to the PEPs list and once the subscription
comes through, will post there linking to the PEP draft.

> My only real objection is to the "incremental" flag in the dictionary. I
> would suggest having two different API methods instead - initDictConfig
> and updateDictConfig. Or is there some specific reason for having the
> choice of incremental update vs replacement configuration in the hands
> of the config author rather than the application developer?

I was thinking that in some environments, configuration changes might be sent as
pickled dicts over the wire. In such cases it might be useful to have the
"incremental" flag in the payload. But I haven't thought that through in detail,
as it's somewhat peripheral to this feature.

> My other question is whether or not it would be worth having the logging
> module able to export it's *current* configuration in dictionary form. I
> don't have a use case other than that it might be useful for debugging
> logging configuration errors when attempting to combine multiple sources
> of logging data, but it's worth considering.

Another fair point - perhaps a getConfig() which returns a dict can be added.

> You may want to allow yourself AttributeError and ImportError here.
> Alternatively, you could commit to converting those to an appropriate
> ValueError that provides more detail on the config setting that
> contained the problematic reference.

Perhaps TypeError, too. The idea is to definitely pinpoint where in the
configuration the error occurred. I'll think about the detail of this a bit
further down the line, but I just wanted to establish that I wasn't planning on
introducing any logging-related exception classes.

Thanks for the quick response and detailed feedback.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Sat Oct 17 14:57:03 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 17 Oct 2009 12:57:03 +0000 (UTC)
Subject: [Python-Dev] A proposal: configuring logging using dictionaries
References: <loom.20091017T091238-255@post.gmane.org>
	<4AD97C09.3070705@gmail.com>
	<loom.20091017T102943-823@post.gmane.org>
Message-ID: <loom.20091017T145605-830@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:

> Perhaps TypeError, too. 

Sorry, brainfade there. I already mentioned TypeError in the original post :-)





From lie.1296 at gmail.com  Sun Oct 18 06:11:16 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Sun, 18 Oct 2009 15:11:16 +1100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4ACA71EB.4050003@voidspace.org.uk>
References: <261064E43172924BBADB00DA14BB4B672A13F6FC1C@XCH-NW-14V.nw.nos.boeing.com>	<ca471dc20910050828o2ed717fbn6434a9c319ef5b15@mail.gmail.com>	<4ACA551F.9090309@gmail.com>	<1afaf6160910051357t32301c3aq593ac1745c807290@mail.gmail.com>
	<4ACA71EB.4050003@voidspace.org.uk>
Message-ID: <hbe4h5$ks1$1@ger.gmane.org>

Michael Foord wrote:
> Benjamin Peterson wrote:
>> 2009/10/5 Nick Coghlan <ncoghlan at gmail.com>:
>>  
>>> So I would agree that method invocation on literals (particularly string
>>> literals) is an already established language idiom.
>>>     
>>
>> And who hasn't ever used 4.56.as_integer_ratio()? :)
 >
> I've tried 4.__add__ a few times (not for a while now though).

That's a Syntax Error
 >>> 4.__add__
   File "<stdin>", line 1
     4.__add__
             ^
SyntaxError: invalid syntax

however this works:
 >>> 4 .__add__
<method-wrapper '__add__' of int object at 0x1E1FA028>


From vinay_sajip at yahoo.co.uk  Sun Oct 18 13:50:23 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 18 Oct 2009 11:50:23 +0000 (UTC)
Subject: [Python-Dev] A proposal: configuring logging using dictionaries
References: <loom.20091017T091238-255@post.gmane.org>
	<4AD97C09.3070705@gmail.com>
Message-ID: <loom.20091018T134542-635@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> My other question is whether or not it would be worth having the logging
> module able to export it's *current* configuration in dictionary form. I
> don't have a use case other than that it might be useful for debugging
> logging configuration errors when attempting to combine multiple sources
> of logging data, but it's worth considering.
> 

One restriction on any output of the current configuration is that it may not be
suitable for round-tripping, i.e. you may not be able to use the output dict to
(re)configure the system. That's because in general, the configurator wants to
know how to instantiate handlers, filters, formatters and so it essentially
needs the constructor arguments in an incoming dict. However, there's no
requirement for the handlers, filters and formatters to keep those constructor
arguments around in the original form; and so, in the general case, they are not
available to output in a dict describing the current configuration.

Regards,

Vinay Sajip


From dsdale24 at gmail.com  Sun Oct 18 16:50:51 2009
From: dsdale24 at gmail.com (Darren Dale)
Date: Sun, 18 Oct 2009 10:50:51 -0400
Subject: [Python-Dev] nonstandard behavior of reflected functions
Message-ID: <a08e5f80910180750o69e099e1w5f9303bc18c2d894@mail.gmail.com>

According to http://docs.python.org/reference/datamodel.html , the
reflected operands functions like __radd__ "are only called if the
left operand does not support the corresponding operation and the
operands are of different types. [3] For instance, to evaluate the
expression x - y, where y is an instance of a class that has an
__rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns
NotImplemented."

Consider the following simple example:

==========================
class Quantity(object):

    def __add__(self, other):
        return '__add__ called'

    def __radd__(self, other):
        return '__radd__ called'

class UnitQuantity(Quantity):

    def __add__(self, other):
        return '__add__ called'

    def __radd__(self, other):
        return '__radd__ called'

print 'Quantity()+Quantity()', Quantity()+Quantity()
print 'UnitQuantity()+UnitQuantity()', UnitQuantity()+UnitQuantity()
print 'UnitQuantity()+Quantity()', UnitQuantity()+Quantity()
print 'Quantity()+UnitQuantity()', Quantity()+UnitQuantity()
==========================

The output should indicate that __add__ was called in all four trials,
but the last trial calls __radd__. Interestingly, if I comment out the
definition of __radd__ in UnitQuantity, then the fourth trial calls
__add__ like it should.

I think this may be an important bug. I'm running Python 2.6.4rc1
(r264rc1:75270, Oct 13 2009, 17:02:06) an ubuntu Karmic. Is it a known
issue, or am I misreading the documentation?

Thanks,
Darren

From dsdale24 at gmail.com  Sun Oct 18 17:03:10 2009
From: dsdale24 at gmail.com (Darren Dale)
Date: Sun, 18 Oct 2009 11:03:10 -0400
Subject: [Python-Dev] nonstandard behavior of reflected functions
In-Reply-To: <a08e5f80910180750o69e099e1w5f9303bc18c2d894@mail.gmail.com>
References: <a08e5f80910180750o69e099e1w5f9303bc18c2d894@mail.gmail.com>
Message-ID: <a08e5f80910180803y41efbf9ci20b014b19ed425a8@mail.gmail.com>

On Sun, Oct 18, 2009 at 10:50 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> According to http://docs.python.org/reference/datamodel.html , the
> reflected operands functions like __radd__ "are only called if the
> left operand does not support the corresponding operation and the
> operands are of different types. [3] For instance, to evaluate the
> expression x - y, where y is an instance of a class that has an
> __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns
> NotImplemented."
>
> Consider the following simple example:
>
> ==========================
> class Quantity(object):
>
> ? ?def __add__(self, other):
> ? ? ? ?return '__add__ called'
>
> ? ?def __radd__(self, other):
> ? ? ? ?return '__radd__ called'
>
> class UnitQuantity(Quantity):
>
> ? ?def __add__(self, other):
> ? ? ? ?return '__add__ called'
>
> ? ?def __radd__(self, other):
> ? ? ? ?return '__radd__ called'
>
> print 'Quantity()+Quantity()', Quantity()+Quantity()
> print 'UnitQuantity()+UnitQuantity()', UnitQuantity()+UnitQuantity()
> print 'UnitQuantity()+Quantity()', UnitQuantity()+Quantity()
> print 'Quantity()+UnitQuantity()', Quantity()+UnitQuantity()
> ==========================
>
> The output should indicate that __add__ was called in all four trials,
> but the last trial calls __radd__. Interestingly, if I comment out the
> definition of __radd__ in UnitQuantity, then the fourth trial calls
> __add__ like it should.
>
> I think this may be an important bug. I'm running Python 2.6.4rc1
> (r264rc1:75270, Oct 13 2009, 17:02:06) an ubuntu Karmic. Is it a known
> issue, or am I misreading the documentation?

I'm sorry, I should have read further down the page in the documentation:

"Note: If the right operand?s type is a subclass of the left operand?s
type and that subclass provides the reflected method for the
operation, this method will be called before the left operand?s
non-reflected method. This behavior allows subclasses to override
their ancestors? operations."

From solipsis at pitrou.net  Sun Oct 18 22:01:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Oct 2009 20:01:54 +0000 (UTC)
Subject: [Python-Dev] SIGCHECK() in longobject.c
Message-ID: <loom.20091018T215434-493@post.gmane.org>


Hello,

In Objects/longobject.c, there's the SIGCHECK() macro which periodically checks
for signals when doing long integer computations (divisions, multiplications).
It does so by messing with the _Py_Ticker variable.

It was added in 1991 under the title "Many small changes", and I suppose it was
useful back then.

However, nowadays long objects are ridiculously fast, witness for example:

$ ./py3k/python -m timeit -s "a=eval('3'*10000+'5');b=eval('8'*6000+'7')"
"str(a//b)"
1000 loops, best of 3: 1.47 msec per loop

Can we remove this check, or are there people doing million-digits calculations
they want to interrupt using Control-C ?

Regards

Antoine.



From dickinsm at gmail.com  Sun Oct 18 22:08:27 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 18 Oct 2009 21:08:27 +0100
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <loom.20091018T215434-493@post.gmane.org>
References: <loom.20091018T215434-493@post.gmane.org>
Message-ID: <5c6f2a5d0910181308s32876221sd20f6df2be004ffb@mail.gmail.com>

On Sun, Oct 18, 2009 at 9:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> In Objects/longobject.c, there's the SIGCHECK() macro which periodically checks
> for signals when doing long integer computations (divisions, multiplications).
> It does so by messing with the _Py_Ticker variable.
>
> It was added in 1991 under the title "Many small changes", and I suppose it was
> useful back then.
>
> However, nowadays long objects are ridiculously fast, witness for example:
>
> $ ./py3k/python -m timeit -s "a=eval('3'*10000+'5');b=eval('8'*6000+'7')"
> "str(a//b)"
> 1000 loops, best of 3: 1.47 msec per loop
>
> Can we remove this check, or are there people doing million-digits calculations
> they want to interrupt using Control-C ?

Yes, I suspect there are.  Though you don't need millions of digits for a single
operation to take a noticeable amount of time:  try str(10**100000),
for example.

Is there a benefit to removing the check?

Mark

From solipsis at pitrou.net  Sun Oct 18 22:23:41 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Oct 2009 20:23:41 +0000 (UTC)
Subject: [Python-Dev] SIGCHECK() in longobject.c
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910181308s32876221sd20f6df2be004ffb@mail.gmail.com>
Message-ID: <loom.20091018T221716-47@post.gmane.org>

Mark Dickinson <dickinsm <at> gmail.com> writes:
> 
> Yes, I suspect there are.  Though you don't need millions of digits for a
single
> operation to take a noticeable amount of time:  try str(10**100000),
> for example.
> 
> Is there a benefit to removing the check?

Apart from casual cleanup, the reason I'm asking is that I'm experimenting with
removing _Py_Ticker, and I want to know what I need to emulate :-)


Antoine.



From ehsanamiri at gmail.com  Sun Oct 18 22:39:15 2009
From: ehsanamiri at gmail.com (Ehsan Amiri)
Date: Sun, 18 Oct 2009 13:39:15 -0700
Subject: [Python-Dev] nonstandard behavior of reflected functions
In-Reply-To: <a08e5f80910180750o69e099e1w5f9303bc18c2d894@mail.gmail.com>
References: <a08e5f80910180750o69e099e1w5f9303bc18c2d894@mail.gmail.com>
Message-ID: <fe13bba80910181339m65ea469bs5d60a2ff08349310@mail.gmail.com>

I see the same behaviour, moreover when I change class Quantity to a classic
class (removing '(object)'), it works as expected. (i.e. Quanitity.__add__()
is called after the fourth print. I run Python 2.6.2 on Vista.






On Sun, Oct 18, 2009 at 7:50 AM, Darren Dale <dsdale24 at gmail.com> wrote:

> According to http://docs.python.org/reference/datamodel.html , the
> reflected operands functions like __radd__ "are only called if the
> left operand does not support the corresponding operation and the
> operands are of different types. [3] For instance, to evaluate the
> expression x - y, where y is an instance of a class that has an
> __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns
> NotImplemented."
>
> Consider the following simple example:
>
> ==========================
> class Quantity(object):
>
>    def __add__(self, other):
>        return '__add__ called'
>
>    def __radd__(self, other):
>        return '__radd__ called'
>
> class UnitQuantity(Quantity):
>
>    def __add__(self, other):
>        return '__add__ called'
>
>    def __radd__(self, other):
>        return '__radd__ called'
>
> print 'Quantity()+Quantity()', Quantity()+Quantity()
> print 'UnitQuantity()+UnitQuantity()', UnitQuantity()+UnitQuantity()
> print 'UnitQuantity()+Quantity()', UnitQuantity()+Quantity()
> print 'Quantity()+UnitQuantity()', Quantity()+UnitQuantity()
> ==========================
>
> The output should indicate that __add__ was called in all four trials,
> but the last trial calls __radd__. Interestingly, if I comment out the
> definition of __radd__ in UnitQuantity, then the fourth trial calls
> __add__ like it should.
>
> I think this may be an important bug. I'm running Python 2.6.4rc1
> (r264rc1:75270, Oct 13 2009, 17:02:06) an ubuntu Karmic. Is it a known
> issue, or am I misreading the documentation?
>
> Thanks,
> Darren
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ehsanamiri%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091018/e998d19b/attachment-0001.htm>

From daniel at stutzbachenterprises.com  Mon Oct 19 00:16:52 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 18 Oct 2009 17:16:52 -0500
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <loom.20091018T215434-493@post.gmane.org>
References: <loom.20091018T215434-493@post.gmane.org>
Message-ID: <eae285400910181516g51ce1a71t616f8bd159a627b2@mail.gmail.com>

On Sun, Oct 18, 2009 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Can we remove this check, or are there people doing million-digits
> calculations
> they want to interrupt using Control-C ?
>

I sometimes do million-digits calculations that I want to interrupt using
Control-C.

(particularly when I didn't *intend* to do a million-digits calculation...
;) )

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091018/0b518faf/attachment.htm>

From solipsis at pitrou.net  Mon Oct 19 00:46:04 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Oct 2009 22:46:04 +0000 (UTC)
Subject: [Python-Dev] SIGCHECK() in longobject.c
References: <loom.20091018T215434-493@post.gmane.org>
	<eae285400910181516g51ce1a71t616f8bd159a627b2@mail.gmail.com>
Message-ID: <loom.20091019T004315-254@post.gmane.org>

Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> 
> I sometimes do million-digits calculations that I want to interrupt using
Control-C.(particularly when I didn't *intend* to do a million-digits
calculation... ;) )--

Sure, but it's no different than doing, e.g.:
    list(range(100000000)).sort()

(don't try this, it just made by computer slow down to a crawl and I had to kill
-9 the Python interpreter)

The question is whether there is still any reason to special-case long objects
and not, say, list.sort() or re.sub().

Regards

Antoine.



From daniel at stutzbachenterprises.com  Mon Oct 19 01:05:54 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 18 Oct 2009 18:05:54 -0500
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <loom.20091019T004315-254@post.gmane.org>
References: <loom.20091018T215434-493@post.gmane.org>
	<eae285400910181516g51ce1a71t616f8bd159a627b2@mail.gmail.com>
	<loom.20091019T004315-254@post.gmane.org>
Message-ID: <eae285400910181605v2def3855m400325d482d230c8@mail.gmail.com>

On Sun, Oct 18, 2009 at 5:46 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> > I sometimes do million-digits calculations that I want to interrupt using
> Control-C.(particularly when I didn't *intend* to do a million-digits
> calculation... ;) )--
>
> Sure, but it's no different than doing, e.g.:
>    list(range(100000000)).sort()
>

That's a good point, although I can't recall the last time I accidently
created a painfully large list.  I can recall the last time I started a
painfully large integer computation.

Being able to stop the interpretter with Control-C instead of kill -9 is a
minor convenience, though.  I could live without it.  (Although I can't
speak for everyone, of course)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091018/5e9d66b0/attachment.htm>

From ncoghlan at gmail.com  Mon Oct 19 03:15:38 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Oct 2009 11:15:38 +1000
Subject: [Python-Dev] nonstandard behavior of reflected functions
In-Reply-To: <fe13bba80910181339m65ea469bs5d60a2ff08349310@mail.gmail.com>
References: <a08e5f80910180750o69e099e1w5f9303bc18c2d894@mail.gmail.com>
	<fe13bba80910181339m65ea469bs5d60a2ff08349310@mail.gmail.com>
Message-ID: <4ADBBDBA.1050002@gmail.com>

Ehsan Amiri wrote:
> I see the same behaviour, moreover when I change class Quantity to a
> classic class (removing '(object)'), it works as expected. (i.e.
> Quanitity.__add__() is called after the fourth print. I run Python 2.6.2
> on Vista.

Darren found the explanation further down the page he was reading - the
fact that the right operand is an instance of a subclass of the left
operand's class makes a difference.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Mon Oct 19 03:20:49 2009
From: barry at python.org (Barry Warsaw)
Date: Sun, 18 Oct 2009 21:20:49 -0400
Subject: [Python-Dev] Python 2.6.4rc2
Message-ID: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>

Hello everyone.

The source tarballs and Windows installers for Python 2.6.4rc2 are now  
available:

http://www.python.org/download/releases/2.6.4/

Please download them, install them, and try to use them with your  
projects and environments.  Let's make 2.6.4 a rock solid release!  If  
there are no more regressions found, we'll do the final release in one  
week, on 25-October.

Enjoy,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091018/e33ec90e/attachment.pgp>

From abhiram.casina at gmail.com  Mon Oct 19 08:29:01 2009
From: abhiram.casina at gmail.com (Abhiram Kasina)
Date: Mon, 19 Oct 2009 11:59:01 +0530
Subject: [Python-Dev] Proposal : Python Trusted Computing API
Message-ID: <fd06c0ca0910182329w501aae84hea3106794bd8fe60@mail.gmail.com>

Hi

Trusted Computing (TC) is a technology developed and promoted by the Trusted
Computing Group (TCG)[3]. So, basically the group came up with these chips
called TPM chips which are present on most motherboards nowadays. The main
purpose of it is to enhance security so that infected executables don't run.
It also provides memory curtaining such that cryptographic keys won't be
accessible and many other features. There was a criticism on this from the
FOSS community as well that it enables DRM. No wonder, it is being pushed by
Intel, Microsoft, AMD, etc.. But personally I think its a good idea from
security point of view.

So, currently there is an TSS (TCG Software Stack)[1] API written in C. And
TrustedJava[2] is a project which ported it to Java and is going to be
included in the standard API of Java soon. They have 2 versions of it. One
is a simple wrapper on top of the API and the other is a whole
implementation of the stack in Java.

My proposal is we create an API for it in python.
*Reason*: I am a developer in Umit and I think Python is a very good
platform for developing applications. So, why not create an API which helps
in developing secure applications?

I would love to learn more and provide you with any more information. Please
let me know what you guys think of it?

Thanks in advance

Cheers
Abhiram

[1]
http://www.trustedcomputinggroup.org/resources/tcg_software_stack_tss_specification
[2] http://trustedjava.sourceforge.net/index.php?item=jtss/about
[3] http://www.trustedcomputinggroup.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091019/9f9ce575/attachment-0001.htm>

From vinay_sajip at yahoo.co.uk  Mon Oct 19 09:59:05 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 19 Oct 2009 07:59:05 +0000 (UTC)
Subject: [Python-Dev] Python 2.6.4rc2
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
Message-ID: <loom.20091019T095712-875@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:

> http://www.python.org/download/releases/2.6.4/

Good news, but just one little nit: the page above appears to link to the NEWS
file for 2.6.4rc1.

Regards,

Vinay Sajip


From dickinsm at gmail.com  Mon Oct 19 11:28:34 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 19 Oct 2009 10:28:34 +0100
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <loom.20091019T004315-254@post.gmane.org>
References: <loom.20091018T215434-493@post.gmane.org>
	<eae285400910181516g51ce1a71t616f8bd159a627b2@mail.gmail.com>
	<loom.20091019T004315-254@post.gmane.org>
Message-ID: <5c6f2a5d0910190228r30b82068rdd22c0c905c3f7b4@mail.gmail.com>

On Sun, Oct 18, 2009 at 11:46 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Sure, but it's no different than doing, e.g.:
> ? ?list(range(100000000)).sort()
>
> (don't try this, it just made by computer slow down to a crawl and I had to kill
> -9 the Python interpreter)

Maybe you were running out of RAM?  On a 64-bit machine,
list(range(10**8)) takes over 3 Gb.

For me, x = list(range(10**8)) takes around 6 seconds, and then
x.sort() takes around 2 seconds.  This is on a machine with
16 Gb of RAM.  (Though I do seem to recall that timsort is
extra fast for already sorted lists: O(n) rather than O(n log n)?)

So I'd say that it *is* different.  A million digit integer takes less
than half a megabyte of RAM, so a single operation can be
horribly slow long before memory limitations are reached.

I'd prefer to keep the SIGCHECK unless there's a really
compelling advantage to getting rid of it.

Mark

From barry at python.org  Mon Oct 19 12:55:00 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 19 Oct 2009 06:55:00 -0400
Subject: [Python-Dev] Python 2.6.4rc2
In-Reply-To: <loom.20091019T095712-875@post.gmane.org>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<loom.20091019T095712-875@post.gmane.org>
Message-ID: <594E6B4A-8D52-4A93-A368-E699FBA42535@python.org>

On Oct 19, 2009, at 3:59 AM, Vinay Sajip wrote:

> Barry Warsaw <barry <at> python.org> writes:
>
>> http://www.python.org/download/releases/2.6.4/
>
> Good news, but just one little nit: the page above appears to link  
> to the NEWS
> file for 2.6.4rc1.

Ooops!  Fixed, thanks.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091019/f65fec09/attachment.pgp>

From dickinsm at gmail.com  Mon Oct 19 13:27:04 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 19 Oct 2009 12:27:04 +0100
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <loom.20091018T215434-493@post.gmane.org>
References: <loom.20091018T215434-493@post.gmane.org>
Message-ID: <5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>

On Sun, Oct 18, 2009 at 9:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Can we remove this check, or are there people doing million-digits calculations
> they want to interrupt using Control-C ?

By the way, here's an example of an *almost* real-life use of million digit
calculations.

For an elementary number theory course that I taught a while ago, there
was an associated (optional) computer lab, where the students used
Python to investigate various ideas, conjectures, examples, etc.  One
of the less open-ended questions might have been[1] something like:

"On August 23rd, 2008 a computer at UCLA found the first example
of a prime with more than 10 million digits: p = 2**43112609-1.  Find

(1) the exact number of digits in p, when written out in decimal
(2) the last 100 digits of p
(3) the first 100 digits of p."

It's wonderfully easy to get answers to these questions with Python:

>>> import math
>>> e = 43112609
>>> p = 2**e - 1
>>> ndigits = int(math.ceil(math.log10(p)))
>>> last_100_digits = '{:0100d}'.format(p % 10**100)
>>> first_100_digits = str(p // 10**(ndigits-100))

Getting the first 100 digits takes a good few seconds;  the
other operations are faster.

But if you (not unreasonably) try to compute str(p),
you'll find it's impossibly slow, and it's very handy
that it's possible to interrupt that calculation, attempt
to understand *why* it's slow, and then try different
methods.

(And yes, there are better ways to get both the first
and last hundred digits.)

Mark


[1] Might have been, but wasn't.  Hence the *almost*.  If I'd been
teaching that course this semester I'd almost certainly have included
something like this.

From guido at python.org  Mon Oct 19 21:35:59 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 19 Oct 2009 12:35:59 -0700
Subject: [Python-Dev] Proposal : Python Trusted Computing API
In-Reply-To: <fd06c0ca0910182329w501aae84hea3106794bd8fe60@mail.gmail.com>
References: <fd06c0ca0910182329w501aae84hea3106794bd8fe60@mail.gmail.com>
Message-ID: <ca471dc20910191235o6322915fic1dec2ee6062b24c@mail.gmail.com>

On Sun, Oct 18, 2009 at 11:29 PM, Abhiram Kasina
<abhiram.casina at gmail.com> wrote:
> Trusted Computing (TC) is a technology developed and promoted by the Trusted
> Computing Group (TCG)[3]. So, basically the group came up with these chips
> called TPM chips which are present on most motherboards nowadays. The main
> purpose of it is to enhance security so that infected executables don't run.
> It also provides memory curtaining such that cryptographic keys won't be
> accessible and many other features. There was a criticism on this from the
> FOSS community as well that it enables DRM. No wonder, it is being pushed by
> Intel, Microsoft, AMD, etc.. But personally I think its a good idea from
> security point of view.

Hm... Given that most infections these days are JavaScript based and
run in the browser, how does this provide any protection? I'm
presuming you're going to say that it doesn't but that there are other
use cases where it *does* provide protection; but most likely those
use cases are only relevant for Windows (since that's what most
attackers attack anyway).

> So, currently there is an TSS (TCG Software Stack)[1] API written in C. And
> TrustedJava[2] is a project which ported it to Java and is going to be
> included in the standard API of Java soon. They have 2 versions of it. One
> is a simple wrapper on top of the API and the other is a whole
> implementation of the stack in Java.

Since this intefaces with the hardware, doesn't it require some kind
of cooperation from the Linux kernel? And wouldn't it be better if
Python was never allowed access to any of the protected resources in
the first place?

> My proposal is we create an API for it in python.
> Reason: I am a developer in Umit

Where/what is Umit? (Google gives several meanings but it's unclear
which you might mean.)

> and I think Python is a very good platform
> for developing applications. So, why not create an API which helps in
> developing secure applications?

You'd first have to tell us more about the security model. What is a
"secure application" and what does it protect against? And how?

> I would love to learn more and provide you with any more information. Please
> let me know what you guys think of it?

This is better directed at python-ideas, so I've redirected this reply
there and Bcc'ed the python-dev list.

> Thanks in advance
>
> Cheers
> Abhiram
>
> [1]
> http://www.trustedcomputinggroup.org/resources/tcg_software_stack_tss_specification
> [2] http://trustedjava.sourceforge.net/index.php?item=jtss/about
> [3] http://www.trustedcomputinggroup.org/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Mon Oct 19 22:18:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Oct 2009 06:18:08 +1000
Subject: [Python-Dev] Proposal : Python Trusted Computing API
In-Reply-To: <fd06c0ca0910182329w501aae84hea3106794bd8fe60@mail.gmail.com>
References: <fd06c0ca0910182329w501aae84hea3106794bd8fe60@mail.gmail.com>
Message-ID: <4ADCC980.9090106@gmail.com>

Abhiram Kasina wrote:
> I would love to learn more and provide you with any more information.
> Please let me know what you guys think of it?

This is really an off-topic question for python-dev. This list is just
about developing the core interpreter and standard library - we have no
control over the APIs that people choose to develop and publish on top
of that.

If you want to develop such an API and put it up on PyPI, then go right
ahead. comp.lang.python (aka python-list) would be the place to ask for
interest from other developers.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at barrys-emacs.org  Mon Oct 19 22:39:06 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Mon, 19 Oct 2009 21:39:06 +0100
Subject: [Python-Dev] Add const to python API - issue 6952
Message-ID: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>

http://bugs.python.org/issue6952

Martin v. L?wis suggested that solutions to this issue should be  
discussed here.

My goal is to avoid compiler warning and the need to cast to remove  
const when
calling the python API.

For example I see compiler messages like this on Mac OS X Snow Leopard  
g++ reports:

example.cxx:633: warning: deprecated conversion from string constant  
to ?char*?

The patch I developed for comment only adds const to the input  
parameters and used casts to
allow output parameters to stay without the const.

This is because adding the const to the input params will not break  
existing code, but
adding const to output parameters may well require code changes for  
users of the Python
API.

What is the best approach to this problem that will be acceptable?

Barry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091019/d7f6ae05/attachment.htm>

From tim.peters at gmail.com  Mon Oct 19 22:58:47 2009
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 19 Oct 2009 16:58:47 -0400
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
Message-ID: <1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>

[Mark Dickinson]
> By the way, here's an example of an *almost* real-life use of million digit
> calculations.
>
> For an elementary number theory course that I taught a while ago, there
> was an associated (optional) computer lab, where the students used
> Python to investigate various ideas, conjectures, examples, etc. ?One
> of the less open-ended questions might have been[1] something like:
>
> "On August 23rd, 2008 a computer at UCLA found the first example
> of a prime with more than 10 million digits: p = 2**43112609-1. ?Find
>
> (1) the exact number of digits in p, when written out in decimal
> (2) the last 100 digits of p
> (3) the first 100 digits of p."
>
> It's wonderfully easy to get answers to these questions with Python:
...
> But if you (not unreasonably) try to compute str(p),
> you'll find it's impossibly slow, and it's very handy
> that it's possible to interrupt that calculation, attempt
> to understand *why* it's slow, and then try different
> methods.

Don't want to hijack this thread, but this is the kind of use case
justifying keeping the 3-argument pow in the decimal module.  People
"playing" with number theory questions can learn a bag of tricks to
worm around that non-decimal arithmetic can make it inconvenient &
slow to get answers about decimal digits, but most people -- and
especially beginners -- find life easier when doing calculations in
decimal to begin with.  Then they can use obvious methods, and not get
sidetracked by wondering why they take forever to finish.

Although, to be fair, I started

>>> decimal.getcontext().prec = 20000000
>>> p10 = pow(decimal.Decimal(2), 43112609)

before I started typing this, and am still waiting for it to finish ;-)

OTOH,

>>> decimal.getcontext().prec = 200
>>> pow(decimal.Decimal(2), 43112609)   # get the first 100 digits (& then some)

and

>>> pow(decimal.Decimal(2), 43112609, 10**100) - 1  # get the last 100 digits

both appear to work instantaneously.

From dalcinl at gmail.com  Tue Oct 20 05:10:45 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 20 Oct 2009 01:10:45 -0200
Subject: [Python-Dev] Python 2.6.4rc2
In-Reply-To: <594E6B4A-8D52-4A93-A368-E699FBA42535@python.org>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<loom.20091019T095712-875@post.gmane.org>
	<594E6B4A-8D52-4A93-A368-E699FBA42535@python.org>
Message-ID: <e7ba66e40910192010o35f8f29v132756ebff40ccc0@mail.gmail.com>

I'm getting this warning. It seems nothing is actually broken, but the
fix is pretty easy.

gcc -pthread -c -fno-strict-aliasing -g -Wall -Wstrict-prototypes  -I.
-IInclude -I./Include   -DPy_BUILD_CORE -o Objects/unicodeobject.o
Objects/unicodeobject.c
Objects/unicodeobject.c: In function 'PyUnicodeUCS2_FromFormatV':
Objects/unicodeobject.c:687: warning: pointer targets in passing
argument 1 of 'strlen' differ in signedness
/usr/include/string.h:397: note: expected 'const char *' but argument
is of type 'unsigned char *'
Objects/unicodeobject.c:687: warning: pointer targets in passing
argument 1 of 'PyUnicodeUCS2_DecodeUTF8' differ in signedness
Include/unicodeobject.h:752: note: expected 'const char *' but
argument is of type 'unsigned char *'

BTW, should Python build with a C++ compiler? It seems this is not
possible with 2.6.4rc2 (and GCC 4.4.1)



On Mon, Oct 19, 2009 at 8:55 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 19, 2009, at 3:59 AM, Vinay Sajip wrote:
>
>> Barry Warsaw <barry <at> python.org> writes:
>>
>>> http://www.python.org/download/releases/2.6.4/
>>
>> Good news, but just one little nit: the page above appears to link to the
>> NEWS
>> file for 2.6.4rc1.
>
> Ooops! ?Fixed, thanks.
> -Barry
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From martin at v.loewis.de  Tue Oct 20 05:35:05 2009
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 20 Oct 2009 05:35:05 +0200
Subject: [Python-Dev] Add const to python API - issue 6952
In-Reply-To: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>
References: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>
Message-ID: <4ADD2FE9.2010807@v.loewis.de>

> The patch I developed for comment only adds const to the input
> parameters and used casts to
> allow output parameters to stay without the const.

What specific APIs are you talking about here?

Regards,
Martin

From dickinsm at gmail.com  Tue Oct 20 10:39:31 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 20 Oct 2009 09:39:31 +0100
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
Message-ID: <5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>

On Mon, Oct 19, 2009 at 9:58 PM, Tim Peters <tim.peters at gmail.com> wrote:
> Don't want to hijack this thread, but this is the kind of use case
> justifying keeping the 3-argument pow in the decimal module. ?People
> "playing" with number theory questions can learn a bag of tricks to
> worm around that non-decimal arithmetic can make it inconvenient &
> slow to get answers about decimal digits, but most people -- and
> especially beginners -- find life easier when doing calculations in
> decimal to begin with. ?Then they can use obvious methods, and not get
> sidetracked by wondering why they take forever to finish.

That's true.  I wouldn't relish the task of trying to teach the decimal
module at the same time as introducing students to Python and to
elementary number theory, though. It's not the most friendly module
to get started with.

> Although, to be fair, I started
>
>>>> decimal.getcontext().prec = 20000000
>>>> p10 = pow(decimal.Decimal(2), 43112609)
>
> before I started typing this, and am still waiting for it to finish ;-)

Hmm, yes.  The decimal module isn't (currently) well set up
for this sort of calculation, mostly because almost every operation
is implemented by converting to binary, invoking the appropriate
int/long arithmetic operation, and converting back.  Since the
conversion is O(n^2), even addition and multiplication of Decimal
instances end up being O(n^2) operations at the moment, instead
of getting the linear and Karatsuba (resp.) running times that they
deserve.

(The exception is rounding operations, which don't do any
decimal <-> binary operations, but operate directly on the
coefficient in its representation as a string of decimal digits.)

This high-precision inefficiency could easily be fixed by
using a dedicated 'decimal natural number' extension
type for the Decimal coefficient, stored internally in base
a suitable power of 10.  I think this may be worth
considering seriously. I'm not proposing this as an alternative
to the huge task of rewriting the entire decimal module in C,
by the way;  it would be more a stop-gap measure that would
be easy to implement, would slightly increase efficiency for
normal operations, and vastly increase efficiency for high-precision
operations.

>
> OTOH,
>
>>>> decimal.getcontext().prec = 200
>>>> pow(decimal.Decimal(2), 43112609) ? # get the first 100 digits (& then some)
>
> and
>
>>>> pow(decimal.Decimal(2), 43112609, 10**100) - 1 ?# get the last 100 digits
>
> both appear to work instantaneously.
>

Right.  Low precision Decimal operations should be fine.  I don't
really see the advantage of the second operation over a simple

pow(2, 43112609, 10**100)

though.

By the way, despite the above use-case, and despite the fact
that I use it frequently, I still don't believe 3-argument pow belongs
in the core.  But that's probably a discussion topic for Python 4000.

Mark

From ncoghlan at gmail.com  Tue Oct 20 12:49:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Oct 2009 20:49:27 +1000
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
	<5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>
Message-ID: <4ADD95B7.8040304@gmail.com>

Mark Dickinson wrote:
> This high-precision inefficiency could easily be fixed by
> using a dedicated 'decimal natural number' extension
> type for the Decimal coefficient, stored internally in base
> a suitable power of 10.  I think this may be worth
> considering seriously. I'm not proposing this as an alternative
> to the huge task of rewriting the entire decimal module in C,
> by the way;  it would be more a stop-gap measure that would
> be easy to implement, would slightly increase efficiency for
> normal operations, and vastly increase efficiency for high-precision
> operations.

Didn't you already start work on that concept with the deccoeff patch?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Tue Oct 20 12:57:45 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 20 Oct 2009 06:57:45 -0400
Subject: [Python-Dev] Python 2.6.4rc2
In-Reply-To: <e7ba66e40910192010o35f8f29v132756ebff40ccc0@mail.gmail.com>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<loom.20091019T095712-875@post.gmane.org>
	<594E6B4A-8D52-4A93-A368-E699FBA42535@python.org>
	<e7ba66e40910192010o35f8f29v132756ebff40ccc0@mail.gmail.com>
Message-ID: <F2F07ECF-1615-462D-A6A0-3363E701FA32@python.org>

On Oct 19, 2009, at 11:10 PM, Lisandro Dalcin wrote:

> I'm getting this warning. It seems nothing is actually broken, but the
> fix is pretty easy.
>
> gcc -pthread -c -fno-strict-aliasing -g -Wall -Wstrict-prototypes  -I.
> -IInclude -I./Include   -DPy_BUILD_CORE -o Objects/unicodeobject.o
> Objects/unicodeobject.c
> Objects/unicodeobject.c: In function 'PyUnicodeUCS2_FromFormatV':
> Objects/unicodeobject.c:687: warning: pointer targets in passing
> argument 1 of 'strlen' differ in signedness
> /usr/include/string.h:397: note: expected 'const char *' but argument
> is of type 'unsigned char *'
> Objects/unicodeobject.c:687: warning: pointer targets in passing
> argument 1 of 'PyUnicodeUCS2_DecodeUTF8' differ in signedness
> Include/unicodeobject.h:752: note: expected 'const char *' but
> argument is of type 'unsigned char *'

This isn't a regression in 2.6.3, nor is it critical enough, to be  
fixed for 2.6.4.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091020/f7fa8e80/attachment.pgp>

From dickinsm at gmail.com  Tue Oct 20 13:22:33 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 20 Oct 2009 12:22:33 +0100
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <4ADD95B7.8040304@gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
	<5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>
	<4ADD95B7.8040304@gmail.com>
Message-ID: <5c6f2a5d0910200422g71560d5eic6503da0d4fdc0fa@mail.gmail.com>

On Tue, Oct 20, 2009 at 11:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Mark Dickinson wrote:
>> This high-precision inefficiency could easily be fixed by
>> using a dedicated 'decimal natural number' extension
>> type for the Decimal coefficient, stored internally in base
>> a suitable power of 10. [...]
>
> Didn't you already start work on that concept with the deccoeff patch?

I did:  the code can be seen at:

http://svn.python.org/view/sandbox/trunk/decimal/decimal_in_c/

This code defines a Deccoeff type as above, providing base 10 natural
number arithmetic, along with a skeletal _Decimal type.  The work on
Deccoeff is pretty much complete, but the _Decimal type is only just
a beginning;  the original intent was to slowly translate the Decimal
code into C and move it into the _Decimal type.

The code was working a few months ago (with all Decimal tests
passing), but there have been some changes and bugfixes since
then.  I might try to resurrect that code, dropping the _Decimal type and
just concentrating on Deccoeff.

Mark

From amk at amk.ca  Tue Oct 20 13:43:25 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 20 Oct 2009 07:43:25 -0400
Subject: [Python-Dev] Volunteer needed to organize summits
Message-ID: <20091020114325.GB6355@amk-desktop.matrixgroup.net>

I'd like to turn over the organization of the VM and Python Language
Summits at PyCon 2010 to someone else, one or two people.  (The same
person doesn't need to organize both of them.)

Why: in November PyCon will be three months away, so the guest list
needs to be finalized and the invitations need to be sent.  Yet I
can't pull together the motivation to work on them; I contemplate the
task for two minutes and then go do something else.  It's obviously
better if the summit tasks are being actively worked on instead of
just drifting, so I want to give it up now.

What's required: chiefly it's just a matter of sending and replying to
e-mail.  Draw up a guest list (I can provide last year's lists); think
of new people & projects to be added, or e-mail someone to ask for
suggestions; send out invitations and requests for agenda items;
collect the responses so we know how many people are coming.

You can also help moderate the summits on the day of the events, but
if that's not feasible someone else could do it, or the groups could
manage themselves.

(Also sent to pycon-organizers, psf-members.)

--amk

From stefan-usenet at bytereef.org  Tue Oct 20 15:15:47 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Tue, 20 Oct 2009 15:15:47 +0200
Subject: [Python-Dev] Interest in integrating C decimal module into Python?
Message-ID: <20091020131547.GA615@mail.bytereef.org>

Hi,

as some of you know, recently I've released an arbitrary precision
C library for decimal arithmetic together with a Python module:

http://www.bytereef.org/libmpdec.html
http://www.bytereef.org/fastdec.html


Both the library and the module have been tested extensively. Fastdec
currently differs from decimal.py in a couple of ways that could be
fixed. The license is AGPL, but if there is interest in integrating it
into Python I'd release it under a Python-compatible license.


There have been several approaches towards getting C decimal arithmetic
into Python:

http://bugs.python.org/issue2486


Fastdec follows Raymond Hettinger's suggestion to provide wrappers for
an independent C implementation. Arguments in favour of fastdec are:

  * Complete implementation of Mike Cowlishaw's specification

  * C library can be tested independently

  * Redundant arithmetic module for tests against decimal.py

  * Faster than Java BigDecimal

  * Compares relatively well in speed against gmpy


To be clear, I would not want to _replace_ decimal.py. Rather I'd like to
see a cdecimal module alongside decimal.py.


I know that ultimately there should be a PEP for module inclusions. The
purpose of this post is to gauge interest. Specifically:


1. Are you generally in favour of a C decimal module in Python?


2. Would fastdec - after achieving full decimal.py compatibility - be
   a serious candidate?


3. Could I use this list to settle a couple of questions, or would perhaps
   a Python developer be willing to work with me to make it compatible? I'm
   asking this to avoid doing work that would not find acceptance afterwards.



Thanks,

Stefan Krah




From ssteinerx at gmail.com  Tue Oct 20 15:27:33 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 20 Oct 2009 09:27:33 -0400
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <20091020131547.GA615@mail.bytereef.org>
References: <20091020131547.GA615@mail.bytereef.org>
Message-ID: <CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>

Shouldn't this be on python-ideas?

S

On Oct 20, 2009, at 9:15 AM, Stefan Krah wrote:

> Hi,
>
> as some of you know, recently I've released an arbitrary precision
> C library for decimal arithmetic together with a Python module:
>
> http://www.bytereef.org/libmpdec.html
> http://www.bytereef.org/fastdec.html
>
>
> Both the library and the module have been tested extensively. Fastdec
> currently differs from decimal.py in a couple of ways that could be
> fixed. The license is AGPL, but if there is interest in integrating it
> into Python I'd release it under a Python-compatible license.
>
>
> There have been several approaches towards getting C decimal  
> arithmetic
> into Python:
>
> http://bugs.python.org/issue2486
>
>
> Fastdec follows Raymond Hettinger's suggestion to provide wrappers for
> an independent C implementation. Arguments in favour of fastdec are:
>
>  * Complete implementation of Mike Cowlishaw's specification
>
>  * C library can be tested independently
>
>  * Redundant arithmetic module for tests against decimal.py
>
>  * Faster than Java BigDecimal
>
>  * Compares relatively well in speed against gmpy
>
>
> To be clear, I would not want to _replace_ decimal.py. Rather I'd  
> like to
> see a cdecimal module alongside decimal.py.
>
>
> I know that ultimately there should be a PEP for module inclusions.  
> The
> purpose of this post is to gauge interest. Specifically:
>
>
> 1. Are you generally in favour of a C decimal module in Python?
>
>
> 2. Would fastdec - after achieving full decimal.py compatibility - be
>   a serious candidate?
>
>
> 3. Could I use this list to settle a couple of questions, or would  
> perhaps
>   a Python developer be willing to work with me to make it  
> compatible? I'm
>   asking this to avoid doing work that would not find acceptance  
> afterwards.
>
>
>
> Thanks,
>
> Stefan Krah
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ssteinerx%40gmail.com


From rdmurray at bitdance.com  Tue Oct 20 15:47:33 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 20 Oct 2009 09:47:33 -0400 (EDT)
Subject: [Python-Dev] Interest in integrating C decimal module into
 Python?
In-Reply-To: <CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>
Message-ID: <Pine.LNX.4.64.0910200945540.18193@kimball.webabinitio.net>

On Tue, 20 Oct 2009 at 09:27, ssteinerX at gmail.com wrote:
> Shouldn't this be on python-ideas?

IMO this question is appropriate for python-dev, not python-ideas.

--David

From stefan-usenet at bytereef.org  Tue Oct 20 15:43:33 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Tue, 20 Oct 2009 15:43:33 +0200
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>
Message-ID: <20091020134333.GA9122@yoda.bytereef.org>

ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
> Shouldn't this be on python-ideas?

I found previous discussions about "Decimal in C" on python-dev, that's why
used this list.


Stefan Krah



From ssteinerx at gmail.com  Tue Oct 20 15:55:06 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 20 Oct 2009 09:55:06 -0400
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <20091020134333.GA9122@yoda.bytereef.org>
References: <20091020131547.GA615@mail.bytereef.org>
	<CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>
	<20091020134333.GA9122@yoda.bytereef.org>
Message-ID: <E41D5316-EAB9-47EB-A3AE-287861ABB867@gmail.com>


On Oct 20, 2009, at 9:43 AM, Stefan Krah wrote:

> ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>> Shouldn't this be on python-ideas?
>
> I found previous discussions about "Decimal in C" on python-dev,  
> that's why
> used this list.

python-ideas:

This list is to contain discussion of speculative language ideas for  
Python for possible inclusion into the language. If an idea gains  
traction it can then be discussed and honed to the point of becoming a  
solid proposal to put to either python-dev or python-3000 as  
appropriate.

I guess it's a fine line...and matter of opinion.  No worries.

S


From chris at simplistix.co.uk  Tue Oct 20 16:18:50 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 20 Oct 2009 15:18:50 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <4ACF6993.5040704@voidspace.org.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<nad-B71566.14284708102009@ger.gmane.org>	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>	<4ACE718D.8090400@cryptelium.net>	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>	<loom.20091009T124309-659@post.gmane.org>	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
Message-ID: <4ADDC6CA.5000809@simplistix.co.uk>

[following up on distutils-sig]

Michael Foord wrote:
> Many Windows users would be quite happy if the standard mechanism for 
> installing non-source distributions on Windows was via the wininst 
> binaries.

...and many users may not be ;-) I know I'd be extremely unhappy if that 
were the case as I routines use multiple versions of packages with 
multiple versions of python on one machine...

> I wonder if it is going to be possible to make this compatible with the 
> upcoming distutils package management 'stuff' (querying for installed 
> packages, uninstallation etc) since installation/uninstallation goes 
> through the Windows system package management feature.  I guess it would 
> be eminently possible but require some reasonably high level Windows-fu 
> to do.

I wouldn't have a problem if integrating with the windows package 
manager was an optional extra, but I think it's one of many types of 
package management that need to be worried about, so might be easier to 
get the others working and let anyone who wants anything beyond a 
pure-python packaging system that works across platforms, regardless of 
whether binary extensions are needed, do the work themselves...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Oct 20 16:22:11 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 20 Oct 2009 15:22:11 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <4ADDC6CA.5000809@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>	<nad-B71566.14284708102009@ger.gmane.org>	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>	<4ACE718D.8090400@cryptelium.net>	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>	<loom.20091009T124309-659@post.gmane.org>	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>	<4ACF6993.5040704@voidspace.org.uk>
	<4ADDC6CA.5000809@simplistix.co.uk>
Message-ID: <4ADDC793.9050507@simplistix.co.uk>

Chris Withers wrote:
> [following up on distutils-sig]

...FAIL, sorry for the noise.

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From rdmurray at bitdance.com  Tue Oct 20 16:40:05 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 20 Oct 2009 10:40:05 -0400 (EDT)
Subject: [Python-Dev] Interest in integrating C decimal module into
 Python?
In-Reply-To: <E41D5316-EAB9-47EB-A3AE-287861ABB867@gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<CEE1DE2D-CA27-43B3-9C67-4370EB39F06D@gmail.com>
	<20091020134333.GA9122@yoda.bytereef.org>
	<E41D5316-EAB9-47EB-A3AE-287861ABB867@gmail.com>
Message-ID: <Pine.LNX.4.64.0910201034470.18193@kimball.webabinitio.net>

On Tue, 20 Oct 2009 at 09:55, ssteinerX at gmail.com wrote:
> On Oct 20, 2009, at 9:43 AM, Stefan Krah wrote:
>> ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>> > Shouldn't this be on python-ideas?
>> 
>> I found previous discussions about "Decimal in C" on python-dev, that's why
>> used this list.
>
> python-ideas:
>
> This list is to contain discussion of speculative language ideas for Python 
> for possible inclusion into the language. If an idea gains traction it can 
> then be discussed and honed to the point of becoming a solid proposal to put 
> to either python-dev or python-3000 as appropriate.
>
> I guess it's a fine line...and matter of opinion.  No worries.

In this case it isn't a language idea under discussion, but the possible
inclusion of an implementation of an idea (and moreover an idea that is
already included in Python in another, less efficient, form).

(I'm expecting that both Mark Dickinson and Raymond Hettinger will
comment on this thread eventually...)

--David (RDM)

From eric at trueblade.com  Tue Oct 20 16:50:07 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 20 Oct 2009 10:50:07 -0400 (EDT)
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <5c6f2a5d0910200422g71560d5eic6503da0d4fdc0fa@mail.gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
	<5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>
	<4ADD95B7.8040304@gmail.com>
	<5c6f2a5d0910200422g71560d5eic6503da0d4fdc0fa@mail.gmail.com>
Message-ID: <41504.63.251.87.214.1256050207.squirrel@mail.trueblade.com>

> On Tue, Oct 20, 2009 at 11:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Mark Dickinson wrote:
>>> This high-precision inefficiency could easily be fixed by
>>> using a dedicated 'decimal natural number' extension
>>> type for the Decimal coefficient, stored internally in base
>>> a suitable power of 10. [...]
>>
>> Didn't you already start work on that concept with the deccoeff patch?
>
> I did:  the code can be seen at:
>
> http://svn.python.org/view/sandbox/trunk/decimal/decimal_in_c/
>
> This code defines a Deccoeff type as above, providing base 10 natural
> number arithmetic, along with a skeletal _Decimal type.  The work on
> Deccoeff is pretty much complete, but the _Decimal type is only just
> a beginning;  the original intent was to slowly translate the Decimal
> code into C and move it into the _Decimal type.
>
> The code was working a few months ago (with all Decimal tests
> passing), but there have been some changes and bugfixes since
> then.  I might try to resurrect that code, dropping the _Decimal type and
> just concentrating on Deccoeff.

My only concern about this is the effect it would have on IronPython,
Jython, PyPy, and other alternate implementations that use the stdlib.


From dickinsm at gmail.com  Tue Oct 20 16:57:13 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 20 Oct 2009 15:57:13 +0100
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <41504.63.251.87.214.1256050207.squirrel@mail.trueblade.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
	<5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>
	<4ADD95B7.8040304@gmail.com>
	<5c6f2a5d0910200422g71560d5eic6503da0d4fdc0fa@mail.gmail.com>
	<41504.63.251.87.214.1256050207.squirrel@mail.trueblade.com>
Message-ID: <5c6f2a5d0910200757s32b814aeod5b7d6d8c202df47@mail.gmail.com>

On Tue, Oct 20, 2009 at 3:50 PM, Eric Smith <eric at trueblade.com> wrote:
>> The code was working a few months ago (with all Decimal tests
>> passing), but there have been some changes and bugfixes since
>> then. ?I might try to resurrect that code, dropping the _Decimal type and
>> just concentrating on Deccoeff.
>
> My only concern about this is the effect it would have on IronPython,
> Jython, PyPy, and other alternate implementations that use the stdlib.

Yes, that worries me a bit, too.  I have the same worry with the idea
of rewriting the entire decimal module in C.

The Deccoeff type is very simple, though.  It would be easy to create
a pure Python version of it, and then do something like:

try:
    from _decimal import Deccoeff  # try to get C version
except ImportError:
    from deccoeff import Deccoeff  # else use Python fallback code.

Mark

From p.f.moore at gmail.com  Tue Oct 20 22:49:42 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 20 Oct 2009 21:49:42 +0100
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <4ADDC6CA.5000809@simplistix.co.uk>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
	<4ADDC6CA.5000809@simplistix.co.uk>
Message-ID: <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com>

2009/10/20 Chris Withers <chris at simplistix.co.uk>:
> I wouldn't have a problem if integrating with the windows package manager
> was an optional extra, but I think it's one of many types of package
> management that need to be worried about, so might be easier to get the
> others working and let anyone who wants anything beyond a pure-python
> packaging system that works across platforms, regardless of whether binary
> extensions are needed, do the work themselves...

There are many (I believe) Windows users for whom bdist_wininst is
just what they want. For those people, where's the incentive to switch
in what you propose? You're not providing the features they currently
have, and frankly "do the work yourself" is no answer (not everyone
can, often for entirely legitimate reasons).

If such users could ignore the new offering, that would be fine - but
they can't, if projects stop providing bdist_wininst installers.
That's what's happening with eggs already, and yes, it *is* a pain.
And I'm in a position where I *can* build my own bdist_wininst
installer - but often, I'll just not bother using a package. So
packages lose users - does this matter? Who can tell?

In my view, the number one priority is to have a single distribution
format. I really don't care what it is. But *one*. bdist_wininst used
to be that on Windows, and for all its limitations it was a huge
benefit. Eggs messed that up, essentially because they didn't provide
all the features of bdist_wininst (uninstallers...) so they didn't
replace bdist_wininst, they sat alongside. Now you're proposing to
make the same mistake again.

Can I repeat that in big letters? The key is a SINGLE DISTRIBUTION FORMAT.

If you can persuade everyone to accept a format which ignores clearly
stated user requirements, go for it. But if you can't, you're making
the problem worse rather than helping. My money's on a solution that
acknowledges and addresses user requirements instead.

Paul.

From brett at python.org  Tue Oct 20 22:54:53 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 20 Oct 2009 13:54:53 -0700
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <5c6f2a5d0910200757s32b814aeod5b7d6d8c202df47@mail.gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com> 
	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com> 
	<5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com> 
	<4ADD95B7.8040304@gmail.com>
	<5c6f2a5d0910200422g71560d5eic6503da0d4fdc0fa@mail.gmail.com> 
	<41504.63.251.87.214.1256050207.squirrel@mail.trueblade.com> 
	<5c6f2a5d0910200757s32b814aeod5b7d6d8c202df47@mail.gmail.com>
Message-ID: <bbaeab100910201354x116cee7eucf341b84919c20bb@mail.gmail.com>

On Tue, Oct 20, 2009 at 07:57, Mark Dickinson <dickinsm at gmail.com> wrote:

> On Tue, Oct 20, 2009 at 3:50 PM, Eric Smith <eric at trueblade.com> wrote:
> >> The code was working a few months ago (with all Decimal tests
> >> passing), but there have been some changes and bugfixes since
> >> then.  I might try to resurrect that code, dropping the _Decimal type
> and
> >> just concentrating on Deccoeff.
> >
> > My only concern about this is the effect it would have on IronPython,
> > Jython, PyPy, and other alternate implementations that use the stdlib.
>
> Yes, that worries me a bit, too.  I have the same worry with the idea
> of rewriting the entire decimal module in C.
>
> The Deccoeff type is very simple, though.  It would be easy to create
> a pure Python version of it, and then do something like:
>
> try:
>    from _decimal import Deccoeff  # try to get C version
> except ImportError:
>    from deccoeff import Deccoeff  # else use Python fallback code.


And this is why you shouldn't have to worry. As long as the tests exercise
both the pure Python and C versions then the other VMs are covered.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091020/99138bec/attachment.htm>

From p.f.moore at gmail.com  Tue Oct 20 23:00:12 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 20 Oct 2009 22:00:12 +0100
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <20091020131547.GA615@mail.bytereef.org>
References: <20091020131547.GA615@mail.bytereef.org>
Message-ID: <79990c6b0910201400k24e5ad4aj3813dc69d5049c9a@mail.gmail.com>

2009/10/20 Stefan Krah <stefan-usenet at bytereef.org>:
> Hi,
>
> as some of you know, recently I've released an arbitrary precision
> C library for decimal arithmetic together with a Python module:
>
> http://www.bytereef.org/libmpdec.html
> http://www.bytereef.org/fastdec.html
>
>
> Both the library and the module have been tested extensively. Fastdec
> currently differs from decimal.py in a couple of ways that could be
> fixed. The license is AGPL, but if there is interest in integrating it
> into Python I'd release it under a Python-compatible license.
>
>
> There have been several approaches towards getting C decimal arithmetic
> into Python:
>
> http://bugs.python.org/issue2486
>
>
> Fastdec follows Raymond Hettinger's suggestion to provide wrappers for
> an independent C implementation. Arguments in favour of fastdec are:
>
> ?* Complete implementation of Mike Cowlishaw's specification
>
> ?* C library can be tested independently
>
> ?* Redundant arithmetic module for tests against decimal.py
>
> ?* Faster than Java BigDecimal
>
> ?* Compares relatively well in speed against gmpy
>
>
> To be clear, I would not want to _replace_ decimal.py. Rather I'd like to
> see a cdecimal module alongside decimal.py.

Why? If it's 100% compatible with decimal.py, just replace it. All the
user should see is improved speed. Let's not do another
pickle/cpickle.

> I know that ultimately there should be a PEP for module inclusions. The
> purpose of this post is to gauge interest. Specifically:
>
>
> 1. Are you generally in favour of a C decimal module in Python?

Yes, although I have to admit my interest is fairly theoretical.

> 2. Would fastdec - after achieving full decimal.py compatibility - be
> ? a serious candidate?

I don't see why not, if it was 100% compatible with decimal.py

> 3. Could I use this list to settle a couple of questions, or would perhaps
> ? a Python developer be willing to work with me to make it compatible? I'm
> ? asking this to avoid doing work that would not find acceptance afterwards.

I can't help much here, but I'd prefer to see discussions on
python-dev so I'm +1 on keeping discussions here.

Waiting eagerly to hear the experts' comments...

Paul.

From barry at barrys-emacs.org  Tue Oct 20 23:03:39 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Tue, 20 Oct 2009 22:03:39 +0100
Subject: [Python-Dev] Add const to python API - issue 6952
In-Reply-To: <4ADD2FE9.2010807@v.loewis.de>
References: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>
	<4ADD2FE9.2010807@v.loewis.de>
Message-ID: <74D1336F-AB87-431F-895C-885EE344CEBD@barrys-emacs.org>


On 20 Oct 2009, at 04:35, Martin v. L?wis wrote:

>> The patch I developed for comment only adds const to the input
>> parameters and used casts to
>> allow output parameters to stay without the const.
>
> What specific APIs are you talking about here?

Checking my patch I have two functions that need to
have output params changed to const to avoid casting.

	PyOS_strtoul			- ptr
	PyLong_FromString 	- pend

Barry


From daniel at stutzbachenterprises.com  Tue Oct 20 23:31:19 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 20 Oct 2009 16:31:19 -0500
Subject: [Python-Dev] Add const to python API - issue 6952
In-Reply-To: <74D1336F-AB87-431F-895C-885EE344CEBD@barrys-emacs.org>
References: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>
	<4ADD2FE9.2010807@v.loewis.de>
	<74D1336F-AB87-431F-895C-885EE344CEBD@barrys-emacs.org>
Message-ID: <eae285400910201431j17d35733vc6423abf85997af9@mail.gmail.com>

On Tue, Oct 20, 2009 at 4:03 PM, Barry Scott <barry at barrys-emacs.org> wrote:

> Checking my patch I have two functions that need to
> have output params changed to const to avoid casting.
>
>        PyOS_strtoul                    - ptr
>        PyLong_FromString       - pend
>

This is a no-win situation.  If the string is const in the caller, they
currently need to cast it.  If you make the change, then if string is not
const in the caller then they will need to cast it.  I've provided a short
example below and marked the lines that generate "an incompatible pointer
type" warning with gcc.

I suggest following POSIX's lead and omitted the const in these cases.

void test_const(const char **foo);
void test_noconst(char **foo);

int main(void) {
        char *noconst_var;
        const char *const_var;
        test_const(&noconst_var);  // generates a warning
        test_const(&const_var);
        test_noconst(&noconst_var);
        test_noconst(&const_var);   // generates a warning
        return 0;
}

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091020/e9510b10/attachment-0001.htm>

From david.lyon at preisshare.net  Tue Oct 20 23:40:07 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 20 Oct 2009 17:40:07 -0400
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
 on Virtualenv, Pip)
In-Reply-To: <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
	<4ADDC6CA.5000809@simplistix.co.uk>
	<79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com>
Message-ID: <52bce2be86dbbd753f612256602d2c70@preisshare.net>

On Tue, 20 Oct 2009 21:49:42 +0100, Paul Moore <p.f.moore at gmail.com> wrote:

> Can I repeat that in big letters? The key is a SINGLE DISTRIBUTION
FORMAT.

ok - but that pretty much exists..

> If you can persuade everyone to accept a format which ignores clearly
> stated user requirements, go for it. But if you can't, you're making
> the problemorse rather than helping. My money's on a solution that
> acknowledges and addresses user requirements instead.

What solution is that exactly? You don't say..

I would be happy to revamp pythonpkgmgr and backgrade to have a tk
interface if it is felt that would help. That's not so difficult -
certainly not on my side.
 
I don't know which iceberg the people that want to keep command line 
interfaces forever still live on, but they seem very good at sinking any 
attempts of passage by forward moving boats..

Distutils roadmap for windows is to apply more freeze spray.. it's a
cryogenic source code freezing iceberg...

It's just there for windows users to crash their boats into..

Try and see for yourself..

David


From fijall at gmail.com  Wed Oct 21 00:49:28 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 20 Oct 2009 16:49:28 -0600
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <79990c6b0910201400k24e5ad4aj3813dc69d5049c9a@mail.gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<79990c6b0910201400k24e5ad4aj3813dc69d5049c9a@mail.gmail.com>
Message-ID: <693bc9ab0910201549o101673fbpda9e7ff5131bfd2@mail.gmail.com>

On Tue, Oct 20, 2009 at 3:00 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/10/20 Stefan Krah <stefan-usenet at bytereef.org>:
>> Hi,
>>
>> as some of you know, recently I've released an arbitrary precision
>> C library for decimal arithmetic together with a Python module:
>>
>> http://www.bytereef.org/libmpdec.html
>> http://www.bytereef.org/fastdec.html
>>
>>
>> Both the library and the module have been tested extensively. Fastdec
>> currently differs from decimal.py in a couple of ways that could be
>> fixed. The license is AGPL, but if there is interest in integrating it
>> into Python I'd release it under a Python-compatible license.
>>
>>
>> There have been several approaches towards getting C decimal arithmetic
>> into Python:
>>
>> http://bugs.python.org/issue2486
>>
>>
>> Fastdec follows Raymond Hettinger's suggestion to provide wrappers for
>> an independent C implementation. Arguments in favour of fastdec are:
>>
>> ?* Complete implementation of Mike Cowlishaw's specification
>>
>> ?* C library can be tested independently
>>
>> ?* Redundant arithmetic module for tests against decimal.py
>>
>> ?* Faster than Java BigDecimal
>>
>> ?* Compares relatively well in speed against gmpy
>>
>>
>> To be clear, I would not want to _replace_ decimal.py. Rather I'd like to
>> see a cdecimal module alongside decimal.py.
>
> Why? If it's 100% compatible with decimal.py, just replace it. All the
> user should see is improved speed. Let's not do another
> pickle/cpickle.

For example other python implementations might decide to use python
version as long as builtin version does not appear. Python versions are
usually also better targets for jit than mixed versions. C level versions also
usually have more bugs (just statistics), so some people might want to
choose pure-python version.

In general - some people have some reasons.

Cheers,
fijal

From cournape at gmail.com  Wed Oct 21 04:44:25 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 21 Oct 2009 11:44:25 +0900
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<4ACE718D.8090400@cryptelium.net>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
	<4ADDC6CA.5000809@simplistix.co.uk>
	<79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com>
Message-ID: <5b8d13220910201944h36577afeu15f96c4cbf86dd88@mail.gmail.com>

On Wed, Oct 21, 2009 at 5:49 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/10/20 Chris Withers <chris at simplistix.co.uk>:
>> I wouldn't have a problem if integrating with the windows package manager
>> was an optional extra, but I think it's one of many types of package
>> management that need to be worried about, so might be easier to get the
>> others working and let anyone who wants anything beyond a pure-python
>> packaging system that works across platforms, regardless of whether binary
>> extensions are needed, do the work themselves...
>
> There are many (I believe) Windows users for whom bdist_wininst is
> just what they want. For those people, where's the incentive to switch
> in what you propose? You're not providing the features they currently
> have, and frankly "do the work yourself" is no answer (not everyone
> can, often for entirely legitimate reasons).

I am not so familiar with msi or wininst internals, but isn't it
possible to install w.r.t. a given prefix ? Basically, making it
possible to use a wininst in a virtualenv if required (in which case I
guess it would not register with the windows db - at least it should
be possible to disable it).

The main problem with bdist_wininst installers is that they don't work
with setuptools dependency stuff (at least, that's the reason given by
windows users for a numpy egg on windows, whereas we used to only
provide an exe). But you could argue it is a setuptools pb as much as
a wininst pb, I guess.

David

From steven.bethard at gmail.com  Wed Oct 21 07:05:51 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 20 Oct 2009 22:05:51 -0700
Subject: [Python-Dev] Distutils and Distribute roadmap (and some words
	on Virtualenv, Pip)
In-Reply-To: <5b8d13220910201944h36577afeu15f96c4cbf86dd88@mail.gmail.com>
References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com>
	<94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com>
	<b654cd2e0910081822p76abea9cud51b307517ea2f2b@mail.gmail.com>
	<loom.20091009T124309-659@post.gmane.org>
	<79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com>
	<b654cd2e0910090934r58439915va8a57ef315c511be@mail.gmail.com>
	<4ACF6993.5040704@voidspace.org.uk>
	<4ADDC6CA.5000809@simplistix.co.uk>
	<79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com>
	<5b8d13220910201944h36577afeu15f96c4cbf86dd88@mail.gmail.com>
Message-ID: <d11dcfba0910202205m6b40c04fs54f051be3dfcf333@mail.gmail.com>

On Tue, Oct 20, 2009 at 7:44 PM, David Cournapeau <cournape at gmail.com> wrote:
> On Wed, Oct 21, 2009 at 5:49 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/10/20 Chris Withers <chris at simplistix.co.uk>:
>> There are many (I believe) Windows users for whom bdist_wininst is
>> just what they want. For those people, where's the incentive to switch
>> in what you propose? You're not providing the features they currently
>> have, and frankly "do the work yourself" is no answer (not everyone
>> can, often for entirely legitimate reasons).
>
> I am not so familiar with msi or wininst internals, but isn't it
> possible to install w.r.t. a given prefix ?

This is definitely supported by the msi. In fact, you can do this at
the command line (although I don't know if this is considered
officially supported or not):

$ argparse-1.0.2.win32-py2.6.msi /quiet TARGETDIR=C:\TempPython
$ dir C:\TempPython\Lib\site-packages
 Volume in drive C has no label.
 Volume Serial Number is B6E6-4B4D

 Directory of C:\TempPython\Lib\site-packages

10/20/2009  10:04 PM    <DIR>          .
10/20/2009  10:04 PM    <DIR>          ..
10/20/2009  10:03 PM             1,584 argparse-1.0.2-py3.1.egg-info
10/09/2009  11:03 AM            86,620 argparse.py

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From martin at v.loewis.de  Wed Oct 21 07:15:14 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 21 Oct 2009 07:15:14 +0200
Subject: [Python-Dev] Add const to python API - issue 6952
In-Reply-To: <eae285400910201431j17d35733vc6423abf85997af9@mail.gmail.com>
References: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>	<4ADD2FE9.2010807@v.loewis.de>	<74D1336F-AB87-431F-895C-885EE344CEBD@barrys-emacs.org>
	<eae285400910201431j17d35733vc6423abf85997af9@mail.gmail.com>
Message-ID: <4ADE98E2.3080707@v.loewis.de>

> I suggest following POSIX's lead and omitted the const in these cases.

Thanks, that sounds reasonable.

Regards,
Martin

From kristjan at ccpgames.com  Wed Oct 21 12:15:52 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 21 Oct 2009 10:15:52 +0000
Subject: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?
In-Reply-To: <4ADE35F0.10803@molden.no>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org> <4ADE35F0.10803@molden.no>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>



> -----Original Message-----
> From: python-ideas-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On
> Behalf Of Sturla Molden
> Sent: 20. okt?ber 2009 22:13
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Remove GIL with CAS instructions?
> 
> 
> - The GIL has consequences on multicore CPUs that are overlooked:
> thread
> switches are usually missed at check intervals. This could be fixed
> without removing the GIL: For example, there could be a wait-queue for
> the GIL; a thread that request the GIL puts itself in the back.
> 


This depends entirely on the platform and primitives used to implement the GIL.
I'm interested in windows.  There, I found this article:
http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html
So, you may be on to something.  Perhaps a simple C test is in order then?

I did that.  I found, on my dual-core vista machine, that running "release", that both Mutexes and CriticalSections behaved as you describe, with no "fairness".  Using a "semaphore" seems to retain fairness, however.
"fairness" was retained in debug builds too, strangely enough.

Now, Python uses none of these.  On windows, it uses an "Event" object coupled with an atomically updated counter.  This also behaves fairly.

The test application is attached.


I think that you ought to sustantiate your claims better, maybe with a specific platform and using some test like the above.

On the other hand, it shows that we must be careful what we use.  There has been some talk of using CriticalSections for the GIL on windows.  This test ought to show the danger of that.  The GIL is different than a regular lock.  It is a reverse-lock, really, and therefore may need to be implemented in its own special way, if we want very fast mutexes for the rest of the system (cc to python-dev)

Cheers,

Kristj?n
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: giltest.cpp
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091021/43a31586/attachment.txt>

From ncoghlan at gmail.com  Wed Oct 21 12:27:58 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Oct 2009 20:27:58 +1000
Subject: [Python-Dev] SIGCHECK() in longobject.c
In-Reply-To: <bbaeab100910201354x116cee7eucf341b84919c20bb@mail.gmail.com>
References: <loom.20091018T215434-493@post.gmane.org>
	<5c6f2a5d0910190427v74a79cacrf4a472017713763@mail.gmail.com>
	<1f7befae0910191358k52496861r4a6b39dae58f5235@mail.gmail.com>
	<5c6f2a5d0910200139p4ef82df2k949eba8fa33365fb@mail.gmail.com>
	<4ADD95B7.8040304@gmail.com>
	<5c6f2a5d0910200422g71560d5eic6503da0d4fdc0fa@mail.gmail.com>
	<41504.63.251.87.214.1256050207.squirrel@mail.trueblade.com>
	<5c6f2a5d0910200757s32b814aeod5b7d6d8c202df47@mail.gmail.com>
	<bbaeab100910201354x116cee7eucf341b84919c20bb@mail.gmail.com>
Message-ID: <4ADEE22E.2090002@gmail.com>

Brett Cannon wrote:
> On Tue, Oct 20, 2009 at 07:57, Mark Dickinson <dickinsm at gmail.com
> <mailto:dickinsm at gmail.com>> wrote:
>     The Deccoeff type is very simple, though.  It would be easy to create
>     a pure Python version of it, and then do something like:
> 
>     try:
>        from _decimal import Deccoeff  # try to get C version
>     except ImportError:
>        from deccoeff import Deccoeff  # else use Python fallback code.
> 
> 
> And this is why you shouldn't have to worry. As long as the tests
> exercise both the pure Python and C versions then the other VMs are covered.

We need to worry at least a little bit, as the pure Python version
doesn't exist at this point in time.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Oct 21 12:37:39 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Oct 2009 20:37:39 +1000
Subject: [Python-Dev] Interest in integrating C decimal module
	into	Python?
In-Reply-To: <693bc9ab0910201549o101673fbpda9e7ff5131bfd2@mail.gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>	<79990c6b0910201400k24e5ad4aj3813dc69d5049c9a@mail.gmail.com>
	<693bc9ab0910201549o101673fbpda9e7ff5131bfd2@mail.gmail.com>
Message-ID: <4ADEE473.9020508@gmail.com>

Maciej Fijalkowski wrote:
> For example other python implementations might decide to use python
> version as long as builtin version does not appear. Python versions are
> usually also better targets for jit than mixed versions. C level versions also
> usually have more bugs (just statistics), so some people might want to
> choose pure-python version.
> 
> In general - some people have some reasons.

Although nobody has broken "sys.modules['_decimal'] = 0", so
deliberately turning off optimisations is pretty easy if you really
don't want them.

There's a reason we moved to implicit import of optimised versions in
Py3k - we're unlikely to revert to the old way of doing things.

As far as decimal.py in particular goes, there are significant
maintenance gains in keeping a lot of the non-performance critical
context management code in pure Python. So we're likely to wait and see
how much speed Mark can wring out of a simple C decimal coefficient
object (that other implementations can also fairly easily provide
natively) before looking seriously at a wholesale replacement of the module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Wed Oct 21 12:51:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 10:51:43 +0000 (UTC)
Subject: [Python-Dev] GIL behaviour under Windows
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>
	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
Message-ID: <loom.20091021T125010-360@post.gmane.org>


Hello Kristjan,

> This depends entirely on the platform and primitives used to implement the GIL.
> I'm interested in windows.

Could you try ccbench (*) under Windows? The only Windows system I have here is
a qemu virtual machine and it wouldn't be very realistic to do concurrency
measurements on it.

(*) http://svn.python.org/view/sandbox/trunk/ccbench/

Thanks

Antoine.



From scott+python-dev at scottdial.com  Wed Oct 21 14:28:48 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 21 Oct 2009 08:28:48 -0400
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091021T125010-360@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
Message-ID: <4ADEFE80.3050002@scottdial.com>

Antoine Pitrou wrote:
> Could you try ccbench (*) under Windows? The only Windows system I have here is
> a qemu virtual machine and it wouldn't be very realistic to do concurrency
> measurements on it.
> 
> (*) http://svn.python.org/view/sandbox/trunk/ccbench/
> 

I don't really know how this test works, so I won't claim to understand
the results either. However, here you go:

C:\>systeminfo
OS Name:                   Microsoft Windows XP Professional
OS Version:                5.1.2600 Service Pack 3 Build 2600

C:\>c:\Python26\python.exe
Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit
(Intel)] on win32

C:\>start /B /HIGH c:\Python26\python.exe c:\ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 377 iterations/s.
threads=2: 376 ( 99 %)
threads=3: 380 ( 100 %)
threads=4: 376 ( 99 %)

regular expression (C)

threads=1: 222 iterations/s.
threads=2: 213 ( 95 %)
threads=3: 223 ( 100 %)
threads=4: 218 ( 97 %)

bz2 compression (C)

threads=1: 324 iterations/s.
threads=2: 324 ( 99 %)
threads=3: 327 ( 100 %)
threads=4: 324 ( 100 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From solipsis at pitrou.net  Wed Oct 21 14:52:40 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 14:52:40 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <4ADEFE80.3050002@scottdial.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
	<4ADEFE80.3050002@scottdial.com>
Message-ID: <1256129560.5069.30.camel@localhost>

> I don't really know how this test works, so I won't claim to understand
> the results either. However, here you go:

Thanks.

Interesting results. I wonder what they would be like on a multi-core
machine. The GIL seems to behave perfectly on your setup (no performance
degradation due to concurrency, and zero latencies).

For a quick explanation of what the benchmark does:

- the "throughput" part launches N computational (CPU-bound) threads and
measures the total work done per second, and then compares the result to
the 1-thread result. It does so with three different workloads, which
have different impacts on the GIL. 100% is the most you can get on a
single-core machine. On a multi-core machine, you can get more than 100%
with the workload that explicitly releases the GIL before taxing the CPU
(bz2 compression).

- the "latency" part launches N computational threads in the background,
and the main thread listens for periodic ping messages on an UDP socket
(the ping messages themselves are emitted from a separate Python
process, so as to decouple it from the process under test). The
latencies are the measured delay between the emission of the UDP message
and the moment at which the recv() call returns in the main thread. This
aims at reproducing the situation where a thread handles IO operations
while one or several other threads perform heavy computations.

Regards

Antoine.


> C:\>systeminfo
> OS Name:                   Microsoft Windows XP Professional
> OS Version:                5.1.2600 Service Pack 3 Build 2600
> 
> C:\>c:\Python26\python.exe
> Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit
> (Intel)] on win32
> 
> C:\>start /B /HIGH c:\Python26\python.exe c:\ccbench.py
> --- Throughput ---
> 
> Pi calculation (Python)
> 
> threads=1: 377 iterations/s.
> threads=2: 376 ( 99 %)
> threads=3: 380 ( 100 %)
> threads=4: 376 ( 99 %)
> 
> regular expression (C)
> 
> threads=1: 222 iterations/s.
> threads=2: 213 ( 95 %)
> threads=3: 223 ( 100 %)
> threads=4: 218 ( 97 %)
> 
> bz2 compression (C)
> 
> threads=1: 324 iterations/s.
> threads=2: 324 ( 99 %)
> threads=3: 327 ( 100 %)
> threads=4: 324 ( 100 %)
> 
> --- Latency ---
> 
> Background CPU task: Pi calculation (Python)
> 
> CPU threads=0: 0 ms. (std dev: 0 ms.)
> CPU threads=1: 0 ms. (std dev: 0 ms.)
> CPU threads=2: 0 ms. (std dev: 0 ms.)
> CPU threads=3: 0 ms. (std dev: 0 ms.)
> CPU threads=4: 0 ms. (std dev: 0 ms.)
> 
> Background CPU task: regular expression (C)
> 
> CPU threads=0: 0 ms. (std dev: 0 ms.)
> CPU threads=1: 0 ms. (std dev: 0 ms.)
> CPU threads=2: 0 ms. (std dev: 0 ms.)
> CPU threads=3: 0 ms. (std dev: 0 ms.)
> CPU threads=4: 0 ms. (std dev: 0 ms.)
> 
> Background CPU task: bz2 compression (C)
> 
> CPU threads=0: 0 ms. (std dev: 0 ms.)
> CPU threads=1: 0 ms. (std dev: 0 ms.)
> CPU threads=2: 0 ms. (std dev: 0 ms.)
> CPU threads=3: 0 ms. (std dev: 0 ms.)
> CPU threads=4: 0 ms. (std dev: 0 ms.)
> 


From scott+python-dev at scottdial.com  Wed Oct 21 15:15:51 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 21 Oct 2009 09:15:51 -0400
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <1256129560.5069.30.camel@localhost>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<4ADEFE80.3050002@scottdial.com>
	<1256129560.5069.30.camel@localhost>
Message-ID: <4ADF0987.9000500@scottdial.com>

Antoine Pitrou wrote:
> Interesting results. I wonder what they would be like on a multi-core
> machine. The GIL seems to behave perfectly on your setup (no performance
> degradation due to concurrency, and zero latencies).

You are correct, my machine is a single-core system. I don't have any
multi-core systems around to test it on, I'm still in the stone age.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From kristjan at ccpgames.com  Wed Oct 21 15:15:59 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 21 Oct 2009 13:15:59 +0000
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091021T125010-360@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF41FFC@exchis.ccp.ad.local>



> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Antoine Pitrou
> Sent: 21. okt?ber 2009 10:52
> To: python-dev at python.org
> Subject: Re: [Python-Dev] GIL behaviour under Windows
> 
> 
> Hello Kristjan,
> 
> > This depends entirely on the platform and primitives used to
> implement the GIL.
> > I'm interested in windows.
> 
> Could you try ccbench (*) under Windows? The only Windows system I have
> here is
> a qemu virtual machine and it wouldn't be very realistic to do
> concurrency
> measurements on it.
> 

Hi, I just want to stress, that according to my test, the current GIL implementation works as intended on windows.  But if we were to reimplement it, say using a CriticalSection, then yielding the GIL as we do in sys.checkinterval wouldn't work as intended anymore.  Just something to keep in mind in case anyone is thinking along those lines.

K


From kristjan at ccpgames.com  Wed Oct 21 15:24:15 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 21 Oct 2009 13:24:15 +0000
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091021T125010-360@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>



> -----Original Message-----
> Could you try ccbench (*) under Windows? The only Windows system I have
> here is
> a qemu virtual machine and it wouldn't be very realistic to do
> concurrency
> measurements on it.
> 
> (*) http://svn.python.org/view/sandbox/trunk/ccbench/

I?ve run it twice on my dual core machine.  It hangs every time, but not in the same place:
D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
--- Throughput ---

Pi calculation (Python)

threads=1: 514 iterations/s.
threads=2: 403 ( 78 %)
threads=3: 392 ( 76 %)
threads=4: 364 ( 70 %)

regular expression (C)

threads=1: 443 iterations/s.
threads=2: 474 ( 106 %)
threads=3: 461 ( 104 %)
threads=4: 466 ( 105 %)

SHA1 hashing (C)

threads=1: 983 iterations/s.
threads=2: 1026 ( 104 %)
^C
D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
--- Throughput ---

Pi calculation (Python)

threads=1: 506 iterations/s.
threads=2: 405 ( 80 %)


From solipsis at pitrou.net  Wed Oct 21 15:34:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 15:34:47 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
Message-ID: <1256132087.5069.39.camel@localhost>


> > (*) http://svn.python.org/view/sandbox/trunk/ccbench/
> 
> I?ve run it twice on my dual core machine.  It hangs every time, but not in the same place:
> D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py

Ah, you should report a bug then. ccbench is pure Python (and not
particularly evil Python), it shouldn't be able to crash the
interpreter.




From dickinsm at gmail.com  Wed Oct 21 16:51:41 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 21 Oct 2009 15:51:41 +0100
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <20091020131547.GA615@mail.bytereef.org>
References: <20091020131547.GA615@mail.bytereef.org>
Message-ID: <5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>

On Tue, Oct 20, 2009 at 2:15 PM, Stefan Krah <stefan-usenet at bytereef.org> wrote:
> 1. Are you generally in favour of a C decimal module in Python?

I'm certainly interested in the general idea;  whether I'd be in favour
of replacing decimal.py with a particular C version would depend on
a lot of factors, with code quality, interchangeability with the current
decimal module, and maintainability by Python core developers
high on the list.  There's also the question of what IronPython
and Jython would do.

> 2. Would fastdec - after achieving full decimal.py compatibility - be
> ? a serious candidate?

Definitely.  As far as I know it's the only real candidate for a full
C version of decimal right now.  Other possibilities that I'm aware of:

* I think Raymond Hettinger is working on a C version of decimal.
I'm not sure what its current status is.  Raymond?

* There's a partially complete rewrite of decimal in C in the sandbox,
dating from the Need for Speed sprint in 2006:

* Wrapping the decNumber library directly would make some sense,
but I think licensing issues rule this out.

http://svn.python.org/view/sandbox/trunk/decimal-c/

Last time I looked at this it wasn't up to date with the decimal
specification:  I'm not sure that functions like exp, log and log10
are currently working.  Georg Brandl might know better than I do.

> 3. Could I use this list to settle a couple of questions, or would perhaps
> ? a Python developer be willing to work with me to make it compatible? I'm
> ? asking this to avoid doing work that would not find acceptance afterwards.

I don't see why not.  Working closely with one of the core developers
on this sounds like a good idea, as well:  if the code were to go into
Python core, at least one (and preferably more than one) core dev
should be familiar enough with it for maintenance.  (BTW, I think that
*if* fastdec went in it's likely you'd be granted commit privileges at
some point.)

It's difficult to gauge the likelihood of eventual acceptance in advance,
though.  Maybe writing a PEP would be an efficient use of time in this
regard?  There are certainly some open issues (e.g., what to do with
the existing Python module;  what should other Python implementations
do).

I think my biggest concern is maintenance:  we'd be replacing
8500 lines of Python code in a single file, that several of the
current core developers understand well, with 30000 (Stefan, is
that about accurate?) lines of C in several files, that presumably
none of the current core devs is familiar with right now.  What
happens when (e.g.,) the number-theoretic transform needs
updating for one reason or another?  Stefan, do you see yourself
having time to spend on maintenance of this code for the
forseeable future?

BTW, does anyone know the current SLOC count for py3k?

Mark

From dickinsm at gmail.com  Wed Oct 21 16:59:44 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 21 Oct 2009 15:59:44 +0100
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <4ADEE473.9020508@gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<79990c6b0910201400k24e5ad4aj3813dc69d5049c9a@mail.gmail.com>
	<693bc9ab0910201549o101673fbpda9e7ff5131bfd2@mail.gmail.com>
	<4ADEE473.9020508@gmail.com>
Message-ID: <5c6f2a5d0910210759i7b2bc12awbc694c9660f68a15@mail.gmail.com>

On Wed, Oct 21, 2009 at 11:37 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As far as decimal.py in particular goes, there are significant
> maintenance gains in keeping a lot of the non-performance critical
> context management code in pure Python. So we're likely to wait and see
> how much speed Mark can wring out of a simple C decimal coefficient
> object

No need to wait for this. :-)  I don't really expect to get much speed gain
at all in normal, low-precision (= precisions less than 100 digits, say)
use.  Even doubling the speed would be way too much to hope for here.

The only real gain from a decimal integer coefficient would be fixing
the asymptotics for high-precision calculations.

Mark

From solipsis at pitrou.net  Wed Oct 21 17:05:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 15:05:16 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Interest_in_integrating_C_decimal_module_i?=
	=?utf-8?q?nto=09Python=3F?=
References: <20091020131547.GA615@mail.bytereef.org>
	<5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
Message-ID: <loom.20091021T170249-555@post.gmane.org>

Mark Dickinson <dickinsm <at> gmail.com> writes:
> 
> There are certainly some open issues (e.g., what to do with
> the existing Python module;  what should other Python implementations
> do).

The existing module could be kept as a fallback. Also, the test suite should be
careful to test both implementations (like what is currently done for the io
module).

> BTW, does anyone know the current SLOC count for py3k?

Here you are, generated using David A. Wheeler's 'SLOCCount':

SLOC	Directory	SLOC-by-Language (Sorted)
261496  Lib             python=261451,sh=45
186769  Modules         ansic=173279,asm=9561,sh=3929
53258   Objects         ansic=53258
40257   Python          ansic=40229,python=28
27220   Tools           python=27117,ansic=67,sh=36
18892   Demo            python=18511,ansic=377,sh=4
9168    PC              ansic=8465,python=703
5840    Include         ansic=5840
5799    Parser          ansic=3914,python=1885
3485    Misc            lisp=2948,python=242,sh=185,ansic=110
3101    Doc             python=2306,ansic=795
3030    Mac             python=2138,objc=775,sh=109,ansic=8
1666    top_dir         python=1140,ansic=286,sh=240
349     PCbuild         python=279,ansic=70
337     build           ansic=295,python=42
0       Grammar         (none)

Totals grouped by language (dominant language first):
python:      315842 (50.89%)
ansic:       286993 (46.24%)
asm:           9561 (1.54%)
sh:            4548 (0.73%)
lisp:          2948 (0.47%)
objc:           775 (0.12%)

Total Physical Source Lines of Code (SLOC)                = 620,667



From john at arbash-meinel.com  Wed Oct 21 17:15:56 2009
From: john at arbash-meinel.com (John Arbash Meinel)
Date: Wed, 21 Oct 2009 10:15:56 -0500
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <1256129560.5069.30.camel@localhost>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<4ADEFE80.3050002@scottdial.com>
	<1256129560.5069.30.camel@localhost>
Message-ID: <4ADF25AC.802@arbash-meinel.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antoine Pitrou wrote:
>> I don't really know how this test works, so I won't claim to understand
>> the results either. However, here you go:
> 
> Thanks.
> 
> Interesting results. I wonder what they would be like on a multi-core
> machine. The GIL seems to behave perfectly on your setup (no performance
> degradation due to concurrency, and zero latencies).
> 

C:\downloads>C:\Python26\python.exe ccbench.py
- --- Throughput ---

Pi calculation (Python)

threads=1: 691 iterations/s.
threads=2: 400 ( 57 %)
threads=3: 453 ( 65 %)
threads=4: 467 ( 67 %)

^- seems to have some contention



regular expression (C)

threads=1: 592 iterations/s.
threads=2: 598 ( 101 %)
threads=3: 587 ( 99 %)
threads=4: 586 ( 99 %)

bz2 compression (C)

threads=1: 536 iterations/s.
threads=2: 1056 ( 196 %)
threads=3: 1040 ( 193 %)
threads=4: 1060 ( 197 %)


^- seems to properly show that I have 2 cores here.


- --- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 38 ms. (std dev: 18 ms.)
CPU threads=2: 173 ms. (std dev: 77 ms.)
CPU threads=3: 518 ms. (std dev: 264 ms.)
CPU threads=4: 661 ms. (std dev: 343 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)


John
=;->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrfJawACgkQJdeBCYSNAANQlgCgwx0TCLh7YhLSJxkfOuMi1/YF
XhkAoIONtdP0rR1YW0nDza+wpKpAlInd
=L4WZ
-----END PGP SIGNATURE-----

From john at arbash-meinel.com  Wed Oct 21 17:27:12 2009
From: john at arbash-meinel.com (John Arbash Meinel)
Date: Wed, 21 Oct 2009 10:27:12 -0500
Subject: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>
	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
Message-ID: <4ADF2850.1040304@arbash-meinel.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kristj?n Valur J?nsson wrote:
...

> This depends entirely on the platform and primitives used to implement the GIL.
> I'm interested in windows.  There, I found this article:
> http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html
> So, you may be on to something.  Perhaps a simple C test is in order then?
> 
> I did that.  I found, on my dual-core vista machine, that running "release", that both Mutexes and CriticalSections behaved as you describe, with no "fairness".  Using a "semaphore" seems to retain fairness, however.
> "fairness" was retained in debug builds too, strangely enough.
> 
> Now, Python uses none of these.  On windows, it uses an "Event" object coupled with an atomically updated counter.  This also behaves fairly.
> 
> The test application is attached.
> 
> 
> I think that you ought to sustantiate your claims better, maybe with a specific platform and using some test like the above.
> 
> On the other hand, it shows that we must be careful what we use.  There has been some talk of using CriticalSections for the GIL on windows.  This test ought to show the danger of that.  The GIL is different than a regular lock.  It is a reverse-lock, really, and therefore may need to be implemented in its own special way, if we want very fast mutexes for the rest of the system (cc to python-dev)
> 
> Cheers,
> 
> Kristj?n

I can compile and run this, but I'm not sure I know how to interpret the
results. If I understand it correctly, then everything but "Critical
Sections" are fair on my Windows Vista machine.

To run, I changed the line "#define EVENT" to EVENT, MUTEX, SEMAPHORE
and CRIT. I then built and ran in "Release" environment (using VS 2008
Express)

For all but CRIT, I saw things like:
thread       5532 reclaims GIL
thread       5532 working 51234 units
thread       5532 worked 51234 units: 1312435761
thread       5532 flashing GIL
thread       5876 reclaims GIL
thread       5876 working 51234 units
thread       5876 worked 51234 units: 1312435761
thread       5876 flashing GIL

Where there would be 4 lines for one thread, then 4 lines for the other
thread.

for CRIT, I saw something more like 50 lines for one thread, and then 50
lines for the other thread.

This is Vista Home Basic, and VS 2008 Express Edition, with a 2-core
machine.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrfKFAACgkQJdeBCYSNAANbuQCgudU0IChylofTwvUk/JglChBd
9gsAoIJHj63/CagKpduUsd68HV8pP3QX
=CuUj
-----END PGP SIGNATURE-----

From dickinsm at gmail.com  Wed Oct 21 17:28:50 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 21 Oct 2009 16:28:50 +0100
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <loom.20091021T170249-555@post.gmane.org>
References: <20091020131547.GA615@mail.bytereef.org>
	<5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
	<loom.20091021T170249-555@post.gmane.org>
Message-ID: <5c6f2a5d0910210828g5dab0aa0qac54815741734e8f@mail.gmail.com>

On Wed, Oct 21, 2009 at 4:05 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Mark Dickinson <dickinsm <at> gmail.com> writes:
>> BTW, does anyone know the current SLOC count for py3k?
>
> Here you are, generated using David A. Wheeler's 'SLOCCount':
> [...]

Thanks, Antoine!  With SLOCCount I can revise my earlier numbers, as well:
Here's Stefan Krah's mpdecimal, version 0.80:

  SLOC	Directory	SLOC-by-Language (Sorted)
  21445   top_dir         ansic=21267,sh=105,python=55,asm=18
  6238    python          python=6177,java=43,sh=18
  1403    tests           ansic=1356,sh=47
  476     literature      lisp=476
  274     cmd             ansic=274
  11      tools           sh=11
  0       doc             (none)

  Totals grouped by language (dominant language first):
  ansic:        22897 (76.71%)
  python:        6232 (20.88%)
  lisp:           476 (1.59%)
  sh:             181 (0.61%)
  java:            43 (0.14%)
  asm:             18 (0.06%)


Lib/decimal.py:

  SLOC	Directory	SLOC-by-Language (Sorted)
  2636    tmp             python=2636

  Totals grouped by language (dominant language first):
  python:        2636 (100.00%)

So it looks like 2636 lines of Python versus 21000-ish
lines of C.

Mark

From sturla at molden.no  Wed Oct 21 17:10:54 2009
From: sturla at molden.no (Sturla Molden)
Date: Wed, 21 Oct 2009 17:10:54 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <1256132087.5069.39.camel@localhost>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost>
Message-ID: <4ADF247E.8040401@molden.no>

Antoine Pitrou skrev:
>>> (*) http://svn.python.org/view/sandbox/trunk/ccbench/
>>>       
>> I?ve run it twice on my dual core machine.  It hangs every time, but not in the same place:
>> D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py
>>     
>
> Ah, you should report a bug then. ccbench is pure Python (and not
> particularly evil Python), it shouldn't be able to crash the
> interpreter.
>
>   
It does not crash the interpreter, but it seems it can deadlock.

Here is what I get con a quadcore (Vista, Python 2.6.3).




D:\>ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 568 iterations/s.
threads=2: 253 ( 44 %)
threads=3: 274 ( 48 %)
threads=4: 283 ( 49 %)

regular expression (C)

threads=1: 510 iterations/s.
threads=2: 508 ( 99 %)
threads=3: 503 ( 98 %)
threads=4: 502 ( 98 %)

bz2 compression (C)

threads=1: 456 iterations/s.
threads=2: 892 ( 195 %)
threads=3: 1320 ( 289 %)
threads=4: 1743 ( 382 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 37 ms. (std dev: 21 ms.)
CPU threads=2: 379 ms. (std dev: 175 ms.)
CPU threads=3: 625 ms. (std dev: 310 ms.)
CPU threads=4: 718 ms. (std dev: 381 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 1 ms. (std dev: 3 ms.)




D:\>ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 554 iterations/s.
threads=2: 400 ( 72 %)
threads=3: 273 ( 49 %)
threads=4: 231 ( 41 %)

regular expression (C)

threads=1: 508 iterations/s.
threads=2: 509 ( 100 %)
threads=3: 509 ( 100 %)
threads=4: 509 ( 100 %)

bz2 compression (C)

threads=1: 456 iterations/s.
threads=2: 897 ( 196 %)
threads=3: 1316 ( 288 %)

DEADLOCK





D:\>ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 559 iterations/s.
threads=2: 397 ( 71 %)
threads=3: 274 ( 49 %)
threads=4: 238 ( 42 %)

regular expression (C)

threads=1: 507 iterations/s.
threads=2: 499 ( 98 %)
threads=3: 505 ( 99 %)
threads=4: 495 ( 97 %)

bz2 compression (C)

threads=1: 455 iterations/s.
threads=2: 896 ( 196 %)
threads=3: 1320 ( 290 %)
threads=4: 1736 ( 381 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 34 ms. (std dev: 21 ms.)
CPU threads=2: 358 ms. (std dev: 174 ms.)
CPU threads=3: 619 ms. (std dev: 312 ms.)
CPU threads=4: 742 ms. (std dev: 382 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 6 ms. (std dev: 13 ms.)










From sturla at molden.no  Wed Oct 21 17:38:13 2009
From: sturla at molden.no (Sturla Molden)
Date: Wed, 21 Oct 2009 17:38:13 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <4ADF247E.8040401@molden.no>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
Message-ID: <4ADF2AE5.5040100@molden.no>

Sturla Molden skrev:
> does not crash the interpreter, but it seems it can deadlock.
>
> Here is what I get con a quadcore (Vista, Python 2.6.3).
>

This what I get with affinity set to CPU  3.

There are deadlocks happening at random locations in ccbench.py. It gets 
worse with affinity set to one processor.

Sturla






D:\>start /AFFINITY 3 /B ccbench.py

D:\>--- Throughput ---

Pi calculation (Python)

threads=1: 554 iterations/s.
threads=2: 257 ( 46 %)
threads=3: 272 ( 49 %)
threads=4: 280 ( 50 %)

regular expression (C)

threads=1: 501 iterations/s.
threads=2: 505 ( 100 %)
threads=3: 493 ( 98 %)
threads=4: 507 ( 101 %)

bz2 compression (C)

threads=1: 455 iterations/s.
threads=2: 889 ( 195 %)
threads=3: 1309 ( 287 %)
threads=4: 1710 ( 375 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 40 ms. (std dev: 22 ms.)
CPU threads=2: 384 ms. (std dev: 179 ms.)
CPU threads=3: 618 ms. (std dev: 314 ms.)
CPU threads=4: 713 ms. (std dev: 379 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 3 ms.)
CPU threads=4: 0 ms. (std dev: 1 ms.)


From solipsis at pitrou.net  Wed Oct 21 19:21:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 17:21:08 +0000 (UTC)
Subject: [Python-Dev] GIL behaviour under Windows
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
Message-ID: <loom.20091021T191933-390@post.gmane.org>


Sturla Molden <sturla <at> molden.no> writes:
> 
> It does not crash the interpreter, but it seems it can deadlock.

Kristj?n sent me a patch which I applied and is supposed to fix this.
Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero
latency with the Pi calculation in the background) than under Linux, although it
may be caused by the limited resolution of time.time() under Windows.

Regards

Antoine.



From g.brandl at gmx.net  Wed Oct 21 19:21:07 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 21 Oct 2009 19:21:07 +0200
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
Message-ID: <hbng32$3ih$2@ger.gmane.org>

Mark Dickinson schrieb:

> * There's a partially complete rewrite of decimal in C in the sandbox,
> dating from the Need for Speed sprint in 2006:
> 
> http://svn.python.org/view/sandbox/trunk/decimal-c/
> 
> Last time I looked at this it wasn't up to date with the decimal
> specification:  I'm not sure that functions like exp, log and log10
> are currently working.  Georg Brandl might know better than I do.

I haven't touched that code since the sprint, but a student named
Mateusz Rukowicz worked on it for a past Summer of Code, I think.
I never heard of the outcome of that particular project.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From john.arbash.meinel at gmail.com  Wed Oct 21 19:42:51 2009
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Wed, 21 Oct 2009 12:42:51 -0500
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091021T191933-390@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>
Message-ID: <4ADF481B.90207@gmail.com>

Antoine Pitrou wrote:
> Sturla Molden <sturla <at> molden.no> writes:
>> It does not crash the interpreter, but it seems it can deadlock.
> 
> Kristj?n sent me a patch which I applied and is supposed to fix this.
> Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero
> latency with the Pi calculation in the background) than under Linux, although it
> may be caused by the limited resolution of time.time() under Windows.
> 
> Regards
> 
> Antoine.

You can use time.clock() instead to get <15ms resolution. Changing all
instances of 'time.time' to 'time.clock' gives me this result:

(2-core machine, python 2.6.2)

$ py ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 675 iterations/s.
threads=2: 388 ( 57 %)
threads=3: 374 ( 55 %)
threads=4: 445 ( 65 %)

regular expression (C)

threads=1: 588 iterations/s.
threads=2: 519 ( 88 %)
threads=3: 511 ( 86 %)
threads=4: 513 ( 87 %)

bz2 compression (C)

threads=1: 536 iterations/s.
threads=2: 949 ( 176 %)
threads=3: 900 ( 167 %)
threads=4: 927 ( 172 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 24727 ms. (std dev: 0 ms.)
CPU threads=1: 27930 ms. (std dev: 0 ms.)
CPU threads=2: 31029 ms. (std dev: 0 ms.)
CPU threads=3: 34170 ms. (std dev: 0 ms.)
CPU threads=4: 37292 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 40454 ms. (std dev: 0 ms.)
CPU threads=1: 43674 ms. (std dev: 21 ms.)
CPU threads=2: 47100 ms. (std dev: 165 ms.)
CPU threads=3: 50441 ms. (std dev: 304 ms.)
CPU threads=4: 53707 ms. (std dev: 377 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 56138 ms. (std dev: 0 ms.)
CPU threads=1: 59332 ms. (std dev: 0 ms.)
CPU threads=2: 62436 ms. (std dev: 0 ms.)
CPU threads=3: 66130 ms. (std dev: 0 ms.)
CPU threads=4: 69859 ms. (std dev: 0 ms.)

From solipsis at pitrou.net  Wed Oct 21 19:55:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 19:55:26 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <4ADF481B.90207@gmail.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>  <4ADF481B.90207@gmail.com>
Message-ID: <1256147726.5069.50.camel@localhost>

Le mercredi 21 octobre 2009 ? 12:42 -0500, John Arbash Meinel a ?crit :
> 
> You can use time.clock() instead to get <15ms resolution. Changing all
> instances of 'time.time' to 'time.clock' gives me this result:
[snip]
> 
> --- Latency ---
> 
> Background CPU task: Pi calculation (Python)
> 
> CPU threads=0: 24727 ms. (std dev: 0 ms.)
> CPU threads=1: 27930 ms. (std dev: 0 ms.)
> CPU threads=2: 31029 ms. (std dev: 0 ms.)
> CPU threads=3: 34170 ms. (std dev: 0 ms.)
> CPU threads=4: 37292 ms. (std dev: 0 ms.)

Well apparently time.clock() has a per-process time reference, which
makes it unusable for this benchmark :-(
(the numbers above are obviously incorrect)

Regards

Antoine.



From john at arbash-meinel.com  Wed Oct 21 20:00:54 2009
From: john at arbash-meinel.com (John Arbash Meinel)
Date: Wed, 21 Oct 2009 13:00:54 -0500
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <1256147726.5069.50.camel@localhost>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com> <1256147726.5069.50.camel@localhost>
Message-ID: <4ADF4C56.3080607@arbash-meinel.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antoine Pitrou wrote:
> Le mercredi 21 octobre 2009 ? 12:42 -0500, John Arbash Meinel a ?crit :
>> You can use time.clock() instead to get <15ms resolution. Changing all
>> instances of 'time.time' to 'time.clock' gives me this result:
> [snip]
>> --- Latency ---
>>
>> Background CPU task: Pi calculation (Python)
>>
>> CPU threads=0: 24727 ms. (std dev: 0 ms.)
>> CPU threads=1: 27930 ms. (std dev: 0 ms.)
>> CPU threads=2: 31029 ms. (std dev: 0 ms.)
>> CPU threads=3: 34170 ms. (std dev: 0 ms.)
>> CPU threads=4: 37292 ms. (std dev: 0 ms.)
> 
> Well apparently time.clock() has a per-process time reference, which
> makes it unusable for this benchmark :-(
> (the numbers above are obviously incorrect)
> 
> Regards
> 
> Antoine.

I believe that 'time.count()' is measured as seconds since the start of
the process. So yeah, I think spawning a background process will reset
this counter back to 0.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrfTFYACgkQJdeBCYSNAAObWQCfRJsRENbcp6kuo2x1k+HvhYGZ
ftsAn2PNnNHNj6D4esNBMhlSdH4IjeMA
=1KWG
-----END PGP SIGNATURE-----

From guido at python.org  Wed Oct 21 20:31:48 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Oct 2009 11:31:48 -0700
Subject: [Python-Dev] Proposal: Moratorium on Python language changes
Message-ID: <ca471dc20910211131i66ae5123m4eae4d5e0b83ce3e@mail.gmail.com>

In the python-ideas list I've proposed a moratorium on language
changes. It seems to be gaining momentum; if you want to have a say,
come over. You can watch the discussion in the archives starting here:
http://mail.python.org/pipermail/python-ideas/2009-October/006305.html
. (Eventually I'll move the proposal over to python-dev, but I thought
it only fair to give early warning to those who don't read
python-ideas.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From kristjan at ccpgames.com  Wed Oct 21 22:33:11 2009
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Wed, 21 Oct 2009 20:33:11 +0000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <1256147726.5069.50.camel@localhost>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>  <4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>

You are right, on windows time.clock() is based relative to its first call in the process.  There is no such promise made on unix.
QueryPerformanceCounter() (what time.clock uses()) is a robust high resolution timer that is processor/core independent.  It should be possible to use it across different processes too, if this annoying rebasing wasn't there.

I wonder if we should consider this a bug?

If so, I see three remedies:
1) simply using the absolute value and stop creating this arbitrary zero point.  this should be ok since the same is done on unix, but it would be a break from the documented behavior.  Never the less, the absolute value of this timer is irrelevant, it is the deltas that matter.
2) Add a flag to time.clock() for it to return absolute value
3) Create yet another api, either something like time.rclock() returning the absolute value or something like time.clockbase() returning the base of the zeroed clock timer.

If you just want to patch locally for your timing pleasure, change line 184 of timemodule.c to:
diff = (double)(now.QuadPart);

K

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Antoine Pitrou
>
 ---
> >
> > Background CPU task: Pi calculation (Python)
> >
> > CPU threads=0: 24727 ms. (std dev: 0 ms.)
> > CPU threads=1: 27930 ms. (std dev: 0 ms.)
> > CPU threads=2: 31029 ms. (std dev: 0 ms.)
> > CPU threads=3: 34170 ms. (std dev: 0 ms.)
> > CPU threads=4: 37292 ms. (std dev: 0 ms.)
> 
> Well apparently time.clock() has a per-process time reference, which
> makes it unusable for this benchmark :-(
> (the numbers above are obviously incorrect)
> 


From solipsis at pitrou.net  Wed Oct 21 22:51:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Oct 2009 20:51:03 +0000 (UTC)
Subject: [Python-Dev] time.clock() on windows
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com> <1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
Message-ID: <loom.20091021T225002-378@post.gmane.org>

Kristj?n Valur J?nsson <kristjan <at> ccpgames.com> writes:
> 
> You are right, on windows time.clock() is based relative to its first call in
the process.  There is no such
> promise made on unix.
> QueryPerformanceCounter() (what time.clock uses()) is a robust high resolution
timer that is
> processor/core independent.  It should be possible to use it across different
processes too, if this
> annoying rebasing wasn't there.

Well, could we simply have a high-resolution time.time()?
Or is Windows just too limited to provide this?

Regards

Antoine.



From curt at hagenlocher.org  Wed Oct 21 23:31:13 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Wed, 21 Oct 2009 14:31:13 -0700
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <loom.20091021T225002-378@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org> <4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
Message-ID: <d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>

On Wed, Oct 21, 2009 at 1:51 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Kristj?n Valur J?nsson <kristjan <at> ccpgames.com> writes:
>>
>> You are right, on windows time.clock() is based relative to its first call
>> in the process. ?There is no such promise made on unix.
>> QueryPerformanceCounter() (what time.clock uses()) is a robust high
>> resolution timer that is processor/core independent. ?It should be
>> possible to use it across different processes too, if this
>> annoying rebasing wasn't there.
>
> Well, could we simply have a high-resolution time.time()?
> Or is Windows just too limited to provide this?

Presumably you could fake something like this by combining output from
an initial time(), an initial QueryPerformanceCounter() and the
current QueryPerformanceCounter(). But it makes more sense to
understand why someone chose to implement time.clock() on Windows the
way they did -- this seems very broken to me, and I think it should be
changed.

Of course, there are no doubt people relying on the broken behavior...

--
Curt Hagenlocher
curt at hagenlocher.org

From scott+python-dev at scottdial.com  Wed Oct 21 23:52:26 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 21 Oct 2009 17:52:26 -0400
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
Message-ID: <4ADF829A.7060509@scottdial.com>

Curt Hagenlocher wrote:
> But it makes more sense to
> understand why someone chose to implement time.clock() on Windows the
> way they did -- this seems very broken to me, and I think it should be
> changed.

Some SVN detective work takes this to all the way back to r7713
(1997-04-02). The original implementation checked by Guido and
attributed to Mark Hammond. So, we should ask Mark why he did that.

Can anyone honestly use it, as it is, without already having normalize
it across platforms themselves?

I don't know how much of an impact it is, but the current implementation
of clock() does not require argument parsing, so the proposal to add a
"absolute" boolean-flag argument is perhaps bad. This is generally a
function used for performance timing and that proposal adds some amount
of latency to the query. The proposal to add a clockbase() function is
perhaps better because of this, you need only call it once, and you can
cache the result for the life of your process.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From zookog at gmail.com  Thu Oct 22 01:00:11 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Wed, 21 Oct 2009 17:00:11 -0600
Subject: [Python-Dev] Python 2.6.4rc2
In-Reply-To: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
Message-ID: <cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>

Barry:

Do you know anything about this alleged regression in 2.6.3 with
regard to the __doc__ property?

https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688

Regards,

Zooko

From kristjan at ccpgames.com  Thu Oct 22 01:05:58 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 21 Oct 2009 23:05:58 +0000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org> <4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF4213E@exchis.ccp.ad.local>



> Presumably you could fake something like this by combining output from
> an initial time(), an initial QueryPerformanceCounter() and the
> current QueryPerformanceCounter(). But it makes more sense to
> understand why someone chose to implement time.clock() on Windows the
> way they did -- this seems very broken to me, and I think it should be
> changed.

Yes.  The problem with QPC is that although it has very high resolution, it is not precise in the long term.  And GetSystemTimeAsFileTime() is high precision in the long term but only updated evey 20ms or so.
In EVE Online we use a combination of the two for high resolution, long term precision.  But I'm not happy with the way I'm doing it.  It needs some sort of smoothing of course.  I've even played with using Kalman filtering to do it...
The idea is to use the low frequency timer to apply correction coefficients to the high frequency timer, yet keep the flow of time smooth (no backwards jumps because of corrections.).  An optimal solution has so far eluded me.

Cheers,

K

From barry at python.org  Thu Oct 22 01:33:12 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Oct 2009 19:33:12 -0400
Subject: [Python-Dev] Python 2.6.4rc2
In-Reply-To: <cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
Message-ID: <1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>

On Oct 21, 2009, at 7:00 PM, Zooko O'Whielacronx wrote:

> Do you know anything about this alleged regression in 2.6.3 with
> regard to the __doc__ property?
>
> https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688

This is the first I've heard of it, but my guess is that it's caused  
by the fix for bug 5890.

http://bugs.python.org/issue5890

It's a change in Python 2.6.3, but I don't believe it's a regression  
(as in, currently broken).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091021/70de2fab/attachment.pgp>

From david.lyon at preisshare.net  Thu Oct 22 02:34:24 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 21 Oct 2009 20:34:24 -0400
Subject: [Python-Dev] Python Package Management Roadmap in Python Releases
In-Reply-To: <1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
Message-ID: <b1535e13caf35563d0b22a78d4b5882d@preisshare.net>


Hi All,

I started out some time ago and wrote a Python Package Manager
with wxpython. It was an interesting learning experience for
me. I was new to python.

Some have pointed out that using wx was not a good idea for
a tool to go into all the python distributions. Because it
required the external dependency of wx.

So, given that, I'm ready to have another go.

Some have suggested using Tk. But I noticed python Tk isn't
standard on ubuntu.

A console mode tool is possible. That would be better than
the nothing that we have under windows today.

Vote [1] - console mode tool for this

Alternatively, a web server based tool along the lines
of the old python docs system might be possible.

Vote [2] - web tool for this

I have come to understand that python-dev is full of
people that know the packages that they like, know how
to install them by hand, and probably only ever go
looking for upgrades.

However, for new users, it is so much harder.

It also seems to me that we have a lot of reasonable
and talented people, who are committed and dedicated.

To me, a relative outsider, it's not clear if this
is a python central issue, or a distutils issue. It
could be either. Advise me please.

By June next year, some people including myself, want
to go to europe for holidays - sorry conferences.. and
we want to take some work with us to talk about.

We can do the work..

But we need concessions...

What can we work on that might make a difference ?

If the current roadmap for distutils package management
on windows for the next 8 months is nothing, I can
live with that.

But seriously, lets get this discussion going again.

If a command line tool is all that windows users will
allowed to have, then it would be better than the
current plan which is for them to have nothing.

It's not just nothing for now, it's nothing for years.

That doesn't seem fair or nice. It's not just one
person saying that, it is a handful who don't use
windows on a regular basis. 

Why can't we have an advocate from each major 
platform? bring their issues, and try to come 
to a consensus.

Even if that tool would simply allow them to choose:
 - PIP
 - Distribute
 - Easy Install
 - Python Package Manager

>From there, users could explore each offer on it's
own merits.

I'm interested in getting all types of opinions and feedback. 

David





From andrew.svetlov at gmail.com  Thu Oct 22 02:44:04 2009
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Wed, 21 Oct 2009 20:44:04 -0400
Subject: [Python-Dev] PEP 384
Message-ID: <270ea8310910211744i250047cewc2d7b26a8be2c2f@mail.gmail.com>

Is there some public branch for this PEP?
I like the idea and glad to look on implementation.
Thanks.

From brett at python.org  Thu Oct 22 02:48:42 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Oct 2009 17:48:42 -0700
Subject: [Python-Dev] PEP 384
In-Reply-To: <270ea8310910211744i250047cewc2d7b26a8be2c2f@mail.gmail.com>
References: <270ea8310910211744i250047cewc2d7b26a8be2c2f@mail.gmail.com>
Message-ID: <bbaeab100910211748x4adb580exc808055af1247042@mail.gmail.com>

On Wed, Oct 21, 2009 at 17:44, Andrew Svetlov <andrew.svetlov at gmail.com>wrote:

> Is there some public branch for this PEP?
> I like the idea and glad to look on implementation.
> Thanks.
>

If there was such a thing the PEP would mention it. But since the PEP is
about not changing ABIs between releases I don't see why some branch would
be needed for it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091021/ffce1d38/attachment.htm>

From brett at python.org  Thu Oct 22 02:56:57 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Oct 2009 17:56:57 -0700
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
In-Reply-To: <b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org> 
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com> 
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
Message-ID: <bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>

On Wed, Oct 21, 2009 at 17:34, David Lyon <david.lyon at preisshare.net> wrote:

>
> Hi All,
>
> I started out some time ago and wrote a Python Package Manager
> with wxpython. It was an interesting learning experience for
> me. I was new to python.
>
> Some have pointed out that using wx was not a good idea for
> a tool to go into all the python distributions. Because it
> required the external dependency of wx.
>
> So, given that, I'm ready to have another go.
>
>
I honestly don't see what this has to do with python-dev or the
distutils-sig. If you want to write a package management tool that's great,
but I don't see why python-dev should have input on that sort of thing.
Asking what kind of GUI to use is a comp.lang.python question and not a
python-dev question as I don't see anything like this going into the stdlib.

If you want distutils to expose something to make it easier to write your
tool then that belong on the distutils-sig. But otherwise this seems
off-topic for python-dev.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091021/b379c62d/attachment.htm>

From david.lyon at preisshare.net  Thu Oct 22 03:17:59 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 21 Oct 2009 21:17:59 -0400
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
In-Reply-To: <bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>
Message-ID: <13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>

On Wed, 21 Oct 2009 17:56:57 -0700, Brett Cannon <brett at python.org> wrote:

> but I don't see why python-dev should have input on that sort of thing.

python-dev is the only place where we could get a change to the
installation
binary release. We'd need a change and the addition of a program
shortcut.

> If you want distutils to expose something to make it easier to write your
> tool then that belong on the distutils-sig. But otherwise this seems
> off-topic for python-dev.

I didn't ask for that - because I know that getting that assistance on the
distutils side is certainly possible.

Distutils is simply just one of the many libraries within python. It
doesn't
have an external interface.

The roadmap for distutils for windows doesn't include getting a shortcut
or utility for windows so that's why I'm asking about it here.

Surely logic says that if it's 'python' and 'development' and it's
not in distutils then some discussion of it should be allowed here.

What I am really talking about is the menu shortcuts in the cpython
distribution for windows. And how they can be improved to help
windows users. This is the only place that I can think to discuss
that.

Best Regards

David






From brett at python.org  Thu Oct 22 03:38:26 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Oct 2009 18:38:26 -0700
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
In-Reply-To: <13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org> 
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com> 
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net> 
	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com> 
	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>
Message-ID: <bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>

On Wed, Oct 21, 2009 at 18:17, David Lyon <david.lyon at preisshare.net> wrote:

> On Wed, 21 Oct 2009 17:56:57 -0700, Brett Cannon <brett at python.org> wrote:
>
> > but I don't see why python-dev should have input on that sort of thing.
>
> python-dev is the only place where we could get a change to the
> installation
> binary release. We'd need a change and the addition of a program
> shortcut.
>
>
But that assumes you can get your tool into the stdlib. It would have been
better to phrase the question as "is there interest in having a package
manager tool included with Python" rather than ask us out of the blue what
GUI you should use.


> > If you want distutils to expose something to make it easier to write your
> > tool then that belong on the distutils-sig. But otherwise this seems
> > off-topic for python-dev.
>
> I didn't ask for that - because I know that getting that assistance on the
> distutils side is certainly possible.
>
> Distutils is simply just one of the many libraries within python. It
> doesn't
> have an external interface.
>
>
And that's on purpose.


> The roadmap for distutils for windows doesn't include getting a shortcut
> or utility for windows so that's why I'm asking about it here.
>
> Surely logic says that if it's 'python' and 'development' and it's
> not in distutils then some discussion of it should be allowed here.
>
>
No it surely does not. We do not need to discuss design decisions of pip,
setuptools, virtualenv, buildout, or various other tools that involve the
terms "python" and "development" and are not in distutils.


> What I am really talking about is the menu shortcuts in the cpython
> distribution for windows. And how they can be improved to help
> windows users. This is the only place that I can think to discuss
> that.
>

David, you are making a huge leap here thinking that we even want a package
manager in the stdlib. You did not ask about menu shortcuts but whether a
package manager should be written using Tk or a web front-end. Then you
start discussing about wanting to add some UI to package management by
default on Windows or add some tool that sounds like what the EU is going to
have MS stick in front of users to get new user by browsers. This extends
beyond adding some shortcut the Windows installer adds to someone's machine.

I realize you are trying to help, David, but you are going in the wrong
direction here and pushing rather hard. At the language summit we discussed
opening up some APIs in distutils about making it easier for people to write
package management tools, but we don't have a burning desire to get into the
tool business. We make a language here. Distutils exists as a bootstrap
mechanism for the package story and for our own building needs of the
stdlib. But I doubt I am the only core developer who has no interest to be
in charge of a package management tool when there already exists several
good ones out there that people seem to find on their own without issue.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091021/8978edbe/attachment.htm>

From mbk.lists at gmail.com  Thu Oct 22 03:56:51 2009
From: mbk.lists at gmail.com (Mike Krell)
Date: Wed, 21 Oct 2009 18:56:51 -0700
Subject: [Python-Dev] nonlocal keyword in 2.x?
Message-ID: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>

Is there any possibility of backporting support for the nonlocal keyword
into a  2.x release?  I see it's not in 2.6, but I don't know if that was an
intentional design choice or due to a lack of demand / round tuits.  I'm
also not sure if this would fall under the scope of the proposed moratorium
on new language features (although my first impression was that it could be
allowed since it already exists in python 3.

One of my motivations for asking is a recent blog post by Fernando Perez of
IPython fame that describes an interesting decorator-based idiom inspired by
Apple's Grand Central Dispatch which would allow many interesting
possibilities for expressing parallelization and other manipulations of
execution context for blocks of python code.  Unfortunately, using the
technique to its fullest extent requires the nonlocal keyword.

The blog post is here:
https://cirl.berkeley.edu/fperez/py4science/decorators.html

   Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091021/6c360fdb/attachment.htm>

From david.lyon at preisshare.net  Thu Oct 22 04:13:09 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 21 Oct 2009 22:13:09 -0400
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
In-Reply-To: <bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>
	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>
	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
Message-ID: <0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>

On Wed, 21 Oct 2009 18:38:26 -0700, Brett Cannon <brett at python.org> wrote:
> But that assumes you can get your tool into the stdlib. 

No I'm not assuming that I can. I am actually assuming that I cannot..

So lets move forward..

> It would have been
> better to phrase the question as "is there interest in having a package
> manager tool included with Python" rather than ask us out of the blue
what
> GUI you should use.

ok - but I think I know the answer to that.. you answer it next.

> David, you are making a huge leap here thinking that we even want a
package
> manager in the stdlib. 

Well - who is 'we'? If it's python-dev people I can accept and respect
that.

If it's regular developers out there in developer land, I'm not so sure
about your assertion. I'd even politely have to disagree from my
experience.

> You did not ask about menu shortcuts but whether a
> package manager should be written using Tk or a web front-end. 

I was thinking about the issue on the fly...

Menu shortcuts that link off to a a standard community web page
would be an excellent compromise - in the case where some tool
could not be added.

That would be a tremendous improvement for windows users over
what they are given at the moment.

> Then you
> start discussing about wanting to add some UI to package management by
> default on Windows or add some tool that sounds like what the EU is going
> to
> have MS stick in front of users to get new user by browsers. This extends
> beyond adding some shortcut the Windows installer adds to someone's
> machine.

That's going further than what I'm saying..

> I realize you are trying to help, David, but you are going in the wrong
> direction here and pushing rather hard. 

On the counter side, others are pushing rather hard for 0 improvement
for the windows platform for the user experience. While everything else
on windows rushes ahead..

My direction is improving the developer experience for windows users. I
can't do compiler writing. I'm not clever enough.

> At the language summit we discussed
> opening up some APIs in distutils about making it easier for people to
> write
> package management tools, but we don't have a burning desire to get into
> the
> tool business. 

ok - but nothing happened there...

I'm not in the tools business either. I'm not doing it for money but
that shouldn't be the point.

> We make a language here. Distutils exists as a bootstrap
> mechanism for the package story and for our own building needs of the
> stdlib. 

I accept that it's a tool for building stdlib. No debate.

> But I doubt I am the only core developer who has no interest to be
> in charge of a package management tool when there already exists several
> good ones out there that people seem to find on their own without issue.

umm.. I disagree with the 'without issue' statement. I'm sure if I tralled
the mailing lists I could find more than one.. 

Enough from me for now. Thanks Brett.

David



From sturla at molden.no  Thu Oct 22 04:56:40 2009
From: sturla at molden.no (Sturla Molden)
Date: Thu, 22 Oct 2009 04:56:40 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091021T191933-390@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>
Message-ID: <4ADFC9E8.1010400@molden.no>

Antoine Pitrou skrev:
> Kristj?n sent me a patch which I applied and is supposed to fix this.
> Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero
> latency with the Pi calculation in the background) than under Linux, although it
> may be caused by the limited resolution of time.time() under Windows.
>
>   

My critisism of the GIL on python-ideas was partly motivated by this:

http://blip.tv/file/2232410

However, David Beazley is not talking about Windows. Since the GIL is 
apparently not a mutex on Windows, it could behave differently. So I 
wrote a small script that contructs a GIL battle, and record how often a 
check-interval results in a thread-switch or not. For monitoring check 
intervals, I used a small C extension to read _Py_Ticker from ceval.c. 
It is not declared static so I could easily hack into it.

With two threads and a check interval og 100, only 61 of 100000 check 
intervals failed to produce a thread-switch in the interpreter. I'd call 
that rather fair. :-)

And in case someone asks, the nthreads=1 case is just for verification.

S.M.




D:\>test.py
check interval = 1
nthreads=1, swiched=0, missed=100000
nthreads=2, swiched=57809, missed=42191
nthreads=3, swiched=91535, missed=8465
nthreads=4, swiched=99751, missed=249
nthreads=5, swiched=95839, missed=4161
nthreads=6, swiched=100000, missed=0

D:\>test.py
check interval = 10
nthreads=1, swiched=0, missed=100000
nthreads=2, swiched=99858, missed=142
nthreads=3, swiched=99992, missed=8
nthreads=4, swiched=100000, missed=0
nthreads=5, swiched=100000, missed=0
nthreads=6, swiched=100000, missed=0

D:\>test.py
check interval = 100
nthreads=1, swiched=0, missed=100000
nthreads=2, swiched=99939, missed=61
nthreads=3, swiched=100000, missed=0
nthreads=4, swiched=100000, missed=0
nthreads=5, swiched=100000, missed=0
nthreads=6, swiched=100000, missed=0

D:\>test.py
check interval = 1000
nthreads=1, swiched=0, missed=100000
nthreads=2, swiched=99999, missed=1
nthreads=3, swiched=100000, missed=0
nthreads=4, swiched=100000, missed=0
nthreads=5, swiched=100000, missed=0
nthreads=6, swiched=100000, missed=0

From sturla at molden.no  Thu Oct 22 05:09:44 2009
From: sturla at molden.no (Sturla Molden)
Date: Thu, 22 Oct 2009 05:09:44 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <4ADFC9E8.1010400@molden.no>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADFC9E8.1010400@molden.no>
Message-ID: <4ADFCCF8.5010807@molden.no>

Sturla Molden skrev:
>
> However, David Beazley is not talking about Windows. Since the GIL is 
> apparently not a mutex on Windows, it could behave differently. So I 
> wrote a small script that contructs a GIL battle, and record how often 
> a check-interval results in a thread-switch or not. For monitoring 
> check intervals, I used a small C extension to read _Py_Ticker from 
> ceval.c. It is not declared static so I could easily hack into it.

Anyway, if anyone wants to run a GIL battle, here is the code I used.

If it turns out the GIL is far worse with pthreads, as it is implemented 
with a mutex, it might be a good idea to reimplement it with an event 
object as it is on Windows.

Sturla Molden

----------------

In python:

from giltest import *
from time import clock
import threading
import sys


def thread(rank, battle, start):

    while not start.isSet():
        if rank == 0:
            start.set()
 
    try:
        while 1:
            battle.record(rank)
    except:
        pass   


if __name__ == '__main__':

    sys.setcheckinterval(1000)

    print "check interval = %d" % sys.getcheckinterval()

    for nthreads  in range(1,7):
   
        start = threading.Event()
        battle = GIL_Battle(100000)
        threads = [threading.Thread(target=thread, args=(i,battle,start))
                    for i in range(1,nthreads)]
        for t in threads:
            t.setDaemon(True)
            t.start()

        thread(0, battle, start)
        for t in threads: t.join()
                                       
        s,m = battle.report()
       
        print "nthreads=%d, swiched=%d, missed=%d" % (nthreads, s, m)


In Cython or Pyrex:


from exceptions import Exception

cdef extern from *:
    ctypedef int vint "volatile int"
    vint _Py_Ticker

class StopBattle(Exception):
    pass

cdef class GIL_Battle:

    """ tests the fairness of the GIL """

    cdef vint prev_tick, prev_rank, switched, missed
    cdef int trials
   
    def __cinit__(GIL_Battle self, int trials=100000):
        self.prev_tick = _Py_Ticker
        self.prev_rank = -1
        self.missed = 0
        self.switched = 0
        self.trials = trials

    def record(GIL_Battle self, int rank):
        if self.trials == self.switched + self.missed:
            raise StopBattle   
        if self.prev_rank == -1:
            self.prev_tick = _Py_Ticker
            self.prev_rank = rank
        else:
            if _Py_Ticker > self.prev_tick:
                if self.prev_rank == rank:
                    self.missed += 1
                else:
                    self.switched += 1
                self.prev_tick = _Py_Ticker
                self.prev_rank = rank
            else:
                self.prev_tick = _Py_Ticker
               
    def report(GIL_Battle self):
        return int(self.switched), int(self.missed)





 



From skippy.hammond at gmail.com  Thu Oct 22 06:21:13 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 22 Oct 2009 15:21:13 +1100
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <4ADF829A.7060509@scottdial.com>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4ADF829A.7060509@scottdial.com>
Message-ID: <4ADFDDB9.3080707@gmail.com>

On 22/10/2009 8:52 AM, Scott Dial wrote:
> Curt Hagenlocher wrote:
>> But it makes more sense to
>> understand why someone chose to implement time.clock() on Windows the
>> way they did -- this seems very broken to me, and I think it should be
>> changed.
>
> Some SVN detective work takes this to all the way back to r7713
> (1997-04-02). The original implementation checked by Guido and
> attributed to Mark Hammond. So, we should ask Mark why he did that.

The thread seems to be at 
http://groups.google.com/group/comp.lang.python/browse_frm/thread/be32478a4b8e77b6/816d6228119a3474 
(although I do seem to recall more discussion of the patch which I 
currently can't find). I'd be very surprised if any applications rely on 
the fact that each process starts counting at zero, so if someone can 
come up with a high-res counter which avoids that artifact I'd expect it 
could be used.

Cheers,

Mark

From nnorwitz at gmail.com  Thu Oct 22 06:46:54 2009
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 21 Oct 2009 21:46:54 -0700
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
Message-ID: <ee2a432c0910212146x413f32f3mbfe3b7c8ff70f337@mail.gmail.com>

On Wed, Oct 21, 2009 at 6:56 PM, Mike Krell <mbk.lists at gmail.com> wrote:
> Is there any possibility of backporting support for the nonlocal keyword
> into a? 2.x release?? I see it's not in 2.6, but I don't know if that was an
> intentional design choice or due to a lack of demand / round tuits.? I'm
> also not sure if this would fall under the scope of the proposed moratorium
> on new language features (although my first impression was that it could be
> allowed since it already exists in python 3.

IIRC, Jeremy Hylton tried to backport it during a Pycon sprint a few
years back.  I think it was difficult and he dropped it.  I don't know
if there were any showstoppers or if Jeremy saved his work...or if my
memory is even correct. :-)

n

From robertc at robertcollins.net  Thu Oct 22 06:45:34 2009
From: robertc at robertcollins.net (Robert Collins)
Date: Thu, 22 Oct 2009 15:45:34 +1100
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <4ADFDDB9.3080707@gmail.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4ADF829A.7060509@scottdial.com>  <4ADFDDB9.3080707@gmail.com>
Message-ID: <1256186734.2929.37.camel@lifeless-64>

On Thu, 2009-10-22 at 15:21 +1100, Mark Hammond wrote:

>  I'd be very surprised if any applications rely on 
> the fact that each process starts counting at zero, so if someone can 
> come up with a high-res counter which avoids that artifact I'd expect it 
> could be used.

Could you offset it by the system time on the first call?

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/619f2ac4/attachment.pgp>

From v+python at g.nevcal.com  Thu Oct 22 07:35:49 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 21 Oct 2009 22:35:49 -0700
Subject: [Python-Dev] Python Package Management Roadmap in
	Python	Releases
In-Reply-To: <0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
	<0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
Message-ID: <4ADFEF35.6000409@g.nevcal.com>

On approximately 10/21/2009 7:13 PM, came the following characters from 
the keyboard of David Lyon:
> On Wed, 21 Oct 2009 18:38:26 -0700, Brett Cannon <brett at python.org> wrote:

>> We make a language here. Distutils exists as a bootstrap
>> mechanism for the package story and for our own building needs of the
>> stdlib. 


Maybe what David is missing is that since python-dev is uninterested in 
the package management issue, the only remaining way to include package 
management in a convenient, single installation, is to

1) Create the package management tool
2) Create a bundled installer that will install both Python, the package 
management tool, and any dependencies of the package management tool
3) Advertise the availability of the bundle, and its usefulness*
4) See how many users try it out, and as new versions of Python are 
created, how many users keep coming back, in preference to the package 
management tool free python.org distribution.

*There does seem to be precedent for mentioning/linking to other 
distributions of Python from http://www.python.org/download/ so that 
would surely be an avenue that is open to David, although I have no idea 
what criteria must be met to obtain a listing on that page.


-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From stephen at xemacs.org  Thu Oct 22 08:54:39 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 22 Oct 2009 15:54:39 +0900
Subject: [Python-Dev] Python Package Management Roadmap in
	Python	Releases
In-Reply-To: <0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>
	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>
	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
	<0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
Message-ID: <87ljj3dikw.fsf@uwakimon.sk.tsukuba.ac.jp>

David Lyon writes:
 > On Wed, 21 Oct 2009 18:38:26 -0700, Brett Cannon <brett at python.org> wrote:

 > > David, you are making a huge leap here thinking that we even want
 > > a package manager in the stdlib.
 > 
 > Well - who is 'we'? If it's python-dev people I can accept and
 > respect that.

Yes.  The stdlib is a product of python-dev.

 > If it's regular developers out there in developer land,

They are not participants in this decision.  python-dev people *do*
care about them.  Nevertheless python-dev is manifestly uninterested
in dealing with *this* issue on behalf of the "regular developers out
there".  (With good reasons IMHO, but you can find those reasons
elsewhere.)

To make progress here on python-dev, you need to get those regular
developers involved in python-dev.  I can think of two ways to do this
that will help.  (1) Turn them into committers via whatever aspect of
the language they are interested in.  This will take a while, and
substantial contribution (ie, effort) on their part.  Once that
happens, they will be in a much stronger position to advocate the
package manager and credibly promise to maintain it (or find a new,
credible maintainer) indefinitely.

(2) Put the package manager on PyPI and get a lot of people on the
bandwagon.  If it's a good product and recommended, the movers and
shakers of python-dev will be very likely to try it and more likely to
second your motion in the future.


From solipsis at pitrou.net  Thu Oct 22 11:09:34 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 09:09:34 +0000 (UTC)
Subject: [Python-Dev] time.clock() on windows
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4ADF829A.7060509@scottdial.com> <4ADFDDB9.3080707@gmail.com>
	<1256186734.2929.37.camel@lifeless-64>
Message-ID: <loom.20091022T110342-476@post.gmane.org>

Robert Collins <robertc <at> robertcollins.net> writes:
> 
> Could you offset it by the system time on the first call?

Two problems:
- the two measurements are not done simultaneously, so the result *still* does
not guarantee you have the same time reference in all processes (but gives you
the illusion you do, which is perhaps worse)
- adding a precise measure to an imprecise measure doesn't make the result
precise, but imprecise (or, rather, precise but inexact); in other words, if the
system time only gives a 0.01 second resolution, adding a high-resolution timer
only gives you an illusion of accuracy

Therefore it seems a very bad solution. The only way, AFAICT, to do this right
is for Windows to provide a high-resolution system time. It sounds astonishing
that this doesn't exist.

Regards

Antoine.



From solipsis at pitrou.net  Thu Oct 22 11:20:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 09:20:32 +0000 (UTC)
Subject: [Python-Dev] GIL behaviour under Windows
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>
	<4ADFC9E8.1010400@molden.no>
Message-ID: <loom.20091022T111452-527@post.gmane.org>

Sturla Molden <sturla <at> molden.no> writes:
> 
> With two threads and a check interval og 100, only 61 of 100000 check 
> intervals failed to produce a thread-switch in the interpreter. I'd call 
> that rather fair. 

This number lacks the elapsed time. 61 switches in one second is probably
enough, the same amount of switches in 10 or 20 seconds is too small (at least
for threads needing good responsivity, e.g. I/O threads).

Also, "fair" has to take into account the average latency and its relative
stability, which is why I wrote ccbench.


Antoine.



From kristjan at ccpgames.com  Thu Oct 22 11:47:50 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 22 Oct 2009 09:47:50 +0000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <4ADFDDB9.3080707@gmail.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4ADF829A.7060509@scottdial.com> <4ADFDDB9.3080707@gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF42178@exchis.ccp.ad.local>



> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Mark Hammond
> The thread seems to be at
> http://groups.google.com/group/comp.lang.python/browse_frm/thread/be324
> 78a4b8e77b6/816d6228119a3474
> (although I do seem to recall more discussion of the patch which I
> currently can't find). I'd be very surprised if any applications rely
> on
> the fact that each process starts counting at zero, so if someone can
> come up with a high-res counter which avoids that artifact I'd expect
> it
> could be used.

The point in question seems to be this this (from the thread):
 * Need some sort of static "start value", which is set when the 
 process starts, so I can return to Python in seconds.  An easy hack 
 is to set this the first time clock() is called, but then it wont 
 reflect any sort of real time - but would be useful for relative 
 times...

But the argumentation is flawed.   There is an implicit "start" value (technically, CPU powerup).  The point concedes that no sort of real time is returned, and so the particular "start" time chosen is immaterial.

K

From kristjan at ccpgames.com  Thu Oct 22 11:51:53 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 22 Oct 2009 09:51:53 +0000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <1256186734.2929.37.camel@lifeless-64>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4ADF829A.7060509@scottdial.com>  <4ADFDDB9.3080707@gmail.com>
	<1256186734.2929.37.camel@lifeless-64>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF4217A@exchis.ccp.ad.local>


> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Robert Collins
> >  I'd be very surprised if any applications rely on the fact that each
> > process starts counting at zero, so if someone can come up with a
> > high-res counter which avoids that artifact I'd expect it could be
> > used.
> 
> Could you offset it by the system time on the first call?
> 

Since system time has low granularity, it would negate our attemt at having time.clock() reflect the same time between processes.
In my opinion, the simplest way is to simply stop setting choosing the first call as a zero base, and use whatever arbitrary time the system has chosen for us.
The documentation could then read:
  "On Windows, this function returns wall-clock seconds elapsed since an
   arbitrary, system wited, epoch, as a floating point number, based on the
   Win32 function QueryPerformanceCounter. The resolution is typically better
   than one microsecond."

K


From phd at phd.pp.ru  Thu Oct 22 12:08:09 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Thu, 22 Oct 2009 14:08:09 +0400
Subject: [Python-Dev] Python Package Management Roadmap in
	Python	Releases
In-Reply-To: <b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
Message-ID: <20091022100809.GE13617@phd.pp.ru>

On Wed, Oct 21, 2009 at 08:34:24PM -0400, David Lyon wrote:
> I started out some time ago and wrote a Python Package Manager
> with wxpython.

   David, your message appeared in the middle of another thread. Please
don't use "Reply" button to start a new thread - send a new message.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From phd at phd.pp.ru  Thu Oct 22 12:12:06 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Thu, 22 Oct 2009 14:12:06 +0400
Subject: [Python-Dev] Python Package Management Roadmap in
	Python	Releases
In-Reply-To: <4ADFEF35.6000409@g.nevcal.com>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>
	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>
	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
	<0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
	<4ADFEF35.6000409@g.nevcal.com>
Message-ID: <20091022101206.GF13617@phd.pp.ru>

On Wed, Oct 21, 2009 at 10:35:49PM -0700, Glenn Linderman wrote:
> Maybe what David is missing is that since python-dev is uninterested in  
> the package management issue, the only remaining way to include package  
> management in a convenient, single installation, is to
>
> 1) Create the package management tool
> 2) Create a bundled installer that will install both Python, the package  
> management tool, and any dependencies of the package management tool
> 3) Advertise the availability of the bundle, and its usefulness*
> 4) See how many users try it out, and as new versions of Python are  
> created, how many users keep coming back, in preference to the package  
> management tool free python.org distribution.

   David, may be my memory betrays me, but I think you ask you question not
for the first but for the second, if not the third time. And every time you
got exactly this answer - "Create the package management tool and see how
many users try it out".

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Thu Oct 22 12:20:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 10:20:03 +0000 (UTC)
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
	<0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
	<4ADFEF35.6000409@g.nevcal.com>
Message-ID: <loom.20091022T121543-839@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:
> 
> Maybe what David is missing is that since python-dev is uninterested in 
> the package management issue, [...]

It's a bit strong to say we are uninterested. Most of us are not interested
enough to tackle it ourselves (*), but we are certainly interested in good
package management being available for Python.

(*) Remember, however, that Tarek and work on Distribute, and also on bringing
pieces of setuptools/Distribute functionality into distutils.


It's true, however, that it is probably not the place to ask for advice with
directions to follow for a yet unwritten piece of software. The only piece of
advice (besides "forget Tk and use a modern toolkit" :-)) I could give would be
to build on Distribute, rather than reinvent a whole bunch of mechanisms.

Regards

Antoine.



From mal at egenix.com  Thu Oct 22 12:57:32 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Oct 2009 12:57:32 +0200
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
Message-ID: <4AE03A9C.9020506@egenix.com>

Curt Hagenlocher wrote:
> On Wed, Oct 21, 2009 at 1:51 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> Kristj?n Valur J?nsson <kristjan <at> ccpgames.com> writes:
>>>
>>> You are right, on windows time.clock() is based relative to its first call
>>> in the process.  There is no such promise made on unix.
>>> QueryPerformanceCounter() (what time.clock uses()) is a robust high
>>> resolution timer that is processor/core independent.  It should be
>>> possible to use it across different processes too, if this
>>> annoying rebasing wasn't there.
>>
>> Well, could we simply have a high-resolution time.time()?
>> Or is Windows just too limited to provide this?
> 
> Presumably you could fake something like this by combining output from
> an initial time(), an initial QueryPerformanceCounter() and the
> current QueryPerformanceCounter(). But it makes more sense to
> understand why someone chose to implement time.clock() on Windows the
> way they did -- this seems very broken to me, and I think it should be
> changed.
> 
> Of course, there are no doubt people relying on the broken behavior...

I'm not sure I understand why time.clock() should be considered
broken.

time.clock() is used for measuring process CPU time and is usually
only used in a relative way, ie. you remember that start value,
do something, then subtract the end value from the start value.

For absolute CPU time values associated with a process, it's usually
better to rely on other APIs such as getrusage() on Unix.

You might want to have a look at the systimes module that
comes with pybench for some alternative timers:

	Tools/pybench/systimes.py

This module tries to provide a cross-platform API for
process and system time:

""" systimes() user and system timer implementations for use by
    pybench.

    This module implements various different strategies for measuring
    performance timings. It tries to choose the best available method
    based on the platforma and available tools.

    On Windows, it is recommended to have the Mark Hammond win32
    package installed. Alternatively, the Thomas Heller ctypes
    packages can also be used.

    On Unix systems, the standard resource module provides the highest
    resolution timings. Unfortunately, it is not available on all Unix
    platforms.

    If no supported timing methods based on process time can be found,
    the module reverts to the highest resolution wall-clock timer
    instead. The system time part will then always be 0.0.

    The module exports one public API:

    def systimes():

        Return the current timer values for measuring user and system
        time as tuple of seconds (user_time, system_time).

    Copyright (c) 2006, Marc-Andre Lemburg (mal at egenix.com). See the
    documentation for further information on copyrights, or contact
    the author. All Rights Reserved.

"""

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 22 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mhammond at skippinet.com.au  Thu Oct 22 12:57:53 2009
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 22 Oct 2009 21:57:53 +1100
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF42178@exchis.ccp.ad.local>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>	<4ADF829A.7060509@scottdial.com>
	<4ADFDDB9.3080707@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F098FF42178@exchis.ccp.ad.local>
Message-ID: <4AE03AB1.5000000@skippinet.com.au>

On 22/10/2009 8:47 PM, Kristj?n Valur J?nsson wrote:
> The point in question seems to be this this (from the thread):
>   * Need some sort of static "start value", which is set when the
>   process starts, so I can return to Python in seconds.  An easy hack
>   is to set this the first time clock() is called, but then it wont
>   reflect any sort of real time - but would be useful for relative
>   times...
>
> But the argumentation is flawed.

It was made in the context of the APIs available to implement this.  The 
code is short-and-sweet in timemodule.c, so please do go ahead and fix 
my flawed reasoning.

For reference:

#if defined(MS_WINDOWS) && !defined(__BORLANDC__)
/* Due to Mark Hammond and Tim Peters */
static PyObject *
time_clock(PyObject *self, PyObject *unused)
{
	static LARGE_INTEGER ctrStart;
	static double divisor = 0.0;
	LARGE_INTEGER now;
	double diff;

	if (divisor == 0.0) {
		LARGE_INTEGER freq;
		QueryPerformanceCounter(&ctrStart);
		if (!QueryPerformanceFrequency(&freq) || freq.QuadPart == 0) {
			/* Unlikely to happen - this works on all intel
			   machines at least!  Revert to clock() */
			return PyFloat_FromDouble(((double)clock()) /
						  CLOCKS_PER_SEC);
		}
		divisor = (double)freq.QuadPart;
	}
	QueryPerformanceCounter(&now);
	diff = (double)(now.QuadPart - ctrStart.QuadPart);
	return PyFloat_FromDouble(diff / divisor);
}


Cheers,

Mark.

From mhammond at skippinet.com.au  Thu Oct 22 13:04:16 2009
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 22 Oct 2009 22:04:16 +1100
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <1256186734.2929.37.camel@lifeless-64>
References: <4ADE2AA9.4030604@molden.no>	
	<loom.20091021T125010-360@post.gmane.org>	
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	
	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	
	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>	
	<1256147726.5069.50.camel@localhost>	
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	
	<loom.20091021T225002-378@post.gmane.org>	
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>	
	<4ADF829A.7060509@scottdial.com> <4ADFDDB9.3080707@gmail.com>
	<1256186734.2929.37.camel@lifeless-64>
Message-ID: <4AE03C30.6060500@skippinet.com.au>

On 22/10/2009 3:45 PM, Robert Collins wrote:
> On Thu, 2009-10-22 at 15:21 +1100, Mark Hammond wrote:
>
>>   I'd be very surprised if any applications rely on
>> the fact that each process starts counting at zero, so if someone can
>> come up with a high-res counter which avoids that artifact I'd expect it
>> could be used.
>
> Could you offset it by the system time on the first call?

Off the top of my head, I guess that depends on the actual accuracy 
required (ie, how many clock ticks elapse between querying the time and 
the high-resolution timer).  Starting at 0 works fine for profiling in a 
single process, the predominant use-case when this was done; I guess it 
depends on the specific requirements and time-intervals being dealt with 
in the cross-process case which determines how suitable that might be?

Cheers,

Mark

From solipsis at pitrou.net  Thu Oct 22 13:19:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 11:19:54 +0000 (UTC)
Subject: [Python-Dev] time.clock() on windows
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4AE03A9C.9020506@egenix.com>
Message-ID: <loom.20091022T131835-449@post.gmane.org>

M.-A. Lemburg <mal <at> egenix.com> writes:
> 
> I'm not sure I understand why time.clock() should be considered
> broken.

Because in some cases you want comparable high-resolution numbers from distinct
processes.

> time.clock() is used for measuring process CPU time

According to the docs, under Windows it measures wall-clock time rather than CPU
time.

Regards

Antoine.



From kristjan at ccpgames.com  Thu Oct 22 15:39:21 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 22 Oct 2009 13:39:21 +0000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <4AE03AB1.5000000@skippinet.com.au>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<4ADF829A.7060509@scottdial.com> <4ADFDDB9.3080707@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F098FF42178@exchis.ccp.ad.local>
	<4AE03AB1.5000000@skippinet.com.au>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF42268@exchis.ccp.ad.local>



> -----Original Message-----
> From: Mark Hammond [mailto:mhammond at skippinet.com.au]
> Sent: 22. okt?ber 2009 10:58
> To: Kristj?n Valur J?nsson
> Cc: Scott Dial; python-dev at python.org
> It was made in the context of the APIs available to implement this.
> The
> code is short-and-sweet in timemodule.c, so please do go ahead and fix
> my flawed reasoning.

...
I'm sorry, I don't want to start a flame war here, it just seems that if you need a zero point, and arbitrarily choose the first call to time.clock(), you could just as well use the implicit zero point already provided by the system.

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of M.-A. Lemburg
> I'm not sure I understand why time.clock() should be considered
> broken.

Ah, well, not broken, but it could be even more useful:  If it used the implicit system-wide epoch rather than the one based on the first call within each process, it could be useful for cross-process high-resolution timings.

Anyway, it is simple enough to patch it on windows using ctypes if one needs that kind of behaviour:

#nuclock.py
import ctypes
import time
counter = ctypes.c_uint64()
pcounter = ctypes.byref(counter)
ctypes.windll.kernel32.QueryPerformanceFrequency(pcounter)
frequency = float(counter.value)
QPC = ctypes.windll.kernel32.QueryPerformanceCounter
def nuclock():
    QPC(pcounter)
    return float(counter.value)/frequency

time.clock = nuclock


Cheers,

Kristjan	

From barry at python.org  Thu Oct 22 16:44:46 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 22 Oct 2009 10:44:46 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
Message-ID: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>

I'd like to get a second opinion on bug 7183:

http://bugs.python.org/issue7183

The Boost folks have reported this as a regression in 2.6.3, making it  
a candidate for Python 2.6.4.  IIUC, the latest version of Boost fixes  
the problem in their code, but if it really is a regression it could  
affect other projects and should probably be fixed.

Robert Collins from the Bazaar team pinged me on IRC and originally  
thought that they'd been bitten by this problem too, though he looked  
around and determined it probably did /not/ affect them after all.  So  
we have only anecdotal evidence of a problem, and no reproducible test  
case.

If the Python 2.6.4 release is to be held up for this issue, we need  
to know today or tomorrow.  Come the weekend, I'm going ahead with the  
tag and release.  Holding up the release will mean another release  
candidate, and a wait of another week for the final.

So does anybody else think bug 7183 should be a release blocker for  
2.6.4 final, or is even a legitimate but that we need to fix?

Thanks,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/77694045/attachment.pgp>

From benjamin at python.org  Thu Oct 22 16:47:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 22 Oct 2009 09:47:17 -0500
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
Message-ID: <1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>

2009/10/22 Barry Warsaw <barry at python.org>:
> So does anybody else think bug 7183 should be a release blocker for 2.6.4
> final, or is even a legitimate but that we need to fix?

I think it cannot hold up a release with out a reproducible code snippet.



-- 
Regards,
Benjamin

From stefan-usenet at bytereef.org  Thu Oct 22 11:10:34 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Thu, 22 Oct 2009 11:10:34 +0200
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <5c6f2a5d0910210828g5dab0aa0qac54815741734e8f@mail.gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
	<loom.20091021T170249-555@post.gmane.org>
	<5c6f2a5d0910210828g5dab0aa0qac54815741734e8f@mail.gmail.com>
Message-ID: <20091022091034.GA2812@mail.bytereef.org>

Mark Dickinson <dickinsm at gmail.com> wrote:
> Thanks, Antoine!  With SLOCCount I can revise my earlier numbers, as well:
> Here's Stefan Krah's mpdecimal, version 0.80:
> 
>   SLOC	Directory	SLOC-by-Language (Sorted)
>   21445   top_dir         ansic=21267,sh=105,python=55,asm=18
>   6238    python          python=6177,java=43,sh=18
>   1403    tests           ansic=1356,sh=47
>   476     literature      lisp=476
>   274     cmd             ansic=274
>   11      tools           sh=11
>   0       doc             (none)

I would say that the relevant code is less than that: The module code is
counted twice (fastdec2.c, fastdec3.c), and sloccount counts the inline
functions in the mpdecimal*.h header files. So, after removing fastdec3.c,
mpdecimal32.h, mpdecimal32vc.h, mpdecimal64vc.h, I get:

SLOC    Directory       SLOC-by-Language (Sorted)
13702   top_dir         ansic=13524,sh=105,python=55,asm=18
6238    python          python=6177,java=43,sh=18
1403    tests           ansic=1356,sh=47
476     literature      lisp=476
274     cmd             ansic=274
11      tools           sh=11
0       doc             (none)


Totals grouped by language (dominant language first):
ansic:        15154 (68.56%)
python:        6232 (28.19%)
lisp:           476 (2.15%)
sh:             181 (0.82%)
java:            43 (0.19%)
asm:             18 (0.08%)


Therefore, my estimate is between 12660 and 13702 lines, depending on whether
to count the remaining mpdecimal64.h (Most inline functions in this header
file are signaling wrappers around the quiet functions and are not necessary
for the module).

If one takes out all library functions that the module does not use, I'm
sure it could be condensed to ~11500 lines.


This is comparable to the expat directory in Modules/:

SLOC    Directory       SLOC-by-Language (Sorted)
11406   expat           ansic=11406


Stefan Krah




From stefan-usenet at bytereef.org  Thu Oct 22 16:50:17 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Thu, 22 Oct 2009 16:50:17 +0200
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
References: <20091020131547.GA615@mail.bytereef.org>
	<5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
Message-ID: <20091022145017.GB2812@mail.bytereef.org>

Mark Dickinson <dickinsm at gmail.com> wrote:
> > 2. Would fastdec - after achieving full decimal.py compatibility - be
> > ? a serious candidate?
> 
> Definitely.  As far as I know it's the only real candidate for a full
> C version of decimal right now.  Other possibilities that I'm aware of:
> 
> * I think Raymond Hettinger is working on a C version of decimal.
> I'm not sure what its current status is.  Raymond?
> 
> * There's a partially complete rewrite of decimal in C in the sandbox,
> dating from the Need for Speed sprint in 2006:
> 
> * Wrapping the decNumber library directly would make some sense,
> but I think licensing issues rule this out.

I hope I'm not spreading misinformation, but I see two problems with
the use of decNumber as an arbitrary precision library. First, the
algorithms are not optimized for large numbers. Then, you have to
define the maximum length of the coefficient *at compile time* by
setting DECNUMDIGITS.


> It's difficult to gauge the likelihood of eventual acceptance in advance,
> though.  Maybe writing a PEP would be an efficient use of time in this
> regard?  There are certainly some open issues (e.g., what to do with
> the existing Python module;  what should other Python implementations
> do).

Good, I'll do that. I mainly wanted to find out if there is fundamental
opposition against C-decimal in the standard library, but this does not
seem to be the case.


> I think my biggest concern is maintenance:  we'd be replacing
> 8500 lines of Python code in a single file, that several of the
> current core developers understand well, with 30000 (Stefan, is
> that about accurate?) lines of C in several files, that presumably
> none of the current core devs is familiar with right now.

Ok, I think we could say that the sloc-counts are around 2600 lines of
Python vs. around 11500-12500 lines of C.


> What happens when (e.g.,) the number-theoretic transform needs
> updating for one reason or another?  Stefan, do you see yourself
> having time to spend on maintenance of this code for the
> forseeable future?

Yes. Generally speaking, I expect the code to be low-maintenance.
It would be even easier if all major compilers supported exact
width C99 types and __uint128_t, but this is wishful thinking.


Stefan Krah




From barry at python.org  Thu Oct 22 17:04:22 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 22 Oct 2009 11:04:22 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
Message-ID: <25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>

On Oct 22, 2009, at 10:47 AM, Benjamin Peterson wrote:

> 2009/10/22 Barry Warsaw <barry at python.org>:
>> So does anybody else think bug 7183 should be a release blocker for  
>> 2.6.4
>> final, or is even a legitimate but that we need to fix?
>
> I think it cannot hold up a release with out a reproducible code  
> snippet.

It may not be reproducible in standard Python, see David's follow up  
to the issue.  If that holds true and we can't reproduce it, I agree  
we should not hold up the release for this.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/6c7c1cad/attachment.pgp>

From fabiofz at gmail.com  Thu Oct 22 17:32:37 2009
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Thu, 22 Oct 2009 12:32:37 -0300
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
Message-ID: <cfb578b20910220832n2d68f88cs76a4561c0da3f049@mail.gmail.com>

On Wed, Oct 21, 2009 at 10:56 PM, Mike Krell <mbk.lists at gmail.com> wrote:
> Is there any possibility of backporting support for the nonlocal keyword
> into a? 2.x release?? I see it's not in 2.6, but I don't know if that was an
> intentional design choice or due to a lack of demand / round tuits.? I'm
> also not sure if this would fall under the scope of the proposed moratorium
> on new language features (although my first impression was that it could be
> allowed since it already exists in python 3.
>
> One of my motivations for asking is a recent blog post by Fernando Perez of
> IPython fame that describes an interesting decorator-based idiom inspired by
> Apple's Grand Central Dispatch which would allow many interesting
> possibilities for expressing parallelization and other manipulations of
> execution context for blocks of python code.? Unfortunately, using the
> technique to its fullest extent requires the nonlocal keyword.
>
> The blog post is here:
> https://cirl.berkeley.edu/fperez/py4science/decorators.html
>

Just as a note, the nonlocal there is not a requirement...

You can just create a mutable object there and change that object (so,
you don't need to actually rebind the object in the outer scope).

E.g.: instead of creating a float in the context, create a list with a
single float and change the float in the list (maybe the nonlocal
would be nicer, but it's certainly still usable)

Cheers,

Fabio

From g.brandl at gmx.net  Thu Oct 22 17:43:04 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 22 Oct 2009 17:43:04 +0200
Subject: [Python-Dev] PyErr_NewExceptionWithDoc
Message-ID: <hbpun8$87o$1@ger.gmane.org>

Hi,

issue #7033 proposes a new C API that creates a new exception class with
a docstring.  Since exception classes should be documented, and adding
the docstring after creating is not a one-liner, I would say it is a useful
addition.  What do you all think?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From stefan_ml at behnel.de  Thu Oct 22 18:41:03 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Oct 2009 18:41:03 +0200
Subject: [Python-Dev] Interest in integrating C decimal module into
	Python?
In-Reply-To: <20091022145017.GB2812@mail.bytereef.org>
References: <20091020131547.GA615@mail.bytereef.org>	<5c6f2a5d0910210751k2851cb31qa0c61ed49545bef3@mail.gmail.com>
	<20091022145017.GB2812@mail.bytereef.org>
Message-ID: <hbq1uv$kn7$1@ger.gmane.org>

Stefan Krah wrote:
> Mark Dickinson wrote:
>> I think my biggest concern is maintenance:  we'd be replacing
>> 8500 lines of Python code in a single file, that several of the
>> current core developers understand well, with 30000 (Stefan, is
>> that about accurate?) lines of C in several files, that presumably
>> none of the current core devs is familiar with right now.
> 
> Ok, I think we could say that the sloc-counts are around 2600 lines of
> Python vs. around 11500-12500 lines of C.

If maintenance is an issue, I would actually propose to compile the
existing decimal.py using Cython (works with a few trivial modifications),
add a few type decorators at the hot spots to make them really fast, and
then use that as both the Python implementation and fast binary module.

Stefan


From solipsis at pitrou.net  Thu Oct 22 18:45:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 16:45:00 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?readonly_=5F=5Fdoc=5F=5F?=
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
Message-ID: <loom.20091022T182906-717@post.gmane.org>


Speaking of the __doc__ property, I just noticed the following thing on py3k:

>>> class C: pass
... 
>>> C.__doc__ = "hop"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: attribute '__doc__' of 'type' objects is not writable


Is this deliberate?

Antoine.



From fetchinson at googlemail.com  Thu Oct 22 19:00:51 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Thu, 22 Oct 2009 19:00:51 +0200
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <loom.20091022T182906-717@post.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
Message-ID: <fbe2e2100910221000m9f055ddld643a73b0dc0e1f1@mail.gmail.com>

> Speaking of the __doc__ property, I just noticed the following thing on
> py3k:
>
>>>> class C: pass
> ...
>>>> C.__doc__ = "hop"
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> AttributeError: attribute '__doc__' of 'type' objects is not writable

Happens also with new style classes in python 2.x:

Python 2.5.1 (r251:54863, Oct 30 2007, 13:45:26)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class C(object): pass
...
>>> C.__doc__ = 'hello'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: attribute '__doc__' of 'type' objects is not writable
>>>


But not with old style classes:


Python 2.5.1 (r251:54863, Oct 30 2007, 13:45:26)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class C: pass
...
>>> C.__doc__ = 'hello'
>>>

Cheers,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From foom at fuhm.net  Thu Oct 22 19:01:02 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 22 Oct 2009 13:01:02 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
Message-ID: <14794788-7FFC-4713-BB4C-8213CA746BA3@fuhm.net>


On Oct 22, 2009, at 11:04 AM, Barry Warsaw wrote:

> On Oct 22, 2009, at 10:47 AM, Benjamin Peterson wrote:
>
>> 2009/10/22 Barry Warsaw <barry at python.org>:
>>> So does anybody else think bug 7183 should be a release blocker  
>>> for 2.6.4
>>> final, or is even a legitimate but that we need to fix?
>>
>> I think it cannot hold up a release with out a reproducible code  
>> snippet.
>
> It may not be reproducible in standard Python, see David's follow up  
> to the issue.  If that holds true and we can't reproduce it, I agree  
> we should not hold up the release for this.

 >>> class Foo(property):
...  __slots__=[]
...
 >>> x=Foo()
 >>> x.__doc__ = "asdf"
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
AttributeError: 'Foo' object attribute '__doc__' is read-only

You can't add arbitrary attributes to instances, since some instances  
don't have the slot to put them in.

Is that an equivalent demonstration to that which boost runs into?  
(except, it's using a C type not a python type).

James

From tseaver at palladion.com  Thu Oct 22 19:05:50 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 22 Oct 2009 13:05:50 -0400
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091022T111452-527@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>	<4ADFC9E8.1010400@molden.no>
	<loom.20091022T111452-527@post.gmane.org>
Message-ID: <hbq3de$p3d$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antoine Pitrou wrote:
> Sturla Molden <sturla <at> molden.no> writes:
>> With two threads and a check interval og 100, only 61 of 100000 check 
>> intervals failed to produce a thread-switch in the interpreter. I'd call 
>> that rather fair. 
> 
> This number lacks the elapsed time. 61 switches in one second is probably
> enough, the same amount of switches in 10 or 20 seconds is too small (at least
> for threads needing good responsivity, e.g. I/O threads).

I read Sturla as saying there were 99,939 switches out of a possible
100,000, with sys.checkinterval set to 100.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrgkO4ACgkQ+gerLs4ltQ4yZgCfZyRfWKPBzRb52v7RtLAMlNts
SOIAnjgxqJ1Wovas5cHoju8kwbvn/9Es
=2SDx
-----END PGP SIGNATURE-----


From guido at python.org  Thu Oct 22 19:12:09 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 22 Oct 2009 10:12:09 -0700
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <loom.20091022T182906-717@post.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org> 
	<loom.20091022T182906-717@post.gmane.org>
Message-ID: <ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>

On Thu, Oct 22, 2009 at 9:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Speaking of the __doc__ property, I just noticed the following thing on py3k:
>
>>>> class C: pass
> ...
>>>> C.__doc__ = "hop"
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> AttributeError: attribute '__doc__' of 'type' objects is not writable
>
> Is this deliberate?

Yes.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Thu Oct 22 19:16:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 17:16:46 +0000 (UTC)
Subject: [Python-Dev] GIL behaviour under Windows
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>	<4ADFC9E8.1010400@molden.no>
	<loom.20091022T111452-527@post.gmane.org>
	<hbq3de$p3d$1@ger.gmane.org>
Message-ID: <loom.20091022T191325-290@post.gmane.org>

Tres Seaver <tseaver <at> palladion.com> writes:
> 
> I read Sturla as saying there were 99,939 switches out of a possible
> 100,000, with sys.checkinterval set to 100.

Oops, you're right.
But depending on the elapsed time (again :-)), it may be too high, because
too many switches per second will add a lot of overhead and decrease performance.

Regards

Antoine.



From tseaver at palladion.com  Thu Oct 22 19:16:32 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 22 Oct 2009 13:16:32 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
Message-ID: <hbq41h$rs1$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
> On Oct 22, 2009, at 10:47 AM, Benjamin Peterson wrote:
> 
>> 2009/10/22 Barry Warsaw <barry at python.org>:
>>> So does anybody else think bug 7183 should be a release blocker for  
>>> 2.6.4
>>> final, or is even a legitimate but that we need to fix?
>> I think it cannot hold up a release with out a reproducible code  
>> snippet.
> 
> It may not be reproducible in standard Python, see David's follow up  
> to the issue.  If that holds true and we can't reproduce it, I agree  
> we should not hold up the release for this.

The fix for 5890 has a funny "smell" to me: copying __doc__ into the
instance dict just feels wrong.  How does that work with a pure Python
class using slots?  E.g.:

  class MyProp(property):
      pass

  class Foo(object):
      @property
      def bar(self):
          '''Get a bar.'''

      @MyProp
      def baz(self):
          '''Get a baz.'''

  print Foo.bar.__doc__
  print Foo.baz.__doc__

  class SlottedFoo(object):
      __slots__ = ()
      @property
      def bar(self):
          '''Get a bar.'''

      @MyProp
      def baz(self):
          '''Get a baz.'''

  print SlottedFoo.bar.__doc__
  print SlottedFoo.baz.__doc__


That being said, I can't this bug as a release blocker:  people can
either upgrade to super-current Boost, or stick with 2.6.2 until they can.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrgk3AACgkQ+gerLs4ltQ6n7gCgmV3zWTazLTHWfmD2zul0BTnF
9dkAn38HXKw4RPwWV8Ht4hUGBC00IB2N
=ssdH
-----END PGP SIGNATURE-----


From ubershmekel at gmail.com  Thu Oct 22 19:18:45 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Thu, 22 Oct 2009 19:18:45 +0200
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
Message-ID: <9d153b7c0910221018u60f2e87x7c2097d369fd9e6@mail.gmail.com>

On Thu, Oct 22, 2009 at 7:12 PM, Guido van Rossum <guido at python.org> wrote:
> Yes.

Why?

--yuv

From solipsis at pitrou.net  Thu Oct 22 19:25:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 17:25:30 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?readonly_=5F=5Fdoc=5F=5F?=
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
Message-ID: <loom.20091022T192246-662@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> On Thu, Oct 22, 2009 at 9:45 AM, Antoine Pitrou <solipsis <at> pitrou.net> 
wrote:
> >
> > Speaking of the __doc__ property, I just noticed the following thing
on py3k:
> >
> >>>> class C: pass
> > ...
> >>>> C.__doc__ = "hop"
> > Traceback (most recent call last):
> > ?File "<stdin>", line 1, in <module>
> > AttributeError: attribute '__doc__' of 'type' objects is not writable
> >
> > Is this deliberate?
> 
> Yes.

I might add why I was asking this question. I was trying to demonstrate the use
of class decorators and the simplest example I found was to add a docstring to
the class. And I was surprised when I saw the following fail with the
aforementioned exception:

def classdeco(cls):
    cls.__doc__ = "decorated!"
    return cls

@classdeco
class C:
    pass


Regards

Antoine.



From phillip.sitbon+python-dev at gmail.com  Thu Oct 22 19:51:01 2009
From: phillip.sitbon+python-dev at gmail.com (Phillip Sitbon)
Date: Thu, 22 Oct 2009 10:51:01 -0700
Subject: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?
In-Reply-To: <4ADF2850.1040304@arbash-meinel.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091020T234433-829@post.gmane.org> <4ADE35F0.10803@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>
	<4ADF2850.1040304@arbash-meinel.com>
Message-ID: <536685ea0910221051p1897770xc517561dccc020f6@mail.gmail.com>

I'd just like to point out some previous discussion about implementing
the GIL as a critical section or semaphore on Windows since it's come
up here (although not quite the center of the OP's proposal AFAICS):

http://bugs.python.org/issue6132
http://mail.python.org/pipermail/python-dev/2009-May/089746.html

Some of this is more low-level. I did see higher performance when
using non-Event objects, although I have not had time to follow up and
do a deeper analysis. The GIL flashing "problem" with critical
sections can very likely be rectified with a call to Sleep(0) or
YieldProcessor() for those who are worried about it. On the subject of
fairness, I tested various forms of the GIL on my multi-threaded ISAPI
extension, where every millisecond counts when under high concurrency,
and fairness wasn't an issue for single- or multi-core systems. It may
be anecdotal, but it also may be that the issue is somewhat
over-blown.

It seems like these discussions come up in one form or another a few
times a year and don't really get anywhere - probably because many
people find that it's easier to just run one instance of Python on
each core/processor. IPC is cheap (cPickle rocks!), and Python's
memory footprint is acceptable by today's standards. Still, it is an
interesting topic to many, myself included.

Also, many people keep talking about inefficiencies due to threads
waking up to a locked GIL. I'd like to see examples of this- most of
the time, the OS should know that the thread is contending on the lock
object and it is skipped over. Granted, a thread may wake up just to
release the GIL shortly thereafter, but that's why
sys.setcheckinterval() is there for us to tinker with.

Anyway, enough of my $0.02.

 - Phillip

2009/10/21 John Arbash Meinel <john at arbash-meinel.com>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Kristj?n Valur J?nsson wrote:
> ...
>
>> This depends entirely on the platform and primitives used to implement the GIL.
>> I'm interested in windows. ?There, I found this article:
>> http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html
>> So, you may be on to something. ?Perhaps a simple C test is in order then?
>>
>> I did that. ?I found, on my dual-core vista machine, that running "release", that both Mutexes and CriticalSections behaved as you describe, with no "fairness". ?Using a "semaphore" seems to retain fairness, however.
>> "fairness" was retained in debug builds too, strangely enough.
>>
>> Now, Python uses none of these. ?On windows, it uses an "Event" object coupled with an atomically updated counter. ?This also behaves fairly.
>>
>> The test application is attached.
>>
>>
>> I think that you ought to sustantiate your claims better, maybe with a specific platform and using some test like the above.
>>
>> On the other hand, it shows that we must be careful what we use. ?There has been some talk of using CriticalSections for the GIL on windows. ?This test ought to show the danger of that. ?The GIL is different than a regular lock. ?It is a reverse-lock, really, and therefore may need to be implemented in its own special way, if we want very fast mutexes for the rest of the system (cc to python-dev)
>>
>> Cheers,
>>
>> Kristj?n
>
> I can compile and run this, but I'm not sure I know how to interpret the
> results. If I understand it correctly, then everything but "Critical
> Sections" are fair on my Windows Vista machine.
>
> To run, I changed the line "#define EVENT" to EVENT, MUTEX, SEMAPHORE
> and CRIT. I then built and ran in "Release" environment (using VS 2008
> Express)
>
> For all but CRIT, I saw things like:
> thread ? ? ? 5532 reclaims GIL
> thread ? ? ? 5532 working 51234 units
> thread ? ? ? 5532 worked 51234 units: 1312435761
> thread ? ? ? 5532 flashing GIL
> thread ? ? ? 5876 reclaims GIL
> thread ? ? ? 5876 working 51234 units
> thread ? ? ? 5876 worked 51234 units: 1312435761
> thread ? ? ? 5876 flashing GIL
>
> Where there would be 4 lines for one thread, then 4 lines for the other
> thread.
>
> for CRIT, I saw something more like 50 lines for one thread, and then 50
> lines for the other thread.
>
> This is Vista Home Basic, and VS 2008 Express Edition, with a 2-core
> machine.
>
> John
> =:->
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Cygwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkrfKFAACgkQJdeBCYSNAANbuQCgudU0IChylofTwvUk/JglChBd
> 9gsAoIJHj63/CagKpduUsd68HV8pP3QX
> =CuUj
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/phillip.sitbon%2Bpython-dev%40gmail.com
>

From guido at python.org  Thu Oct 22 19:58:08 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 22 Oct 2009 10:58:08 -0700
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <loom.20091022T192246-662@post.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org> 
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com> 
	<loom.20091022T192246-662@post.gmane.org>
Message-ID: <ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com>

Well __doc__ isn't a normal attribute -- it doesn't follow inheritance rules.

On Thu, Oct 22, 2009 at 10:25 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Guido van Rossum <guido <at> python.org> writes:
>>
>> On Thu, Oct 22, 2009 at 9:45 AM, Antoine Pitrou <solipsis <at> pitrou.net>
> wrote:
>> >
>> > Speaking of the __doc__ property, I just noticed the following thing
> on py3k:
>> >
>> >>>> class C: pass
>> > ...
>> >>>> C.__doc__ = "hop"
>> > Traceback (most recent call last):
>> > ?File "<stdin>", line 1, in <module>
>> > AttributeError: attribute '__doc__' of 'type' objects is not writable
>> >
>> > Is this deliberate?
>>
>> Yes.
>
> I might add why I was asking this question. I was trying to demonstrate the use
> of class decorators and the simplest example I found was to add a docstring to
> the class. And I was surprised when I saw the following fail with the
> aforementioned exception:
>
> def classdeco(cls):
> ? ?cls.__doc__ = "decorated!"
> ? ?return cls
>
> @classdeco
> class C:
> ? ?pass
>
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From alexander.belopolsky at gmail.com  Thu Oct 22 19:52:51 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 22 Oct 2009 13:52:51 -0400
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <loom.20091022T192246-662@post.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
	<loom.20091022T192246-662@post.gmane.org>
Message-ID: <d38f5330910221052r1abff46cl2213dbfb094d279e@mail.gmail.com>

On Thu, Oct 22, 2009 at 1:25 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
..
> I might add why I was asking this question. I was trying to demonstrate the use
> of class decorators and the simplest example I found was to add a docstring to
> the class.

I always thought that read-only __doc__ was an artifact of new classes
evolving from (C-implemented) types.  IIRC, NumPy project implemented
a somewhat elaborate hack to allow docstrings to be added to
C-implemented classes in a .py file.

From ssteinerx at gmail.com  Thu Oct 22 20:18:37 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 22 Oct 2009 14:18:37 -0400
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
	<loom.20091022T192246-662@post.gmane.org>
	<ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com>
Message-ID: <198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>


On Oct 22, 2009, at 1:58 PM, Guido van Rossum wrote:

> Well __doc__ isn't a normal attribute -- it doesn't follow  
> inheritance rules.

Maybe we could add a ticket to flag this in the docs.

Is __doc__ not normal due to its general underscorishness, or is it  
not normal because it isn't?

Any others that deserve special notice, while we're at it?

Thanks,

S


From phillip.sitbon+python-dev at gmail.com  Thu Oct 22 20:19:37 2009
From: phillip.sitbon+python-dev at gmail.com (Phillip Sitbon)
Date: Thu, 22 Oct 2009 11:19:37 -0700
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091022T191325-290@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org> <4ADFC9E8.1010400@molden.no>
	<loom.20091022T111452-527@post.gmane.org> <hbq3de$p3d$1@ger.gmane.org>
	<loom.20091022T191325-290@post.gmane.org>
Message-ID: <536685ea0910221119t66cc1b37ua940e9a8eda8294f@mail.gmail.com>

I know I already posted some relevant threads to the other discussion,
but I wanted to point out a couple of specific comments on GIL
fairness from the discussion:

http://mail.python.org/pipermail/python-dev/2009-May/089752.html
http://mail.python.org/pipermail/python-dev/2009-May/089755.html

- Phillip

On Thu, Oct 22, 2009 at 10:16 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tres Seaver <tseaver <at> palladion.com> writes:
>>
>> I read Sturla as saying there were 99,939 switches out of a possible
>> 100,000, with sys.checkinterval set to 100.
>
> Oops, you're right.
> But depending on the elapsed time (again :-)), it may be too high, because
> too many switches per second will add a lot of overhead and decrease performance.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/phillip.sitbon%2Bpython-dev%40gmail.com
>

From brett at python.org  Thu Oct 22 20:30:38 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 22 Oct 2009 11:30:38 -0700
Subject: [Python-Dev] PyErr_NewExceptionWithDoc
In-Reply-To: <hbpun8$87o$1@ger.gmane.org>
References: <hbpun8$87o$1@ger.gmane.org>
Message-ID: <bbaeab100910221130r476f6f71x6956dea2736a76e1@mail.gmail.com>

On Thu, Oct 22, 2009 at 08:43, Georg Brandl <g.brandl at gmx.net> wrote:

> Hi,
>
> issue #7033 proposes a new C API that creates a new exception class with
> a docstring.  Since exception classes should be documented, and adding
> the docstring after creating is not a one-liner, I would say it is a useful
> addition.  What do you all think?
>
>
Seems fine to me.


> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent
> thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/cc11e35a/attachment.htm>

From barry at python.org  Thu Oct 22 20:36:37 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 22 Oct 2009 14:36:37 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <hbq41h$rs1$1@ger.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org>
Message-ID: <65DE256C-9EE8-4B72-A647-0C47EE4C27E1@python.org>

On Oct 22, 2009, at 1:16 PM, Tres Seaver wrote:

> That being said, I can't this bug as a release blocker:  people can
> either upgrade to super-current Boost, or stick with 2.6.2 until  
> they can.

Agreed.  I've knocked it down from release-blocker.

-Barryu

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/573f9574/attachment-0001.pgp>

From brett at python.org  Thu Oct 22 20:39:53 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 22 Oct 2009 11:39:53 -0700
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org> 
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com> 
	<loom.20091022T192246-662@post.gmane.org>
	<ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com> 
	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
Message-ID: <bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>

On Thu, Oct 22, 2009 at 11:18, ssteinerX at gmail.com <ssteinerx at gmail.com>wrote:

>
> On Oct 22, 2009, at 1:58 PM, Guido van Rossum wrote:
>
>  Well __doc__ isn't a normal attribute -- it doesn't follow inheritance
>> rules.
>>
>
> Maybe we could add a ticket to flag this in the docs.
>
>
Sure, go for it.


> Is __doc__ not normal due to its general underscorishness, or is it not
> normal because it isn't?
>
>
I honestly don't follow that sentence. But __doc__ is special because of its
use; documenting how to use of an object. In this case when you call
something like help() on an instance of an object it skips the instance's
value for __doc__ and goes straight to the defining class and stops there as
you don't care how a subclass says to use itself as that is not what you are
working with.


> Any others that deserve special notice, while we're at it?
>

Don't know.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/29c5e926/attachment.htm>

From rdmurray at bitdance.com  Thu Oct 22 20:44:05 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 22 Oct 2009 14:44:05 -0400 (EDT)
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <hbq41h$rs1$1@ger.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org>
Message-ID: <Pine.LNX.4.64.0910221441300.18193@kimball.webabinitio.net>

On Thu, 22 Oct 2009 at 13:16, Tres Seaver wrote:
> The fix for 5890 has a funny "smell" to me: copying __doc__ into the
> instance dict just feels wrong.  How does that work with a pure Python
> class using slots?  E.g.:

It doesn't.  There's even a test to make sure it doesn't :)
(It raises an attribute error.)

__doc__ is a funny beast.  If someone can come up with a better
fix for 5890, that would be great.

--David

From barry at barrys-emacs.org  Thu Oct 22 21:31:42 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu, 22 Oct 2009 20:31:42 +0100
Subject: [Python-Dev] Add const to python API - issue 6952
In-Reply-To: <4ADE98E2.3080707@v.loewis.de>
References: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>	<4ADD2FE9.2010807@v.loewis.de>	<74D1336F-AB87-431F-895C-885EE344CEBD@barrys-emacs.org>
	<eae285400910201431j17d35733vc6423abf85997af9@mail.gmail.com>
	<4ADE98E2.3080707@v.loewis.de>
Message-ID: <A043C4A3-A6BA-4F68-A092-B9CB890DD462@barrys-emacs.org>


On 21 Oct 2009, at 06:15, Martin v. L?wis wrote:

>> I suggest following POSIX's lead and omitted the const in these  
>> cases.

Ah... Yes I see this in strstr for example.

>
> Thanks, that sounds reasonable.


This is basically what I have done in the patch I hope.
I have updated the patch to include a comment explaining
why output  param is char ** and the need for a cast.

Barry


From ssteinerx at gmail.com  Thu Oct 22 21:48:36 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 22 Oct 2009 15:48:36 -0400
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
	<loom.20091022T192246-662@post.gmane.org>
	<ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com>
	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
	<bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
Message-ID: <995C225C-8141-43D3-A840-75F7F336FA5E@gmail.com>


On Oct 22, 2009, at 2:39 PM, Brett Cannon wrote:

>
> On Thu, Oct 22, 2009 at 11:18, ssteinerX at gmail.com <ssteinerx at gmail.com 
> > wrote:
>
> On Oct 22, 2009, at 1:58 PM, Guido van Rossum wrote:
>
> Well __doc__ isn't a normal attribute -- it doesn't follow  
> inheritance rules.
>
> Maybe we could add a ticket to flag this in the docs.
> Sure, go for it.

Filed: http://bugs.python.org/issue7186

>
> Is __doc__ not normal due to its general underscorishness, or is it  
> not normal because it isn't?
>
>
> I honestly don't follow that sentence.

It means, is it special because it's a "special" i.e. __X__ type  
attribute or because of something special about __doc__ itself,  
independent of it being a "special attribute".

> But __doc__ is special because of its use; documenting how to use of  
> an object. In this case when you call something like help() on an  
> instance of an object it skips the instance's value for __doc__ and  
> goes straight to the defining class and stops there as you don't  
> care how a subclass says to use itself as that is not what you are  
> working with.
>
> Any others that deserve special notice, while we're at it?
>
> Don't know.

Ok, issue recorded.

Thanks,

S


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/2a7823cc/attachment.htm>

From g.brandl at gmx.net  Thu Oct 22 21:47:26 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 22 Oct 2009 21:47:26 +0200
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <hbq41h$rs1$1@ger.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org>
Message-ID: <hbqd1f$qp3$1@ger.gmane.org>

Tres Seaver schrieb:
> Barry Warsaw wrote:
>> On Oct 22, 2009, at 10:47 AM, Benjamin Peterson wrote:
> 
>>> 2009/10/22 Barry Warsaw <barry at python.org>:
>>>> So does anybody else think bug 7183 should be a release blocker for  
>>>> 2.6.4
>>>> final, or is even a legitimate but that we need to fix?
>>> I think it cannot hold up a release with out a reproducible code  
>>> snippet.
> 
>> It may not be reproducible in standard Python, see David's follow up  
>> to the issue.  If that holds true and we can't reproduce it, I agree  
>> we should not hold up the release for this.
> 
> The fix for 5890 has a funny "smell" to me: copying __doc__ into the
> instance dict just feels wrong.  How does that work with a pure Python
> class using slots?  E.g.:
> 
>   class MyProp(property):
>       pass
> 
>   class Foo(object):
>       @property
>       def bar(self):
>           '''Get a bar.'''
> 
>       @MyProp
>       def baz(self):
>           '''Get a baz.'''
> 
>   print Foo.bar.__doc__
>   print Foo.baz.__doc__
> 
>   class SlottedFoo(object):
>       __slots__ = ()
>       @property
>       def bar(self):
>           '''Get a bar.'''
> 
>       @MyProp
>       def baz(self):
>           '''Get a baz.'''
> 
>   print SlottedFoo.bar.__doc__
>   print SlottedFoo.baz.__doc__

Works just fine.  I think you meant

class MyProp(property):
    __slots__ = ()

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From robertc at robertcollins.net  Thu Oct 22 21:53:43 2009
From: robertc at robertcollins.net (Robert Collins)
Date: Fri, 23 Oct 2009 06:53:43 +1100
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <hbq41h$rs1$1@ger.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org>
Message-ID: <1256241223.2929.56.camel@lifeless-64>

On Thu, 2009-10-22 at 13:16 -0400, Tres Seaver wrote:
...
> That being said, I can't this bug as a release blocker:  people can
> either upgrade to super-current Boost, or stick with 2.6.2 until they can.

Thats the challenge Ubuntu faces:
https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688

We've just announced our Karmic RC, boost 1.40 isn't released, and
python 2.6.3 doesn't work with a released boost :(

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/29127dbb/attachment-0001.pgp>

From martin at v.loewis.de  Thu Oct 22 22:24:35 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 22 Oct 2009 22:24:35 +0200
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
Message-ID: <4AE0BF83.4080903@v.loewis.de>

Mike Krell wrote:
> Is there any possibility of backporting support for the nonlocal keyword
> into a  2.x release?  

If so, only into 2.7. Can you please explain why it would be desirable
to do that? 2.7 will likely be the last 2.x release, so only a fairly
small portion of the applications would be actually able to use this (or
any other new feature added to 2.7): most code supporting 2.x will also
have to support 2.6, so the keyword won't be available to such code,
anyway.

Regards,
Martin

From p.f.moore at gmail.com  Thu Oct 22 22:27:04 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 22 Oct 2009 21:27:04 +0100
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <1256241223.2929.56.camel@lifeless-64>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org> <1256241223.2929.56.camel@lifeless-64>
Message-ID: <79990c6b0910221327h37a678cfw4e566e4cb9f8083e@mail.gmail.com>

2009/10/22 Robert Collins <robertc at robertcollins.net>:
> On Thu, 2009-10-22 at 13:16 -0400, Tres Seaver wrote:
> ...
>> That being said, I can't this bug as a release blocker: ?people can
>> either upgrade to super-current Boost, or stick with 2.6.2 until they can.
>
> Thats the challenge Ubuntu faces:
> https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688
>
> We've just announced our Karmic RC, boost 1.40 isn't released, and
> python 2.6.3 doesn't work with a released boost :(

?? Boost 1.40 was released on 27th August, according to www.boost.org...

Paul.

From barry at python.org  Thu Oct 22 22:29:18 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 22 Oct 2009 16:29:18 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <1256241223.2929.56.camel@lifeless-64>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org> <1256241223.2929.56.camel@lifeless-64>
Message-ID: <FA25D6A9-5A77-4576-8311-046A21CE1F35@python.org>

On Oct 22, 2009, at 3:53 PM, Robert Collins wrote:

> On Thu, 2009-10-22 at 13:16 -0400, Tres Seaver wrote:
> ...
>> That being said, I can't this bug as a release blocker:  people can
>> either upgrade to super-current Boost, or stick with 2.6.2 until  
>> they can.
>
> Thats the challenge Ubuntu faces:
> https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688
>
> We've just announced our Karmic RC, boost 1.40 isn't released, and
> python 2.6.3 doesn't work with a released boost :(

Aren't we stuck either way though? :(

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/d5e79270/attachment.pgp>

From foom at fuhm.net  Thu Oct 22 22:29:28 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 22 Oct 2009 16:29:28 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <1256241223.2929.56.camel@lifeless-64>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org> <1256241223.2929.56.camel@lifeless-64>
Message-ID: <0272128F-3703-411C-BA1B-2F7BF8AC63A4@fuhm.net>


On Oct 22, 2009, at 3:53 PM, Robert Collins wrote:

> On Thu, 2009-10-22 at 13:16 -0400, Tres Seaver wrote:
> ...
>> That being said, I can't this bug as a release blocker:  people can
>> either upgrade to super-current Boost, or stick with 2.6.2 until  
>> they can.
>
> Thats the challenge Ubuntu faces:
> https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688
>
> We've just announced our Karmic RC, boost 1.40 isn't released, and
> python 2.6.3 doesn't work with a released boost :(

If I were running a Linux distro, I'd revert the patch in 2.6.3.

And if I were running a Python release process, I'd revert that patch  
for python 2.6.4, and reopen the bug that it fixed, so a less-breaky  
patch can be made.

James


From exarkun at twistedmatrix.com  Thu Oct 22 22:46:21 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Thu, 22 Oct 2009 20:46:21 -0000
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE0BF83.4080903@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
Message-ID: <20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>

On 08:24 pm, martin at v.loewis.de wrote:
>Mike Krell wrote:
>>Is there any possibility of backporting support for the nonlocal 
>>keyword
>>into a  2.x release?
>
>If so, only into 2.7. Can you please explain why it would be desirable
>to do that? 2.7 will likely be the last 2.x release, so only a fairly
>small portion of the applications would be actually able to use this 
>(or
>any other new feature added to 2.7): most code supporting 2.x will also
>have to support 2.6, so the keyword won't be available to such code,
>anyway.

For the same reason that it is desirable to backport all of the other 
changes from 3.x - because it makes the 2.x to 3.x transition easier.

If Python 2.7 supports the nonlocal keyword, then 2.7 becomes that much 
better of a stepping stone towards 3.x.

You've suggested that most 2.x code will have to support 2.6 and so 
won't be able to use the nonlocal keyword even if it is added to 2.7. 
This precise argument could be applied to all of the features in 2.6 
which aim to bring it closer to 3.x.  Any program which must retain 
Python 2.5 compatibility will not be able to use them.  Yet 2.6 is a 
more useful stepping stone towards 3.x than 2.5 is.

So yes, it would be quite desirable to see nonlocal and as many other 
3.x features as possible backported for 2.7.  And depending on how close 
2.7 manages to get, it may make sense to backport anything that doesn't 
make it into 2.7 for a 2.8 release.

The 3.x transition is *hard*.  Anything that makes it easier is good.

Jean-Paul

From ncoghlan at gmail.com  Thu Oct 22 22:49:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Oct 2009 06:49:19 +1000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF4213E@exchis.ccp.ad.local>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F098FF4213E@exchis.ccp.ad.local>
Message-ID: <4AE0C54F.20705@gmail.com>

Kristj?n Valur J?nsson wrote:
> Yes.  The problem with QPC is that although it has very high
> resolution, it is not precise in the long term.  And
> GetSystemTimeAsFileTime() is high precision in the long term but only
> updated evey 20ms or so. In EVE Online we use a combination of the
> two for high resolution, long term precision.  But I'm not happy with
> the way I'm doing it.  It needs some sort of smoothing of course.
> I've even played with using Kalman filtering to do it... The idea is
> to use the low frequency timer to apply correction coefficients to
> the high frequency timer, yet keep the flow of time smooth (no
> backwards jumps because of corrections.).  An optimal solution has so
> far eluded me.

That sounds very similar to the problem spec for system time corrections
in Network Time Protocol client implementations. Perhaps the time
drifting algorithms in the NTP specs are relevant? Or are they too slow
to correct discrepancies?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Thu Oct 22 22:49:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 20:49:37 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?readonly_=5F=5Fdoc=5F=5F?=
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
	<loom.20091022T192246-662@post.gmane.org>
	<ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com>
	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
	<bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
Message-ID: <loom.20091022T224813-373@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> 
> I honestly don't follow that sentence. But __doc__ is special because of its 
> use; documenting how to use of an object. In this case when you call 
> something like help() on an instance of an object it skips the instance's 
> value for __doc__ and goes straight to the defining class and stops there as 
> you don't care how a subclass says to use itself as that is not what you are 
> working with.

I don't really understand how this explains the read-only __doc__.
I am talking about modifying __doc__ on a class, not on an instance.
(sure, a new-style class is also an instance of type, but still...)


Antoine.



From robertc at robertcollins.net  Thu Oct 22 22:51:31 2009
From: robertc at robertcollins.net (Robert Collins)
Date: Fri, 23 Oct 2009 07:51:31 +1100
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <79990c6b0910221327h37a678cfw4e566e4cb9f8083e@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org> <1256241223.2929.56.camel@lifeless-64>
	<79990c6b0910221327h37a678cfw4e566e4cb9f8083e@mail.gmail.com>
Message-ID: <1256244691.2929.69.camel@lifeless-64>

On Thu, 2009-10-22 at 21:27 +0100, Paul Moore wrote:
> 2009/10/22 Robert Collins <robertc at robertcollins.net>:
> > On Thu, 2009-10-22 at 13:16 -0400, Tres Seaver wrote:
> > ...
> >> That being said, I can't this bug as a release blocker:  people can
> >> either upgrade to super-current Boost, or stick with 2.6.2 until they can.
> >
> > Thats the challenge Ubuntu faces:
> > https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688
> >
> > We've just announced our Karmic RC, boost 1.40 isn't released, and
> > python 2.6.3 doesn't work with a released boost :(
> 
> ?? Boost 1.40 was released on 27th August, according to www.boost.org...

Oh bah, teach me to trust comments in bug reports :)

Still, a full release is high risk, one of the Ubuntu devs is looking
for the single patch they added to fix things.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/3ab419d0/attachment.pgp>

From martin at v.loewis.de  Thu Oct 22 22:57:32 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 22 Oct 2009 22:57:32 +0200
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
Message-ID: <4AE0C73C.9070805@v.loewis.de>

exarkun at twistedmatrix.com wrote:
> On 08:24 pm, martin at v.loewis.de wrote:
>> Mike Krell wrote:
>>> Is there any possibility of backporting support for the nonlocal keyword
>>> into a  2.x release?
>>
>> If so, only into 2.7. Can you please explain why it would be desirable
>> to do that? 2.7 will likely be the last 2.x release, so only a fairly
>> small portion of the applications would be actually able to use this (or
>> any other new feature added to 2.7): most code supporting 2.x will also
>> have to support 2.6, so the keyword won't be available to such code,
>> anyway.
> 
> For the same reason that it is desirable to backport all of the other
> changes from 3.x - because it makes the 2.x to 3.x transition easier.

Hmm. Really?

> If Python 2.7 supports the nonlocal keyword, then 2.7 becomes that much
> better of a stepping stone towards 3.x.

What use has such a stepping stone? Why, and (more importantly) when
would anybody currently supporting 2.x give up 2.6 and earlier, and
only support 2.7? And, if they chose to do so, why would they not move
the code base to 3.x right away?

> You've suggested that most 2.x code will have to support 2.6 and so
> won't be able to use the nonlocal keyword even if it is added to 2.7.

Correct.

> This precise argument could be applied to all of the features in 2.6
> which aim to bring it closer to 3.x.

Not so. One of the features added to 2.6 was the 3k warning. This
warning can be used without any modification to the code. So code
can run on 2.6 and use the feature, while running unmodified on 2.5
and earlier (not using it).

As for actual language and library changes (such as any new future
import): there was indeed little point adding them. However, given
that the possible migration paths weren't as clear back then as they
are now, it is understandable that people considered this a viable
path.

In addition, for 2.6, it's a bit more realistic to assume that people
might drop 2.5 support and still support 2.x for some more time (in
particular as people wouldn't rule out a 2.8 release back then, either).

> Any program which must retain
> Python 2.5 compatibility will not be able to use them.  Yet 2.6 is a
> more useful stepping stone towards 3.x than 2.5 is.

I disagree fairly much (except that the 3k warnings probably *are*
useful - even though I haven't ever used them myself).

> So yes, it would be quite desirable to see nonlocal and as many other
> 3.x features as possible backported for 2.7.  And depending on how close
> 2.7 manages to get, it may make sense to backport anything that doesn't
> make it into 2.7 for a 2.8 release.

There might not be a 2.8 release at all, though.

> The 3.x transition is *hard*.  Anything that makes it easier is good.

I agree. I question whether backporting features actually makes the
transition easier.

In addition, in the *specific* case: the nonlocal keyword isn't
necessary for a transition *at all*. Code that currently works without
it won't need it when ported to 3.x. You may not be able to use it while
maintaining 2.x and 3.x simultaneously, but you can certainly do the
transition just fine without it.

Regards,
Martin

From guido at python.org  Thu Oct 22 22:59:17 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 22 Oct 2009 13:59:17 -0700
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <loom.20091022T224813-373@post.gmane.org>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org> 
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com> 
	<loom.20091022T192246-662@post.gmane.org>
	<ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com> 
	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
	<bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com> 
	<loom.20091022T224813-373@post.gmane.org>
Message-ID: <ca471dc20910221359v1df03dbcr281ad822c7aff88e@mail.gmail.com>

On Thu, Oct 22, 2009 at 1:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Brett Cannon <brett <at> python.org> writes:
>> I honestly don't follow that sentence. But __doc__ is special because of its
>> use; documenting how to use of an object. In this case when you call
>> something like help() on an instance of an object it skips the instance's
>> value for __doc__ and goes straight to the defining class and stops there as
>> you don't care how a subclass says to use itself as that is not what you are
>> working with.
>
> I don't really understand how this explains the read-only __doc__.
> I am talking about modifying __doc__ on a class, not on an instance.
> (sure, a new-style class is also an instance of type, but still...)

Antoine, it's not clear from the questions you're asking whether
you've read the code in typobject.c that implements __doc__ or not.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Thu Oct 22 23:26:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Oct 2009 23:26:35 +0200
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <ca471dc20910221359v1df03dbcr281ad822c7aff88e@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<loom.20091022T182906-717@post.gmane.org>
	<ca471dc20910221012t1bf1a259ya4213c2a467b8930@mail.gmail.com>
	<loom.20091022T192246-662@post.gmane.org>
	<ca471dc20910221058k63d23e23q37f25dc9984a23cf@mail.gmail.com>
	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
	<bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
	<loom.20091022T224813-373@post.gmane.org>
	<ca471dc20910221359v1df03dbcr281ad822c7aff88e@mail.gmail.com>
Message-ID: <1256246795.5828.0.camel@localhost>

Le jeudi 22 octobre 2009 ? 13:59 -0700, Guido van Rossum a ?crit :
> > I don't really understand how this explains the read-only __doc__.
> > I am talking about modifying __doc__ on a class, not on an instance.
> > (sure, a new-style class is also an instance of type, but still...)
> 
> Antoine, it's not clear from the questions you're asking whether
> you've read the code in typobject.c that implements __doc__ or not.

Ah, well I haven't :-/ My bad.

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Thu Oct 22 23:08:06 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 22 Oct 2009 22:08:06 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE0C73C.9070805@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<4AE0BF83.4080903@v.loewis.de>	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
Message-ID: <4AE0C9B6.60109@voidspace.org.uk>

Martin v. L?wis wrote:
> exarkun at twistedmatrix.com wrote:
>   
>> On 08:24 pm, martin at v.loewis.de wrote:
>>     
>>> Mike Krell wrote:
>>>       
>>>> Is there any possibility of backporting support for the nonlocal keyword
>>>> into a  2.x release?
>>>>         
>>> If so, only into 2.7. Can you please explain why it would be desirable
>>> to do that? 2.7 will likely be the last 2.x release, so only a fairly
>>> small portion of the applications would be actually able to use this (or
>>> any other new feature added to 2.7): most code supporting 2.x will also
>>> have to support 2.6, so the keyword won't be available to such code,
>>> anyway.
>>>       
>> For the same reason that it is desirable to backport all of the other
>> changes from 3.x - because it makes the 2.x to 3.x transition easier.
>>     
>
> Hmm. Really?
>
>   
>> If Python 2.7 supports the nonlocal keyword, then 2.7 becomes that much
>> better of a stepping stone towards 3.x.
>>     
>
> What use has such a stepping stone? Why, and (more importantly) when
> would anybody currently supporting 2.x give up 2.6 and earlier, and
> only support 2.7? And, if they chose to do so, why would they not move
> the code base to 3.x right away?
>   


 From the Django roadmap for supporting 3.0, using 2.6 as a stepping 
stone (and if 2.7 was a *better* stepping stone then it would make it 
easier):

    http://groups.google.com/group/django-developers/msg/0888b1c8f2518059?


* First half of 2009: Django 1.1 is released, with a notification that
  it will be the final release supporting Python 2.3.

* Second half of 2009: Django 1.2 is released, drops Python 2.3
  support and is the final release supporting Python 2.4.

* First half of 2010: Django 1.3 is released, drops Python 2.4 support
  and is the final release supporting Python 2.5.

* Second half of 2010: Django 1.4 is released and drops Python 2.5
  support.

This gets us to a situation where, about a year after the release of
Python 3.0, Django will be ready to make the transition -- the only
2.x Python we'll be supporting is Python 2.6, and 2to3 plus manual
effort and available supporting libraries should make it possible to
also run Django on Python 3.0 either at that point or not long after.

 From there, 2.6 support can be dropped whenever convenient, and Django
can move to running only on Python 3.x at whatever time is judged
appropriate.


All the best,

Michael Foord

>   
>> You've suggested that most 2.x code will have to support 2.6 and so
>> won't be able to use the nonlocal keyword even if it is added to 2.7.
>>     
>
> Correct.
>
>   
>> This precise argument could be applied to all of the features in 2.6
>> which aim to bring it closer to 3.x.
>>     
>
> Not so. One of the features added to 2.6 was the 3k warning. This
> warning can be used without any modification to the code. So code
> can run on 2.6 and use the feature, while running unmodified on 2.5
> and earlier (not using it).
>
> As for actual language and library changes (such as any new future
> import): there was indeed little point adding them. However, given
> that the possible migration paths weren't as clear back then as they
> are now, it is understandable that people considered this a viable
> path.
>
> In addition, for 2.6, it's a bit more realistic to assume that people
> might drop 2.5 support and still support 2.x for some more time (in
> particular as people wouldn't rule out a 2.8 release back then, either).
>
>   
>> Any program which must retain
>> Python 2.5 compatibility will not be able to use them.  Yet 2.6 is a
>> more useful stepping stone towards 3.x than 2.5 is.
>>     
>
> I disagree fairly much (except that the 3k warnings probably *are*
> useful - even though I haven't ever used them myself).
>
>   
>> So yes, it would be quite desirable to see nonlocal and as many other
>> 3.x features as possible backported for 2.7.  And depending on how close
>> 2.7 manages to get, it may make sense to backport anything that doesn't
>> make it into 2.7 for a 2.8 release.
>>     
>
> There might not be a 2.8 release at all, though.
>
>   
>> The 3.x transition is *hard*.  Anything that makes it easier is good.
>>     
>
> I agree. I question whether backporting features actually makes the
> transition easier.
>
> In addition, in the *specific* case: the nonlocal keyword isn't
> necessary for a transition *at all*. Code that currently works without
> it won't need it when ported to 3.x. You may not be able to use it while
> maintaining 2.x and 3.x simultaneously, but you can certainly do the
> transition just fine without it.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From sturla at molden.no  Thu Oct 22 23:33:22 2009
From: sturla at molden.no (Sturla Molden)
Date: Thu, 22 Oct 2009 23:33:22 +0200
Subject: [Python-Dev] GIL behaviour under Windows
In-Reply-To: <loom.20091022T111452-527@post.gmane.org>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>	<4ADFC9E8.1010400@molden.no>
	<loom.20091022T111452-527@post.gmane.org>
Message-ID: <4AE0CFA2.5070701@molden.no>

Antoine Pitrou skrev:
> This number lacks the elapsed time. 61 switches in one second is probably
> enough, the same amount of switches in 10 or 20 seconds is too small (at least
> for threads needing good responsivity, e.g. I/O threads).
>
> Also, "fair" has to take into account the average latency and its relative
> stability, which is why I wrote ccbench.
>   

Since I am a scientist and statistics interests me, let's do this 
properly :-) Here is a suggestion:

_Py_Ticker is a circular variable. Thus, it can be transformed to an 
angle measured in radians, using:

   a = 2 * pi * _Py_Ticker / _Py_CheckInterval

With simultaneous measurements of a, check interval count x, and time y 
(?s), we can fit the multiple regression:

   y = b0 + b1*cos(a) + b2*sin(a) + b3*x + err

using some non-linear least squares solver. We can then extract all the 
statistics we need on interpreter latencies for "ticks" with and without 
periodic checks.

On a Python setup with many missed thread switches (pthreads according 
to D. Beazley), we could just extend the model to take into account 
successful and unsccessful check intervals:

   y = b0 + b1*cos(a) + b2*sin(a) + b3*x1 + b4*x2 + err
 
where x1 being successful thread switches and x2 being missed thread 
switches. But at least on Windows we can use the simpler model.

The reason why multiple regression is needed, is that the record method 
of my GIL_Battle class is not called on every interpreter tick. I thus 
cannot measure precicesly each latency, which I could have done with a 
direct hook into ceval.c. So statistics to the rescue. But on the bright 
side, it reduces the overhead of the profiler.

Would that help?


Sturla Molden


From sturla at molden.no  Thu Oct 22 23:42:32 2009
From: sturla at molden.no (Sturla Molden)
Date: Thu, 22 Oct 2009 23:42:32 +0200
Subject: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?
In-Reply-To: <536685ea0910221051p1897770xc517561dccc020f6@mail.gmail.com>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091020T234433-829@post.gmane.org>
	<4ADE35F0.10803@molden.no>	<930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local>	<4ADF2850.1040304@arbash-meinel.com>
	<536685ea0910221051p1897770xc517561dccc020f6@mail.gmail.com>
Message-ID: <4AE0D1C8.8050404@molden.no>

Phillip Sitbon skrev:
> Some of this is more low-level. I did see higher performance when
> using non-Event objects, although I have not had time to follow up and
> do a deeper analysis. The GIL flashing "problem" with critical
> sections can very likely be rectified with a call to Sleep(0) or
> YieldProcessor() for those who are worried about it. 
For those who don't know what Sleep(0) on Windows does: It returns the 
reminder of the current time-slice back to the system is a thread with 
equal or higher-priority is ready to run. Otherwise it does nothing.

GIL flashing is a serious issue if it happens often; with the current 
event-based GIL on Windows, it never happens (61 cases of GIL flash in 
100,000 periodic checks is as good as never).

S.M.



From martin at v.loewis.de  Thu Oct 22 23:51:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 22 Oct 2009 23:51:29 +0200
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE0C9B6.60109@voidspace.org.uk>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<4AE0BF83.4080903@v.loewis.de>	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de> <4AE0C9B6.60109@voidspace.org.uk>
Message-ID: <4AE0D3E1.2010207@v.loewis.de>

> From the Django roadmap for supporting 3.0, using 2.6 as a stepping
> stone (and if 2.7 was a *better* stepping stone then it would make it
> easier):
> 
>    http://groups.google.com/group/django-developers/msg/0888b1c8f2518059?

Is that still a current plan? It's from November 2008.

> This gets us to a situation where, about a year after the release of
> Python 3.0, Django will be ready to make the transition -- the only
> 2.x Python we'll be supporting is Python 2.6, and 2to3 plus manual
> effort and available supporting libraries should make it possible to
> also run Django on Python 3.0 either at that point or not long after.
> 
> From there, 2.6 support can be dropped whenever convenient, and Django
> can move to running only on Python 3.x at whatever time is judged
> appropriate.

I would claim that this specific plan was ignoring opportunities
for migrating to Python 3.0. My Django port demonstrates that you
can very well support 2.3 and 3.0 simultaneously:

http://wiki.python.org/moin/PortingDjangoTo3k

Regards,
Martin

From steve at pearwood.info  Fri Oct 23 04:13:55 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 23 Oct 2009 13:13:55 +1100
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>
	<bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
Message-ID: <200910231313.55925.steve@pearwood.info>

On Fri, 23 Oct 2009 05:39:53 am Brett Cannon wrote:

> > Is __doc__ not normal due to its general underscorishness, or is it
> > not normal because it isn't?
>
> I honestly don't follow that sentence. But __doc__ is special because
> of its use; documenting how to use of an object. In this case when
> you call something like help() on an instance of an object it skips
> the instance's value for __doc__ and goes straight to the defining
> class and stops there as you don't care how a subclass says to use
> itself as that is not what you are working with.

Classes don't normally inherit behaviour from their subclasses. 
Presumably you meant superclass.

I expected __doc__ to be just like the other double-underscore objects: 
lookup skips the instance and goes to the class, then follows the 
normal rules of inheritance. Consequently I've just discovered that a 
few of my classes, which I assumed inherited __doc__, don't actually 
have a docstring. This has consequences beyond help(obj) not working as 
expected: doctests which I thought were running aren't.

Magic behaviour like this is nasty because it breaks the Rule of Least 
Astonishment. I thought I understood Python's object model and it's 
rules of inheritance, but not only do double-underscore attributes 
follow a different rule for inheritance than other attributes, but 
__doc__ follows a different rule than other double-underscore 
attributes.


-- 
Steven D'Aprano

From brett at python.org  Fri Oct 23 06:47:06 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 22 Oct 2009 21:47:06 -0700
Subject: [Python-Dev] language summit topic: issue tracker
Message-ID: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>

Another summit, another potential time to see if people want to change
anything about the issue tracker. I would bring up:

- Dropping Stage in favor of some keywords (e.g. 'needs unit test', 'needs
docs')
- Adding a freestyle text box to delineate which, if any, stdlib module is
the cause of a bug and tie that into Misc/maintainers.rst; would potentially
scale back the Component box

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091022/a5311a6e/attachment.htm>

From ben+python at benfinney.id.au  Fri Oct 23 07:51:03 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 23 Oct 2009 16:51:03 +1100
Subject: [Python-Dev] language summit topic: issue tracker
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
Message-ID: <871vku7j5k.fsf@benfinney.id.au>

Brett Cannon <brett at python.org> writes:

> Another summit, another potential time to see if people want to change
> anything about the issue tracker.

It requires some coding, but I see OpenID authentication support
<URL:http://issues.roundup-tracker.org/issue2550523> to be important for
lowering the barrier to getting bug reports.

-- 
 \        ?I was once walking through the forest alone and a tree fell |
  `\       right in front of me, and I didn't hear it.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From fperez.net at gmail.com  Fri Oct 23 08:07:41 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 23 Oct 2009 06:07:41 +0000 (UTC)
Subject: [Python-Dev] nonlocal keyword in 2.x?
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<cfb578b20910220832n2d68f88cs76a4561c0da3f049@mail.gmail.com>
Message-ID: <hbrh7c$ere$1@ger.gmane.org>

On Thu, 22 Oct 2009 12:32:37 -0300, Fabio Zadrozny wrote:


> Just as a note, the nonlocal there is not a requirement...
> 
> You can just create a mutable object there and change that object (so,
> you don't need to actually rebind the object in the outer scope).
> 
> E.g.: instead of creating a float in the context, create a list with a
> single float and change the float in the list (maybe the nonlocal would
> be nicer, but it's certainly still usable)

Yup, that's what I meant by 'some slightly ugly solutions' in this note:

http://mail.scipy.org/pipermail/ipython-dev/2009-September/005529.html

in the thread that spawned those notes.  nonlocal allows for this pattern 
to work without the ugliness of writing code like:

s = [s]

@somedeco
def foo():
  s[0] += 1

s = s[0]

just to be able to 'change s' inside the foo() scope.


I felt this was both obvious and ugly enough not to warrant too much 
explicit mention, but I probably should have left it there for the sake of 
completeness.  Thanks for the feedback.

Cheers,

f

ps - the above shouldn't be taken as either pro or con on the idea of 
nonlocal in 2.x, just a clarification on why I didn't add the mutable 
container trick to the original notes.


From mbk.lists at gmail.com  Fri Oct 23 09:30:00 2009
From: mbk.lists at gmail.com (Mike Krell)
Date: Fri, 23 Oct 2009 00:30:00 -0700
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE0BF83.4080903@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
Message-ID: <da7032ce0910230030o1f95a77esb8ad706aca7dcd36@mail.gmail.com>

On Thu, Oct 22, 2009 at 1:24 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Can you please explain why it would be desirable
> to [backport nonlocal]? 2.7 will likely be the last 2.x release, so only a
> fairly
> small portion of the applications would be actually able to use this (or
> any other new feature added to 2.7): most code supporting 2.x will also
> have to support 2.6, so the keyword won't be available to such code,
> anyway.
>

Others have explained the rationale for the backport, so I won't bother
repeating those arguments.

I understand your point about code supporting 2.6, but as you say, that
applies to any new features being added in 2.7.  I'm therefore confused as
to what the rationale for a 2.7 release is.  It's a bump in minor release
number, so it's purpose is to add new features, correct?

Could someone please explain what new features are currently envisioned for
2.7?  Why would those make the cut but a not backport of nonlocal?

   Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/09838cb0/attachment.htm>

From martin at v.loewis.de  Fri Oct 23 10:15:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 23 Oct 2009 10:15:21 +0200
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <da7032ce0910230030o1f95a77esb8ad706aca7dcd36@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<4AE0BF83.4080903@v.loewis.de>
	<da7032ce0910230030o1f95a77esb8ad706aca7dcd36@mail.gmail.com>
Message-ID: <4AE16619.2030100@v.loewis.de>

> Others have explained the rationale for the backport, so I won't bother
> repeating those arguments.
> 
> I understand your point about code supporting 2.6, but as you say, that
> applies to any new features being added in 2.7.  I'm therefore confused
> as to what the rationale for a 2.7 release is.  It's a bump in minor
> release number, so it's purpose is to add new features, correct?

That, but not only - it's purpose is also to allow for certain
incompatible changes, and to arrive at a version that will be maintained
until 2015 (wrt. security fixes).

> Could someone please explain what new features are currently envisioned
> for 2.7?

Read through Misc/NEWS for details of what has been added already.

> Why would those make the cut but a not backport of nonlocal?

Just to name a few changes:
- Issue #1655: Make imaplib IPv6-capable. Patch by Derek Morr.
- Issue #4915: Port sysmodule to Windows CE.
- Issue #6101: A new opcode, SETUP_WITH, has been added to speed up the
  with statement and correctly lookup the __enter__ and __exit__
  special methods.
- Issue #1616979: Added the cp720 (Arabic DOS) encoding.

In all of these cases, in order to use the new feature, no change to
Python code will be necessary: the feature just becomes available
transparently.

This is the kind of new features I like to see in 2.7.
The other kind (i.e. features that do require application changes
in order to use them) I find questionable.

Regards,
Martin

From ncoghlan at gmail.com  Fri Oct 23 11:46:00 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Oct 2009 19:46:00 +1000
Subject: [Python-Dev] readonly __doc__
In-Reply-To: <200910231313.55925.steve@pearwood.info>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>	<198E579C-0FFF-4CF7-AC5A-9D0AB2CEA509@gmail.com>	<bbaeab100910221139j1c147785m97f9951f8802cdb4@mail.gmail.com>
	<200910231313.55925.steve@pearwood.info>
Message-ID: <4AE17B58.1060306@gmail.com>

Steven D'Aprano wrote:
> On Fri, 23 Oct 2009 05:39:53 am Brett Cannon wrote:
> 
>>> Is __doc__ not normal due to its general underscorishness, or is it
>>> not normal because it isn't?
>> I honestly don't follow that sentence. But __doc__ is special because
>> of its use; documenting how to use of an object. In this case when
>> you call something like help() on an instance of an object it skips
>> the instance's value for __doc__ and goes straight to the defining
>> class and stops there as you don't care how a subclass says to use
>> itself as that is not what you are working with.
> 
> Classes don't normally inherit behaviour from their subclasses. 
> Presumably you meant superclass.
> 
> I expected __doc__ to be just like the other double-underscore objects: 
> lookup skips the instance and goes to the class, then follows the 
> normal rules of inheritance. Consequently I've just discovered that a 
> few of my classes, which I assumed inherited __doc__, don't actually 
> have a docstring. This has consequences beyond help(obj) not working as 
> expected: doctests which I thought were running aren't.
> 
> Magic behaviour like this is nasty because it breaks the Rule of Least 
> Astonishment. I thought I understood Python's object model and it's 
> rules of inheritance, but not only do double-underscore attributes 
> follow a different rule for inheritance than other attributes, but 
> __doc__ follows a different rule than other double-underscore 
> attributes.

This actually breaks a recipe I recently suggested in python-ideas,
where I recommended inheriting from a namedtuple instance in order to
insert additional argument validity checks in __new__.

Probably not invalid for it to break there though - in the case of that
recipe, the default docstring from the parent class really should be
replaced with one that details the additional constraints being placed
on the arguments. Then again, the parent class docstring wouldn't have
been wrong - merely incomplete.

The point you mentioned about doctests silently failing to run strikes
me as a pretty major glitch due to this behaviour though. Having a base
class that defines the doctests to run in its docstring and then
creating subclasses to specialise the test execution sounds like a
perfectly reasonable use case to me (even though I personally tend to
write unittest based test rather than doctest based ones).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Fri Oct 23 11:49:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Oct 2009 19:49:06 +1000
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
Message-ID: <4AE17C12.1080007@gmail.com>

Brett Cannon wrote:
> Another summit, another potential time to see if people want to change
> anything about the issue tracker. I would bring up:
> 
> - Dropping Stage in favor of some keywords (e.g. 'needs unit test',
> 'needs docs')
> - Adding a freestyle text box to delineate which, if any, stdlib module
> is the cause of a bug and tie that into Misc/maintainers.rst; would
> potentially scale back the Component box

+lots on adding a module field (independent of automatically adding
maintainers to the nosy list, it would assist in "I just did a major
cleanup of module X, are there any old bugs I can kill off").

Of course, it will take a while for the field to be filled in on
existing issues, but even having it be possible would be very nice.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From w.richert at gmx.net  Fri Oct 23 11:32:45 2009
From: w.richert at gmx.net (Willi Richert)
Date: Fri, 23 Oct 2009 11:32:45 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
Message-ID: <200910231132.45504.w.richert@gmx.net>

Hi,

recently I wrote an algorithm, in which very often I had to get an arbitrary 
element from a set without removing it.

Three possibilities came to mind:

1. 
x = some_set.pop()
some_set.add(x)

2. 
for x in some_set: 
	break

3.
x = iter(some_set).next()


Of course, the third should be the fastest. It nevertheless goes through all 
the iterator creation stuff, which costs some time. I wondered, why the builtin 
set does not provide a more direct and efficient way for retrieving some element 
without removing it. Is there any reason for this?

I imagine something like

x = some_set.get()

or 

x = some_set.pop(False)

and am thinking about providing a patch against setobject.c (preferring the 
.get() solution being a stripped down pop()).

Before, I would like to know whether I have overlooked something or whether 
this can be done in an already existing way.

Thanks,
wr


From w.richert at gmx.net  Fri Oct 23 12:23:08 2009
From: w.richert at gmx.net (Willi Richert)
Date: Fri, 23 Oct 2009 12:23:08 +0200
Subject: [Python-Dev] First shot at some_set.get()
Message-ID: <200910231223.08470.w.richert@gmx.net>

Hi,

here is the first shot to provide a faster means of retrieving an arbitrary 
element from a set without removing it.

The times for 

=========================
from timeit import *

stat1 = "for i in xrange(100): iter(s).next()"
stat2 = "for i in xrange(100): s.get()"

for stat in [stat1, stat2]:
    t = Timer(stat, setup="s=set(range(10000))")       # outside the 
try/except
    try:
        print t.timeit(number=1000) 
    except:
        t.print_exc()
=========================

are 

stat1: 0.0451760292053
stat2: 0.0148510932922

The patch is attached. Feel free to criticize.

I would love to see something like that in the standard Python interpreter.

Regards,
wr
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setobject_get.patch
Type: text/x-patch
Size: 2211 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/1e025bce/attachment.bin>

From phd at phd.pp.ru  Fri Oct 23 12:37:09 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Fri, 23 Oct 2009 14:37:09 +0400
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <200910231223.08470.w.richert@gmx.net>
References: <200910231223.08470.w.richert@gmx.net>
Message-ID: <20091023103709.GA29267@phd.pp.ru>

On Fri, Oct 23, 2009 at 12:23:08PM +0200, Willi Richert wrote:
> The patch is attached.

   Patches should be put to the issue tracker. Thank you.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From steve at pearwood.info  Fri Oct 23 12:54:52 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 23 Oct 2009 21:54:52 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910231132.45504.w.richert@gmx.net>
References: <200910231132.45504.w.richert@gmx.net>
Message-ID: <200910232154.53188.steve@pearwood.info>

On Fri, 23 Oct 2009 08:32:45 pm Willi Richert wrote:
> Hi,
>
> recently I wrote an algorithm, in which very often I had to get an
> arbitrary element from a set without removing it.
>
> Three possibilities came to mind:
...
> Of course, the third should be the fastest.

If you need one or more items chosen randomly without replacement, use a 
list:

L = list(some_set)
x = random.choice(L)

If you don't need a randomly chosen item, merely an arbitrary item, you 
can still use a list but avoid the call to random.choice:

x = list(some_set)[0]

> It nevertheless goes 
> through all the iterator creation stuff, which costs some time. I
> wondered, why the builtin set does not provide a more direct and
> efficient way for retrieving some element without removing it. Is
> there any reason for this?

I can see it being useful to iterate over the entire set, without 
removing anything. I can see it being useful to pop an arbitrary item, 
and remove it. But I can't see the use for retrieving an arbitrary 
item, leaving it in the set. What do you use this for?


> I imagine something like
>
> x = some_set.get()
>
> or
>
> x = some_set.pop(False)
>
> and am thinking about providing a patch against setobject.c
> (preferring the .get() solution being a stripped down pop()).

-1 on .pop(flag)

+0 on .get()



-- 
Steven D'Aprano

From steve at pearwood.info  Fri Oct 23 13:09:50 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 23 Oct 2009 22:09:50 +1100
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <20091023103709.GA29267@phd.pp.ru>
References: <200910231223.08470.w.richert@gmx.net>
	<20091023103709.GA29267@phd.pp.ru>
Message-ID: <200910232209.50723.steve@pearwood.info>

On Fri, 23 Oct 2009 09:37:09 pm Oleg Broytman wrote:
> On Fri, Oct 23, 2009 at 12:23:08PM +0200, Willi Richert wrote:
> > The patch is attached.
>
>    Patches should be put to the issue tracker. Thank you.

Is there any point? Even if accepted, it's too late to make it into 3.1, 
and with the overwhelming approval for a moratorium on changes to 
built-ins, it is likely to just sit in the tracker, forgotten, until 
2013 or later. How likely is it that the patch will still be 
applicable?


-- 
Steven D'Aprano

From ubershmekel at gmail.com  Fri Oct 23 13:19:18 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 23 Oct 2009 13:19:18 +0200
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <200910232209.50723.steve@pearwood.info>
References: <200910231223.08470.w.richert@gmx.net>
	<20091023103709.GA29267@phd.pp.ru>
	<200910232209.50723.steve@pearwood.info>
Message-ID: <9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>

On Fri, Oct 23, 2009 at 1:09 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Is there any point? Even if accepted, it's too late to make it into 3.1,
> and with the overwhelming approval for a moratorium on changes to
> built-ins, it is likely to just sit in the tracker, forgotten, until
> 2013 or later. How likely is it that the patch will still be
> applicable?


+1 on throwing it away completely even if it's a good idea. I suggest
Willi go invent a new language (and hope for it to become popular :-)
if he wants to experiment.

--yuv

From kristjan at ccpgames.com  Fri Oct 23 13:48:25 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Fri, 23 Oct 2009 11:48:25 +0000
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <4AE0C54F.20705@gmail.com>
References: <4ADE2AA9.4030604@molden.no>
	<loom.20091021T125010-360@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>
	<1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no>
	<loom.20091021T191933-390@post.gmane.org> <4ADF481B.90207@gmail.com>
	<1256147726.5069.50.camel@localhost>
	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>
	<loom.20091021T225002-378@post.gmane.org>
	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F098FF4213E@exchis.ccp.ad.local>
	<4AE0C54F.20705@gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF4246E@exchis.ccp.ad.local>

Thanks, I'll take a look in that direction.

> -----Original Message-----
> From: Nick Coghlan [mailto:ncoghlan at gmail.com]
> I've even played with using Kalman filtering to do it... The idea is
> > to use the low frequency timer to apply correction coefficients to
> > the high frequency timer, yet keep the flow of time smooth (no
> > backwards jumps because of corrections.).  An optimal solution has so
> > far eluded me.
> 
> That sounds very similar to the problem spec for system time
> corrections
> in Network Time Protocol client implementations. Perhaps the time
> drifting algorithms in the NTP specs are relevant? Or are they too slow
> to correct discrepancies?

K

From solipsis at pitrou.net  Fri Oct 23 14:24:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 23 Oct 2009 12:24:46 +0000 (UTC)
Subject: [Python-Dev] language summit topic: issue tracker
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
Message-ID: <hbs7ad$if1$1@ger.gmane.org>

Le Thu, 22 Oct 2009 21:47:06 -0700, Brett Cannon a ?crit?:
> 
> - Dropping Stage in favor of some keywords (e.g. 'needs unit test',
> 'needs docs')

What would it bring? We don't have a very strict process and the current 
"stage" looks sufficient to me. Saying that unit tests or docs are 
lacking is part of the review and doesn't need a specific indicator IMO.

Besides, the more keywords there are, the messier it is.


From ncoghlan at gmail.com  Fri Oct 23 14:47:22 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Oct 2009 22:47:22 +1000
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>
References: <200910231223.08470.w.richert@gmx.net>	<20091023103709.GA29267@phd.pp.ru>	<200910232209.50723.steve@pearwood.info>
	<9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>
Message-ID: <4AE1A5DA.40603@gmail.com>

Yuvgoog Greenle wrote:
> On Fri, Oct 23, 2009 at 1:09 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> Is there any point? Even if accepted, it's too late to make it into 3.1,
>> and with the overwhelming approval for a moratorium on changes to
>> built-ins, it is likely to just sit in the tracker, forgotten, until
>> 2013 or later. How likely is it that the patch will still be
>> applicable?
> 
> 
> +1 on throwing it away completely even if it's a good idea. I suggest
> Willi go invent a new language (and hope for it to become popular :-)
> if he wants to experiment.

Careful folks - these kinds of throwaway comments may be clearly tongue
in cheek for anyone following the moratorium discussion on python-ideas,
but will be justifiably confusing to anyone else, especially newcomers.

Willi - Oleg is right that patches should go on the issue tracker. They
tend to get lost if they only exist on the mailing list.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From skip at pobox.com  Fri Oct 23 15:00:25 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 23 Oct 2009 08:00:25 -0500
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
Message-ID: <19169.43241.70138.170239@montanaro.dyndns.org>

    Brett> Another summit, another potential time to see if people want to
    Brett> change anything about the issue tracker.

I have no idea how hard this would be to implement and won't be at the
language summit to formally present the idea, but it seems to me that some
integration between the issue tracker and Rietveld would be beneficial.  If
someone attaches a patch to an issue the current next step is essentially a
code review without the benefits provided by a code review tool.  I'd
envision a bit of workflow like this:

    * A patch is attached to an issue.
    * The user clicks the "Create Review" button.  (Maybe not all patches
      require review?)
    * This generates a review request in Rietveld with all on the nosy list
      invited as reviewers.  (Or should this be a side effect of attaching a
      patch?)
    * The "needs review" keyword is added to the selected keywords.
    * A link to the review request is added as a comment to the issue so
      other people not on the nosy list can evaluate the patch.
    * If an updated diff is uploaded the review request would be updated.
      That might necessitate adding a "Replace Patch" button next to all
      uploaded patches instead of adding a new one then deleting a previous
      one.

Skip

From stephen at xemacs.org  Fri Oct 23 16:18:39 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 23 Oct 2009 23:18:39 +0900
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <hbs7ad$if1$1@ger.gmane.org>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
	<hbs7ad$if1$1@ger.gmane.org>
Message-ID: <87tyxqb3cw.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > Besides, the more keywords there are, the messier it is.

That's what I've found in the XEmacs tracker.  Keywords are a
reasonable way (in the context of the Roundup implementation) to test
new classifications before going to the effort of messing with the
page templates.  But if you don't think "stage" is needed, I'd say
drop it entirely rather than demote it to keywords.

Keywords themselves are rather confusing to non-tracker-hackers,
anyway.  A lot of people seem to think anything that occurs to them
should be allowed.  That's not true in Roundup, you need to register a
keyword before using it.

In the XEmacs tracker I don't allow non-committers to do that.  I'm
open to changing that policy, but as yet I haven't seen a keyword
suggestion from a non-committer that either (1) helps the committers
to do their work or (2) seems likely to help users find relevant bugs.
The suggestions are always of the form "it would make the interface
more complete/consistent."  So I'm going to maintain editorial control
for now....


From van.lindberg at gmail.com  Fri Oct 23 17:49:40 2009
From: van.lindberg at gmail.com (VanL)
Date: Fri, 23 Oct 2009 10:49:40 -0500
Subject: [Python-Dev] Volunteer needed to organize summits
In-Reply-To: <20091020114325.GB6355@amk-desktop.matrixgroup.net>
References: <20091020114325.GB6355@amk-desktop.matrixgroup.net>
Message-ID: <hbsjak$ta9$1@ger.gmane.org>

Have you had any bites?

Thanks,

Van


From status at bugs.python.org  Fri Oct 23 18:07:40 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 23 Oct 2009 18:07:40 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091023160740.7468A7864F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/16/09 - 10/23/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2474 open (+27) / 16541 closed (+16) / 19015 total (+43)

Open issues with patches:   987

Average duration of open issues: 673 days.
Median duration of open issues: 428 days.

Open Issues Breakdown
   open  2436 (+27)
pending    37 ( +0)

Issues Created Or Reopened (43)
_______________________________

regrtest -j sometimes fails if output gets written to stderr by  10/16/09
CLOSED http://bugs.python.org/issue7151    created  r.david.murray                
       patch, patch                                                            

urllib2.build_opener() skips ProxyHandler                        10/16/09
       http://bugs.python.org/issue7152    created  barry                         
                                                                               

add "start" arg to max and min functions                         10/16/09
CLOSED http://bugs.python.org/issue7153    created  phr                           
                                                                               

urllib.request system proxy configuration lookup broken for OS X 10/16/09
       http://bugs.python.org/issue7154    created  ned.deily                     
                                                                               

urllib2 and python 3 urllib do not document default use of syste 10/16/09
CLOSED http://bugs.python.org/issue7155    created  ned.deily                     
       patch                                                                   

curses can't find _curses                                        10/17/09
CLOSED http://bugs.python.org/issue7156    created  Archon                        
                                                                               

Fix Download Current Documentation link                          10/17/09
       http://bugs.python.org/issue7157    created  ryles                         
                                                                               

os.path.basename/split fails                                     10/17/09
CLOSED http://bugs.python.org/issue7158    created  kuiper                        
                                                                               

Urllib2 authentication memory.                                   10/17/09
       http://bugs.python.org/issue7159    created  bradobro                      
                                                                               

Crash when returning a 64-bit char pointer in Python 2.6.3 ctype 10/17/09
CLOSED http://bugs.python.org/issue7160    created  creachadair                   
                                                                               

raise of SyntaxError in codeop was ported incorrectly to Py3     10/17/09
CLOSED http://bugs.python.org/issue7161    created  Trundle                       
       patch                                                                   

2to3 does not convert __builtins__.file                          10/17/09
       http://bugs.python.org/issue7162    created  joe.amenta                    
                                                                               

IDLE suppresses sys.stdout.write() return value                  10/17/09
       http://bugs.python.org/issue7163    created  tjreedy                       
                                                                               

pickle test failure after test_imp/test_import (_make_stat_resul 10/18/09
CLOSED http://bugs.python.org/issue7164    created  r.david.murray                
                                                                               

xmlrpc.server assumes sys.stdout will have a buffer attribute    10/18/09
       http://bugs.python.org/issue7165    created  ncoghlan                      
       patch                                                                   

IDLE (python 3.1.1) syntax coloring for b'bytestring' and u'unic 10/18/09
       http://bugs.python.org/issue7166    created  lieryan                       
       easy                                                                    

Smarter FTP passive mode                                         10/18/09
       http://bugs.python.org/issue7167    created  pitrou                        
       patch                                                                   

Document PyFloat_AsString and PyFloat_AsReprString as deprecated 10/19/09
CLOSED http://bugs.python.org/issue7168    created  eric.smith                    
       patch                                                                   

zipfile leaves a file handle open if file is zero size           10/19/09
       http://bugs.python.org/issue7169    created  skelker                       
                                                                               

subclasses of (some) built-in types are not weakref-able         10/19/09
CLOSED http://bugs.python.org/issue7170    created  stutzbach                     
                                                                               

Add inet_ntop and inet_pton support for Windows                  10/19/09
       http://bugs.python.org/issue7171    created  jaraco                        
                                                                               

BaseHTTPServer.BaseHTTPRequestHandler.responses[405] has a small 10/19/09
       http://bugs.python.org/issue7172    created  dalke                         
                                                                               

Cython compiler run crashes CPython 3.1.1 and 3.2                10/20/09
       http://bugs.python.org/issue7173    created  scoder                        
       patch                                                                   

modeule: queue class: PriorityQueue                              10/20/09
CLOSED http://bugs.python.org/issue7174    created  asuiu                         
                                                                               

unify pydistutils.cfg and distutils.cfg and use .local           10/20/09
       http://bugs.python.org/issue7175    created  tarek                         
                                                                               

sum() doesn't work for lists.                                    10/20/09
CLOSED http://bugs.python.org/issue7176    created  oggust                        
                                                                               

Unclear warning for subprocess.call                              10/20/09
       http://bugs.python.org/issue7177    created  azumanga                      
                                                                               

open function's buffering parameter is not completely doc'ed     10/20/09
CLOSED http://bugs.python.org/issue7178    created  kuhnsjohn                     
                                                                               

Unportable test(1) construct                                     10/21/09
       http://bugs.python.org/issue7179    created  wiz                           
       patch                                                                   

"pydoc -k" can generate AttributeError on Mac OS X               10/21/09
       http://bugs.python.org/issue7180    created  kmgrant                       
                                                                               

No logging when two loggers with same qualname                   10/21/09
CLOSED http://bugs.python.org/issue7181    created  cristiroma                    
                                                                               

For non-debug builds, the cygwinccompiler.py should define NDEBU 10/21/09
       http://bugs.python.org/issue7182    created  stutzbach                     
                                                                               

did 2.6.3 regress for some uses of the __doc__ property?         10/21/09
       http://bugs.python.org/issue7183    created  zooko                         
                                                                               

build failures on Snow Leopard                                   10/22/09
       http://bugs.python.org/issue7184    created  richard                       
                                                                               

csv reader utf-8 BOM error                                       10/22/09
       http://bugs.python.org/issue7185    created  W00D00                        
                                                                               

Document specialness of __doc__, and possibly other "special" at 10/22/09
       http://bugs.python.org/issue7186    created  ssteiner                      
                                                                               

importlib/_bootstrap.py write_bytecode raises an IOError if it c 10/22/09
       http://bugs.python.org/issue7187    created  dmalcolm                      
       patch                                                                   

optionxform documentation confusing                              10/23/09
CLOSED http://bugs.python.org/issue7188    created  loewis                        
                                                                               

struct.calcsize returns strange size                             10/23/09
CLOSED http://bugs.python.org/issue7189    created  igor.mikushkin                
                                                                               

TCP/IP?                                                          10/23/09
       http://bugs.python.org/issue7190    created  jlr2383                       
                                                                               

Odd behaviour with zlib.decompressobj optional parameter "wbits" 10/23/09
       http://bugs.python.org/issue7191    created  pythonhacker                  
                                                                               

webbrowser.get("firefox") does not work on Mac with installed Fi 10/23/09
       http://bugs.python.org/issue7192    created  dalke                         
       patch                                                                   

Popen blocks program from another thread                         10/23/09
       http://bugs.python.org/issue7193    created  dgriff1                       
                                                                               



Issues Now Closed (41)
______________________

fromfd() and dup() for _socket on WIndows                         716 days
       http://bugs.python.org/issue1378    planders                      
       patch                                                                   

GC optimization: don't track simple tuples and dicts              303 days
       http://bugs.python.org/issue4688    esam                          
       patch                                                                   

optimize bytecode for conditional branches                        299 days
       http://bugs.python.org/issue4715    esam                          
       patch                                                                   

Bugs in _ssl object read() when a buffer is specified             278 days
       http://bugs.python.org/issue4967    pitrou                        
       patch                                                                   

"in" expression falls back to __iter__ before __getitem__         122 days
       http://bugs.python.org/issue6324    georg.brandl                  
                                                                               

Printing the 'The Python Tutorial'                                 76 days
       http://bugs.python.org/issue6670    brimac                        
                                                                               

uuid creates zombies                                               42 days
       http://bugs.python.org/issue6882    eric.smith                    
       patch                                                                   

Metaclass doc (Ref 3.3.3) errors                                   36 days
       http://bugs.python.org/issue6927    georg.brandl                  
                                                                               

Getopt documentation ambiguity                                     29 days
       http://bugs.python.org/issue6977    georg.brandl                  
                                                                               

codecs error handlers lack documentation                           20 days
       http://bugs.python.org/issue7035    georg.brandl                  
                                                                               

Doc/reference/datamodel: Slots description needs update            20 days
       http://bugs.python.org/issue7036    georg.brandl                  
       patch                                                                   

urllib.urlopen crashes when used on Mac OS 10.6.1 through a prox   15 days
       http://bugs.python.org/issue7044    ronaldoussoren                
                                                                               

No docs for module 'IN'                                            17 days
       http://bugs.python.org/issue7062    georg.brandl                  
                                                                               

Python 2.6.3 / setuptools 0.6c9: extension module builds fail wi    6 days
       http://bugs.python.org/issue7064    ned.deily                     
                                                                               

Suggest better documentation of 2 versus 3 on 3.1.1 site.          17 days
       http://bugs.python.org/issue7067    srl                           
                                                                               

locale.strxfrm raises MemoryError                                  12 days
       http://bugs.python.org/issue7080    pitrou                        
       patch                                                                   

New (in 2.6) functions in signal module are not documented as un   13 days
       http://bugs.python.org/issue7088    georg.brandl                  
                                                                               

Problems building pyhton from source on Snow Leopard (Mac OS X      7 days
       http://bugs.python.org/issue7102    ronaldoussoren                
                                                                               

regrtest -j fails when tests write to stderr                        4 days
       http://bugs.python.org/issue7127    r.david.murray                
       easy                                                                    

test_ssl failure                                                    5 days
       http://bugs.python.org/issue7133    pitrou                        
       patch                                                                   

Socket documentation not updated                                    7 days
       http://bugs.python.org/issue7137    georg.brandl                  
                                                                               

Remove WITHOUT_COMPLEX from 3.x trunk                               3 days
       http://bugs.python.org/issue7147    skip.montanaro                
       patch, patch, easy, needs review                                        

2.6.4rc1 regression: test_urllib2 fails on OS X with UnboundLoca    3 days
       http://bugs.python.org/issue7149    ned.deily                     
                                                                               

regrtest -j sometimes fails if output gets written to stderr by     3 days
       http://bugs.python.org/issue7151    r.david.murray                
       patch, patch                                                            

add "start" arg to max and min functions                            2 days
       http://bugs.python.org/issue7153    rhettinger                    
                                                                               

urllib2 and python 3 urllib do not document default use of syste    1 days
       http://bugs.python.org/issue7155    orsenthil                     
       patch                                                                   

curses can't find _curses                                           6 days
       http://bugs.python.org/issue7156    georg.brandl                  
                                                                               

os.path.basename/split fails                                        0 days
       http://bugs.python.org/issue7158    ezio.melotti                  
                                                                               

Crash when returning a 64-bit char pointer in Python 2.6.3 ctype    2 days
       http://bugs.python.org/issue7160    creachadair                   
                                                                               

raise of SyntaxError in codeop was ported incorrectly to Py3        0 days
       http://bugs.python.org/issue7161    benjamin.peterson             
       patch                                                                   

pickle test failure after test_imp/test_import (_make_stat_resul    0 days
       http://bugs.python.org/issue7164    ncoghlan                      
                                                                               

Document PyFloat_AsString and PyFloat_AsReprString as deprecated    1 days
       http://bugs.python.org/issue7168    eric.smith                    
       patch                                                                   

subclasses of (some) built-in types are not weakref-able            1 days
       http://bugs.python.org/issue7170    georg.brandl                  
                                                                               

modeule: queue class: PriorityQueue                                 0 days
       http://bugs.python.org/issue7174    r.david.murray                
                                                                               

sum() doesn't work for lists.                                       0 days
       http://bugs.python.org/issue7176    rhettinger                    
                                                                               

open function's buffering parameter is not completely doc'ed        0 days
       http://bugs.python.org/issue7178    benjamin.peterson             
                                                                               

No logging when two loggers with same qualname                      0 days
       http://bugs.python.org/issue7181    vinay.sajip                   
                                                                               

optionxform documentation confusing                                 0 days
       http://bugs.python.org/issue7188    georg.brandl                  
                                                                               

struct.calcsize returns strange size                                0 days
       http://bugs.python.org/issue7189    mark.dickinson                
                                                                               

os.path, %HOME% set: realpath contradicts expanduser on '~'       993 days
       http://bugs.python.org/issue1646838 ezio.melotti                  
                                                                               

PyThreadState_SetAsyncExc and the main thread                     788 days
       http://bugs.python.org/issue1779233 pitrou                        
                                                                               



Top Issues Most Discussed (10)
______________________________

 10 add ftp-tls support to ftplib - RFC 4217                         623 days
open    http://bugs.python.org/issue2054   

  9 did 2.6.3 regress for some uses of the __doc__ property?           2 days
open    http://bugs.python.org/issue7183   

  9 unify pydistutils.cfg and distutils.cfg and use .local             3 days
open    http://bugs.python.org/issue7175   

  9 Add gamma function, error functions and other C99 math.h functi  465 days
open    http://bugs.python.org/issue3366   

  6 add "start" arg to max and min functions                           2 days
closed  http://bugs.python.org/issue7153   

  6 locale.strxfrm raises MemoryError                                 12 days
closed  http://bugs.python.org/issue7080   

  5 subclasses of (some) built-in types are not weakref-able           1 days
closed  http://bugs.python.org/issue7170   

  5 os.path.basename/split fails                                       0 days
closed  http://bugs.python.org/issue7158   

  5 2.6.4rc1 regression: test_urllib2 fails on OS X with UnboundLoc    3 days
closed  http://bugs.python.org/issue7149   

  5 deprecated conversion from string constant to char *              33 days
open    http://bugs.python.org/issue6952   




From fuzzyman at voidspace.org.uk  Fri Oct 23 18:49:58 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 23 Oct 2009 17:49:58 +0100
Subject: [Python-Dev] Volunteer needed to organize summits
In-Reply-To: <hbsjak$ta9$1@ger.gmane.org>
References: <20091020114325.GB6355@amk-desktop.matrixgroup.net>
	<hbsjak$ta9$1@ger.gmane.org>
Message-ID: <4AE1DEB6.7070303@voidspace.org.uk>

VanL wrote:
> Have you had any bites?
>

I'm going to help Andrew with the invites and working out agendas. He's 
sent me a bunch of stuff to get me started.

Michael

> Thanks,
>
> Van
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From algorias at gmail.com  Fri Oct 23 19:04:33 2009
From: algorias at gmail.com (Vitor Bosshard)
Date: Fri, 23 Oct 2009 14:04:33 -0300
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910231132.45504.w.richert@gmx.net>
References: <200910231132.45504.w.richert@gmx.net>
Message-ID: <2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>

2009/10/23 Willi Richert <w.richert at gmx.net>:
> Hi,
>
> recently I wrote an algorithm, in which very often I had to get an arbitrary
> element from a set without removing it.
>
> Three possibilities came to mind:
>
> 1.
> x = some_set.pop()
> some_set.add(x)
>
> 2.
> for x in some_set:
> ? ? ? ?break
>
> 3.
> x = iter(some_set).next()
>
>
> Of course, the third should be the fastest. It nevertheless goes through all
> the iterator creation stuff, which costs some time. I wondered, why the builtin
> set does not provide a more direct and efficient way for retrieving some element
> without removing it. Is there any reason for this?
>
> I imagine something like
>
> x = some_set.get()
>


I see this as being useful for frozensets as well, where you can't get
an arbitrary element easily due to the obvious lack of .pop(). I ran
into this recently, when I had a frozenset that I knew had 1 element
(it was the difference between 2 other sets), but couldn't get to that
element easily (get the pun?)

From sturla at molden.no  Fri Oct 23 19:18:46 2009
From: sturla at molden.no (Sturla Molden)
Date: Fri, 23 Oct 2009 19:18:46 +0200
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF4246E@exchis.ccp.ad.local>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>
	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>
	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>	<930F189C8A437347B80DF2C156F7EC7F098FF4213E@exchis.ccp.ad.local>	<4AE0C54F.20705@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F098FF4246E@exchis.ccp.ad.local>
Message-ID: <4AE1E576.5000903@molden.no>

Kristj?n Valur J?nsson skrev:
> Thanks, I'll take a look in that direction.
>
>   
I have a suggestion, forgive me if I am totally ignorant. :-)

Sturla Molden



#include <windows.h>

union __reftime {
    double   us;
    __int64  bits;
};

static volatile union __reftime __ref_perftime, __ref_filetime;


double clock()
{
    __int64 cnt, hz, init;
    double us;
    union __reftime ref_filetime;
    union __reftime ref_perftime; 
    for (;;) {
        ref_filetime.bits = __ref_filetime.bits;
        ref_perftime.bits = __ref_perftime.bits;
        if(!QueryPerformanceCounter((LARGE_INTEGER*)&cnt)) goto error;
        if(!QueryPerformanceFrequency((LARGE_INTEGER*)&hz)) goto error;
        us = ref_filetime.us + ((double)(1000000*cnt)/(double)hz - 
ref_perftime.us);
        /* verify that values did not change */
        init = InterlockedCompareExchange64((LONGLONG*)&__ref_filetime.bits,
                                            (LONGLONG)ref_filetime.bits,
                                            (LONGLONG)ref_filetime.bits);
        if (init != ref_filetime.bits) continue;
        init = InterlockedCompareExchange64((LONGLONG*)&__ref_perftime.bits,
                                            (LONGLONG)ref_perftime.bits,
                                            (LONGLONG)ref_perftime.bits);
        if (init == ref_perftime.bits) break;
    }
    return us;
error:
    /* only if there is no performance counter */
    return -1; /* or whatever */
}


int periodic_reftime_check()
{
    /* call this function at regular intervals, e.g. once every second 
*/   
    __int64 cnt1, cnt2, hz;
    FILETIME systime;
    double ft;
    if(!QueryPerformanceFrequency((LARGE_INTEGER*)&hz)) goto error;
    if(!QueryPerformanceCounter((LARGE_INTEGER*)&cnt1)) goto error;
    GetSystemTimeAsFileTime(&systime);
    __ref_filetime.us = (double)(((((__int64)(systime.dwHighDateTime)) 
<< 32)
            | ((__int64)(systime.dwLowDateTime)))/10); 
    if(!QueryPerformanceCounter((LARGE_INTEGER*)&cnt2)) goto error;
    __ref_perftime.us = 500000*(cnt1 + cnt2)/((double)hz);
    return 0;
error:
    /* only if there is no performance counter */
    return -1;   
}










From john.arbash.meinel at gmail.com  Fri Oct 23 19:25:48 2009
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Fri, 23 Oct 2009 12:25:48 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
Message-ID: <4AE1E71C.7020806@gmail.com>

Vitor Bosshard wrote:
> 2009/10/23 Willi Richert <w.richert at gmx.net>:
>> Hi,
>>
>> recently I wrote an algorithm, in which very often I had to get an arbitrary
>> element from a set without removing it.
>>
>> Three possibilities came to mind:
>>
>> 1.
>> x = some_set.pop()
>> some_set.add(x)
>>
>> 2.
>> for x in some_set:
>>        break
>>
>> 3.
>> x = iter(some_set).next()
>>
>>
>> Of course, the third should be the fastest. It nevertheless goes through all
>> the iterator creation stuff, which costs some time. I wondered, why the builtin
>> set does not provide a more direct and efficient way for retrieving some element
>> without removing it. Is there any reason for this?
>>
>> I imagine something like
>>
>> x = some_set.get()
>>
> 
> 
> I see this as being useful for frozensets as well, where you can't get
> an arbitrary element easily due to the obvious lack of .pop(). I ran
> into this recently, when I had a frozenset that I knew had 1 element
> (it was the difference between 2 other sets), but couldn't get to that
> element easily (get the pun?)

So in my testing (2) was actually the fastest. I assumed because .next()
was a function call overhead, while:
for x in some_set:
  break

Was evaluated inline. It probably still has to call PyObject_GetIter,
however it doesn't have to create a stack frame for it.

This is what "timeit" tells me. All runs are of the form:
python -m timeit -s "s = set([10])" ...

0.101us	"for x in s: break; x"
0.130us "for x in s: pass; x"
0.234us -s "n = next; i = iter" "x = n(i(s)); x"
0.248us "x = next(iter(s)); x"
0.341us "x = iter(s).next(); x"

So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
faster than (iter(s).next()).
I was pretty surprised that it was 30% faster than "for x in s: pass". I
assume it has something to do with a potential "else:" statement?

Note that all of these are significantly < 1us. So this only matters if
it is something you are doing often.

I don't know your specific timings, but I would guess that:
  for x in s: break

Is actually going to be faster than your
  s.get()

Primarily because s.get() requires an attribute lookup. I would think
your version might be faster for:
  stat2 = "g = s.get; for i in xrange(100): g()"

However, that is still a function call, which may be treated differently
by the interpreter than the for:break loop. I certainly suggest you try
it and compare.

John
=:->

From kmtracey at gmail.com  Fri Oct 23 20:05:49 2009
From: kmtracey at gmail.com (Karen Tracey)
Date: Fri, 23 Oct 2009 14:05:49 -0400
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE0D3E1.2010207@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de> <4AE0C9B6.60109@voidspace.org.uk>
	<4AE0D3E1.2010207@v.loewis.de>
Message-ID: <af3536270910231105h46dd96dbhdf4a2e425789d0fc@mail.gmail.com>

On Thu, Oct 22, 2009 at 5:51 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> > From the Django roadmap for supporting 3.0, using 2.6 as a stepping
> > stone (and if 2.7 was a *better* stepping stone then it would make it
> > easier):
> >
> >    http://groups.google.com/group/django-developers/msg/0888b1c8f2518059
> ?
>
> Is that still a current plan? It's from November 2008.
>
>
Django 1.1 came out in 2H09, not 1H09, and Django 1.2 is now looking to come
out in 1H10, not 2H09, so the dates in that note have slipped out by 3-6
months. (The labeling of some features as must-have for a release has been
dropped for 1.2, so as to hopefully prevent the kind of slip seen with
1.1.)  Django 1.2 is still scheduled to drop Python 2.3 support. I think
it's too early to say whether Python 2.4 support will be dropped with 1.3,
nor do I have any good idea when supporting 3.x will become a priority.

Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/81bab303/attachment.htm>

From p.f.moore at gmail.com  Fri Oct 23 20:13:33 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 23 Oct 2009 19:13:33 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE1E71C.7020806@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<4AE1E71C.7020806@gmail.com>
Message-ID: <79990c6b0910231113p11adc0e7g970116822090f9d6@mail.gmail.com>

2009/10/23 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> I was pretty surprised that it was 30% faster than "for x in s: pass". I
> assume it has something to do with a potential "else:" statement?

I'd imagine it's actually because it has to call next() a second time
and deal with the StopIteration exception - the loop has to end
normally, whereas the break form exits prematurely.

Paul.

From stefan_ml at behnel.de  Fri Oct 23 21:01:23 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 23 Oct 2009 21:01:23 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910231132.45504.w.richert@gmx.net>
References: <200910231132.45504.w.richert@gmx.net>
Message-ID: <hbsui3$2u3$1@ger.gmane.org>

Willi Richert wrote:
> recently I wrote an algorithm, in which very often I had to get an arbitrary 
> element from a set without removing it.

See this discussion:

http://comments.gmane.org/gmane.comp.python.ideas/5606

Stefan


From tjreedy at udel.edu  Fri Oct 23 21:04:12 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 23 Oct 2009 15:04:12 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE1E71C.7020806@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<4AE1E71C.7020806@gmail.com>
Message-ID: <hbsun9$432$1@ger.gmane.org>

John Arbash Meinel wrote:
> So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
> faster than (iter(s).next()).
> I was pretty surprised that it was 30% faster than "for x in s: pass". I
> assume it has something to do with a potential "else:" statement?

for x in s: pass

iterates through *all* the elements in s and leaves x bound to the 
arbritrary *last* one instead of the arbitrary *first* one. For a large 
set, this would be a lot slower, not just a little.

fwiw, I think the use case for this is sufficiently rare that it does 
not need a separate method just for this purpose.

tjr


From barry at python.org  Fri Oct 23 21:16:19 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 23 Oct 2009 15:16:19 -0400
Subject: [Python-Dev] Bug 7183 and Python 2.6.4
In-Reply-To: <0272128F-3703-411C-BA1B-2F7BF8AC63A4@fuhm.net>
References: <92004188-2DD4-4708-8737-0AFD98201B65@python.org>
	<1afaf6160910220747v41b441f6h69e3778b906dc96c@mail.gmail.com>
	<25FBFDCF-7D07-464E-8117-12AA0C380484@python.org>
	<hbq41h$rs1$1@ger.gmane.org> <1256241223.2929.56.camel@lifeless-64>
	<0272128F-3703-411C-BA1B-2F7BF8AC63A4@fuhm.net>
Message-ID: <484091E4-AC53-4314-BFDB-7067504AC8F1@python.org>

While I think there is some risk of exposure on this bug, I haven't  
yet heard a compelling argument for delaying 2.6.4 final for it.  I  
think we should go ahead and do the release this Sunday as planned  
with the code from 2.6.4rc2.

If you strongly disagree, please private email me.  Otherwise...  
there's always 2.6.5! :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/d86b72e3/attachment.pgp>

From john.arbash.meinel at gmail.com  Fri Oct 23 22:11:57 2009
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Fri, 23 Oct 2009 15:11:57 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <hbsun9$432$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org>
Message-ID: <4AE20E0D.4050405@gmail.com>

Terry Reedy wrote:
> John Arbash Meinel wrote:
>> So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
>> faster than (iter(s).next()).
>> I was pretty surprised that it was 30% faster than "for x in s: pass". I
>> assume it has something to do with a potential "else:" statement?
> 
> for x in s: pass
> 
> iterates through *all* the elements in s and leaves x bound to the
> arbritrary *last* one instead of the arbitrary *first* one. For a large
> set, this would be a lot slower, not just a little.
> 
> fwiw, I think the use case for this is sufficiently rare that it does
> not need a separate method just for this purpose.
> 
> tjr

The point of my test was that it was a set with a *single* item, and
'break' was 30% faster than 'pass'. Which was surprising. Certainly the
difference is huge if there are 10k items in the set.

John
=:->


From w.richert at gmx.net  Fri Oct 23 22:53:24 2009
From: w.richert at gmx.net (Willi Richert)
Date: Fri, 23 Oct 2009 22:53:24 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE1E71C.7020806@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<4AE1E71C.7020806@gmail.com>
Message-ID: <200910232253.24375.w.richert@gmx.net>

Hi,

surprised about the performance of for/break provided by Vitor, I did some 
more testing. It revealed that indeed  we can forget the get() (which was 
implemented as a stripped down pop()):

from timeit import *
stats = ["for i in xrange(1000): iter(s).next()   ",
         "for i in xrange(1000): \n\tfor x in s: \n\t\tbreak            ",
         "for i in xrange(1000): s.add(s.pop())   ",
         "for i in xrange(1000): s.get()          "]

for stat in stats:
    t = Timer(stat, setup="s=set(range(100))")
    try:
        print "Time for %s:\t %f"%(stat, t.timeit(number=1000))
    except:
        t.print_exc()



$ ./test_get.py
Time for for i in xrange(1000): iter(s).next()   :       0.433080
Time for for i in xrange(1000):
        for x in s:
                break                            :       0.148695
Time for for i in xrange(1000): s.add(s.pop())   :       0.317418
Time for for i in xrange(1000): s.get()          :       0.146673


In some tests, for/break was even slightly faster then get().

As always, intuition regarding performance bottlenecks is flawed ;-)

Anyway, thanks for all the helpful comments, especially to Stefan for the 
http://comments.gmane.org/gmane.comp.python.ideas/5606 link.

Regards,
wr

Am Freitag, 23. Oktober 2009 19:25:48 schrieb John Arbash Meinel:
> Vitor Bosshard wrote:
> > 2009/10/23 Willi Richert <w.richert at gmx.net>:
> >> Hi,
> >>
> >> recently I wrote an algorithm, in which very often I had to get an
> >> arbitrary element from a set without removing it.
> >>
> >> Three possibilities came to mind:
> >>
> >> 1.
> >> x = some_set.pop()
> >> some_set.add(x)
> >>
> >> 2.
> >> for x in some_set:
> >>        break
> >>
> >> 3.
> >> x = iter(some_set).next()
> >>
> >>
> >> Of course, the third should be the fastest. It nevertheless goes through
> >> all the iterator creation stuff, which costs some time. I wondered, why
> >> the builtin set does not provide a more direct and efficient way for
> >> retrieving some element without removing it. Is there any reason for
> >> this?
> >>
> >> I imagine something like
> >>
> >> x = some_set.get()
> >
> > I see this as being useful for frozensets as well, where you can't get
> > an arbitrary element easily due to the obvious lack of .pop(). I ran
> > into this recently, when I had a frozenset that I knew had 1 element
> > (it was the difference between 2 other sets), but couldn't get to that
> > element easily (get the pun?)
> 
> So in my testing (2) was actually the fastest. I assumed because .next()
> was a function call overhead, while:
> for x in some_set:
>   break
> 
> Was evaluated inline. It probably still has to call PyObject_GetIter,
> however it doesn't have to create a stack frame for it.
> 
> This is what "timeit" tells me. All runs are of the form:
> python -m timeit -s "s = set([10])" ...
> 
> 0.101us	"for x in s: break; x"
> 0.130us "for x in s: pass; x"
> 0.234us -s "n = next; i = iter" "x = n(i(s)); x"
> 0.248us "x = next(iter(s)); x"
> 0.341us "x = iter(s).next(); x"
> 
> So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
> faster than (iter(s).next()).
> I was pretty surprised that it was 30% faster than "for x in s: pass". I
> assume it has something to do with a potential "else:" statement?
> 
> Note that all of these are significantly < 1us. So this only matters if
> it is something you are doing often.
> 
> I don't know your specific timings, but I would guess that:
>   for x in s: break
> 
> Is actually going to be faster than your
>   s.get()
> 
> Primarily because s.get() requires an attribute lookup. I would think
> your version might be faster for:
>   stat2 = "g = s.get; for i in xrange(100): g()"
> 
> However, that is still a function call, which may be treated differently
> by the interpreter than the for:break loop. I certainly suggest you try
> it and compare.
> 
> John
> =:->
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091023/755690d7/attachment-0001.htm>

From steve at pearwood.info  Sat Oct 24 00:44:22 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Oct 2009 09:44:22 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE20E0D.4050405@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<hbsun9$432$1@ger.gmane.org> <4AE20E0D.4050405@gmail.com>
Message-ID: <200910240944.22917.steve@pearwood.info>

On Sat, 24 Oct 2009 07:11:57 am John Arbash Meinel wrote:

> The point of my test was that it was a set with a *single* item, and
> 'break' was 30% faster than 'pass'. Which was surprising. 

Not really. See below.

> Certainly 
> the difference is huge if there are 10k items in the set.


Earlier you suggested that the difference may have been because of a 
potential "else" statement. That won't be it: if there's no "else" in 
the code, it's not compiled in:

>>> import dis
>>> dis.dis(compile("for x in s: break", "", "exec"))
  1           0 SETUP_LOOP              15 (to 18)
              3 LOAD_NAME                0 (s)
              6 GET_ITER
        >>    7 FOR_ITER                 7 (to 17)
             10 STORE_NAME               1 (x)
             13 BREAK_LOOP
             14 JUMP_ABSOLUTE            7
        >>   17 POP_BLOCK
        >>   18 LOAD_CONST               0 (None)
             21 RETURN_VALUE
>>>
>>> dis.dis(compile("for x in s: pass", "", "exec"))
  1           0 SETUP_LOOP              14 (to 17)
              3 LOAD_NAME                0 (s)
              6 GET_ITER
        >>    7 FOR_ITER                 6 (to 16)
             10 STORE_NAME               1 (x)
             13 JUMP_ABSOLUTE            7
        >>   16 POP_BLOCK
        >>   17 LOAD_CONST               0 (None)
             20 RETURN_VALUE



The difference is likely to be this:

for x in s: break

retrieves the first (only) element of the set, then immediately breaks 
out of the loop. This is very different from:

for x in s: pass

which retrieves the first element of the set, then tries to retrieve a 
second element, which fails and raises StopIteration, which is then 
caught, ending the loop. That's much more expensive.



-- 
Steven D'Aprano

From steve at pearwood.info  Sat Oct 24 00:46:48 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Oct 2009 09:46:48 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <hbsun9$432$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com> <hbsun9$432$1@ger.gmane.org>
Message-ID: <200910240946.48542.steve@pearwood.info>

On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:
> John Arbash Meinel wrote:
> > So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
> > faster than (iter(s).next()).
> > I was pretty surprised that it was 30% faster than "for x in s:
> > pass". I assume it has something to do with a potential "else:"
> > statement?
>
> for x in s: pass
>
> iterates through *all* the elements in s and leaves x bound to the
> arbritrary *last* one instead of the arbitrary *first* one. For a
> large set, this would be a lot slower, not just a little.
>
> fwiw, I think the use case for this is sufficiently rare that it does
> not need a separate method just for this purpose.


And yet it keeps coming up, again and again... obviously people using 
sets in code think it has a use-case.

I did ask earlier for a use-case, and the OP hasn't replied, but Vitor 
Bosshard did point out one of his use-cases: he had the difference 
between two frozensets, which he knew had only one element, but due to 
the lack of pop() he had no straightforward way of finding out what 
that element was.

The lack of get() in sets and frozensets is sounding more and more to me 
like the victory of purity over practicality.



-- 
Steven D'Aprano

From sturla at molden.no  Sat Oct 24 00:47:28 2009
From: sturla at molden.no (Sturla Molden)
Date: Sat, 24 Oct 2009 00:47:28 +0200
Subject: [Python-Dev] time.clock() on windows
In-Reply-To: <4AE1E576.5000903@molden.no>
References: <4ADE2AA9.4030604@molden.no>	<loom.20091021T125010-360@post.gmane.org>	<930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local>	<1256132087.5069.39.camel@localhost>	<4ADF247E.8040401@molden.no>	<loom.20091021T191933-390@post.gmane.org>	<4ADF481B.90207@gmail.com>	<1256147726.5069.50.camel@localhost>	<930F189C8A437347B80DF2C156F7EC7F098FF42135@exchis.ccp.ad.local>	<loom.20091021T225002-378@post.gmane.org>	<d2155e360910211431t325511e2gaaa9b8a614faed26@mail.gmail.com>	<930F189C8A437347B80DF2C156F7EC7F098FF4213E@exchis.ccp.ad.local>	<4AE0C54F.20705@gmail.com>	<930F189C8A437347B80DF2C156F7EC7F098FF4246E@exchis.ccp.ad.local>
	<4AE1E576.5000903@molden.no>
Message-ID: <4AE23280.50209@molden.no>

Sturla Molden skrev: 
> I have a suggestion, forgive me if I am totally ignorant. :-)
>
Ah, damn... Since there is a GIL, we don't need any of that crappy 
synchronization. And my code does not correct for the 20 ms time jitter 
in GetSystemTimeAsFileTime. Sorry!


S.M.




From steve at pearwood.info  Sat Oct 24 00:49:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Oct 2009 09:49:38 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910232253.24375.w.richert@gmx.net>
References: <200910231132.45504.w.richert@gmx.net> <4AE1E71C.7020806@gmail.com>
	<200910232253.24375.w.richert@gmx.net>
Message-ID: <200910240949.40150.steve@pearwood.info>

On Sat, 24 Oct 2009 07:53:24 am Willi Richert wrote:
> Hi,
>
> surprised about the performance of for/break provided by Vitor, I did
> some more testing. It revealed that indeed  we can forget the get()
> (which was implemented as a stripped down pop()):


I don't understand that conclusion. According to your tests, your 
implementation of get() is as fast as "for x in set: break", and it's 
certainly much, much more readable and straightforward.



-- 
Steven D'Aprano

From ben+python at benfinney.id.au  Sat Oct 24 01:26:27 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 24 Oct 2009 10:26:27 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com> <hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>
Message-ID: <87skd966ak.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:
> > fwiw, I think the use case for this is sufficiently rare that it
> > does not need a separate method just for this purpose.
>
> And yet it keeps coming up, again and again... obviously people using
> sets in code think it has a use-case.

The desire for this may be obvious, but what seems to be lacking is One
Obvious Way to Do It.

I agree that ?for x in foo_set: break? is not an Obvious Way.

> The lack of get() in sets and frozensets is sounding more and more to
> me like the victory of purity over practicality.

What would be the input to ?set.get??

If it's the value, that makes it rather non-obvious; if I already know
about ?dict.get? which takes the key, I'm not going to be thinking about
the ?get? method if I don't have a key to feed it. Once I learn it, I'm
going to forget it easily, because it's inconsistent with ?dict.get?. So
I don't think that meets the ?obvious way? criterion.

By analogy with ?list.pop?, the method that takes the ?top? item off the
?stack?, I would expect to see ?list.peek? and ?set.peek?, to see the
item without altering the container.

-- 
 \        ?Odious ideas are not entitled to hide from criticism behind |
  `\          the human shield of their believers' feelings.? ?Richard |
_o__)                                                         Stallman |
Ben Finney


From steve at pearwood.info  Sat Oct 24 03:48:09 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Oct 2009 12:48:09 +1100
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <4AE1A5DA.40603@gmail.com>
References: <200910231223.08470.w.richert@gmx.net>
	<9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>
	<4AE1A5DA.40603@gmail.com>
Message-ID: <200910241248.09530.steve@pearwood.info>

On Fri, 23 Oct 2009 11:47:22 pm Nick Coghlan wrote:
> Yuvgoog Greenle wrote:
> > On Fri, Oct 23, 2009 at 1:09 PM, Steven D'Aprano 
<steve at pearwood.info> wrote:
> >> Is there any point? Even if accepted, it's too late to make it
> >> into 3.1, and with the overwhelming approval for a moratorium on
> >> changes to built-ins, it is likely to just sit in the tracker,
> >> forgotten, until 2013 or later. How likely is it that the patch
> >> will still be applicable?
> >
> > +1 on throwing it away completely even if it's a good idea. I
> > suggest Willi go invent a new language (and hope for it to become
> > popular :-) if he wants to experiment.
>
> Careful folks - these kinds of throwaway comments may be clearly
> tongue in cheek for anyone following the moratorium discussion on
> python-ideas, but will be justifiably confusing to anyone else,
> especially newcomers.

I'm not being tongue-in-cheek or sarcastic. My question was serious -- 
if there is a moratorium, is there any reason to bother submitting 
patches for functional changes to built-ins? A lot can change between 
now and 2013, and I for one wouldn't bother making a patch that I knew 
wouldn't even be considered for inclusion for four years, and would 
likely need to be re-done even if it was accepted. Guido has said that 
the purpose of the moratorium is to discourage changes to the language. 
If we're discouraging changes, shouldn't we actually discourage them 
rather than waste people's time and give them false hope by telling 
them to put them in the tracker?

Nick, you made the comparison with C in another thread. I don't think 
that the comparison with C is apt, for various reasons, but putting 
that aside, given that C is so stable, what do they do with suggested 
changes to the language? Is there a C issue tracker with ten years of 
accumulated patches, or even proposals, in the queue?


-- 
Steven D'Aprano

From steve at pearwood.info  Sat Oct 24 04:07:10 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Oct 2009 13:07:10 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <87skd966ak.fsf@benfinney.id.au>
References: <200910231132.45504.w.richert@gmx.net>
	<200910240946.48542.steve@pearwood.info>
	<87skd966ak.fsf@benfinney.id.au>
Message-ID: <200910241307.10303.steve@pearwood.info>

On Sat, 24 Oct 2009 10:26:27 am Ben Finney wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
> > On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:
> > > fwiw, I think the use case for this is sufficiently rare that it
> > > does not need a separate method just for this purpose.
> >
> > And yet it keeps coming up, again and again... obviously people
> > using sets in code think it has a use-case.
>
> The desire for this may be obvious, but what seems to be lacking is
> One Obvious Way to Do It.
>
> I agree that ?for x in foo_set: break? is not an Obvious Way.
>
> > The lack of get() in sets and frozensets is sounding more and more
> > to me like the victory of purity over practicality.
>
> What would be the input to ?set.get??

It wouldn't take any input.


> If it's the value, that makes it rather non-obvious; 

It makes it pointless if it takes the value. If you already have the 
value, why would you need to retrieve it from the set?


> if I already 
> know about ?dict.get? which takes the key, I'm not going to be
> thinking about the ?get? method if I don't have a key to feed it.
> Once I learn it, I'm going to forget it easily, because it's
> inconsistent with ?dict.get?. So I don't think that meets the
> ?obvious way? criterion.

"get" is such a generic term that I don't believe that is a problem. 
There are already quite a number of get() methods in the 2.6 std lib:

$ grep "def get[(]" *.py
_abcoll.py:    def get(self, key, default=None):
ConfigParser.py:    def get(self, section, option):
ConfigParser.py:    def get(self, section, option, raw=False, 
vars=None):
doctest.py:    def get(self):
mailbox.py:    def get(self, key, default=None):
os.py:            def get(self, key, failobj=None):
pickle.py:    def get(self, i, pack=struct.pack):
Queue.py:    def get(self, block=True, timeout=None):
shelve.py:    def get(self, key, default=None):
sre_parse.py:    def get(self):
threading.py:        def get(self):
UserDict.py:    def get(self, key, failobj=None):
UserDict.py:    def get(self, key, default=None):
weakref.py:    def get(self, key, default=None):
weakref.py:    def get(self, key, default=None):
webbrowser.py:def get(using=None):


I think you over-estimate the degree of confusion people suffer from 
similar sounding methods. I don't think people are confused that 
dict[key] and list[index] have different semantics, and I don't see why 
dict.get(key[, default]) and set.get() would be any different.

But if you don't like get(), spell it differently. There's plenty of 
opportunity for bike-shedding:

getitem()
getelement()
get_arbitrary_element()
peek()

etc.



> By analogy with ?list.pop?, the method that takes the ?top? item off
> the ?stack?, I would expect to see ?list.peek? and ?set.peek?, to see
> the item without altering the container.

You don't need list.peek() because there already is an obvious way to 
retrieve an item from a list without removing it: list[index]. 





-- 
Steven D'Aprano

From ben+python at benfinney.id.au  Sat Oct 24 05:02:48 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 24 Oct 2009 14:02:48 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
References: <200910231132.45504.w.richert@gmx.net>
	<200910240946.48542.steve@pearwood.info>
	<87skd966ak.fsf@benfinney.id.au>
	<200910241307.10303.steve@pearwood.info>
Message-ID: <87aazh5w9z.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> On Sat, 24 Oct 2009 10:26:27 am Ben Finney wrote:
> > Steven D'Aprano <steve at pearwood.info> writes:
> > > The lack of get() in sets and frozensets is sounding more and more
> > > to me like the victory of purity over practicality.
> >
> > What would be the input to ?set.get??
>
> It wouldn't take any input.

That is even less obvious. I would expect a parameter-less ?set.get? to
get the set. Not terribly useful, but the name and function signature
doesn't suggest anything else.

> "get" is such a generic term that I don't believe that is a problem.

The problem above is made less problematic by the fact that the function
signature (e.g. ?foo_dict.get(key)?) clarifies the answer to the
question ?get what??. Whereas ?foo_set.get()? doesn't communicate much
at all to the reader.

If we want a method that gets one item from a set, perhaps the name can
make it clearer: name it ?set.getitem?. But which item should it get?
The ?__getitem__? special method of lists and dictionaries requires an
index or key as parameter.

-- 
 \             ?Roll dice!? ?Why?? ?Shut up! I don't need your fucking |
  `\     *input*, I need you to roll dice!? ?Luke Crane, demonstrating |
_o__)                       his refined approach to play testing, 2009 |
Ben Finney


From steve at pearwood.info  Sat Oct 24 05:38:52 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Oct 2009 14:38:52 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <87aazh5w9z.fsf@benfinney.id.au>
References: <200910231132.45504.w.richert@gmx.net>
	<200910241307.10303.steve@pearwood.info>
	<87aazh5w9z.fsf@benfinney.id.au>
Message-ID: <200910241438.53974.steve@pearwood.info>

On Sat, 24 Oct 2009 02:02:48 pm Ben Finney wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
> > On Sat, 24 Oct 2009 10:26:27 am Ben Finney wrote:
> > > Steven D'Aprano <steve at pearwood.info> writes:
> > > > The lack of get() in sets and frozensets is sounding more and
> > > > more to me like the victory of purity over practicality.
> > >
> > > What would be the input to ?set.get??
> >
> > It wouldn't take any input.
>
> That is even less obvious. I would expect a parameter-less ?set.get?
> to get the set. Not terribly useful, but the name and function
> signature doesn't suggest anything else.

You already have the set. Why would you need a method that you call that 
returns the set you already have?

A method called "get" obviously gets something, and if it takes no 
arguments the only thing is has access to is the set. The obvious 
things it could get are the set as a whole or some part of the set. 
Since getting the set as a whole would be pointless, and we're allowed 
to assume that the language designers aren't lunatics who would waste 
time creating pointless methods, the obvious answer is that it gets 
some part of the set.

Since there's no obvious reason to choose one subset over another 
subset, the only useful thing it could return is a single arbitrary 
item. But if you're not willing to guess what it gets, you are 
permitted to read the docstring.


> > "get" is such a generic term that I don't believe that is a
> > problem.
>
> The problem above is made less problematic by the fact that the
> function signature (e.g. ?foo_dict.get(key)?) clarifies the answer to
> the question ?get what??. Whereas ?foo_set.get()? doesn't communicate
> much at all to the reader.

You are demanding a level of intuitiveness that few, if any, functions 
in the standard library would be able to meet. Consider list.index(x): 
would you expect it to return the value at index x, or the index of 
value x? Either description is compatible with the name, and in fact 
people sometimes mix them up.

Or sorted() -- from the name alone, I'd expect this to work, but it 
doesn't:

>>> sorted(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable


That's not a problem with the built-in function. I took a guess about 
the API, and guessed wrong. I'll learn from this, and get it right next 
time.


> If we want a method that gets one item from a set, perhaps the name
> can make it clearer: name it ?set.getitem?. But which item should it
> get? The ?__getitem__? special method of lists and dictionaries
> requires an index or key as parameter.

Neither of which is appropriate for sets -- sets are unordered, so 
elements don't have indexes, and they don't map keys to values. They 
just have elements.

Sets are not lists, and they're not dicts. Analogies only go so far, and 
you can't reason about set methods by considering how lists or dicts 
will behave.



-- 
Steven D'Aprano

From ben+python at benfinney.id.au  Sat Oct 24 06:06:28 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 24 Oct 2009 15:06:28 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
References: <200910231132.45504.w.richert@gmx.net>
	<200910241307.10303.steve@pearwood.info>
	<87aazh5w9z.fsf@benfinney.id.au>
	<200910241438.53974.steve@pearwood.info>
Message-ID: <8763a55tbv.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> On Sat, 24 Oct 2009 02:02:48 pm Ben Finney wrote:
> > I would expect a parameter-less ?set.get? to get the set. Not
> > terribly useful, but the name and function signature doesn't suggest
> > anything else.
>
> You already have the set. Why would you need a method that you call
> that returns the set you already have?

Exactly, hence the confusion. I think the method name ?set.get? is poor
for that reason.

> A method called "get" obviously gets something, and if it takes no
> arguments the only thing is has access to is the set. The obvious
> things it could get are the set as a whole or some part of the set.

Which then raises the question ?what part of the set does it get??,
which the function signature does nothing to answer. I'm proposing that
a no-parameters ?set.get? is needlessly confusing to think about.

> Since there's no obvious reason to choose one subset over another
> subset, the only useful thing it could return is a single arbitrary
> item.

That's not at all obvious, IMO. The name doesn't give much help at all
in getting to that conclusion, and isn't easily associated with that
connotation.

> You are demanding a level of intuitiveness that few, if any, functions
> in the standard library would be able to meet.

I'm not demanding intuitiveness; all programming interfaces are learned.
I'm asking for ease of learning and later recall.

> That's not a problem with the built-in function. I took a guess about
> the API, and guessed wrong. I'll learn from this, and get it right
> next time.

Which is partly my point. Such a narrow use case (?get an arbitrary item
from the set, without specifying anything about what that item might
be?) is a mismatch for such a generic name. It makes it unnecessarily
difficult to make the association.

Since the use case is so specific, I would expect the name to be
specific too, to better match the use case.

(Since I know how your mind works, I can anticipate facetious
fifty-character-long names bubbling up already in a response from you.
Let's just assume that joke is already done :-)

-- 
 \                             ?Holy tintinnabulation, Batman!? ?Robin |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From stephen at xemacs.org  Sat Oct 24 08:04:33 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 24 Oct 2009 15:04:33 +0900
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <200910241248.09530.steve@pearwood.info>
References: <200910231223.08470.w.richert@gmx.net>
	<9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>
	<4AE1A5DA.40603@gmail.com> <200910241248.09530.steve@pearwood.info>
Message-ID: <87fx99ba4u.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > I'm not being tongue-in-cheek or sarcastic. My question was serious -- 
 > if there is a moratorium, is there any reason to bother submitting 
 > patches for functional changes to built-ins?

Yes.  Python is open source.  Private and public forks are possible
and (at least in principle) encouraged where the core project decides
that the proposed changes are inappropriate (or should be deferred, as
here).  Nevertheless, isn't the core Python project the obvious common
point of contact for sharing such ideas, even if there is a moratorium
on the code base itself?  I know that a few patches to Mailman have
lived for nearly a decade, beloved by their users and not (much)
bothering the Mailman maintainers with their obvious pitfalls for
naive users.

Whether the tracker is the appropriate place is another question, but
I think it's easier to search for "serious proposal with patch" than
the mailing lists.


From smiles at worksmail.net  Sat Oct 24 08:04:25 2009
From: smiles at worksmail.net (C or L Smith)
Date: Sat, 24 Oct 2009 11:49:25 +0545
Subject: [Python-Dev] tokenize string literal problem
Message-ID: <DE0B6F1B3F8E4597B72B4D3E20148EDB@kisc.edu.np>

BACKGROUND
    I'm trying to modify the doctest DocTestParser so it will parse docstring code snippets out of a *py file. (Although doctest can parse these with another method out of *pyc, it is missing certain decorated functions and we would also like to insist of import of needed modules rather and that method automatically loads everything from the module containing the code.)

PROBLEM
    I need to find code snippets which are located in docstrings. Docstrings, being string literals should be able to be parsed out with tokenize. But tokenize is giving the wrong results (or I am doing something wrong) for this (pathological) case:

foo.py:
+----
def bar():
    """
    A quoted triple quote is not a closing
    of this docstring:
    >>> print '"""'
    """
    """ # <-- this is the closing quote
    pass
+----

Here is how I tokenize the file:

###
import re, tokenize
DOCSTRING_START_RE = re.compile('\s+[ru]*("""|' + "''')")

o=open('foo.py','r')
for ti in tokenize.generate_tokens(o.next):
    typ = ti[0]
    text = ti[-1]
    if typ == tokenize.STRING:
        if DOCSTRING_START_RE.match(text):
            print "DOCSTRING:",repr(text)
o.close()
###

which outputs:

DOCSTRING: '    """\n    A quoted triple quote is not a closing\n    of this docstring:\n    >>> print \'"""\'\n'
DOCSTRING: '    """\n    """ # <-- this is the closing quote\n'

There should be only one string tokenized, I believe. The PythonWin editor parses (and colorizes) this correctly, but tokenize (or I) are making an error.

Thanks for any help,
Chris

From stephen at xemacs.org  Sat Oct 24 08:22:25 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 24 Oct 2009 15:22:25 +0900
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	without	removing it
In-Reply-To: <87aazh5w9z.fsf@benfinney.id.au>
References: <200910231132.45504.w.richert@gmx.net>
	<200910240946.48542.steve@pearwood.info>
	<87skd966ak.fsf@benfinney.id.au>
	<200910241307.10303.steve@pearwood.info>
	<87aazh5w9z.fsf@benfinney.id.au>
Message-ID: <87d44db9b2.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:
 > Steven D'Aprano <steve at pearwood.info> writes:

 > > "get" is such a generic term that I don't believe that is a problem.
 > 
 > The problem above is made less problematic by the fact that the function
 > signature (e.g. 'foo_dict.get(key)') clarifies the answer to the
 > question "get what?". Whereas 'foo_set.get()' doesn't communicate much
 > at all to the reader.

I agree.

This is precisely why a couple of months ago people were proposing
names like ".getany()" for this API.

The problem brought up then was that pretty clearly people would then
do things like

    x = foo.getany()
    y = foo.getany()

expecting x and y to be different (ie, interpreting "any" as "random").
Pretty soon there was a whole family of proposals for such methods:
.getfirst(), .getlast(), .getrandom(), .getonly(), ....

I think it would be better to document the various ways of doing this,
and let each program define its own oneliner for the MySet.get() that
makes idiomatic sense in its use case.

From smiles at worksmail.net  Sat Oct 24 08:33:10 2009
From: smiles at worksmail.net (C or L Smith)
Date: Sat, 24 Oct 2009 12:18:10 +0545
Subject: [Python-Dev] tokenize string literal problem
Message-ID: <B2E11BBF469C43479F40D13E0950ED14@kisc.edu.np>

C or L Smith wrote:
> PROBLEM
>     I need to find code snippets which are located in docstrings.
> Docstrings, being string literals should be able to be parsed out
> with tokenize. But tokenize is giving the wrong results (or I am
> doing something wrong) for this (pathological) case:   
> 
> foo.py:
> +----
> def bar():
>     """
>     A quoted triple quote is not a closing
>     of this docstring:
>     >>> print '"""'
>     """
>     """ # <-- this is the closing quote
>     pass
> +----
> 

I now see that I've created a code snippet that is invalid. Myopia. The thing that pythonWin was doing correctly was displaying my sample STRING not code. I had delimited the code with triple-single-quotes so it showed up correctly. In fact, if entered as code it would show the need to delimit the docstring contents with ''' rather than """.

Sorry!
/c

From ncoghlan at gmail.com  Sat Oct 24 09:19:54 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Oct 2009 17:19:54 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <8763a55tbv.fsf@benfinney.id.au>
References: <200910231132.45504.w.richert@gmx.net>	<200910241307.10303.steve@pearwood.info>	<87aazh5w9z.fsf@benfinney.id.au>	<200910241438.53974.steve@pearwood.info>
	<8763a55tbv.fsf@benfinney.id.au>
Message-ID: <4AE2AA9A.6090107@gmail.com>

Ben Finney wrote:
> Which then raises the question ?what part of the set does it get??,
> which the function signature does nothing to answer. I'm proposing that
> a no-parameters ?set.get? is needlessly confusing to think about.

The fact that set.get() is just set.pop() without removing the result
from the set seems perfectly straightforward.

> Since the use case is so specific, I would expect the name to be
> specific too, to better match the use case.

The use case is no more specific than set.pop().

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From gzlist at googlemail.com  Sat Oct 24 11:12:26 2009
From: gzlist at googlemail.com (Martin (gzlist))
Date: Sat, 24 Oct 2009 10:12:26 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE2AA9A.6090107@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<200910241307.10303.steve@pearwood.info>
	<87aazh5w9z.fsf@benfinney.id.au>
	<200910241438.53974.steve@pearwood.info>
	<8763a55tbv.fsf@benfinney.id.au> <4AE2AA9A.6090107@gmail.com>
Message-ID: <d80792120910240212v16bac9e0l302c223ecda37e30@mail.gmail.com>

On 24/10/2009, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Ben Finney wrote:
>> Which then raises the question ?what part of the set does it get??,
>> which the function signature does nothing to answer. I'm proposing that
>> a no-parameters ?set.get? is needlessly confusing to think about.
>
> The fact that set.get() is just set.pop() without removing the result
> from the set seems perfectly straightforward.

There's a different proposed meaning for `set.get` that's been
discussed on python-dev before:

<http://mail.python.org/pipermail/python-dev/2009-April/088128.html>

That one I've had cause for before and no clean and fast way of
writing, this one I've always just done the for/break thing.

Martin

From steve at pearwood.info  Sat Oct 24 16:20:37 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 25 Oct 2009 01:20:37 +1100
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <87fx99ba4u.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <200910231223.08470.w.richert@gmx.net>
	<200910241248.09530.steve@pearwood.info>
	<87fx99ba4u.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <200910250120.38748.steve@pearwood.info>

On Sat, 24 Oct 2009 05:04:33 pm Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
>
> ?> I'm not being tongue-in-cheek or sarcastic. My question was
> serious -- > if there is a moratorium, is there any reason to bother
> submitting > patches for functional changes to built-ins?
>
> Yes. ?Python is open source. ?Private and public forks are possible
> and (at least in principle) encouraged where the core project decides
> that the proposed changes are inappropriate (or should be deferred,
> as here). ?Nevertheless, isn't the core Python project the obvious
> common point of contact for sharing such ideas, even if there is a
> moratorium on the code base itself?

No.

It's not obvious to me that the CPython tracker is the right place for 
patches for implementations that aren't for CPython.

It's not even obvious that there would be a common point of contact for 
private and public forks, let alone that it would be CPython's tracker. 
There are, by my count, 14 active and defunct implementations of Python 
to date, apart from CPython itself. How many of them currently use the 
CPython tracker to share patches? If the answer is "None", why would 
you expect future implementations and forks to be any different?



-- 
Steven D'Aprano

From w.richert at gmx.net  Sat Oct 24 16:29:32 2009
From: w.richert at gmx.net (Willi Richert)
Date: Sat, 24 Oct 2009 16:29:32 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910240949.40150.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>
	<200910232253.24375.w.richert@gmx.net>
	<200910240949.40150.steve@pearwood.info>
Message-ID: <200910241629.32395.w.richert@gmx.net>

Hi,

I agree. But, here are the pros/cons collected from the recent list repsonses:

Pro:
 - more readable
 - newbies will encounter one of the fastest solution (.get()) before trying 
slower "first solutions" like (iter(set).next())

Cons:
 - no name consensus. get() getany() arbitrary() ?
 - BDFL moratorium, which I find very wise (get() is, however, no language 
extension, but std lib extension, which Guido did not moratorize, didn't he?)
 - other classes should then also be extended, like frozenset

Regards,
wr

PS: what is the correct verb form of moratorium?

Am Samstag, 24. Oktober 2009 00:49:38 schrieb Steven D'Aprano:
> On Sat, 24 Oct 2009 07:53:24 am Willi Richert wrote:
> > Hi,
> >
> > surprised about the performance of for/break provided by Vitor, I did
> > some more testing. It revealed that indeed  we can forget the get()
> > (which was implemented as a stripped down pop()):
> 
> I don't understand that conclusion. According to your tests, your
> implementation of get() is as fast as "for x in set: break", and it's
> certainly much, much more readable and straightforward.
> 

From steve at pearwood.info  Sat Oct 24 17:18:28 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 25 Oct 2009 02:18:28 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <d80792120910240212v16bac9e0l302c223ecda37e30@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net> <4AE2AA9A.6090107@gmail.com>
	<d80792120910240212v16bac9e0l302c223ecda37e30@mail.gmail.com>
Message-ID: <200910250218.29454.steve@pearwood.info>

On Sat, 24 Oct 2009 08:12:26 pm Martin (gzlist) wrote:

> On 24/10/2009, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > Ben Finney wrote:
> >> Which then raises the question ?what part of the set does it
> >> get??, which the function signature does nothing to answer. I'm
> >> proposing that a no-parameters ?set.get? is needlessly confusing
> >> to think about.
> >
> > The fact that set.get() is just set.pop() without removing the
> > result from the set seems perfectly straightforward.
>
> There's a different proposed meaning for `set.get` that's been
> discussed on python-dev before:
>
> <http://mail.python.org/pipermail/python-dev/2009-April/088128.html>

That thread has an example of a use-case for extracting an item from a 
set, given the item itself: interning.

The current Python idiom for interning is to use a dict:

# store the canonical version
_cache[item] = item

# retrieve the interned version
item = _cache[item]

It has been argued that such interning is better done by sets rather 
than dicts, since this will save a pointer per item (plus any 
additional overhead dicts may have over that of sets). If this argument 
is accepted, then we want a fast, efficient operation to perform this:

def get(self, item):
    """Return an element of the set equal to item.
    Useful for retrieving the canonical version of an element 
    and for interning.
    """
    for x in self:
        if x == item:
            return x
    raise ValueError('item not in set')


The above is O(N); ideally it should be O(1).

Normally we don't care about identity, only equality, so if x and item 
above are equal we are indifferent about which one we use. But 
interning is a real example of a use-case where we do care about 
identity. Arguably it is inefficient and wasteful to use a dict for 
interning when sets could be just as fast while using less memory.

The other proposed semantics for set.get() are to retrieve an arbitrary 
element. It must be arbitrary, because elements in a set are unordered 
and unkeyed. This gives us the semantics of pop() without removing the 
element:

def get(self):
    """Return an arbitrary element of the set without removing it."""
    for x in self:
        return x
    raise ValueError("empty set")

This is a frequent request, but I still don't know what the use-case is.

If you'll excuse me stating the obvious, both of these can easily be 
written as helper-functions. The main arguments for making them methods 
are:

(1) The first one is needlessly O(N) when it could be O(1).

(2) The second one is apparently a common need (although I personally 
have never needed it), forcing people to re-invent the wheel, sometimes 
incorrectly or inefficiently, e.g.:

def get(aset):
    for element in aset:
        pass
    return element



-- 
Steven D'Aprano

From martin at v.loewis.de  Sat Oct 24 17:28:58 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 24 Oct 2009 17:28:58 +0200
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <200910241248.09530.steve@pearwood.info>
References: <200910231223.08470.w.richert@gmx.net>	<9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>	<4AE1A5DA.40603@gmail.com>
	<200910241248.09530.steve@pearwood.info>
Message-ID: <4AE31D3A.6090207@v.loewis.de>

> I'm not being tongue-in-cheek or sarcastic. My question was serious -- 
> if there is a moratorium, is there any reason to bother submitting 
> patches for functional changes to built-ins? A lot can change between 
> now and 2013, and I for one wouldn't bother making a patch that I knew 
> wouldn't even be considered for inclusion for four years, and would 
> likely need to be re-done even if it was accepted. Guido has said that 
> the purpose of the moratorium is to discourage changes to the language. 

I haven't been following the discussion, but I wouldn't expect that
a moratorium on language changes would rule out adding a method to the
set type.

Regards,
Martin

From w.richert at gmx.net  Sat Oct 24 17:41:18 2009
From: w.richert at gmx.net (Willi Richert)
Date: Sat, 24 Oct 2009 17:41:18 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910232253.24375.w.richert@gmx.net>
References: <200910231132.45504.w.richert@gmx.net> <4AE1E71C.7020806@gmail.com>
	<200910232253.24375.w.richert@gmx.net>
Message-ID: <200910241741.18388.w.richert@gmx.net>

Hi,

someone on this list mentioned that much of the s.get() time is spend on the 
name lookup for get(). That is indeed the case:

===================
from timeit import *

stats = ["for i in xrange(1000): iter(s).next()   ",
         "for i in xrange(1000): \n\tfor x in s: \n\t\tbreak",
         "for i in xrange(1000): s.add(s.pop())   ",
         "for i in xrange(1000): s.get()          ",
         "g=s.get;\nfor i in xrange(1000): g()          "]

for stat in stats:
    t = Timer(stat, setup="s=set(range(1000))")
    print "Time for %s:\t %f"%(stat, t.timeit(number=1000))
==================

Time for for i in xrange(1000): iter(s).next()   :       0.448227
Time for for i in xrange(1000):
        for x in s:
                break:   0.141669
Time for for i in xrange(1000): s.add(s.pop())   :       0.348055
Time for for i in xrange(1000): s.get()          :       0.148580
Time for g=s.get;
for i in xrange(1000): g()          :    0.080563

So, now set.get() is indeed the fastest and preferable solution if you need 
massive amounts of retrieving elements from a set without removing them.

wr

Am Freitag, 23. Oktober 2009 22:53:24 schrieb Willi Richert:
> Hi,
> 
> surprised about the performance of for/break provided by Vitor, I did some
> more testing. It revealed that indeed  we can forget the get() (which was
> implemented as a stripped down pop()):
> 
> from timeit import *
> stats = ["for i in xrange(1000): iter(s).next()   ",
>          "for i in xrange(1000): \n\tfor x in s: \n\t\tbreak            ",
>          "for i in xrange(1000): s.add(s.pop())   ",
>          "for i in xrange(1000): s.get()          "]
> 
> for stat in stats:
>     t = Timer(stat, setup="s=set(range(100))")
>     try:
>         print "Time for %s:\t %f"%(stat, t.timeit(number=1000))
>     except:
>         t.print_exc()
> 
> 
> 
> $ ./test_get.py
> Time for for i in xrange(1000): iter(s).next()   :       0.433080
> Time for for i in xrange(1000):
>         for x in s:
>                 break                            :       0.148695
> Time for for i in xrange(1000): s.add(s.pop())   :       0.317418
> Time for for i in xrange(1000): s.get()          :       0.146673
> 
> 
> In some tests, for/break was even slightly faster then get().
> 
> As always, intuition regarding performance bottlenecks is flawed ;-)
> 
> Anyway, thanks for all the helpful comments, especially to Stefan for the
> http://comments.gmane.org/gmane.comp.python.ideas/5606 link.
> 
> Regards,
> wr
> 
> Am Freitag, 23. Oktober 2009 19:25:48 schrieb John Arbash Meinel:
> > Vitor Bosshard wrote:
> > > 2009/10/23 Willi Richert <w.richert at gmx.net>:
> > >> Hi,
> > >>
> > >> recently I wrote an algorithm, in which very often I had to get an
> > >> arbitrary element from a set without removing it.
> > >>
> > >> Three possibilities came to mind:
> > >>
> > >> 1.
> > >> x = some_set.pop()
> > >> some_set.add(x)
> > >>
> > >> 2.
> > >> for x in some_set:
> > >>        break
> > >>
> > >> 3.
> > >> x = iter(some_set).next()
> > >>
> > >>
> > >> Of course, the third should be the fastest. It nevertheless goes
> > >> through all the iterator creation stuff, which costs some time. I
> > >> wondered, why the builtin set does not provide a more direct and
> > >> efficient way for retrieving some element without removing it. Is
> > >> there any reason for this?
> > >>
> > >> I imagine something like
> > >>
> > >> x = some_set.get()
> > >
> > > I see this as being useful for frozensets as well, where you can't get
> > > an arbitrary element easily due to the obvious lack of .pop(). I ran
> > > into this recently, when I had a frozenset that I knew had 1 element
> > > (it was the difference between 2 other sets), but couldn't get to that
> > > element easily (get the pun?)
> >
> > So in my testing (2) was actually the fastest. I assumed because .next()
> > was a function call overhead, while:
> > for x in some_set:
> >   break
> >
> > Was evaluated inline. It probably still has to call PyObject_GetIter,
> > however it doesn't have to create a stack frame for it.
> >
> > This is what "timeit" tells me. All runs are of the form:
> > python -m timeit -s "s = set([10])" ...
> >
> > 0.101us	"for x in s: break; x"
> > 0.130us "for x in s: pass; x"
> > 0.234us -s "n = next; i = iter" "x = n(i(s)); x"
> > 0.248us "x = next(iter(s)); x"
> > 0.341us "x = iter(s).next(); x"
> >
> > So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
> > faster than (iter(s).next()).
> > I was pretty surprised that it was 30% faster than "for x in s: pass". I
> > assume it has something to do with a potential "else:" statement?
> >
> > Note that all of these are significantly < 1us. So this only matters if
> > it is something you are doing often.
> >
> > I don't know your specific timings, but I would guess that:
> >   for x in s: break
> >
> > Is actually going to be faster than your
> >   s.get()
> >
> > Primarily because s.get() requires an attribute lookup. I would think
> > your version might be faster for:
> >   stat2 = "g = s.get; for i in xrange(100): g()"
> >
> > However, that is still a function call, which may be treated differently
> > by the interpreter than the for:break loop. I certainly suggest you try
> > it and compare.
> >
> > John
> > =:->
> 

From alexander.belopolsky at gmail.com  Sat Oct 24 19:34:48 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 24 Oct 2009 13:34:48 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910240946.48542.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net> <4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org> <200910240946.48542.steve@pearwood.info>
Message-ID: <d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>

On Fri, Oct 23, 2009 at 6:46 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:
..
>> fwiw, I think the use case for this is sufficiently rare that it does
>> not need a separate method just for this purpose.
>
>
> And yet it keeps coming up, again and again... obviously people using
> sets in code think it has a use-case.
>
This reminds me a debate I had with Martin several years ago:

http://bugs.python.org/issue1507011

Here is the essence:

AB> I disagree with Martin. I think interning is a set
AB> operation and it is unfortunate that set API does not
AB> support it directly.

ML> I disagree with Alexander's last remark in several respects:
ML> set is indeed a container, and there is a way to get the
ML> elements back (namely, by enumerating them, or picking an
ML> arbitrary element for removal). There is no operation to check,
ML> on insertion of E, what the the prior element equal to E was
ML> (if any); there is only a check to determine whether E is in the
ML> set. The operation "give me the member equal but not identical
ML> to E" is conceptually a lookup operation; the mathematical set
ML> construct has no such operation, and the Python set models
ML> it closely. IOW, set is *not* a dict with key==value.

To me, however, a set seems to be a container that is a specialization
of a dict with values and keys being the same.   In this model, a
get() method, or even a __getitem__ with s[k] is k, is only natural.

From debatem1 at gmail.com  Sat Oct 24 19:55:06 2009
From: debatem1 at gmail.com (geremy condra)
Date: Sat, 24 Oct 2009 13:55:06 -0400
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <4AE31D3A.6090207@v.loewis.de>
References: <200910231223.08470.w.richert@gmx.net>
	<9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com>
	<4AE1A5DA.40603@gmail.com> <200910241248.09530.steve@pearwood.info>
	<4AE31D3A.6090207@v.loewis.de>
Message-ID: <f3cc57c60910241055y5dcc8ff9n5d403ee9dc29ef82@mail.gmail.com>

On Sat, Oct 24, 2009 at 11:28 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I'm not being tongue-in-cheek or sarcastic. My question was serious --
>> if there is a moratorium, is there any reason to bother submitting
>> patches for functional changes to built-ins? A lot can change between
>> now and 2013, and I for one wouldn't bother making a patch that I knew
>> wouldn't even be considered for inclusion for four years, and would
>> likely need to be re-done even if it was accepted. Guido has said that
>> the purpose of the moratorium is to discourage changes to the language.
>
> I haven't been following the discussion, but I wouldn't expect that
> a moratorium on language changes would rule out adding a method to the
> set type.
>
> Regards,
> Martin

My understanding is that the moratorium would preclude changes to
the builtins. Is that not the case here?

Geremy Condra

From guido at python.org  Sat Oct 24 19:55:20 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 24 Oct 2009 10:55:20 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com> 
	<hbsun9$432$1@ger.gmane.org> <200910240946.48542.steve@pearwood.info> 
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
Message-ID: <ca471dc20910241055v57df990j7476e75465011309@mail.gmail.com>

On Sat, Oct 24, 2009 at 10:34 AM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> To me, however, a set seems to be a container that is a specialization
> of a dict with values and keys being the same.

That's absurd; the mapping provides nothing useful.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.

From guido at python.org  Sat Oct 24 20:01:21 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 24 Oct 2009 11:01:21 -0700
Subject: [Python-Dev] First shot at some_set.get()
In-Reply-To: <f3cc57c60910241055y5dcc8ff9n5d403ee9dc29ef82@mail.gmail.com>
References: <200910231223.08470.w.richert@gmx.net>
	<9d153b7c0910230419t5e2e6af8n4ecd847a2c9ef329@mail.gmail.com> 
	<4AE1A5DA.40603@gmail.com> <200910241248.09530.steve@pearwood.info> 
	<4AE31D3A.6090207@v.loewis.de>
	<f3cc57c60910241055y5dcc8ff9n5d403ee9dc29ef82@mail.gmail.com>
Message-ID: <ca471dc20910241101y68a4b99xe3c8449189577814@mail.gmail.com>

On Sat, Oct 24, 2009 at 10:55 AM, geremy condra <debatem1 at gmail.com> wrote:
> My understanding is that the moratorium would preclude changes to
> the builtins. Is that not the case here?

It is as yet undecided.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.

From solipsis at pitrou.net  Sat Oct 24 20:02:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 24 Oct 2009 18:02:30 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Clean_up_Python/thread=5F*=2Eh_=3F?=
Message-ID: <loom.20091024T192722-458@post.gmane.org>


Hello,

I am wondering which of the files in Python/thread_*.h are really necessary
today. Looking at these files, I think most of them could perhaps be removed in
py3k. I've identified three categories of potentially removable files:

* The unused file: thread_wince.h

Windows CE has actually been using thread_nt.h since January 2009 (see
http://bugs.python.org/issue4893 ). thread_wince.h is still there, but it's
unused and grep brings no instance of it being mentioned anywhere in the source
tree.

* The (unsupported, untested?) files: thread_atheos.h, thread_cthread.h,
thread_lwp.h, thread_os2.h, thread_pth.h, thread_sgi.h.

These files refer to architectures which we probably have stopped caring about.
It is not even sure whether these files compile and work ok. Most of them have
not seen maintenance for years, except for the OS/2 file. In any case, it is
obvious they haven't received the level of attention and support that the
pthreads and NT versions have.
(also, according to http://www.lesstif.org/~amai/os2/html/porting.html, there is
a pthreads interface available for OS/2)

* The questionable files: thread_pth.h, thread_solaris.h.

GNU pth is probably obsolete on all platforms and superseced by the pthreads
API. According to http://www.gnu.org/software/pth/, the last pth version was
released in 2006 (and the changelog is rather unimpressive).
As for Solaris, according to the Sun website it should provide the pthreads API.
It was already documented in a 1999 manpage for SunOS 5.9:
http://docs.sun.com/app/docs/doc/816-0216/6m6ngupon?a=view .


Making a decision and removing the files considered unnecessary would clarify
what platforms still are/should be supported. Having less supported platforms
would also make improvements easier. Right now making an addition to the core
threading primitives is almost impossible since it would require to provide all
those platform-specific versions which almost nobody is competent for.

Regards

Antoine.



From solipsis at pitrou.net  Sat Oct 24 20:13:44 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 24 Oct 2009 18:13:44 +0000 (UTC)
Subject: [Python-Dev] interning
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com> <hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
Message-ID: <loom.20091024T200726-340@post.gmane.org>

Alexander Belopolsky <alexander.belopolsky <at> gmail.com> writes:
> 
> AB> I disagree with Martin. I think interning is a set
> AB> operation and it is unfortunate that set API does not
> AB> support it directly.
> 
> ML> I disagree with Alexander's last remark in several respects:
[...]
> ML> The operation "give me the member equal but not identical
> ML> to E" is conceptually a lookup operation; the mathematical set
> ML> construct has no such operation, and the Python set models
> ML> it closely. IOW, set is *not* a dict with key==value.

This looks like a debate of purity vs. practicality.

I don't have any opinion on a Python-facing API, but the interpreter's dict of
interned strings could probably converted to a set as a simple optimization.

Regards

Antoine.



From lists at cheimes.de  Sat Oct 24 21:08:08 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 24 Oct 2009 21:08:08 +0200
Subject: [Python-Dev] Clean up Python/thread_*.h ?
In-Reply-To: <loom.20091024T192722-458@post.gmane.org>
References: <loom.20091024T192722-458@post.gmane.org>
Message-ID: <4AE35098.6070306@cheimes.de>

Antoine Pitrou wrote:
> * The unused file: thread_wince.h
> 
> Windows CE has actually been using thread_nt.h since January 2009 (see
> http://bugs.python.org/issue4893 ). thread_wince.h is still there, but it's
> unused and grep brings no instance of it being mentioned anywhere in the source
> tree.

What about older versions of Windows CE? Maybe they still require the
wince thread header?

> 
> * The (unsupported, untested?) files: thread_atheos.h, thread_cthread.h,
> thread_lwp.h, thread_os2.h, thread_pth.h, thread_sgi.h.

thread_atheos.h
---------------

According to svn annotate the file was added on one revision and never
touched afterwards. It was added by MvL.

r27146 | loewis | 2002-06-11 08:22:31 +0200 (Tue, 11 Jun 2002) | 2 lines
Patch #488073: AtheOS port.

thread_cthread.h
----------------

Most lines were last modified or added in in revision 4018 (!), 12178,
16421 and 24967 by Guido, Thomas Wouters and MvL. The last change was:

r24967 | loewis | 2002-01-01 19:41:33 +0100 (Di, 01. Jan 2002) | 2 Zeilen
Patch #497098: build support for GNU/Hurd.

thread_lwp.h
------------

The history of this file is almost identical to thread_cthread.h. No
changes since r23732 (2001-10-16)

thread_os2.h
------------

The file seems to be maintained by Andrew Macintyre. The last change was
on r46919 (2006-06-13). We should ask him before we remove the file.

thread_pth.h
------------

No changes since r15372 (2000-05-08), most of the files was last changed
by Guido and Thomas Wouters.

thread_sgi.h
------------

Same as thread_cthread.h


I'mm +1 on removing the files. Let's ask Martin and Andrew before we
remove the wince and os2 thread files.

Christian

From steven.bethard at gmail.com  Sat Oct 24 21:10:34 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 24 Oct 2009 12:10:34 -0700
Subject: [Python-Dev] updated PEP 389: argparse
Message-ID: <d11dcfba0910241210h62a23bf8x660a08584552001a@mail.gmail.com>

Sorry for the delay, but I've finally updated PEP 389, the argparse
PEP, based on all the feedback from python-dev. The full PEP is below,
but in short, the important changes are:

* The getopt module will *not* be deprecated.
* In Python 2, the -3 flag will cause deprecation warnings to be
issued for optparse.
* No casually visible deprecation warnings for optparse are expected
until Jun 2013.
* Support for {}-format strings will be added when one of the
converters is approved. This can be done at any point though, so it
shouldn't hold up inclusion of argparse.

Steve

----------------------------------------------------------------------
PEP: 389
Title: argparse - new command line parsing module
Version: $Revision: 75674 $
Last-Modified: $Date: 2009-10-24 12:01:49 -0700 (Sat, 24 Oct 2009) $
Author: Steven Bethard <steven.bethard at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 25-Sep-2009
Python-Version: 2.7 and 3.2
Post-History: 27-Sep-2009, 24-Oct-2009


Abstract
========
This PEP proposes inclusion of the argparse [1]_ module in the Python
standard library in Python 2.7 and 3.2.


Motivation
==========
The argparse module is a command line parsing library which provides
more functionality than the existing command line parsing modules in
the standard library, getopt [2]_ and optparse [3]_. It includes
support for positional arguments (not just options), subcommands,
required options, options syntaxes like "/f" and "+rgb", zero-or-more
and one-or-more style arguments, and many other features the other
two lack.

The argparse module is also already a popular third-party replacement
for these modules. It is used in projects like IPython (the Scipy
Python shell) [4]_, is included in Debian testing and unstable [5]_,
and since 2007 has had various requests for its inclusion in the
standard library [6]_ [7]_ [8]_. This popularity suggests it may be
a valuable addition to the Python libraries.


Why aren't getopt and optparse enough?
======================================
One argument against adding argparse is that thare are "already two
different option parsing modules in the standard library" [9]_. The
following is a list of features provided by argparse but not present
in getopt or optparse:

* While it is true there are two *option* parsing libraries, there
  are no full command line parsing libraries -- both getopt and
  optparse support only options and have no support for positional
  arguments. The argparse module handles both, and as a result, is
  able to generate better help messages, avoiding redundancies like
  the ``usage=`` string usually required by optparse.

* The argparse module values practicality over purity. Thus, argparse
  allows required options and customization of which characters are
  used to identify options, while optparse explicitly states "the
  phrase 'required option' is self-contradictory" and that the option
  syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file``
  "are not supported by optparse, and they never will be".

* The argparse module allows options to accept a variable number of
  arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The
  optparse module provides an untested recipe for some part of this
  functionality [10]_ but admits that "things get hairy when you want
  an option to take a variable number of arguments."

* The argparse module supports subcommands, where a main command
  line parser dispatches to other command line parsers depending on
  the command line arguments. This is a common pattern in command
  line interfaces, e.g. ``svn co`` and ``svn up``.


Why isn't the functionality just being added to optparse?
=========================================================
Clearly all the above features offer improvements over what is
available through optparse. A reasonable question then is why these
features are not simply provided as patches to optparse, instead of
introducing an entirely new module. In fact, the original development
of argparse intended to do just that, but because of various fairly
constraining design decisions of optparse, this wasn't really
possible. Some of the problems included:

* The optparse module exposes the internals of its parsing algorithm.
  In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed
  to be available to callbacks [11]_. This makes it extremely
  difficult to improve the parsing algorithm as was necessary in
  argparse for proper handling of positional arguments and variable
  length arguments. For example, ``nargs='+'`` in argparse is matched
  using regular expressions and thus has no notion of things like
  ``parser.largs``.

* The optparse extension APIs are extremely complex. For example,
  just to use a simple custom string-to-object conversion function,
  you have to subclass ``Option``, hack class attributes, and then
  specify your custom option type to the parser, like this::

    class MyOption(Option):
        TYPES = Option.TYPES + ("mytype",)
        TYPE_CHECKER = copy(Option.TYPE_CHECKER)
        TYPE_CHECKER["mytype"] = check_mytype
    parser = optparse.OptionParser(option_class=MyOption)
    parser.add_option("-m", type="mytype")

  For comparison, argparse simply allows conversion functions to be
  used as ``type=`` arguments directly, e.g.::

    parser = argparse.ArgumentParser()
    parser.add_option("-m", type=check_mytype)

  But given the baroque customization APIs of optparse, it is unclear
  how such a feature should interact with those APIs, and it is
  quite possible that introducing the simple argparse API would break
  existing custom Option code.

* Both optparse and argparse parse command line arguments and assign
  them as attributes to an object returned by ``parse_args``.
  However, the optparse module guarantees that the ``take_action``
  method of custom actions will always be passed a ``values`` object
  which provides an ``ensure_value`` method [12]_, while the argparse
  module allows attributes to be assigned to any object, e.g.::

    foo_object = ...
    parser.parse_args(namespace=foo_object)
    foo_object.some_attribute_parsed_from_command_line

  Modifying optparse to allow any object to be passed in would be
  difficult because simply passing the ``foo_object`` around instead
  of a ``Values`` instance will break existing custom actions that
  depend on the ``ensure_value`` method.

Because of issues like these, which made it unreasonably difficult
for argparse to stay compatible with the optparse APIs, argparse was
developed as an independent module. Given these issues, merging all
the argparse features into optparse with no backwards
incompatibilities seems unlikely.


Deprecation of optparse
=======================
Because all of optparse's features are available in argparse, the
optparse module will be deprecated. The following very conservative
deprecation strategy will be followed:

* Python 2.7+ and 3.2+ -- The following note will be added to the
  optparse documentation:

    The optparse module is deprecated, and has been replaced by the
    argparse module.

* Python 2.7+ -- If the Python 3 compatibility flag, ``-3``, is
  provided at the command line, then importing optparse will issue a
  DeprecationWarning. Otherwise no warnings will be issued.

* Python 3.2 (estimated Jun 2010) -- Importing optparse will issue
  a PendingDeprecationWarning, which is not displayed by default.

* Python 3.3 (estimated Jan 2012) -- Importing optparse will issue
  a PendingDeprecationWarning, which is not displayed by default.

* Python 3.4 (estimated Jun 2013) -- Importing optparse will issue
  a DeprecationWarning, which *is* displayed by default.

* Python 3.5 (estimated Jan 2015) -- The optparse module will be
  removed.

Though this is slower than the usual deprecation process, with two
releases of PendingDeprecationWarnings instead of the usual one, it
seems prudent to avoid producing any casually visible Deprecation
warnings until Python 3.X has had some additional time to attract
developers.


Updates to getopt documentation
===============================
The getopt module will not be deprecated. However, its documentation
will be updated to point to argparse in a couple of places. At the
top of the module, the following note will be added:

  The getopt module is a parser for command line options whose API
  is designed to be familiar to users of the C getopt function.
  Users who are unfamiliar with the C getopt function or who would
  like to write less code and get better help and error messages
  should consider using the argparse module instead.

Additionally, after the final getopt example, the following note will
be added:

  Note that an equivalent command line interface could be produced
  with less code by using the argparse module::

    import argparse

    if __name__ == '__main__':
        parser = argparse.ArgumentParser()
        parser.add_argument('-o', '--output')
        parser.add_argument('-v', dest='verbose', action='store_true')
        args = parser.parse_args()
        # ... do something with args.output ...
        # ... do something with args.verbose ..


Deferred: string formatting
===========================
The argparse module supports Python from 2.3 up through 3.2 and as a
result relies on traditional ``%(foo)s`` style string formatting. It
has been suggested that it might be better to use the new style
``{foo}`` string formatting [13]_. There was some discussion about
how best to do this for modules in the standard library [14]_ and
several people are developing functions for automatically converting
%-formatting to {}-formatting [15]_ [16]_. When one of these is added
to the standard library, argparse will use them to support both
formatting styles.


Rejected: getopt compatibility methods
======================================
Previously, when this PEP was suggesting the deprecation of getopt
as well as optparse, there was some talk of adding a method like::

  ArgumentParser.add_getopt_arguments(options[, long_options])

However, this method will not be added for a number of reasons:

* The getopt module is not being deprecated, so there is less need.
* This method would not actually ease the transition for any getopt
  users who were already maintaining usage messages, because the API
  above gives no way of adding help messages to the arguments.
* Some users of getopt consider it very important that only a single
  function call is necessary. The API above does not satisfy this
  requirement because both ``ArgumentParser()`` and ``parse_args()``
  must also be called.


Out of Scope: Various Feature Requests
======================================
Several feature requests for argparse were made in the discussion of
this PEP:

* Support argument defaults from environment variables
* Support argument defaults from configuration files
* Support "foo --help subcommand" in addition to the currently
  supported "foo subcommand --help"

These are all reasonable feature requests for the argparse module,
but are out of the scope of this PEP, and have been redirected to
the argparse issue tracker.


Discussion: sys.err and sys.exit
================================
There were some concerns that argparse by default always writes to
``sys.err`` and always calls ``sys.exit`` when invalid arguments are
provided. This is the desired behavior for the vast majority of
argparse use cases which revolve around simple command line
interfaces. However, in some cases, it may be desirable to keep
argparse from exiting, or to have it write its messages to something
other than ``sys.err``. These use cases can be supported by
subclassing ``ArgumentParser`` and overriding the ``exit`` or
``_print_message`` methods. The latter is an undocumented
implementation detail, but could be officially exposed if this turns
out to be a common need.


References
==========
.. [1] argparse
   (http://code.google.com/p/argparse/)

.. [2] getopt
   (http://docs.python.org/library/getopt.html)

.. [3] optparse
   (http://docs.python.org/library/optparse.html)

.. [4] argparse in IPython
   (http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html)

.. [5] argparse in Debian
   (http://packages.debian.org/search?keywords=python-argparse)

.. [6] 2007-01-03 request for argparse in the standard library
   (http://mail.python.org/pipermail/python-list/2007-January/592646.html)

.. [7] 2009-06-09 request for argparse in the standard library
   (http://bugs.python.org/issue6247)

.. [8] 2009-09-10 request for argparse in the standard library
   (http://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html)

.. [9] Fredrik Lundh response to [6]_
   (http://mail.python.org/pipermail/python-list/2007-January/592675.html)

.. [10] optparse variable args
   (http://docs.python.org/library/optparse.html#callback-example-6-variable-arguments)

.. [11] parser.largs and parser.rargs
   (http://docs.python.org/library/optparse.html#how-callbacks-are-called)

.. [12] take_action values argument
   (http://docs.python.org/library/optparse.html#adding-new-actions)

.. [13] use {}-formatting instead of %-formatting
   (http://bugs.python.org/msg89279)

.. [14] transitioning from % to {} formatting
   (http://mail.python.org/pipermail/python-dev/2009-September/092326.html)

.. [15] Vinay Sajip's %-to-{} converter
   (http://gist.github.com/200936)

.. [16] Benjamin Peterson's %-to-{} converter
   (http://bazaar.launchpad.net/~gutworth/+junk/mod2format/files)


Copyright
=========
This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From tjreedy at udel.edu  Sat Oct 24 21:18:50 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 24 Oct 2009 15:18:50 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <200910250218.29454.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE2AA9A.6090107@gmail.com>	<d80792120910240212v16bac9e0l302c223ecda37e30@mail.gmail.com>
	<200910250218.29454.steve@pearwood.info>
Message-ID: <hbvjum$e0t$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Sat, 24 Oct 2009 08:12:26 pm Martin (gzlist) wrote:

>> There's a different proposed meaning for `set.get` that's been
>> discussed on python-dev before:
>>
>> <http://mail.python.org/pipermail/python-dev/2009-April/088128.html>

For that case, I think the OP was mistaken mistaken to reject using a 
dict. He had objects with a key and wanted an index based on that key.

> That thread has an example of a use-case for extracting an item from a 
> set, given the item itself: interning.
> 
> The current Python idiom for interning is to use a dict:
> 
> # store the canonical version
> _cache[item] = item
> 
> # retrieve the interned version
> item = _cache[item]

This is an interesting use case as it turns on the difference between 
mathematics, where value is identity, and informatics, where the two 
concepts split. The above is meaningless in math, but *is* meaningful in 
informatics, where different objects can have the same set-membership 
value. For Python's builtin sets, set-membership 'value' is based on 
hash comparisons followed by equality comparison, rather than identity. 
The purpose of this is to make them more like mathematical sets.

But because of the value-identity distinction, they are not exactly like 
mathematical sets. Given that set()s implicitly map equivalence classes 
(which are not Python objects) to representitive members of that class 
(which are), then I agree and would even argue that there should be 
method of retrieving the one representative given any member.

(The above assumes that .__eq__ is properly defined for potential 
members so that they are properly partitioned into equivalence classes. 
This is not true when Decimals are mixed with other number classes.)

A counter-argument is that the implicit mapping is an implementation 
detail. A bit-mapped set class for a finite range of ints would not have 
this mapping. So an ABC for sets probably should not include such a method.

> It has been argued that such interning is better done by sets rather 
> than dicts, since this will save a pointer per item (plus any 
> additional overhead dicts may have over that of sets). If this argument 
> is accepted, then we want a fast, efficient operation to perform this:
> 
> def get(self, item):
>     """Return an element of the set equal to item.
>     Useful for retrieving the canonical version of an element 
>     and for interning.
>     """
>     for x in self:
>         if x == item:
>             return x
>     raise ValueError('item not in set')
> 
> 
> The above is O(N); ideally it should be O(1).
> 
> Normally we don't care about identity, only equality, so if x and item 
> above are equal we are indifferent about which one we use. But 
> interning is a real example of a use-case where we do care about 
> identity. Arguably it is inefficient and wasteful to use a dict for 
> interning when sets could be just as fast while using less memory.
> 
> The other proposed semantics for set.get() are to retrieve an arbitrary 
> element. It must be arbitrary, because elements in a set are unordered 
> and unkeyed. This gives us the semantics of pop() without removing the 
> element:
> 
> def get(self):
>     """Return an arbitrary element of the set without removing it."""
>     for x in self:
>         return x
>     raise ValueError("empty set")
> 
> This is a frequent request, but I still don't know what the use-case is.
> 
> If you'll excuse me stating the obvious, both of these can easily be 
> written as helper-functions. The main arguments for making them methods 
> are:
> 
> (1) The first one is needlessly O(N) when it could be O(1).
> 
> (2) The second one is apparently a common need (although I personally 
> have never needed it), forcing people to re-invent the wheel, sometimes 
> incorrectly or inefficiently, e.g.:
> 
> def get(aset):
>     for element in aset:
>         pass
>     return element

Terry Jan Reedy



From martin at v.loewis.de  Sat Oct 24 21:24:47 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 24 Oct 2009 21:24:47 +0200
Subject: [Python-Dev] Clean up Python/thread_*.h ?
In-Reply-To: <loom.20091024T192722-458@post.gmane.org>
References: <loom.20091024T192722-458@post.gmane.org>
Message-ID: <4AE3547F.40800@v.loewis.de>

> Making a decision and removing the files considered unnecessary would clarify
> what platforms still are/should be supported.

I think any such removal should go through the PEP 11 process. Put a
#error into the files for 3.2, and a removal notice into the PEP, then
remove them in 3.3.

As for OS/2: this should probably stay.

Regards,
Martin

From tjreedy at udel.edu  Sat Oct 24 21:37:13 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 24 Oct 2009 15:37:13 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <ca471dc20910241055v57df990j7476e75465011309@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<ca471dc20910241055v57df990j7476e75465011309@mail.gmail.com>
Message-ID: <hbvl15$gne$1@ger.gmane.org>

Guido van Rossum wrote:
> On Sat, Oct 24, 2009 at 10:34 AM, Alexander Belopolsky
> <alexander.belopolsky at gmail.com> wrote:
>> To me, however, a set seems to be a container that is a specialization
>> of a dict with values and keys being the same.
> 
> That's absurd; the mapping provides nothing useful.

Given Alexander's premise, I agree with your response. But his premise 
is wrong. Python's current builtin set class maps abstract equivalence 
classes to representative members. And this *is* useful. Mapping 
arbitrary members of such classes to representative members, sometimes 
called 'canonical', is a standard problem/goal in math. String interning 
is an application of this idea.

Terry Jan Reedy




From tjreedy at udel.edu  Sat Oct 24 21:37:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 24 Oct 2009 15:37:27 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com>	<hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
Message-ID: <hbvl1i$gne$2@ger.gmane.org>

Alexander Belopolsky wrote:
> On Fri, Oct 23, 2009 at 6:46 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:
> ..
>>> fwiw, I think the use case for this is sufficiently rare that it does
>>> not need a separate method just for this purpose.
>>
>> And yet it keeps coming up, again and again... obviously people using
>> sets in code think it has a use-case.
>>
> This reminds me a debate I had with Martin several years ago:
> 
> http://bugs.python.org/issue1507011
> 
> Here is the essence:
> 
> AB> I disagree with Martin. I think interning is a set
> AB> operation and it is unfortunate that set API does not
> AB> support it directly.
> 
> ML> I disagree with Alexander's last remark in several respects:
> ML> set is indeed a container, and there is a way to get the
> ML> elements back (namely, by enumerating them, or picking an
> ML> arbitrary element for removal). There is no operation to check,
> ML> on insertion of E, what the the prior element equal to E was
> ML> (if any); there is only a check to determine whether E is in the
> ML> set. The operation "give me the member equal but not identical
> ML> to E" is conceptually a lookup operation; the mathematical set
> ML> construct has no such operation, and the Python set models
> ML> it closely. IOW, set is *not* a dict with key==value.
> 
> To me, however, a set seems to be a container that is a specialization
> of a dict with values and keys being the same.

As I explained in response to Steven, set()s *implicitly* map of 
abstract, non-object equivalence classes to a concrete, representative 
object/member of that class.

 >  In this model, a
> get() method, or even a __getitem__ with s[k] is k, is only natural.

No, if key and value were the same thing, the get method would be 
nonesense, as Guido said. But since they are not, since the implict key 
is an abstract class, retrieving the representative, perhaps canonical 
object given *any* member *is* natural. Being able to do so is also a 
standard practice in mathematics.

Terry Jan Reedy



From solipsis at pitrou.net  Sat Oct 24 21:41:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 24 Oct 2009 19:41:54 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Clean_up_Python/thread=5F*=2Eh_=3F?=
References: <loom.20091024T192722-458@post.gmane.org>
	<4AE3547F.40800@v.loewis.de>
Message-ID: <loom.20091024T213930-937@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Making a decision and removing the files considered unnecessary would clarify
> > what platforms still are/should be supported.
> 
> I think any such removal should go through the PEP 11 process. Put a
> #error into the files for 3.2, and a removal notice into the PEP, then
> remove them in 3.3.

Ok, thanks! I see that AtheOS and BeOS are already marked unsupported in PEP 11.

What is your opinion about the Solaris-specific thread support? Is it still
necessary?

Regards

Antoine.



From rhamph at gmail.com  Sat Oct 24 22:09:41 2009
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 24 Oct 2009 14:09:41 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
Message-ID: <aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>

On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard <algorias at gmail.com> wrote:
> I see this as being useful for frozensets as well, where you can't get
> an arbitrary element easily due to the obvious lack of .pop(). I ran
> into this recently, when I had a frozenset that I knew had 1 element
> (it was the difference between 2 other sets), but couldn't get to that
> element easily (get the pun?)

item, = set_of_one


-- 
Adam Olsen, aka Rhamphoryncus

From ben+python at benfinney.id.au  Sun Oct 25 01:34:35 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 25 Oct 2009 10:34:35 +1100
Subject: [Python-Dev] updated PEP 389: argparse
References: <d11dcfba0910241210h62a23bf8x660a08584552001a@mail.gmail.com>
Message-ID: <871vks5ptg.fsf@benfinney.id.au>

Steven Bethard <steven.bethard at gmail.com> writes:

> Discussion: sys.err and sys.exit
> ================================
> There were some concerns that argparse by default always writes to
> ``sys.err``
[?]

Unless, I'm missing something, this should replace ?sys.err? with
?sys.stderr? throughout.

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\        Brain, but why would anyone want to see Snow White and the |
_o__)                           Seven Samurai?? ?_Pinky and The Brain_ |
Ben Finney


From steven.bethard at gmail.com  Sun Oct 25 01:43:24 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 24 Oct 2009 16:43:24 -0700
Subject: [Python-Dev] updated PEP 389: argparse
In-Reply-To: <871vks5ptg.fsf@benfinney.id.au>
References: <d11dcfba0910241210h62a23bf8x660a08584552001a@mail.gmail.com>
	<871vks5ptg.fsf@benfinney.id.au>
Message-ID: <d11dcfba0910241643h15a6e384sbd748d96c251142d@mail.gmail.com>

On Sat, Oct 24, 2009 at 4:34 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Steven Bethard <steven.bethard at gmail.com> writes:
>
>> Discussion: sys.err and sys.exit
>> ================================
>> There were some concerns that argparse by default always writes to
>> ``sys.err``
> [?]
>
> Unless, I'm missing something, this should replace ?sys.err? with
> ?sys.stderr? throughout.

Yep, that's a typo. Thanks for the catch. It's fixed in SVN and the website:

http://www.python.org/dev/peps/pep-0389/

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From john.meinel at canonical.com  Sun Oct 25 02:47:42 2009
From: john.meinel at canonical.com (John Arbash Meinel)
Date: Sat, 24 Oct 2009 20:47:42 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>
Message-ID: <4AE3AE3E.80402@canonical.com>

Adam Olsen wrote:
> On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard <algorias at gmail.com> wrote:
>> I see this as being useful for frozensets as well, where you can't get
>> an arbitrary element easily due to the obvious lack of .pop(). I ran
>> into this recently, when I had a frozenset that I knew had 1 element
>> (it was the difference between 2 other sets), but couldn't get to that
>> element easily (get the pun?)
> 
> item, = set_of_one
> 
> 

Interesting. It depends a bit on the speed of tuple unpacking, but
presumably that is quite fast. On my system it is pretty darn good:

0.101us "for x in s: break"
0.112us "x, = s"
0.122us "for x in s: pass"

So not quite as good as the for loop, but quite close.

John
=:->

From aleaxit at gmail.com  Sun Oct 25 05:22:03 2009
From: aleaxit at gmail.com (Alex Martelli)
Date: Sat, 24 Oct 2009 21:22:03 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE3AE3E.80402@canonical.com>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>
	<4AE3AE3E.80402@canonical.com>
Message-ID: <BF8D86B7-587E-4229-B503-93A706597047@gmail.com>

Next(s) would seem good...

Alex

Sent from my iPhone

On Oct 24, 2009, at 6:47 PM, John Arbash Meinel <john.meinel at canonical.com 
 > wrote:

> Adam Olsen wrote:
>> On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard <algorias at gmail.com>  
>> wrote:
>>> I see this as being useful for frozensets as well, where you can't  
>>> get
>>> an arbitrary element easily due to the obvious lack of .pop(). I ran
>>> into this recently, when I had a frozenset that I knew had 1 element
>>> (it was the difference between 2 other sets), but couldn't get to  
>>> that
>>> element easily (get the pun?)
>>
>> item, = set_of_one
>>
>>
>
> Interesting. It depends a bit on the speed of tuple unpacking, but
> presumably that is quite fast. On my system it is pretty darn good:
>
> 0.101us "for x in s: break"
> 0.112us "x, = s"
> 0.122us "for x in s: pass"
>
> So not quite as good as the for loop, but quite close.
>
> John
> =:->
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com

From ncoghlan at gmail.com  Sun Oct 25 06:48:54 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 25 Oct 2009 15:48:54 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <hbvl1i$gne$2@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>	<4AE1E71C.7020806@gmail.com>	<hbsun9$432$1@ger.gmane.org>	<200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org>
Message-ID: <4AE3E6C6.4010008@gmail.com>

Terry Reedy wrote:
>>  In this model, a
>> get() method, or even a __getitem__ with s[k] is k, is only natural.
> 
> No, if key and value were the same thing, the get method would be
> nonesense, as Guido said. But since they are not, since the implict key
> is an abstract class, retrieving the representative, perhaps canonical
> object given *any* member *is* natural. Being able to do so is also a
> standard practice in mathematics.

To formalise this invariant to some degree: given a set "s" and two
items "k1" and "k2", such that "k1 in s" and "k2 in s" and "k1 == k2",
there is a single item "k" in s such that "k1 == k" and "k2 == k".

If __getitem__ were to be provided by sets, then the last part of that
invariant could be written "s[k1] is s[k2]".

If you actually care about that aspect of the semantics for a particular
application, it would be far clearer to use a full dict() and live with
the additional memory usage. While I can see how saving that
pointer-per-entry might make sense in some circumstances, for most I
would see it as a needlessly confusing micro-optimisation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sun Oct 25 07:00:14 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 25 Oct 2009 16:00:14 +1000
Subject: [Python-Dev] updated PEP 389: argparse
In-Reply-To: <d11dcfba0910241210h62a23bf8x660a08584552001a@mail.gmail.com>
References: <d11dcfba0910241210h62a23bf8x660a08584552001a@mail.gmail.com>
Message-ID: <4AE3E96E.9030506@gmail.com>

Steven Bethard wrote:
> Sorry for the delay, but I've finally updated PEP 389, the argparse
> PEP, based on all the feedback from python-dev. The full PEP is below,
> but in short, the important changes are:
> 
> * The getopt module will *not* be deprecated.
> * In Python 2, the -3 flag will cause deprecation warnings to be
> issued for optparse.
> * No casually visible deprecation warnings for optparse are expected
> until Jun 2013.
> * Support for {}-format strings will be added when one of the
> converters is approved. This can be done at any point though, so it
> shouldn't hold up inclusion of argparse.

The new version provides a very good summary of and response to the
feedback on the previous version of the PEP.

+1 on adding the module :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tjreedy at udel.edu  Sun Oct 25 07:50:06 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 25 Oct 2009 02:50:06 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <BF8D86B7-587E-4229-B503-93A706597047@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>
	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>
Message-ID: <hc0sev$uo1$1@ger.gmane.org>

Alex Martelli wrote:
> Next(s) would seem good...

That does not work. It has to be next(iter(s)), and that has been tried 
and eliminated because it is significantly slower.

>> Interesting. It depends a bit on the speed of tuple unpacking, but
>> presumably that is quite fast. On my system it is pretty darn good:
>>
>> 0.101us "for x in s: break"
>> 0.112us "x, = s"


From dickinsm at gmail.com  Sun Oct 25 09:54:46 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 25 Oct 2009 08:54:46 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
Message-ID: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>

Would it be worth spending some time discussing the buildbot situation
at the PyCon 2010 language summit?  In the past, I've found the
buildbots to be an incredibly valuable resource;  especially when
working with aspects of Python or C that tend to vary significantly
from platform to platform (for me, this usually means floating-point,
and platform math libraries, but there are surely many other things it
applies to).  But more recently there seem to have been some
difficulties keeping a reasonable number of buildbots up and running.
A secondary problem is that it can be awkward to debug some of the
more obscure test failures on buildbots without having direct access
to the machine.  From conversations on IRC, I don't think I'm alone in
wanting to find ways to make the buildbots more useful.

So the question is: how best to invest time and possibly money to
improve the buildbot situation (and as a result, I hope, improve the
quality of Python)?  What could be done to make maintenance of build
slaves easier?  Or to encourage interested third parties to donate
hardware and time?  Are there good alternatives to Buildbot that might
make a difference? What do other projects do?

These are probably the wrong questions;  I'm hoping that a discussion
would help produce the right questions, and possibly some answers.

Mark

From martin at v.loewis.de  Sun Oct 25 10:47:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 10:47:10 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
Message-ID: <4AE41E9E.4050706@v.loewis.de>

Mark Dickinson wrote:
> Would it be worth spending some time discussing the buildbot situation
> at the PyCon 2010 language summit?  In the past, I've found the
> buildbots to be an incredibly valuable resource;  especially when
> working with aspects of Python or C that tend to vary significantly
> from platform to platform (for me, this usually means floating-point,
> and platform math libraries, but there are surely many other things it
> applies to).  But more recently there seem to have been some
> difficulties keeping a reasonable number of buildbots up and running.
> A secondary problem is that it can be awkward to debug some of the
> more obscure test failures on buildbots without having direct access
> to the machine.  From conversations on IRC, I don't think I'm alone in
> wanting to find ways to make the buildbots more useful.

These are actually two issues:
a) where do we get buildbot hardware and operators?
b) how can we reasonably debug problems occurring on buildbots

For a), I think we can solve this only by redundancy, i.e. create
more build slaves, hoping that a sufficient number would be up at
any point in time.

So: what specific kinds of buildbots do you think are currently lacking?
A call for volunteers will likely be answered quickly.

> So the question is: how best to invest time and possibly money to
> improve the buildbot situation (and as a result, I hope, improve the
> quality of Python)?

I don't think money will really help (I'm skeptical in general that
money helps in open source projects). As for time: "buildbot scales",
meaning that the buildbot slave admins will all share the load, being
responsible only for their own slaves.

On the master side: would you be interested in tracking slave admins?

> What could be done to make maintenance of build
> slaves easier?

This is something that only the slave admins can answer. I don't think
it's difficult - it's just that people are really unlikely to contribute
to the same thing over a period of five years at a steady rate. So we
need to make sure to find replacements when people drop out.

> Or to encourage interested third parties to donate
> hardware and time?

Again: I think a call for volunteers would do (Steve, if you are
reading this, please hold back just a few days before actually
making such a call :-)

> Are there good alternatives to Buildbot that might
> make a difference?

I think people have started working on such a thing. There are
certainly alternatives; I'm fairly skeptical that they are *good*
alternatives (but then, I'm the one who set up the buildbot
installation in the first place).

> What do other projects do?

I think that's really difficult to compare, since their testing
often has a very different scope. I think CruiseControl is widely
used.

> These are probably the wrong questions;  I'm hoping that a discussion
> would help produce the right questions, and possibly some answers.

I think these are good questions - just not for the summit. Setting
up such a system is, conceptually, easy. It's also just a little work
to set it up initially; the difficult part then is to keep it running
(and no, a system where anybody can just post test results at any time
without prior registration is *still* difficult to keep running).

The source of the problem is that such a system can degrade without
anybody taking action. If the web server's hard disk breaks down,
people panic and look for a solution quickly. If the source control
is down, somebody *will* "volunteer" to fix it. If the automated
build system produces results less useful, people will worry, but
not take action.

Regards,
Martin

From andymac at bullseye.apana.org.au  Sun Oct 25 12:00:03 2009
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sun, 25 Oct 2009 22:00:03 +1100
Subject: [Python-Dev] Clean up Python/thread_*.h ?
In-Reply-To: <4AE3547F.40800@v.loewis.de>
References: <loom.20091024T192722-458@post.gmane.org>
	<4AE3547F.40800@v.loewis.de>
Message-ID: <4AE42FB3.9090606@bullseye.andymac.org>

Martin v. L?wis wrote:
>> Making a decision and removing the files considered unnecessary would clarify
>> what platforms still are/should be supported.
> 
> I think any such removal should go through the PEP 11 process. Put a
> #error into the files for 3.2, and a removal notice into the PEP, then
> remove them in 3.3.
> 
> As for OS/2: this should probably stay.

Yes please (to keeping the OS/2 thread support as-is).

Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia


From solipsis at pitrou.net  Sun Oct 25 13:16:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Oct 2009 12:16:26 +0000 (UTC)
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
Message-ID: <hc1fiq$5ec$1@ger.gmane.org>


> For a), I think we can solve this only by redundancy, i.e. create more
> build slaves, hoping that a sufficient number would be up at any point
> in time.

We are already doing this, aren't we?
http://www.python.org/dev/buildbot/3.x/

It doesn't seem to work very well, it's a bit like a Danaides vessel.

> The source of the problem is that such a system can degrade without
> anybody taking action. If the web server's hard disk breaks down, people
> panic and look for a solution quickly. If the source control is down,
> somebody *will* "volunteer" to fix it. If the automated build system
> produces results less useful, people will worry, but not take action.

Well, to be fair, buildbots breaking also happens much more frequently 
(perhaps one or two orders of magnitude) than the SVN server or the Web 
site going down. Maintaining them looks like a Sisyphean task, and nobody 
wants that.

I don't know what kind of machines are the current slaves, but if they 
are 24/7 servers, isn't it a bit surprising that the slaves would go down 
so often? Is the buildbot software fragile? Does it require a lot of 
(maintenance, repair) work from the slave owners?


From ctb at msu.edu  Sun Oct 25 13:48:16 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 25 Oct 2009 05:48:16 -0700
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
Message-ID: <20091025124816.GA20329@idyll.org>

On Sun, Oct 25, 2009 at 08:54:46AM +0000, Mark Dickinson wrote:
> Would it be worth spending some time discussing the buildbot situation
> at the PyCon 2010 language summit?  In the past, I've found the
> buildbots to be an incredibly valuable resource;  especially when
> working with aspects of Python or C that tend to vary significantly
> from platform to platform (for me, this usually means floating-point,
> and platform math libraries, but there are surely many other things it
> applies to).  But more recently there seem to have been some
> difficulties keeping a reasonable number of buildbots up and running.
> A secondary problem is that it can be awkward to debug some of the
> more obscure test failures on buildbots without having direct access
> to the machine.  From conversations on IRC, I don't think I'm alone in
> wanting to find ways to make the buildbots more useful.
> 
> So the question is: how best to invest time and possibly money to
> improve the buildbot situation (and as a result, I hope, improve the
> quality of Python)?  What could be done to make maintenance of build
> slaves easier?  Or to encourage interested third parties to donate
> hardware and time?  Are there good alternatives to Buildbot that might
> make a difference? What do other projects do?
> 
> These are probably the wrong questions;  I'm hoping that a discussion
> would help produce the right questions, and possibly some answers.

[ x-posting to testing-in-python; please redirect followups to one list or
the other! ]

Hi Mark,

a few bits of information...

---

I have a set of VM machines running some "core" build archs -- Linux, Mac OS X,
Win XP, Win Vista, and Win 7 -- that I am going to dedicate to this purpose.
I am happy to give out remote admin access to a few people.  This
infrastructure is also going to increase slowly as I build up my lab's internal
network.

I'm giving Tarek an account on my Linux box later today to serve as a build
slave for Distribute.

--

More machines, and more esoteric machines, will be coming online as Snakebite
unfolds.  Trent Nelson (Snakepit master) has been finishing up some consulting
work and is going to dedicate his time to it starting in November.  This means
more 64 bit, bigmem, and "weird" archs, with full login access.

---

I've also been working on a buildbot alternative that I'm calling pony-build.
pony-build is based on a client-push architecture in which client machines
do builds and push results to the master, which then acts as a record-keeper
rather than a slave driver.  The result is a less coordinated but (AFAICT) much
less fragile continuous integration system.  I'm hoping to give a talk at PyCon
on the differences, and there will be a sprint on pony-build + pyhton-dev at
PyCon, regardless.

The current status of pony-build is "functional but ugly inside".  In
particular, the data model is horrible, and the internal API needs much
more fleshing out.  Nonetheless, my server has been running for two months
or so, and you can look at the results here,

	http://lyorn.idyll.org/ctb/pb-dev/

The most fully-fleshed out set of build clients is for pygr,

	http://lyorn.idyll.org/ctb/pb-dev/pygr/

but you can view daily build results for Python 2.7 at

	http://lyorn.idyll.org/ctb/pb-dev/python/

with an exhaustive list here

	http://lyorn.idyll.org/ctb/pb-dev/python/show_all

(and why the heck am I sorting in reverse date order, anyway?!)

The most interesting (?) part of pony-build right now is the client
config, which I'm struggling to make simple and potentially universal
enough to serve under buildbot as well:

	http://github.com/ctb/pony-build/blob/master/client/build-cpython

(see 'commands' list).

The most *exciting* part of pony-build, apart from the always-riveting
spectacle of "titus rediscovering problems that buildbot solved 5 years ago",
is the loose coupling of recording server to the build slaves and build
reporters.  My plan is to enable a simple and lightweight XML-RPC and/or
REST-ish interface for querying the recording server from scripts or other Web
sites.  This has Brett aquiver with anticipation, I gather -- no more visual
inspection of buildbot waterfall pages ;)

--

If you're interested in bashing on, contributing to, or helping figure out
what color the pony-build main screen should be, contact me off-list; I'm
reluctant to spam up python-dev or testing-in-python.

That having been said, the results of taking it and trying it out -- you can
post results to my own recording server at 

	http://lyorn.idyll.org/ctb/pb-dev/xmlrpc

-- would be most welcome.

Once I fix the data model, code collaboration will be much more feasible,
too.

---

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From exarkun at twistedmatrix.com  Sun Oct 25 14:50:18 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 25 Oct 2009 13:50:18 -0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <hc1fiq$5ec$1@ger.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
Message-ID: <20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>

On 12:16 pm, solipsis at pitrou.net wrote:
>
>>For a), I think we can solve this only by redundancy, i.e. create more
>>build slaves, hoping that a sufficient number would be up at any point
>>in time.
>
>We are already doing this, aren't we?
>http://www.python.org/dev/buildbot/3.x/
>
>It doesn't seem to work very well, it's a bit like a Danaides vessel.
>>The source of the problem is that such a system can degrade without
>>anybody taking action. If the web server's hard disk breaks down, 
>>people
>>panic and look for a solution quickly. If the source control is down,
>>somebody *will* "volunteer" to fix it. If the automated build system
>>produces results less useful, people will worry, but not take action.
>
>Well, to be fair, buildbots breaking also happens much more frequently
>(perhaps one or two orders of magnitude) than the SVN server or the Web
>site going down. Maintaining them looks like a Sisyphean task, and 
>nobody
>wants that.

Perhaps this is a significant portion of the problem.  Maintaining a 
build slave is remarkably simple and easy.  I maintain about half a 
dozen slaves and spend at most a few minutes a month operating them. 
Actually setting one up in the first place might take a bit longer, 
since it involves installing the necessary software and making sure 
everything's set up right, but the actual slave configuration itself is 
one command:

  buildbot create-slave <path> <master address> <slave name> <slave 
password>

Perhaps this will help dispel the idea that it is a serious undertaking 
to operate a slave.

The real requirement which some people may find challenging is that the 
slave needs to operate on a host which is actually online almost all of 
the time.  If you don't such a machine, then there's little point 
offering to host a slave.
>I don't know what kind of machines are the current slaves, but if they
>are 24/7 servers, isn't it a bit surprising that the slaves would go 
>down
>so often? Is the buildbot software fragile? Does it require a lot of
>(maintenance, repair) work from the slave owners?

As I have no specific experience maintaining any of the CPython build 
slaves, I can't speak to any maintenance issues which these slaves have 
encountered.  I would expect that they are as minimal as the issues I 
have encountered maintaining slaves for other projects, but perhaps this 
is wrong.  I do recall that there were some win32 issues (discussed on 
this list, I think) quite a while back, but I think those were resolved. 
I haven't heard of any other issues since then.  If there are some, 
perhaps the people who know about them could raise them and we could try 
to figure out how to resolve them.

Jean-Paul

From exarkun at twistedmatrix.com  Sun Oct 25 15:05:12 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 25 Oct 2009 14:05:12 -0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE41E9E.4050706@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
Message-ID: <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>

On 09:47 am, martin at v.loewis.de wrote:
>Mark Dickinson wrote:
>>Would it be worth spending some time discussing the buildbot situation
>>at the PyCon 2010 language summit?  In the past, I've found the
>>buildbots to be an incredibly valuable resource;  especially when
>>working with aspects of Python or C that tend to vary significantly
>>from platform to platform (for me, this usually means floating-point,
>>and platform math libraries, but there are surely many other things it
>>applies to).  But more recently there seem to have been some
>>difficulties keeping a reasonable number of buildbots up and running.
>>A secondary problem is that it can be awkward to debug some of the
>>more obscure test failures on buildbots without having direct access
>>to the machine.  From conversations on IRC, I don't think I'm alone in
>>wanting to find ways to make the buildbots more useful.
>
>These are actually two issues:
>a) where do we get buildbot hardware and operators?
>b) how can we reasonably debug problems occurring on buildbots
>
>For a), I think we can solve this only by redundancy, i.e. create
>more build slaves, hoping that a sufficient number would be up at
>any point in time.
>
>So: what specific kinds of buildbots do you think are currently 
>lacking?
>A call for volunteers will likely be answered quickly.
>>So the question is: how best to invest time and possibly money to
>>improve the buildbot situation (and as a result, I hope, improve the
>>quality of Python)?
>
>I don't think money will really help (I'm skeptical in general that
>money helps in open source projects). As for time: "buildbot scales",
>meaning that the buildbot slave admins will all share the load, being
>responsible only for their own slaves.

I think that money can help in two ways in this case.

First, there are now a multitude of cloud hosting providers which will 
operate a slave machine for you.  BuildBot has even begun to support 
this deployment use-case by allowing you to start up and shut down vms 
on demand to save on costs.  Amazon's EC2 service is supported out of 
the box in the latest release.

Second, there are a number of active BuildBot developers.  One of them 
has even recently taken a contract from Mozilla to implement some non- 
trivial BuildBot enhancements.  I think it very likely that he would 
consider taking such a contract from the PSF for whatever enhancements 
would help out the CPython buildbot.
>On the master side: would you be interested in tracking slave admins?
>>What could be done to make maintenance of build
>>slaves easier?
>
>This is something that only the slave admins can answer. I don't think
>it's difficult - it's just that people are really unlikely to 
>contribute
>to the same thing over a period of five years at a steady rate. So we
>need to make sure to find replacements when people drop out.

This is a good argument for VMs.  It's certainly *possible* to chase an 
ever changing set of platforms, but it strikes me as something of a 
waste of time.
>The source of the problem is that such a system can degrade without
>anybody taking action. If the web server's hard disk breaks down,
>people panic and look for a solution quickly. If the source control
>is down, somebody *will* "volunteer" to fix it. If the automated
>build system produces results less useful, people will worry, but
>not take action.

To me, that raises the question of why people aren't more concerned with 
the status of the build system.  Shouldn't developers care if the code 
they're writing works or not?

Jean-Paul

From jnoller at gmail.com  Sun Oct 25 15:10:31 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Sun, 25 Oct 2009 10:10:31 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025124816.GA20329@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
Message-ID: <4222a8490910250710i6e3d2959s5da2ddf6f53a9fa4@mail.gmail.com>

On Sun, Oct 25, 2009 at 8:48 AM, C. Titus Brown <ctb at msu.edu> wrote:

> [ x-posting to testing-in-python; please redirect followups to one list or
> the other! ]
>
> Hi Mark,
>
> a few bits of information...
>
> ---
>
> I have a set of VM machines running some "core" build archs -- Linux, Mac OS X,
> Win XP, Win Vista, and Win 7 -- that I am going to dedicate to this purpose.
> I am happy to give out remote admin access to a few people. ?This
> infrastructure is also going to increase slowly as I build up my lab's internal
> network.
>
> I'm giving Tarek an account on my Linux box later today to serve as a build
> slave for Distribute.

Just to add to what titus said; I'm trying to price out a decent
Desktop machine with enough ram/disk/cpu to run VMWare ESXi and serve
a variety of virtual machines. I had planned on (ab)using the free
MSDN license Microsoft provided to get a variety of platforms up and
running.

The end goal (since I have excess bandwidth and cooling where I live)
would be to maintain this box as a series of buildslaves python-dev
would have near unlimited shell/remote desktop access to.

The nice thing about this would be that once the initial cost was sunk
for the machine itself, and making all the virtual machines, in theory
the machine could run a set of "common" virtual machines all the time,
with a set of virtual machines on standby if someone needed to debug a
less common problem.

Yes, this is a mini-snakebite concept. Right now the main blocker is
funding and time - that and I need to spec something that doesn't
sound like a jet engine ;)

jesse

From exarkun at twistedmatrix.com  Sun Oct 25 15:13:17 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 25 Oct 2009 14:13:17 -0000
Subject: [Python-Dev] [TIP]  Possible language summit topic: buildbots
In-Reply-To: <20091025124816.GA20329@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
Message-ID: <20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>

On 12:48 pm, ctb at msu.edu wrote:
>
>[snip]
>
>The most *exciting* part of pony-build, apart from the always-riveting
>spectacle of "titus rediscovering problems that buildbot solved 5 years 
>ago",
>is the loose coupling of recording server to the build slaves and build
>reporters.  My plan is to enable a simple and lightweight XML-RPC 
>and/or
>REST-ish interface for querying the recording server from scripts or 
>other Web
>sites.  This has Brett aquiver with anticipation, I gather -- no more 
>visual
>inspection of buildbot waterfall pages ;)

BuildBot has an XML-RPC interface.  So Brett can probably do what he 
wants with BuildBot right now.

Jean-Paul

From ssteinerx at gmail.com  Sun Oct 25 15:27:55 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 25 Oct 2009 10:27:55 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE41E9E.4050706@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
Message-ID: <73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>


On Oct 25, 2009, at 5:47 AM, Martin v. L?wis wrote:
>
> These are actually two issues:
> a) where do we get buildbot hardware and operators?

I've been trying to get some feedback about firing up buildbots on  
Cloud Servers for a while now and haven't had much luck.  I'd love to  
find a way of having buildbots come to life, report to the mother  
ship, do the build, then go away 'till next time they're required.

S


From ssteinerx at gmail.com  Sun Oct 25 15:35:14 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 25 Oct 2009 10:35:14 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
Message-ID: <110EE89E-8313-4E08-8861-7772D0F2D722@gmail.com>


On Oct 25, 2009, at 9:50 AM, exarkun at twistedmatrix.com wrote:
> Actually setting one up in the first place might take a bit longer,  
> since it involves installing the necessary software and making sure  
> everything's set up right, but the actual slave configuration itself  
> is one command:
>
> buildbot create-slave <path> <master address> <slave name> <slave  
> password>

I have written a Fabric script for the distutils-buildbot project (on  
bitbucket, under Tarek) that puts everything necessary up onto an  
Ubuntu server, runs all the build steps, and fires up the buildbot.

Obviously it will have to be modified to correctly configure other  
types of servers but the implementation should be fairly trivial for  
someone who could have done it by hand in the first place.    Once  
it's done, it's in the script and may require an occasional tweak but  
not much more.

The next step is to have the slaves themselves created in the cloud,  
fired up and then report to the mother ship that they're available.   
This last step is the one that doesn't seem to be supported by the  
current system.

Thanks,

S


From solipsis at pitrou.net  Sun Oct 25 15:36:51 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Oct 2009 14:36:51 +0000 (UTC)
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
Message-ID: <loom.20091025T153356-443@post.gmane.org>

<exarkun <at> twistedmatrix.com> writes:
> 
> To me, that raises the question of why people aren't more concerned with 
> the status of the build system.  Shouldn't developers care if the code 
> they're writing works or not?

The fact that we ask questions and publicly express worries should hint that we
/are/ concerned :-)
However, being mostly developers rather than system admins, and not knowing
anything about the details of how buildbot does its work (not to mention the
details of this or that particular buildslave and slave owner), makes us (at
least me) quite clueless when faced with a buildbot-not-working-as-expected
problem.

Regards

Antoine.



From ssteinerx at gmail.com  Sun Oct 25 15:37:41 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 25 Oct 2009 10:37:41 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
Message-ID: <DD0B1F2E-E7C5-47A3-BE0F-3908A49EA3AB@gmail.com>


On Oct 25, 2009, at 10:05 AM, exarkun at twistedmatrix.com wrote:
> First, there are now a multitude of cloud hosting providers which  
> will operate a slave machine for you.  BuildBot has even begun to  
> support this deployment use-case by allowing you to start up and  
> shut down vms on demand to save on costs.  Amazon's EC2 service is  
> supported out of the box in the latest release.

I have been working to expand this support to Rackspace's Cloud  
Servers as well.

S


From foom at fuhm.net  Sun Oct 25 18:09:11 2009
From: foom at fuhm.net (James Y Knight)
Date: Sun, 25 Oct 2009 13:09:11 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <hc0sev$uo1$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>
	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>
	<hc0sev$uo1$1@ger.gmane.org>
Message-ID: <3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>


On Oct 25, 2009, at 2:50 AM, Terry Reedy wrote:

> Alex Martelli wrote:
>> Next(s) would seem good...
>
> That does not work. It has to be next(iter(s)), and that has been  
> tried and eliminated because it is significantly slower.

But who cares about the speed of getting an arbitrary element from a  
set? How can it *possibly* be a problem in a real program?

If you want to optimize python, this operation is certainly not the  
right place to start...

James

From p.f.moore at gmail.com  Sun Oct 25 18:47:54 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 25 Oct 2009 17:47:54 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
Message-ID: <79990c6b0910251047s3edb0d73mfdad918b04c68b1@mail.gmail.com>

2009/10/25  <exarkun at twistedmatrix.com>:
> Perhaps this is a significant portion of the problem. ?Maintaining a build
> slave is remarkably simple and easy. ?I maintain about half a dozen slaves
> and spend at most a few minutes a month operating them. Actually setting one
> up in the first place might take a bit longer, since it involves installing
> the necessary software and making sure everything's set up right, but the
> actual slave configuration itself is one command:
>
> ?buildbot create-slave <path> <master address> <slave name> <slave password>
>
> Perhaps this will help dispel the idea that it is a serious undertaking to
> operate a slave.
>
> The real requirement which some people may find challenging is that the
> slave needs to operate on a host which is actually online almost all of the
> time. ?If you don't such a machine, then there's little point offering to
> host a slave.

I have been seriously considering setting up one or more buildslaves
for a while now. However, my biggest issue is that they would be
running as VMs on my normal PC, which means that it's the issue of
keeping them continually online that hurts me.

If I could (say) just fire the slaves up for a set period, or fire
them up, have them do a build and report back, and then shut down,
that would make my life easier (regular activities rather than ongoing
sysadmin works better for me).

It sounds like a buildslave isn't really what I should be looking at.
Maybe Titus' push model pony-build project would make more sense for
me.

Paul.

From exarkun at twistedmatrix.com  Sun Oct 25 18:59:47 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 25 Oct 2009 17:59:47 -0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <79990c6b0910251047s3edb0d73mfdad918b04c68b1@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<79990c6b0910251047s3edb0d73mfdad918b04c68b1@mail.gmail.com>
Message-ID: <20091025175947.11571.1887104266.divmod.xquotient.1569@localhost.localdomain>

On 05:47 pm, p.f.moore at gmail.com wrote:
>2009/10/25  <exarkun at twistedmatrix.com>:
>>Perhaps this is a significant portion of the problem. ?Maintaining a 
>>build
>>slave is remarkably simple and easy. ?I maintain about half a dozen 
>>slaves
>>and spend at most a few minutes a month operating them. Actually 
>>setting one
>>up in the first place might take a bit longer, since it involves 
>>installing
>>the necessary software and making sure everything's set up right, but 
>>the
>>actual slave configuration itself is one command:
>>
>>?buildbot create-slave <path> <master address> <slave name> <slave 
>>password>
>>
>>Perhaps this will help dispel the idea that it is a serious 
>>undertaking to
>>operate a slave.
>>
>>The real requirement which some people may find challenging is that 
>>the
>>slave needs to operate on a host which is actually online almost all 
>>of the
>>time. ?If you don't such a machine, then there's little point offering 
>>to
>>host a slave.
>
>I have been seriously considering setting up one or more buildslaves
>for a while now. However, my biggest issue is that they would be
>running as VMs on my normal PC, which means that it's the issue of
>keeping them continually online that hurts me.
>
>If I could (say) just fire the slaves up for a set period, or fire
>them up, have them do a build and report back, and then shut down,
>that would make my life easier (regular activities rather than ongoing
>sysadmin works better for me).
>
>It sounds like a buildslave isn't really what I should be looking at.
>Maybe Titus' push model pony-build project would make more sense for
>me.

Maybe.  I wonder if Titus' "push model" (I don't really understand this 
term in this context) makes sense for continuous integration at all, 
though.  As a developer, I don't want to have access to build results 
across multiple platforms when someone else feels like it.  I want 
access when *I* feel like it.

Anyway, BuildBot is actually perfectly capable of dealing with this.  I 
failed to separate my assumptions about how everyone would want to use 
the system from what the system is actually capable of.

If you run a build slave and it's offline when a build is requested, the 
build will be queued and run when the slave comes back online.  So if 
the CPython developers want to work this way (I wouldn't), then we don't 
need pony-build; BuildBot will do just fine.

Jean-Paul

From p.f.moore at gmail.com  Sun Oct 25 19:15:38 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 25 Oct 2009 18:15:38 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025175947.11571.1887104266.divmod.xquotient.1569@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<79990c6b0910251047s3edb0d73mfdad918b04c68b1@mail.gmail.com>
	<20091025175947.11571.1887104266.divmod.xquotient.1569@localhost.localdomain>
Message-ID: <79990c6b0910251115x3983334bh12d618d3016a720f@mail.gmail.com>

2009/10/25  <exarkun at twistedmatrix.com>:
> If you run a build slave and it's offline when a build is requested, the
> build will be queued and run when the slave comes back online. ?So if the
> CPython developers want to work this way (I wouldn't), then we don't need
> pony-build; BuildBot will do just fine.

OK, sounds useful. If I'm offline for a while, do multiple builds get
queued, or only the "last" one? If the former, I can imagine coming
back to a pretty huge load if the slave breaks while I'm on holiday
:-(

I should look all of this up somewhere. Is there a reference to
buildbot for slave maintainers? Are there any specifics for Python
slaves that I should refer to?

(From what I've been able to find, it seems to me that setting up a
slave first requires getting things sorted with the admins, which
sadly precludes experimenting to find things out - I can understand
why the python admins don't want people "playing" on the live buildbot
infrastructure, though :-))

Paul.

From fw at deneb.enyo.de  Sun Oct 25 18:57:07 2009
From: fw at deneb.enyo.de (Florian Weimer)
Date: Sun, 25 Oct 2009 18:57:07 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net> (James
	Y. Knight's message of "Sun, 25 Oct 2009 13:09:11 -0400")
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>
	<4AE3AE3E.80402@canonical.com>
	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>
	<hc0sev$uo1$1@ger.gmane.org>
	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>
Message-ID: <87ljizxsp8.fsf@mid.deneb.enyo.de>

* James Y. Knight:

> On Oct 25, 2009, at 2:50 AM, Terry Reedy wrote:
>
>> Alex Martelli wrote:
>>> Next(s) would seem good...
>>
>> That does not work. It has to be next(iter(s)), and that has been
>> tried and eliminated because it is significantly slower.
>
> But who cares about the speed of getting an arbitrary element from a
> set? How can it *possibly* be a problem in a real program?

Hmm, perhaps when using sets as work queues?

From martin at v.loewis.de  Sun Oct 25 19:32:52 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 19:32:52 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>
Message-ID: <4AE499D4.4010104@v.loewis.de>

> I've been trying to get some feedback about firing up buildbots on Cloud
> Servers for a while now and haven't had much luck.  I'd love to find a
> way of having buildbots come to life, report to the mother ship, do the
> build, then go away 'till next time they're required.

I'm not quite sure whom you have been trying to get feedback from, and
can't quite picture your proposed setup from above description.

In any case, ISTM that your approach isn't compatible with how buildbot
works today (not sure whether you are aware of that): a build slave
needs to stay connected all the time, so that the build master can
trigger a build when necessary.

So if your setup requires the slaves to shut down after a build, I don't
think this can possibly work.

Regards,
Martin

From ctb at msu.edu  Sun Oct 25 19:38:59 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 25 Oct 2009 11:38:59 -0700
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE499D4.4010104@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>
	<4AE499D4.4010104@v.loewis.de>
Message-ID: <20091025183858.GA7460@idyll.org>

On Sun, Oct 25, 2009 at 07:32:52PM +0100, "Martin v. L?wis" wrote:
> > I've been trying to get some feedback about firing up buildbots on Cloud
> > Servers for a while now and haven't had much luck.  I'd love to find a
> > way of having buildbots come to life, report to the mother ship, do the
> > build, then go away 'till next time they're required.
> 
> I'm not quite sure whom you have been trying to get feedback from, and
> can't quite picture your proposed setup from above description.
> 
> In any case, ISTM that your approach isn't compatible with how buildbot
> works today (not sure whether you are aware of that): a build slave
> needs to stay connected all the time, so that the build master can
> trigger a build when necessary.

Hi Martin,

it shouldn't be difficult to cobble together a build script that spins up a
buildslave on EC2 and runs the tests there; I wrote something similar a
few years ago for an infrequently connected home machine.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From martin at v.loewis.de  Sun Oct 25 19:43:52 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 19:43:52 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <hbvl15$gne$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>	<4AE1E71C.7020806@gmail.com>	<hbsun9$432$1@ger.gmane.org>	<200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<ca471dc20910241055v57df990j7476e75465011309@mail.gmail.com>
	<hbvl15$gne$1@ger.gmane.org>
Message-ID: <4AE49C68.9090108@v.loewis.de>

> Given Alexander's premise, I agree with your response. But his premise
> is wrong. Python's current builtin set class maps abstract equivalence
> classes to representative members. And this *is* useful. Mapping
> arbitrary members of such classes to representative members, sometimes
> called 'canonical', is a standard problem/goal in math. String interning
> is an application of this idea.

Right. However, this is conceptually a function. In some cases (like
string interning), it is mutable (over time) and finite (at any point in
time). We already have a data type that can perfectly represent mutable
finite funcitons, namely dictionaries. And indeed, interning is
implemented by a dictionary.

What Alexander wants is that the set type also becomes useful for
storing canonical representations. I find that inappropriate, since
dicts already provide the one obvious way to do it.

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 19:45:54 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 19:45:54 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <4AE3E6C6.4010008@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<4AE1E71C.7020806@gmail.com>	<hbsun9$432$1@ger.gmane.org>	<200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>
	<4AE3E6C6.4010008@gmail.com>
Message-ID: <4AE49CE2.5050502@v.loewis.de>

> If you actually care about that aspect of the semantics for a particular
> application, it would be far clearer to use a full dict() and live with
> the additional memory usage. While I can see how saving that
> pointer-per-entry might make sense in some circumstances, for most I
> would see it as a needlessly confusing micro-optimisation.

That's exactly my view.

Regards,
Martin

From exarkun at twistedmatrix.com  Sun Oct 25 19:45:53 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 25 Oct 2009 18:45:53 -0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE499D4.4010104@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>
	<4AE499D4.4010104@v.loewis.de>
Message-ID: <20091025184553.31526.2015752889.divmod.xquotient.2@localhost.localdomain>




On 06:32 pm, martin at v.loewis.de wrote:
>>I've been trying to get some feedback about firing up buildbots on 
>>Cloud
>>Servers for a while now and haven't had much luck.  I'd love to find a
>>way of having buildbots come to life, report to the mother ship, do 
>>the
>>build, then go away 'till next time they're required.
>
>I'm not quite sure whom you have been trying to get feedback from, and
>can't quite picture your proposed setup from above description.
>
>In any case, ISTM that your approach isn't compatible with how buildbot
>works today (not sure whether you are aware of that): a build slave
>needs to stay connected all the time, so that the build master can
>trigger a build when necessary.
>
>So if your setup requires the slaves to shut down after a build, I 
>don't
>think this can possibly work.

This is supported in recent versions of BuildBot with a special kind of 
slave:

http://djmitche.github.com/buildbot/docs/0.7.11/#On_002dDemand- 
_0028_0022Latent_0022_0029-Buildslaves

Jean-Paul

From martin at v.loewis.de  Sun Oct 25 19:51:26 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 19:51:26 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <87ljizxsp8.fsf@mid.deneb.enyo.de>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>	<hc0sev$uo1$1@ger.gmane.org>	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>
	<87ljizxsp8.fsf@mid.deneb.enyo.de>
Message-ID: <4AE49E2E.3090506@v.loewis.de>

> Hmm, perhaps when using sets as work queues?

A number of comments:

- it's somewhat confusing to use a set as a *queue*, given
  that it won't provide FIFO semantics.
- there are more appropriate and direct container structures
  available, including a dedicated Queue module (even though
  this might be doing to much with its thread-safety).
- if you absolutely want to use a set as a work queue,
  then the .pop() method should be sufficient, right?

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 19:57:48 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 19:57:48 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <hc1fiq$5ec$1@ger.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>
Message-ID: <4AE49FAC.70207@v.loewis.de>

Antoine Pitrou wrote:
>> For a), I think we can solve this only by redundancy, i.e. create more
>> build slaves, hoping that a sufficient number would be up at any point
>> in time.
> 
> We are already doing this, aren't we?
> http://www.python.org/dev/buildbot/3.x/
> 
> It doesn't seem to work very well, it's a bit like a Danaides vessel.

Both true. However, it seems that Mark is unhappy with the current set
of systems, so we probably need to do it again.

> Well, to be fair, buildbots breaking also happens much more frequently 
> (perhaps one or two orders of magnitude) than the SVN server or the Web 
> site going down. Maintaining them looks like a Sisyphean task, and nobody 
> wants that.

It only looks so. It is like any server management task - it takes
constant effort. However, it is not Sisyphean (feeling Greek today,
ain't you :-); since you actually achieve something. It's not hard to
restart a buildbot when it has crashed, and it gives a warm feeling of
having achieved something.

> I don't know what kind of machines are the current slaves, but if they 
> are 24/7 servers, isn't it a bit surprising that the slaves would go down 
> so often? Is the buildbot software fragile? 

Not really. It sometimes happens that the slaves don't reconnect after
a master restart, but more often, it is just a change on the slave side
that breaks it (such as a reboot done to the machine, and not having
the machine configured to restart the slave after the reboot).

> Does it require a lot of 
> (maintenance, repair) work from the slave owners?

On Unix, not really. On Windows, there is still the issue that
sometimes, some error message pops up which you need to click away.
Over several builds, you may find that you have to click away dozens
of such messages. This could use some improvement.

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 20:06:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:06:49 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
Message-ID: <4AE4A1C9.5010002@v.loewis.de>

> As I have no specific experience maintaining any of the CPython build
> slaves, I can't speak to any maintenance issues which these slaves have
> encountered.  I would expect that they are as minimal as the issues I
> have encountered maintaining slaves for other projects, but perhaps this
> is wrong.  I do recall that there were some win32 issues (discussed on
> this list, I think) quite a while back, but I think those were resolved.
> I haven't heard of any other issues since then.

Only partially. One old issue was that previous builds would not
complete, keeping the executable files open, preventing further
runs. Buildbot is supposed to kill a build, but only kills the
parent process (as it is really difficult to kill the entire process
tree (*)). We work around this by explicitly killing any stale Python
processes at the beginning of a new build.

The remaining issue is the popups; if a process still has a popup,
you can't even terminate it properly. There are two kinds of popups:
system-generated ones, and CRT-generated ones. For the CRT ones, we
once had a way to turn them off, but I'm not sure whether that mechanism
might have been removed. For the system messages, there is a way to
turn them off in the parent process. David Bolen (IIRC) had developed
a patch, but I think this patch only runs on his system(s).

Regards,
Martin

(*) it may help if Buildbot would create a Win32 job object, and
then use TerminateJobObject. Contributions are welcome.

From brett at python.org  Sun Oct 25 20:12:05 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 25 Oct 2009 12:12:05 -0700
Subject: [Python-Dev] [TIP] Possible language summit topic: buildbots
In-Reply-To: <20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> 
	<20091025124816.GA20329@idyll.org>
	<20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
Message-ID: <bbaeab100910251212u7450e8f4q44fa784373712575@mail.gmail.com>

On Sun, Oct 25, 2009 at 07:13, <exarkun at twistedmatrix.com> wrote:

> On 12:48 pm, ctb at msu.edu wrote:
>
>>
>> [snip]
>>
>>
>> The most *exciting* part of pony-build, apart from the always-riveting
>> spectacle of "titus rediscovering problems that buildbot solved 5 years
>> ago",
>> is the loose coupling of recording server to the build slaves and build
>> reporters.  My plan is to enable a simple and lightweight XML-RPC and/or
>> REST-ish interface for querying the recording server from scripts or other
>> Web
>> sites.  This has Brett aquiver with anticipation, I gather -- no more
>> visual
>> inspection of buildbot waterfall pages ;)
>>
>
> BuildBot has an XML-RPC interface.  So Brett can probably do what he wants
> with BuildBot right now.
>

Brett actually wants web hooks so pony-build will ping an App Engine web app
when there is more data, ala PubSubHubbub. Or hell, just have pony-build
have an Atom feed with updates and simply use PuSH. In other words I want to
be told when there is an update, not have to poll to find out.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091025/8a46d9f6/attachment-0001.htm>

From martin at v.loewis.de  Sun Oct 25 20:15:09 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:15:09 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <79990c6b0910251115x3983334bh12d618d3016a720f@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<79990c6b0910251047s3edb0d73mfdad918b04c68b1@mail.gmail.com>	<20091025175947.11571.1887104266.divmod.xquotient.1569@localhost.localdomain>
	<79990c6b0910251115x3983334bh12d618d3016a720f@mail.gmail.com>
Message-ID: <4AE4A3BD.8010002@v.loewis.de>

> OK, sounds useful. If I'm offline for a while, do multiple builds get
> queued, or only the "last" one? 

IIRC, it will only build the last one, then with a huge blame list.

> If the former, I can imagine coming
> back to a pretty huge load if the slave breaks while I'm on holiday
> :-(

If it's offline too often, I'm skeptical that it would be useful. If
you report breakage after a day, then it will be difficult to attribute
this to a specific commit. It is most useful to have continuous
integration if error reports are instantaneous.

> I should look all of this up somewhere. Is there a reference to
> buildbot for slave maintainers? Are there any specifics for Python
> slaves that I should refer to?

Hmm. I thought I had send you this before:

http://wiki.python.org/moin/BuildbotOnWindows

> (From what I've been able to find, it seems to me that setting up a
> slave first requires getting things sorted with the admins, which
> sadly precludes experimenting to find things out - I can understand
> why the python admins don't want people "playing" on the live buildbot
> infrastructure, though :-))

That's really not an issue. Feel free to play as much as you want, with
the live infrastructure. You can't break anything doing so (perhaps
except for spamming the mailing list with faulty reports). If you then
decide to withdraw your offer, that's fine as well (just make sure to
notify us instead of just silently taking the slave down).

Regards,
Martin

From solipsis at pitrou.net  Sun Oct 25 21:22:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Oct 2009 21:22:20 +0100
Subject: [Python-Dev] Reworking the GIL
Message-ID: <1256502140.5621.163.camel@localhost>


Hello there,

The last couple of days I've been working on an experimental rewrite of
the GIL. Since the work has been turning out rather successful (or, at
least, not totally useless and crashing!) I thought I'd announce it
here.

First I want to stress this is not about removing the GIL. There still
is a Global Interpreter Lock which serializes access to most parts of
the interpreter. These protected parts haven't changed either, so Python
doesn't become really better at extracting computational parallelism out
of several cores.

Goals
-----

The new GIL (which is also the name of the sandbox area I've committed
it in, "newgil") addresses the following issues :

1) Switching by opcode counting. Counting opcodes is a very crude way of
estimating times, since the time spent executing a single opcode can
very wildly. Litterally, an opcode can be as short as a handful of
nanoseconds (think something like "... is not None") or as long as a
fraction of second, or even longer (think calling a heavy non-GIL
releasing C function, such as re.search()). Therefore, releasing the GIL
every 100 opcodes, regardless of their length, is a very poor policy.

The new GIL does away with this by ditching _Py_Ticker entirely and
instead using a fixed interval (by default 5 milliseconds, but settable)
after which we ask the main thread to release the GIL and let another
thread be scheduled.

2) GIL overhead and efficiency in contended situations. Apparently, some
OSes (OS X mainly) have problems with lock performance when the lock is
already taken: the system calls are heavy. This is the "Dave Beazley
effect", where he took a very trivial loop, therefore made of very short
opcodes and therefore releasing the GIL very often (probably 100000
times a second), and runs it in one or two threads on an OS with poor
lock performance (OS X). He sees a 50% increase in runtime when using
two threads rather than one, in what is admittedly a pathological case.

Even on better platforms such as Linux, eliminating the overhead of many
GIL acquires and releases (since the new GIL is released on a fixed time
basis rather than on an opcode counting basis) yields slightly better
performance (read: a smaller performance degradation :-)) when there are
several pure Python computation threads running.

3) Thread switching latency. The traditional scheme merely releases the
GIL for a couple of CPU cycles, and reacquires it immediately.
Unfortunately, this doesn't mean the OS will automatically switch to
another, GIL-awaiting thread. In many situations, the same thread will
continue running. This, with the opcode counting scheme, is the reason
why some people have been complaining about latency problems when an I/O
thread competes with a computational thread (the I/O thread wouldn't be
scheduled right away when e.g. a packet arrives; or rather, it would be
scheduled by the OS, but unscheduled immediately when trying to acquire
the GIL, and it would be scheduled again only much later).

The new GIL improves on this by combinating two mechanisms:
- forced thread switching, which means that when the switching interval
is terminated (mentioned in 1) and the GIL is released, we will force
any of the threads waiting on the GIL to be scheduled instead of the
formerly GIL-holding thread. Which thread exactly is an OS decision,
however: the goal here is not to have our own scheduler (this could be
discussed but I wanted the design to remain simple :-) After all,
man-years of work have been invested in scheduling algorithms by kernel
programming teams).
- priority requests, which is an option for a thread requesting the GIL
to be scheduled as soon as possible, and forcibly (rather than any other
threads). This is meant to be used by GIL-releasing methods such as
read() on files and sockets. The scheme, again, is very simple: when a
priority request is done by a thread, the GIL is released as soon as
possible by the thread holding it (including in the eval loop), and then
the thread making the priority request is forcibly scheduled (by making
all other GIL-awaiting threads wait in the meantime).

Implementation
--------------

The new GIL is implemented using a couple of mutexes and condition
variables. A {mutex, condition} pair is used to protect the GIL itself,
which is a mere variable named `gil_locked` (there are a couple of other
variables for bookkeeping). Another {mutex, condition} pair is used for
forced thread switching (described above). Finally, a separate mutex is
used for priority requests (described above).

The code is in the sandbox:
http://svn.python.org/view/sandbox/trunk/newgil/

The file of interest is Python/ceval_gil.h. Changes in other files are
very minimal, except for priority requests which have been added at
strategic places (some methods of I/O modules). Also, the code remains
rather short, while of course being less trivial than the old one.

NB : this is a branch of py3k. There should be no real difficulty
porting it back to trunk, provided someone wants to do the job.

Platforms
---------

I've implemented the new GIL for POSIX and Windows (tested under Linux
and Windows XP (running in a VM)). Judging by what I can read in the
online MSDN docs, the Windows support should include everything from
Windows 2000, and probably recent versions of Windows CE.

Other platforms aren't implemented, because I don't have access to the
necessary hardware. Besides, I must admit I'm not very motivated in
working on niche/obsolete systems. I've e-mailed Andrew MacIntyre in
private to ask him if he'd like to do the OS/2 support.

Supporting a new platform is not very difficult: it's a matter of
writing the 50-or-so lines of necessary platform-specific macros at the
beginning of Python/ceval_gil.h.

The reason I couldn't use the existing thread support
(Python/thread_*.h) is that these abstractions are too poor. Mainly,
they don't provide:
- events, conditions or an equivalent thereof
- the ability to acquire a resource with a timeout

Measurements
------------

Before starting this work, I wrote ccbench (*), a little benchmark
script ("ccbench" being a shorthand for "concurrency benchmark") which
measures two things:
- computation throughput with one or several concurrent threads
- latency to external events (I use an UDP socket) when there is zero,
one, or several background computation threads running

(*) http://svn.python.org/view/sandbox/trunk/ccbench/

The benchmark involves several computation workloads with different GIL
characteristics. By default there are 3 of them:
A- one pure Python workload (computation of a number of digits of pi):
that is, something which spends its time in the eval loop
B- one mostly C workload where the C implementation doesn't release the
GIL (regular expression matching)
C- one mostly C workload where the implementation does release the GIL
(bz2 compression)

In the ccbench directory you will find benchmark results, under Linux,
for two different systems I have here. The new GIL shows roughly similar
but slightly better throughput results than the old one. And it is much
better in the latency tests, especially in workload B (going down from
almost a second of average latency with the old GIL, to a couple of
milliseconds with the new GIL). This is the combined result of using a
time-based scheme (rather than opcode-based) and of forced thread
switching (rather than relying on the OS to actually switch threads when
we speculatively release the GIL).

As a sidenote, I might mention that single-threaded performance is not
degraded at all. It is, actually, theoretically a bit better because the
old ticker check in the eval loop becomes simpler; however, this goes
mostly unnoticed.


Now what remains to be done?

Having other people test it would be fine. Even better if you have an
actual multi-threaded py3k application. But ccbench results for other
OSes would be nice too :-)
(I get good results under the Windows XP VM but I feel that a VM is not
an ideal setup for a concurrency benchmark)

Of course, studying and reviewing the code is welcome. As for
integrating it into the mainline py3k branch, I guess we have to answer
these questions:
- is the approach interesting? (we could decide that it's just not worth
it, and that a good GIL can only be a dead (removed) GIL)
- is the patch good, mature and debugged enough?
- how do we deal with the unsupported platforms (POSIX and Windows
support should cover most bases, but the fate of OS/2 support depends on
Andrew)?

Regards

Antoine.



From ssteinerx at gmail.com  Sun Oct 25 20:19:05 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 25 Oct 2009 15:19:05 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE499D4.4010104@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>
	<4AE499D4.4010104@v.loewis.de>
Message-ID: <C071A21C-EF99-461A-A659-883BECE1F3A3@gmail.com>


On Oct 25, 2009, at 2:32 PM, Martin v. L?wis wrote:

>> I've been trying to get some feedback about firing up buildbots on  
>> Cloud
>> Servers for a while now and haven't had much luck.  I'd love to  
>> find a
>> way of having buildbots come to life, report to the mother ship, do  
>> the
>> build, then go away 'till next time they're required.
>
> I'm not quite sure whom you have been trying to get feedback from, and
> can't quite picture your proposed setup from above description.

I posted a couple of messages on testing-in-python, and have sent  
around some queries to others that I know are using buildbot type  
setups with various tools/platforms, not necessarily Python.

> In any case, ISTM that your approach isn't compatible with how  
> buildbot
> works today (not sure whether you are aware of that): a build slave
> needs to stay connected all the time, so that the build master can
> trigger a build when necessary.
>
> So if your setup requires the slaves to shut down after a build, I  
> don't
> think this can possibly work.

It can't possibly work within the way the Python buildbot structure  
currently works, as I understand it so far.

What I'm implementing is less of a 'continuous integration' tool like  
you would use for something like Python itself, and more of a testing  
tool for things that have to be installed on multiple versions of  
Python, on multiple platforms.

I don't need to know that it works on every checkin, just every once  
in a while I'd like to start from scratch and make sure everything  
still works on all supported versions of Python on all the platforms I  
test on and cloud servers are great for this since I'll usually only  
need them for an hour or so.

S



From solipsis at pitrou.net  Sun Oct 25 20:23:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Oct 2009 19:23:07 +0000 (UTC)
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<79990c6b0910251047s3edb0d73mfdad918b04c68b1@mail.gmail.com>	<20091025175947.11571.1887104266.divmod.xquotient.1569@localhost.localdomain>
	<79990c6b0910251115x3983334bh12d618d3016a720f@mail.gmail.com>
	<4AE4A3BD.8010002@v.loewis.de>
Message-ID: <loom.20091025T202116-645@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> If it's offline too often, I'm skeptical that it would be useful. If
> you report breakage after a day, then it will be difficult to attribute
> this to a specific commit. It is most useful to have continuous
> integration if error reports are instantaneous.

Not only, but it's useful to have a stable set of buildbots, so that when a test
fails, you know whether it's a new problem caused by a recent revision, or
rather an already existing problem on this platform.

Regards

Antoine.



From martin at v.loewis.de  Sun Oct 25 20:24:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:24:12 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
Message-ID: <4AE4A5DC.2010906@v.loewis.de>

> I think that money can help in two ways in this case.
> 
> First, there are now a multitude of cloud hosting providers which will
> operate a slave machine for you.  BuildBot has even begun to support
> this deployment use-case by allowing you to start up and shut down vms
> on demand to save on costs.  Amazon's EC2 service is supported out of
> the box in the latest release.

Here I'm skeptical. I think we can find people donating always-online
machines still; no need to throw donated money to Amazon.

> Second, there are a number of active BuildBot developers.  One of them
> has even recently taken a contract from Mozilla to implement some non-
> trivial BuildBot enhancements.  I think it very likely that he would
> consider taking such a contract from the PSF for whatever enhancements
> would help out the CPython buildbot.

That could indeed be interesting, assuming we had a clear requirement.
But then, most of us can "easily" fix things in buildbot themselves -
this is python-dev, after all.

>> This is something that only the slave admins can answer. I don't think
>> it's difficult - it's just that people are really unlikely to contribute
>> to the same thing over a period of five years at a steady rate. So we
>> need to make sure to find replacements when people drop out.
> 
> This is a good argument for VMs.

Not really - you still would need somebody to manage them.

> It's certainly *possible* to chase an
> ever changing set of platforms, but it strikes me as something of a
> waste of time.

Hmm - can you really get "strange" operating systems "in the cloud"?
Some of the operating systems that we would like to test don't
even support VMs.

> To me, that raises the question of why people aren't more concerned with
> the status of the build system.  Shouldn't developers care if the code
> they're writing works or not?

I think there are two issues here:
1. some developers actually *don't* care to much whether their code
   works in all cases. If it fails on some strange platform they never
   heard of (such as "Solaris", or "Windows"), they are not bothered by
   the failure. Or, if they care, they still don't know what to do about
   the failure.
2. the buildbots sometimes report false positives. Some tests fail in
   a non-repeatable fashion, only on selected systems. So when you
   change something, the tests break, and you cannot really see how
   this could be possibly related to your change. Then you start
   ignoring these reports - both the bogus ones, and the real ones.

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 20:27:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:27:39 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025183858.GA7460@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	<4AE499D4.4010104@v.loewis.de>
	<20091025183858.GA7460@idyll.org>
Message-ID: <4AE4A6AB.8020801@v.loewis.de>

> it shouldn't be difficult to cobble together a build script that spins up a
> buildslave on EC2 and runs the tests there; I wrote something similar a
> few years ago for an infrequently connected home machine.

Ok - so it would be the master running this script? Sounds reasonable to me.

As for EC2 (and other cloud providers); I'm somewhat skeptical about
platform coverage, also. How many different processors and operating
systems can they possibly support?

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 20:33:31 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:33:31 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025184553.31526.2015752889.divmod.xquotient.2@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	<4AE499D4.4010104@v.loewis.de>
	<20091025184553.31526.2015752889.divmod.xquotient.2@localhost.localdomain>
Message-ID: <4AE4A80B.1050107@v.loewis.de>

> This is supported in recent versions of BuildBot with a special kind of
> slave:
> 
> http://djmitche.github.com/buildbot/docs/0.7.11/#On_002dDemand-
> _0028_0022Latent_0022_0029-Buildslaves

Interesting. Coming back to "PSF may spend money", let me say this:

If somebody would volunteer to set up slaves in EC2, and operate them,
I'm fairly certain that the PSF would pay the bill. It might be useful
to have two people operating them, so that the knowledge and the load
is shared.

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 20:35:22 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:35:22 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <C071A21C-EF99-461A-A659-883BECE1F3A3@gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	<4AE499D4.4010104@v.loewis.de>
	<C071A21C-EF99-461A-A659-883BECE1F3A3@gmail.com>
Message-ID: <4AE4A87A.1060602@v.loewis.de>

> I don't need to know that it works on every checkin

For us, that is a fairly important requirement, though.
Reports get more and more useless if they aren't instantaneous.
Sometimes, people check something in just to see how the build
slaves react.

Regards,
Martin

From ssteinerx at gmail.com  Sun Oct 25 20:38:00 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 25 Oct 2009 15:38:00 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE4A87A.1060602@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	<4AE499D4.4010104@v.loewis.de>
	<C071A21C-EF99-461A-A659-883BECE1F3A3@gmail.com>
	<4AE4A87A.1060602@v.loewis.de>
Message-ID: <D914BF6E-CA5A-4AD1-B162-67735C838592@gmail.com>


On Oct 25, 2009, at 3:35 PM, Martin v. L?wis wrote:

>> I don't need to know that it works on every checkin
>
> For us, that is a fairly important requirement, though.
> Reports get more and more useless if they aren't instantaneous.
> Sometimes, people check something in just to see how the build
> slaves react.

Understood -- that's why I mentioned it.  This is a different use-case.

It may still have some use for Python itself, but my idea is more for  
testing things like libraries where the developer may only work on or  
have one platform and may want to test installing on other platforms  
and Python versions during development and/or before release.

S


From martin at v.loewis.de  Sun Oct 25 20:42:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 20:42:11 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091025124816.GA20329@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
Message-ID: <4AE4AA13.8010903@v.loewis.de>

> The most *exciting* part of pony-build, apart from the always-riveting
> spectacle of "titus rediscovering problems that buildbot solved 5 years ago",
> is the loose coupling of recording server to the build slaves and build
> reporters.  My plan is to enable a simple and lightweight XML-RPC and/or
> REST-ish interface for querying the recording server from scripts or other Web
> sites.  This has Brett aquiver with anticipation, I gather -- no more visual
> inspection of buildbot waterfall pages ;)

If that's something that people want to have, then buildbot could also
provide it, of course. Do you have a spec of the interface somewhere?

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 21:10:41 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 25 Oct 2009 21:10:41 +0100
Subject: [Python-Dev] [TIP] Possible language summit topic: buildbots
In-Reply-To: <bbaeab100910251212u7450e8f4q44fa784373712575@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>	<20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
	<bbaeab100910251212u7450e8f4q44fa784373712575@mail.gmail.com>
Message-ID: <4AE4B0C1.5060706@v.loewis.de>

> Brett actually wants web hooks so pony-build will ping an App Engine web
> app when there is more data, ala PubSubHubbub. Or hell, just have
> pony-build have an Atom feed with updates and simply use PuSH. In other
> words I want to be told when there is an update, not have to poll to
> find out.

Not sure what exactly it is that Brett wants to do, but perhaps Brett
could take a look at

http://www.python.org/dev/buildbot/all/atom

As JP says, there is also XML-RPC (at all/xmlrpc)

For a true push notifications: buildbot sends messages into an
IRC channel - not sure whether an App Engine App could listen
to that.

Regards,
Martin

From martin at v.loewis.de  Sun Oct 25 21:19:13 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 21:19:13 +0100
Subject: [Python-Dev] Buildbot alternate renderings
Message-ID: <4AE4B2C1.7080504@v.loewis.de>

Triggered by the recent discussion, I looked at the changes in status
display that the buildbot people have implemented recently. I added
a few links into alternate renderings; see

http://www.python.org/dev/buildbot/

Regards,
Martin

From p.f.moore at gmail.com  Sun Oct 25 22:17:40 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 25 Oct 2009 21:17:40 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE499D4.4010104@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>
	<4AE499D4.4010104@v.loewis.de>
Message-ID: <79990c6b0910251417l515d29b7iaedd2a338eb0b9d2@mail.gmail.com>

2009/10/25 "Martin v. L?wis" <martin at v.loewis.de>:
>> I've been trying to get some feedback about firing up buildbots on Cloud
>> Servers for a while now and haven't had much luck. ?I'd love to find a
>> way of having buildbots come to life, report to the mother ship, do the
>> build, then go away 'till next time they're required.
>
> I'm not quite sure whom you have been trying to get feedback from, and
> can't quite picture your proposed setup from above description.

Sorry, feedback was the wrong word. I've been digging round the
documentation I've been able to find and looking into what's needed to
set up a slave.

> In any case, ISTM that your approach isn't compatible with how buildbot
> works today (not sure whether you are aware of that): a build slave
> needs to stay connected all the time, so that the build master can
> trigger a build when necessary.
>
> So if your setup requires the slaves to shut down after a build, I don't
> think this can possibly work.

It's not so much that I *require* the slave to shut down, more that
I'm not sure how well I'll be able to ensure that it's up all the
time, and I'm trying to understand the implications of that. My basic
impression is that it's not really going to work, unfortunately.

Paul.

From glyph at twistedmatrix.com  Sun Oct 25 22:36:08 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Sun, 25 Oct 2009 17:36:08 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE4A1C9.5010002@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<4AE4A1C9.5010002@v.loewis.de>
Message-ID: <BEA8AEEC-AE6F-4A74-8F08-6381AFCFE492@twistedmatrix.com>

On Oct 25, 2009, at 3:06 PM, Martin v. L?wis wrote:

> (*) it may help if Buildbot would create a Win32 job object, and
> then use TerminateJobObject. Contributions are welcome.

Some work has already been done on this, but it needs help.  At the  
root it's a Twisted issue:

     http://twistedmatrix.com/trac/ticket/2726


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091025/ad93750e/attachment.htm>

From db3l.net at gmail.com  Sun Oct 25 22:36:31 2009
From: db3l.net at gmail.com (David Bolen)
Date: Sun, 25 Oct 2009 17:36:31 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<4AE4A1C9.5010002@v.loewis.de>
Message-ID: <m2aazfb1gg.fsf@valheru.db3l.homeip.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> The remaining issue is the popups; if a process still has a popup,
> you can't even terminate it properly. There are two kinds of popups:
> system-generated ones, and CRT-generated ones. For the CRT ones, we
> once had a way to turn them off, but I'm not sure whether that mechanism
> might have been removed. For the system messages, there is a way to
> turn them off in the parent process. David Bolen (IIRC) had developed
> a patch, but I think this patch only runs on his system(s).

Yes, process-stopping dialogs have probably been the single most
annoying issue over time running a Windows build slave - certainly
from my perspective in terms of maintenance and detection they have
taken up the largest amount of time.

I believe the CRT disabling is still active in the 3.x branches (the
"-n" flag to regrtest in the buildbot's test.bat), after you restored
it this past March (it had regressed during an earlier py3k branch set
of patches and caused a bunch of problems for a bit) but not in the
2.x branches or trunk.  So there's still a bit of exposure there and
I'd certainly be in favor of porting the regrtest -n support over to
all current development branches.

I think the other issue most likely to cause a perceived "downtime"
with the Windows build slave that I've had a handful of cases over the
past two years where the build slave appears to be operating properly,
but the master seems to just queue up jobs as if it were down.  The
slave still shows an established TCP link to the master, so I
generally only catch this when I happen to peek at the status web
page, or catch a remark here on python-dev, so that can reduce
availability.

My build slave (based on 0.7.5 I think) runs with local patches to:

1. Protect again Win32 pop-up error boxes in child processes.
2. A fix for a recursive chaining of Twisted Deferreds during uploads which
   could break for large file transfers.  This only came up when my build
   slave was generating daily MSI builds and uploading them to the master.
3. Handle clock jumps.  It's a general flaw in the presence of system clock
   adjustments, but I only encountered it while in my FreeBSD build slave
   under VMWare with a Linux host.

(2) and (3) are useful, but not likely to be an issue with most build
slaves in normal operation.  (2) probably isn't needed on my end any
more now that the daily MSI builds aren't run, and it's possible that
it got corrected in later buildbot updates, since I did report it on
the development list at the time.

(1) is a pretty trivial patch, but added a dependency on pywin32, so
passing it back up to the buildbot maintainers (back in 2007) stalled
while determining if that was ok, and I don't I ever closed the loop.
I did later make it fall back to ctypes if pywin32 was missing, but
then I think buildbot was using a minimum of Python 2.3 at the time,
so even ctypes was a new dependency.  Anyway, it became less crucial
when Python's regrtest started executing similar code, though the
buildbot patch covers anything run under it and not just the python
process.

I'd of course be happy to pass along the patch to anyone interested.

I believe that Thomas Heller had run his Windows buildbot with some
similar local code, but implemented with a modified buildbot script
for building Python, rather than tweaking buildbot itself.

Of course, the patch only protects against system pop-ups - it can't
control the CRT assertion dialogs when Python is built in debug mode,
which is why I've argued in the past that the test process for Python
should ensure those are disabled.  The CRT errors themselves are still
important, but can be redirected to stderr rather than a blocking GUI
dialog.

-- David


From martin at v.loewis.de  Sun Oct 25 22:43:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 25 Oct 2009 22:43:03 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <79990c6b0910251417l515d29b7iaedd2a338eb0b9d2@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	
	<4AE41E9E.4050706@v.loewis.de>	
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	
	<4AE499D4.4010104@v.loewis.de>
	<79990c6b0910251417l515d29b7iaedd2a338eb0b9d2@mail.gmail.com>
Message-ID: <4AE4C667.5030305@v.loewis.de>


> It's not so much that I *require* the slave to shut down, more that
> I'm not sure how well I'll be able to ensure that it's up all the
> time, and I'm trying to understand the implications of that. My basic
> impression is that it's not really going to work, unfortunately.

There is a significant difference between "not able to ensure that
it is up all the time", and "it is down most of the time, and only
up once a day for a short period of time".

Regular and irregular short downtimes are no problem at all. When
the slave comes back, it will pick up pending work. Longer
scheduled downtimes (e.g. for vacations) are acceptable.
Only turning on the slave occasionally makes it useless.

Regards,
Martin

From ssteinerx at gmail.com  Sun Oct 25 22:50:50 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 25 Oct 2009 17:50:50 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE4C667.5030305@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	
	<4AE41E9E.4050706@v.loewis.de>	
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	
	<4AE499D4.4010104@v.loewis.de>
	<79990c6b0910251417l515d29b7iaedd2a338eb0b9d2@mail.gmail.com>
	<4AE4C667.5030305@v.loewis.de>
Message-ID: <09C914A0-C9F2-46B4-9133-BDC9B67C828F@gmail.com>


On Oct 25, 2009, at 5:43 PM, Martin v. L?wis wrote:

> Only turning on the slave occasionally makes it useless.

For certain use cases; not mine.

S


From solipsis at pitrou.net  Sun Oct 25 22:56:42 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Oct 2009 21:56:42 +0000 (UTC)
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	
	<4AE41E9E.4050706@v.loewis.de>	
	<73E21491-E39E-4CFA-8C77-2E9AFDFCF3BE@gmail.com>	
	<4AE499D4.4010104@v.loewis.de>
	<79990c6b0910251417l515d29b7iaedd2a338eb0b9d2@mail.gmail.com>
	<4AE4C667.5030305@v.loewis.de>
	<09C914A0-C9F2-46B4-9133-BDC9B67C828F@gmail.com>
Message-ID: <loom.20091025T225544-619@post.gmane.org>

ssteinerX <at> gmail.com <ssteinerx <at> gmail.com> writes:
> On Oct 25, 2009, at 5:43 PM, Martin v. L?wis wrote:
> 
> > Only turning on the slave occasionally makes it useless.
> 
> For certain use cases; not mine.

Let's say that for the use case we are talking here (this is python-dev),
Martin's statement holds true.




From brett at python.org  Sun Oct 25 22:57:21 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 25 Oct 2009 14:57:21 -0700
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1256502140.5621.163.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
Message-ID: <bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>

>
> [SNIP - a lot of detail on what sounds like a good design]
>
> Now what remains to be done?
>
> Having other people test it would be fine. Even better if you have an
> actual multi-threaded py3k application. But ccbench results for other
> OSes would be nice too :-)
> (I get good results under the Windows XP VM but I feel that a VM is not
> an ideal setup for a concurrency benchmark)
>
> Of course, studying and reviewing the code is welcome. As for
> integrating it into the mainline py3k branch, I guess we have to answer
> these questions:
> - is the approach interesting? (we could decide that it's just not worth
> it, and that a good GIL can only be a dead (removed) GIL)
>

I think it's worth it. Removal of the GIL is a totally open-ended problem
with no solution in sight. This, on the other hand, is a performance benefit
now. I say move forward with this. If it happens to be short-lived because
some actually figures out how to remove the GIL then great, but is that
really going to happen between now and Python 3.2? I doubt it.


> - is the patch good, mature and debugged enough?
> - how do we deal with the unsupported platforms (POSIX and Windows
> support should cover most bases, but the fate of OS/2 support depends on
> Andrew)?
>
>
It's up to Andrew to get the support in. While I have faith he will, this is
why we have been scaling back the support for alternative OSs for a while
and will continue to do so. I suspect the day Andrew stops keeping up will
be the day we push to have OS/2 be externally maintained.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091025/0b663d91/attachment-0001.htm>

From steve at pearwood.info  Mon Oct 26 00:17:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 26 Oct 2009 10:17:03 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE49C68.9090108@v.loewis.de>
References: <200910231132.45504.w.richert@gmx.net>
	<hbvl15$gne$1@ger.gmane.org> <4AE49C68.9090108@v.loewis.de>
Message-ID: <200910261017.04518.steve@pearwood.info>

On Mon, 26 Oct 2009 05:43:52 am Martin v. L?wis wrote:
> What Alexander wants is that the set type also becomes useful for
> storing canonical representations. I find that inappropriate, since
> dicts already provide the one obvious way to do it.

Only because dicts came first in Python rather than sets. There's 
nothing inherently obvious about storing the canonical representation 
of an object *twice* (although I'm not Dutch). A more natural 
representation would be to store the canonical representation once.

An abstract intern operation might be written:

if s equals an element in the data store
    return the element which s equals
otherwise
    insert s into the data store and return s

Notice we don't say:

if s equals an element in the data store
    return the value associated with the element which s equals

which is what the dict-based solution does.

We can implement that abstract algorithm using lists. The algorithm is 
exactly the same as it is for dicts, except that search/retrieval 
becomes two operations rather than one:

_cache = ['x', 'spam', 'parrot']
def intern(s):
    try:
        s = _cache[ _cache.index(s) ]
    except ValueError:
        _cache.append(s)
    return s

We don't do this because O(N) searching is too expensive. Using a dict 
{s: s} is a workaround for the lack of a data structure that combines 
O(1) searching and ability to retrieve the element just found.

Memory is cheap, but not so cheap that doubling the size of a data 
structure (two pointers per element for {s:s} versus one for {s}) is 
insignificant. Part of the reason we intern in the first place is to 
save memory. We shouldn't dismiss this out of hand based on the 
argument that retrieval from a set is insufficiently pure. As soon as 
you allow iteration over sets, you have allowed retrieval.



-- 
Steven D'Aprano

From python at mrabarnett.plus.com  Mon Oct 26 00:54:09 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 25 Oct 2009 23:54:09 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <m2aazfb1gg.fsf@valheru.db3l.homeip.net>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<4AE4A1C9.5010002@v.loewis.de>
	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>
Message-ID: <4AE4E521.600@mrabarnett.plus.com>

David Bolen wrote:
[snip]
> I think the other issue most likely to cause a perceived "downtime"
> with the Windows build slave that I've had a handful of cases over the
> past two years where the build slave appears to be operating properly,
> but the master seems to just queue up jobs as if it were down.  The
> slave still shows an established TCP link to the master, so I
> generally only catch this when I happen to peek at the status web
> page, or catch a remark here on python-dev, so that can reduce
> availability.
> 
[snip]
Couldn't you write a script to check the status periodically?


From exarkun at twistedmatrix.com  Mon Oct 26 01:58:15 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Mon, 26 Oct 2009 00:58:15 -0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <m2aazfb1gg.fsf@valheru.db3l.homeip.net>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<4AE4A1C9.5010002@v.loewis.de>
	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>
Message-ID: <20091026005815.31526.130687717.divmod.xquotient.88@localhost.localdomain>

On 25 Oct, 09:36 pm, db3l.net at gmail.com wrote:
>
>I think the other issue most likely to cause a perceived "downtime"
>with the Windows build slave that I've had a handful of cases over the
>past two years where the build slave appears to be operating properly,
>but the master seems to just queue up jobs as if it were down.  The
>slave still shows an established TCP link to the master, so I
>generally only catch this when I happen to peek at the status web
>page, or catch a remark here on python-dev, so that can reduce
>availability.

This sounds like something that should be reported upstream. 
Particularly if you know how to reproduce it.  Has it been?

Jean-Paul

From db3l.net at gmail.com  Mon Oct 26 02:28:33 2009
From: db3l.net at gmail.com (David Bolen)
Date: Sun, 25 Oct 2009 21:28:33 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<4AE4A1C9.5010002@v.loewis.de>
	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>
	<20091026005815.31526.130687717.divmod.xquotient.88@localhost.localdomain>
Message-ID: <m263a3aqpq.fsf@valheru.db3l.homeip.net>

exarkun at twistedmatrix.com writes:

> This sounds like something that should be reported
> upstream. Particularly if you know how to reproduce it.  Has it been?

No, largely because I can't reproduce it at all.  It's happened maybe
4-5 times in the past 2 years or so.  All that I see is that my end
looks good yet the master end seems not to be dispatching jobs (it
never shows an explicit disconnect for my slave though).

My best guess is that something disrupted the TCP connection, and that
the slave isn't doing anything that would let it know its connection
was dropped.  Although I thought there were periodic pings even from
the slave side.

Given the frequency, it's not quite high priority to me, though having
the master let the owner of a slave know when it's down would help cut
down on lost availability due to this case, so I suppose I could suggest
that feature to the buildbot developers.

-- David


From db3l.net at gmail.com  Mon Oct 26 02:31:55 2009
From: db3l.net at gmail.com (David Bolen)
Date: Sun, 25 Oct 2009 21:31:55 -0400
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<4AE4A1C9.5010002@v.loewis.de>
	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>
	<4AE4E521.600@mrabarnett.plus.com>
Message-ID: <m21vkraqk4.fsf@valheru.db3l.homeip.net>

MRAB <python at mrabarnett.plus.com> writes:

> Couldn't you write a script to check the status periodically?

Sure, I suppose scraping the web status page would work.  If it
happened frequently I'd probably be forced to do something like that,
but it's relatively low frequency (though I guess it does have a big
impact in terms of availability) makes it hard to dedicate time to
that compared to my "real world" work :-)

-- David


From exarkun at twistedmatrix.com  Mon Oct 26 02:56:50 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Mon, 26 Oct 2009 01:56:50 -0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <m263a3aqpq.fsf@valheru.db3l.homeip.net>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de> <hc1fiq$5ec$1@ger.gmane.org>
	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>
	<4AE4A1C9.5010002@v.loewis.de>
	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>
	<20091026005815.31526.130687717.divmod.xquotient.88@localhost.localdomain>
	<m263a3aqpq.fsf@valheru.db3l.homeip.net>
Message-ID: <20091026015650.31526.600593069.divmod.xquotient.94@localhost.localdomain>

On 01:28 am, db3l.net at gmail.com wrote:
>exarkun at twistedmatrix.com writes:
>>This sounds like something that should be reported
>>upstream. Particularly if you know how to reproduce it.  Has it been?
>
>No, largely because I can't reproduce it at all.  It's happened maybe
>4-5 times in the past 2 years or so.  All that I see is that my end
>looks good yet the master end seems not to be dispatching jobs (it
>never shows an explicit disconnect for my slave though).
>
>My best guess is that something disrupted the TCP connection, and that
>the slave isn't doing anything that would let it know its connection
>was dropped.  Although I thought there were periodic pings even from
>the slave side.
>
>Given the frequency, it's not quite high priority to me, though having
>the master let the owner of a slave know when it's down would help cut
>down on lost availability due to this case, so I suppose I could 
>suggest
>that feature to the buildbot developers.

This feature exists, at least.  BuildBot can email people when slaves 
are offline for more than some configured time limit.  I'm not sure if 
the CPython master is configured to do this or not.

It's easy to set up if not, the BuildSlave initializer accepts a list of 
email addresses that will be notified when that slave goes offline, 
notify_on_missing:

http://buildbot.net/apidocs/buildbot.buildslave.AbstractBuildSlave- 
class.html#__init__

Jean-Paul

From john at arbash-meinel.com  Mon Oct 26 03:20:51 2009
From: john at arbash-meinel.com (John Arbash Meinel)
Date: Sun, 25 Oct 2009 21:20:51 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <4AE49E2E.3090506@v.loewis.de>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>	<hc0sev$uo1$1@ger.gmane.org>	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>	<87ljizxsp8.fsf@mid.deneb.enyo.de>
	<4AE49E2E.3090506@v.loewis.de>
Message-ID: <4AE50783.90303@arbash-meinel.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
>> Hmm, perhaps when using sets as work queues?
> 
> A number of comments:
> 
> - it's somewhat confusing to use a set as a *queue*, given
>   that it won't provide FIFO semantics.
> - there are more appropriate and direct container structures
>   available, including a dedicated Queue module (even though
>   this might be doing to much with its thread-safety).
> - if you absolutely want to use a set as a work queue,
>   then the .pop() method should be sufficient, right?
> 
> Regards,
> Martin

We were using sets to track the tips of a graph, and to compare whether
one node was an ancestor of another one. We were caching that answer
into frozensets, since that made them immutable. If

res = heads(node1, node2)
if len(res) == 1:
  # What is the 'obvious' way to get the node out?

I posit that there *isn't* an obvious way to get the single item out of
a 1-entry frozenset.

for x in res: break
list(res)[0]
set(res).pop()
iter(res).next()
[x for x in res][0]
x, = res   # I didn't think of this one before recently

Are all answers, but none of them I would consider *obvious*. At the
least, none of them are obviously better than another, so you look into
the performance characteristics to give you a reason to pick one over
the other.

res.get() would be a fairly obvious way to do it. Enough that I would
probably never have gone searching for any of the other answers. Though
personally, I think I would call it "set.peek()", but the specific name
doesn't really matter to me.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrlB4MACgkQJdeBCYSNAAN0lQCgrtyXWlqIbjj01qB4AKPhKrMq
QH8An0z2gCWZHoceEJsqRJOUdEl/VLTB
=fJXI
-----END PGP SIGNATURE-----

From debatem1 at gmail.com  Mon Oct 26 03:46:31 2009
From: debatem1 at gmail.com (geremy condra)
Date: Sun, 25 Oct 2009 22:46:31 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE50783.90303@arbash-meinel.com>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>
	<4AE3AE3E.80402@canonical.com>
	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>
	<hc0sev$uo1$1@ger.gmane.org>
	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>
	<87ljizxsp8.fsf@mid.deneb.enyo.de> <4AE49E2E.3090506@v.loewis.de>
	<4AE50783.90303@arbash-meinel.com>
Message-ID: <f3cc57c60910251946v88d69eerf8c10c3f76ca4de7@mail.gmail.com>

> Martin v. L?wis wrote:
>>> Hmm, perhaps when using sets as work queues?
>>
>> A number of comments:
>>
>> - it's somewhat confusing to use a set as a *queue*, given
>> ? that it won't provide FIFO semantics.
>> - there are more appropriate and direct container structures
>> ? available, including a dedicated Queue module (even though
>> ? this might be doing to much with its thread-safety).
>> - if you absolutely want to use a set as a work queue,
>> ? then the .pop() method should be sufficient, right?
>>
>> Regards,
>> Martin
>
> We were using sets to track the tips of a graph, and to compare whether
> one node was an ancestor of another one. We were caching that answer
> into frozensets, since that made them immutable. If
>
> res = heads(node1, node2)
> if len(res) == 1:
> ?# What is the 'obvious' way to get the node out?
>
> I posit that there *isn't* an obvious way to get the single item out of
> a 1-entry frozenset.
>
> for x in res: break
> list(res)[0]
> set(res).pop()
> iter(res).next()
> [x for x in res][0]
> x, = res ? # I didn't think of this one before recently
>
> Are all answers, but none of them I would consider *obvious*. At the
> least, none of them are obviously better than another, so you look into
> the performance characteristics to give you a reason to pick one over
> the other.
>
> res.get() would be a fairly obvious way to do it. Enough that I would
> probably never have gone searching for any of the other answers. Though
> personally, I think I would call it "set.peek()", but the specific name
> doesn't really matter to me.
>
> John
>

When I first wrote Graphine (graph library), I did something very
similar to the last solution. The code has since been rewritten to
avoid the issue, but it would be nice if it didn't have to be the
next time it comes up- the comma is easy to miss, and while the
results are common-sense once you get what the line is doing,
it doesn't lend itself to immediate comprehension.

Geremy Condra

From tjreedy at udel.edu  Mon Oct 26 06:07:26 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 26 Oct 2009 01:07:26 -0400
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1256502140.5621.163.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
Message-ID: <hc3aqd$eh2$1@ger.gmane.org>

Antoine Pitrou wrote:
> Hello there,
> 
> The last couple of days I've been working on an experimental rewrite of
> the GIL. Since the work has been turning out rather successful (or, at
> least, not totally useless and crashing!) I thought I'd announce it
> here.

I am curious as to whether the entire mechanism is or can be turned off 
when not needed -- when there are not threads (other than the main, 
starting thread)?

tjr


From martin at v.loewis.de  Mon Oct 26 06:53:30 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 26 Oct 2009 06:53:30 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <4AE50783.90303@arbash-meinel.com>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>	<hc0sev$uo1$1@ger.gmane.org>	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>	<87ljizxsp8.fsf@mid.deneb.enyo.de>
	<4AE49E2E.3090506@v.loewis.de> <4AE50783.90303@arbash-meinel.com>
Message-ID: <4AE5395A.7010902@v.loewis.de>

> We were using sets to track the tips of a graph, and to compare whether
> one node was an ancestor of another one. We were caching that answer
> into frozensets, since that made them immutable. If
> 
> res = heads(node1, node2)
> if len(res) == 1:
>   # What is the 'obvious' way to get the node out?

What specifically is that that you want to do in this case?
IIUC, the possible result could be that either node1 is an ancestor
of node2, or vice versa.

So I would write that as

if len(res) == 1:
   if node1 in res:
     # action to take if node1 is the head
   else:
     # action to take if node2 is the head
else:
   # they are unrelated

In any case, this is a use case different from "work queue" (AFAICT),
so I'm unsure why you responded to my message where Florian and me were
talking about work queues.

> res.get() would be a fairly obvious way to do it. Enough that I would
> probably never have gone searching for any of the other answers. Though
> personally, I think I would call it "set.peek()", but the specific name
> doesn't really matter to me.

Somebody proposed to call it .any(); this I like best (except that one
might expect that any takes a predicate as the argument).

Regards,
Martin

From Scott.Daniels at Acm.Org  Mon Oct 26 07:14:06 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun, 25 Oct 2009 23:14:06 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE50783.90303@arbash-meinel.com>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>	<hc0sev$uo1$1@ger.gmane.org>	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>	<87ljizxsp8.fsf@mid.deneb.enyo.de>	<4AE49E2E.3090506@v.loewis.de>
	<4AE50783.90303@arbash-meinel.com>
Message-ID: <hc3e2c$js3$1@ger.gmane.org>

John Arbash Meinel wrote:
> res = heads(node1, node2)
> if len(res) == 1:
>   # What is the 'obvious' way to get the node out?
> 
> I posit that there *isn't* an obvious way to get the single item out of
> a 1-entry frozenset.
> 
> for x in res: break
> list(res)[0]
> set(res).pop()
> iter(res).next()
> [x for x in res][0]
> x, = res   # I didn't think of this one before recently
> 
> Are all answers, but none of them I would consider *obvious*. 
And from my SQL-hacking experience:

     x = min(s)

--Scott David Daniels
Scott.Daniels at Acm.Org


From python at rcn.com  Mon Oct 26 08:29:36 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 26 Oct 2009 00:29:36 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<4AE1E71C.7020806@gmail.com><hbsun9$432$1@ger.gmane.org>
	<4AE20E0D.4050405@gmail.com>
Message-ID: <B29044CF942646EDB5D208ACEBEF5B49@RaymondLaptop1>


[John Arbash Meine]
> So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
> faster than (iter(s).next()).

The first version uses direct calls for the for-loop opcodes.
The other two have to do functions/method look-up and
make a pure python function call (neither is very fast).

FWIW, a "choose" method had been previously proposed,
discussed and rejected.

I don't find the optimization issue to be very interesting in
the case of retrieving an arbitrary element.  This isn't
the kind of thing that typically appears in an inner-loop;
afterall, if you've retrieved an arbitrary element without
removing it, then successive calls to "choose" could
potentially retrieve the exact same element again and again.


Raymond

From stephen at xemacs.org  Mon Oct 26 08:48:39 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 26 Oct 2009 16:48:39 +0900
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <4AE5395A.7010902@v.loewis.de>
References: <200910231132.45504.w.richert@gmx.net>
	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>
	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>
	<4AE3AE3E.80402@canonical.com>
	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>
	<hc0sev$uo1$1@ger.gmane.org>
	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>
	<87ljizxsp8.fsf@mid.deneb.enyo.de> <4AE49E2E.3090506@v.loewis.de>
	<4AE50783.90303@arbash-meinel.com> <4AE5395A.7010902@v.loewis.de>
Message-ID: <873a56a948.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > > res.get() would be a fairly obvious way to do it. Enough that I would
 > > probably never have gone searching for any of the other answers. Though
 > > personally, I think I would call it "set.peek()", but the specific name
 > > doesn't really matter to me.
 > 
 > Somebody proposed to call it .any(); this I like best (except that one
 > might expect that any takes a predicate as the argument).

Why not allow that?

    def any(self, predicate=lambda x: True, default=None)
        for a in self:
            if predicate(a):
                break
        else:
            return default
        return a



From martin at v.loewis.de  Mon Oct 26 09:02:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 26 Oct 2009 09:02:10 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <873a56a948.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <200910231132.45504.w.richert@gmx.net>	<2987c46d0910231004p216e68d3tdc93a68e9462b4f6@mail.gmail.com>	<aac2c7cb0910241309n6c3170cfu886ba07982fa40ee@mail.gmail.com>	<4AE3AE3E.80402@canonical.com>	<BF8D86B7-587E-4229-B503-93A706597047@gmail.com>	<hc0sev$uo1$1@ger.gmane.org>	<3699AEA0-B268-4DDE-8739-8CC9CE7C7FB1@fuhm.net>	<87ljizxsp8.fsf@mid.deneb.enyo.de>	<4AE49E2E.3090506@v.loewis.de>	<4AE50783.90303@arbash-meinel.com>	<4AE5395A.7010902@v.loewis.de>
	<873a56a948.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4AE55782.3070805@v.loewis.de>

> Why not allow that?
> 
>     def any(self, predicate=lambda x: True, default=None)
>         for a in self:
>             if predicate(a):
>                 break
>         else:
>             return default
>         return a

I'm +0 (given that I'm still skeptical about the need to have something
like this). Also setting aside the moratorium here, which may disallow
that kind of change for a foreseeable feature in any form.

Regards,
Martin

From w.richert at gmx.net  Mon Oct 26 09:22:37 2009
From: w.richert at gmx.net (Willi Richert)
Date: Mon, 26 Oct 2009 09:22:37 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <B29044CF942646EDB5D208ACEBEF5B49@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net> <4AE20E0D.4050405@gmail.com>
	<B29044CF942646EDB5D208ACEBEF5B49@RaymondLaptop1>
Message-ID: <200910260922.37375.w.richert@gmx.net>

Hi,

I totally agree regarding the efficiency. Code that relies on a fast "non-
removing .pop()" probably has other worse bottlenecks that should be targetted 
first.

This would, however, relief every programmer who needs this the first time in 
his Python experience, to research how this could be most reasonably done. And 
it is more of a style question. I find the "for x in some_set: break" rather 
ugly.

wr

Am Montag, 26. Oktober 2009 08:29:36 schrieben Sie:
> I don't find the optimization issue to be very interesting in
> the case of retrieving an arbitrary element.  This isn't
> the kind of thing that typically appears in an inner-loop;
> afterall, if you've retrieved an arbitrary element without
> removing it, then successive calls to "choose" could
> potentially retrieve the exact same element again and again.
> 
> 
> Raymond

From martin at v.loewis.de  Mon Oct 26 09:25:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 26 Oct 2009 09:25:41 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <m21vkraqk4.fsf@valheru.db3l.homeip.net>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<4AE4A1C9.5010002@v.loewis.de>	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>	<4AE4E521.600@mrabarnett.plus.com>
	<m21vkraqk4.fsf@valheru.db3l.homeip.net>
Message-ID: <4AE55D05.3050305@v.loewis.de>

David Bolen wrote:
> MRAB <python at mrabarnett.plus.com> writes:
> 
>> Couldn't you write a script to check the status periodically?
> 
> Sure, I suppose scraping the web status page would work.  If it
> happened frequently I'd probably be forced to do something like that,
> but it's relatively low frequency (though I guess it does have a big
> impact in terms of availability) makes it hard to dedicate time to
> that compared to my "real world" work :-)

In addition, if it was happening frequently, we would rather investigate
the problem and fix it, than working around.

Regards,
Martin

From martin at v.loewis.de  Mon Oct 26 09:30:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 26 Oct 2009 09:30:49 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091026015650.31526.600593069.divmod.xquotient.94@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<4AE4A1C9.5010002@v.loewis.de>	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>	<20091026005815.31526.130687717.divmod.xquotient.88@localhost.localdomain>	<m263a3aqpq.fsf@valheru.db3l.homeip.net>
	<20091026015650.31526.600593069.divmod.xquotient.94@localhost.localdomain>
Message-ID: <4AE55E39.20000@v.loewis.de>

>>> This sounds like something that should be reported
>>> upstream. Particularly if you know how to reproduce it.  Has it been?
>>
>> No, largely because I can't reproduce it at all.  It's happened maybe
>> 4-5 times in the past 2 years or so.  All that I see is that my end
>> looks good yet the master end seems not to be dispatching jobs (it
>> never shows an explicit disconnect for my slave though).

It's not really reproducible. I think it sometimes happens when I
restart the master; sometimes, some clients fail to reconnect
(properly).

> It's easy to set up if not, the BuildSlave initializer accepts a list of
> email addresses that will be notified when that slave goes offline,
> notify_on_missing:
> 
> http://buildbot.net/apidocs/buildbot.buildslave.AbstractBuildSlave-
> class.html#__init__

I tried that out a couple of weeks ago, but never received any email.
I didn't have the time to look into this further since.

Regards,
Martin

From solipsis at pitrou.net  Mon Oct 26 11:19:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Oct 2009 10:19:00 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost> <hc3aqd$eh2$1@ger.gmane.org>
Message-ID: <loom.20091026T111126-392@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:
> 
> I am curious as to whether the entire mechanism is or can be turned off 
> when not needed -- when there are not threads (other than the main, 
> starting thread)?

It is an implicit feature: when no thread is waiting on the GIL, the GIL-holding
thread isn't notified and doesn't try to release it at all (in the eval loop,
that is; GIL-releasing C extensions still release it).

Note that "no thread is waiting on the GIL" can mean one of two things:
- either there is only one Python thread
- or the other Python threads are doing things with the GIL released (zlib/bz2
compression, waiting on I/O, sleep()ing, etc.)

So, yes, it automatically "turns itself off".

Regards

Antoine.



From eckhardt at satorlaser.com  Mon Oct 26 12:10:13 2009
From: eckhardt at satorlaser.com (Ulrich Eckhardt)
Date: Mon, 26 Oct 2009 12:10:13 +0100
Subject: [Python-Dev] Clean up Python/thread_*.h ?
In-Reply-To: <4AE35098.6070306@cheimes.de>
References: <loom.20091024T192722-458@post.gmane.org> 
	<4AE35098.6070306@cheimes.de>
Message-ID: <200910261210.13736.eckhardt@satorlaser.com>

On Saturday 24 October 2009, Christian Heimes wrote:
> Antoine Pitrou wrote:
> > * The unused file: thread_wince.h
> >
> > Windows CE has actually been using thread_nt.h since January 2009 (see
> > http://bugs.python.org/issue4893 ). thread_wince.h is still there, but
> > it's unused and grep brings no instance of it being mentioned anywhere in
> > the source tree.
>
> What about older versions of Windows CE? Maybe they still require the
> wince thread header?

Windows CE support in recent 2.x (and probably 3.x) is broken due to heavy 
bitrot. I can't imagine any CE platform this code would compile against. 
Further, I don't see many people using CE < 4, with CE 6 being current, in 
fact they are _very_ rare judging from Usenet activities.

For current CE versions, porting the NT-threads code is the easier way out.

+1 for removal of thread_wince.{h,c}

Uli

-- 
Sator Laser GmbH
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932

**************************************************************************************
Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
           Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************


From andymac at bullseye.apana.org.au  Mon Oct 26 13:37:14 2009
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Mon, 26 Oct 2009 23:37:14 +1100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
Message-ID: <4AE597FA.7070701@bullseye.andymac.org>

Brett Cannon wrote:
> It's up to Andrew to get the support in. While I have faith he will, 
> this is why we have been scaling back the support for alternative OSs 
> for a while and will continue to do so. I suspect the day Andrew stops 
> keeping up will be the day we push to have OS/2 be externally maintained.

Notwithstanding my desire to keep OS/2 supported in the Python tree,
keeping up has been more difficult of late:
- OS/2 is unquestionably a "legacy" environment, with system APIs
different in flavour and semantics from the current mainstream (though
surprisingly capable in many ways despite its age).
- The EMX runtime my OS/2 port currently relies on to abstract the 
system API to a Posix-ish API is itself a legacy package, essentially
unmaintained for some years :-(  This has been a source of increasing
pain as Python has moved with the mainstream... with regard to Unicode
support and threads in conjunction with multi-processing, in particular.

Real Life hasn't been favourably disposed either...

I have refrained from applying the extensive patches required to make
the port feature complete for 2.6 and later while I investigate an
alternate Posix emulating runtime (derived from FreeBSD's C library,
and which is used by Mozilla on OS/2), which would allow me to dispense
with most of these patches.  But it has an issue or two of its own...

The cost in effort has been compounded by effectively having to try and
maintain two ports - 2.x and 3.x.  And the 3.x port has suffered more
as its demands are higher.

So while I asked to keep the OS/2 thread support alive, if a decision 
were to be taken to remove OS/2 support from the Python 3.x sources I
could live with that.  A completed migration to Mercurial might well
make future port maintenance easier for me.

Regards,
Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia

From sturla at molden.no  Mon Oct 26 14:43:04 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 26 Oct 2009 14:43:04 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1256502140.5621.163.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
Message-ID: <4AE5A768.6080900@molden.no>

Antoine Pitrou skrev:
> - priority requests, which is an option for a thread requesting the GIL
> to be scheduled as soon as possible, and forcibly (rather than any other
> threads). 
So Python threads become preemptive rather than cooperative? That would 
be great. :-)

time.sleep should generate a priority request to re-acquire the GIL; and 
so should all other blocking standard library functions with a time-out.


S.M.


From kristjan at ccpgames.com  Mon Oct 26 15:09:43 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 26 Oct 2009 14:09:43 +0000
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AE5A768.6080900@molden.no>
References: <1256502140.5621.163.camel@localhost> <4AE5A768.6080900@molden.no>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF42768@exchis.ccp.ad.local>



> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Sturla Molden
> time.sleep should generate a priority request to re-acquire the GIL;
> and
> so should all other blocking standard library functions with a time-
> out.

I don't agree.  You have to be very careful with priority.  time.sleep() does not promise to wake up in any timely manner, and neither do the timeout functions.  Rather, the timeout is a way to prevent infinite wait.

In my experience (from stackless python) using priority wakeup for IO can result in very erratic scheduling when there is much IO going on, every IO trumping another.  You should stick to round robin except for very special and carefully analysed cases.
K

From ssteinerx at gmail.com  Mon Oct 26 15:25:25 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Mon, 26 Oct 2009 10:25:25 -0400
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF42768@exchis.ccp.ad.local>
References: <1256502140.5621.163.camel@localhost> <4AE5A768.6080900@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF42768@exchis.ccp.ad.local>
Message-ID: <B68ADB0D-3022-45F5-ADA4-20E62F4B12DB@gmail.com>


On Oct 26, 2009, at 10:09 AM, Kristj?n Valur J?nsson wrote:

>
>
>> -----Original Message-----
>> From: python-dev-bounces+kristjan=ccpgames.com at python.org
>> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On  
>> Behalf
>> Of Sturla Molden
>> time.sleep should generate a priority request to re-acquire the GIL;
>> and
>> so should all other blocking standard library functions with a time-
>> out.
>
> I don't agree.  You have to be very careful with priority.   
> time.sleep() does not promise to wake up in any timely manner, and  
> neither do the timeout functions.  Rather, the timeout is a way to  
> prevent infinite wait.
>
> In my experience (from stackless python) using priority wakeup for  
> IO can result in very erratic scheduling when there is much IO going  
> on, every IO trumping another.  You should stick to round robin  
> except for very special and carefully analysed cases.

All the IO tasks can also go in their own round robin so that CPU time  
is correctly shared among all waiting IO tasks.

IOW, to make sure that all IO tasks get a fair share *in relation to  
all other IO tasks*.

Tasks can be put into the IO round robin when they "pull the IO alarm"  
so to speak, so there's no need to decide before-hand which task goes  
in which round robin pool.

I'm not familiar with this particular code in Python, but I've used  
this in other systems for years to make sure that IO tasks don't  
starve the rest of the system and that the most "insistent" IO task  
doesn't starve all the others.

S


From alexander.belopolsky at gmail.com  Mon Oct 26 16:04:06 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 26 Oct 2009 11:04:06 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <4AE3E6C6.4010008@gmail.com>
References: <200910231132.45504.w.richert@gmx.net> <4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org> <200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
Message-ID: <d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>

On Sun, Oct 25, 2009 at 1:48 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Terry Reedy wrote:
>>>  In this model, a
>>> get() method, or even a __getitem__ with s[k] is k, is only natural.
>>
>> No, if key and value were the same thing, the get method would be
>> nonesense, as Guido said. But since they are not, since the implict key
>> is an abstract class, retrieving the representative, perhaps canonical
>> object given *any* member *is* natural. Being able to do so is also a
>> standard practice in mathematics.
>
> To formalise this invariant to some degree: given a set "s" and two
> items "k1" and "k2", such that "k1 in s" and "k2 in s" and "k1 == k2",
> there is a single item "k" in s such that "k1 == k" and "k2 == k".
>
> If __getitem__ were to be provided by sets, then the last part of that
> invariant could be written "s[k1] is s[k2]".
>

Thanks, Nick and Terry for a much more precise formulation of the
point that I was trying to present.  When I wrote s[k] is k, I had in
mind for k stored is the set or among the keys of an equivalent dict.
Formally alltrue(s[k] is k for k in s).  Nick's invariant is of course
a better expression of the same idea.

I believe interning is a worthwhile use case for Python to have "one
obvious way to do it."   Martin suggests that such a way already
exists and it involves storing interned objects as both keys and
values in a dictionary.  While this may have been obvious before sets
became available, I agree with Steven that in post 2.4 python users
are likely to look at set first and will only turn to dictionary after
discovering that the functionality that they are looking for is not
available in set.

Even if you know to use a dictionary, there are still some non-obvious
tricks to learn about.  First, you need to learn about setdefault, a
much criticized method that led to a new class being introduced to the
standard library:

http://mail.python.org/pipermail/python-dev/2003-February/033321.html
http://docs.python.org/library/collections.html?highlight=defaultdict#collections.defaultdict

Second, there is no obvious way to pre-seed the dictionary, i.e.,
given a list l of keys,

d = {}
for k in l:
   d[k] = k

When I was looking for a dict method to accomplish that, I discovered
dict.fromkeys, which of course was not the right method.  I still
don't know if a better solution exists than calling setdefault(k, k)
in a loop.

As experience with defaultdict has shown, it may be more appropriate
to introduce a specialized and properly named class in collections
rather than overloading set with new methods, but such approach would
lead to steep learning curve.  Collections.defaultdict, could be
cross-referenced from dict.setdefault, but a hypothetical
collections.interningset would most likely remain undiscovered for the
majority of users.

Here is an alternative idea on how storing interned objects in a set
can be supported.  Currently set.add method returns None and had no
effect when set already has an object equal to the one being added.  I
propose to consider changing that behavior to make set.add return the
added object or the set member that is equal to the object being
added.  It is unlikely that many programs rely on the return value
being None (with doctests being a probable exception), so adding this
feature is unlikely to cause much grief.

From sturla at molden.no  Mon Oct 26 16:46:34 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 26 Oct 2009 16:46:34 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1256502140.5621.163.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
Message-ID: <4AE5C45A.9090900@molden.no>

Antoine Pitrou skrev:
> - priority requests, which is an option for a thread requesting the GIL
> to be scheduled as soon as possible, and forcibly (rather than any other
> threads). T
Should a priority request for the GIL take a priority number?

- If two threads make a priority requests for the GIL, the one with the 
higher priority should get the GIL first. 

- If a thread with a low priority make a priority request for the GIL, 
it should not be allowed to "preempt" (take the GIL away from) a 
higher-priority thread, in which case the priority request would be 
ignored. 

Related issue: Should Python threads have priorities? They are after all 
real OS threads.

S.M.


From solipsis at pitrou.net  Mon Oct 26 16:58:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Oct 2009 15:58:49 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost> <4AE5C45A.9090900@molden.no>
Message-ID: <loom.20091026T165251-219@post.gmane.org>

Sturla Molden <sturla <at> molden.no> writes:
> 
> Antoine Pitrou skrev:
> > - priority requests, which is an option for a thread requesting the GIL
> > to be scheduled as soon as possible, and forcibly (rather than any other
> > threads). T
> Should a priority request for the GIL take a priority number?

Er, I prefer to keep things simple. If you have lots of I/O you should probably
use an event loop rather than separate threads.

> Related issue: Should Python threads have priorities? They are after all 
> real OS threads.

Well, precisely they are OS threads, and the OS already assigns them (static or
dynamic) priorities. No need to replicate this.

(to answer another notion expressed in another message, there's no "round-robin"
scheduling either)

Regards

Antoine.



From tjreedy at udel.edu  Mon Oct 26 17:02:05 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 26 Oct 2009 12:02:05 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com>	<hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>
	<4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
Message-ID: <hc4h5s$vch$1@ger.gmane.org>

Alexander Belopolsky wrote:

> Here is an alternative idea on how storing interned objects in a set
> can be supported.  Currently set.add method returns None and had no
> effect when set already has an object equal to the one being added.  I
> propose to consider changing that behavior to make set.add return the
> added object or the set member that is equal to the object being
> added.  It is unlikely that many programs rely on the return value
> being None (with doctests being a probable exception), so adding this
> feature is unlikely to cause much grief.

I had exactly the same idea, but did not post because it violates the 
general rule that mutators return None. On the other hand, the returned 
value here would not be the mutated collection, so no chaining is 
possible. And 'add' is clearly intended to change something.

On the other hand, frozensets do not have an add method.

Terry Jan Reedy


From daniel at stutzbachenterprises.com  Mon Oct 26 17:18:59 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 26 Oct 2009 11:18:59 -0500
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091026T165251-219@post.gmane.org>
References: <1256502140.5621.163.camel@localhost> <4AE5C45A.9090900@molden.no>
	<loom.20091026T165251-219@post.gmane.org>
Message-ID: <eae285400910260918t7596115bg77c48265f7d62790@mail.gmail.com>

On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> Er, I prefer to keep things simple. If you have lots of I/O you should
> probably
> use an event loop rather than separate threads.
>

On Windows, sometimes using a single-threaded event loop is sometimes
impossible.  WaitForMultipleObjects(), which is the Windows equivalent to
select() or poll(), can handle a maximum of only 64 objects.

Do we really need priority requests at all?  They seem counter to your
desire for simplicity and allowing the operating system's scheduler to do
its work.

That said, if a thread's time budget is merely paused during I/O rather than
reset, then a thread making frequent (but short) I/O requests cannot starve
the system.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091026/c99a9f24/attachment.htm>

From guido at python.org  Mon Oct 26 17:38:13 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Oct 2009 09:38:13 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <hc4h5s$vch$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com> 
	<hbsun9$432$1@ger.gmane.org> <200910240946.48542.steve@pearwood.info> 
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com> 
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com> 
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
Message-ID: <ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>

All this talk of equivalence classes makes me dizzy. :-)

- If sets were to grow an API to non-destructively access the object
stored in the set for a particular key, then dicts should have such a
method too.

- Ditto for an API to non-destructively get an arbitrary element.

- I'm far from convinced that we urgently need either API. But I'm
also not convinced it's unneeded.

- I still wish we could go back in time and unify sets and dicts, if
only to find out how that experiment would turn out.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.

From alexander.belopolsky at gmail.com  Mon Oct 26 17:43:09 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 26 Oct 2009 12:43:09 -0400
Subject: [Python-Dev] One obvious way to do interning [Was: Retrieve an
	arbitrary element from a set without removing it]
Message-ID: <d38f5330910260943r74d67d42l8791b6cf17e90b23@mail.gmail.com>

Changing the subject to reflect branched discussion and forwarding to
python-ideas where it probably belongs.

On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Alexander Belopolsky wrote:
>
>> Here is an alternative idea on how storing interned objects in a set
>> can be supported. ?Currently set.add method returns None and has no
>> effect when set already has an object equal to the one being added. ?I
>> propose to consider changing that behavior to make set.add return the
>> added object or the set member that is equal to the object being
>> added. ?It is unlikely that many programs rely on the return value
>> being None (with doctests being a probable exception), so adding this
>> feature is unlikely to cause much grief.
>
> I had exactly the same idea, but did not post because it violates the
> general rule that mutators return None.

Is there such a rule?  What about set/dict pop?

> On the other hand, the returned
> value here would not be the mutated collection, so no chaining is possible.

I assume you refer to chaining as in s.add(1).add(2) which would be
enabled if s.add(..) returned s.  My proposal would enable a different
type of "chaining" which I would find useful, but ready to hear
objections:

v = compute_value()
s.add(v)
# use v

can, with my proposal, be rewritten as v = s.add(compute_value()) with
an added benefit that v that is used is the "canonical" value.

> And 'add' is clearly intended to change something.
>
Is this an argument for or against the proposal?

> On the other hand, frozensets do not have an add method.

However, PySet_Add "works with frozenset instances (like
PyTuple_SetItem() it can be used to fill-in the values of brand new
frozensets before they are exposed to other code). "
<http://docs.python.org/3.1/c-api/set.html#PySet_Add>

I will experiment with changing  PySet_Add to see how much it would
break and whether it will be helpful in implementing set-based
interning of python strings.

From solipsis at pitrou.net  Mon Oct 26 19:10:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Oct 2009 18:10:12 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost> <4AE5C45A.9090900@molden.no>
	<loom.20091026T165251-219@post.gmane.org>
	<eae285400910260918t7596115bg77c48265f7d62790@mail.gmail.com>
Message-ID: <loom.20091026T190502-788@post.gmane.org>

Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> 
> Do we really need priority requests at all?? They seem counter to your
> desire for simplicity and allowing the operating system's scheduler to do
> its work.

No, they can be disabled (removed) if we prefer. With priority requests
disabled, latency results becomes less excellent but still quite good.

Running ccbench on a dual core machine gives the following latency results,
first with then without priority requets.

--- Latency --- (with prio requests)

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 0 ms. (std dev: 2 ms.)
CPU threads=3: 0 ms. (std dev: 2 ms.)
CPU threads=4: 0 ms. (std dev: 2 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 3 ms. (std dev: 2 ms.)
CPU threads=2: 3 ms. (std dev: 2 ms.)
CPU threads=3: 3 ms. (std dev: 2 ms.)
CPU threads=4: 4 ms. (std dev: 3 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 2 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 2 ms.)
CPU threads=4: 0 ms. (std dev: 1 ms.)

--- Latency --- (without prio requests)

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 2 ms.)
CPU threads=1: 5 ms. (std dev: 0 ms.)
CPU threads=2: 3 ms. (std dev: 3 ms.)
CPU threads=3: 9 ms. (std dev: 7 ms.)
CPU threads=4: 22 ms. (std dev: 23 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 1 ms.)
CPU threads=1: 8 ms. (std dev: 2 ms.)
CPU threads=2: 5 ms. (std dev: 4 ms.)
CPU threads=3: 21 ms. (std dev: 32 ms.)
CPU threads=4: 19 ms. (std dev: 26 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 1 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)




From chris at subtlety.com  Mon Oct 26 19:54:03 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Mon, 26 Oct 2009 13:54:03 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE1E71C.7020806@gmail.com> 
	<hbsun9$432$1@ger.gmane.org> <200910240946.48542.steve@pearwood.info> 
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com> 
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com> 
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
Message-ID: <7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>

On Mon, Oct 26, 2009 at 11:38 AM, Guido van Rossum <guido at python.org> wrote:
> - If sets were to grow an API to non-destructively access the object
> stored in the set for a particular key, then dicts should have such a
> method too.
>
> - Ditto for an API to non-destructively get an arbitrary element.
>
> - I'm far from convinced that we urgently need either API. But I'm
> also not convinced it's unneeded.

   These clearly aren't urgently needed, but I do think they're needed
and useful.  For those who want a use-case for getting an arbitrary
element from a set, I've run into the need several times over the last
year, and each time I'm a little surprised I had the need and a little
surprised there wasn't an good way of going about it.
   In the most recent example, I was writing some UI code.  I had a
set of all the open window references so I could clean them up at the
end of the program.  I needed to call some API function that required
a window reference as the first argument, but it returned a global
value common to all the window references.

   I like the proposed set.get() method, personally.  list.get(index)
gets the item at that index, dict.get(key) gets the item associated
with that key, set.get() gets an item, but doesn't place any
guarantees on which item is returned.  Makes sense to me.  I also like
the idea there aren't any guarantees about which item is returned--it
allows subclasses to implement different behavior (so one might always
return the last item placed in the set, another could always return a
random item, another could implement some round-robin behavior, and
all would fulfill the contract of the set class).
   The existing methods aren't great for accomplishing this, mainly
because they're obfuscatory.  "iter(s).next()" is probably clearest,
and at least will throw a StopIteration exception if the set is empty.
 "for x in s: break" is just confusing, easy to accidentally confuse
with "for x in s: pass", and causes an unrevealing NameError if the
set is empty.  Add in all the other idioms for accomplishing the same
thing ("x, = s", etc.) and I think there's a good argument for adding
the method to sets, if only to provide a single obvious way of doing
it--and throwing a single, appropriate exception if the set is empty.

-- Chris

From tjreedy at udel.edu  Mon Oct 26 20:20:55 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 26 Oct 2009 15:20:55 -0400
Subject: [Python-Dev] One obvious way to do interning [Was: Retrieve an
 arbitrary element from a set without removing it]
In-Reply-To: <d38f5330910260943r74d67d42l8791b6cf17e90b23@mail.gmail.com>
References: <d38f5330910260943r74d67d42l8791b6cf17e90b23@mail.gmail.com>
Message-ID: <hc4sqm$cek$1@ger.gmane.org>

Alexander Belopolsky wrote:
> Changing the subject to reflect branched discussion and forwarding to
> python-ideas where it probably belongs.
> 
> On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> Alexander Belopolsky wrote:
>>
>>> Here is an alternative idea on how storing interned objects in a set
>>> can be supported.  Currently set.add method returns None and has no
>>> effect when set already has an object equal to the one being added.  I
>>> propose to consider changing that behavior to make set.add return the
>>> added object or the set member that is equal to the object being
>>> added.  It is unlikely that many programs rely on the return value
>>> being None (with doctests being a probable exception), so adding this
>>> feature is unlikely to cause much grief.
>> I had exactly the same idea, but did not post because it violates the
>> general rule that mutators return None.
> 
> Is there such a rule?  What about set/dict pop?

The rule perhaps should be restated as 'Collection mutators return None 
or possible an item but not the collection.'

>> On the other hand, the returned
>> value here would not be the mutated collection, so no chaining is possible.


>> And 'add' is clearly intended to change something.
>>
> Is this an argument for or against the proposal?

Mildly for.

>> On the other hand, frozensets do not have an add method.
> 
> However, PySet_Add "works with frozenset instances (like
> PyTuple_SetItem() it can be used to fill-in the values of brand new
> frozensets before they are exposed to other code). "
> <http://docs.python.org/3.1/c-api/set.html#PySet_Add>
> 
> I will experiment with changing  PySet_Add to see how much it would
> break and whether it will be helpful in implementing set-based
> interning of python strings.

Terry Jan Reedy



From python at rcn.com  Mon Oct 26 20:23:31 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 26 Oct 2009 12:23:31 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
References: <200910231132.45504.w.richert@gmx.net><4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
Message-ID: <55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>


Chris Bergstresser]
>   I like the proposed set.get() method, personally. 

Sets have been implemented in many languages, but  I've only 
seen one that implemented this functionality  (the "arb" function 
in SETL).  For the most part, it seems that this is an uncommon need.

Also consider that there is value to keeping the set-api as
simple as possible.  Right now, anyone who was exposed
to the basics of sets in school can understand the set-api
with a near zero learning curve.  Some of that would be
lost if you add methods that make identity/equality distinctions
or that fetch the same arbitrary value over and over again.

Besides, it is trivial to write a short function that encapsulates
this behavior if it is part of your personal style of expression.



>   The existing methods aren't great for accomplishing this, mainly
> because they're obfuscatory.  "iter(s).next()" is probably clearest,
> and at least will throw a StopIteration exception if the set is empty.
> "for x in s: break" is just confusing ...

A short comment would do wonders.


Raymond


P.S.  It is weird that this thread is gaining traction at the same
time as the moratorium thread.  Go figure :-)

From guido at python.org  Mon Oct 26 20:56:02 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Oct 2009 12:56:02 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net>
	<200910240946.48542.steve@pearwood.info> 
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com> 
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com> 
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com> 
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com> 
	<55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
Message-ID: <ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>

On Mon, Oct 26, 2009 at 12:23 PM, Raymond Hettinger <python at rcn.com> wrote:
> P.S. ?It is weird that this thread is gaining traction at the same
> time as the moratorium thread. ?Go figure :-)

I'm beginning to think maybe adding a method to a built-in object type
*should* fall under the moratorium.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.

From python at rcn.com  Mon Oct 26 20:56:54 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 26 Oct 2009 12:56:54 -0700
Subject: [Python-Dev] Set methods for mapping views
Message-ID: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>

[GvR]
> I still wish we could go back in time and unify sets and dicts, if
> only to find out how that experiment would turn out.

I'm curious about the outcome of another experiment along those lines.
Is anyone seeing uptake for the set methods on mapping views in Py3.x?

I haven't seen any code using it, nor any bug reports or documentation
requests, nor any code in the ASPN cookbook, nor mention of it 
on the newsgroup or python-help.

Has anyone here seen any hints about how this is faring in the wild?


Raymond


From collinw at gmail.com  Mon Oct 26 21:01:34 2009
From: collinw at gmail.com (Collin Winter)
Date: Mon, 26 Oct 2009 13:01:34 -0700
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1256502140.5621.163.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
Message-ID: <43aa6ff70910261301l2961a513m817796019ee1d13f@mail.gmail.com>

On Sun, Oct 25, 2009 at 1:22 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Having other people test it would be fine. Even better if you have an
> actual multi-threaded py3k application. But ccbench results for other
> OSes would be nice too :-)

My results for an 2.4 GHz Intel Core 2 Duo MacBook Pro (OS X 10.5.8):

Control (py3k @ r75723)

--- Throughput ---

Pi calculation (Python)

threads=1: 633 iterations/s.
threads=2: 468 ( 74 %)
threads=3: 443 ( 70 %)
threads=4: 442 ( 69 %)

regular expression (C)

threads=1: 281 iterations/s.
threads=2: 282 ( 100 %)
threads=3: 282 ( 100 %)
threads=4: 282 ( 100 %)

bz2 compression (C)

threads=1: 379 iterations/s.
threads=2: 735 ( 193 %)
threads=3: 733 ( 193 %)
threads=4: 724 ( 190 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 1 ms. (std dev: 1 ms.)
CPU threads=2: 1 ms. (std dev: 2 ms.)
CPU threads=3: 3 ms. (std dev: 6 ms.)
CPU threads=4: 2 ms. (std dev: 3 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 975 ms. (std dev: 577 ms.)
CPU threads=2: 1035 ms. (std dev: 571 ms.)
CPU threads=3: 1098 ms. (std dev: 556 ms.)
CPU threads=4: 1195 ms. (std dev: 557 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 4 ms. (std dev: 5 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 1 ms. (std dev: 4 ms.)



Experiment (newgil branch @ r75723)

--- Throughput ---

Pi calculation (Python)

threads=1: 651 iterations/s.
threads=2: 643 ( 98 %)
threads=3: 637 ( 97 %)
threads=4: 625 ( 95 %)

regular expression (C)

threads=1: 298 iterations/s.
threads=2: 296 ( 99 %)
threads=3: 288 ( 96 %)
threads=4: 287 ( 96 %)

bz2 compression (C)

threads=1: 378 iterations/s.
threads=2: 720 ( 190 %)
threads=3: 724 ( 191 %)
threads=4: 718 ( 189 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 1 ms.)
CPU threads=2: 0 ms. (std dev: 1 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 1 ms. (std dev: 5 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 1 ms. (std dev: 0 ms.)
CPU threads=2: 2 ms. (std dev: 1 ms.)
CPU threads=3: 2 ms. (std dev: 2 ms.)
CPU threads=4: 2 ms. (std dev: 1 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 2 ms. (std dev: 3 ms.)
CPU threads=3: 0 ms. (std dev: 1 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)


I also ran this through Unladen Swallow's threading microbenchmark,
which is a straight copy of what David Beazley was experimenting with
(simply iterating over 1000000 ints in pure Python) [1].
"iterative_count" is doing the loops one after the other,
"threaded_count" is doing the loops in parallel using threads.

The results below are benchmarking py3k as the control, newgil as the
experiment. When it says "x% faster", that is a measure of newgil's
performance over py3k's.

With two threads:

iterative_count:
Min: 0.336573 -> 0.387782: 13.21% slower  # I've run this
configuration multiple times and gotten the same slowdown.
Avg: 0.338473 -> 0.418559: 19.13% slower
Significant (t=-38.434785, a=0.95)

threaded_count:
Min: 0.529859 -> 0.397134: 33.42% faster
Avg: 0.581786 -> 0.429933: 35.32% faster
Significant (t=70.100445, a=0.95)


With four threads:

iterative_count:
Min: 0.766617 -> 0.734354: 4.39% faster
Avg: 0.771954 -> 0.751374: 2.74% faster
Significant (t=22.164103, a=0.95)
Stddev: 0.00262 -> 0.00891: 70.53% larger

threaded_count:
Min: 1.175750 -> 0.829181: 41.80% faster
Avg: 1.224157 -> 0.867506: 41.11% faster
Significant (t=161.715477, a=0.95)
Stddev: 0.01900 -> 0.01120: 69.65% smaller


With eight threads:

iterative_count:
Min: 1.527794 -> 1.447421: 5.55% faster
Avg: 1.536911 -> 1.479940: 3.85% faster
Significant (t=35.559595, a=0.95)
Stddev: 0.00394 -> 0.01553: 74.61% larger

threaded_count:
Min: 2.424553 -> 1.677180: 44.56% faster
Avg: 2.484922 -> 1.723093: 44.21% faster
Significant (t=184.766131, a=0.95)
Stddev: 0.02874 -> 0.02956: 2.78% larger


I'd be interested in multithreaded benchmarks with less-homogenous workloads.

Collin Winter

[1] - http://code.google.com/p/unladen-swallow/source/browse/tests/performance/bm_threading.py

From jnoller at gmail.com  Mon Oct 26 21:06:29 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 26 Oct 2009 16:06:29 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
	<ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
Message-ID: <4222a8490910261306p4d72bb54u403d60ea843c962f@mail.gmail.com>

On Mon, Oct 26, 2009 at 3:56 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Oct 26, 2009 at 12:23 PM, Raymond Hettinger <python at rcn.com> wrote:
>> P.S. ?It is weird that this thread is gaining traction at the same
>> time as the moratorium thread. ?Go figure :-)
>
> I'm beginning to think maybe adding a method to a built-in object type
> *should* fall under the moratorium.

So far, fiddling with the PEP, I'm on the fence - adding a method to a
built-in object type is sort of a grey area (at least in my mind). It
depends on if doing so significantly alters the language/syntax.

jesse

From debatem1 at gmail.com  Mon Oct 26 21:08:45 2009
From: debatem1 at gmail.com (geremy condra)
Date: Mon, 26 Oct 2009 16:08:45 -0400
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
Message-ID: <f3cc57c60910261308j59fefe6duec00ec1a98903776@mail.gmail.com>

On Mon, Oct 26, 2009 at 3:56 PM, Raymond Hettinger <python at rcn.com> wrote:
> [GvR]
>>
>> I still wish we could go back in time and unify sets and dicts, if
>> only to find out how that experiment would turn out.
>
> I'm curious about the outcome of another experiment along those lines.
> Is anyone seeing uptake for the set methods on mapping views in Py3.x?
>
> I haven't seen any code using it, nor any bug reports or documentation
> requests, nor any code in the ASPN cookbook, nor mention of it on the
> newsgroup or python-help.
>
> Has anyone here seen any hints about how this is faring in the wild?
>
>
> Raymond
>

Next version of Graphine will use them heavily for graph
merges and set operations on graphs.

Geremy Condra

From debatem1 at gmail.com  Mon Oct 26 21:14:58 2009
From: debatem1 at gmail.com (geremy condra)
Date: Mon, 26 Oct 2009 16:14:58 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
	<ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
Message-ID: <f3cc57c60910261314n38103d4eu489ce0c96fd58d2b@mail.gmail.com>

On Mon, Oct 26, 2009 at 3:56 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Oct 26, 2009 at 12:23 PM, Raymond Hettinger <python at rcn.com> wrote:
>> P.S. ?It is weird that this thread is gaining traction at the same
>> time as the moratorium thread. ?Go figure :-)
>
> I'm beginning to think maybe adding a method to a built-in object type
> *should* fall under the moratorium.
>
> --
> --Guido van Rossum

Seems like any changes requiring support from uninvolved
developers should fall under the moratorium. If the proposal
is to add a set.get() function to the set API, then this clearly
falls under that heading.

Geremy Condra

From solipsis at pitrou.net  Mon Oct 26 21:19:22 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Oct 2009 20:19:22 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Retrieve_an_arbitrary_element_from_a_set?=
	=?utf-8?q?=09withoutremoving_it?=
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
	<ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
	<4222a8490910261306p4d72bb54u403d60ea843c962f@mail.gmail.com>
Message-ID: <loom.20091026T211711-119@post.gmane.org>

Jesse Noller <jnoller <at> gmail.com> writes:
> 
> So far, fiddling with the PEP, I'm on the fence - adding a method to a
> built-in object type is sort of a grey area (at least in my mind). It
> depends on if doing so significantly alters the language/syntax.

We have recently added things like float.fromhex() which IMHO shouldn't be
blocked by the moratorium (*), although they technically add a method to a 
built-in.

(*) it is a minor new feature aimed at catching up with some established
standard for an exact, non-ambiguous string representation of floats

Regards

Antoine.



From python at rcn.com  Mon Oct 26 21:20:24 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 26 Oct 2009 13:20:24 -0700
Subject: [Python-Dev] Set methods for mapping views
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
	<f3cc57c60910261308j59fefe6duec00ec1a98903776@mail.gmail.com>
Message-ID: <C1401E9BC73E47D081015A4198B97630@RaymondLaptop1>


>> I'm curious about the outcome of another experiment along those lines.
>> Is anyone seeing uptake for the set methods on mapping views in Py3.x?

> Next version of Graphine will use them heavily for graph
> merges and set operations on graphs.

That's great!  Thanks for the data point.
I look forward to reading the code.


Raymond

From dickinsm at gmail.com  Mon Oct 26 21:26:51 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 26 Oct 2009 20:26:51 +0000
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
	<ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com>
Message-ID: <5c6f2a5d0910261326x518a8892w86f9b76ee33b9f9d@mail.gmail.com>

On Mon, Oct 26, 2009 at 7:56 PM, Guido van Rossum <guido at python.org> wrote:
> I'm beginning to think maybe adding a method to a built-in object type
> *should* fall under the moratorium.

Another possible test case for this decision is the int.from_bytes and
int.to_bytes methods, proposed a while ago, and implemented
by Alexandre Vassalotti.  See:

http://bugs.python.org/issue1023290

Mark

From guido at python.org  Mon Oct 26 21:32:32 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Oct 2009 13:32:32 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <loom.20091026T211711-119@post.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE3E6C6.4010008@gmail.com> 
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com> 
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com> 
	<55A46E12112349EFAE8C2C8E7E6532B6@RaymondLaptop1>
	<ca471dc20910261256j21146f93p378497545099a2c@mail.gmail.com> 
	<4222a8490910261306p4d72bb54u403d60ea843c962f@mail.gmail.com> 
	<loom.20091026T211711-119@post.gmane.org>
Message-ID: <ca471dc20910261332r2132fb12wf32f5a27103a55b1@mail.gmail.com>

On Mon, Oct 26, 2009 at 1:19 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Jesse Noller <jnoller <at> gmail.com> writes:
>>
>> So far, fiddling with the PEP, I'm on the fence - adding a method to a
>> built-in object type is sort of a grey area (at least in my mind). It
>> depends on if doing so significantly alters the language/syntax.
>
> We have recently added things like float.fromhex() which IMHO shouldn't be
> blocked by the moratorium (*), although they technically add a method to a
> built-in.
>
> (*) it is a minor new feature aimed at catching up with some established
> standard for an exact, non-ambiguous string representation of floats

Okay, so it remains a gray area.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.

From w.richert at gmx.net  Mon Oct 26 22:14:11 2009
From: w.richert at gmx.net (Willi Richert)
Date: Mon, 26 Oct 2009 22:14:11 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <ca471dc20910261332r2132fb12wf32f5a27103a55b1@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<loom.20091026T211711-119@post.gmane.org>
	<ca471dc20910261332r2132fb12wf32f5a27103a55b1@mail.gmail.com>
Message-ID: <200910262214.11662.w.richert@gmx.net>

For those of you who want to tinker with it, I posted the patch against the 
current trunk at http://bugs.python.org/issue7212

Have fun,
wr

Am Montag, 26. Oktober 2009 21:32:32 schrieb Guido van Rossum:
> On Mon, Oct 26, 2009 at 1:19 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Jesse Noller <jnoller <at> gmail.com> writes:
> >> So far, fiddling with the PEP, I'm on the fence - adding a method to a
> >> built-in object type is sort of a grey area (at least in my mind). It
> >> depends on if doing so significantly alters the language/syntax.
> >
> > We have recently added things like float.fromhex() which IMHO shouldn't
> > be blocked by the moratorium (*), although they technically add a method
> > to a built-in.
> >
> > (*) it is a minor new feature aimed at catching up with some established
> > standard for an exact, non-ambiguous string representation of floats
> 
> Okay, so it remains a gray area.
> 

From solipsis at pitrou.net  Mon Oct 26 22:43:50 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Oct 2009 21:43:50 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
	<43aa6ff70910261301l2961a513m817796019ee1d13f@mail.gmail.com>
Message-ID: <loom.20091026T223740-432@post.gmane.org>

Collin Winter <collinw <at> gmail.com> writes:
> 
> My results for an 2.4 GHz Intel Core 2 Duo MacBook Pro (OS X 10.5.8):

Thanks!


[the Dave Beazley benchmark]
> The results below are benchmarking py3k as the control, newgil as the
> experiment. When it says "x% faster", that is a measure of newgil's
> performance over py3k's.
> 
> With two threads:
> 
> iterative_count:
> Min: 0.336573 -> 0.387782: 13.21% slower  # I've run this
> configuration multiple times and gotten the same slowdown.
> Avg: 0.338473 -> 0.418559: 19.13% slower

Those numbers are not very in line with the other "iterative_count" results.
Since iterative_count just runs the loop N times in a row, results should be
proportional to the number N ("number of threads").

Besides, there's no reason for single-threaded performance to be degraded since
the fast path of the eval loop actually got a bit streamlined (there is no
volatile ticker to decrement).

> I'd be interested in multithreaded benchmarks with less-homogenous workloads.

So would I.

Regards

Antoine.



From collinw at gmail.com  Mon Oct 26 22:50:14 2009
From: collinw at gmail.com (Collin Winter)
Date: Mon, 26 Oct 2009 14:50:14 -0700
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091026T223740-432@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<43aa6ff70910261301l2961a513m817796019ee1d13f@mail.gmail.com>
	<loom.20091026T223740-432@post.gmane.org>
Message-ID: <43aa6ff70910261450m23e66c4er654e7e425ddda31b@mail.gmail.com>

On Mon, Oct 26, 2009 at 2:43 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Collin Winter <collinw <at> gmail.com> writes:
> [the Dave Beazley benchmark]
>> The results below are benchmarking py3k as the control, newgil as the
>> experiment. When it says "x% faster", that is a measure of newgil's
>> performance over py3k's.
>>
>> With two threads:
>>
>> iterative_count:
>> Min: 0.336573 -> 0.387782: 13.21% slower ?# I've run this
>> configuration multiple times and gotten the same slowdown.
>> Avg: 0.338473 -> 0.418559: 19.13% slower
>
> Those numbers are not very in line with the other "iterative_count" results.
> Since iterative_count just runs the loop N times in a row, results should be
> proportional to the number N ("number of threads").
>
> Besides, there's no reason for single-threaded performance to be degraded since
> the fast path of the eval loop actually got a bit streamlined (there is no
> volatile ticker to decrement).

I agree those numbers are out of line with the others and make no
sense. I've run it with two threads several times and the results are
consistent on this machine. I'm digging into it a bit more.

Collin

From ncoghlan at gmail.com  Mon Oct 26 22:50:56 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 27 Oct 2009 07:50:56 +1000
Subject: [Python-Dev] One obvious way to do interning [Was: Retrieve an
 arbitrary element from a set without removing it]
In-Reply-To: <hc4sqm$cek$1@ger.gmane.org>
References: <d38f5330910260943r74d67d42l8791b6cf17e90b23@mail.gmail.com>
	<hc4sqm$cek$1@ger.gmane.org>
Message-ID: <4AE619C0.803@gmail.com>

Terry Reedy wrote:
> Alexander Belopolsky wrote:
>> Terry Reedy wrote:
>>> I had exactly the same idea, but did not post because it violates the
>>> general rule that mutators return None.
>>
>> Is there such a rule?  What about set/dict pop?
> 
> The rule perhaps should be restated as 'Collection mutators return None
> or possible an item but not the collection.'

And to clarify the rationale for that guideline: it is to make it clear
that the mutator is changing the container in place and *not* creating a
new container object.

myset.pop()     # No new container, returns popped object
mylist.sort()   # No new container, returns None
sorted(mylist)  # New container, so return it
mystr.lower()   # Creates new string, so return it

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Oct 26 22:54:52 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 27 Oct 2009 07:54:52 +1000
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
Message-ID: <4AE61AAC.6020806@gmail.com>

Raymond Hettinger wrote:
> [GvR]
>> I still wish we could go back in time and unify sets and dicts, if
>> only to find out how that experiment would turn out.
> 
> I'm curious about the outcome of another experiment along those lines.
> Is anyone seeing uptake for the set methods on mapping views in Py3.x?
> 
> I haven't seen any code using it, nor any bug reports or documentation
> requests, nor any code in the ASPN cookbook, nor mention of it on the
> newsgroup or python-help.
> 
> Has anyone here seen any hints about how this is faring in the wild?

If anyone is looking for further explanation as to why Guido's
moratorium on core language changes is a good idea, allowing a chance
for answers to questions like Raymond's above a chance to evolve
naturally is what I see as the most important rationale.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From exarkun at twistedmatrix.com  Mon Oct 26 23:45:25 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Mon, 26 Oct 2009 22:45:25 -0000
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <eae285400910260918t7596115bg77c48265f7d62790@mail.gmail.com>
References: <1256502140.5621.163.camel@localhost> <4AE5C45A.9090900@molden.no>
	<loom.20091026T165251-219@post.gmane.org>
	<eae285400910260918t7596115bg77c48265f7d62790@mail.gmail.com>
Message-ID: <20091026224525.31526.1835917647.divmod.xquotient.110@localhost.localdomain>

On 04:18 pm, daniel at stutzbachenterprises.com wrote:
>On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou 
><solipsis at pitrou.net>wrote:
>>Er, I prefer to keep things simple. If you have lots of I/O you should
>>probably
>>use an event loop rather than separate threads.
>
>On Windows, sometimes using a single-threaded event loop is sometimes
>impossible.  WaitForMultipleObjects(), which is the Windows equivalent 
>to
>select() or poll(), can handle a maximum of only 64 objects.

This is only partially accurate.  For one thing, WaitForMultipleObjects 
calls are nestable.  For another thing, Windows also has I/O completion 
ports which are not limited to 64 event sources.  The situation is 
actually better than on a lot of POSIXes.
>Do we really need priority requests at all?  They seem counter to your
>desire for simplicity and allowing the operating system's scheduler to 
>do
>its work.

Despite what I said above, however, I would also take a default position 
against adding any kind of more advanced scheduling system here.  It 
would, perhaps, make sense to expose the APIs for controlling the 
platform scheduler, though.

Jean-Paul

From barry at python.org  Mon Oct 26 23:48:46 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 26 Oct 2009 18:48:46 -0400
Subject: [Python-Dev] RELEASED Python 2.6.4
Message-ID: <F28950ED-BA85-4275-948D-416D6820E184@python.org>

On behalf of the Python community, I'm happy to announce the  
availability of Python 2.6.4.  This is the latest production-ready  
version in the Python 2.6 series.

We had a little trouble with the Python 2.6.3 release; a number of  
unfortunate regressions were introduced.  I take responsibility for  
rushing it out, but the good news is that Python 2.6.4 fixes the known  
regressions in 2.6.3.  We've had a lengthy release candidate cycle  
this time, and are confident that 2.6.4 is a solid release.  We highly  
recommend you upgrade to Python 2.6.4.

Please see the NEWS file for all the gory details.

     http://www.python.org/download/releases/2.6.4/NEWS.txt

Source tarballs and the Windows installers can be downloaded from the  
Python 2.6.4 page.  The Mac OS X disk image will be uploaded soon.

    http://www.python.org/download/releases/2.6.4/

For more information on Python 2.6 in general, please see

    http://docs.python.org/whatsnew/2.6.html

Please report bugs for any Python version in the Python tracker.

    http://bugs.python.org

Enjoy,
-Barry

Barry Warsaw
barry at python.org
Python 2.6 Release Manager
(on behalf of the entire python-dev team)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091026/edf7605a/attachment-0001.pgp>

From steve at pearwood.info  Mon Oct 26 23:59:43 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 27 Oct 2009 09:59:43 +1100
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <4AE61AAC.6020806@gmail.com>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
	<4AE61AAC.6020806@gmail.com>
Message-ID: <200910270959.44539.steve@pearwood.info>

On Tue, 27 Oct 2009 08:54:52 am Nick Coghlan wrote:
> Raymond Hettinger wrote:
> > [GvR]
> >
> >> I still wish we could go back in time and unify sets and dicts, if
> >> only to find out how that experiment would turn out.
> >
> > I'm curious about the outcome of another experiment along those
> > lines. Is anyone seeing uptake for the set methods on mapping views
> > in Py3.x?
> >
> > I haven't seen any code using it, nor any bug reports or
> > documentation requests, nor any code in the ASPN cookbook, nor
> > mention of it on the newsgroup or python-help.
> >
> > Has anyone here seen any hints about how this is faring in the
> > wild?
>
> If anyone is looking for further explanation as to why Guido's
> moratorium on core language changes is a good idea, allowing a chance
> for answers to questions like Raymond's above a chance to evolve
> naturally is what I see as the most important rationale.

I don't understand that rationale. Let's take a concrete example. The 
new `yield from` syntax was accepted but now will be delayed by the 
moratorium. How would the addition of `yield from` delay or prevent 
people using set methods on mapping views?

If a proposed feature directly competes with a feature in 3.x, then it 
might delay usage of the 3.x feature -- but if that were the case, the 
proposal would almost certainly be rejected on the basis that 3.x 
already has a feature to do that very thing.


-- 
Steven D'Aprano

From solipsis at pitrou.net  Tue Oct 27 00:06:15 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Oct 2009 23:06:15 +0000 (UTC)
Subject: [Python-Dev] Set methods for mapping views
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
	<4AE61AAC.6020806@gmail.com>
	<200910270959.44539.steve@pearwood.info>
Message-ID: <loom.20091027T000556-818@post.gmane.org>

Steven D'Aprano <steve <at> pearwood.info> writes:
> 
> I don't understand that rationale. Let's take a concrete example. The 
> new `yield from` syntax was accepted

Was it?



From guido at python.org  Tue Oct 27 00:09:14 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Oct 2009 16:09:14 -0700
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <loom.20091027T000556-818@post.gmane.org>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1> 
	<4AE61AAC.6020806@gmail.com> <200910270959.44539.steve@pearwood.info> 
	<loom.20091027T000556-818@post.gmane.org>
Message-ID: <ca471dc20910261609p2e5c5134l4ab1a53d5e3d13e8@mail.gmail.com>

On Mon, Oct 26, 2009 at 4:06 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Steven D'Aprano <steve <at> pearwood.info> writes:
>>
>> I don't understand that rationale. Let's take a concrete example. The
>> new `yield from` syntax was accepted
>
> Was it?

No.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.

From steve at pearwood.info  Tue Oct 27 00:29:58 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 27 Oct 2009 10:29:58 +1100
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <ca471dc20910261609p2e5c5134l4ab1a53d5e3d13e8@mail.gmail.com>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>
	<loom.20091027T000556-818@post.gmane.org>
	<ca471dc20910261609p2e5c5134l4ab1a53d5e3d13e8@mail.gmail.com>
Message-ID: <200910271029.58591.steve@pearwood.info>

On Tue, 27 Oct 2009 10:09:14 am Guido van Rossum wrote:
> On Mon, Oct 26, 2009 at 4:06 PM, Antoine Pitrou <solipsis at pitrou.net> 
wrote:
> > Steven D'Aprano <steve <at> pearwood.info> writes:
> >> I don't understand that rationale. Let's take a concrete example.
> >> The new `yield from` syntax was accepted
> >
> > Was it?
>
> No.

I thought it had been. My mistake. Serves me right for not checking the 
PEP first. But my point still stands.



-- 
Steven D'Aprano

From ssteinerx at gmail.com  Tue Oct 27 01:02:03 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Mon, 26 Oct 2009 20:02:03 -0400
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <20091026224525.31526.1835917647.divmod.xquotient.110@localhost.localdomain>
References: <1256502140.5621.163.camel@localhost> <4AE5C45A.9090900@molden.no>
	<loom.20091026T165251-219@post.gmane.org>
	<eae285400910260918t7596115bg77c48265f7d62790@mail.gmail.com>
	<20091026224525.31526.1835917647.divmod.xquotient.110@localhost.localdomain>
Message-ID: <C540E67D-3107-4608-BA42-84290A10F6DF@gmail.com>


On Oct 26, 2009, at 6:45 PM, exarkun at twistedmatrix.com wrote:
>> Despite what I said above, however, I would also take a default  
>> position against adding any kind of more advanced scheduling system  
>> here.  It would, perhaps, make sense to expose the APIs for  
>> controlling the platform scheduler, though.

I would also like to have an exposed API and optional profiling  
(optional as in conditional compilation, not as in some sort of  
profiling 'flag' that slows down non-profiling runs) so that you can  
see what's going on well enough to use the API to tune a particular  
platform for a particular workload.

S



From cs at zip.com.au  Tue Oct 27 02:09:53 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 27 Oct 2009 12:09:53 +1100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <20091026224525.31526.1835917647.divmod.xquotient.110@localhost.localdomain>
References: <20091026224525.31526.1835917647.divmod.xquotient.110@localhost.localdomain>
Message-ID: <20091027010953.GA19484@cskk.homeip.net>

On 26Oct2009 22:45, exarkun at twistedmatrix.com <exarkun at twistedmatrix.com> wrote:
| On 04:18 pm, daniel at stutzbachenterprises.com wrote:
| >On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou write:
| >Do we really need priority requests at all?  They seem counter to your
| >desire for simplicity and allowing the operating system's
| >scheduler to do
| >its work.
| 
| Despite what I said above, however, I would also take a default
| position against adding any kind of more advanced scheduling system
| here.  It would, perhaps, make sense to expose the APIs for
| controlling the platform scheduler, though.

+1 to both sentences from me.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Plague, Famine, Pestilence, and C++ stalk the land. We're doomed! Doomed!
        - Simon E Spero

From brett at python.org  Tue Oct 27 04:13:28 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 26 Oct 2009 20:13:28 -0700
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <200910270959.44539.steve@pearwood.info>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1> 
	<4AE61AAC.6020806@gmail.com> <200910270959.44539.steve@pearwood.info>
Message-ID: <bbaeab100910262013y19fbb5e3h7b53b9fc34726a87@mail.gmail.com>

On Mon, Oct 26, 2009 at 15:59, Steven D'Aprano <steve at pearwood.info> wrote:

> On Tue, 27 Oct 2009 08:54:52 am Nick Coghlan wrote:
> > Raymond Hettinger wrote:
> > > [GvR]
> > >
> > >> I still wish we could go back in time and unify sets and dicts, if
> > >> only to find out how that experiment would turn out.
> > >
> > > I'm curious about the outcome of another experiment along those
> > > lines. Is anyone seeing uptake for the set methods on mapping views
> > > in Py3.x?
> > >
> > > I haven't seen any code using it, nor any bug reports or
> > > documentation requests, nor any code in the ASPN cookbook, nor
> > > mention of it on the newsgroup or python-help.
> > >
> > > Has anyone here seen any hints about how this is faring in the
> > > wild?
> >
> > If anyone is looking for further explanation as to why Guido's
> > moratorium on core language changes is a good idea, allowing a chance
> > for answers to questions like Raymond's above a chance to evolve
> > naturally is what I see as the most important rationale.
>
> I don't understand that rationale. Let's take a concrete example. The
> new `yield from` syntax was accepted but now will be delayed by the
> moratorium. How would the addition of `yield from` delay or prevent
> people using set methods on mapping views?
>
>
It doesn't, but the point is we have already added several things to the
language in Python 3 that have gone mostly unused from the community thus
far. We do not need to continue to pile on the new features when we already
have a stack that we need to see if they pan out.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091026/6a9e281f/attachment-0001.htm>

From solipsis at pitrou.net  Tue Oct 27 11:58:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 27 Oct 2009 10:58:47 +0000 (UTC)
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>
	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<4AE4A1C9.5010002@v.loewis.de>	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>	<20091026005815.31526.130687717.divmod.xquotient.88@localhost.localdomain>	<m263a3aqpq.fsf@valheru.db3l.homeip.net>
	<20091026015650.31526.600593069.divmod.xquotient.94@localhost.localdomain>
	<4AE55E39.20000@v.loewis.de>
Message-ID: <loom.20091027T115542-139@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> It's not really reproducible. I think it sometimes happens when I
> restart the master; sometimes, some clients fail to reconnect
> (properly).

Another common problem is that some buildbot fails in the middle of the test
suite, with the following kind of message:

command timed out: 1800 seconds without output, killing pid 12325
process killed by signal 9
program finished with exit code -1
elapsedTime=10910.362981

See for example :
http://www.python.org/dev/buildbot/trunk.stable/builders/ia64%20Ubuntu%20trunk/builds/73/steps/test/logs/stdio

(notice, by the way, the elapsed time (10910s, that is, close to 3 hours...))

Regards

Antoine.



From zookog at gmail.com  Tue Oct 27 16:41:42 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Tue, 27 Oct 2009 09:41:42 -0600
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <4AE4A5DC.2010906@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<4AE4A5DC.2010906@v.loewis.de>
Message-ID: <cd6401a0910270841i6336288dh5fd196d5cb0809b2@mail.gmail.com>

Right, how do developers benefit from a buildbot?

>From my experience (five large buildbots with many developers plus two
with only a couple of developers), a buildbot does little good unless
the tests are reliable and not too noisy.  "Reliable" is best achieved
by having tests be deterministic and reproducible.  "Not too noisy"
means that the builders are all green all the time (at least for a
"supported" subset of the buildslaves).

Beyond that, then I think there has to be a culture change where the
development team decides that it is really, really not okay to leave a
builder red after you turned it red, and that instead you need to
revert the patch that made it go from green to red before you do
anything else.  It has taken me a long time to acculturate to that and
I wouldn't expect most people to do it quickly or easily.

(It is interesting to think of what would happen if that policy were
automated -- any patch which caused any "supported" builder to go from
green to red would be automatically be reverted.)

Also, of course, this is mostly meaningless unless the code that is
being changed by the patches is well-covered by tests.

Regards,

Zooko

From g.brandl at gmx.net  Tue Oct 27 17:06:44 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 27 Oct 2009 17:06:44 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org>
	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
Message-ID: <hc75qn$a0k$1@ger.gmane.org>

Chris Bergstresser schrieb:

>    I like the proposed set.get() method, personally.  list.get(index)
> gets the item at that index, dict.get(key) gets the item associated
> with that key, set.get() gets an item, but doesn't place any
> guarantees on which item is returned.

Sorry to nitpick, but there is no list.get().

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From chris at subtlety.com  Tue Oct 27 18:33:29 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Tue, 27 Oct 2009 12:33:29 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <hc75qn$a0k$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>
	<200910240946.48542.steve@pearwood.info> 
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com> 
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com> 
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com> 
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com> 
	<hc75qn$a0k$1@ger.gmane.org>
Message-ID: <7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>

On Tue, Oct 27, 2009 at 11:06 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Sorry to nitpick, but there is no list.get().

   No?  How ... odd.  I guess it wouldn't have come up, but I was sure
there was a .get method which took an optional default parameter if
the index didn't exist, mirroring the dict method.  Still, I think my
point stands--it's a clear extrapolation from the existing dict.get().

-- Chris

From g.brandl at gmx.net  Tue Oct 27 18:40:45 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 27 Oct 2009 18:40:45 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
Message-ID: <hc7bb2$35u$1@ger.gmane.org>

Chris Bergstresser schrieb:
> On Tue, Oct 27, 2009 at 11:06 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Sorry to nitpick, but there is no list.get().
> 
>    No?  How ... odd.  I guess it wouldn't have come up, but I was sure
> there was a .get method which took an optional default parameter if
> the index didn't exist, mirroring the dict method.  Still, I think my
> point stands--it's a clear extrapolation from the existing dict.get().

I don't see that.  Both dict.get() and your hypothetical list.get() are
variants of the [] subscription operator, i.e. __getitem__, that have a
default value, defaulting to None.  The [] operator retrieves an element
from the object at the specified "position", be it dict key, list index
or some other abstract position.

Contrary to that, sets don't support subscript access, there is no notion
of a value at a specified position, so I would find the set.get() naming
confusing.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From python at rcn.com  Tue Oct 27 18:47:59 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 27 Oct 2009 10:47:59 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
References: <200910231132.45504.w.richert@gmx.net><200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
Message-ID: <B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>


[Chris Bergstresser]
  Still, I think my
> point stands--it's a clear extrapolation from the existing dict.get().

Not really.  One looks-up a key and supplies a default value if not found.
The other, set.get(), doesn't have a key to lookup.

A dict.get() can be meaningfully used in a loop (because the key can vary).
A set.get() returns the same value over and over again (because there is no key).


Raymond

From alexander.belopolsky at gmail.com  Tue Oct 27 18:50:56 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 27 Oct 2009 13:50:56 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
Message-ID: <d38f5330910271050y5f9e53a7sbca23b3058e1bf61@mail.gmail.com>

On Tue, Oct 27, 2009 at 1:33 PM, Chris Bergstresser <chris at subtlety.com> wrote:
> On Tue, Oct 27, 2009 at 11:06 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Sorry to nitpick, but there is no list.get().
>
> ? No? ?How ... odd.

Odd indeed.  My first reaction was: it is not needed because lists
support slicing, but when I tried to construct a list.get() using
slicing the best I could come up with was the following hack

>>> def lget(l, i, v):  return (l[i:] or [v])[0]
...
>>> lget(range(10), 20, 200)
200
>>> lget(range(10), 5, 50)
5

Yet for some reason I never missed this functionality ...

From solipsis at pitrou.net  Tue Oct 27 18:59:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 27 Oct 2009 17:59:00 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Retrieve_an_arbitrary_element_from_a_set?=
	=?utf-8?q?=09withoutremoving_it?=
References: <200910231132.45504.w.richert@gmx.net><200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
Message-ID: <loom.20091027T185838-675@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> [Chris Bergstresser]
>   Still, I think my
> > point stands--it's a clear extrapolation from the existing dict.get().
> 
> Not really.  One looks-up a key and supplies a default value if not found.
> The other, set.get(), doesn't have a key to lookup.

set.getone() then ?




From debatem1 at gmail.com  Tue Oct 27 19:20:04 2009
From: debatem1 at gmail.com (geremy condra)
Date: Tue, 27 Oct 2009 14:20:04 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <loom.20091027T185838-675@post.gmane.org>
References: <200910231132.45504.w.richert@gmx.net> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<loom.20091027T185838-675@post.gmane.org>
Message-ID: <f3cc57c60910271120j68f594a3vb505046f0f4ca875@mail.gmail.com>

On Tue, Oct 27, 2009 at 1:59 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Raymond Hettinger <python <at> rcn.com> writes:
>>
>> [Chris Bergstresser]
>> ? Still, I think my
>> > point stands--it's a clear extrapolation from the existing dict.get().
>>
>> Not really. ?One looks-up a key and supplies a default value if not found.
>> The other, set.get(), doesn't have a key to lookup.
>
> set.getone() then ?

ugh- other spellings much preferred.

Geremy Condra

From phd at phd.pp.ru  Tue Oct 27 19:30:30 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Tue, 27 Oct 2009 21:30:30 +0300
Subject: [Python-Dev] Retrieve an arbitrary element from a
	set	withoutremoving it
In-Reply-To: <f3cc57c60910271120j68f594a3vb505046f0f4ca875@mail.gmail.com>
References: <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<loom.20091027T185838-675@post.gmane.org>
	<f3cc57c60910271120j68f594a3vb505046f0f4ca875@mail.gmail.com>
Message-ID: <20091027183030.GA29335@phd.pp.ru>

On Tue, Oct 27, 2009 at 02:20:04PM -0400, geremy condra wrote:
> On Tue, Oct 27, 2009 at 1:59 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Raymond Hettinger <python <at> rcn.com> writes:
> > set.getone() then ?
> 
> ugh- other spellings much preferred.

   set[] ? (Just kidding, really.)

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From tjreedy at udel.edu  Tue Oct 27 19:50:28 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 27 Oct 2009 14:50:28 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net><200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>
	<4AE3E6C6.4010008@gmail.com>	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>	<hc75qn$a0k$1@ger.gmane.org>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
Message-ID: <hc7fdl$ia8$1@ger.gmane.org>

Raymond Hettinger wrote:
>
> A dict.get() can be meaningfully used in a loop (because the key can vary).
> A set.get() returns the same value over and over again (because there is 
> no key).

There are two ideas of set.get floating about:
1) get an arbitrary object
2) get the object in the set with the same 'value'(hash+eq) as an input 
arg (the intern case). In this case, there is a 'key', even if it is 
somewhat abstract rather that being an object.

Both could be done with the same method, depending on whether an arg is 
passed or not. The former is not useful in a loop; the latter could be.

If there is no match in case 2, the method could a) raise an exception, 
b) return None (which by its nature would never sensibly be looked up), 
or c) return an object specified by 'default=xxx' keyword arg.

Terry Jan Reedy


From chris at subtlety.com  Tue Oct 27 20:02:18 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Tue, 27 Oct 2009 14:02:18 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com> 
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com> 
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com> 
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
Message-ID: <7b0045ed0910271202h4e8f762cqdd48373c52af8f0a@mail.gmail.com>

On Tue, Oct 27, 2009 at 12:47 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Chris Bergstresser]
> ?Still, I think my
>>
>> point stands--it's a clear extrapolation from the existing dict.get().
>
> Not really. ?One looks-up a key and supplies a default value if not found.
> The other, set.get(), doesn't have a key to lookup.

    Right, which is why dict.get() takes a key as an argument, while
the proposed set.get() wouldn't.

> A dict.get() can be meaningfully used in a loop (because the key can vary).
> A set.get() returns the same value over and over again (because there is no
> key).

    I don't think "can it be used meaningfully in a loop?" is an
especially interesting or useful way of evaluating language features.
    Besides, why would set.get() necessarily return the same value
over and over again?  I thought it would be defined to return an
arbitrary value--which could be the same value over and over again,
but could just as easily be defined to return a round-robin value, or
the minimum value, or some *other* value as the container defined it.
   The fact is, set.get() succinctly describes an action which is
otherwise obscure in Python.  It doesn't come up all that frequently,
but when it does the alternatives are poor at best.  Add in the
uncertainty about which is optimized (imagine the situation where the
set you're calling is actually a proxy for an object across the
network, and constructing an iterator is expensive) and you start to
see the value.

-- Chris

From ssteinerx at gmail.com  Tue Oct 27 20:06:14 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 27 Oct 2009 15:06:14 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <hc7fdl$ia8$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net><200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>
	<4AE3E6C6.4010008@gmail.com>	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>	<hc75qn$a0k$1@ger.gmane.org>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<hc7fdl$ia8$1@ger.gmane.org>
Message-ID: <CC725036-196D-468A-BDA4-9BD1217F3669@gmail.com>


On Oct 27, 2009, at 2:50 PM, Terry Reedy wrote more and more and more  
and more and more and more and more and more and more:

This topic needs its own flippin' newsgroup.

S


From tseaver at palladion.com  Tue Oct 27 20:10:25 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 27 Oct 2009 15:10:25 -0400
Subject: [Python-Dev] RELEASED Python 2.6.4
In-Reply-To: <F28950ED-BA85-4275-948D-416D6820E184@python.org>
References: <F28950ED-BA85-4275-948D-416D6820E184@python.org>
Message-ID: <hc7gj1$l5l$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
> On behalf of the Python community, I'm happy to announce the  
> availability of Python 2.6.4.  This is the latest production-ready  
> version in the Python 2.6 series.
> 
> We had a little trouble with the Python 2.6.3 release; a number of  
> unfortunate regressions were introduced.  I take responsibility for  
> rushing it out, but the good news is that Python 2.6.4 fixes the known  
> regressions in 2.6.3.  We've had a lengthy release candidate cycle  
> this time, and are confident that 2.6.4 is a solid release.  We highly  
> recommend you upgrade to Python 2.6.4.
> 
> Please see the NEWS file for all the gory details.
> 
>      http://www.python.org/download/releases/2.6.4/NEWS.txt
> 
> Source tarballs and the Windows installers can be downloaded from the  
> Python 2.6.4 page.  The Mac OS X disk image will be uploaded soon.
> 
>     http://www.python.org/download/releases/2.6.4/
> 
> For more information on Python 2.6 in general, please see
> 
>     http://docs.python.org/whatsnew/2.6.html
> 
> Please report bugs for any Python version in the Python tracker.
> 
>     http://bugs.python.org

Congratulations, and thaks for working to correct the issues with the
earlier release so quickly.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrnRaEACgkQ+gerLs4ltQ5yHACgt9CqCyIE8HfkKCNjSEAU3MB1
A70AoL/yJB9lcI+RrBe3/BlqChCweXvr
=K/lR
-----END PGP SIGNATURE-----


From jnoller at gmail.com  Tue Oct 27 20:13:23 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 27 Oct 2009 15:13:23 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <CC725036-196D-468A-BDA4-9BD1217F3669@gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<hc7fdl$ia8$1@ger.gmane.org>
	<CC725036-196D-468A-BDA4-9BD1217F3669@gmail.com>
Message-ID: <4222a8490910271213w7966f501l5456798ff9858bc2@mail.gmail.com>

On Tue, Oct 27, 2009 at 3:06 PM, ssteinerX at gmail.com
<ssteinerx at gmail.com> wrote:
>
> On Oct 27, 2009, at 2:50 PM, Terry Reedy wrote more and more and more and
> more and more and more and more and more and more:
>
> This topic needs its own flippin' newsgroup.
>
> S

Don't like it? Mute the conversation (thank you gmail) or unsub.

From guido at python.org  Tue Oct 27 20:13:13 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 27 Oct 2009 12:13:13 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <hc7fdl$ia8$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>
	<4AE3E6C6.4010008@gmail.com> 
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com> 
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com> 
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com> 
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com> 
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<hc7fdl$ia8$1@ger.gmane.org>
Message-ID: <ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>

On Tue, Oct 27, 2009 at 11:50 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> There are two ideas of set.get floating about:
> 1) get an arbitrary object
> 2) get the object in the set with the same 'value'(hash+eq) as an input arg
> (the intern case). In this case, there is a 'key', even if it is somewhat
> abstract rather that being an object.
>
> Both could be done with the same method, depending on whether an arg is
> passed or not.

My gut tells me it is bad API design to collapse these two use cases.
Probably because the implementations won't have much in common: (1)
should just pick the first valid element, while (2) should use the
normal hash lookup algorithm (shared with 'in', .add() etc.).

-- 
--Guido van Rossum (python.org/~guido)

From debatem1 at gmail.com  Tue Oct 27 20:27:50 2009
From: debatem1 at gmail.com (geremy condra)
Date: Tue, 27 Oct 2009 15:27:50 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<hc7fdl$ia8$1@ger.gmane.org>
	<ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>
Message-ID: <f3cc57c60910271227p2f4edc0xcdf6213b36a5cac2@mail.gmail.com>

On Tue, Oct 27, 2009 at 3:13 PM, Guido van Rossum <guido at python.org> wrote:
> On Tue, Oct 27, 2009 at 11:50 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> There are two ideas of set.get floating about:
>> 1) get an arbitrary object
>> 2) get the object in the set with the same 'value'(hash+eq) as an input arg
>> (the intern case). In this case, there is a 'key', even if it is somewhat
>> abstract rather that being an object.
>>
>> Both could be done with the same method, depending on whether an arg is
>> passed or not.
>
> My gut tells me it is bad API design to collapse these two use cases.
> Probably because the implementations won't have much in common: (1)
> should just pick the first valid element, while (2) should use the
> normal hash lookup algorithm (shared with 'in', .add() etc.).
>
> --
> --Guido van Rossum (python.org/~guido)


Was it ever decided whether this would fall under the moratorium?

Geremy Condra

From python at rcn.com  Tue Oct 27 20:49:01 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 27 Oct 2009 12:49:01 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
References: <200910231132.45504.w.richert@gmx.net><d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com><hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com><7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com><hc75qn$a0k$1@ger.gmane.org><7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com><B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1><hc7fdl$ia8$1@ger.gmane.org><ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>
	<f3cc57c60910271227p2f4edc0xcdf6213b36a5cac2@mail.gmail.com>
Message-ID: <9D7624CE452343719A7ED512B2FD17F5@RaymondLaptop1>


[geremy condra]
> Was it ever decided whether this would fall under the moratorium?

Decided isn't the right word:
http://mail.python.org/pipermail/python-dev/2009-October/093373.html

FWIW, I'm a strong -1 on both proposals.

Just add a short get_one() function and a get_equivalent() recipe
to your utils directory.  That will get the job done (thought I don't
expect that you will *ever* make much use of either one).
 No need to complexify a type that is currently very simple. 


Raymond



P.S.  get_equivalent:  http://code.activestate.com/recipes/499299/
         get_one = lambda s, default=None:  next(iter(s), default)

The first works with all type that defines __contains__.
The second works for any iterable.
Neither of these concepts are specific to set objects.

From ziade.tarek at gmail.com  Tue Oct 27 22:55:53 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 27 Oct 2009 22:55:53 +0100
Subject: [Python-Dev] Refactoring installation schemes
Message-ID: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>

Hello,

Since the addition of PEP 370, (per-user site packages), site.py and
distutils/command/install.py are *both* providing the various
installation directories for Python,
depending on the system and the Python version.

We have also started to discuss lately in various Mailing Lists the
addition of new schemes for IronPython and Jython, meaning that we
might add some more in both places.

I would like to suggest a simplification by adding a dedicated module
to manage these installation schemes in one single place in the
stdlib.

This new independant module would be used by site.py and distutils and
would also make it easier for third party code to work with these
schemes.
Of course this new module would be rather simple and not add any new
import statement to avoid any overhead when Python starts and loads
site.py

Regards
Tarek

From tjreedy at udel.edu  Wed Oct 28 04:02:06 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 27 Oct 2009 23:02:06 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <CC725036-196D-468A-BDA4-9BD1217F3669@gmail.com>
References: <200910231132.45504.w.richert@gmx.net><200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>	<4AE3E6C6.4010008@gmail.com>	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>	<hc75qn$a0k$1@ger.gmane.org>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>	<hc7fdl$ia8$1@ger.gmane.org>
	<CC725036-196D-468A-BDA4-9BD1217F3669@gmail.com>
Message-ID: <hc8c7e$2d8$1@ger.gmane.org>

ssteinerX at gmail.com wrote:
> 
> On Oct 27, 2009, at 2:50 PM, Terry Reedy wrote more and more and more 
> and more and more and more and more and more and more:

Actually, I wrote 7 succinct lines that summarized and made a proposal.
In general, I snip when quoting and write concisely as possible.

> This topic needs its own flippin' newsgroup.

You need to learn politeness. You could have said just that, appropriate 
or not, without dumping on anyone in particular.

Terry Jan Reedy



From ssteinerx at gmail.com  Wed Oct 28 04:27:00 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 27 Oct 2009 23:27:00 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <hc8c7e$2d8$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net><200910240946.48542.steve@pearwood.info>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>	<4AE3E6C6.4010008@gmail.com>	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>	<hc4h5s$vch$1@ger.gmane.org><ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>	<hc75qn$a0k$1@ger.gmane.org>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>	<hc7fdl$ia8$1@ger.gmane.org>
	<CC725036-196D-468A-BDA4-9BD1217F3669@gmail.com>
	<hc8c7e$2d8$1@ger.gmane.org>
Message-ID: <5CA39441-BE18-40C3-BEA5-330FF38D2AAE@gmail.com>


On Oct 27, 2009, at 11:02 PM, Terry Reedy wrote:

> ssteinerX at gmail.com wrote:

> This topic needs its own flippin' newsgroup.
> You could have said just that, appropriate or not, without dumping  
> on anyone in particular.

I was not trying to dump on you in particular, I picked a random  
message of the trillions that went by and happened to get you.  I  
apologize if you felt dumped on.

No offense intended to you in particular but this thread has gone on  
and on and on and on and on and on...

S


From debatem1 at gmail.com  Wed Oct 28 04:34:34 2009
From: debatem1 at gmail.com (geremy condra)
Date: Tue, 27 Oct 2009 23:34:34 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <9D7624CE452343719A7ED512B2FD17F5@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<hc7fdl$ia8$1@ger.gmane.org>
	<ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>
	<f3cc57c60910271227p2f4edc0xcdf6213b36a5cac2@mail.gmail.com>
	<9D7624CE452343719A7ED512B2FD17F5@RaymondLaptop1>
Message-ID: <f3cc57c60910272034y9cf2551qc98b85ca0f05eaf1@mail.gmail.com>

On Tue, Oct 27, 2009 at 3:49 PM, Raymond Hettinger <python at rcn.com> wrote:
>
> [geremy condra]
>>
>> Was it ever decided whether this would fall under the moratorium?
>
> Decided isn't the right word:
> http://mail.python.org/pipermail/python-dev/2009-October/093373.html
>

<snip>

I'm unclear- does that imply that this is this going to be decided on a
case-by-case basis in the future or have the details for the big M just
not been hammered out yet?

Geremy Condra

From david.lyon at preisshare.net  Wed Oct 28 06:13:21 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 28 Oct 2009 01:13:21 -0400
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
In-Reply-To: <loom.20091022T121543-839@post.gmane.org>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>	<cd6401a0910211600m6c4c83ah898e810c97d5fbef@mail.gmail.com>	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
	<0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
	<4ADFEF35.6000409@g.nevcal.com>
	<loom.20091022T121543-839@post.gmane.org>
Message-ID: <5e4f7ac667b4bd5ffdd40f2b45d969df@preisshare.net>

On Thu, 22 Oct 2009 10:20:03 +0000 (UTC), Antoine Pitrou
<solipsis at pitrou.net> wrote:
> (*) Remember, however, that Tarek and work on Distribute, and also on
> bringing pieces of setuptools/Distribute functionality into distutils.

But if that's the case then why not work on any third party tool..? like 
pip or setuptools?

It seems are very longwinded process if the only way to work on
python is to work on distutils but before doing that you have to
first work on distribute and then wait for all the changes to work
their way back up the chain..

Actually, I have finally worked out what I want. That is shell support
in the python windows distribution so that you can right click an
.egg and install it.

I don't see how that can be achieved by following the workprocess
that you describe above.

David


From regebro at gmail.com  Wed Oct 28 07:30:44 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 28 Oct 2009 07:30:44 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE0C73C.9070805@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
Message-ID: <319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>

2009/10/22 "Martin v. L?wis" <martin at v.loewis.de>:
> What use has such a stepping stone? Why, and (more importantly) when
> would anybody currently supporting 2.x give up 2.6 and earlier, and
> only support 2.7? And, if they chose to do so, why would they not move
> the code base to 3.x right away?

Python 2 support is not only about supporting old versions of Python,
but also supporting users of Python2-only modules.

If you have a module that runs only on Python 2.7, the people who for
some reason can not move to Python 3 yet, can still use it, by running
Python 2.7. For that your module doesn't have to run on 2.6 or 2.5.


So 2.7 support will for the most part be a case not of supporting
Python versions, but Python *users*.
Which is a good thing.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From ncoghlan at gmail.com  Wed Oct 28 07:40:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 28 Oct 2009 16:40:08 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from a
 set	withoutremoving it
In-Reply-To: <ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>
	<hc75qn$a0k$1@ger.gmane.org>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>	<hc7fdl$ia8$1@ger.gmane.org>
	<ca471dc20910271213y6292d813jcfbc3de823973db@mail.gmail.com>
Message-ID: <4AE7E748.9060400@gmail.com>

Guido van Rossum wrote:
> My gut tells me it is bad API design to collapse these two use cases.
> Probably because the implementations won't have much in common: (1)
> should just pick the first valid element, while (2) should use the
> normal hash lookup algorithm (shared with 'in', .add() etc.).

As a side note, a fairly obvious method name for "add if missing and
then return canonical representation" did occur to me: s.intern(x)

I'd be -0 on expanding the set API for this though (since the cookbook
recipe overloading __eq__ already provides an efficient solution).

As far as the moratorium in general goes, perhaps we should draw a line
between API changes that affect the ABCs in collections and numbers and
new convenience methods on the builtin types themselves?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Oct 28 07:42:40 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 28 Oct 2009 16:42:40 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
 removing it
In-Reply-To: <d38f5330910271050y5f9e53a7sbca23b3058e1bf61@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>	<hbvl1i$gne$2@ger.gmane.org>
	<4AE3E6C6.4010008@gmail.com>	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>	<hc4h5s$vch$1@ger.gmane.org>	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>	<7b0045ed0910261154t558dfe97h80b0ccd513890d77@mail.gmail.com>	<hc75qn$a0k$1@ger.gmane.org>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<d38f5330910271050y5f9e53a7sbca23b3058e1bf61@mail.gmail.com>
Message-ID: <4AE7E7E0.50602@gmail.com>

Alexander Belopolsky wrote:
> Odd indeed.  My first reaction was: it is not needed because lists
> support slicing, but when I tried to construct a list.get() using
> slicing the best I could come up with was the following hack
> 
>>>> def lget(l, i, v):  return (l[i:] or [v])[0]
> ...
>>>> lget(range(10), 20, 200)
> 200
>>>> lget(range(10), 5, 50)
> 5
> 
> Yet for some reason I never missed this functionality ...

People tend to do length checks on lists to decide which items are and
are not present. You can't do that with a dict.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Oct 28 08:00:44 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 28 Oct 2009 17:00:44 +1000
Subject: [Python-Dev] Set methods for mapping views
In-Reply-To: <200910271029.58591.steve@pearwood.info>
References: <617D5DBD3D6549958A4AF1EBA25C163C@RaymondLaptop1>	<loom.20091027T000556-818@post.gmane.org>	<ca471dc20910261609p2e5c5134l4ab1a53d5e3d13e8@mail.gmail.com>
	<200910271029.58591.steve@pearwood.info>
Message-ID: <4AE7EC1C.809@gmail.com>

Steven D'Aprano wrote:
> On Tue, 27 Oct 2009 10:09:14 am Guido van Rossum wrote:
>> On Mon, Oct 26, 2009 at 4:06 PM, Antoine Pitrou <solipsis at pitrou.net> 
> wrote:
>>> Steven D'Aprano <steve <at> pearwood.info> writes:
>>>> I don't understand that rationale. Let's take a concrete example.
>>>> The new `yield from` syntax was accepted
>>> Was it?
>> No.
> 
> I thought it had been. My mistake. Serves me right for not checking the 
> PEP first. But my point still stands.

Rather than going through and saying "oh, change X touches on area Y,
where we're waiting to see how change Z plays out, so we shouldn't do X
yet" it is simpler to declare the core of the language off limits for a
release or two.

We made a lot of big changes to fairly core parts of the language in a
relatively short period. Giving those a chance to settle down before we
start fiddling further is unlikely to be detrimental in the long run.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Wed Oct 28 10:12:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 28 Oct 2009 09:12:14 +0000 (UTC)
Subject: [Python-Dev] nonlocal keyword in 2.x?
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>
Message-ID: <loom.20091028T101119-295@post.gmane.org>

Lennart Regebro <regebro <at> gmail.com> writes:
> 
> So 2.7 support will for the most part be a case not of supporting
> Python versions, but Python *users*.

That's still not a good reason to backport nonlocal. The same reasoning could be
used to backport new features to the 2.6 branch after all.

Regards

Antoine.



From mal at egenix.com  Wed Oct 28 11:43:54 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 28 Oct 2009 11:43:54 +0100
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
Message-ID: <4AE8206A.9020108@egenix.com>

Tarek Ziad? wrote:
> Hello,
> 
> Since the addition of PEP 370, (per-user site packages), site.py and
> distutils/command/install.py are *both* providing the various
> installation directories for Python,
> depending on the system and the Python version.
> 
> We have also started to discuss lately in various Mailing Lists the
> addition of new schemes for IronPython and Jython, meaning that we
> might add some more in both places.
> 
> I would like to suggest a simplification by adding a dedicated module
> to manage these installation schemes in one single place in the
> stdlib.
> 
> This new independant module would be used by site.py and distutils and
> would also make it easier for third party code to work with these
> schemes.
> Of course this new module would be rather simple and not add any new
> import statement to avoid any overhead when Python starts and loads
> site.py

+1

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 28 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fetchinson at googlemail.com  Wed Oct 28 12:01:36 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Wed, 28 Oct 2009 12:01:36 +0100
Subject: [Python-Dev] Python Package Management Roadmap in Python
	Releases
In-Reply-To: <5e4f7ac667b4bd5ffdd40f2b45d969df@preisshare.net>
References: <42E841ED-E8C3-4F97-9E56-8DFDCB687386@python.org>
	<1998C055-4E0F-46F0-A0EE-142866C1B114@python.org>
	<b1535e13caf35563d0b22a78d4b5882d@preisshare.net>
	<bbaeab100910211756o3977ef3al5ddea5a1421f671e@mail.gmail.com>
	<13a7dd22a702e20a2526b48bd932e5a8@preisshare.net>
	<bbaeab100910211838w680262edjcba33938316f5284@mail.gmail.com>
	<0841bc282e204bc4dadfbdb0e87feb51@preisshare.net>
	<4ADFEF35.6000409@g.nevcal.com>
	<loom.20091022T121543-839@post.gmane.org>
	<5e4f7ac667b4bd5ffdd40f2b45d969df@preisshare.net>
Message-ID: <fbe2e2100910280401v454dd76et353f2211a10b7d0f@mail.gmail.com>

>> (*) Remember, however, that Tarek and work on Distribute, and also on
>> bringing pieces of setuptools/Distribute functionality into distutils.
>
> But if that's the case then why not work on any third party tool..? like
> pip or setuptools?
>
> It seems are very longwinded process if the only way to work on
> python is to work on distutils but before doing that you have to
> first work on distribute and then wait for all the changes to work
> their way back up the chain..
>
> Actually, I have finally worked out what I want. That is shell support
> in the python windows distribution so that you can right click an
> .egg and install it.
>
> I don't see how that can be achieved by following the workprocess
> that you describe above.

As has been said by many, you are entirely welcome to work on whatever
tool you think is useful. Once you are done you are again welcome to
distribute your tool or application to users and see how many users
are happy with it. Once you are done with this step as well, you are
again encouraged to come back to python-dev and say:

"In the last X months my app/tool became very popular in the python
community. There are Y developers working on the app/tool and there
are Z happy users. I'd suggest including it in the python stdlib or
I'd suggest coordinating the releases of my app/tool with that of
python."

At this point a useful conversation can start. Please note that a
similarly useful conversation is impossible to take place before all
the above steps have been completed.

HTH,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From fuzzyman at voidspace.org.uk  Wed Oct 28 12:02:50 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 28 Oct 2009 11:02:50 +0000
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <4AE8206A.9020108@egenix.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com>
Message-ID: <4AE824DA.4090701@voidspace.org.uk>

M.-A. Lemburg wrote:
> Tarek Ziad? wrote:
>   
>> Hello,
>>
>> Since the addition of PEP 370, (per-user site packages), site.py and
>> distutils/command/install.py are *both* providing the various
>> installation directories for Python,
>> depending on the system and the Python version.
>>
>> We have also started to discuss lately in various Mailing Lists the
>> addition of new schemes for IronPython and Jython, meaning that we
>> might add some more in both places.
>>
>> I would like to suggest a simplification by adding a dedicated module
>> to manage these installation schemes in one single place in the
>> stdlib.
>>
>> This new independant module would be used by site.py and distutils and
>> would also make it easier for third party code to work with these
>> schemes.
>> Of course this new module would be rather simple and not add any new
>> import statement to avoid any overhead when Python starts and loads
>> site.py
>>     
>
> +1
>
>   
Also +1. It seems like this would make things easier for the alternative 
implementations.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Wed Oct 28 12:36:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 28 Oct 2009 11:36:27 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost> <4AE5A768.6080900@molden.no>
	<930F189C8A437347B80DF2C156F7EC7F098FF42768@exchis.ccp.ad.local>
Message-ID: <loom.20091028T123311-608@post.gmane.org>

Kristj?n Valur J?nsson <kristjan <at> ccpgames.com> writes:
> 
> In my experience (from stackless python) using priority wakeup for IO can
result in very erratic
> scheduling when there is much IO going on, every IO trumping another.

I whipped up a trivial multithreaded HTTP server using
socketserver.ThreadingMixin and wsgiref, and used apachebench against it with a
reasonable concurrency level (10 requests at once). Enabling/disabling priority
requests doesn't seem to make a difference.

Regards

Antoine.



From skip at pobox.com  Wed Oct 28 14:35:54 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 28 Oct 2009 08:35:54 -0500
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <loom.20091028T101119-295@post.gmane.org>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>
	<loom.20091028T101119-295@post.gmane.org>
Message-ID: <19176.18618.662587.613771@montanaro.dyndns.org>


    >> So 2.7 support will for the most part be a case not of supporting
    >> Python versions, but Python *users*.

    Antoine> That's still not a good reason to backport nonlocal. The same
    Antoine> reasoning could be used to backport new features to the 2.6
    Antoine> branch after all.

No, because 2.6 is in feature freeze (bug fixes only).  2.7 is the current
version of 2.x where new features are allowed to be added.

Skip

From ssteinerx at gmail.com  Wed Oct 28 15:30:29 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 28 Oct 2009 10:30:29 -0400
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <4AE824DA.4090701@voidspace.org.uk>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
Message-ID: <ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>


On Oct 28, 2009, at 7:02 AM, Michael Foord wrote:

> M.-A. Lemburg wrote:
>> Tarek Ziad? wrote:
>>
>>> Hello,
>>>
>>> Since the addition of PEP 370, (per-user site packages), site.py and
>>> distutils/command/install.py are *both* providing the various
>>> installation directories for Python,
>>> depending on the system and the Python version.
>>>
>>> We have also started to discuss lately in various Mailing Lists the
>>> addition of new schemes for IronPython and Jython, meaning that we
>>> might add some more in both places.
>>>
>>> I would like to suggest a simplification by adding a dedicated  
>>> module
>>> to manage these installation schemes in one single place in the
>>> stdlib.
>>>
>>> This new independant module would be used by site.py and distutils  
>>> and
>>> would also make it easier for third party code to work with these
>>> schemes.
>>> Of course this new module would be rather simple and not add any new
>>> import statement to avoid any overhead when Python starts and loads
>>> site.py

+1

This would help unpollute ~/ which is getting quite full of 'stuff.'

S


From solipsis at pitrou.net  Wed Oct 28 15:49:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 28 Oct 2009 14:49:47 +0000 (UTC)
Subject: [Python-Dev] nonlocal keyword in 2.x?
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>
	<loom.20091028T101119-295@post.gmane.org>
	<19176.18618.662587.613771@montanaro.dyndns.org>
Message-ID: <loom.20091028T154811-994@post.gmane.org>

<skip <at> pobox.com> writes:
> 
>     >> So 2.7 support will for the most part be a case not of supporting
>     >> Python versions, but Python *users*.
> 
>     Antoine> That's still not a good reason to backport nonlocal. The same
>     Antoine> reasoning could be used to backport new features to the 2.6
>     Antoine> branch after all.
> 
> No, because 2.6 is in feature freeze (bug fixes only).  2.7 is the current
> version of 2.x where new features are allowed to be added.

That was precisely my point. There are development practices which mitigate the
idea that backporting is always helpful to the user.

Regards

Antoine.



From dreamingforward at gmail.com  Wed Oct 28 17:22:46 2009
From: dreamingforward at gmail.com (average)
Date: Wed, 28 Oct 2009 09:22:46 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net> <4AE1E71C.7020806@gmail.com>
	<hbsun9$432$1@ger.gmane.org> <200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
Message-ID: <913f9f570910280922v5687db91m8bb0557ef2480c56@mail.gmail.com>

[Guido wrote:]
> - If sets were to grow an API to non-destructively access the object
> stored in the set for a particular key, then dicts should have such a
> method too.
> - I still wish we could go back in time and unify sets and dicts, if
> only to find out how that experiment would turn out.

+5.  If Python3 were to have this feature it would make it worth
migrating to (and nearly worthy of the intent that was behind the idea
of python3k).

FWIW ... :)

marcos

From dreamingforward at gmail.com  Wed Oct 28 17:34:45 2009
From: dreamingforward at gmail.com (average)
Date: Wed, 28 Oct 2009 09:34:45 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <913f9f570910280931p3da32519kd56f251264ff867@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<200910240946.48542.steve@pearwood.info>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<913f9f570910280922v5687db91m8bb0557ef2480c56@mail.gmail.com>
	<913f9f570910280931p3da32519kd56f251264ff867@mail.gmail.com>
Message-ID: <913f9f570910280934n4030f54u19e865c9fbed0c88@mail.gmail.com>

[I wrote:]
>?If Python3 were to have this feature it would make it worth
> migrating to

Sorry that may have sounded more harsh than I expected. ?If I had more
resources, I'd propose (and volunteer) a python3000 branch where any
and all who were disappointed at the *lack* of compatability changes
could continue working on the core language. ?(Moratorium controversy
solved and quaranteened--any desireable features for the working
branch would be back-propagated via the time machine as BDFL permits.)

marcos

From ziade.tarek at gmail.com  Wed Oct 28 19:05:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Oct 2009 19:05:06 +0100
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
Message-ID: <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>

M.-A. Lemburg wrote:
> +1

On Oct 28, 2009, at 7:02 AM, Michael Foord wrote:
> Also +1. It seems like this would make things easier for the alternative implementations.

On Wed, Oct 28, 2009 at 3:30 PM, ssteinerX at gmail.com
<ssteinerx at gmail.com> wrote:
> +1

Ok then I'll work on a patch for that change and start an issue about it soon.

> This would help unpollute ~/ which is getting quite full of 'stuff.'

Notice that there's a discussion going on about that at
http://bugs.python.org/issue7175

Regards
Tarek

From debatem1 at gmail.com  Wed Oct 28 19:59:52 2009
From: debatem1 at gmail.com (geremy condra)
Date: Wed, 28 Oct 2009 14:59:52 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <913f9f570910280934n4030f54u19e865c9fbed0c88@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<913f9f570910280922v5687db91m8bb0557ef2480c56@mail.gmail.com>
	<913f9f570910280931p3da32519kd56f251264ff867@mail.gmail.com>
	<913f9f570910280934n4030f54u19e865c9fbed0c88@mail.gmail.com>
Message-ID: <f3cc57c60910281159q5e1532b3gc1740d917a33c072@mail.gmail.com>

On Wed, Oct 28, 2009 at 12:34 PM, average <dreamingforward at gmail.com> wrote:
> [I wrote:]
>>?If Python3 were to have this feature it would make it worth
>> migrating to
>
> Sorry that may have sounded more harsh than I expected. ?If I had more
> resources, I'd propose (and volunteer) a python3000 branch where any
> and all who were disappointed at the *lack* of compatability changes
> could continue working on the core language. ?(Moratorium controversy
> solved and quaranteened--any desireable features for the working
> branch would be back-propagated via the time machine as BDFL permits.)
>
> marcos

This is effectively the sandbox idea I proposed, except with backporting.

Geremy Condra

From olemis at gmail.com  Wed Oct 28 22:28:30 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 28 Oct 2009 16:28:30 -0500
Subject: [Python-Dev] Where is `ctypes.com` ?
Message-ID: <24ea26600910281428n26497103v48e96a45b464d804@mail.gmail.com>

Hello !

Recently I found a code snippet [1]_ illustrating integration between
Python and COM technology in Win32 systems. I tried to reproduce it
and I can't import module `ctypes.com`.

Q:

  - Is it (`ctypes.com`) distributed with stdlib ?

If that's true then I'm affraid that those py files were removed
somehow (accidentally ? who knows ? ) ...

Thnx in advance !

.. [1] ctypes.win32.com.server @ Py
        (http://svn.python.org/projects/ctypes/tags/release_0_6_2/ctypes/win32/com/server.py)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
[visualization-api] ANN: TracGViz 1.3.4 released: msg#00244 ...  -
http://feedproxy.google.com/~r/TracGViz-full/~3/OBfKGS_sy2E/msg00244.html

From theller at ctypes.org  Wed Oct 28 22:39:32 2009
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 28 Oct 2009 22:39:32 +0100
Subject: [Python-Dev] Where is `ctypes.com` ?
In-Reply-To: <24ea26600910281428n26497103v48e96a45b464d804@mail.gmail.com>
References: <24ea26600910281428n26497103v48e96a45b464d804@mail.gmail.com>
Message-ID: <hcadmk$a0h$1@ger.gmane.org>

Olemis Lang schrieb:
> Hello !
> 
> Recently I found a code snippet [1]_ illustrating integration between
> Python and COM technology in Win32 systems. I tried to reproduce it
> and I can't import module `ctypes.com`.

First, the python-dev mailing list is used for developing the Python language
itself, not developing WITH Python.  For questions about developing with Python,
please use the newsgroup comp.lang.python.

> Q:
> 
>   - Is it (`ctypes.com`) distributed with stdlib ?
> 
> If that's true then I'm affraid that those py files were removed
> somehow (accidentally ? who knows ? ) ...

ctypes.com is very old and has been superseeded by the comtypes
package: http://starship.python.net/crew/theller/comtypes/

An even more popular package for using COM with python is
pywin32: http://sourceforge.net/projects/pywin32/

-- 
Thanks,
Thomas


From olemis at gmail.com  Wed Oct 28 22:48:56 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 28 Oct 2009 16:48:56 -0500
Subject: [Python-Dev] Where is `ctypes.com` ?
In-Reply-To: <hcadmk$a0h$1@ger.gmane.org>
References: <24ea26600910281428n26497103v48e96a45b464d804@mail.gmail.com>
	<hcadmk$a0h$1@ger.gmane.org>
Message-ID: <24ea26600910281448x7f586efdpf72171eb4972cbe9@mail.gmail.com>

On Wed, Oct 28, 2009 at 4:39 PM, Thomas Heller <theller at ctypes.org> wrote:
> Olemis Lang schrieb:
>> Hello !
>>
>> Recently I found a code snippet [1]_ illustrating integration between
>> Python and COM technology in Win32 systems. I tried to reproduce it
>> and I can't import module `ctypes.com`.
>
> First, the python-dev mailing list is used for developing the Python language
> itself, not developing WITH Python. ?For questions about developing with Python,
> please use the newsgroup comp.lang.python.
>

I apologize if needed ... but I didn't ask about ?how to program
things using `ctypes.com` ?. I asked here because you are the ones who
should know about modules in stdlib . That's why I thought that it was
not ?very? OT

>> Q:
>>
>> ? - Is it (`ctypes.com`) distributed with stdlib ?
>>
>> If that's true then I'm affraid that those py files were removed
>> somehow (accidentally ? who knows ? ) ...
>
> ctypes.com is very old and has been superseeded by the comtypes
> package: http://starship.python.net/crew/theller/comtypes/
>

Thnx !

> An even more popular package for using COM with python is
> pywin32: http://sourceforge.net/projects/pywin32/
>

Yes, I've just tried that one, but the fact is that I'm experiencing a
problem with it (that I won't detail in here because this is not the
list to talk about such subjects , isn't it ? ... just learning fast
... :P ) and I thought I could switch to that one ...

Thnx very much for your reply !

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Suggestions: Wave (private) Groups, integration - Google Wave API ...
- http://feedproxy.google.com/~r/TracGViz-full/~3/cuwdwGkX1WA/90bf35ca0c38caf0

From stephen at xemacs.org  Thu Oct 29 03:31:54 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 29 Oct 2009 11:31:54 +0900
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	without	removing it
In-Reply-To: <f3cc57c60910281159q5e1532b3gc1740d917a33c072@mail.gmail.com>
References: <200910231132.45504.w.richert@gmx.net>
	<d38f5330910241034j698d3395xecba0b7b580ca92f@mail.gmail.com>
	<hbvl1i$gne$2@ger.gmane.org> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<913f9f570910280922v5687db91m8bb0557ef2480c56@mail.gmail.com>
	<913f9f570910280931p3da32519kd56f251264ff867@mail.gmail.com>
	<913f9f570910280934n4030f54u19e865c9fbed0c88@mail.gmail.com>
	<f3cc57c60910281159q5e1532b3gc1740d917a33c072@mail.gmail.com>
Message-ID: <87d4476icl.fsf@uwakimon.sk.tsukuba.ac.jp>

geremy condra writes:
 > On Wed, Oct 28, 2009 at 12:34 PM, average <dreamingforward at gmail.com> wrote:
 > > [I wrote:]
 > >>?If Python3 were to have this feature it would make it worth
 > >> migrating to
 > >
 > > Sorry that may have sounded more harsh than I expected. ?If I had more
 > > resources,

 > This is effectively the sandbox idea I proposed,

Um, guys?  Python 3000 was a project involving fairly concentrated
effort by about a dozen senior, very skilled, developers plus many
others over a period of several (3 or 4?) years.  The "sandbox" is
going to be played in by people without the resources to create such
an industrial-strength "code workshop," and as such is not going to be
given anywhere near the credibility that the py3k effort got.  As
evidence of how difficult an effort like "Python 3000" is, consider:
where is Perl 6?  There are hundreds, perhaps thousands, of developers
saying "we want Perl 6" but the resources aren't there to *finish* it.

On the other hand, "this feature" is already present in Python in
great generality.  You just have to spell it

    for x in container:
        break

(or any of several other ways).  The multiple spellings (of different
degrees of efficiency) and the lack of an obvious spelling are indeed
warts IMO, but "this feature would make Python 3 worth migrating to"
and "+5" are hard for me to understand except as extreme exaggeration.

The Python 3000 effort succeeded because the resources were sufficient
to the goal.  Part of that is because some very powerful developers
found some itches they needed to scratch: Guido is clearly happy with
P3tL (Python 3 the Language), both its current state and the fact that
all that effort was devoted to it.

But now he (inter alia) wants to focus effort (mostly not his own ;-)
on P3tSL (Python 3 the Standard Library) which still needs substantial
cleanup though it's already very usable for many applications, and on
P3S3PA (Python 3 Supported 3rd Party Applications).  The resources
available for language evolution have dried up; *you* who want more
language evolution are going to have to supply them.

OTOH, there are at present few attractive improvements.  set.getany()
is an example of the kind of decidedly unattractive improvement
currently available.  It has an efficient if not obvious spelling as
above; it has several less efficient but more transparent spellings as
have been posted several times.  The *need* is not infinitesimal; some
good programmers have testified to being stumped by this one.  But
it's not big, either.  The *resources* required are surprisingly large:
although we already have a satisfactory implementation, about half of
this thread has been devoted to discussing the various names proposed,
and no conclusion reached in this thread or the previous one (of
similar length).



From debatem1 at gmail.com  Thu Oct 29 04:23:06 2009
From: debatem1 at gmail.com (geremy condra)
Date: Wed, 28 Oct 2009 23:23:06 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
In-Reply-To: <87d4476icl.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <200910231132.45504.w.richert@gmx.net> <4AE3E6C6.4010008@gmail.com>
	<d38f5330910260804o5a23d928k20f039929a9f287c@mail.gmail.com>
	<hc4h5s$vch$1@ger.gmane.org>
	<ca471dc20910260938s15b46e7ahe9e2f0caa33b6ff7@mail.gmail.com>
	<913f9f570910280922v5687db91m8bb0557ef2480c56@mail.gmail.com>
	<913f9f570910280931p3da32519kd56f251264ff867@mail.gmail.com>
	<913f9f570910280934n4030f54u19e865c9fbed0c88@mail.gmail.com>
	<f3cc57c60910281159q5e1532b3gc1740d917a33c072@mail.gmail.com>
	<87d4476icl.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <f3cc57c60910282023s3d1cb36fpbda4983d672f524@mail.gmail.com>

[Stephen Turbull]
>?The resources available for language evolution have dried up;
> *you* who want more language evolution are going to have to
> supply them.

That's the idea I've been advocating- a code-first-jaw-later
approach and the sandbox to support it. Its not even that
hard to find where I've said exactly that in the moratorium
discussion.

Geremy Condra

From holdenweb at gmail.com  Thu Oct 29 04:34:25 2009
From: holdenweb at gmail.com (Steve Holden)
Date: Wed, 28 Oct 2009 23:34:25 -0400
Subject: [Python-Dev] MSDN subscribers: Using Visual Studio?
Message-ID: <4AE90D41.8080107@holdenweb.com>

I just wondered, with the recent flood of new MSDN subscriptions loosed
on the developer community, how many people have installed the required
version of Visual Studio and built Python for Windows from source? Not
being that familiar with the process myself I was hoping for some advice
from the inexperienced who have overcome any hurdles there might be. I
have a pristine virtual machine just waiting for the right pieces ...

Also what other uses have you found for the licenses? It would be good
to get some information about how useful the licenses have been, and how
they have helped people to improve Python's quality or ease of
distribution (if they have). I'm sure Microsoft would also appreciate
some positive feedback to their generosity, and I'll undertake to
provide that if this message elicits much by way of reply.

Since I'm pretty much too busy to follow the dev list right now I'd
appreciate direct Cc's.

regards
 Steve

PS: If any further core developers need licenses, I plan to apply to
Microsoft again in the new year. I'll be sending out a message then, I
don't intend to keep a waiting list.
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Watch PyCon on video now!          http://pycon.blip.tv/

From regebro at gmail.com  Thu Oct 29 09:15:47 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 29 Oct 2009 09:15:47 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <loom.20091028T154811-994@post.gmane.org>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>
	<loom.20091028T101119-295@post.gmane.org>
	<19176.18618.662587.613771@montanaro.dyndns.org>
	<loom.20091028T154811-994@post.gmane.org>
Message-ID: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>

2009/10/28 Antoine Pitrou <solipsis at pitrou.net>:
> <skip <at> pobox.com> writes:
>>
>> ? ? >> So 2.7 support will for the most part be a case not of supporting
>> ? ? >> Python versions, but Python *users*.
>>
>> ? ? Antoine> That's still not a good reason to backport nonlocal. The same
>> ? ? Antoine> reasoning could be used to backport new features to the 2.6
>> ? ? Antoine> branch after all.
>>
>> No, because 2.6 is in feature freeze (bug fixes only). ?2.7 is the current
>> version of 2.x where new features are allowed to be added.
>
> That was precisely my point.

Then I don't understand what you are saying. Obviously we shouldn't
backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I
don't see what 2.6 has to do with it.

> There are development practices which mitigate the
> idea that backporting is always helpful to the user.

And those are?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From ncoghlan at gmail.com  Thu Oct 29 10:23:36 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 29 Oct 2009 19:23:36 +1000
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<4AE0BF83.4080903@v.loewis.de>	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>	<4AE0C73C.9070805@v.loewis.de>	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>	<loom.20091028T101119-295@post.gmane.org>	<19176.18618.662587.613771@montanaro.dyndns.org>	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
Message-ID: <4AE95F18.4040807@gmail.com>

Lennart Regebro wrote:
> 2009/10/28 Antoine Pitrou <solipsis at pitrou.net>:
>> <skip <at> pobox.com> writes:
>>>     >> So 2.7 support will for the most part be a case not of supporting
>>>     >> Python versions, but Python *users*.
>>>
>>>     Antoine> That's still not a good reason to backport nonlocal. The same
>>>     Antoine> reasoning could be used to backport new features to the 2.6
>>>     Antoine> branch after all.
>>>
>>> No, because 2.6 is in feature freeze (bug fixes only).  2.7 is the current
>>> version of 2.x where new features are allowed to be added.
>> That was precisely my point.
> 
> Then I don't understand what you are saying. Obviously we shouldn't
> backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I
> don't see what 2.6 has to do with it.
> 
>> There are development practices which mitigate the
>> idea that backporting is always helpful to the user.
> 
> And those are?

You said it above yourself: "bugfix mode"

That's all Antoine's point was - backporting of new features to previous
branches is not automatically a good idea.

In the case of 3.2 -> 2.7 backports, there are issues with the initial
development time investment to do the backport, future double-keying of
additional maintenance issues, consideration of possible poor
interaction with legacy features in the 2.x series. It's a bunch of
additional work that isn't going to happen without someone volunteering
to do it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Thu Oct 29 12:57:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 29 Oct 2009 12:57:49 +0100
Subject: [Python-Dev] MSDN subscribers: Using Visual Studio?
In-Reply-To: <4AE90D41.8080107@holdenweb.com>
Message-ID: <1256817469.5534.4.camel@localhost>

Steve Holden <holdenweb <at> gmail.com> writes:
> 
> I just wondered, with the recent flood of new MSDN subscriptions loosed
> on the developer community, how many people have installed the required
> version of Visual Studio and built Python for Windows from source? Not
> being that familiar with the process myself I was hoping for some advice
> from the inexperienced who have overcome any hurdles there might be. I
> have a pristine virtual machine just waiting for the right pieces ...

If it's just a matter of building and testing it you don't need any MSDN
subscription, you just need Visual Studio Express which is free (as in
free beer...). I don't know if it allows you to build an installer
though.

I won't reply to the other part of the message since I haven't asked for
(nor received) any MSDN subscription :)

Regards

Antoine.



From solipsis at pitrou.net  Thu Oct 29 15:30:05 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 29 Oct 2009 14:30:05 +0000 (UTC)
Subject: [Python-Dev] "Buildbot" category on the tracker
Message-ID: <loom.20091029T152906-465@post.gmane.org>


Hello,

What do you think of creating a "buildbot" category in the tracker? There are
often problems on specific buildbots which would be nice to track, but there's
nowhere to do so.

Regards

Antoine.



From daniel at stutzbachenterprises.com  Thu Oct 29 18:15:36 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 29 Oct 2009 12:15:36 -0500
Subject: [Python-Dev] MSDN subscribers: Using Visual Studio?
In-Reply-To: <1256817469.5534.4.camel@localhost>
References: <4AE90D41.8080107@holdenweb.com>
	<1256817469.5534.4.camel@localhost>
Message-ID: <eae285400910291015v6a5a6aa7u139175c04a2ed2dd@mail.gmail.com>

On Thu, Oct 29, 2009 at 6:57 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> If it's just a matter of building and testing it you don't need any MSDN
> subscription, you just need Visual Studio Express which is free (as in
> free beer...). I don't know if it allows you to build an installer
> though.
>

It does.  If I recall correctly, in addition to Visual Studio Express, I
also needed the Windows SDK (which is also free as in beer).

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091029/4f90aa82/attachment.htm>

From lists at cheimes.de  Thu Oct 29 19:21:50 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 29 Oct 2009 19:21:50 +0100
Subject: [Python-Dev] MSDN subscribers: Using Visual Studio?
In-Reply-To: <eae285400910291015v6a5a6aa7u139175c04a2ed2dd@mail.gmail.com>
References: <4AE90D41.8080107@holdenweb.com>	<1256817469.5534.4.camel@localhost>
	<eae285400910291015v6a5a6aa7u139175c04a2ed2dd@mail.gmail.com>
Message-ID: <hccmfu$11r$1@ger.gmane.org>

Daniel Stutzbach wrote:
> It does.  If I recall correctly, in addition to Visual Studio Express, I
> also needed the Windows SDK (which is also free as in beer).

The VS 2008 Express Edition is sufficient to build X86 binaries on
Windows. The express edition doesn't support X64_86. though.

Christian


From martin at v.loewis.de  Thu Oct 29 23:05:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Oct 2009 23:05:16 +0100
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <loom.20091029T152906-465@post.gmane.org>
References: <loom.20091029T152906-465@post.gmane.org>
Message-ID: <4AEA119C.5070707@v.loewis.de>

> What do you think of creating a "buildbot" category in the tracker? There are
> often problems on specific buildbots which would be nice to track, but there's
> nowhere to do so.

Do you have any specific reports that you would want to classify with
this category?

Regards,
Martin

From Bruno.Harbulot at manchester.ac.uk  Thu Oct 29 19:32:29 2009
From: Bruno.Harbulot at manchester.ac.uk (Bruno Harbulot)
Date: Thu, 29 Oct 2009 18:32:29 +0000
Subject: [Python-Dev] ssl module
Message-ID: <hccn3u$3eq$1@ger.gmane.org>

Hello,

I would like to ask a few questions and suggestions regarding the ssl 
module (in Python 2.6). (I gather from [1] that there is some effort 
going on to enhance the ssl API, but I'm not sure if this is the right 
place to discuss it.)

Like other Python users, I was a bit surprised by the lack of 
verification of httplib/urllib2 (hence I started to write a small 
library a while back, only published today [2]), but the following 
points are not HTTP-specific.

1. Hostname checking.

   From what I gather by reading the archives on this list, the issue of 
hostname checking seems controversial [3]. It seems widely admitted by 
browser communities nowadays to check that the hostname the CN field of 
the subject DN or the DNS entries of subjectAltName. I'd feel more 
comfortable if this was the default behaviour of the client in Python's 
SSL module, although having a mechanism to override this would be useful 
indeed. It's more or less a basic security requirement to check the 
identity of the server before doing anything else.


2. Cipher suite selection.

   It's useful to restrict the list of cipher suites that can be used, 
not just for speed (as mentioned in [1]), but also because some cipher 
suites may be considered insecure by some institutions. This would be a 
good feature to have indeed.


3. Full chain of certificates.

   The PyOpenSSL module is able to take a callback function that 
verifies each certificate in the chain (using depth). According to the 
documentation, the ssl module only exposes the first certificate in the 
chain (no CA). In some applications, it is useful to verify certain 
policies according to attributes further up in the chain.
I'd like to suggest having an 
"SSLSocket.getpeercerts(binary_form=False)" (plural) that returns a list 
of certificates in the verification chain.


Is there a place where the status of the ssl module is summarized, or a 
better place to discuss this? I could try to provide contributions or 
further details if appropriate.


Best wishes,

Bruno.



[1] http://mail.python.org/pipermail/python-dev/2009-September/091636.html
[2] http://code.google.com/p/python-httpclient
[2] http://bugs.python.org/issue1589


From solipsis at pitrou.net  Thu Oct 29 23:21:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 29 Oct 2009 22:21:31 +0000 (UTC)
Subject: [Python-Dev] (no subject)
References: <loom.20091029T152906-465@post.gmane.org>
	<4AEA119C.5070707@v.loewis.de>
Message-ID: <loom.20091029T232059-824@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > What do you think of creating a "buildbot" category in the tracker? There are
> > often problems on specific buildbots which would be nice to track, but
there's
> > nowhere to do so.
> 
> Do you have any specific reports that you would want to classify with
> this category?

I was thinking of http://bugs.python.org/issue4970

Regards

Antoine.



From exarkun at twistedmatrix.com  Fri Oct 30 00:04:43 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Thu, 29 Oct 2009 23:04:43 -0000
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <loom.20091029T152906-465@post.gmane.org>
References: <loom.20091029T152906-465@post.gmane.org>
Message-ID: <20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>

On 02:30 pm, solipsis at pitrou.net wrote:
>
>Hello,
>
>What do you think of creating a "buildbot" category in the tracker? 
>There are
>often problems on specific buildbots which would be nice to track, but 
>there's
>nowhere to do so.

Is your idea that this would be for tracking issues with the *bots* 
themselves?  That is, not just for tracking cases where some test method 
fails on a particular bot, but for tracking cases where, say, a bot's 
host has run out of disk space and cannot run the tests at all?

For the case where a test is failing because of some platform or 
environment issue, it seems more sensible to track the ticket as 
relating to that platform or environment, or track it in relation to the 
feature it affects.

Of course, tickets could move between these classifications as 
investigation reveals new information about the problem.

Jean-Paul

From jnoller at gmail.com  Fri Oct 30 00:41:59 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 29 Oct 2009 19:41:59 -0400
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
Message-ID: <4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>

On Thu, Oct 29, 2009 at 7:04 PM,  <exarkun at twistedmatrix.com> wrote:
> On 02:30 pm, solipsis at pitrou.net wrote:
>>
>> Hello,
>>
>> What do you think of creating a "buildbot" category in the tracker? There
>> are
>> often problems on specific buildbots which would be nice to track, but
>> there's
>> nowhere to do so.
>
> Is your idea that this would be for tracking issues with the *bots*
> themselves? ?That is, not just for tracking cases where some test method
> fails on a particular bot, but for tracking cases where, say, a bot's host
> has run out of disk space and cannot run the tests at all?
>
> For the case where a test is failing because of some platform or environment
> issue, it seems more sensible to track the ticket as relating to that
> platform or environment, or track it in relation to the feature it affects.
>
> Of course, tickets could move between these classifications as investigation
> reveals new information about the problem.

Then again, I know for a fact certain tests fail ONLY on certain
buildbots because of the way they're configured. For example, certain
multiprocessing tests will fail if /dev/shm isn't accessible on Linux,
and several of the buildbosts are in tight chroot jails and don't have
that exposed.

Is it a bug in that buildbot, a platform specific bug, etc?

jesse

From janssen at parc.com  Fri Oct 30 00:55:03 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 29 Oct 2009 16:55:03 PDT
Subject: [Python-Dev] ssl module
In-Reply-To: <hccn3u$3eq$1@ger.gmane.org>
References: <hccn3u$3eq$1@ger.gmane.org>
Message-ID: <48451.1256860503@parc.com>

Bruno Harbulot <Bruno.Harbulot at manchester.ac.uk> wrote:

> Hello,
> 
> I would like to ask a few questions and suggestions regarding the ssl
> module (in Python 2.6). (I gather from [1] that there is some effort
> going on to enhance the ssl API, but I'm not sure if this is the right
> place to discuss it.)
> 
> Is there a place where the status of the ssl module is summarized, or
> a better place to discuss this? I could try to provide contributions
> or further details if appropriate.

Yes, please look at the issues in the issue tracker already, and
contribute there.  We could particularly use more test cases to support
patches that people have already submitted.

> Like other Python users, I was a bit surprised by the lack of
> verification of httplib/urllib2 (hence I started to write a small
> library a while back, only published today [2]), but the following
> points are not HTTP-specific.
> 
> 1. Hostname checking.
> 
>   From what I gather by reading the archives on this list, the issue
> of hostname checking seems controversial [3]. It seems widely admitted
> by browser communities nowadays to check that the hostname the CN
> field of the subject DN or the DNS entries of subjectAltName. I'd feel
> more comfortable if this was the default behaviour of the client in
> Python's SSL module, although having a mechanism to override this
> would be useful indeed. It's more or less a basic security requirement
> to check the identity of the server before doing anything else.

I don't think it should happen by default in the ssl module client code,
but I agree it makes sense to do that in various application uses of
SSL, such as the httplib support for https, since that behavior is
(annoyingly) called for in the (experimental, not standard) RFC which
defines HTTP over SSL, and, as you say, it's widely done in Web browsers
when https is being used.  If you check the issues, you'll see that I
think there should be a helper function in the ssl module to do this
checking.  Would you like to contribute one?  Please either attach it
to an already open issue, or start a new feature request issue.

> 2. Cipher suite selection.
> 
>   It's useful to restrict the list of cipher suites that can be used,
> not just for speed (as mentioned in [1]), but also because some cipher
> suites may be considered insecure by some institutions. This would be
> a good feature to have indeed.

Yes, I agree.

> 3. Full chain of certificates.
> 
>   The PyOpenSSL module is able to take a callback function that
> verifies each certificate in the chain (using depth). According to the
> documentation, the ssl module only exposes the first certificate in
> the chain (no CA). In some applications, it is useful to verify
> certain policies according to attributes further up in the chain.
> I'd like to suggest having an
> "SSLSocket.getpeercerts(binary_form=False)" (plural) that returns a
> list of certificates in the verification chain.

Feel free to use PyOpenSSL for more complicated applications, like the
one you mention, and to file more issues on the Python issue tracker
about this.  Though, we were striving to have something small and simple
in the ssl module, not a feature-full binding to OpenSSL.  Oh, and
there's also a stdlib-sig, which (since the ssl module is part of the
stdlib) might be an appropriate place to discuss it.

Bill

From rdmurray at bitdance.com  Fri Oct 30 01:31:58 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 29 Oct 2009 20:31:58 -0400 (EDT)
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>

On Thu, 29 Oct 2009 at 19:41, Jesse Noller wrote:
> Then again, I know for a fact certain tests fail ONLY on certain
> buildbots because of the way they're configured. For example, certain
> multiprocessing tests will fail if /dev/shm isn't accessible on Linux,
> and several of the buildbosts are in tight chroot jails and don't have
> that exposed.
>
> Is it a bug in that buildbot, a platform specific bug, etc?

I'd say that particular one is a bug in the tests.  If /dev/shm is
not available and is required, then the tests should be skipped with
an appropriate message.  It would also secondarily be an issue with
the buildbot fleet, since multiprocessing would then not be getting
thoroughly tested by those buildbots.

IMO a buildbot category might be useful for bugs that show up in a
buildbot but no one can (currently) reproduce, or problems with the
buildbots themselves.  I don't think we currently have any bugs filed that
fall in the second category, but multiprocessing not getting completely
tested because of lack of /dev/shm would fall into that category.
Issue 4970 was in the first category until recently.

But the real reason for having a buildbot category (or at least a keyword)
would be to be able to tag all bugs that are currently making buildbots
fail that are _not_ the result of a recent checkin.  This would make
the task of finding the bugs that need to be cleaned up to stabilize
the buildbot fleet easier.  I'm currently aware of issues 4970, 3892,
and 6462 in this category, and there are a few more that we can/will file
if we continue to pay attention to the failure reports now arriving on
the irc channel.

--David (RDM)

From exarkun at twistedmatrix.com  Fri Oct 30 01:58:05 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 30 Oct 2009 00:58:05 -0000
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
Message-ID: <20091030005805.12668.1011941081.divmod.xquotient.13@localhost.localdomain>

On 29 Oct, 11:41 pm, jnoller at gmail.com wrote:
>On Thu, Oct 29, 2009 at 7:04 PM,  <exarkun at twistedmatrix.com> wrote:
>>On 02:30 pm, solipsis at pitrou.net wrote:
>>>
>>>Hello,
>>>
>>>What do you think of creating a "buildbot" category in the tracker? 
>>>There
>>>are
>>>often problems on specific buildbots which would be nice to track, 
>>>but
>>>there's
>>>nowhere to do so.
>>
>>Is your idea that this would be for tracking issues with the *bots*
>>themselves? ?That is, not just for tracking cases where some test 
>>method
>>fails on a particular bot, but for tracking cases where, say, a bot's 
>>host
>>has run out of disk space and cannot run the tests at all?
>>
>>For the case where a test is failing because of some platform or 
>>environment
>>issue, it seems more sensible to track the ticket as relating to that
>>platform or environment, or track it in relation to the feature it 
>>affects.
>>
>>Of course, tickets could move between these classifications as 
>>investigation
>>reveals new information about the problem.
>
>Then again, I know for a fact certain tests fail ONLY on certain
>buildbots because of the way they're configured. For example, certain
>multiprocessing tests will fail if /dev/shm isn't accessible on Linux,
>and several of the buildbosts are in tight chroot jails and don't have
>that exposed.
>
>Is it a bug in that buildbot, a platform specific bug, etc?

It's a platform configuration that can exist.  If you're rejecting that 
configuration and saying that CPython will not support it, then it's 
silly to have a buildbot set up that way, and presumably that should be 
changed.

The point is that this isn't about buildbot.  It's about CPython and 
what platforms it supports.  Categorizing it by "buildbot" is not 
useful, because no one is going to be cruising along looking for 
multiprocessing issues to fix by browsing tickets by the "buildbot" 
category.

If, on the other hand, (sticking with this example) /dev/shm-less 
systems are not a platform that CPython wants to support, then having a 
buildbot running on one is a bit silly.  It will probably always fail, 
and all it does is contribute another column of red.  Who does that 
help?



Jean-Paul

From jnoller at gmail.com  Fri Oct 30 02:58:48 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 29 Oct 2009 21:58:48 -0400
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
Message-ID: <4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com>

On Thu, Oct 29, 2009 at 8:31 PM, R. David Murray <rdmurray at bitdance.com> wrote:

> I'd say that particular one is a bug in the tests. ?If /dev/shm is
> not available and is required, then the tests should be skipped with
> an appropriate message. ?It would also secondarily be an issue with
> the buildbot fleet, since multiprocessing would then not be getting
> thoroughly tested by those buildbots.

Fwiw: The tests skip on those platforms; but multiprocessing can't
function properly on platforms like that.

From steve at pearwood.info  Fri Oct 30 03:58:16 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 30 Oct 2009 13:58:16 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
Message-ID: <200910301358.17752.steve@pearwood.info>

On Wed, 28 Oct 2009 04:47:59 am Raymond Hettinger wrote:

> A dict.get() can be meaningfully used in a loop (because the key can
> vary). A set.get() returns the same value over and over again
> (because there is no key).

I don't believe anyone has requested those semantics. The suggested 
semantics for set.get() with no arguments, as I understand them, are:

(1) it will only fail if the set is empty;

(2) it should be efficient;

(3) if you call it repeatedly on a set without modifying the set, you 
will cycle through each element in turn in some unspecified arbitrary 
order.

To clarify point 3, given:

x = set.get()
y = set.get()

then x and y will only be the same element if set has length one. 
However, given:

x = set.get()
set.add(el)
set.remove(el)
y = set.get()

there are no guarantees about x and y being different.

I believe that the patch supplied by Willi Richart implemented these 
behaviours.

http://bugs.python.org/issue7212



-- 
Steven D'Aprano

From python at rcn.com  Fri Oct 30 05:00:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 29 Oct 2009 21:00:27 -0700
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
References: <200910231132.45504.w.richert@gmx.net><7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com><B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<200910301358.17752.steve@pearwood.info>
Message-ID: <2585AF2DFDDE4C61B3596633F5001945@RaymondLaptop1>


[Steven D'Aprano]
> The suggested
> semantics for set.get() with no arguments, as I understand them, are:
>
>(1) it will only fail if the set is empty;

Just like next() except that next() gives you the option to supply a default
and can be used on any iterator (perhaps iter(s) or itertools.cycle(s) etc).


> (2) it should be efficient;

Is this about optimization?

I wouldn't expect "x=s.get()" to beat "for x in s: break".
Attribute lookup and method calls usually are slower
than equivalents using built-in syntax with specific opcodes.


> (3) if you call it repeatedly on a set without modifying the set, you
> will cycle through each element in turn in some unspecified arbitrary
> order.

What's wrong with using next()?  That is what it's for.

What about this proposal is specific to sets, i.e. why don't you want the same thing for lists. tuples, strings, file objects, or 
any other iterable?

Does this proposal pass the test of being self-descriptive?  Can you write a code fragment that exercises the cycling behavior, show 
it to another programmer, and have them correctly deduce what the code does (i.e. that different values are returned, that it fails 
when the set it empty, that it wraps around and never terminates)?  Can they readily differentiate it for dict.get() which has 
decidedly different semantics?



> To clarify point 3, given:
>
> x = set.get()
> y = set.get()
>
> then x and y will only be the same element if set has length one.

So, it can't even be used for looping through a set because there is no termination?



> I believe that the patch supplied by Willi Richart implemented these
> behaviours.
>
> http://bugs.python.org/issue7212

So you want to introduce additional, hidden state to sets? (to make sure that successive invocations return different values)

Do you want a thread local version too? (so that two threads can call gets without stomping on each other's guarantees that 
successive calls will produce distinct elements)

Do you have any real-world use-cases where next(), for-loops, or itertools wouldn't suffice?

Is there a precedent in *any* other language you've ever seen?  (setl has an "arb" function but it makes no promises about returning 
different values on consequetive calls; otherwise, I've never seen an equivalent in any other set implementation).

Do you think the return-different-values-on-successive-calls semantics is self-evident and non-magical as compared to a straight 
for-loop or next(it)?

ISTM, that when streams have non-destructive getters with self-advancing pointers, they also have a seek() function so that it can 
be controlled.  Will this proposal need a seek() method too?

Sorry for so many questions, but I honestly think there are too many unresolved design issues.  We've seen no real-world source code 
that would be improved fwith the proposal.  I think it sounds conceptually tempting and is fun to theorize about, but it actual 
implementation it will make sets more difficult to learn and it would quickly become a piece of rarely used, poorly understood 
cruft.


Raymond


From sumerc at gmail.com  Fri Oct 30 08:07:57 2009
From: sumerc at gmail.com (=?ISO-8859-1?Q?S=FCmer_Cip?=)
Date: Fri, 30 Oct 2009 09:07:57 +0200
Subject: [Python-Dev] yappi - any thoughts?
Message-ID: <65b2175c0910300007i2cf614dq30893434fdd9721a@mail.gmail.com>

Hi there,

yappi(Yet Another Python Profiler) is a multithreaded profiler for 2.x
series(do not know if it will work on 3k, not tested yet). I have done my
best to make it efficient as possible.(details are under the technical
bullet in the website). It is aimed for long-running programs, to
attach/detach profiler and retrieve stats on the fly. The current cProfile
module needs substantial amount of work to accomplish this task and also
have some problems with recursive functions, basically that is why I have
written a profiler from scratch.

yappi is currently(v0.2) modifying the profilefunc variable fo the
ThreadState object to profile an exsiting thread. The newly created threads
are catched via threading.setprofile() call(from VM to our extension). And
for the deleted threads, we rely on the recycling of the ThreadState
objects.(Please see the website for details.)

I have tested it under a game server (100k players per-day) for about a
month for any issues and now releasing it.

Please see:

http://code.google.com/p/yappi/

Any thoughts/comments/ideas, anything?:)

-- 
Sumer Cip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091030/61a07d77/attachment.htm>

From w.richert at gmx.net  Fri Oct 30 09:16:02 2009
From: w.richert at gmx.net (Willi Richert)
Date: Fri, 30 Oct 2009 09:16:02 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200910301358.17752.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<200910301358.17752.steve@pearwood.info>
Message-ID: <200910300916.02924.w.richert@gmx.net>

Am Freitag, 30. Oktober 2009 03:58:16 schrieb Steven D'Aprano:
> To clarify point 3, given:
> 
> x = set.get()
> y = set.get()
> 
> then x and y will only be the same element if set has length one. 
> However, given:
> 
> x = set.get()
> set.add(el)
> set.remove(el)
> y = set.get()
> 
> there are no guarantees about x and y being different.
> 
> I believe that the patch supplied by Willi Richart implemented these 
> behaviours.
> 
> http://bugs.python.org/issue7212
> 


Actually, no. The patch makes no assumption about x and y being different.

It does not try to extend the core functionality of set nor does it need to 
maintain any state that would be necessary for that. 

It is just a more obvious and cleaner way for saying "for x in some_set: 
break". So, as Raymond says, it is the Pythonic version of "arb" in setl.


wr

From martin at v.loewis.de  Fri Oct 30 09:53:19 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Oct 2009 09:53:19 +0100
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com>
References: <loom.20091029T152906-465@post.gmane.org>	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
	<4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com>
Message-ID: <4AEAA97F.7060300@v.loewis.de>

Jesse Noller wrote:
> On Thu, Oct 29, 2009 at 8:31 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> 
>> I'd say that particular one is a bug in the tests.  If /dev/shm is
>> not available and is required, then the tests should be skipped with
>> an appropriate message.  It would also secondarily be an issue with
>> the buildbot fleet, since multiprocessing would then not be getting
>> thoroughly tested by those buildbots.
> 
> Fwiw: The tests skip on those platforms; but multiprocessing can't
> function properly on platforms like that.

I'm confused: first you said they fail, now you say they get skipped.
Which one is it? I agree with R. David's analysis: if they fail, it's
a multiprocessing bug, if they get skipped, it's a flaw in the build
slave configuration (but perhaps only slightly so, because it is good
if both cases are tested - and we do have machines also that provide
/dev/shm).

Regards,
Martin

From martin at v.loewis.de  Fri Oct 30 09:57:47 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Oct 2009 09:57:47 +0100
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
References: <loom.20091029T152906-465@post.gmane.org>	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
Message-ID: <4AEAAA8B.3000007@v.loewis.de>

> But the real reason for having a buildbot category (or at least a keyword)
> would be to be able to tag all bugs that are currently making buildbots
> fail that are _not_ the result of a recent checkin.  This would make
> the task of finding the bugs that need to be cleaned up to stabilize
> the buildbot fleet easier.  I'm currently aware of issues 4970, 3892,
> and 6462 in this category, and there are a few more that we can/will file
> if we continue to pay attention to the failure reports now arriving on
> the irc channel.

That's convincing; I've created a "buildbot" keyword. I gave it the
description

  "indicates that tests fail on a buildslave for uncertain reasons"

If that is indeed the intended purpose of this classification, please
keep it in mind when assigning the tag. If I misunderstood the purpose
of the keyword, please provide a better description.

Regards,
Martin

From martin at v.loewis.de  Fri Oct 30 10:00:40 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Oct 2009 10:00:40 +0100
Subject: [Python-Dev] ssl module
In-Reply-To: <hccn3u$3eq$1@ger.gmane.org>
References: <hccn3u$3eq$1@ger.gmane.org>
Message-ID: <4AEAAB38.6090609@v.loewis.de>

> Is there a place where the status of the ssl module is summarized

The documentation of the ssl module should describe its features
correctly and precisely.

> or a better place to discuss this? I could try to provide contributions or
> further details if appropriate.

For contributions, this is indeed the right place for discussion.
For mere inquiries for help, other places are better (such as
python-list). For specific feature requests, use the bug tracker.

Regards,
Martin

From martin at v.loewis.de  Fri Oct 30 10:02:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Oct 2009 10:02:12 +0100
Subject: [Python-Dev] yappi - any thoughts?
In-Reply-To: <65b2175c0910300007i2cf614dq30893434fdd9721a@mail.gmail.com>
References: <65b2175c0910300007i2cf614dq30893434fdd9721a@mail.gmail.com>
Message-ID: <4AEAAB94.5060006@v.loewis.de>

> Any thoughts/comments/ideas, anything?:)

Announce it to comp.lang.python.announce if you haven't done so, yet.

Regards,
Martin

From jnoller at gmail.com  Fri Oct 30 13:55:08 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 30 Oct 2009 08:55:08 -0400
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4AEAA97F.7060300@v.loewis.de>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
	<4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com>
	<4AEAA97F.7060300@v.loewis.de>
Message-ID: <4222a8490910300555g5d2ba753o7237171c3af1aad9@mail.gmail.com>

On Fri, Oct 30, 2009 at 4:53 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> I'm confused: first you said they fail, now you say they get skipped.
> Which one is it? I agree with R. David's analysis: if they fail, it's
> a multiprocessing bug, if they get skipped, it's a flaw in the build
> slave configuration (but perhaps only slightly so, because it is good
> if both cases are tested - and we do have machines also that provide
> /dev/shm).

They failed until we had the tests skip those platforms - at the time,
I felt that it was more of a bug with the build slave configuration
than a multiprocessing issue, I don't like skipping tests unless the
platform fundamentally isn't supported (e.g. broken semaphores for
some actions on OS/X) - linux platforms support this functionality
just fine - except when in locked-down chroot jails.

The only reason I brought it up was to point out the a buildbot
configuration on a given host can make tests fail even if those tests
would normally pass on that operating system.

jesse

From exarkun at twistedmatrix.com  Fri Oct 30 15:15:35 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 30 Oct 2009 14:15:35 -0000
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4222a8490910300555g5d2ba753o7237171c3af1aad9@mail.gmail.com>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
	<4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com>
	<4AEAA97F.7060300@v.loewis.de>
	<4222a8490910300555g5d2ba753o7237171c3af1aad9@mail.gmail.com>
Message-ID: <20091030141535.12668.1812365877.divmod.xquotient.85@localhost.localdomain>

On 12:55 pm, jnoller at gmail.com wrote:
>On Fri, Oct 30, 2009 at 4:53 AM, "Martin v. L?wis" <martin at v.loewis.de> 
>wrote:
>>I'm confused: first you said they fail, now you say they get skipped.
>>Which one is it? I agree with R. David's analysis: if they fail, it's
>>a multiprocessing bug, if they get skipped, it's a flaw in the build
>>slave configuration (but perhaps only slightly so, because it is good
>>if both cases are tested - and we do have machines also that provide
>>/dev/shm).
>
>They failed until we had the tests skip those platforms - at the time,
>I felt that it was more of a bug with the build slave configuration
>than a multiprocessing issue, I don't like skipping tests unless the
>platform fundamentally isn't supported (e.g. broken semaphores for
>some actions on OS/X) - linux platforms support this functionality
>just fine - except when in locked-down chroot jails.
>
>The only reason I brought it up was to point out the a buildbot
>configuration on a given host can make tests fail even if those tests
>would normally pass on that operating system.

Just as a build slave can be run in a chroot, so can any other Python 
program.  This is a real shortcoming of the multiprocessing module. 
It's entirely possible that people will want to run Python software in 
chroots sometimes.  So it's proper to acknowledge that this is an 
unsupported environment.  The fact that the kernel in use is the same as 
the kernel in use on another supported platform is sort of irrelevant. 
The kernel is just one piece of the system, there are many other 
important pieces.

Jean-Paul

From rdmurray at bitdance.com  Fri Oct 30 15:35:22 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 30 Oct 2009 10:35:22 -0400 (EDT)
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4222a8490910300555g5d2ba753o7237171c3af1aad9@mail.gmail.com>
References: <loom.20091029T152906-465@post.gmane.org> 
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com> 
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net> 
	<4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com> 
	<4AEAA97F.7060300@v.loewis.de>
	<4222a8490910300555g5d2ba753o7237171c3af1aad9@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0910300952390.21420@kimball.webabinitio.net>

On Fri, 30 Oct 2009 at 08:55, Jesse Noller wrote:
> On Fri, Oct 30, 2009 at 4:53 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I'm confused: first you said they fail, now you say they get skipped.
>> Which one is it? I agree with R. David's analysis: if they fail, it's
>> a multiprocessing bug, if they get skipped, it's a flaw in the build
>> slave configuration (but perhaps only slightly so, because it is good
>> if both cases are tested - and we do have machines also that provide
>> /dev/shm).
>
> They failed until we had the tests skip those platforms - at the time,
> I felt that it was more of a bug with the build slave configuration
> than a multiprocessing issue, I don't like skipping tests unless the
> platform fundamentally isn't supported (e.g. broken semaphores for
> some actions on OS/X) - linux platforms support this functionality
> just fine - except when in locked-down chroot jails.

As Martin pointed out, Python supports both configurations (chroot and
non-chroot), and needs to be tested in both.  Somewhere we should probably
have a list of what tests are getting skipped on what buildslaves so we
can inspect the buildbot fleet for complete coverage, but I'm not sure
who is going to volunteer to create and maintain that list :)

> The only reason I brought it up was to point out the a buildbot
> configuration on a given host can make tests fail even if those tests
> would normally pass on that operating system.

Yes, and that's a kind of ticket that should end up getting tagged
with the new 'buildbot' keyword (thanks, Martin), IMO.

--David

From jnoller at gmail.com  Fri Oct 30 15:44:59 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 30 Oct 2009 10:44:59 -0400
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <20091030141535.12668.1812365877.divmod.xquotient.85@localhost.localdomain>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
	<4222a8490910291858sc1caa57k7f46e03ab56262ca@mail.gmail.com>
	<4AEAA97F.7060300@v.loewis.de>
	<4222a8490910300555g5d2ba753o7237171c3af1aad9@mail.gmail.com>
	<20091030141535.12668.1812365877.divmod.xquotient.85@localhost.localdomain>
Message-ID: <4222a8490910300744q61414bc9m1a82047c89e741fe@mail.gmail.com>

On Fri, Oct 30, 2009 at 10:15 AM,  <exarkun at twistedmatrix.com> wrote:
> On 12:55 pm, jnoller at gmail.com wrote:
>>
>> On Fri, Oct 30, 2009 at 4:53 AM, "Martin v. L?wis" <martin at v.loewis.de>
>> wrote:
>>>
>>> I'm confused: first you said they fail, now you say they get skipped.
>>> Which one is it? I agree with R. David's analysis: if they fail, it's
>>> a multiprocessing bug, if they get skipped, it's a flaw in the build
>>> slave configuration (but perhaps only slightly so, because it is good
>>> if both cases are tested - and we do have machines also that provide
>>> /dev/shm).
>>
>> They failed until we had the tests skip those platforms - at the time,
>> I felt that it was more of a bug with the build slave configuration
>> than a multiprocessing issue, I don't like skipping tests unless the
>> platform fundamentally isn't supported (e.g. broken semaphores for
>> some actions on OS/X) - linux platforms support this functionality
>> just fine - except when in locked-down chroot jails.
>>
>> The only reason I brought it up was to point out the a buildbot
>> configuration on a given host can make tests fail even if those tests
>> would normally pass on that operating system.
>
> Just as a build slave can be run in a chroot, so can any other Python
> program. ?This is a real shortcoming of the multiprocessing module. It's
> entirely possible that people will want to run Python software in chroots
> sometimes. ?So it's proper to acknowledge that this is an unsupported
> environment. ?The fact that the kernel in use is the same as the kernel in
> use on another supported platform is sort of irrelevant. The kernel is just
> one piece of the system, there are many other important pieces.
>
> Jean-Paul

I'm well aware of that.

From steve at pearwood.info  Fri Oct 30 16:11:32 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 31 Oct 2009 02:11:32 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <2585AF2DFDDE4C61B3596633F5001945@RaymondLaptop1>
References: <200910231132.45504.w.richert@gmx.net>
	<200910301358.17752.steve@pearwood.info>
	<2585AF2DFDDE4C61B3596633F5001945@RaymondLaptop1>
Message-ID: <200910310211.33590.steve@pearwood.info>

On Fri, 30 Oct 2009 03:00:27 pm Raymond Hettinger wrote:

[skipping to the last paragraph]

> Sorry for so many questions

Don't be sorry. These are good questions, and I'll try to answer them. 
But keep in mind that this isn't my proposal -- I vary between +0 and 
+1 on the proposal depending on the time of the day *wink*


> >(1) it will only fail if the set is empty;
>
> Just like next() except that next() gives you the option to supply a
> default and can be used on any iterator (perhaps iter(s) or
> itertools.cycle(s) etc).

Yes. I had considered suggesting that sets become their own iterator, so 
you could do the following:

>>> s = set([1,2])
>>> next(s)
2

but that leads to the problem that if you accept a set from somewhere 
else, you have no idea whether next(s) will succeed or raise 
StopIteration. It may have already been exhausted before it reached 
you.


> > (2) it should be efficient;
>
> Is this about optimization?

Not primarily. Perhaps I should have said it should not be inefficient. 
It's primarily about there being One Obvious Way to extract an 
arbitrary item from a set -- this is a reoccurring question on 
comp.lang.python. Being efficient is a bonus, but it shouldn't be 
inefficient.


> I wouldn't expect "x=s.get()" to beat "for x in s: break".
> Attribute lookup and method calls usually are slower
> than equivalents using built-in syntax with specific opcodes.

Obviously any hypothetical solution would need to be benchmarked, but I 
wouldn't be concerned if s.get() was marginally slower than for `x in 
s: break`. 

When people are left to come up with their own ad hoc solutions, we've 
seen some poor solutions. I've seen, possibly in this very thread, the 
following O(N) suggestions:

for x in s:
    pass

x = list(s)[0]

The usual technique people tend to come up with is:

x = s.pop()
s.add(x)

Which strikes many people (including myself) as inelegant. Surely "get 
an element" is a more fundamental operation than "get an element and 
remove it"?


> > (3) if you call it repeatedly on a set without modifying the set,
> > you will cycle through each element in turn in some unspecified
> > arbitrary order.
>
> What's wrong with using next()?  That is what it's for.

You can't call next() on a set. You have to call next(iter(set)). From 
Python 3.0:

>>> next(set([1,2]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: set object is not an iterator
>>> next(iter(set([1,2])))
1

Unless I missed something, this is so unobvious that nobody has 
suggested it in the thread yet.


> What about this proposal is specific to sets, i.e. why don't you want
> the same thing for lists. tuples, strings, file objects, or any other
> iterable?

Sequences such as lists, tuples and strings have easy random access. 
It's easy to get an arbitrary element: pick an index i by whatever 
method you like, and get seq[i]. Many file objects are similar, you 
have random access with seek().

It is unpractical and an over-generalisation to apply this to general 
iterables, for reasons I'm sure I don't need to go into. But sets and 
frozensets aren't lazily generated streams, they actually do store the 
elements inside their data structure in the same way that lists do.


> Does this proposal pass the test of being self-descriptive?  Can you
> write a code fragment that exercises the cycling behavior, show it to
> another programmer, and have them correctly deduce what the code does
> (i.e. that different values are returned, that it fails when the set
> it empty, that it wraps around and never terminates)?  Can they
> readily differentiate it for dict.get() which has decidedly different
> semantics?

I don't expect ConfigParser.get() to have the same semantics as 
dict.get(), and they're much more closely related than sets and dicts. 
I don't understand why so many people apparently have a problem with 
the name get(), but that's okay, I'm not wedded to the idea either. If 
folks prefer a different name, by all means suggest it. I like the name 
suggested by Wikipedia, "pick".

As for being self-descriptive, I don't know, I haven't tried the 
experiment.


> > To clarify point 3, given:
> >
> > x = set.get()
> > y = set.get()
> >
> > then x and y will only be the same element if set has length one.
>
> So, it can't even be used for looping through a set because there is
> no termination?

That's a feature, not a bug! This is not intended to be a replacement 
for standard set iteration. Anyone writing the following:

for _ in len(s):
    process(s.get())

instead of 

for x in s:
    process(x)

will be taken out and slapped with a large fish.

My reasoning for the given behaviour is as follows:

* If you want the same element twice from a set, the One Obvious Way is 
to get an element from the set (somehow!) and assign it to a local 
variable:

    x = s.get()
    process(x)
    # later...
    process(x)

So having get() return the same value each time is not useful or 
necessary.

* If you want a random element on each call, the One Obvious Way is to 
call random.choice(list(s)).

* Since sets aren't mappings, or sequences, there's no sensible way to 
look up an index or key, so any variation of "get element #1" are out.

* Which leaves returning the elements in some unspecified arbitrary 
order. The most obvious arbitrary order is to cycle through them, which 
conveniently is exactly what iter(set) does, but we don't need to 
guarantee that order.


> > I believe that the patch supplied by Willi Richart implemented
> > these behaviours.
> >
> > http://bugs.python.org/issue7212
>
> So you want to introduce additional, hidden state to sets? (to make
> sure that successive invocations return different values)

If you can think of any other way to efficiently cycle over the elements 
in a set, I'm all for it :)

Presumably this is no different from what dict views do, except they 
don't wrap around when exhausted.


> Do you want a thread local version too? (so that two threads can call
> gets without stomping on each other's guarantees that successive
> calls will produce distinct elements)

I think that's overkill. I wouldn't think we need to guarantee that two 
threads don't see the same element, or that each thread will see each 
element in the same order. We need only the much weaker guarantee that 
two subsequent calls to get() in the one thread with no modification to 
the set between the calls won't retrieve the same element each time 
(unless there is only one element to retrieve), and that each element 
will eventually be seen.


> Do you have any real-world use-cases where next(), for-loops, or
> itertools wouldn't suffice?

Someone in this thread -- forgive me, I don't recall who -- had the case 
where they had a frozenset which they knew only had one element, but no 
obvious way to retrieve that element.

To my mind, the major rationalisation for set.get() is not that there's 
no way to get a single element from a set, but that there is no 
obvious, easily discoverable way to retrieve a single element.


> Is there a precedent in *any* other language you've ever seen?  (setl
> has an "arb" function but it makes no promises about returning
> different values on consequetive calls; otherwise, I've never seen an
> equivalent in any other set implementation).

I can't say I've seen one in any other languages, but Wikipedia 
lists "pick" as a fundamental set operation:

pick(S): returns an arbitrary element of S.

http://en.wikipedia.org/wiki/Set_(computer_science)



This page claims that Icon has an operator that returns a random element 
of a set:

? set( [1, 2, 3, 4, 5] )

http://stackoverflow.com/questions/124671/picking-a-random-element-from-a-set

but I've never used Icon myself and so can't confirm that.



> Do you think the return-different-values-on-successive-calls
> semantics is self-evident and non-magical as compared to a straight
> for-loop or next(it)?

I'm going to sit on the fence on that and say "maybe".


> ISTM, that when streams have non-destructive getters with
> self-advancing pointers, they also have a seek() function so that it
> can be controlled.  Will this proposal need a seek() method too?

No. Since sets are unordered, there's no way to seek to a specific 
element.


> Sorry for so many questions, but I honestly think there are too many
> unresolved design issues.  We've seen no real-world source code that
> would be improved fwith the proposal.  I think it sounds conceptually
> tempting and is fun to theorize about, but it actual implementation
> it will make sets more difficult to learn and it would quickly become
> a piece of rarely used, poorly understood cruft.



-- 
Steven D'Aprano

From solipsis at pitrou.net  Fri Oct 30 16:35:33 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 30 Oct 2009 15:35:33 +0000 (UTC)
Subject: [Python-Dev] Retrieve an arbitrary element from a set without
	removing it
References: <200910231132.45504.w.richert@gmx.net>
	<200910301358.17752.steve@pearwood.info>
	<2585AF2DFDDE4C61B3596633F5001945@RaymondLaptop1>
	<200910310211.33590.steve@pearwood.info>
Message-ID: <loom.20091030T161729-797@post.gmane.org>

Steven D'Aprano <steve <at> pearwood.info> writes:
> 
> If you can think of any other way to efficiently cycle over the elements 
> in a set, I'm all for it :)

How about "for x in s"?

Or if you want to cycle:

>>> s = set('abc')
>>> it = itertools.cycle(s)
>>> next(it)
'a'
>>> next(it)
'c'
>>> next(it)
'b'
>>> next(it)
'a'

Or if you don't want the overhead of itertools.cycle() keeping a copy of the
set's elements:

>>> s = set('abc')
>>> it = itertools.chain.from_iterable(itertools.cycle([s]))
>>> next(it)
'a'
>>> next(it)
'c'
>>> next(it)
'b'
>>> next(it)
'a'
>>> next(it)
'c'
>>> next(it)
'b'

> I can't say I've seen one in any other languages, but Wikipedia 
> lists "pick" as a fundamental set operation:
> 
> pick(S): returns an arbitrary element of S.

Well, it's an arbitrary element. It isn't specified that it will try to return
different results in a row to satisfy the developer's aesthetical preferences...

> This page claims that Icon has an operator that returns a random element 
> of a set:
> 
> ? set( [1, 2, 3, 4, 5] )

random != arbitrary != weak-guaranteedly distinct



From ncoghlan at gmail.com  Fri Oct 30 16:46:01 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 31 Oct 2009 01:46:01 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from
 a	setwithoutremoving it
In-Reply-To: <200910310211.33590.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>	<200910301358.17752.steve@pearwood.info>	<2585AF2DFDDE4C61B3596633F5001945@RaymondLaptop1>
	<200910310211.33590.steve@pearwood.info>
Message-ID: <4AEB0A39.2070707@gmail.com>

Steven D'Aprano wrote:
>> So you want to introduce additional, hidden state to sets? (to make
>> sure that successive invocations return different values)
> 
> If you can think of any other way to efficiently cycle over the elements 
> in a set, I'm all for it :)

for x in itertools.cycle(s):
  # this is an infinite loop

Having a pick() or get() method that returns an arbitrary member of a
set makes sense to me. Having any state on the set that guarantees
successive calls to get will return different values feels wrong -
creating an object with that extra state is what iter(s) is for.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Fri Oct 30 17:21:06 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 30 Oct 2009 16:21:06 +0000 (UTC)
Subject: [Python-Dev] Possible language summit topic: buildbots
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
Message-ID: <hcf3pi$vgs$1@ger.gmane.org>


Hello,

Sorry for the little redundancy, I would like to underline Jean-Paul's
suggestion here:

Le Sun, 25 Oct 2009 14:05:12 +0000, exarkun a ?crit?:
> I think that money can help in two ways in this case.
> 
> First, there are now a multitude of cloud hosting providers which will 
> operate a slave machine for you.  BuildBot has even begun to support 
> this deployment use-case by allowing you to start up and shut down vms 
> on demand to save on costs.  Amazon's EC2 service is supported out of 
> the box in the latest release.

I'm not a PSF member, but it seems to me that the PSF could ask Amazon 
(or any other virtual machine business anyway) to donate a small number
of permanent EC2 instances in order to run buildslaves on. After all, big
companies often like sponsoring open-source projects, especially when the
project is well-known and the donation is cheap for them.

This would have several advantages:
- the machines are administered by the provider: we don't have to worry 
about failed hardware, connectivity loss etc.
- any Python core developer could get ssh access to the VMs to run tests
directly, since they would be dedicated buildbot instances
- they are not tied to a particular owner when it comes to fixing system
problems, which means we eliminate a single point of failure: if a 
volunteer gets demotivated/bored/missing in action, someone can replace 
him/her easily
- there are a number of various OS images available (of course, we still 
need competent people to install the required software -- buildbot, etc.)

Since I've never used any such service ("cloud"-based VMs), I'm not sure 
what the downsides would be. But it seems to be that it would be at least 
worth trying. Right now we have around 15 buildbots but two thirds of 
them are down, the others sometimes fail or disconnect in erratic ways 
and it's difficult for "regular" core developers to be aware of what's 
precisely going on.

Of course this could also be a broken idea, for whatever reason I'm not 
aware of.

Regards

Antoine.



From ctb at msu.edu  Fri Oct 30 17:31:28 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 30 Oct 2009 09:31:28 -0700
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <hcf3pi$vgs$1@ger.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org>
Message-ID: <20091030163128.GA30191@idyll.org>

On Fri, Oct 30, 2009 at 04:21:06PM +0000, Antoine Pitrou wrote:
> 
> Hello,
> 
> Sorry for the little redundancy, I would like to underline Jean-Paul's
> suggestion here:
> 
> Le Sun, 25 Oct 2009 14:05:12 +0000, exarkun a ??crit??:
> > I think that money can help in two ways in this case.
> > 
> > First, there are now a multitude of cloud hosting providers which will 
> > operate a slave machine for you.  BuildBot has even begun to support 
> > this deployment use-case by allowing you to start up and shut down vms 
> > on demand to save on costs.  Amazon's EC2 service is supported out of 
> > the box in the latest release.
> 
> I'm not a PSF member, but it seems to me that the PSF could ask Amazon 
> (or any other virtual machine business anyway) to donate a small number
> of permanent EC2 instances in order to run buildslaves on.

[ ... ]

I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
ia64; Linux x86; and Mac OS X.  Others have made similar offers.  The
architectures supported by the cloud services don't really add anything
(and generally don't have Mac OS X support, AFAIK).

What we really need (IMO) is someone to dig into the tests to figure out which
tests fail randomly and why, and to fix them on specific architectures that
most of us don't personally use.  This is hard work that is neither glamorous
nor popular.

I think the idea of paying a dedicated developer to make the CPython+buildbot
tests reliable is better, although I would still be -0 on it (I don't think
the PSF should be paying for this kind of thing at all).

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From solipsis at pitrou.net  Fri Oct 30 17:41:39 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 30 Oct 2009 17:41:39 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091030163128.GA30191@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org>  <20091030163128.GA30191@idyll.org>
Message-ID: <1256920899.5472.5.camel@localhost>

Le vendredi 30 octobre 2009 ? 09:31 -0700, C. Titus Brown a ?crit :
> [ ... ]
> 
> I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
> ia64; Linux x86; and Mac OS X.  Others have made similar offers.  The
> architectures supported by the cloud services don't really add anything
> (and generally don't have Mac OS X support, AFAIK).

Well these VMs would have to run buildslaves on them, then. Are you
ready to host some and connect them to the current buildbot
infrastructure?
(VMs without buildslaves are less interesting IMO)

> What we really need (IMO) is someone to dig into the tests to figure out which
> tests fail randomly and why, and to fix them on specific architectures that
> most of us don't personally use.  This is hard work that is neither glamorous
> nor popular.

I'm sure some of us are ready to do so (*). The situation has already
improved quite a lot in the recent times. But fixing platform- or,
worse, setup-specific issues often requires shell access to the target
system, otherwise you spend too much time trying fixes on the SVN and
waiting for the buildbot to react.

(*) After all, if we weren't, we wouldn't even care about buildbots,
we'd be content with running the test suite on our own machines

> I think the idea of paying a dedicated developer to make the CPython+buildbot
> tests reliable is better, although I would still be -0 on it (I don't think
> the PSF should be paying for this kind of thing at all).

Paying developers in volunteer communities is always more contentious
than paying for other kinds of resources.
(It's generally more expensive too)




From olemis at gmail.com  Fri Oct 30 17:42:30 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 30 Oct 2009 11:42:30 -0500
Subject: [Python-Dev] [TIP] Possible language summit topic: buildbots
In-Reply-To: <20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
	<20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
Message-ID: <24ea26600910300942j3790e8a0m83c011ac17e57bd9@mail.gmail.com>

On Sun, Oct 25, 2009 at 9:13 AM,  <exarkun at twistedmatrix.com> wrote:
> On 12:48 pm, ctb at msu.edu wrote:
>>
>> [snip]
>>
>> The most *exciting* part of pony-build, apart from the always-riveting
>> spectacle of "titus rediscovering problems that buildbot solved 5 years
>> ago",
>> is the loose coupling of recording server to the build slaves and build
>> reporters. ?My plan is to enable a simple and lightweight XML-RPC and/or
>> REST-ish interface for querying the recording server from scripts or other
>> Web
>> sites. ?This has Brett aquiver with anticipation, I gather -- no more
>> visual
>> inspection of buildbot waterfall pages ;)
>
> BuildBot has an XML-RPC interface. ?So Brett can probably do what he wants
> with BuildBot right now.
>

... but pony-build follows a different model

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From ctb at msu.edu  Fri Oct 30 17:43:31 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 30 Oct 2009 09:43:31 -0700
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <1256920899.5472.5.camel@localhost>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
	<1256920899.5472.5.camel@localhost>
Message-ID: <20091030164331.GA19953@idyll.org>

On Fri, Oct 30, 2009 at 05:41:39PM +0100, Antoine Pitrou wrote:
> Le vendredi 30 octobre 2009 ?? 09:31 -0700, C. Titus Brown a ??crit :
> > [ ... ]
> > 
> > I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
> > ia64; Linux x86; and Mac OS X.  Others have made similar offers.  The
> > architectures supported by the cloud services don't really add anything
> > (and generally don't have Mac OS X support, AFAIK).
> 
> Well these VMs would have to run buildslaves on them, then. Are you
> ready to host some and connect them to the current buildbot
> infrastructure?
> (VMs without buildslaves are less interesting IMO)

No, I'm not willing to spend the time to install and maintain buildbot. But
I'm happy to give the necessary access to those who are interested and
willing.

(...and let me tell you, getting these !#%!#$! Windows VMs up and running
already took an immense amount of effort ;)

> > What we really need (IMO) is someone to dig into the tests to figure out which
> > tests fail randomly and why, and to fix them on specific architectures that
> > most of us don't personally use.  This is hard work that is neither glamorous
> > nor popular.
> 
> I'm sure some of us are ready to do so (*). The situation has already
> improved quite a lot in the recent times. But fixing platform- or,
> worse, setup-specific issues often requires shell access to the target
> system, otherwise you spend too much time trying fixes on the SVN and
> waiting for the buildbot to react.
> 
> (*) After all, if we weren't, we wouldn't even care about buildbots,
> we'd be content with running the test suite on our own machines

I look forward to it!

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From ctb at msu.edu  Fri Oct 30 17:45:49 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 30 Oct 2009 09:45:49 -0700
Subject: [Python-Dev] [TIP] Possible language summit topic: buildbots
In-Reply-To: <24ea26600910300942j3790e8a0m83c011ac17e57bd9@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
	<20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
	<24ea26600910300942j3790e8a0m83c011ac17e57bd9@mail.gmail.com>
Message-ID: <20091030164549.GC19953@idyll.org>

On Fri, Oct 30, 2009 at 11:42:30AM -0500, Olemis Lang wrote:
> On Sun, Oct 25, 2009 at 9:13 AM,  <exarkun at twistedmatrix.com> wrote:
> > On 12:48 pm, ctb at msu.edu wrote:
> >>
> >> [snip]
> >>
> >> The most *exciting* part of pony-build, apart from the always-riveting
> >> spectacle of "titus rediscovering problems that buildbot solved 5 years
> >> ago",
> >> is the loose coupling of recording server to the build slaves and build
> >> reporters. ?My plan is to enable a simple and lightweight XML-RPC and/or
> >> REST-ish interface for querying the recording server from scripts or other
> >> Web
> >> sites. ?This has Brett aquiver with anticipation, I gather -- no more
> >> visual
> >> inspection of buildbot waterfall pages ;)
> >
> > BuildBot has an XML-RPC interface. ?So Brett can probably do what he wants
> > with BuildBot right now.
> 
> ... but pony-build follows a different model

I'd rather not get into discussions of why my vaporware is going to be
way, way better than anything anyone else could possibly do -- that's
flamebait and not very friendly, in the end.  Let's just say that I'm wasting
my own time on it to scratch my own itch and leave it at that!

thanks,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From p.f.moore at gmail.com  Fri Oct 30 17:49:51 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 30 Oct 2009 16:49:51 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091030163128.GA30191@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
Message-ID: <79990c6b0910300949ybc11b29q11900f08a79cdb9e@mail.gmail.com>

2009/10/30 C. Titus Brown <ctb at msu.edu>:
> On Fri, Oct 30, 2009 at 04:21:06PM +0000, Antoine Pitrou wrote:
>>
>> Hello,
>>
>> Sorry for the little redundancy, I would like to underline Jean-Paul's
>> suggestion here:
>>
>> Le Sun, 25 Oct 2009 14:05:12 +0000, exarkun a ??crit??:
>> > I think that money can help in two ways in this case.
>> >
>> > First, there are now a multitude of cloud hosting providers which will
>> > operate a slave machine for you. ?BuildBot has even begun to support
>> > this deployment use-case by allowing you to start up and shut down vms
>> > on demand to save on costs. ?Amazon's EC2 service is supported out of
>> > the box in the latest release.
>>
>> I'm not a PSF member, but it seems to me that the PSF could ask Amazon
>> (or any other virtual machine business anyway) to donate a small number
>> of permanent EC2 instances in order to run buildslaves on.
>
> [ ... ]
>
> I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
> ia64; Linux x86; and Mac OS X. ?Others have made similar offers. ?The
> architectures supported by the cloud services don't really add anything
> (and generally don't have Mac OS X support, AFAIK).

As Antione pointed out, it's not clear (at least, it isn't to me) what
that leaves to be done.

As a counter-offer: Given remote access to however many Windows VMs
you want to provide, I'll get them up and running with buildslaves on
them. If that requires software such as Visual Studio, I have copies
via the MSDN licenses that I am happy to provide.

Once things are up and running, I'll be prepared to do basic care and
feeding of the buildslave, but as my time is limited, it would be nice
if others would pitch in to help.

In other words, if it's setup effort that's lacking, I'll provide it.
As long as someone else can cover systems admin, and we get some level
of volunteers to cover ongoing support, that should give us better
Windows coverage on the buildbots.

Paul.

From ctb at msu.edu  Fri Oct 30 17:56:15 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 30 Oct 2009 09:56:15 -0700
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <79990c6b0910300949ybc11b29q11900f08a79cdb9e@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
	<79990c6b0910300949ybc11b29q11900f08a79cdb9e@mail.gmail.com>
Message-ID: <20091030165615.GB5132@idyll.org>

On Fri, Oct 30, 2009 at 04:49:51PM +0000, Paul Moore wrote:
> 2009/10/30 C. Titus Brown <ctb at msu.edu>:
> > On Fri, Oct 30, 2009 at 04:21:06PM +0000, Antoine Pitrou wrote:
> >>
> >> Hello,
> >>
> >> Sorry for the little redundancy, I would like to underline Jean-Paul's
> >> suggestion here:
> >>
> >> Le Sun, 25 Oct 2009 14:05:12 +0000, exarkun a ??crit??:
> >> > I think that money can help in two ways in this case.
> >> >
> >> > First, there are now a multitude of cloud hosting providers which will
> >> > operate a slave machine for you. ?BuildBot has even begun to support
> >> > this deployment use-case by allowing you to start up and shut down vms
> >> > on demand to save on costs. ?Amazon's EC2 service is supported out of
> >> > the box in the latest release.
> >>
> >> I'm not a PSF member, but it seems to me that the PSF could ask Amazon
> >> (or any other virtual machine business anyway) to donate a small number
> >> of permanent EC2 instances in order to run buildslaves on.
> >
> > [ ... ]
> >
> > I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
> > ia64; Linux x86; and Mac OS X. ?Others have made similar offers. ?The
> > architectures supported by the cloud services don't really add anything
> > (and generally don't have Mac OS X support, AFAIK).
> 
> As Antione pointed out, it's not clear (at least, it isn't to me) what
> that leaves to be done.

Great!  We've solved the problem ;)

> As a counter-offer: Given remote access to however many Windows VMs
> you want to provide, I'll get them up and running with buildslaves on
> them. If that requires software such as Visual Studio, I have copies
> via the MSDN licenses that I am happy to provide.

I, too, have MSDN licenses, and I have functioning build environments
on all of the VMs (I think -- I've only tested Win XP currently:

http://lyorn.idyll.org/ctb/pb-dev/python/detail?result_key=8276

)

I also have an OS X 10.5 machine that I can let you into through a firewall;
it's building Python 2.7 quite nicely:

http://lyorn.idyll.org/ctb/pb-dev/python/detail?result_key=8229

> Once things are up and running, I'll be prepared to do basic care and
> feeding of the buildslave, but as my time is limited, it would be nice
> if others would pitch in to help.

I would be somewhat unhappy about giving more than three or four people
admin access, but am prepared to lie back and think of England.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From exarkun at twistedmatrix.com  Fri Oct 30 17:58:29 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 30 Oct 2009 16:58:29 -0000
Subject: [Python-Dev] Cloud build slaves (was Re: Possible language
	summit	topic: buildbots)
In-Reply-To: <20091030163128.GA30191@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
Message-ID: <20091030165829.12668.1546393452.divmod.xquotient.118@localhost.localdomain>

On 04:31 pm, ctb at msu.edu wrote:
>On Fri, Oct 30, 2009 at 04:21:06PM +0000, Antoine Pitrou wrote:
>>
>>Hello,
>>
>>Sorry for the little redundancy, I would like to underline Jean-Paul's
>>suggestion here:
>>
>>Le Sun, 25 Oct 2009 14:05:12 +0000, exarkun a ??crit??:
>> > I think that money can help in two ways in this case.
>> >
>> > First, there are now a multitude of cloud hosting providers which 
>>will
>> > operate a slave machine for you.  BuildBot has even begun to support
>> > this deployment use-case by allowing you to start up and shut down 
>>vms
>> > on demand to save on costs.  Amazon's EC2 service is supported out 
>>of
>> > the box in the latest release.
>>
>>I'm not a PSF member, but it seems to me that the PSF could ask Amazon
>>(or any other virtual machine business anyway) to donate a small 
>>number
>>of permanent EC2 instances in order to run buildslaves on.
>
>[ ... ]
>
>I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); 
>Linux
>ia64; Linux x86; and Mac OS X.

Okay, let's move on this.  Martin has, I believe, said that potential 
slave operators only need to contact him to get credentials for new 
slaves.  Can you make sure to follow up with him to get slaves running 
on these machines?  Or would you rather give out access to someone else 
and have them do the build slave setup?
>Others have made similar offers.

I'll similarly encourage them to take action, then.  Do you happen to 
remember who?
>The
>architectures supported by the cloud services don't really add anything
>(and generally don't have Mac OS X support, AFAIK).

That's not entirely accurate.  Currently, CPython has slaves on these 
platforms:

  - x86
    - FreeBSD
    - Windows XP
    - Gentoo Linux
    - OS X
  - ia64
    - Ubuntu Linux
  - Alpha
    - Debian Linux

So, assuming we don't want to introduce any new OS, Amazon could fill in 
the following holes:

  - x86
    - Ubuntu Linux
  - ia64
    - FreeBSD
    - Windows XP
    - Gentoo Linux

So very modestly, that's 4 currently missing slaves which Amazon's cloud 
service *does* add.  It's easy to imagine further additions it could 
make as well.
>What we really need (IMO) is someone to dig into the tests to figure 
>out which
>tests fail randomly and why, and to fix them on specific architectures 
>that
>most of us don't personally use.  This is hard work that is neither 
>glamorous
>nor popular.

Sure.  That's certainly necessary.  I don't think anyone is suggesting 
that it's not.  Fortunately, adding more build slaves is not mutually 
exclusive with a developer fixing bugs in CPython.
>I think the idea of paying a dedicated developer to make the 
>CPython+buildbot
>tests reliable is better, although I would still be -0 on it (I don't 
>think
>the PSF should be paying for this kind of thing at all).

I hope everyone is on board with the idea of fixing bugs in CPython, 
either in the actual implementation of features or in the tests for 
those features.  That being the case, the discussion of whether or not 
the PSF should try to fund such a task is perhaps best discussed on the 
PSF members list.

Jean-Paul

From exarkun at twistedmatrix.com  Fri Oct 30 18:01:29 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 30 Oct 2009 17:01:29 -0000
Subject: [Python-Dev] [TIP] Possible language summit topic: buildbots
In-Reply-To: <24ea26600910300942j3790e8a0m83c011ac17e57bd9@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
	<20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
	<24ea26600910300942j3790e8a0m83c011ac17e57bd9@mail.gmail.com>
Message-ID: <20091030170129.12668.399242170.divmod.xquotient.122@localhost.localdomain>

On 04:42 pm, olemis at gmail.com wrote:
>On Sun, Oct 25, 2009 at 9:13 AM,  <exarkun at twistedmatrix.com> wrote:
>>On 12:48 pm, ctb at msu.edu wrote:
>>>
>>>[snip]
>>>
>>>The most *exciting* part of pony-build, apart from the always- 
>>>riveting
>>>spectacle of "titus rediscovering problems that buildbot solved 5 
>>>years
>>>ago",
>>>is the loose coupling of recording server to the build slaves and 
>>>build
>>>reporters. ?My plan is to enable a simple and lightweight XML-RPC 
>>>and/or
>>>REST-ish interface for querying the recording server from scripts or 
>>>other
>>>Web
>>>sites. ?This has Brett aquiver with anticipation, I gather -- no more
>>>visual
>>>inspection of buildbot waterfall pages ;)
>>
>>BuildBot has an XML-RPC interface. ?So Brett can probably do what he 
>>wants
>>with BuildBot right now.
>
>... but pony-build follows a different model

But BuildBot exists, is deployed, and can be used now, without waiting.

(Sorry, I don't really understand what point you were hoping to make 
with your message, so I just thought I'd follow up in the same style and 
hope that you'll understand my message even if I don't understand yours 
:).

Jean-Paul

From status at bugs.python.org  Fri Oct 30 18:07:15 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 30 Oct 2009 18:07:15 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091030170715.D6AED781CD@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/23/09 - 10/30/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2483 open (+31) / 16582 closed (+19) / 19065 total (+50)

Open issues with patches:   989

Average duration of open issues: 675 days.
Median duration of open issues: 430 days.

Open Issues Breakdown
   open  2447 (+31)
pending    35 ( +0)

Issues Created Or Reopened (53)
_______________________________

distutils and IronPython compatibility                           10/27/09
CLOSED http://bugs.python.org/issue7071    reopened michael.foord                 
       26backport                                                              

Popen blocks program from another thread                         10/23/09
CLOSED http://bugs.python.org/issue7193    reopened dgriff1                       
                                                                               

test_thread is flaky                                             10/23/09
CLOSED http://bugs.python.org/issue7194    created  pitrou                        
                                                                               

Value error 'path is on drive c: start on drive d:' in os.path.r 10/23/09
       http://bugs.python.org/issue7195    created  jaraco                        
                                                                               

Clarify str.split() behavior                                     10/24/09
       http://bugs.python.org/issue7196    created  gagenellina                   
       patch                                                                   

test_multiprocessing crashes under Windows when run in verbose m 10/24/09
       http://bugs.python.org/issue7197    created  pitrou                        
       patch                                                                   

csv.writer                                                       10/24/09
CLOSED http://bugs.python.org/issue7198    created  zacktu                        
                                                                               

Doc: Logging level order seems inconsistent                      10/24/09
CLOSED http://bugs.python.org/issue7199    created  tjreedy                       
                                                                               

multiprocessing deadlock on Mac OS X when queue collected before 10/24/09
       http://bugs.python.org/issue7200    created  bquinlan                      
                                                                               

double Endian problem and more on arm                            10/24/09
       http://bugs.python.org/issue7201    created  mancausoft                    
                                                                               

"python setup.py MYCOMMAND --verbose" does not yield an unrecogn 10/25/09
       http://bugs.python.org/issue7202    created  zooko                         
                                                                               

fixer for map(None, ...) needs to consider multi-argument case   10/25/09
CLOSED http://bugs.python.org/issue7203    created  georg.brandl                  
                                                                               

Strange shebang line in test_pep263                              10/25/09
CLOSED http://bugs.python.org/issue7204    created  wiz                           
                                                                               

bz2file deadlock                                                 10/25/09
CLOSED http://bugs.python.org/issue7205    created  rbcollins                     
       patch                                                                   

64 bit python fails on Windows 7                                 10/25/09
       http://bugs.python.org/issue7206    created  richo                         
                                                                               

test_telnetlib fails on OS X 10.6                                10/26/09
CLOSED http://bugs.python.org/issue7207    created  mark.dickinson                
       patch, needs review                                                     

Getpass echo's password to screen on 2.6, but not on 2.5 or 3.1  10/26/09
       http://bugs.python.org/issue7208    created  pajs at fodder.org.uk            
       patch                                                                   

Prevents uint_t from being used on QNX                           10/26/09
       http://bugs.python.org/issue7209    created  kraai                         
       patch                                                                   

Proposed Syntax Checks in Test Suite                             10/26/09
       http://bugs.python.org/issue7210    created  ChuckRhode                    
       patch                                                                   

select module - kevent ident field 64 bit issue                  10/26/09
       http://bugs.python.org/issue7211    created  mbroughton                    
       patch                                                                   

Retrieve an arbitrary element from a set without removing it     10/26/09
       http://bugs.python.org/issue7212    created  wrichert                      
       patch                                                                   

Popen.subprocess change close_fds default to True                10/26/09
       http://bugs.python.org/issue7213    created  milko.krachounov              
                                                                               

TreeBuilder.end(tag) differs between cElementTree and ElementTre 10/26/09
       http://bugs.python.org/issue7214    created  merrellb                      
                                                                               

TreeBuilder.end(tag) differs between cElementTree and ElementTre 10/26/09
CLOSED http://bugs.python.org/issue7215    created  merrellb                      
                                                                               

low performance of zipfile readline()                            10/27/09
       http://bugs.python.org/issue7216    created  volker_siepmann               
                                                                               

IDLE Subprocess Startup Error                                    10/27/09
       http://bugs.python.org/issue7217    created  irdb                          
                                                                               

test_site failure under Windows                                  10/27/09
CLOSED http://bugs.python.org/issue7218    created  pitrou                        
                                                                               

Unhelpful error message when a distutils package install fails d 10/27/09
       http://bugs.python.org/issue7219    created  michael.foord                 
       26backport                                                              

python -m no longer executes packages directly                   10/27/09
CLOSED http://bugs.python.org/issue7220    created  Martijn                       
                                                                               

DispatcherWithSendTests_UsePoll with test_asyncore does nothing  10/27/09
       http://bugs.python.org/issue7221    created  pitrou                        
                                                                               

thread reaping is imperfect                                      10/27/09
       http://bugs.python.org/issue7222    created  pitrou                        
       patch                                                                   

Simplify and improve Thread.join()                               10/27/09
CLOSED http://bugs.python.org/issue7223    created  pitrou                        
       patch                                                                   

One obvious way to do interning                                  10/28/09
       http://bugs.python.org/issue7224    created  belopolsky                    
       patch                                                                   

fwrite() compiler warnings                                       10/28/09
       http://bugs.python.org/issue7225    created  benjamin.peterson             
                                                                               

IDLE right-clicks don't work on Mac OS 10.5                      10/28/09
       http://bugs.python.org/issue7226    created  MLModel                       
                                                                               

Shell Support for installation of Python Packages (.EGG)         10/28/09
CLOSED http://bugs.python.org/issue7227    created  djlyon                        
                                                                               

%lld for PyErr_Format (Modules/_io/bufferedio.c)                 10/28/09
       http://bugs.python.org/issue7228    created  ocean-city                    
                                                                               

Manual entry for time.daylight can be misleading                 10/28/09
       http://bugs.python.org/issue7229    created  napik                         
                                                                               

test_hotshot fails on solaris                                    10/28/09
       http://bugs.python.org/issue7230    created  csernazs                      
                                                                               

Windows installer does not add \Scripts folder to the path       10/28/09
       http://bugs.python.org/issue7231    created  sorin                         
                                                                               

Support of 'with' statement fo TarFile class                     10/29/09
       http://bugs.python.org/issue7232    created  jaime.buelta                  
       patch                                                                   

decimal.py: two rotate() issues                                  10/29/09
CLOSED http://bugs.python.org/issue7233    created  skrah                         
                                                                               

Complete your registration to Python tracker -- keyRVZuTYMju0Lif 10/29/09
CLOSED http://bugs.python.org/issue7234    created  kaka                          
                                                                               

IO libary have some error                                        10/29/09
CLOSED http://bugs.python.org/issue7235    created  kaka                          
                                                                               

the reply's additional "Re:" is ok                               10/29/09
CLOSED http://bugs.python.org/issue7236    created  kaka                          
                                                                               

Syntax error with not                                            10/29/09
CLOSED http://bugs.python.org/issue7237    created  LambertDW                     
                                                                               

frame.f_lineno doesn't get updated after local trace function as 10/29/09
       http://bugs.python.org/issue7238    created  eggy                          
                                                                               

Error when running a code                                        10/29/09
CLOSED http://bugs.python.org/issue7239    created  MNilson                       
                                                                               

subprocess.Popen.stdout.flush fails os OS-X 10.6.1               10/30/09
       http://bugs.python.org/issue7240    created  petegibson                    
                                                                               

tkinter fails to import                                          10/30/09
CLOSED http://bugs.python.org/issue7241    reopened python.noob                   
                                                                               

Forking in a thread raises RuntimeError                          10/30/09
       http://bugs.python.org/issue7242    created  csernazs                      
                                                                               

mac binary download link for 2.6.4 broken                        10/30/09
       http://bugs.python.org/issue7243    created  beard                         
                                                                               

0.0 and -0.0 identified, with surprising results                 10/24/09
       http://bugs.python.org/issue1678380 reopened mark.dickinson                
       patch                                                                   



Issues Now Closed (47)
______________________

Option to ignore or substitute ~/.pydistutils.cfg                 769 days
       http://bugs.python.org/issue1180    tarek                         
       patch, easy                                                             

Extension module build fails for MinGW: missing vcvarsall.bat     550 days
       http://bugs.python.org/issue2698    tarek                         
                                                                               

py3k warn on use of frame.f_exc*                                  486 days
       http://bugs.python.org/issue3223    benjamin.peterson             
       easy                                                                    

'dictionary changed size' error in test_multiprocessing           433 days
       http://bugs.python.org/issue3578    pitrou                        
                                                                               

tarfile keeps excessive dir structure in compressed files         307 days
       http://bugs.python.org/issue4750    lars.gustaebel                
       patch                                                                   

tarfile: path problem in arcname under windows                    226 days
       http://bugs.python.org/issue5500    lars.gustaebel                
                                                                               

Incorrect DST transition                                          217 days
       http://bugs.python.org/issue5582    acummings                     
                                                                               

Fix reference leak in io.StringIO                                 138 days
       http://bugs.python.org/issue6242    pitrou                        
       patch                                                                   

zipfile: Invalid argument when opening zero-sized files           102 days
       http://bugs.python.org/issue6511    skelker                       
                                                                               

PEP 314 inconsistency (authors/author/maintainer)                  28 days
       http://bugs.python.org/issue6992    tarek                         
                                                                               

utf-8 encoding error                                               25 days
       http://bugs.python.org/issue7045    benjamin.peterson             
                                                                               

'g'/'G' format docs need a little more explanation                 24 days
       http://bugs.python.org/issue7051    mark.dickinson                
                                                                               

Automatic test___all__                                             23 days
       http://bugs.python.org/issue7055    pitrou                        
       patch                                                                   

Add some test execution environment protection to regrtest         23 days
       http://bugs.python.org/issue7058    r.david.murray                
       patch                                                                   

archive_util.make_archive doesn't restore the cwd if an error is   19 days
       http://bugs.python.org/issue7066    tarek                         
                                                                               

distutils and IronPython compatibility                              0 days
       http://bugs.python.org/issue7071    r.david.murray                
       26backport                                                              

struct help in the interpreter does not explain about the fmt op   20 days
       http://bugs.python.org/issue7078    mark.dickinson                
       patch, easy                                                             

Patch for get_filename in email.message when content-disposition   19 days
       http://bugs.python.org/issue7082    r.david.murray                
       patch                                                                   

printing a list releases the GIL carelessly                        19 days
       http://bugs.python.org/issue7084    pitrou                        
       patch                                                                   

Decimal.is_normal should return True even for numbers with expon   17 days
       http://bugs.python.org/issue7099    mark.dickinson                
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat      12 days
       http://bugs.python.org/issue7131    tarek                         
                                                                               

Fix Download Current Documentation link                            10 days
       http://bugs.python.org/issue7157    georg.brandl                  
                                                                               

zipfile leaves a file handle open if file is zero size              9 days
       http://bugs.python.org/issue7169    r.david.murray                
                                                                               

sum() doesn't work for lists.                                       6 days
       http://bugs.python.org/issue7176    ezio.melotti                  
                                                                               

Popen blocks program from another thread                            0 days
       http://bugs.python.org/issue7193    dgriff1                       
                                                                               

test_thread is flaky                                                5 days
       http://bugs.python.org/issue7194    ned.deily                     
                                                                               

csv.writer                                                          0 days
       http://bugs.python.org/issue7198    zacktu                        
                                                                               

Doc: Logging level order seems inconsistent                         4 days
       http://bugs.python.org/issue7199    vinay.sajip                   
                                                                               

fixer for map(None, ...) needs to consider multi-argument case      2 days
       http://bugs.python.org/issue7203    segfaulthunter                
                                                                               

Strange shebang line in test_pep263                                 0 days
       http://bugs.python.org/issue7204    georg.brandl                  
                                                                               

bz2file deadlock                                                    2 days
       http://bugs.python.org/issue7205    pitrou                        
       patch                                                                   

test_telnetlib fails on OS X 10.6                                   1 days
       http://bugs.python.org/issue7207    mark.dickinson                
       patch, needs review                                                     

TreeBuilder.end(tag) differs between cElementTree and ElementTre    0 days
       http://bugs.python.org/issue7215    r.david.murray                
                                                                               

test_site failure under Windows                                     0 days
       http://bugs.python.org/issue7218    tarek                         
                                                                               

python -m no longer executes packages directly                      0 days
       http://bugs.python.org/issue7220    benjamin.peterson             
                                                                               

Simplify and improve Thread.join()                                  0 days
       http://bugs.python.org/issue7223    pitrou                        
       patch                                                                   

Shell Support for installation of Python Packages (.EGG)            0 days
       http://bugs.python.org/issue7227    tarek                         
                                                                               

decimal.py: two rotate() issues                                     0 days
       http://bugs.python.org/issue7233    mark.dickinson                
                                                                               

Complete your registration to Python tracker -- keyRVZuTYMju0Lif    0 days
       http://bugs.python.org/issue7234    eric.smith                    
                                                                               

IO libary have some error                                           0 days
       http://bugs.python.org/issue7235    kaka                          
                                                                               

the reply's additional "Re:" is ok                                  0 days
       http://bugs.python.org/issue7236    eric.smith                    
                                                                               

Syntax error with not                                               0 days
       http://bugs.python.org/issue7237    benjamin.peterson             
                                                                               

Error when running a code                                           0 days
       http://bugs.python.org/issue7239    MNilson                       
                                                                               

tkinter fails to import                                             0 days
       http://bugs.python.org/issue7241    python.noob                   
                                                                               

bdist_deb - Debian packager                                       251 days
       http://bugs.python.org/issue1054967 tarek                         
       patch                                                                   

long int bitwise ops speedup (patch included)                    1773 days
       http://bugs.python.org/issue1087418 mark.dickinson                
       patch                                                                   

Thread shutdown exception in Thread.notify()                      630 days
       http://bugs.python.org/issue1722344 pitrou                        
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 19 Backport py3k float repr to trunk                                 17 days
open    http://bugs.python.org/issue7117   

 12 %lld for PyErr_Format (Modules/_io/bufferedio.c)                   2 days
open    http://bugs.python.org/issue7228   

 11 unify pydistutils.cfg and distutils.cfg and use .local            10 days
open    http://bugs.python.org/issue7175   

 10 64 bit python fails on Windows 7                                   5 days
open    http://bugs.python.org/issue7206   

 10 long int bitwise ops speedup (patch included)                   1773 days
closed  http://bugs.python.org/issue1087418

  9 select module - kevent ident field 64 bit issue                    4 days
open    http://bugs.python.org/issue7211   

  9 bz2file deadlock                                                   2 days
closed  http://bugs.python.org/issue7205   

  9 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1         1029 days
open    http://bugs.python.org/issue1628484

  7 Proposed Syntax Checks in Test Suite                               4 days
open    http://bugs.python.org/issue7210   

  7 fixer for map(None, ...) needs to consider multi-argument case     2 days
closed  http://bugs.python.org/issue7203   




From olemis at gmail.com  Fri Oct 30 18:27:14 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 30 Oct 2009 12:27:14 -0500
Subject: [Python-Dev] [TIP] Possible language summit topic: buildbots
In-Reply-To: <20091030164549.GC19953@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<20091025124816.GA20329@idyll.org>
	<20091025141317.11571.1000888001.divmod.xquotient.1552@localhost.localdomain>
	<24ea26600910300942j3790e8a0m83c011ac17e57bd9@mail.gmail.com>
	<20091030164549.GC19953@idyll.org>
Message-ID: <24ea26600910301027j3724ab79x80de968a118047e8@mail.gmail.com>

On Fri, Oct 30, 2009 at 11:45 AM, C. Titus Brown <ctb at msu.edu> wrote:
> On Fri, Oct 30, 2009 at 11:42:30AM -0500, Olemis Lang wrote:
>> On Sun, Oct 25, 2009 at 9:13 AM, ?<exarkun at twistedmatrix.com> wrote:
>> > On 12:48 pm, ctb at msu.edu wrote:
>> >>
>> >> [snip]
>> >>
>> >> The most *exciting* part of pony-build, apart from the always-riveting
>> >> spectacle of "titus rediscovering problems that buildbot solved 5 years
>> >> ago",
>> >> is the loose coupling of recording server to the build slaves and build
>> >> reporters. ?My plan is to enable a simple and lightweight XML-RPC and/or
>> >> REST-ish interface for querying the recording server from scripts or other
>> >> Web
>> >> sites. ?This has Brett aquiver with anticipation, I gather -- no more
>> >> visual
>> >> inspection of buildbot waterfall pages ;)
>> >
>> > BuildBot has an XML-RPC interface. ?So Brett can probably do what he wants
>> > with BuildBot right now.
>>
>> ... but pony-build follows a different model
>

that was just a brief comment to mention that even if both (buildbot +
pony-build) support RPC, they are not just ?the same?  .

> I'd rather not get into discussions of why my vaporware is going to be
> way, way better than anything anyone else could possibly do

+1 ... I'll be the first one that won't follow it since I have no time
for that and my intention was not to suggest that ?pb is better than
bb? (but if you follow, please do it in a separate thread ;o)

@exarkun at twistedmatrix.com
> But BuildBot exists, is deployed, and can be used now, without waiting.
>

+1  as I mentioned before I was not talking about eliminating buildbots

> (Sorry, I don't really understand what point you were hoping to make with your
> message, so I just thought I'd follow up in the same style and hope that you'll
> understand my message even if I don't understand yours :).

well, I understand that you don't understand, since I barely
understand that I will never be able to understand myself ... :)

The only thing I can say so far is that if pb is seriously considered
as an option ... then I could give a hand (... and I'll possibly do it
anyway , once I have time :-/ )

... turning myself off ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From g.brandl at gmx.net  Fri Oct 30 21:37:36 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 30 Oct 2009 21:37:36 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200910301358.17752.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<200910301358.17752.steve@pearwood.info>
Message-ID: <hcfff8$6vr$1@ger.gmane.org>

Steven D'Aprano schrieb:
> On Wed, 28 Oct 2009 04:47:59 am Raymond Hettinger wrote:
> 
>> A dict.get() can be meaningfully used in a loop (because the key can
>> vary). A set.get() returns the same value over and over again
>> (because there is no key).
> 
> I don't believe anyone has requested those semantics. The suggested 
> semantics for set.get() with no arguments, as I understand them, are:
> 
> (1) it will only fail if the set is empty;
> 
> (2) it should be efficient;
> 
> (3) if you call it repeatedly on a set without modifying the set, you 
> will cycle through each element in turn in some unspecified arbitrary 
> order.

I don't like this.  It gives a set object a hidden state, something that
AFAICS no other builtin has.  Iterating over an iterable is what iterators
are for.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ctb at msu.edu  Fri Oct 30 20:42:45 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 30 Oct 2009 12:42:45 -0700
Subject: [Python-Dev] Cloud build slaves (was Re: Possible
	language	summit topic: buildbots)
In-Reply-To: <20091030165829.12668.1546393452.divmod.xquotient.118@localhost.localdomain>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
	<20091030165829.12668.1546393452.divmod.xquotient.118@localhost.localdomain>
Message-ID: <20091030194244.GA18135@idyll.org>

On Fri, Oct 30, 2009 at 04:58:29PM -0000, exarkun at twistedmatrix.com wrote:
> On 04:31 pm, ctb at msu.edu wrote:
>> On Fri, Oct 30, 2009 at 04:21:06PM +0000, Antoine Pitrou wrote:
>>>
>>> Hello,
>>>
>>> Sorry for the little redundancy, I would like to underline Jean-Paul's
>>> suggestion here:
>>>
>>> Le Sun, 25 Oct 2009 14:05:12 +0000, exarkun a ??crit??:
>>> > I think that money can help in two ways in this case.
>>> >
>>> > First, there are now a multitude of cloud hosting providers which  
>>> will
>>> > operate a slave machine for you.  BuildBot has even begun to support
>>> > this deployment use-case by allowing you to start up and shut down  
>>> vms
>>> > on demand to save on costs.  Amazon's EC2 service is supported out  
>>> of
>>> > the box in the latest release.
>>>
>>> I'm not a PSF member, but it seems to me that the PSF could ask Amazon
>>> (or any other virtual machine business anyway) to donate a small  
>>> number
>>> of permanent EC2 instances in order to run buildslaves on.
>>
>> [ ... ]
>>
>> I'm happy to provide VMs or shell access for Windows (XP, Vista, 7);  
>> Linux
>> ia64; Linux x86; and Mac OS X.
>
> Okay, let's move on this.  Martin has, I believe, said that potential  
> slave operators only need to contact him to get credentials for new  
> slaves.  Can you make sure to follow up with him to get slaves running  
> on these machines?  Or would you rather give out access to someone else  
> and have them do the build slave setup?

I think we crossed threads here; I can provide the VMs, and access to them,
but I won't (empirically, don't have the regular time available to ;) maintain
buildbot buildslaves.

You or Antoine or others are welcome to contact me off-list.  Just give me an
account name and ssh key, and I'll give you login access via tunneled Remote
Desktop to the Windows machines.

>> Others have made similar offers.
>
> I'll similarly encourage them to take action, then.  Do you happen to  
> remember who?

Every few months this thread seems to pop up and then fizzles when people
realize the level of work and attention involved (<- he says cynically) in
exploiting the offer of resources; I hope that anyone interested in
offering resources will pop their head up again to look around.

> I hope everyone is on board with the idea of fixing bugs in CPython,  
> either in the actual implementation of features or in the tests for  
> those features.  That being the case, the discussion of whether or not  
> the PSF should try to fund such a task is perhaps best discussed on the  
> PSF members list.

Sure.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From p.f.moore at gmail.com  Fri Oct 30 20:46:34 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 30 Oct 2009 19:46:34 +0000
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <20091030165615.GB5132@idyll.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
	<79990c6b0910300949ybc11b29q11900f08a79cdb9e@mail.gmail.com>
	<20091030165615.GB5132@idyll.org>
Message-ID: <79990c6b0910301246p60dcd5e5s4c1e7b9c0f3662db@mail.gmail.com>

2009/10/30 C. Titus Brown <ctb at msu.edu>:
>> As a counter-offer: Given remote access to however many Windows VMs
>> you want to provide, I'll get them up and running with buildslaves on
>> them. If that requires software such as Visual Studio, I have copies
>> via the MSDN licenses that I am happy to provide.
>
> I, too, have MSDN licenses, and I have functioning build environments
> on all of the VMs (I think -- I've only tested Win XP currently:
>
> http://lyorn.idyll.org/ctb/pb-dev/python/detail?result_key=8276

OK, so I guess it's just setting the buildbot stuff up.

> I also have an OS X 10.5 machine that I can let you into through a firewall;
> it's building Python 2.7 quite nicely:
>
> http://lyorn.idyll.org/ctb/pb-dev/python/detail?result_key=8229

Sorry, I've no experience with OS X at all.

>> Once things are up and running, I'll be prepared to do basic care and
>> feeding of the buildslave, but as my time is limited, it would be nice
>> if others would pitch in to help.
>
> I would be somewhat unhappy about giving more than three or four people
> admin access, but am prepared to lie back and think of England.

Greetings from England... :-)

I doubt it'll be a huge issue, I just didn't want to end up doing
nothing more than delivering 5 more red boxes on the buildbot status
page. We can see how it goes. I just think that maintaining the
buildbots as more of a community effort means that there's a better
chance of issues being fixed quickly.

Paul.

From rdmurray at bitdance.com  Fri Oct 30 20:51:01 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 30 Oct 2009 15:51:01 -0400 (EDT)
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <4AEAAA8B.3000007@v.loewis.de>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
	<4AEAAA8B.3000007@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0910301534350.21420@kimball.webabinitio.net>

On Fri, 30 Oct 2009 at 09:57, "Martin v. L?wis" wrote:
>> But the real reason for having a buildbot category (or at least a keyword)
>> would be to be able to tag all bugs that are currently making buildbots
>> fail that are _not_ the result of a recent checkin.  This would make
>> the task of finding the bugs that need to be cleaned up to stabilize
>> the buildbot fleet easier.  I'm currently aware of issues 4970, 3892,
>> and 6462 in this category, and there are a few more that we can/will file
>> if we continue to pay attention to the failure reports now arriving on
>> the irc channel.
>
> That's convincing; I've created a "buildbot" keyword. I gave it the
> description
>
>  "indicates that tests fail on a buildslave for uncertain reasons"
>
> If that is indeed the intended purpose of this classification, please
> keep it in mind when assigning the tag. If I misunderstood the purpose
> of the keyword, please provide a better description.

How about:
     "indicates that related test failures are causing buildbot
     instability"

My thought is that sometimes we more-or-less know the reasons for the
failure, but for one reason or another we can't fix it immediately, and
I'd like to keep such a bug visible when looking at buildbot related
issues.

IMO it would be no bad thing for this tag to be applied to any issue
that is created as a result of an observed test failure on a buildbot.
Such an issue should only get created if the person who did the checkin
that caused it can't reproduce the problem themselves (ie: they ran
the test suite and on their platform it was clean).  Now, we know that
in practice some bugs show up on buildbots because a committer forgot
to run the test suite prior to check in (we all make mistakes), but if
such a bug gets tagged 'buildbot' I think that's fine, since it will
still be affecting the stability of the buildbots.

--David

From rdmurray at bitdance.com  Fri Oct 30 21:08:14 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 30 Oct 2009 16:08:14 -0400 (EDT)
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <79990c6b0910301246p60dcd5e5s4c1e7b9c0f3662db@mail.gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <20091030163128.GA30191@idyll.org>
	<79990c6b0910300949ybc11b29q11900f08a79cdb9e@mail.gmail.com>
	<20091030165615.GB5132@idyll.org>
	<79990c6b0910301246p60dcd5e5s4c1e7b9c0f3662db@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0910301603360.21420@kimball.webabinitio.net>

On Fri, 30 Oct 2009 at 19:46, Paul Moore wrote:
> 2009/10/30 C. Titus Brown <ctb at msu.edu>:
>>> Once things are up and running, I'll be prepared to do basic care and
>>> feeding of the buildslave, but as my time is limited, it would be nice
>>> if others would pitch in to help.
>>
>> I would be somewhat unhappy about giving more than three or four people
>> admin access, but am prepared to lie back and think of England.
>
> Greetings from England... :-)
>
> I doubt it'll be a huge issue, I just didn't want to end up doing
> nothing more than delivering 5 more red boxes on the buildbot status
> page. We can see how it goes. I just think that maintaining the
> buildbots as more of a community effort means that there's a better
> chance of issues being fixed quickly.

I'd be happy to help with keeping these (or any other) buildbots running,
but I'm not much of a Windows geek (I can work with it, but I know a _lot_
more about Linux).  Same goes for OS/X, though since that is unix based
I'm a little more comfortable with it.

I guess what I'm saying is, if you don't get responses from more
Windows-savvy developers, then let me know and I'll be glad to help.

That said, the idea of EC2 buildslaves seems pretty attractive...

--David

From amk at amk.ca  Fri Oct 30 21:27:22 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 30 Oct 2009 16:27:22 -0400
Subject: [Python-Dev] Retrieve an arbitrary element from a
	set	withoutremoving it
In-Reply-To: <hcfff8$6vr$1@ger.gmane.org>
References: <200910231132.45504.w.richert@gmx.net>
	<7b0045ed0910271033h69fcc06aybbff0fa3192bb060@mail.gmail.com>
	<B50D88D04FD74B2B8733F6B4207F9BE9@RaymondLaptop1>
	<200910301358.17752.steve@pearwood.info>
	<hcfff8$6vr$1@ger.gmane.org>
Message-ID: <20091030202722.GA23370@amk-desktop.matrixgroup.net>

On Fri, Oct 30, 2009 at 09:37:36PM +0100, Georg Brandl wrote:
> I don't like this.  It gives a set object a hidden state, something that
> AFAICS no other builtin has.  Iterating over an iterable is what iterators
> are for.

It also makes the object thread-unsafe; there's no way for two threads
to iterate over it at the same time.  It's a terrible idea to
introduce new things that won't work under threaded usage.

--amk

From solipsis at pitrou.net  Fri Oct 30 23:04:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 30 Oct 2009 22:04:12 +0000 (UTC)
Subject: [Python-Dev] "Buildbot" category on the tracker
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
Message-ID: <loom.20091030T225651-346@post.gmane.org>

<exarkun <at> twistedmatrix.com> writes:
> 
> Is your idea that this would be for tracking issues with the *bots* 
> themselves?  That is, not just for tracking cases where some test method 
> fails on a particular bot, but for tracking cases where, say, a bot's 
> host has run out of disk space and cannot run the tests at all?

Well the general situation would be slightly easier to appreciate if there was a
public medium where buildbot info was exchanged, announcements done, and
problems tracked. Some kind of tracker, tracker keyword, mailing-list, or
anything else.

Regards

Antoine.



From martin at v.loewis.de  Fri Oct 30 23:36:32 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Oct 2009 23:36:32 +0100
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <loom.20091030T225651-346@post.gmane.org>
References: <loom.20091029T152906-465@post.gmane.org>	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<loom.20091030T225651-346@post.gmane.org>
Message-ID: <4AEB6A70.7090007@v.loewis.de>

> Well the general situation would be slightly easier to appreciate if there was a
> public medium where buildbot info was exchanged, announcements done, and
> problems tracked. Some kind of tracker, tracker keyword, mailing-list, or
> anything else.

As for the tracker keyword - I created one (in case you didn't notice).

As for exchanging info: if you talk about the specific Python buildbot
installation (instead of info about buildbot, the software), then
python-dev is the proper place to exchange info about it.

Regards,
Martin

From martin at v.loewis.de  Fri Oct 30 23:43:18 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 30 Oct 2009 23:43:18 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <hcf3pi$vgs$1@ger.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org>
Message-ID: <4AEB6C06.3070907@v.loewis.de>

> Since I've never used any such service ("cloud"-based VMs), I'm not sure 
> what the downsides would be. But it seems to be that it would be at least 
> worth trying.

Not sure whether it's still relevant after the offers of individually
donated hardware. However, if you want to look into this, feel free to
set up EC2 slaves. When it comes to the point of actually having to pay
money, please send a request to psf-board at python.org (make sure you
don't pay anything until the request is approved). Exact processing
would have to be decided, then, traditionally, it would be most simple
if you could invoice the PSF (IIUC). Giving the PSF as the billing
address would probably also work (check with the treasurer).

Regards,
Martin

From steve at pearwood.info  Sat Oct 31 02:29:28 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 31 Oct 2009 12:29:28 +1100
Subject: [Python-Dev]
	=?iso-8859-1?q?Retrieve_an_arbitrary_element_from_a_?=
	=?iso-8859-1?q?set=09withoutremoving_it?=
In-Reply-To: <20091030202722.GA23370@amk-desktop.matrixgroup.net>
References: <200910231132.45504.w.richert@gmx.net> <hcfff8$6vr$1@ger.gmane.org>
	<20091030202722.GA23370@amk-desktop.matrixgroup.net>
Message-ID: <200910311229.29250.steve@pearwood.info>

On Sat, 31 Oct 2009 07:27:22 am A.M. Kuchling wrote:
> On Fri, Oct 30, 2009 at 09:37:36PM +0100, Georg Brandl wrote:
> > I don't like this.  It gives a set object a hidden state, something
> > that AFAICS no other builtin has.

All objects have a reference count field which is hidden from Python 
code. The C API for objects has a flags field which specifies whether 
objects are read-only or read/write from Python code.

As of Python 2.6, type objects have an internal method cache. C code can 
clear it with PyType_ClearCache(), Python codes can't even see it.

Lists and dicts pre-allocate extra space, and record hidden state of how 
much of the space is actually in use. Sets may do the same. File 
objects may use internal buffers, with all the state that implies.


> > Iterating over an iterable is 
> > what iterators are for.

set.get(), or set.pick() as Wikipedia calls it, isn't for iterating over 
sets. It is for getting an arbitrary element from the set.

If the requirement that get/pick() cycles through the sets elements is 
the killer objection to this proposal, I'd be willing to drop it. I 
thought that was part of the OP's request, but apparently it isn't. I 
would argue that it's hardly "arbitrary" if you get the same element 
every time you call the method, but if others are happy with that 
behaviour, I'm not going to stand in the way.


> It also makes the object thread-unsafe; there's no way for two
> threads to iterate over it at the same time.  It's a terrible idea to
> introduce new things that won't work under threaded usage.

I would agree if the purpose of get/pick() was to iterate over the 
elements of the set, but that's not the purpose. The purpose is to 
return an arbitrary item each time it is called. If two threads get the 
same element, that's not a problem; if one thread misses an element 
because another thread grabbed it first, that's not a problem either. 
If people prefer a random element instead, I have no problem with 
that -- personally I think that's overkill, but maybe that's just me.


-- 
Steven D'Aprano

From benjamin at python.org  Sat Oct 31 02:38:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 30 Oct 2009 20:38:17 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200910311229.29250.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net> <hcfff8$6vr$1@ger.gmane.org>
	<20091030202722.GA23370@amk-desktop.matrixgroup.net>
	<200910311229.29250.steve@pearwood.info>
Message-ID: <1afaf6160910301838u37a12582h2e0a8ece2b3e6a57@mail.gmail.com>

2009/10/30 Steven D'Aprano <steve at pearwood.info>:
> On Sat, 31 Oct 2009 07:27:22 am A.M. Kuchling wrote:
>> On Fri, Oct 30, 2009 at 09:37:36PM +0100, Georg Brandl wrote:
>> > I don't like this. ?It gives a set object a hidden state, something
>> > that AFAICS no other builtin has.
>
> All objects have a reference count field which is hidden from Python
> code. The C API for objects has a flags field which specifies whether
> objects are read-only or read/write from Python code.
>
> As of Python 2.6, type objects have an internal method cache. C code can
> clear it with PyType_ClearCache(), Python codes can't even see it.
>
> Lists and dicts pre-allocate extra space, and record hidden state of how
> much of the space is actually in use. Sets may do the same. File
> objects may use internal buffers, with all the state that implies.

Those are all examples of states which are implementation details. The
proposed get() semantics would require that allow implementations keep
this state around.



-- 
Regards,
Benjamin

From chris at subtlety.com  Sat Oct 31 02:43:31 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Fri, 30 Oct 2009 20:43:31 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200910311229.29250.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>
	<hcfff8$6vr$1@ger.gmane.org> 
	<20091030202722.GA23370@amk-desktop.matrixgroup.net>
	<200910311229.29250.steve@pearwood.info>
Message-ID: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>

On Fri, Oct 30, 2009 at 8:29 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> > Iterating over an iterable is
>> > what iterators are for.
>
> set.get(), or set.pick() as Wikipedia calls it, isn't for iterating over
> sets. It is for getting an arbitrary element from the set.
>
> If the requirement that get/pick() cycles through the sets elements is
> the killer objection to this proposal, I'd be willing to drop it. I
> thought that was part of the OP's request, but apparently it isn't. I
> would argue that it's hardly "arbitrary" if you get the same element
> every time you call the method, but if others are happy with that
> behaviour, I'm not going to stand in the way.

   It's arbitrary in the sense that the API doesn't make any
assurances which item the caller will get, not that it's arbitrary for
any particular * implementation*.

> The purpose is to
> return an arbitrary item each time it is called. If two threads get the
> same element, that's not a problem; if one thread misses an element
> because another thread grabbed it first, that's not a problem either.
> If people prefer a random element instead, I have no problem with
> that -- personally I think that's overkill, but maybe that's just me.

   I also think returning a random one is overkill, in the base set.
And I'd have the base implementation do the simplest thing possible:
return a fixed element (either the first returned when iterated over,
or the last stored, or whatever's easiest based on the internals).
For me, leaving the choice of *which* element to return on each call
is a feature.  It allows subclasses to change the behavior to support
other use cases, like a random or round-robin behavior.

-- Chris

From stephen at xemacs.org  Sat Oct 31 03:42:46 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 31 Oct 2009 11:42:46 +0900
Subject: [Python-Dev] Retrieve an arbitrary element from
	a	setwithoutremoving it
In-Reply-To: <200910310211.33590.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>
	<200910301358.17752.steve@pearwood.info>
	<2585AF2DFDDE4C61B3596633F5001945@RaymondLaptop1>
	<200910310211.33590.steve@pearwood.info>
Message-ID: <874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > The usual technique people tend to come up with is:
 > 
 > x = s.pop()
 > s.add(x)
 > 
 > Which strikes many people (including myself) as inelegant. Surely "get 
 > an element" is a more fundamental operation than "get an element and 
 > remove it"?

Not in a literal urn or bag.  See "sampling with replacement" in any
statistics text.  Note that in order to "have your cake and eat it
too" there's an implicit copy operation (although in Python it will be
a cheap ref copy rather than an expensive object copy).

I'm afraid that the various conflicting intuitions here are all
correct.  That is going to make design impossible without some
compelling use cases.

 > > What's wrong with using next()?  That is what it's for.
 > 
 > You can't call next() on a set. You have to call next(iter(set)).
 > [...]
 > Unless I missed something, this is so unobvious that nobody has
 > suggested it in the thread yet.

I've seen it at least twice (Nick suggested it IIRC), although that
may have been on Python-Ideas (which is where this thread belongs
IMHO).

 > If folks prefer a different name, by all means suggest it. I like
 > the name suggested by Wikipedia, "pick".

"Pick" has a connotation of removal in many contexts: "Pick a card,
any card".  "Pick it up off the floor" (it's not in the set of things
on the floor any more).  Like get, to me it has a flavor of "according
to some criterion": "a band of picked men".  I would expect a pick or
get operation to take an optional predicate.  But then TOOWTDI is

    for x in container:
        if predicate(x):
            break
    else:
        x = default_x    # or perhaps raise in cases where there
                         # theoretically should be an element

 > My reasoning for the given behaviour is as follows:
 > 
 > * If you want the same element twice from a set,

Nobody is asking for that AFAICS.  The use case I have so far observed
people to have in mind is a cast from an equivalence class to a
representative member.  They don't care whether the member is the same
or not.  If they wanted iterator behavior, getting it would not be a
problem.  next(iter(foo_set)) is TOOWTDI.  If they wanted cyclical
behavior, itertools.cycle.  If they wanted random behavior, make a
list and choose from it.

In one leading subcase, the equivalence class is a singleton.  In this
use case what people really want, I suspect, is behavior like Python
strings and characters: a singleton set should provide the same
operations as its unique element and vice versa.

 > So having get() return the same value each time is not useful or 
 > necessary.

Which is not the point.  The point is that having get return a
different value if possible is not useful or necessary in at least
some of the leading use cases.  OTOH, we don't know enough about
"weird" use cases where having a different value returned is important
to decide how that should be done, where "weird" means "outside of the
list of already Obvious Ways".

 > No. Since sets are unordered, there's no way to seek to a specific 
 > element.

I think people will realize that in fact *these* sets are ordered and
they will demand a seek function for various practical purposes.

 > > Sorry for so many questions, but I honestly think there are too many
 > > unresolved design issues.

I have to agree with Raymond.


From barry at barrys-emacs.org  Sat Oct 31 11:07:12 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Sat, 31 Oct 2009 10:07:12 +0000
Subject: [Python-Dev] Add const to python API - issue 6952
In-Reply-To: <A043C4A3-A6BA-4F68-A092-B9CB890DD462@barrys-emacs.org>
References: <D1755C55-BC5A-4144-9952-CBF55B99D9BB@barrys-emacs.org>	<4ADD2FE9.2010807@v.loewis.de>	<74D1336F-AB87-431F-895C-885EE344CEBD@barrys-emacs.org>
	<eae285400910201431j17d35733vc6423abf85997af9@mail.gmail.com>
	<4ADE98E2.3080707@v.loewis.de>
	<A043C4A3-A6BA-4F68-A092-B9CB890DD462@barrys-emacs.org>
Message-ID: <38B5518D-20C5-438E-8300-D0E92066B391@barrys-emacs.org>


On 22 Oct 2009, at 20:31, Barry Scott wrote:

>
> On 21 Oct 2009, at 06:15, Martin v. L?wis wrote:
>
>>> I suggest following POSIX's lead and omitted the const in these  
>>> cases.
>
> Ah... Yes I see this in strstr for example.
>
>>
>> Thanks, that sounds reasonable.
>
>
> This is basically what I have done in the patch I hope.
> I have updated the patch to include a comment explaining
> why output  param is char ** and the need for a cast.

Have I address the issues sufficiently for the patch to be accepted?

Should I go ahead a create the python 3 version of the patch?

Barry


From martin at v.loewis.de  Sat Oct 31 15:57:31 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 31 Oct 2009 15:57:31 +0100
Subject: [Python-Dev] "Buildbot" category on the tracker
In-Reply-To: <Pine.LNX.4.64.0910301534350.21420@kimball.webabinitio.net>
References: <loom.20091029T152906-465@post.gmane.org>
	<20091029230443.15635.1202637346.divmod.xquotient.10@localhost.localdomain>
	<4222a8490910291641jaa66329uf8ff59f0e13c708b@mail.gmail.com>
	<Pine.LNX.4.64.0910292011460.21420@kimball.webabinitio.net>
	<4AEAAA8B.3000007@v.loewis.de>
	<Pine.LNX.4.64.0910301534350.21420@kimball.webabinitio.net>
Message-ID: <4AEC505B.8040701@v.loewis.de>

> How about:
>     "indicates that related test failures are causing buildbot
>     instability"

Ok, changed!

Martin

From martin at v.loewis.de  Sat Oct 31 19:24:20 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 31 Oct 2009 19:24:20 +0100
Subject: [Python-Dev] Possible language summit topic: buildbots
In-Reply-To: <loom.20091027T115542-139@post.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<hc1fiq$5ec$1@ger.gmane.org>	<20091025135018.11571.560646061.divmod.xquotient.1516@localhost.localdomain>	<4AE4A1C9.5010002@v.loewis.de>	<m2aazfb1gg.fsf@valheru.db3l.homeip.net>	<20091026005815.31526.130687717.divmod.xquotient.88@localhost.localdomain>	<m263a3aqpq.fsf@valheru.db3l.homeip.net>	<20091026015650.31526.600593069.divmod.xquotient.94@localhost.localdomain>	<4AE55E39.20000@v.loewis.de>
	<loom.20091027T115542-139@post.gmane.org>
Message-ID: <4AEC80D4.5090004@v.loewis.de>

Antoine Pitrou wrote:
> Martin v. L?wis <martin <at> v.loewis.de> writes:
>> It's not really reproducible. I think it sometimes happens when I
>> restart the master; sometimes, some clients fail to reconnect
>> (properly).
> 
> Another common problem is that some buildbot fails in the middle of the test
> suite, with the following kind of message:
> 
> command timed out: 1800 seconds without output, killing pid 12325
> process killed by signal 9
> program finished with exit code -1
> elapsedTime=10910.362981
> 
> See for example :
> http://www.python.org/dev/buildbot/trunk.stable/builders/ia64%20Ubuntu%20trunk/builds/73/steps/test/logs/stdio
> 
> (notice, by the way, the elapsed time (10910s, that is, close to 3 hours...))

That's not really a challenge to the buildbot operator, though - the
buildbot will continue just fine afterwards, right?

For some reason, the test suite stopped producing output, and
eventually, the buildbot decided to kill the build process.
Most likely, the machine ran severely out of memory, so everything
stopped working.

Regards,
Martin

From solipsis at pitrou.net  Sat Oct 31 21:13:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 31 Oct 2009 20:13:57 +0000 (UTC)
Subject: [Python-Dev] EC2 buildslaves
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
Message-ID: <loom.20091031T210845-748@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> Not sure whether it's still relevant after the offers of individually
> donated hardware.

We'll see, indeed.

> However, if you want to look into this, feel free to
> set up EC2 slaves.

I only know to setup mainstream Linux distros though (Debian- or
Redhat-lookalikes :-)). I've just played a bit and, after the hassle of juggling
with a bunch of different keys and credentials, setting up an instance and
saving an image for future use is quite easy.

Regards

Antoine.



From brett at python.org  Sat Oct 31 22:47:43 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 31 Oct 2009 14:47:43 -0700
Subject: [Python-Dev] thanks to everyone cleaning up the tests
Message-ID: <bbaeab100910311447l656c4f17tee3d88225e1cdcfc@mail.gmail.com>

Just wanted to publicly thank everyone who has been causing all the
checkins to fix and stabilize the test suite (I think it's mostly
Antoine and Mark, but I could be missing somebody; I'm under a
deadline so only have marginal higher brain functionality).

-Brett

From solipsis at pitrou.net  Sat Oct 31 23:03:55 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 31 Oct 2009 22:03:55 +0000 (UTC)
Subject: [Python-Dev] thanks to everyone cleaning up the tests
References: <bbaeab100910311447l656c4f17tee3d88225e1cdcfc@mail.gmail.com>
Message-ID: <loom.20091031T230330-509@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> Just wanted to publicly thank everyone who has been causing all the
> checkins to fix and stabilize the test suite (I think it's mostly
> Antoine and Mark, but I could be missing somebody; I'm under a
> deadline so only have marginal higher brain functionality).

A lot of work was done by David (RDM).

Thanks !

Antoine.