From pnorvig at google.com  Thu Apr  1 00:00:55 2004
From: pnorvig at google.com (Peter Norvig)
Date: Thu Apr  1 00:00:59 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
Message-ID: <42B02151.420DCBB8@mail.google.com>

I have to say I favor the "last before colon" approach, but if it has
to be before the def, then I think it should have a keyword, and if
you don't want to introduce a new keyword, then it looks like "is" is
the only reasonable candidate.  And if you do have a keyword, you
don't need the square brackets.  So you have

is: classmethod
def f(x): 
    pass


is: author("Guido"), signature(int, result=None)
def g(x):
    pass

From jack at performancedrivers.com  Thu Apr  1 00:27:55 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Thu Apr  1 00:28:01 2004
Subject: [Python-Dev] PEP 318: Decorators last before colon
In-Reply-To: <1080790860.21196.23.camel@geddy.wooz.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<1080790860.21196.23.camel@geddy.wooz.org>
Message-ID: <20040401052755.GA18083@performancedrivers.com>

On Wed, Mar 31, 2004 at 10:41:00PM -0500, Barry Warsaw wrote:
> On Tue, 2004-03-30 at 16:21, Guido van Rossum wrote:
> > > Another possibility that has been suggested is
> > > 
> > > [decorator] 
> > > def func(arg, arg):
> > 
> > And one that I currently favor.  I'm out of bandwidth to participate
> > on a msg-by-msg basis, but perhaps folks can see if they can come to
> > terms with this solution?
> 
> I don't like it.  It already has a meaning (albeit fairly useless) and
> it doesn't seem obvious from just looking at it that the decorator is
> connected to the following method.  It doesn't taste Pythonic to me.
> 
Me too, but no one contrasted my actual use cases with any in any other
format, so I must be missing something fundamental.
I support an all Guido ticket now and forever, but I don't get the
pulling-the-rabbit-out-of-a-hat decorator syntax anymore than the at-sign
assignment.  Something in the water at PyCon?

-jackdied

From guido at python.org  Thu Apr  1 00:44:44 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 00:44:53 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: Your message of "Wed, 31 Mar 2004 21:00:55 PST."
	<42B02151.420DCBB8@mail.google.com> 
References: <42B02151.420DCBB8@mail.google.com> 
Message-ID: <200404010544.i315iiB15938@guido.python.org>

Despite some positive responses, my proposal (which was originally due
to Jim Hugunin, who probably has more C# experience than all of us
arguing together) is getting some tough opposition.  I'd like to have
a bake-off, where we view a serious amount of code using decorators to
the hilt with each of three proposed syntaxes:

1) Last-before-colon:

   def foo(cls, arg1, arg2) [funcattrs(foo=42), deprecated,
                             overrides, classmethod]:
       pass

2) Prefix list:

   [funcattrs(foo=42), deprecated, overrides, classmethod]
   def foo(cls, arg1, arg2):
       pass

2a) Prefix list with leading *:

   *[funcattrs(foo=42), deprecated, overrides, classmethod]
   def foo(cls, arg1, arg2):
       pass

3) Prefix suite (could use a different keyword than 'decorate'):

   decorate:
       funcattrs(foo=42)
       deprecated
       overrides
       classmethod
   def foo(cls, arg1, arg2):
       pass

None of the other positions between def and arglist are acceptable any
more; a keyword after the argument list isn't acceptable; I don't
think any syntax that puts the decorators inside the body (between the
colon and the docstring) will work.  So this is the field.

What I'm asking (especially of Phillip) is to collect a set of
realistic method declarations using decorators; we can then
collectively format these using any of the possible syntaxes, and see
how they look.  We can also review different ways of spreading
multiple decorators across several lines, e.g.

   [funcattrs(foo=42),
    deprecated,
    overrides,
    classmethod]
   def foo(cls, arg1, arg2):
       pass

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tanzer at swing.co.at  Thu Apr  1 01:46:47 2004
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu Apr  1 01:48:44 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Wed, 31 Mar 2004 20:42:39 MDT."
	<16491.33183.102141.327504@montanaro.dyndns.org>
Message-ID: <E1B8vyR-0006IZ-00@swing.co.at>


> On the advisability of
>
>     [decorator]
>     def func():
>         pass
>
> vs.
>
>     def func() [decorator]:
>         pass
>
> I'm beginning to think Guido has staged an elaborate April Fool's joke.
>
> <hopeful wink>

+1

-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From paul at prescod.net  Thu Apr  1 02:11:20 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Apr  1 02:13:37 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200403312332.i2VNWpe15013@guido.python.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<c4dbo8$ddh$1@sea.gmane.org>
	<2CF5B513-82EB-11D8-BD7A-0003934AD54A@chello.se>
	<200403311542.i2VFgWf13002@guido.python.org>
	<406B4FD2.1040506@prescod.net>
	<200403312332.i2VNWpe15013@guido.python.org>
Message-ID: <406BC098.7040209@prescod.net>

Guido van Rossum wrote:
>>>Why does <...> look better than [...]?  To me, <...> just reminds me
>>>of XML, which is totally the wrong association.
>>
>>I vote for << >>. The brackets and parens have too many meanings in 
>>Python already. <<staticmethod, foobar(baz)>> looks more like French 
>>than XML. ;)
> 
> 
> <<...>> has the same practical problems as <...>: no automatic line
> breaking, >> is ambiguous.

Sorry. I forgot that ">>" is also an operator. ;)

Anyhow, I just meant that if you find a pair of characters that are 
illegal today then many of the objections about abusing existing syntax 
would go away.

  Paul Prescod



From pyth at devel.trillke.net  Thu Apr  1 03:10:02 2004
From: pyth at devel.trillke.net (Holger Krekel)
Date: Thu Apr  1 03:10:39 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <8CF77A2E-8356-11D8-8201-000A95686CD8@redivi.com>
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331203701.GC6361@solar.trillke>
	<8CF77A2E-8356-11D8-8201-000A95686CD8@redivi.com>
Message-ID: <20040401081002.GD6361@solar.trillke>

Bob Ippolito wrote:
> On Mar 31, 2004, at 3:37 PM, Holger Krekel wrote:
> >Specifically, in PyPy we use something like
> >
> >    def add_List_List(self, w_list1, w_list2):
> >        ...
> >    def add_Int_Int(self, w_list1, w_list2):
> >        ...
> >
> >and later on just one:
> >
> >    register_all(vars(), W_ListType)
> >
> >(which will register the methods to some multimethod dispatcher).  It's
> >a matter of taste i guess if using decorators - whatever syntax - at  
> >each
> >function definition would be more readable.
> 
> Would you have even considered writing something as crazy as the  
> aforementioned example if decorators were already in Python?  I doubt  
> it.

Yes, we might have have used some decorator syntax.  But then again,
we also might use full Macros if they were available.  The question
(to me) is if there are enough use cases to warrant a new special 
decorator syntax.  

At least consider that Python's battery library only has ~10 
occurences of 'classmethod' and no 'staticmethod' at all.  

And let me add that IMO decorators generally make things harder to 
read because you have to wonder what magic takes place in those 
decorators.  IOW, i am not sure that seeing decorators occurring 
everywhere (because they become so convenient to write) will improve 
readability and debuggability of a python program. 

> >for example you could try to use the first line of the docstring
> >for it - maybe building on the typical way to document C-defined python
> >functions. e.g.:
> >
> >    "(some_type_signature) -> (some_return_type)"
> >
> >I am not saying straight away this is better but i doubt that the
> >only good solution to the above problem is to add new syntax.
> 
> Mangling *doc* strings is really stupid and is not even an option,  
> IMHO.  The only reason people do this is because we don't have a  
> decorator syntax.

Well, the above is (supposed to be) used for documenting the signature 
of C-functions.  Read "Documentation strings" in 

    http://www.python.org/peps/pep-0007.html

Anyway, I just happen to think that adding decorator syntax is not the only
viable solution to all problems reported so far. 

> >>If decorators do not make
> >>Python 2.4, that is another major release cycle that extensions such  
> >>as
> >>PyObjC and ctypes will be hampered by lack of syntax... to the point
> >>where I'd be inclined to just fork Python (or Stackless, more likely)
> >>in order to get the syntax.
> >
> >Nah, just work with us on PyPy to allow it to add decorator syntax
> >at runtime :-)
> 
> That'd be an option, if I could wait a few years until it's capable  
> enough to do everything it needs to do.

I am not sure it will take that long but i see your point :-)

cheers,

    holger

From mwh at python.net  Thu Apr  1 07:13:40 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Apr  1 07:13:47 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <20040331234312.GA14993@mems-exchange.org> (Neil Schemenauer's
	message of "Wed, 31 Mar 2004 18:43:12 -0500")
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
Message-ID: <2mekr7hp57.fsf@starship.python.net>

Neil Schemenauer <nas-python@python.ca> writes:

> On Wed, Mar 31, 2004 at 03:04:18PM -0500, Bob Ippolito wrote:
>> On Mar 31, 2004, at 1:59 PM, Michel Pelletier wrote:
>> >Please don't add any decorator syntax to Python, at least not yet.  All
>> >of the proposals I have seen so far are, to be blunt, and in my opinion
>> >of course, ugly and are getting uglier as the discussion ensues.
>
> I agree with Michel.  The decorator syntax being discussed looks
> ugly.  I think it would be okay if the set of valid decorations were
> limited to 'classmethod', 'staticmethod' and maybe a few others.
> Allowing more general expressions seems to asking for abuse.

I humbly submit that that's a terrible idea: it knackers Bob's PyObjC
example, for one thing.

>> Decorators solve a *huge* problem with the current syntax:
>> 
>> def someObjectiveCSelector_yesTheyCanBeThisLong_sometimesLonger_(takes,  
>> some, args, here):
>>     pass
>> someObjectiveCSelector_yesTheyCanBeThisLong_sometimesLonger_ =  
>> objc.selector(someObjectiveCSelector_yesTheyCanBeThisLong_sometimesLonge 
>> r_, signature='some type signature')
>
> I would be happer if there was an easier way for you to do what you
> want _without_ introducing new syntax to that language.  For
> example, what if '_' was bound to the last defined function?  You
> could then do something like this:
>
>   def someObjectiveCSelector_itsReallyLong_(takes, some, args, here):
>       pass
>   objc.selector(_, signature='some type signature')
>
> That looks pretty nice and is even shorter to type than the proposed
> syntax.

And doesn't work.  Oh well!

>> Please understand that just because you haven't need them yet doesn't  
>> make them worthless, ugly, etc.
>
> I don't think Michel is saying they are worthless.  However, the
> proposed syntax is highly contentious.  It would be good if there
> was a short term solution that wouldn't require new syntax.  That
> would give Guido and the Python community time to figure out the
> best syntax.

We've been discussing this off and on for OVER A YEAR!  If 'the best
syntax' hasn't been figured out yet after N thousand emails on the
subject, I see no reason to believe enlightenment is going to arrive
soon (or ever).

Cheers,
mwh

-- 
  If trees could scream, would we be so cavalier about cutting them
  down? We might, if they screamed all the time, for no good reason.
                                                        -- Jack Handey

From FBatista at uniFON.com.ar  Thu Apr  1 08:24:51 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr  1 08:26:53 2004
Subject: [Python-Dev] Expert floats
Message-ID: <A128D751272CD411BC9200508BC2194D03383823@escpl.tcp.com.ar>

Greg Ewing wrote:

#- > Decimal floating-point has almost all the pitfalls of binary
#- > floating-point, yet I do not see anyone arguing against decimal
#- > floating-point on the basis that it makes the pitfalls less
#- > apparent.
#- 
#- But they're not the pitfalls at issue here. The pitfalls at
#- issue are the ones due to binary floating point behaving
#- *differently* from decimal floating point.
#- 
#- Most people's mental model of arithmetic, including floating
#- point, works in decimal. They can reason about it based on

This is a problem for future, outside of this world, portability. 

When an alien, with, say, twelve fingers in each hand, uses Python? The
standard fp may be will still be the binary one, so they can make a
Twodecimal.py (I damn my english).

:)

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040401/e1118d43/attachment.html
From skip at pobox.com  Thu Apr  1 08:43:22 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  1 08:43:36 2004
Subject: [Python-Dev] PEP 318: Decorators last before colon
In-Reply-To: <1080790860.21196.23.camel@geddy.wooz.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<1080790860.21196.23.camel@geddy.wooz.org>
Message-ID: <16492.7290.766907.377725@montanaro.dyndns.org>

    >> > [decorator] 
    >> > def func(arg, arg):

    Barry> I don't like it.  It already has a meaning (albeit fairly
    Barry> useless) and it doesn't seem obvious from just looking at it that
    Barry> the decorator is connected to the following method.  It doesn't
    Barry> taste Pythonic to me.

I'm with Barry.  It seems magic to me.  If I write

    def f():
        "doc"
        [47]
        return 3

today, although a bit weird, [47] doesn't affect the following statement in
any way.  Now the proposal (still, assuming this isn't an elaborate AFJ) on
the table means to change the semantics of an unassigned list expression in
one special case.  I suggest:

    Special cases aren't special enough to break the rules.

though I know someone will follow with

    Although practicality beats purity.

IN this case there are other practical proposals on the table, not the least
of which is the status quo.

Skip

From skip at pobox.com  Thu Apr  1 08:46:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  1 08:46:48 2004
Subject: [Python-Dev] PEP 318: Decorators last before colon
In-Reply-To: <1080794825.22892.113.camel@localhost.localdomain>
References: <200404010430.i314UGCj017063@cosc353.cosc.canterbury.ac.nz>
	<1080794825.22892.113.camel@localhost.localdomain>
Message-ID: <16492.7475.436062.172933@montanaro.dyndns.org>


    Jeremy> PS Unspeakably horrible is a minor objection on the Internet.
    Jeremy> It's not really bad unless its untype-ably bad.

I take it the test for "untype-ably bad" is you check to see if you have to
wash your keyboard (or buy a new one) after typing such a construct. <wink>

Skip

From bob at redivi.com  Thu Apr  1 09:53:20 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Apr  1 09:49:18 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <20040401081002.GD6361@solar.trillke>
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331203701.GC6361@solar.trillke>
	<8CF77A2E-8356-11D8-8201-000A95686CD8@redivi.com>
	<20040401081002.GD6361@solar.trillke>
Message-ID: <50F44CDC-83EC-11D8-8201-000A95686CD8@redivi.com>

On Apr 1, 2004, at 3:10 AM, Holger Krekel wrote:

> Bob Ippolito wrote:
>> On Mar 31, 2004, at 3:37 PM, Holger Krekel wrote:
>>> Specifically, in PyPy we use something like
>>>
>>>    def add_List_List(self, w_list1, w_list2):
>>>        ...
>>>    def add_Int_Int(self, w_list1, w_list2):
>>>        ...
>>>
>>> and later on just one:
>>>
>>>    register_all(vars(), W_ListType)
>>>
>>> (which will register the methods to some multimethod dispatcher).  
>>> It's
>>> a matter of taste i guess if using decorators - whatever syntax - at
>>> each
>>> function definition would be more readable.
>>
>> Would you have even considered writing something as crazy as the
>> aforementioned example if decorators were already in Python?  I doubt
>> it.
>
> Yes, we might have have used some decorator syntax.  But then again,
> we also might use full Macros if they were available.  The question
> (to me) is if there are enough use cases to warrant a new special
> decorator syntax.

Well, I would be happy with full macros too.. but I'm not crazy enough 
to propose adding them :)

> At least consider that Python's battery library only has ~10
> occurences of 'classmethod' and no 'staticmethod' at all.

That's a bogus argument, using decorators is currently such a PITA that 
people avoid them, and not much of the standard library was written 
after the release of 2.2.

> And let me add that IMO decorators generally make things harder to
> read because you have to wonder what magic takes place in those
> decorators.  IOW, i am not sure that seeing decorators occurring
> everywhere (because they become so convenient to write) will improve
> readability and debuggability of a python program.

It's not really about readability and debuggability, this is a case 
where the information is absolutely necessary for the program to work 
at all.. the information needs to be there, right up against that def, 
or else it's not readable and hard to debug.  Putting it in a doc 
string and having a metaclass understand how to parse it is not more 
readable and is harder to debug.

>>> for example you could try to use the first line of the docstring
>>> for it - maybe building on the typical way to document C-defined 
>>> python
>>> functions. e.g.:
>>>
>>>    "(some_type_signature) -> (some_return_type)"
>>>
>>> I am not saying straight away this is better but i doubt that the
>>> only good solution to the above problem is to add new syntax.
>>
>> Mangling *doc* strings is really stupid and is not even an option,
>> IMHO.  The only reason people do this is because we don't have a
>> decorator syntax.
>
> Well, the above is (supposed to be) used for documenting the signature
> of C-functions.  Read "Documentation strings" in

I am *not talking about documentation*!  This is *code*!  It tells the 
bridge how to mangle the input and output of the method when it is 
crossing the Objective C <-> Python bridge!  Without these selectors, 
the code does not work at all, because it needs to convert objects into 
C types (int, float, struct, etc.) and vice versa in many cases.. 
sometimes even worse (pointers, yay!).

-bob


From barry at python.org  Thu Apr  1 10:19:53 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr  1 10:19:59 2004
Subject: [Python-Dev] PEP 318: Decorators last before colon
In-Reply-To: <16492.7290.766907.377725@montanaro.dyndns.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<1080790860.21196.23.camel@geddy.wooz.org>
	<16492.7290.766907.377725@montanaro.dyndns.org>
Message-ID: <1080832792.1021.82.camel@anthem.wooz.org>

On Thu, 2004-04-01 at 08:43, Skip Montanaro wrote:

> IN this case there are other practical proposals on the table, not the least
> of which is the status quo.

So I hacked up python-mode a bit to support the syntax coloring of
Guido's previously MFS (most favored syntax).  I wanted to see if the
concerns about visual obscurity were real or not.  Then I rewrote a few
methods of mine that would benefit from decorators (with some
elaboration).  A screen shot of my XEmacs buffer is here:

http://barry.warsaw.us/xemacs.png

Now, I'm sure most of you will hate my color scheme, but I'm used to it
so for me, the decorator stands out perfectly fine.  I'd have no problem
supporting decorator-before-colon and vastly prefer it to
decorator-before-def.

You can play with this yourself by checking out the latest version of
python-mode.el (from sf.net/projects/python-mode).  Note that you won't
get the nice colorization on multiline decorators on the fly because
XEmacs doesn't handle as-you-type font-locking across newlines well. 
Re-fontifying the buffer will pick up the right colors though.

-Barry



From jeremy at alum.mit.edu  Thu Apr  1 10:24:02 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Apr  1 10:27:59 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <2mekr7hp57.fsf@starship.python.net>
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
Message-ID: <1080833042.22892.122.camel@localhost.localdomain>

On Thu, 2004-04-01 at 07:13, Michael Hudson wrote:
> > I don't think Michel is saying they are worthless.  However, the
> > proposed syntax is highly contentious.  It would be good if there
> > was a short term solution that wouldn't require new syntax.  That
> > would give Guido and the Python community time to figure out the
> > best syntax.
> 
> We've been discussing this off and on for OVER A YEAR!  If 'the best
> syntax' hasn't been figured out yet after N thousand emails on the
> subject, I see no reason to believe enlightenment is going to arrive
> soon (or ever).

There's no particular reason to believe that effort alone will arrive at
an elegant solution.  On the other hand, maybe there isn't a good syntax
for arbitrary decorators.

Jeremy




From guido at python.org  Thu Apr  1 10:44:39 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 10:44:45 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Wed, 31 Mar 2004 23:11:20 PST."
	<406BC098.7040209@prescod.net> 
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<c4dbo8$ddh$1@sea.gmane.org>
	<2CF5B513-82EB-11D8-BD7A-0003934AD54A@chello.se>
	<200403311542.i2VFgWf13002@guido.python.org>
	<406B4FD2.1040506@prescod.net>
	<200403312332.i2VNWpe15013@guido.python.org> 
	<406BC098.7040209@prescod.net> 
Message-ID: <200404011544.i31Fiea17172@guido.python.org>

> Anyhow, I just meant that if you find a pair of characters that are 
> illegal today then many of the objections about abusing existing syntax 
> would go away.

So how about *[...]* ?  Or perhaps [*...*]?  Or [|...|]?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Apr  1 10:56:40 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr  1 10:56:49 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404011544.i31Fiea17172@guido.python.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<c4dbo8$ddh$1@sea.gmane.org>
	<2CF5B513-82EB-11D8-BD7A-0003934AD54A@chello.se>
	<200403311542.i2VFgWf13002@guido.python.org>
	<406B4FD2.1040506@prescod.net>
	<200403312332.i2VNWpe15013@guido.python.org>
	<406BC098.7040209@prescod.net>
	<200404011544.i31Fiea17172@guido.python.org>
Message-ID: <1080834999.1021.119.camel@anthem.wooz.org>

On Thu, 2004-04-01 at 10:44, Guido van Rossum wrote:
> > Anyhow, I just meant that if you find a pair of characters that are 
> > illegal today then many of the objections about abusing existing syntax 
> > would go away.
> 
> So how about *[...]* ?  Or perhaps [*...*]?  Or [|...|]?

I'm -0 on whether it's necessary, but of these choices the latter seems
okay.

-Barry



From tjreedy at udel.edu  Thu Apr  1 11:16:31 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu Apr  1 11:16:30 2004
Subject: [Python-Dev] Re: Re: PEP 318: Decorators last before colon
References: <42B02151.420DCBB8@mail.google.com>
Message-ID: <c4hf8o$7lu$1@sea.gmane.org>


"Peter Norvig" <pnorvig@google.com> wrote in message
news:42B02151.420DCBB8@mail.google.com...
> I have to say I favor the "last before colon" approach,

The way I see it today, we are trying to select syntax to define something
like a 'function metatype' (FMT) in order to get non-standard function-like
objects.  The idea and result strike me as similar (but *not* identical) to
running the pieces of a would-be class thru a metaclass.  If FMTs were
required to be defined and named before use, then [FMTidentifier] would be
analogous to __metaclass__ = MCidentifier after a class statement.  While a
metaclass can be defined in-place, over multiple lines, this does not work
nearly so well for an anonymous FMT.

(Has any consideration been given to an actual metafunc mechanism more
directly analogous to metaclasses, that would be given the *pieces* of a
would-be function (name, param names, default args, code body, etc), so
that there would not necessarily ever be a standard function object?)

> but if it has to be before the def, then I think it should have a
keyword,

Yes.  My support for 'as' was based on misremembering its quasi status.

> and if
> you don't want to introduce a new keyword, then it looks like "is" is
> the only reasonable candidate.  And if you do have a keyword, you
> don't need the square brackets.

Since the sequence is not going to be mutated, I would see them as a
positive distraction.

>  So you have
>
> is: classmethod
> def f(x):
>     pass
>
>
> is: author("Guido"), signature(int, result=None)
> def g(x):
>     pass

For this usage, 'is' is semantically better than 'as' anyway.

Terry J. Reedy




From s.percivall at chello.se  Thu Apr  1 11:24:25 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Thu Apr  1 11:24:30 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404011544.i31Fiea17172@guido.python.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<c4dbo8$ddh$1@sea.gmane.org>
	<2CF5B513-82EB-11D8-BD7A-0003934AD54A@chello.se>
	<200403311542.i2VFgWf13002@guido.python.org>
	<406B4FD2.1040506@prescod.net>
	<200403312332.i2VNWpe15013@guido.python.org>
	<406BC098.7040209@prescod.net>
	<200404011544.i31Fiea17172@guido.python.org>
Message-ID: <0A6B20B6-83F9-11D8-AF78-0003934AD54A@chello.se>

On 2004-04-01, at 17.44, Guido van Rossum wrote:

>> Anyhow, I just meant that if you find a pair of characters that are
>> illegal today then many of the objections about abusing existing 
>> syntax
>> would go away.
>
> So how about *[...]* ?  Or perhaps [*...*]?  Or [|...|]?

Also: Is it definite that it should be square brackets instead of 
parentheses?

---
(sigh ... mail problems)


From pje at telecommunity.com  Thu Apr  1 11:26:00 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr  1 11:26:07 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: <200404010544.i315iiB15938@guido.python.org>
References: <Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com>
Message-ID: <5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>

At 09:44 PM 3/31/04 -0800, Guido van Rossum wrote:

>What I'm asking (especially of Phillip) is to collect a set of
>realistic method declarations using decorators; we can then
>collectively format these using any of the possible syntaxes, and see
>how they look.

I'd be happy to scrounge up some samples from existing code using 
'property' and 'classmethod' as well as some of PEAK's decorators, and I 
definitely think that Jack Diedrich and Bob Ippolito's samples should be 
included as well.

Important question, though: do we include code bodies, or just use 'pass' 
for the bodies?  If we include the body, how much of the body?  Should we 
include entire classes, especially if the class itself needs a decorator, 
and multiple methods have decorators?

Next, does anybody have any actual use cases for attribute decoration 
today?  We're probably not going to get a lot of that from current code 
samples.  I can make up some examples that throw in every possible option 
that PEAK provides in order to get some "lots of decoration" samples, but 
they wouldn't be "real" uses in that case.  But I guess that maybe Bob's 
examples might be wordy enough.


From guido at python.org  Thu Apr  1 11:36:15 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 11:36:19 2004
Subject: [Python-Dev] Re: Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Thu, 01 Apr 2004 11:16:31 EST."
	<c4hf8o$7lu$1@sea.gmane.org> 
References: <42B02151.420DCBB8@mail.google.com>  <c4hf8o$7lu$1@sea.gmane.org> 
Message-ID: <200404011636.i31GaFg17456@guido.python.org>

> (Has any consideration been given to an actual metafunc mechanism
> more directly analogous to metaclasses, that would be given the
> *pieces* of a would-be function (name, param names, default args,
> code body, etc), so that there would not necessarily ever be a
> standard function object?)

Deconstructing a function like that is too invasive -- I don't want to
touch the calling sequence, for example, because it's so performance
critical.  None of the people arguing for decorators has shown a use
case for that either.  However, if you really want to do that, you
*can* take the function apart and construct a new one using the 'new'
module.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From evan at 4-am.com  Thu Apr  1 11:47:21 2004
From: evan at 4-am.com (Evan Simpson)
Date: Thu Apr  1 11:48:49 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <200404010544.i315iiB15938@guido.python.org>
References: <42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org>
Message-ID: <406C4799.6030502@4-am.com>

Guido van Rossum wrote:
> 2a) Prefix list with leading *:
> 
>    *[funcattrs(foo=42), deprecated, overrides, classmethod]
>    def foo(cls, arg1, arg2):
>        pass

How about adding:

2b) Prefix list with repeated keyword:

def [funcattrs(foo=42), deprecated, overrides, classmethod]
def foo(cls, arg1, arg2):
     pass

class [singleton]
class foo:
     pass

3a) Prefix suite with repeated keyword:

def:
     funcattrs(foo=42)
     deprecated, overrides, classmethod
def foo(cls, arg1, arg2):
     pass

Cheers,

Evan @ 4-am



From edloper at gradient.cis.upenn.edu  Thu Apr  1 11:56:39 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Thu Apr  1 11:55:12 2004
Subject: [Python-Dev] PEP 318 -- a couple use cases
Message-ID: <406C49C7.80304@gradient.cis.upenn.edu>

I saw both of the following use cases mentioned on the list, and they 
seemed interesting, so I went ahead and wrote up implementations:

def property_getter(func):
     """
     A decorator that can be used to incrementally construct
     properties.  For example, the following code constructs a
     property 'x' from an fget, an fset, and an fdel function:

         >>> class A:
         ...     def x(self) [property_getter]:
         ...         return self.__x
         ...     def x(self, val) [property_setter]:
         ...         self.__x = val
         ...     def x(self) [property_deleter]:
         ...         del self.__x

     In particular, this decorator checks if a property named
     'func' is defined in the enclosing frame.  If so, then it
     updates that property's fget to be 'func'.  If not, then
     it creates a new property whose fget is 'func'.  The
     property's docstring is taken from the first decorated
     property function to define a docstring.
     """

def generic(*type_signature):
     """
     A decorator-generator that can be used to incrementally construct
     a generic function that delegates to individual functions based on
     the type signature of the arguments.  For example, the following
     code defines a generic function that uses two different actual
     functions, depending on whether its argument is a string or an
     int:

         >>> def f(x) [generic(int)]:
         ...     print x, 'is an int'
         >>> def f(x) [generic(str)]:
         ...     print x, 'is a string'

     In particular, the decorator returned by this function checks if a
     Generic function-delegation object with the same name as the
     decorated function is defined in the enclosing frame.  If so, then
     it registers the function with that generic object, under
     'type_signature'.  If not, then it creates a new Generic
     function-delegation object, and registers the function.
     """

Full code is at <http://www.cis.upenn.edu/~edloper/pydecorators.html>.

But I still don't feel like I have a good handle on what "acceptable" 
uses of decorators are..  So, for each of these 2 use cases, is it...
     - an excellent example that should be included in the stdlib
     - perfectly acceptable, and belongs in the python cookbook
     - somewhat hackish, but ok under the right circumstances
     - an abhorition of nature

(If one of the first 3, then maybe they should be added to the pep?) 
Thanks for the feedback.

-Edward


From mike at nospam.com  Thu Apr  1 12:10:04 2004
From: mike at nospam.com (Mike Rovner)
Date: Thu Apr  1 12:10:14 2004
Subject: [Python-Dev] Re: ANNOUNCE: 'goto' for Python
References: <n98m60drsk5ep9ugq6rh6uvg4jdnjtkihh@4ax.com>
Message-ID: <c4hidb$hfd$1@sea.gmane.org>

> 'goto' example: breaking out from a deeply nested loop:
>            goto .end
mark that space--^

Shall I file a bug about python lexer which allows space between class and
attribute? Or that's a fiature?

Regards,
Mike




From michel at dialnetwork.com  Thu Apr  1 12:08:28 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Thu Apr  1 12:27:14 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <E1B8r42-0002bc-JK@mail.python.org>
References: <E1B8r42-0002bc-JK@mail.python.org>
Message-ID: <1080839290.2036.29.camel@heinlein>


I apologize for not changing my digest subjects in earlier messages.

> Message: 4
> Date: Wed, 31 Mar 2004 19:16:06 -0500
> From: "Phillip J. Eby" <pje@telecommunity.com>
> Subject: Re: [Python-Dev] Re: PEP 318: Decorators last before colon
> To: Neil Schemenauer <nas-python@python.ca>, python-dev@python.org
> Message-ID: <5.1.1.6.0.20040331184627.01e86a10@telecommunity.com>
> Content-Type: text/plain; charset="us-ascii"; format=flowed
> 
> Ugly or not, *all* of the proposed syntaxes that don't involve creating an 
> extra suite have the practical effect of improving the semantic readability 
> of decorator use over today's Python.  In addition, they also have the 
> practical effect of making decorator use more convenient, since the two 
> extra repetitions of the function or class name go away.

Agreed. 

> I suspect this is part of why there is such disagreement on the subject of 
> decorators: people who make heavy use of them today know exactly what 
> problems today's syntax has from both the readability and writability 
> viewpoints.  Whereas people who do not use them, don't get why limiting 
> their use to fixed subsets, or single decorators, or any number of other 
> ideas just negate the usefulness of having a syntax in the first 
> place.  Or, like Michel, they don't see a point to having a syntax at 
> all.

I'd like to clarify that I'm not against a new syntax (I'd be more than
a bit hypocritical having written two PEPs proposing new syntax!).  My
original comment was in fact that the line-preceding-def was ugly, not
pointless. 

My second comment is that I wonder if decoration of any syntax has been
thought out in the light of other objects being decorated, like classes,
functions, or interfaces.  I just feel like there's more thought to be
applied here on decoration syntax in general.  When we take decoration
to the next level, will this syntax suffice?  Will it be abused for
reasons not considered, like run-time type checking?  I'm not positive
but it seems this is what Bob I. wants it for.

And have we considered all the options?  I'm not making any proposals,
but perhaps decoration can be bundled into a separate object that a
class refers to, similar to interfaces.  This could allow a lot more
flexibility and still look like Python.    Or perhaps this separate
object could be mixed in (blech).  But you see my point, the upshot is
you could apply different decorations at different times, decorators
would be separately managed and it wouldn't look like a shark fin duct
taped to a VW bug.

But having said that, I concede: I know that some people make a lot of
redundant finger movements working around the lack of decorations. 
Phillip is right that I don't use decorators (I have used classmethod
quite a bit, but that's all) and that he does, so his needs outweigh
mine.  So I cave.  See below for my vote.

Later, Barry said:

> So I hacked up python-mode a bit to support the syntax coloring of
> Guido's previously MFS (most favored syntax).  I wanted to see if the
> concerns about visual obscurity were real or not.  Then I rewrote a
> few
> methods of mine that would benefit from decorators (with some
> elaboration).  A screen shot of my XEmacs buffer is here:
> 
> http://barry.warsaw.us/xemacs.png

Mmm.. I like your colors, it reminds me of a scheme Ken M. shared with
me once.  I've long lost it, along with his handy pop-to-shell script.

Later, Guido said:

> 3) Prefix suite (could use a different keyword than 'decorate'):
> 
>    decorate:
>        funcattrs(foo=42)
>        deprecated
>        overrides
>        classmethod
>    def foo(cls, arg1, arg2):
>        pass


I don't really like it, but it's the only one that looks like Python to
me.  Otherwise, I vote last-before-colon.  Line-preceding-def would be
my last choice.

-Michel




From barry at python.org  Thu Apr  1 12:30:53 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr  1 12:31:05 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1080839290.2036.29.camel@heinlein>
References: <E1B8r42-0002bc-JK@mail.python.org>
	<1080839290.2036.29.camel@heinlein>
Message-ID: <1080840653.3054.8.camel@anthem.wooz.org>

On Thu, 2004-04-01 at 12:08, Michel Pelletier wrote:

> Mmm.. I like your colors, it reminds me of a scheme Ken M. shared with
> me once.  I've long lost it, along with his handy pop-to-shell script.

That probably shouldn't be surprising. :)

-Barry



From ark-mlist at att.net  Thu Apr  1 12:38:21 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Thu Apr  1 12:38:18 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404011544.i31Fiea17172@guido.python.org>
Message-ID: <00f001c41810$20afc4f0$6402a8c0@arkdesktop>

> > Anyhow, I just meant that if you find a pair of characters that are
> > illegal today then many of the objections about abusing existing syntax
> > would go away.
> 
> So how about *[...]* ?  Or perhaps [*...*]?  Or [|...|]?

How about INDENT[...]:DEDENT?  <0.5 serious>


From bob at redivi.com  Thu Apr  1 12:48:56 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Apr  1 12:44:51 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1080839290.2036.29.camel@heinlein>
References: <E1B8r42-0002bc-JK@mail.python.org>
	<1080839290.2036.29.camel@heinlein>
Message-ID: <D90AB4B6-8404-11D8-8B4D-000A95686CD8@redivi.com>

On Apr 1, 2004, at 12:08 PM, Michel Pelletier wrote:

> My second comment is that I wonder if decoration of any syntax has been
> thought out in the light of other objects being decorated, like 
> classes,
> functions, or interfaces.  I just feel like there's more thought to be
> applied here on decoration syntax in general.  When we take decoration
> to the next level, will this syntax suffice?  Will it be abused for
> reasons not considered, like run-time type checking?  I'm not positive
> but it seems this is what Bob I. wants it for.

It's not run-time type *checking*.. it's run-time type *bridge* between 
the Python and Objective C runtimes.  Without these decorators (and 
other FFI machinery), the bridge does not work, because a PyObject* is 
not a struct _NSRect or a NSObject*, etc.  It's essentially the same 
kind of thing as ctypes, except the Objective C runtime has a lot more 
RTTI available than the regular 'ol C runtime and it also uses 
reference counting so it's more safe and sane.  ctypes probably needs 
decorator syntax more than PyObjC does, but the brave few have hobbled 
along without it :)

> And have we considered all the options?  I'm not making any proposals,
> but perhaps decoration can be bundled into a separate object that a
> class refers to, similar to interfaces.  This could allow a lot more
> flexibility and still look like Python.    Or perhaps this separate
> object could be mixed in (blech).  But you see my point, the upshot is
> you could apply different decorations at different times, decorators
> would be separately managed and it wouldn't look like a shark fin duct
> taped to a VW bug.

In my use case it doesn't really make sense to keep them separate, as 
the functions are broken without their decorator because they can not 
be called from the Objective C side of the bridge.  The only time you 
need to use these decorators are when you are creating a class that 
complies with an informal protocol, whose selectors have non-object 
type signatures (returning void or BOOL is exceedingly common, as is 
taking integers, and sometimes nasty pointer stuff).  In any case, it's 
actually really common with AppKit and Foundation because most of it is 
event driven and these informal protocols are part of the delegate 
callback machinery for everything from URL loading to printing.

-bob


From pje at telecommunity.com  Thu Apr  1 12:47:11 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr  1 12:47:20 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1080839290.2036.29.camel@heinlein>
References: <E1B8r42-0002bc-JK@mail.python.org>
	<E1B8r42-0002bc-JK@mail.python.org>
Message-ID: <5.1.1.6.0.20040401124254.02c2acd0@telecommunity.com>

At 09:08 AM 4/1/04 -0800, Michel Pelletier wrote:
>Later, Guido said:
>
> > 3) Prefix suite (could use a different keyword than 'decorate'):
> >
> >    decorate:
> >        funcattrs(foo=42)
> >        deprecated
> >        overrides
> >        classmethod
> >    def foo(cls, arg1, arg2):
> >        pass
>
>
>I don't really like it, but it's the only one that looks like Python to
>me.  Otherwise, I vote last-before-colon.  Line-preceding-def would be
>my last choice.

There is another option:

     as [classmethod]
     def blah(cls):
         ...

     as [singleton]
     class SignalManager:
         ...

     as [implementorOf(IFoo, forTypes=[Bar])]
     class FooToBarAdapter:
         ...

and it could also be on the same line, e.g.:

     as [classmethod] def blah(cls):
         ...

It reads a bit better with function definitions than with classes, but it's 
not too bad.  It's more of a replacement for the *[decorators] syntax than 
the 'decorate:' syntax, but it's an English word rather than a symbol, 
which brings back a bit more of the "Python look and feel".


From skip at pobox.com  Thu Apr  1 13:05:59 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  1 13:06:29 2004
Subject: [Python-Dev] Re: Re: PEP 318: Decorators last before colon
In-Reply-To: <200404011636.i31GaFg17456@guido.python.org>
References: <42B02151.420DCBB8@mail.google.com> <c4hf8o$7lu$1@sea.gmane.org>
	<200404011636.i31GaFg17456@guido.python.org>
Message-ID: <16492.23047.867494.68059@montanaro.dyndns.org>


    >> (Has any consideration been given to an actual metafunc mechanism
    >> more directly analogous to metaclasses, that would be given the
    >> *pieces* of a would-be function (name, param names, default args,
    >> code body, etc), so that there would not necessarily ever be a
    >> standard function object?)

    Guido> Deconstructing a function like that is too invasive -- I don't
    Guido> want to touch the calling sequence, for example, because it's so
    Guido> performance critical.  None of the people arguing for decorators
    Guido> has shown a use case for that either.  However, if you really
    Guido> want to do that, you *can* take the function apart and construct
    Guido> a new one using the 'new' module.

One thing that occurred to me is that a function's func_name attribute might
be made read-write so that decorator functions can easily make that aspect
of a function wrapper behave the same as the original function.  Calling
new.function() shouldn't be required in such simple cases.

Skip

From michel at dialnetwork.com  Thu Apr  1 13:07:35 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Thu Apr  1 13:23:24 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <5.1.1.6.0.20040401124254.02c2acd0@telecommunity.com>
References: <E1B8r42-0002bc-JK@mail.python.org>
	<E1B8r42-0002bc-JK@mail.python.org>
	<5.1.1.6.0.20040401124254.02c2acd0@telecommunity.com>
Message-ID: <1080842227.2804.7.camel@heinlein>

On Thu, 2004-04-01 at 09:47, Phillip J. Eby wrote:

>      as [implementorOf(IFoo, forTypes=[Bar])]
>      class FooToBarAdapter:
>          ...
> 
> and it could also be on the same line, e.g.:
> 
>      as [classmethod] def blah(cls):
>          ...
> 
> It reads a bit better with function definitions than with classes, but it's 
> not too bad.

Agreed, I missed this one.  It's not too bad with classes if you can
warp your mind to think of "to class" as a verb like "to define".

>   It's more of a replacement for the *[decorators] syntax than 
> the 'decorate:' syntax, but it's an English word rather than a symbol, 
> which brings back a bit more of the "Python look and feel".

One minor nit, why use the brackets at all?

  as sychronized, sharkfin(size=10) def VWBug(...):

that looks better to me than:

  as [sychronized, sharkfin(size=10)] def VWBug(...):

without brackets it looks more to me like Python, you'd never see:

  from foo import [bar, baz]

For longer lists of decorators use the familiar tuple rule, wrap in
parens:

  as (sychronized,
      sharkfin(size=10),
      deprecated) def VWBug(...):

-Michel


From jim.jewett at eds.com  Thu Apr  1 13:42:34 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu Apr  1 13:43:06 2004
Subject: [Python-Dev] PEP 318: Decorators last before colon/bakeoff
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D429@USAHM010.amer.corp.eds.com>

>> Typically the suite (things indented after the colon) are all
>> of the same type.

Josiah Carlson:

> Funny, but this isn't the case with Python anywhere else; I can have
> functions, class and variable definitions inside any other suite.

I phrased that badly.  

The statements following a suite are all interchangable from the *suite's*
perspective.  If not, then a second header clause (e.g. "else:") is added.
If a particular expression is special, it comes before the ":" within
the header.  (x, y in "for x in y:")

Docstrings are arguably an exception, but they get away with it because
they evaluate to themselves, without a side-effect.  If you remove their
special status, programs don't change their behavior.  (Which is why
optimized code can just remove them.)

Decorators behave differently from other statements, and this should be
clear.

Everything in Guido's bakeoff meets this requirement.  Option 2 *might*
cause confusion with normal lists, which is why he also listed 2.a.

(2)  Individually fine, but the tie between the list and def is week.

    [d1, d2, d3]
    def func (args):
        pass

(2a) Also warns you that the list is strange, but doesn't say how.

    *[d1, d2, d3] 
    def func (args):
        pass

(2b) Explicitly relates the list to the def, but does add a keyword.
     On the other hand, adding this keyword makes it slightly less
     difficult to extend the syntax again, if that ever becomes
     desirable.

    [d1, d2, d3] of 
    def func (args):
        pass

-jJ

From goodger at python.org  Thu Apr  1 13:54:30 2004
From: goodger at python.org (David Goodger)
Date: Thu Apr  1 13:54:35 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
References: <Your message of "Wed, 31 Mar 2004 21:00:55 PST."
	<42B02151.420DCBB8@mail.google.com>	<42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
Message-ID: <406C6566.3040104@python.org>

Phillip J. Eby wrote:
> Next, does anybody have any actual use cases for attribute decoration 
> today?

If you mean function attributes, Docutils uses them a lot.
They're described in
<http://docutils.sf.net/docutils/parsers/rst/directives/__init__.py>
and used in all the modules in that directory, such as
<docutils/docutils/parsers/rst/directives/images.py>.

-- David Goodger

From guido at python.org  Thu Apr  1 14:08:26 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 14:08:33 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: Your message of "Thu, 01 Apr 2004 11:26:00 EST."
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com> 
References: <Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com> 
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com> 
Message-ID: <200404011908.i31J8Qu17815@guido.python.org>

> >What I'm asking (especially of Phillip) is to collect a set of
> >realistic method declarations using decorators; we can then
> >collectively format these using any of the possible syntaxes, and see
> >how they look.
> 
> I'd be happy to scrounge up some samples from existing code using 
> 'property' and 'classmethod' as well as some of PEAK's decorators, and I 
> definitely think that Jack Diedrich and Bob Ippolito's samples should be 
> included as well.
> 
> Important question, though: do we include code bodies, or just use 'pass' 
> for the bodies?  If we include the body, how much of the body?  Should we 
> include entire classes, especially if the class itself needs a decorator, 
> and multiple methods have decorators?

Why not provide the bodies, for added realism?

(I still think class decorators are a separate case, and much weaker
-- you can do this by having a single 'decoratable' metaclass and
setting __decorators__ = [...] in the class body.)

> Next, does anybody have any actual use cases for attribute decoration 
> today?  We're probably not going to get a lot of that from current code 
> samples.  I can make up some examples that throw in every possible option 
> that PEAK provides in order to get some "lots of decoration" samples, but 
> they wouldn't be "real" uses in that case.  But I guess that maybe Bob's 
> examples might be wordy enough.

I think that SPARK syntax and everything else that people have
traditionally added to docstring markup that isn't strictly speaking
documentation (even some extreme cases of doctest usage) ought to be
considered as candidates for attribute-ification.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Apr  1 14:10:39 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr  1 14:10:46 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <406C6566.3040104@python.org>
References: <5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
Message-ID: <5.1.1.6.0.20040401140508.02d80090@telecommunity.com>

At 01:54 PM 4/1/04 -0500, David Goodger wrote:
>Phillip J. Eby wrote:
>>Next, does anybody have any actual use cases for attribute decoration today?
>
>If you mean function attributes, Docutils uses them a lot.
>They're described in
><http://docutils.sf.net/docutils/parsers/rst/directives/__init__.py>
>and used in all the modules in that directory, such as
><docutils/docutils/parsers/rst/directives/images.py>.

Thanks!  So for example, this:


def admonition(*args):
     return make_admonition(nodes.admonition, *args)

admonition.arguments = (1, 0, 1)
admonition.options = {'class': directives.class_option}
admonition.content = 1

def attention(*args):
     return make_admonition(nodes.attention, *args)

attention.content = 1


might be rephrased as (say):


as [rst_directive(
         arguments=(1,0,1),
         options={'class': directives.class_option},
         content=1
     )
] def admonition(*args):
     return make_admonition(nodes.admonition, *args)


as [rst_directive(content=1)]
def attention(*args):
     return make_admonition(nodes.attention, *args)


From guido at python.org  Thu Apr  1 14:16:33 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 14:16:40 2004
Subject: [Python-Dev] PEP 318 -- a couple use cases
In-Reply-To: Your message of "Thu, 01 Apr 2004 11:56:39 EST."
	<406C49C7.80304@gradient.cis.upenn.edu> 
References: <406C49C7.80304@gradient.cis.upenn.edu> 
Message-ID: <200404011916.i31JGXm17863@guido.python.org>

> I saw both of the following use cases mentioned on the list, and they 
> seemed interesting, so I went ahead and wrote up implementations:
> 
> def property_getter(func):
[...] 
> def generic(*type_signature):
[...]
> Full code is at <http://www.cis.upenn.edu/~edloper/pydecorators.html>.
> 
> But I still don't feel like I have a good handle on what "acceptable" 
> uses of decorators are..  So, for each of these 2 use cases, is it...
>      - an excellent example that should be included in the stdlib
>      - perfectly acceptable, and belongs in the python cookbook
>      - somewhat hackish, but ok under the right circumstances
>      - an abhorition of nature

I'm wavering between 2 (perfectly acceptable) and 3 (somewhat
hackish), only because I consider anything that uses sys._getframe()
to be a danger to society unless proven innocent.

I would probably prefer to do both of these using explicit, different
names.  In particular for properties, I *like* the fact that I can
also explicitly call the getter or setter functions, so I'd probably
continue to write those like this:

  class C(object):

     def getX(self): return self.__x
     def setX(self, x): self.__x = x
     x = property(getX, setX)

I'm not sufficiently comfortable with generic functions to quite know
what feels best there.  But I like the notation you proposed (I would
like it better with the decorators up front though).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr  1 14:21:12 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 14:21:20 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Thu, 01 Apr 2004 12:38:21 EST."
	<00f001c41810$20afc4f0$6402a8c0@arkdesktop> 
References: <00f001c41810$20afc4f0$6402a8c0@arkdesktop> 
Message-ID: <200404011921.i31JLCF17896@guido.python.org>

> How about INDENT[...]:DEDENT?  <0.5 serious>

Alas, won't work -- if the decoration is the first thing in a class
body, you'd have something starting with double indent, but the lexer
never gives you that.

class C:

        [staticmethod]
    def foo():
        pass

This would be tokenized as

'class'
'C'
':'
NEWLINE
INDENT
'['
'staticmethod'
']'
NEWLINE
<ILLEGAL DEDENT>

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bkc at murkworks.com  Thu Apr  1 14:28:28 2004
From: bkc at murkworks.com (Brad Clements)
Date: Thu Apr  1 14:27:03 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: <200404011908.i31J8Qu17815@guido.python.org>
References: Your message of "Thu, 01 Apr 2004 11:26:00
	EST."	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com> 
Message-ID: <406C26A3.28912.6817562C@localhost>

On 1 Apr 2004 at 11:08, Guido van Rossum wrote:

> I think that SPARK syntax and everything else that people have
> traditionally added to docstring markup that isn't strictly speaking
> documentation (even some extreme cases of doctest usage) ought to be
> considered as candidates for attribute-ification.

Where do method attribute type signatures and DBC fit in? As a decorator, or in the 
docstring?

I'm concerned that the funcattrs(a="xyz" .. ) sample tossed around here will be rather 
ugly for specifying DBC strings.

DBC will also need class invariants, so a funcattrs work-alike at the class level will be 
needed.

Finally,  I don't have a need to access DBC annotations at runtime once my module is 
distributed. I would not want to pay the memory cost overhead of loading DBC 
information or attribute type signatures at runtime.

However another person at PyCon poo-poo'd my concern over bloating .pyc files and 
subsequent memory use. As a compromise I suggested that "annotation" information 
could go into the .pyc, but be loaded "on demand" at runtime. For example, a traceback 
handler might be able to use DBC info to suggest the call level that may have caused 
the problem.

To do "on demand loading", I suggested giving .pyc files a "resource section" that could 
hold this meta information. Normal imports would not load the resource section, but 
resource information could be loaded later using another mechanism.

I thought that putting meta data into a different file was more complicated, and that 
using -OO to "strip out" annotation was too heavy handed. The standard library might 
benefit from the addition of method attribute type information. However I think it would 
be better to not load this information at runtime unless needed. 

The same could be said for docstrings. If docstrings were "packed" in a resource 
section of the .pyc file, they might be loaded "on-demand", thereby saving some 
memory overhead.


-- 
Brad Clements,                bkc@murkworks.com   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
http://www.wecanstopspam.org/                   AOL-IM: BKClements


From bob at redivi.com  Thu Apr  1 14:33:24 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Apr  1 14:29:19 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: <200404011908.i31J8Qu17815@guido.python.org>
References: <Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<200404011908.i31J8Qu17815@guido.python.org>
Message-ID: <712CE402-8413-11D8-8B4D-000A95686CD8@redivi.com>

On Apr 1, 2004, at 2:08 PM, Guido van Rossum wrote:

>>> What I'm asking (especially of Phillip) is to collect a set of
>>> realistic method declarations using decorators; we can then
>>> collectively format these using any of the possible syntaxes, and see
>>> how they look.
>>
>> I'd be happy to scrounge up some samples from existing code using
>> 'property' and 'classmethod' as well as some of PEAK's decorators, 
>> and I
>> definitely think that Jack Diedrich and Bob Ippolito's samples should 
>> be
>> included as well.
>>
>> Important question, though: do we include code bodies, or just use 
>> 'pass'
>> for the bodies?  If we include the body, how much of the body?  
>> Should we
>> include entire classes, especially if the class itself needs a 
>> decorator,
>> and multiple methods have decorators?
>
> Why not provide the bodies, for added realism?
>
> (I still think class decorators are a separate case, and much weaker
> -- you can do this by having a single 'decoratable' metaclass and
> setting __decorators__ = [...] in the class body.)

Here's something I wrote today.. it's a delegate for the exception 
handling mechanism so you can listen in on (any of) the exceptions that 
ObjC throws whether or not they are caught by something else.. the 
Python->PyObjC exceptions are logged as tracebacks and the ObjC 
exceptions are thrown over to atos so they turn into human-readable 
stack traces.

As you can see, it's rather ugly with regular old Python syntax.

class PyObjCDebuggingDelegate(NSObject):
     def exceptionHandler_shouldLogException_mask_(self, sender, 
exception, aMask):
         try:
             if isPythonException(exception):
                 if self.verbosity() & LOGSTACKTRACE:
                     nsLogObjCException(exception)
                 return nsLogPythonException(exception)
             elif self.verbosity() & LOGSTACKTRACE:
                 return nsLogObjCException(exception)
             else:
                 return False
         except:
             print >>sys.stderr, "*** Exception occurred during 
exception handler ***"
             traceback.print_exc(sys.stderr)
             return True
     exceptionHandler_shouldLogException_mask_ = 
objc.selector(exceptionHandler_shouldLogException_mask_, 
signature='c@:@@I')

     def exceptionHandler_shouldHandleException_mask_(self, sender, 
exception, aMask):
         return False
     exceptionHandler_shouldHandleException_mask_ = 
objc.selector(exceptionHandler_shouldHandleException_mask_, 
signature='c@:@@I')

the objc.selector signatures say that they return a char (a BOOL, 
actually), the following @: represents "self" and the selector (the 
"method name"), the next two @@ say that the sender and exception 
arguments are both ObjC objects, and the trailing I means that aMask is 
an unsigned int.

-bob


From aahz at pythoncraft.com  Thu Apr  1 14:47:08 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr  1 14:47:22 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404011544.i31Fiea17172@guido.python.org>
References: <Pine.LNX.4.58.0403300136280.18028@server1.LFW.org>
	<1080674306.12643.76.camel@localhost.localdomain>
	<200403302121.i2ULLIM09840@guido.python.org>
	<c4dbo8$ddh$1@sea.gmane.org>
	<2CF5B513-82EB-11D8-BD7A-0003934AD54A@chello.se>
	<200403311542.i2VFgWf13002@guido.python.org>
	<406B4FD2.1040506@prescod.net>
	<200403312332.i2VNWpe15013@guido.python.org>
	<406BC098.7040209@prescod.net>
	<200404011544.i31Fiea17172@guido.python.org>
Message-ID: <20040401194707.GB3825@panix.com>

On Thu, Apr 01, 2004, Guido van Rossum wrote:
> 
> So how about *[...]* ?  Or perhaps [*...*]?  Or [|...|]?

-0, +1, +0

(That last is mainly 'cause it's four pinky keystrokes, or I'd be +1 for
it, too.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From edloper at gradient.cis.upenn.edu  Thu Apr  1 14:57:09 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Thu Apr  1 14:55:40 2004
Subject: [Python-Dev] PEP 318 -- a couple use cases
In-Reply-To: <200404011916.i31JGXm17863@guido.python.org>
References: <406C49C7.80304@gradient.cis.upenn.edu>
	<200404011916.i31JGXm17863@guido.python.org>
Message-ID: <406C7415.5000005@gradient.cis.upenn.edu>

> I'm wavering between 2 (perfectly acceptable) and 3 (somewhat
> hackish), only because I consider anything that uses sys._getframe()
> to be a danger to society unless proven innocent.

Would these be preferable:?

     class C(object):
         [property]
         def x(self): return self.__x

         [property_setter(x)]
         def x(self, val): self.__x = val

         [property_deleter(x)]
         def x(self): del self.__x
and:
     [generic(None, (int,int))]
     def f(x,y): print 'two ints'

     [generic(f, (float,float))]
     def f(x,y): print 'two floats'

I.e., explicitly pass the old value in, rather than grabbing it from 
sys._getframe.

Advantages:
   - no sys._getframe magic
Disadvantages:
   - you have to type the property/function name twice
   - you have to do something special for the first def
   - you can't easily rearrange the defs

-Edward


From python at rcn.com  Thu Apr  1 15:47:26 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Apr  1 15:47:44 2004
Subject: [Python-Dev] Vote No on 318
In-Reply-To: <1080833042.22892.122.camel@localhost.localdomain>
Message-ID: <002a01c4182a$8b057420$7329cb97@oemcomputer>

[Michael]
> > We've been discussing this off and on for OVER A YEAR!  If 'the best
> > syntax' hasn't been figured out yet after N thousand emails on the
> > subject, I see no reason to believe enlightenment is going to arrive
> > soon (or ever).

[Jeremy]
> There's no particular reason to believe that effort alone will arrive
at
> an elegant solution.  On the other hand, maybe there isn't a good
syntax
> for arbitrary decorators.

IMHO, the best syntax is what we have now.  Its only weakness is being
distant from the first line in the definition.  Other than that, it is
understandable, unambiguous, simple, backward compatible, and flexible.

A failing common to all of the proposals is that they turn action words
(like classmethod) into adjectives with implicit actions.  By making
them implicit, it becomes less obvious what is occurring, when it is
occurring, and, for multiple decorators, which order they occur.  That
will make the wrapped functions harder to debug.

A second failing is introducing a second way to do it.  The current
approach will always be there for some purposes, properties,
metaclasses, existing code, or dynamic wrapping.  This is a prime
example where two ways to do something is especially bad; if you do not
see [staticmethod] in the definition, can you presume that it is not a
staticmethod -- the answer is clearly no.   So, why create optical
illusions.

A third failing is clashing with a design preference for
functions/expressions over statements/keywords/syntax.  This concept was
at the foundation for discussions about having a print() function, using
sorted() in an expression, or implementing if-then-else expressions.  My
understanding of language design history is that of regrets for not
using functions wherever possible.

Also, some proposed uses were strange enough that I wonder whether the
syntax encourages people to do weird things with their code.

How much complexity has to be introduced just to bring decorations
closer to the first line in a definition?


Raymond Hettinger
Chairman, Citizens Against 318


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From guido at python.org  Thu Apr  1 16:56:37 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 16:56:46 2004
Subject: [Python-Dev] PEP 318 -- a couple use cases
In-Reply-To: Your message of "Thu, 01 Apr 2004 14:57:09 EST."
	<406C7415.5000005@gradient.cis.upenn.edu> 
References: <406C49C7.80304@gradient.cis.upenn.edu>
	<200404011916.i31JGXm17863@guido.python.org> 
	<406C7415.5000005@gradient.cis.upenn.edu> 
Message-ID: <200404012156.i31Lubc18438@guido.python.org>

> > I'm wavering between 2 (perfectly acceptable) and 3 (somewhat
> > hackish), only because I consider anything that uses sys._getframe()
> > to be a danger to society unless proven innocent.
> 
> Would these be preferable:?
> 
>      class C(object):
>          [property]
>          def x(self): return self.__x
> 
>          [property_setter(x)]
>          def x(self, val): self.__x = val
> 
>          [property_deleter(x)]
>          def x(self): del self.__x
> and:
>      [generic(None, (int,int))]
>      def f(x,y): print 'two ints'
> 
>      [generic(f, (float,float))]
>      def f(x,y): print 'two floats'
> 
> I.e., explicitly pass the old value in, rather than grabbing it from 
> sys._getframe.
> 
> Advantages:
>    - no sys._getframe magic
> Disadvantages:
>    - you have to type the property/function name twice
>    - you have to do something special for the first def
>    - you can't easily rearrange the defs

I see no big advantage to this solution; the requirement to type the
name an extra time seems arbitrary.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dave at boost-consulting.com  Thu Apr  1 17:25:20 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Apr  1 17:40:26 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
Message-ID: <u65cjpc8f.fsf@boost-consulting.com>

Jeremy Hylton <jeremy@alum.mit.edu> writes:

> On Thu, 2004-04-01 at 07:13, Michael Hudson wrote:
>> > I don't think Michel is saying they are worthless.  However, the
>> > proposed syntax is highly contentious.  It would be good if there
>> > was a short term solution that wouldn't require new syntax.  That
>> > would give Guido and the Python community time to figure out the
>> > best syntax.
>> 
>> We've been discussing this off and on for OVER A YEAR!  If 'the best
>> syntax' hasn't been figured out yet after N thousand emails on the
>> subject, I see no reason to believe enlightenment is going to arrive
>> soon (or ever).
>
> There's no particular reason to believe that effort alone will arrive at
> an elegant solution.  On the other hand, maybe there isn't a good syntax
> for arbitrary decorators.

Has something along these lines been discussed?

  with [staticmethod, classmethod]:

     def foo(x):
         pass

     def bar(x):
         pass

IIUC, the PyObjC application needs whole swathes of functions with
the same decoration, but this syntax isn't much worse for one
function than for many.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From barry at python.org  Thu Apr  1 17:56:16 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr  1 17:56:23 2004
Subject: [Python-Dev] Vote No on 318
In-Reply-To: <002a01c4182a$8b057420$7329cb97@oemcomputer>
References: <002a01c4182a$8b057420$7329cb97@oemcomputer>
Message-ID: <1080860175.5481.19.camel@anthem.wooz.org>

On Thu, 2004-04-01 at 15:47, Raymond Hettinger wrote:

> IMHO, the best syntax is what we have now.  Its only weakness is being
> distant from the first line in the definition.  

A significant drawback.  Plus the fact that you have to type everything
twice.

> A failing common to all of the proposals is that they turn action words
> (like classmethod) into adjectives with implicit actions.

I've never thought of "classmethod" as an action, i.e. verb.  To me, and
in my code, the things inside the decorator syntax have always been
nouns.  That's why I favored decorator-after-def (although I've given up
on that now) because it read like English: "define a class method named
foo with arguments bar and baz".

I'm becoming more dejected that we're just not going to reach agreement
on this for Python 2.4.  In the hopes of avoiding hanging our Chad, I'm
throwing my endorsement to Les "Decorator Before Colon" Decidawredy.  

plus-python-mode-already-supports-that-syntax-so-what-more-do-you-need?-ly y'rs,
-Barry



From bob at redivi.com  Thu Apr  1 18:02:44 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Apr  1 17:58:41 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <u65cjpc8f.fsf@boost-consulting.com>
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
Message-ID: <AFB1C252-8430-11D8-8B4D-000A95686CD8@redivi.com>


On Apr 1, 2004, at 5:25 PM, David Abrahams wrote:

> Jeremy Hylton <jeremy@alum.mit.edu> writes:
>
>> On Thu, 2004-04-01 at 07:13, Michael Hudson wrote:
>>>> I don't think Michel is saying they are worthless.  However, the
>>>> proposed syntax is highly contentious.  It would be good if there
>>>> was a short term solution that wouldn't require new syntax.  That
>>>> would give Guido and the Python community time to figure out the
>>>> best syntax.
>>>
>>> We've been discussing this off and on for OVER A YEAR!  If 'the best
>>> syntax' hasn't been figured out yet after N thousand emails on the
>>> subject, I see no reason to believe enlightenment is going to arrive
>>> soon (or ever).
>>
>> There's no particular reason to believe that effort alone will arrive 
>> at
>> an elegant solution.  On the other hand, maybe there isn't a good 
>> syntax
>> for arbitrary decorators.
>
> Has something along these lines been discussed?
>
>   with [staticmethod, classmethod]:
>
>      def foo(x):
>          pass
>
>      def bar(x):
>          pass
>
> IIUC, the PyObjC application needs whole swathes of functions with
> the same decoration, but this syntax isn't much worse for one
> function than for many.

No, that's a misunderstanding.. it was by coincidence alone that the 
last example I gave had two callbacks with an identical type signature. 
  Using a "with" block for decorators is as nonsensical as this:

with args(x):

     def foo:
         pass

     def bar:
         pass

-bob


From pje at telecommunity.com  Thu Apr  1 18:18:47 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr  1 18:19:03 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: <200404011908.i31J8Qu17815@guido.python.org>
References: <Your message of "Thu, 01 Apr 2004 11:26:00 EST."
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
Message-ID: <5.1.1.6.0.20040401175455.0271c6c0@telecommunity.com>

At 11:08 AM 4/1/04 -0800, Guido van Rossum wrote:
> > >What I'm asking (especially of Phillip) is to collect a set of
> > >realistic method declarations using decorators; we can then
> > >collectively format these using any of the possible syntaxes, and see
> > >how they look.
> >
> > I'd be happy to scrounge up some samples from existing code using
> > 'property' and 'classmethod' as well as some of PEAK's decorators, and I
> > definitely think that Jack Diedrich and Bob Ippolito's samples should be
> > included as well.
> >
> > Important question, though: do we include code bodies, or just use 'pass'
> > for the bodies?  If we include the body, how much of the body?  Should we
> > include entire classes, especially if the class itself needs a decorator,
> > and multiple methods have decorators?
>
>Why not provide the bodies, for added realism?

Okay, here's one using two decorators, in today's syntax.  It's excerpted 
from an I/O scheduling component in 'peak.events'.  The component manages a 
set of read/write/error file handles, and allows pseudothreads waiting on 
those handles to resume when I/O is possible.  The 'select()' operation is 
itself performed by a pseudothread called 'monitor'.  Each instance of the 
component should have exactly one such pseudothread, which should begin 
running as soon as the component is "assembled" (attached to an application).

Two decorators control this: 'events.taskFactory', which accepts a 
generator function and returns a function that returns a new pseudothread 
each time it's invoked.  That is, it's roughly equivalent to:

def taskFactory(func):
     return lambda *args,**kw: Task(func(*args,**kw))

except that there is some extra magic so that introspecting the returned 
function still shows the same argument signature.  (Which is important for 
documentation tools like pydoc and epydoc).

The second decorator is 'binding.Make', which takes a 1-argument callable 
and returns a descriptor that will invoke the callable only once: when the 
attribute is first accessed for a given instance.  The result of the 
callable is cached in the object's instance dictionary, where it will be 
retrieved on any subsequent access.

So, applying the two decorators (i.e. [events.taskFactory, binding.Make]) 
to a 1-argument function results in an attribute that will be automatically 
initialized when first used.  By applying an extra keyword argument to 
'binding.Make' in the current implementation, we can tell the descriptor to 
automatically initialize itself when the componet is assembled.  (Note: 
this is not the same as __init__ time; a PEAK component is considered 
"assembled" when it is made the child of a component that knows its "root" 
component, and thus can be certain of its entire configuration environment.)

So, I would probably render this example with these decorators:

[events.taskFactory, binding.Make(uponAssembly=True)]

in order to specify that the function is a generator that should be run as 
a task (pseudothread), it should be run at most once, and it should exist 
as soon as the component knows its configuration environment.  (Note: the 
'uponAssembly' bit works today only with old-style 
'foo=binding.Make(foo,...)' syntax, not decorator syntax.)  Anyway, here's 
the example:




     def monitor(self) [events.taskFactory, binding.Make(uponAssembly=True)]:

         r,w,e = self.rwe
         count = self.count
         sleep = self.scheduler.sleep()
         time_available = self.scheduler.time_available
         select = self.select
         error = self._error

         while True:
             yield count; resume()   # wait until there are selectables
             yield sleep; resume()   # ensure we are in top-level loop

             delay = time_available()
             if delay is None:
                 delay = self.checkInterval

             try:
                 rwe = self.select(r.keys(),w.keys(),e.keys(),delay)
             except error, v:
                 if v.args[0]==EINTR:
                     continue    # signal received during select, try again
                 else:
                     raise

             for fired,events in zip(rwe,self.rwe):
                 for stream in fired:
                     events[stream]().send(True)


>(I still think class decorators are a separate case, and much weaker
>-- you can do this by having a single 'decoratable' metaclass and
>setting __decorators__ = [...] in the class body.)

Or you can use the "class advisors" mechanism I implemented for PyProtocols 
and Zope 3, which is clean and convenient in today's syntax.  Its only 
pitfall is that you absolutely *must* specify __metaclass__ first if you 
are specifying one, or your class advisors won't work right.  Actually, the 
other pitfall for anybody taking this approach is that a correct 
implementation of class advisors is *hard*, because it depends on a correct 
re-implementation of much of Python's metaclass validation logic.  But, at 
least there is one working implementation available to steal from.  :)



>I think that SPARK syntax and everything else that people have
>traditionally added to docstring markup that isn't strictly speaking
>documentation (even some extreme cases of doctest usage) ought to be
>considered as candidates for attribute-ification.

David Goodger mentioned docutils, so I mocked up a couple of 
'rst_directive' examples in a seperate message.


From pje at telecommunity.com  Thu Apr  1 18:21:47 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr  1 18:21:53 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <AFB1C252-8430-11D8-8B4D-000A95686CD8@redivi.com>
References: <u65cjpc8f.fsf@boost-consulting.com>
	<E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
Message-ID: <5.1.1.6.0.20040401181954.0271fec0@telecommunity.com>

At 06:02 PM 4/1/04 -0500, Bob Ippolito wrote:

>No, that's a misunderstanding.. it was by coincidence alone that the last 
>example I gave had two callbacks with an identical type signature.  Using 
>a "with" block for decorators is as nonsensical as this:
>
>with args(x):
>
>     def foo:
>         pass
>
>     def bar:
>         pass

I'm not sure I'd go that far.  For 'classmethod' it can actually be 
attractive to group a bunch of them together.  On the other hand, that's 
the only use case that comes to mind for grouping a bunch of functions 
under the same decorator list.


From goodger at python.org  Thu Apr  1 18:43:07 2004
From: goodger at python.org (David Goodger)
Date: Thu Apr  1 18:43:13 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <5.1.1.6.0.20040401140508.02d80090@telecommunity.com>
References: <5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>	<Your
	message of "Wed, 31 Mar 2004 21:00:55 PST."
	<42B02151.420DCBB8@mail.google.com>	<42B02151.420DCBB8@mail.google.com>	<200404010544.i315iiB15938@guido.python.org>	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<406C6566.3040104@python.org>
	<5.1.1.6.0.20040401140508.02d80090@telecommunity.com>
Message-ID: <406CA90B.5020302@python.org>

Phillip J. Eby wrote:
> Thanks!  So for example, this:
> 
> def admonition(*args):
>     return make_admonition(nodes.admonition, *args)
> 
> admonition.arguments = (1, 0, 1)
> admonition.options = {'class': directives.class_option}
> admonition.content = 1
...
> 
> might be rephrased as (say):
> 
> as [rst_directive(
>         arguments=(1,0,1),
>         options={'class': directives.class_option},
>         content=1
>     )
> ] def admonition(*args):
>     return make_admonition(nodes.admonition, *args)

I suppose so, but a generic function attribute decorator
would do just as well.  IOW, it doesn't have to be
"rst_directive", just "attributes" would do fine.

-- David Goodger

From dave at boost-consulting.com  Thu Apr  1 18:45:26 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Apr  1 18:46:13 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <u65cjpc8f.fsf@boost-consulting.com>
	<E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
	<AFB1C252-8430-11D8-8B4D-000A95686CD8@redivi.com>
	<5.1.1.6.0.20040401181954.0271fec0@telecommunity.com>
Message-ID: <uptarntyh.fsf@boost-consulting.com>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> I'm not sure I'd go that far.  For 'classmethod' it can actually be
> attractive to group a bunch of them together.  On the other hand,
> that's the only use case that comes to mind for grouping a bunch of
> functions under the same decorator list.

I really suggested it because I think it solves some of the syntactic
problems of other approaches.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From guido at python.org  Thu Apr  1 18:47:29 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 18:47:35 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: Your message of "Thu, 01 Apr 2004 18:43:07 EST."
	<406CA90B.5020302@python.org> 
References: <5.1.1.6.0.20040401111202.02d2a530@telecommunity.com> <Your
	message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<406C6566.3040104@python.org>
	<5.1.1.6.0.20040401140508.02d80090@telecommunity.com> 
	<406CA90B.5020302@python.org> 
Message-ID: <200404012347.i31NlTH18844@guido.python.org>

> > as [rst_directive(
> >         arguments=(1,0,1),
> >         options={'class': directives.class_option},
> >         content=1
> >     )
> > ] def admonition(*args):
> >     return make_admonition(nodes.admonition, *args)
> 
> I suppose so, but a generic function attribute decorator
> would do just as well.  IOW, it doesn't have to be
> "rst_directive", just "attributes" would do fine.

I like rst_directive(), because it's more specific -- it would catch
mistakes earlier.  E.g. if you misspelled argumnets, attributes()
would have no clue about it; but rst_directive() should have specific
keywords.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From shane at zope.com  Thu Apr  1 18:54:11 2004
From: shane at zope.com (Shane Hathaway)
Date: Thu Apr  1 18:54:49 2004
Subject: [Python-Dev] PEP 318 bake-off?
In-Reply-To: <200404010544.i315iiB15938@guido.python.org>
References: Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org>
Message-ID: <406CABA3.7080805@zope.com>

Guido van Rossum wrote:
> 1) Last-before-colon:
> 
>    def foo(cls, arg1, arg2) [funcattrs(foo=42), deprecated,
>                              overrides, classmethod]:
>        pass
> 
> 2) Prefix list:
> 
>    [funcattrs(foo=42), deprecated, overrides, classmethod]
>    def foo(cls, arg1, arg2):
>        pass

I bet most of us who prefer #2 would be just as happy with #1.  Am I right?

Shane

From greg at cosc.canterbury.ac.nz  Thu Apr  1 19:05:39 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Apr  1 19:05:51 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <200403312049.i2VKnEi14444@guido.python.org>
Message-ID: <200404020005.i3205dhf018441@cosc353.cosc.canterbury.ac.nz>

I think most of the worry about long, complicated
decorator lists with baroque parameters is FUD.
What are the realistic use cases for such things?
I don't remember seeing any proposed.

The ones I have seen proposed (e.g. PyObjC) use
only one or two decorators at a time, and if there
are any parameters, they're not lengthly.

Furthermore, if in some circumstance someone finds
themselves writing decorator lists so long that they
become unreadable, well, there are ways of dealing
with that. For example,

foo_decorators = decorator_list(
  # Big long-winded
  # list of
  # decorators
)

def foo(args) [foo_decorators]:
  ...

which is not much different from Guido's preceding-list
version, with the advantage that it's explicit about
the fact that you need to look above to get the full
picture.

Please let's not allow unfounded speculation to get
in the way of coming up with a nice syntax that works
well in the majority of cases we expect to actually
see.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Thu Apr  1 19:10:03 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr  1 19:10:21 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: Your message of "Thu, 01 Apr 2004 10:47:21 CST."
	<406C4799.6030502@4-am.com> 
References: <42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org> 
	<406C4799.6030502@4-am.com> 
Message-ID: <200404020010.i320A3X18910@guido.python.org>

> How about adding:
> 
> 2b) Prefix list with repeated keyword:
> 
> def [funcattrs(foo=42), deprecated, overrides, classmethod]
> def foo(cls, arg1, arg2):
>      pass
> 
> class [singleton]
> class foo:
>      pass
> 
> 3a) Prefix suite with repeated keyword:
> 
> def:
>      funcattrs(foo=42)
>      deprecated, overrides, classmethod
> def foo(cls, arg1, arg2):
>      pass

You know, that one occurred to me in the shower, because it should
parse easily.  But I immediately rejected it as too weird.  The
stutter doesn't have any semantic connotation to it and is bound to
confuse source-scanning tools.

(The nice thing about plain prefix list syntax is that tools which
look for 'def' but don't process other statements are just as
oblivious to it as they are to current decorator syntax.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pje at telecommunity.com  Thu Apr  1 19:12:35 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr  1 19:12:41 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <200404012347.i31NlTH18844@guido.python.org>
References: <Your message of "Thu,
	01 Apr 2004 18:43:07 EST." <406CA90B.5020302@python.org>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<Your message of "Wed,
	31 Mar 2004 21:00:55 PST." <42B02151.420DCBB8@mail.google.com>
	<42B02151.420DCBB8@mail.google.com>
	<200404010544.i315iiB15938@guido.python.org>
	<5.1.1.6.0.20040401111202.02d2a530@telecommunity.com>
	<406C6566.3040104@python.org>
	<5.1.1.6.0.20040401140508.02d80090@telecommunity.com>
	<406CA90B.5020302@python.org>
Message-ID: <5.1.1.6.0.20040401190831.028443c0@telecommunity.com>

At 03:47 PM 4/1/04 -0800, Guido van Rossum wrote:
> > > as [rst_directive(
> > >         arguments=(1,0,1),
> > >         options={'class': directives.class_option},
> > >         content=1
> > >     )
> > > ] def admonition(*args):
> > >     return make_admonition(nodes.admonition, *args)
> >
> > I suppose so, but a generic function attribute decorator
> > would do just as well.  IOW, it doesn't have to be
> > "rst_directive", just "attributes" would do fine.
>
>I like rst_directive(), because it's more specific -- it would catch
>mistakes earlier.  E.g. if you misspelled argumnets, attributes()
>would have no clue about it; but rst_directive() should have specific
>keywords.

Yeah, that's why I wrote the example that way.  Once you have an 
'attributes()' decorator, it's easy to create task-specific versions of it, 
e.g.:

def my_decorator(foo,bar,baz):
     return attributes(**locals())

And of course you can have defaults, and do validation of the attributes 
before the return statement.

And there's an additional reason to use task-specific decorators: if you 
later decide that function attributes aren't as useful, or you need to 
change what the attributes are named, or you decide to stick all the 
options into an object and use only one attribute, you need only change the 
task-specific decorator, not all the things that call it.


From shane at zope.com  Thu Apr  1 19:24:18 2004
From: shane at zope.com (Shane Hathaway)
Date: Thu Apr  1 19:24:22 2004
Subject: [Python-Dev] Vote No on 318
In-Reply-To: <002a01c4182a$8b057420$7329cb97@oemcomputer>
References: <1080833042.22892.122.camel@localhost.localdomain>
	<002a01c4182a$8b057420$7329cb97@oemcomputer>
Message-ID: <406CB2B2.1060209@zope.com>

Raymond Hettinger wrote:
> A failing common to all of the proposals is that they turn action words
> (like classmethod) into adjectives with implicit actions.  By making
> them implicit, it becomes less obvious what is occurring, when it is
> occurring, and, for multiple decorators, which order they occur.  That
> will make the wrapped functions harder to debug.

"foo = classmethod(foo)" also uses a noun as a verb.  We're no worse 
than before.  I agree that the meaning of 'y' in "def foo(x) [y]" is not 
obvious, but it is easy to explain: "Python creates function foo, which 
accepts one parameter named x, then applies the y decorator to the 
function."

> A second failing is introducing a second way to do it.  The current
> approach will always be there for some purposes, properties,
> metaclasses, existing code, or dynamic wrapping.  This is a prime
> example where two ways to do something is especially bad; if you do not
> see [staticmethod] in the definition, can you presume that it is not a
> staticmethod -- the answer is clearly no.   So, why create optical
> illusions.

In all places I've seen it used, the current approach sacrifices a lot 
of clarity.  Python should advise against the current approach.

> A third failing is clashing with a design preference for
> functions/expressions over statements/keywords/syntax.  This concept was
> at the foundation for discussions about having a print() function, using
> sorted() in an expression, or implementing if-then-else expressions.  My
> understanding of language design history is that of regrets for not
> using functions wherever possible.

That's an excellent pattern, but Python is much clearer than Lisp 
because it does not try to take that pattern to its ultimate extreme.

> Also, some proposed uses were strange enough that I wonder whether the
> syntax encourages people to do weird things with their code.

The code many of are working with is already weird, and we're grasping 
for some way to clean it up.

Shane

From jack at performancedrivers.com  Thu Apr  1 20:11:41 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Thu Apr  1 20:11:47 2004
Subject: [Python-Dev] Vote No on 318
In-Reply-To: <002a01c4182a$8b057420$7329cb97@oemcomputer>
References: <1080833042.22892.122.camel@localhost.localdomain>
	<002a01c4182a$8b057420$7329cb97@oemcomputer>
Message-ID: <20040402011141.GB26450@performancedrivers.com>

On Thu, Apr 01, 2004 at 03:47:26PM -0500, Raymond Hettinger wrote:
> [Jeremy]
> > There's no particular reason to believe that effort alone will arrive
> > at
> > an elegant solution.  On the other hand, maybe there isn't a good
> > syntax
> > for arbitrary decorators.
> 
> IMHO, the best syntax is what we have now.  Its only weakness is being
> distant from the first line in the definition.  Other than that, it is
> understandable, unambiguous, simple, backward compatible, and flexible.
> 
> A failing common to all of the proposals is that they turn action words
> (like classmethod) into adjectives with implicit actions.  By making
> them implicit, it becomes less obvious what is occurring, when it is
> occurring, and, for multiple decorators, which order they occur.  That
> will make the wrapped functions harder to debug.
I think of them as annotations instead of adjectives.  I sympathize for
the 'when it is occuring' but I much prefer it included in the func def
than very far away.  If it is way after the func/class def it is obvious
the action happens afterwards.  The trick is noticing it happens at all.

As for implicit,
  def foo(cls, arg1, arg2): pass # implicitly a classmethod
  def foo(cls, arg1, arg2) [classmethod]: pass # unambiguously classmethod

> A second failing is introducing a second way to do it.  The current
> approach will always be there for some purposes, properties,
> metaclasses, existing code, or dynamic wrapping.  This is a prime
> example where two ways to do something is especially bad; if you do not
> see [staticmethod] in the definition, can you presume that it is not a
> staticmethod -- the answer is clearly no.   So, why create optical
> illusions.
Decorators will be unifying, there is currently more than one equal way 
to do it (metaclasses, frame hacks, inspection, eval'ed docstrings).
'import this' says there should be one _obvious_ way to do it.
Decorators (especially the before-colon kind) are _obvious_.
I think 'classmethod' when I've typed 'def foo(cls' *next thought classmethod*
I don't think *foo = classmethod(foo)* when I've typed 'return'
 
> A third failing is clashing with a design preference for
> functions/expressions over statements/keywords/syntax.  This concept was
> at the foundation for discussions about having a print() function, using
> sorted() in an expression, or implementing if-then-else expressions.  My
> understanding of language design history is that of regrets for not
> using functions wherever possible.
'readability counts', this helps readability for the places where you need it.
I'll use decorators more often than genexps, but I like not having to import
itertools and calling a function to make an iterator.  genexps read like
intention, uneccessary function calls read like a recipe.
Java is a recipe language, we've all seen comments like
"I'm about to do the boilerplate for X, page down for the important stuff"
in Java (if you haven't been exposed to Java, feel lucky).

def foo(cls): # see modifiers below
  a = 1 + 1
  b = a * 10
  return true
foo = classmethod(foo) # boilerplate
foo.__doc__ = "This function returns true" # boilerplate (which was rectified)

> Also, some proposed uses were strange enough that I wonder whether the
> syntax encourages people to do weird things with their code.
As above, we're cutting out the weird ways people do things now (eval'ing
docstrings, examining stackframes, abusing metaclasses) to achieve
decorator functionality by making decorators standard.
They already had plenty of rope, let's make it all the same color.

-jackdied

From jim.jewett at eds.com  Thu Apr  1 20:37:04 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu Apr  1 20:37:04 2004
Subject: [Python-Dev] Re: PEP 318: "as", barelist
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D42F@USAHM010.amer.corp.eds.com>

Phillip J. Eby:

> 'as' is not currently a keyword, and making it so would break
> any programs that use it as a name.

It has been documented as a pseudo-keyword for some time, with
the warning that it will may become a real keyword later.

> Really, of the current "before the def" proposals, I think I
> like Samuele's:

	*[classmethod]
	def func(*args):
	     pass

	approach the best.  The '*' seems to say "special note, pay
attention".  :)

I agree that it says special note; it just doesn't say *why* 
the list is special.  We happen to know that it secretly means 

	"After you evaluate this list, keep a reference, then 
	call each member on the function or class I'm about to
	define."

but a keyword (even as) makes it more explicit why this list
is special.  Of course,

	as:  d1, d2, d2
	def func(args):
	   pass

also works; if you put a keyword at the front, then [] becomes
redundant.  And that was one of the bakeoff options, except 
that Guido used "decorate:" instead of "as:" in his example.

-jJ

From jack at performancedrivers.com  Thu Apr  1 20:38:36 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Thu Apr  1 20:38:42 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <200404020005.i3205dhf018441@cosc353.cosc.canterbury.ac.nz>
References: <200403312049.i2VKnEi14444@guido.python.org>
	<200404020005.i3205dhf018441@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040402013836.GD26450@performancedrivers.com>

On Fri, Apr 02, 2004 at 12:05:39PM +1200, Greg Ewing wrote:
> I think most of the worry about long, complicated
> decorator lists with baroque parameters is FUD.
> What are the realistic use cases for such things?
> I don't remember seeing any proposed.
> 
> The ones I have seen proposed (e.g. PyObjC) use
> only one or two decorators at a time, and if there
> are any parameters, they're not lengthly.
> 
> Furthermore, if in some circumstance someone finds
> themselves writing decorator lists so long that they
> become unreadable, well, there are ways of dealing
> with that. For example,
> 
> foo_decorators = decorator_list(
>   # Big long-winded
>   # list of
>   # decorators
> )
> 
> def foo(args) [foo_decorators]:
>   ...
> 
I was actually about post the same in the bake-off thread,
but with the super-obvious

see_above = decorator_list(
   decorator1,
   ..
   decoratorN,
)
def foo(args) [see_above]:
  ...

You can't beat that for obviousness, even newbies know _something_
is going on and it happens just above.

no_joy = decorator_list(
  decorator1,
  ..
  decoratorN,
)
[no_joy]
def foo(args): # modified by no_joy

So the best case is a comment that will vary by style.
Do newbies read comments? dunno.

-jackdied

From greg at cosc.canterbury.ac.nz  Thu Apr  1 21:31:06 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Apr  1 21:31:22 2004
Subject: [Python-Dev] Re: ANNOUNCE: 'goto' for Python
In-Reply-To: <c4hidb$hfd$1@sea.gmane.org>
Message-ID: <200404020231.i322V6iC018697@cosc353.cosc.canterbury.ac.nz>

Mike Rovner <mike@nospam.com>:

> Shall I file a bug about python lexer which allows space between class and
> attribute? Or that's a fiature?

It's certainly not a bug. An identifier and a dot following it
are separate lexical tokens, so whitespace should be allowed
between them in the interests of consistency, even if it's not
all that useful.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Thu Apr  1 22:13:59 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Apr  1 22:14:12 2004
Subject: [Python-Dev] Vote No on 318
In-Reply-To: <002a01c4182a$8b057420$7329cb97@oemcomputer>
Message-ID: <200404020313.i323DxL7024193@cosc353.cosc.canterbury.ac.nz>

Raymond Hettinger <python@rcn.com>:

> A failing common to all of the proposals is that they turn action
> words (like classmethod) into adjectives with implicit actions.

Action words? I would have said "classmethod" is a noun, and as such
it's a perfectly good name for a decorator.

The fact that an action occurs at function definition time is an
implementation detail.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From perky at i18n.org  Thu Apr  1 22:24:24 2004
From: perky at i18n.org (Hye-Shik Chang)
Date: Thu Apr  1 22:24:36 2004
Subject: [Python-Dev] Re: ANNOUNCE: 'goto' for Python
In-Reply-To: <200404020231.i322V6iC018697@cosc353.cosc.canterbury.ac.nz>
References: <c4hidb$hfd$1@sea.gmane.org>
	<200404020231.i322V6iC018697@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040402032424.GA96150@i18n.org>

On Fri, Apr 02, 2004 at 02:31:06PM +1200, Greg Ewing wrote:
> Mike Rovner <mike@nospam.com>:
> 
> > Shall I file a bug about python lexer which allows space between class and
> > attribute? Or that's a fiature?
> 
> It's certainly not a bug. An identifier and a dot following it
> are separate lexical tokens, so whitespace should be allowed
> between them in the interests of consistency, even if it's not
> all that useful.
> 

Agreed. At least it has one usage:

>>> 1 .__add__
<method-wrapper object at 0x81c4dec>
>>> 1.__add__
  File "<stdin>", line 1
    1.__add__
            ^
SyntaxError: invalid syntax
>>> addone = 1 .__add__
>>> addone(5)
6

Even though it's not so beautiful. :)


Hye-Shik

From greg at cosc.canterbury.ac.nz  Thu Apr  1 22:34:10 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Apr  1 22:34:33 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <200404020005.i3205dhf018441@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>

Another thought about:

  [decorator]
  def foo(args):
    ...

What are newcomers going to make of this?

It looks like a thrown-away list followed by a def, and
there's no clue that there's any connection between the
two. And even if the newcomer suspects that there is some
kind of connection, what are they going to look up
in the manual?

At least with

  def foo(args) [decorator]:
    ...

it's clear that (1) something different is going on, and
(2) it's somehow a part of the 'def' statement.

Everywhere else in Python where you can say

  something
  someotherthing

it means "do something and then do someotherthing", and
the combined result can be deduced from knowing what
something and someotherthing each do on their own.
This breaks that.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From DavidA at ActiveState.com  Thu Apr  1 23:09:24 2004
From: DavidA at ActiveState.com (David Ascher)
Date: Thu Apr  1 23:10:49 2004
Subject: [Python-Dev] Re: PEP 318: "as", barelist
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D42F@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D42F@USAHM010.amer.corp.eds.com>
Message-ID: <406CE774.60400@ActiveState.com>

Jewett, Jim J wrote:

>>Really, of the current "before the def" proposals, I think I
>>like Samuele's:
> 
> 
> 	*[classmethod]
> 	def func(*args):
> 	     pass
> 
> 	approach the best.  The '*' seems to say "special note, pay
> attention".  :)

I know -- just like Perl6 uses unicode characters (see e.g. 
http://www.perl.com/pub/a/2004/02/27/exegesis7.html?page=3), we could 
use the unicode DAGGER character (?).

--david


From bob at redivi.com  Thu Apr  1 23:28:59 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Apr  1 23:24:53 2004
Subject: [Python-Dev] Re: PEP 318: "as", barelist
In-Reply-To: <406CE774.60400@ActiveState.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D42F@USAHM010.amer.corp.eds.com>
	<406CE774.60400@ActiveState.com>
Message-ID: <433D680A-845E-11D8-8B4D-000A95686CD8@redivi.com>


On Apr 1, 2004, at 11:09 PM, David Ascher wrote:

> Jewett, Jim J wrote:
>
>>> Really, of the current "before the def" proposals, I think I
>>> like Samuele's:
>> 	*[classmethod]
>> 	def func(*args):
>> 	     pass
>> 	approach the best.  The '*' seems to say "special note, pay
>> attention".  :)
>
> I know -- just like Perl6 uses unicode characters (see e.g. 
> http://www.perl.com/pub/a/2004/02/27/exegesis7.html?page=3), we could 
> use the unicode DAGGER character (?).

if datetime.date.today().isoformat().endswith('04-01'):
	or we could just ?start using chevrons?.  They're particularly easy to 
type on the Mac (opt-\ and shift-opt-\), and some people are already 
familiar with them from lowlevel applescript hackery :)

-bob


From jeremy at alum.mit.edu  Thu Apr  1 23:28:50 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Apr  1 23:30:49 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1080880130.22892.158.camel@localhost.localdomain>

On Thu, 2004-04-01 at 22:34, Greg Ewing wrote:
> Another thought about:
> 
>   [decorator]
>   def foo(args):
>     ...
> 
> What are newcomers going to make of this?
> 
> It looks like a thrown-away list followed by a def, and
> there's no clue that there's any connection between the
> two. 

I think it's obvious that there's a connection between the two.  A bare
list by itself would be nonsense and the actual list would probably say
something like [classmethod] which suggests it's saying something about
a method.

> And even if the newcomer suspects that there is some
> kind of connection, what are they going to look up
> in the manual?

Maybe they'll lookup functions in the reference manual?  I assume they'd
be just as puzzled by:

    def f(x, y, *args, **kwargs):

Who came up with that bizarre syntax? <wink>

Jeremy



From python at rcn.com  Thu Apr  1 23:40:45 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Apr  1 23:40:59 2004
Subject: [Python-Dev] Vote No on 318
In-Reply-To: <200404020313.i323DxL7024193@cosc353.cosc.canterbury.ac.nz>
Message-ID: <008201c4186c$a9c96820$7329cb97@oemcomputer>

> > A failing common to all of the proposals is that they turn action
> > words (like classmethod) into adjectives with implicit actions.

[Greg Ewing]
> Action words? I would have said "classmethod" is a noun, and as such
> it's a perfectly good name for a decorator.

Sure, the current name happens to be a noun instead of "classmethodify",
but what it does is action oriented.   So, the real issue is declarative
versus imperative.  Both are great, but don't mix well.


 
> The fact that an action occurs at function definition time is an
> implementation detail.

It's a somewhat important detail.



Raymond


From tim.one at comcast.net  Fri Apr  2 00:07:33 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr  2 00:07:35 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <007801c41739$cf5b2c10$6402a8c0@arkdesktop>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEGFJPAB.tim.one@comcast.net>

[Andrew Koenig, on the move-to-wider-platform thing]
> ...
> OK, but either that objection is relevant for Python or it isn't.

It's not important to me.  It may be important to some Python users now; I
don't know.  It may be more or less important in the future, because things
do change.

> ...
> On the other hand, maybe ?!ng is right about the desirable properties
> of display for people being different from those for
> marshalling/unmarshalling.

There's no necessary connection between the two, as I've granted for all the
years this debate has raged <wink>.

OTOH, there's no fixed choice for displaying floats that will make everyone
happy all the time, or, I think, even most people happy most of the time.
You gave a list of 4 different behaviors you sometimes want.  I have my own
list, including some not on yours:

+ Display numbers with a comma every third digit.

+ Display numbers with just a few significant digits, but in
  "engineering" notation (== exponent always a multiple of 3).

Because of this, arguments about repr() and str() are futile, IMO.  The
flexibility most people want some of the time can't be gotten by changing
one fixed behavior for another.  This isn't limited to floats, either.

...

>> Python would love to grab their I/O implementation then <0.8 wink>.

> http://www.research.att.com/sw/tools/sfio/
>
> I think the licensing terms are compatible with Python, if you're
> serious.

It was considered before, but its portablility to non-Unixish system looks
poor, and in particular its Windows story appears to require commercial
users to buy a UWIN license.

Guido started on something different in the Python CVS sandbox/sio.  I don't
think he wrote a design doc, but the thrust is more to make composable
filtering streams (like streams that deal with Unicode gracefully, and that
hide differences among the different line end conventions (it's increasingly
common to have to deal with a variety of file formats on a single box)).


From barry at python.org  Fri Apr  2 08:24:43 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr  2 08:24:55 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <1080880130.22892.158.camel@localhost.localdomain>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
Message-ID: <1080912282.5481.158.camel@anthem.wooz.org>

On Thu, 2004-04-01 at 23:28, Jeremy Hylton wrote:
> On Thu, 2004-04-01 at 22:34, Greg Ewing wrote:
> > Another thought about:
> > 
> >   [decorator]
> >   def foo(args):
> >     ...
> > 
> > What are newcomers going to make of this?
> > 
> > It looks like a thrown-away list followed by a def, and
> > there's no clue that there's any connection between the
> > two. 
> 
> I think it's obvious that there's a connection between the two.  A bare
> list by itself would be nonsense and the actual list would probably say
> something like [classmethod] which suggests it's saying something about
> a method.

It may be nonsense, but it means something today.  So it can't be
obvious that they're connected because today, they aren't.

If tomorrow this same code means something different, users looking at
the code will have to know what version of Python they're using, and
make sure it's the right one ("uh, how do I do that?").  If they were to
use decorator-before-def code in an older version of Python, the program
would be accepted but silently do the wrong thing.

At least with decorator-before-colon trying to run the code in older
interpreters will barf loudly.

-Barry



From ark-mlist at att.net  Fri Apr  2 09:40:08 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Apr  2 09:40:07 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon -- another
	new syntax idea
In-Reply-To: <u65cjpc8f.fsf@boost-consulting.com>
Message-ID: <007501c418c0$655eb6f0$6402a8c0@arkdesktop>

> Has something along these lines been discussed?
> 
>   with [staticmethod, classmethod]:
> 
>      def foo(x):
>          pass
> 
>      def bar(x):
>          pass
> 
> IIUC, the PyObjC application needs whole swathes of functions with
> the same decoration, but this syntax isn't much worse for one
> function than for many.

If I understand it correctly, the motivation for this idea is to try to
avoid cluttering the definitions with large numbers of decorators.

In which case, how about this?

	sc = [staticmethod, classmethod]

	def foo(x) sc:
		pass

	def bar(x) sc:
		pass

In other words, if the decorators are too voluminous, do what you always do
with a complicated expression:  Bind a variable to it and use the variable
instead.


From casey at zope.com  Fri Apr  2 10:55:35 2004
From: casey at zope.com (Casey Duncan)
Date: Fri Apr  2 10:58:46 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
Message-ID: <20040402105535.0526c27d.casey@zope.com>

On Thu, 01 Apr 2004 17:25:20 -0500
David Abrahams <dave@boost-consulting.com> wrote:

> Jeremy Hylton <jeremy@alum.mit.edu> writes:
> 
> > On Thu, 2004-04-01 at 07:13, Michael Hudson wrote:
> >> > I don't think Michel is saying they are worthless.  However, the
> >> > proposed syntax is highly contentious.  It would be good if there
> >> > was a short term solution that wouldn't require new syntax.  That
> >> > would give Guido and the Python community time to figure out the
> >> > best syntax.
> >> 
> >> We've been discussing this off and on for OVER A YEAR!  If 'the
> >best> syntax' hasn't been figured out yet after N thousand emails on
> >the> subject, I see no reason to believe enlightenment is going to
> >arrive> soon (or ever).
> >
> > There's no particular reason to believe that effort alone will
> > arrive at an elegant solution.  On the other hand, maybe there isn't
> > a good syntax for arbitrary decorators.
> 
> Has something along these lines been discussed?
> 
>   with [staticmethod, classmethod]:
> 
>      def foo(x):
>          pass
> 
>      def bar(x):
>          pass
> 
> IIUC, the PyObjC application needs whole swathes of functions with
> the same decoration, but this syntax isn't much worse for one
> function than for many.

What if you dropped the keyword?

  [classmethod]:

      def splat(cls):
          pass

      def baz(cls):
          pass

Or how about as?

  as classmethod:

     def jinkies(cls):
         pass

Which seems nice in the face of other declarations:

  as protected(some_permission):

     def zoinks(self, scooby, snack):
         pass

     def jinkies(self):
         pass

or

  [protected_by(some_permission)]:

      def dangerous(self, risk):
          pass

At least then the ambiguity is gone wrt unassigned lists before defs
(and I won't have to rewrite all that code where I'm using list
comprehensions as docs before method defs ;^). Also the grouping seems
useful.

OTOH, madatory bracketing seems unpythonic to me, which makes as/with
option (without mandatory brackets) seem compelling.

-Casey

 


From mwh at python.net  Fri Apr  2 11:34:02 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Apr  2 11:34:06 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <1080912282.5481.158.camel@anthem.wooz.org> (Barry Warsaw's
	message of "Fri, 02 Apr 2004 08:24:43 -0500")
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
Message-ID: <2m1xn6gwzp.fsf@starship.python.net>

Barry Warsaw <barry@python.org> writes:

> If tomorrow this same code means something different, users looking at
> the code will have to know what version of Python they're using, and
> make sure it's the right one ("uh, how do I do that?").  If they were to
> use decorator-before-def code in an older version of Python, the program
> would be accepted but silently do the wrong thing.
>
> At least with decorator-before-colon trying to run the code in older
> interpreters will barf loudly.

I think this is a good point that hadn't occurred to me: silent
failures suck.

Cheers,
mwh

-- 
  Like most people, I don't always agree with the BDFL (especially
  when he wants to change things I've just written about in very 
  large books), ... 
         -- Mark Lutz, http://python.oreilly.com/news/python_0501.html

From jim.jewett at eds.com  Fri Apr  2 11:44:17 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Fri Apr  2 11:45:12 2004
Subject: [Python-Dev] PEP 318 bake-off?
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D433@USAHM010.amer.corp.eds.com>

Brad Clements:
> Guido van Rossum wrote:

>> I think that SPARK syntax and everything else that people have
>> traditionally added to docstring markup that isn't strictly speaking
>> documentation (even some extreme cases of doctest usage) ought to be
>> considered as candidates for attribute-ification.

> Where do method attribute type signatures and DBC fit in? 
> As a decorator, or in the docstring?

While it will still be *possible* to abuse the docstring, it 
*should* go as a decorator.  Not every machine-readable class 
invariant is useful documentation to someone who isn't already 
debugging that code in particular.

> I'm concerned that the funcattrs(a="xyz" .. ) sample tossed
> around here will be rather ugly for specifying DBC strings.

I agree that it would be better to create a DBC class.

If there is a DBC attribute on a code object, then DBC-aware
tools will look to that object for the DBC conditions.  Whether
you express them as strings or predicate functions or ... your
choice.

As a specific case, a debugger could use object.__DBC__.__call__
instead of object.__call__.  (If you wanted to enforce the
DBC checks at all times, then your decorator could just return
a new object that checks and delegates, rather than an annotated
version of the original.)

> Finally,  I don't have a need to access DBC annotations at 
> runtime once my module is distributed. I would not want to
> pay the memory cost overhead of loading DBC information or
> attribute type signatures at runtime.

Then define a release-DBC class whose constructor is pass, and
whose decoration is to return the object unchanged (without a
DBC attribute).  Use that in your released code.  Whether to
strip DBC info entirely or just throw it away on loading is up 
to you.

> However another person at PyCon poo-poo'd my concern over
> bloating .pyc files and subsequent memory use. As a compromise
> I suggested that "annotation" information could go into the
> .pyc, but be loaded "on demand" at runtime. 

Changing the format of .pyc is beyond the scope of PEP318.  

If you want a special case for DBC, you can write it.  Make 
your DBC class save the annotations for example.py to 
example.dbc, and retrieve them on demand. You may have to 
go through a rewrite/reload step to get them out of the 
.pyc without removing them from the .py, but you can do it.

If on-demand is not required, you could probably change the 
compiler to ignore any attribute registered as ignorable
during optimization, instead of just __doc__.

-jJ

From fdrake at acm.org  Fri Apr  2 11:59:47 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Apr  2 12:01:02 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <2m1xn6gwzp.fsf@starship.python.net>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<2m1xn6gwzp.fsf@starship.python.net>
Message-ID: <200404021159.47419.fdrake@acm.org>

Barry Warsaw <barry@python.org> writes:
 > At least with decorator-before-colon trying to run the code in older
 > interpreters will barf loudly.

On Friday 02 April 2004 11:34 am, Michael Hudson wrote:
 > I think this is a good point that hadn't occurred to me: silent
 > failures suck.

There's also the little matter that using syntax only, without new keywords, 
allows us to avoid __future__ imports and all the implementation cruft that 
entails.

Decorator-before-colon wins out big time for me.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jeremy at alum.mit.edu  Fri Apr  2 12:05:13 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Apr  2 12:07:16 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <1080912282.5481.158.camel@anthem.wooz.org>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
Message-ID: <1080925513.22892.173.camel@localhost.localdomain>

On Fri, 2004-04-02 at 08:24, Barry Warsaw wrote:
> It may be nonsense, but it means something today.  So it can't be
> obvious that they're connected because today, they aren't.

Your original complaint was: "What are newcomers going to make of
this?"  Newcomers aren't going to be worried about the small change in
semantics that decorator-before would entail, because they won't know
the old semantics.  It's only a problem for people who already know the
language, but one would hope they read the "What's New in Python 2.4"
document.

I expect a newcomer to read the code in the most natural way.  Only a
language lawyer would worry about whether the expression "[classmethod]"
was evaluated and thrown away rather than referring to fromFoo. 
Language lawyers should be clever enough to figure out the new rules
:-).

class Quux:

    def __init__(self):
        ...

    [classmethod]
    def fromFoo(cls, arg):
        """Create a Quux instance from a Foo."""
        ...


> If tomorrow this same code means something different, users looking at
> the code will have to know what version of Python they're using, and
> make sure it's the right one ("uh, how do I do that?").  If they were to
> use decorator-before-def code in an older version of Python, the program
> would be accepted but silently do the wrong thing.

I agree there's a risk here, but we've faced that kind of risk before. 
We used future statements for nested scopes, but only one version.  If
you're looking at code with free variables, you need to know whether it
was written to run with or without nested scopes.  Code written for one
version will fail at runtime on the other.  (It may or may not fail
silently.)

Jeremy



From theller at python.net  Fri Apr  2 12:11:42 2004
From: theller at python.net (Thomas Heller)
Date: Fri Apr  2 12:26:04 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <2m1xn6gwzp.fsf@starship.python.net> (Michael Hudson's message
	of "Fri, 02 Apr 2004 17:34:02 +0100")
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<2m1xn6gwzp.fsf@starship.python.net>
Message-ID: <vfkicnjl.fsf@python.net>

Michael Hudson <mwh@python.net> writes:

> Barry Warsaw <barry@python.org> writes:
>
>> If tomorrow this same code means something different, users looking at
>> the code will have to know what version of Python they're using, and
>> make sure it's the right one ("uh, how do I do that?").  If they were to
>> use decorator-before-def code in an older version of Python, the program
>> would be accepted but silently do the wrong thing.
>>
>> At least with decorator-before-colon trying to run the code in older
>> interpreters will barf loudly.
>
> I think this is a good point that hadn't occurred to me: silent
> failures suck.

Wouldn't the decorator-before-def require a 'from __future__ import
decorators' (Although I'm still on favor of the decorator-before-colon
version)?

Thomas


From barry at python.org  Fri Apr  2 14:08:56 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr  2 14:09:09 2004
Subject: [Python-Dev] PEP 318: Let's not give in to FUD
In-Reply-To: <1080925513.22892.173.camel@localhost.localdomain>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
Message-ID: <1080932935.5481.282.camel@anthem.wooz.org>

On Fri, 2004-04-02 at 12:05, Jeremy Hylton wrote:
> On Fri, 2004-04-02 at 08:24, Barry Warsaw wrote:
> > It may be nonsense, but it means something today.  So it can't be
> > obvious that they're connected because today, they aren't.
> 
> Your original complaint was: "What are newcomers going to make of
> this?"  Newcomers aren't going to be worried about the small change in
> semantics that decorator-before would entail, because they won't know
> the old semantics.  

They already know the old semantics.  If they were to encounter that
syntax today, they'd know that Python throws away the results.  If /I/
saw that construct in today's Python code, I'd start looking for side
effects.

> > If tomorrow this same code means something different, users looking at
> > the code will have to know what version of Python they're using, and
> > make sure it's the right one ("uh, how do I do that?").  If they were to
> > use decorator-before-def code in an older version of Python, the program
> > would be accepted but silently do the wrong thing.
> 
> I agree there's a risk here, but we've faced that kind of risk before. 
> We used future statements for nested scopes, but only one version.  If
> you're looking at code with free variables, you need to know whether it
> was written to run with or without nested scopes.  Code written for one
> version will fail at runtime on the other.  (It may or may not fail
> silently.)

Adding a future statement would make decorator-before-def slightly more
acceptable.  Adding a keyword to introduce decorator-before-def would
make it slightly more acceptable.  All-in-all, I still think
decorator-before-colon is plenty readable and the more obvious of the
two choices.

-Barry



From guido at python.org  Fri Apr  2 14:50:35 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr  2 14:50:42 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in decorators
In-Reply-To: Your message of "Fri, 02 Apr 2004 14:08:56 EST."
	<1080932935.5481.282.camel@anthem.wooz.org> 
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain> 
	<1080932935.5481.282.camel@anthem.wooz.org> 
Message-ID: <200404021950.i32JoZN21741@guido.python.org>

While I ponder the decorator syntax, let's propose some built-in
decorators.

We've already got classmethod and staticmethod.

I propose this one to set function attributes:

  class func_attrs(objects):

      def __init__(self, **kwds):
          self.attrs = kwds

      def __call__(self, funcobj):
          funcobj.__dict__.update(self.attrs)

Why a class?  So you can subclass it like this:

  class rst_attrs(func_attrs):

      def __init__(self, arguments, options, content):
          func_attrs.__init__(self, arguments=arguments, options=options,
                                    content=content)

Why would you want to do that?  So that mistakes in the attribute
names would be caught early, and perhaps default values could be
provided.

We could also add a standard implementation of synchronized.  Or
perhaps that should be imported from threading.  (But is that really a
good thing to copy from Java?)

Other possibilities (all these are pretty much thinking aloud and very
much up for votes; and not original, I've seen these proposed
before!):

deprecated(comment_string) -- mostly for documentation, but would set
the deprecated attribute.  The comment_string argument is a string
explaining why it is deprecated.  Maybe this should also have
arguments specifying the first Python release in which it was
considered deprecated and the last Python release (if known) where it
will be available.

overrides -- indicates that this overrides a base class method.  Maybe
the default metaclass could check that if this is used there actually
is a corresponding base class method, and we might have a "strict"
metaclass that checks this is set for all overriding methods.

doctest_script(multi_line_string) -- specifies a doctest script, for
use by the doctest module (if this is absent, it will continue to look
in the doc string).  I like to separate the doctest script from the
actual documentation string because, when used for rigorous unit
testing (as in Jim&Tim's talk at PyCON), the doctest script is too
long to double as reasonable documentation.  Maybe this should be
imported from the doctest module.

decorator_chain(*args) -- takes any number of decorator arguments and
applies them sequentially.  Implementation could be:

  def decorator_chain(*args):
      def decorate(func):
          for arg in args:
              func = arg(func)
          return func
      return decorate

(I don't see a reason to do this with a class.)

I'm not a fan of using function attributes for specifying author,
version, copyright etc.; those things usually work on a larger
granularity than methods anyway, and belong in the module doc string.

I'm still torn whether to promote defining properties this way:

  [propget]
  def x(self):
      "Doc string for x"
      return self.__x

  [propset]
  def x(self, newx):
      self.__x = newx

  [propdel]
  def x(self):
      del self.__x

but if people like this (whatever the decorator syntax :) we might as
well make this the recommended way to define properties.

Should there be a a separate module from which all those decorators
are imported, or should we make them built-ins, following the trend
set by classmethod etc.?  Not counting the ones that I already mark as
to be imported from a special place, like synchronized (from
threading.py) and doctest_script (from doctest.py).

Proposals for other standard decorators are welcome!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jacobs at theopalgroup.com  Fri Apr  2 15:03:15 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Fri Apr  2 15:03:18 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404021950.i32JoZN21741@guido.python.org>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>	<1080880130.22892.158.camel@localhost.localdomain>	<1080912282.5481.158.camel@anthem.wooz.org>	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
Message-ID: <406DC703.8050708@theopalgroup.com>

Guido van Rossum wrote:

>While I ponder the decorator syntax, let's propose some built-in
>decorators.
>
>We've already got classmethod and staticmethod.
>
>I propose this one to set function attributes:
>
>  class func_attrs(objects):
>
>      def __init__(self, **kwds):
>          self.attrs = kwds
>
>      def __call__(self, funcobj):
>          funcobj.__dict__.update(self.attrs)
>
>  
>
Did you leave out the 'return funcobj' from the end of __call__?  I thought
that decorators were supposed to be inherently cooperative, and should 
return
their modified funcobj, or a new func-like-obj.

Or maybe I haven't been paying close enough attention...

-Kevin


From guido at python.org  Fri Apr  2 15:08:03 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr  2 15:08:14 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: Your message of "Fri, 02 Apr 2004 15:03:15 EST."
	<406DC703.8050708@theopalgroup.com> 
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org> 
	<406DC703.8050708@theopalgroup.com> 
Message-ID: <200404022008.i32K83F21842@guido.python.org>

> >  class func_attrs(objects):
> >
> >      def __init__(self, **kwds):
> >          self.attrs = kwds
> >
> >      def __call__(self, funcobj):
> >          funcobj.__dict__.update(self.attrs)
> >
> Did you leave out the 'return funcobj' from the end of __call__?  I
> thought that decorators were supposed to be inherently cooperative,
> and should return their modified funcobj, or a new func-like-obj.

Sorry, you're right.  (I've been thinking of interpreting a None
result as "keep the input object" but the code generation would be too
messy.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jacobs at theopalgroup.com  Fri Apr  2 15:15:20 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Fri Apr  2 15:15:25 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404022008.i32K83F21842@guido.python.org>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
	<406DC703.8050708@theopalgroup.com>
	<200404022008.i32K83F21842@guido.python.org>
Message-ID: <406DC9D8.8040206@theopalgroup.com>

Guido van Rossum wrote:

>>> class func_attrs(objects):
>>>
>>>     def __init__(self, **kwds):
>>>         self.attrs = kwds
>>>
>>>     def __call__(self, funcobj):
>>>         funcobj.__dict__.update(self.attrs)
>>>
>>>      
>>>
>>Did you leave out the 'return funcobj' from the end of __call__?  I
>>thought that decorators were supposed to be inherently cooperative,
>>and should return their modified funcobj, or a new func-like-obj.
>>    
>>
>
>Sorry, you're right.  (I've been thinking of interpreting a None
>result as "keep the input object" but the code generation would be too
>messy.
>  
>

Cool.  And now that I have my pedantic hat on, I may as well go all 
out.  First,
why call it func_attrs, when staticmethod and classmethod are 
underscoreless?
Second, I know it is effectively the same, but shouldn't the .update 
line use
vars(funcobj) instead of funcobj.__dict__?   This is something that I am 
asked
(often!) by my Python students.  I use vars(obj) since it looks less 
magical.

-Kevin



From guido at python.org  Fri Apr  2 15:32:16 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr  2 15:32:27 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: Your message of "Fri, 02 Apr 2004 15:15:20 EST."
	<406DC9D8.8040206@theopalgroup.com> 
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
	<406DC703.8050708@theopalgroup.com>
	<200404022008.i32K83F21842@guido.python.org> 
	<406DC9D8.8040206@theopalgroup.com> 
Message-ID: <200404022032.i32KWG821938@guido.python.org>

> Cool.  And now that I have my pedantic hat on, I may as well go all
> out.  First, why call it func_attrs, when staticmethod and
> classmethod are underscoreless?

Just for variety. :-)

We had a discussion about naming conventions recently, and I believe
the outcome was that most folks like underscores in their names.  OTOH
if I was writing this just for *me*, I would indeed make them
underscoreless.

> Second, I know it is effectively the same, but shouldn't the .update
> line use vars(funcobj) instead of funcobj.__dict__?  This is
> something that I am asked (often!) by my Python students.  I use
> vars(obj) since it looks less magical.

No, vars() might return a copy.  __dict__ is the real thing (unless it
isn't -- try updating the __dict__ of a new-style class :-).

But perhaps the real implementation should use setattr() anyway --
some function attributes are not stored in the __dict__ (like __doc__)
and these should still be settable this way (if they are settable at
all).  And if they are *not* settable, this should raise an error
rather than silently creating an inaccessible entry in __dict__.

So let's rephrase that class as:

class funcattrs(object):
  def __init__(self, **kwds):
    self.attrs = kwds
  def __call__(self, func):
    for name, value in self.attrs.iteritems():
      setattr(func, name, value)
    return func

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz at pythoncraft.com  Fri Apr  2 15:33:17 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Apr  2 15:33:25 2004
Subject: [Python-Dev] PEP 327: Decimal data type
In-Reply-To: <A128D751272CD411BC9200508BC2194D0338381A@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D0338381A@escpl.tcp.com.ar>
Message-ID: <20040402203317.GA8514@panix.com>

On Wed, Mar 31, 2004, Batista, Facundo wrote:
> 
> This PEP is ready for your review.

Excellent work!  Only one nitpick:

> Syntax for All Cases
> ''''''''''''''''''''
> 
> ::
> 
>     Decimal(value1)
>     Decimal.from_float(value2, [decimal_digits])
> 
> where ``value1`` can be int, long, string, tuple or Decimal,
> ``value1`` can be only float, and ``decimal_digits`` is an optional
> int.

You mean value2 in the second case, right?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From pje at telecommunity.com  Fri Apr  2 15:35:26 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Apr  2 15:35:49 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <406DC9D8.8040206@theopalgroup.com>
References: <200404022008.i32K83F21842@guido.python.org>
	<200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
	<406DC703.8050708@theopalgroup.com>
	<200404022008.i32K83F21842@guido.python.org>
Message-ID: <5.1.1.6.0.20040402153139.02d704a0@telecommunity.com>

At 03:15 PM 4/2/04 -0500, Kevin Jacobs wrote:
>Guido van Rossum wrote:
>
>>>>class func_attrs(objects):
>>>>
>>>>     def __init__(self, **kwds):
>>>>         self.attrs = kwds
>>>>
>>>>     def __call__(self, funcobj):
>>>>         funcobj.__dict__.update(self.attrs)
>>>>
>>>>
>>>Did you leave out the 'return funcobj' from the end of __call__?  I
>>>thought that decorators were supposed to be inherently cooperative,
>>>and should return their modified funcobj, or a new func-like-obj.
>>>
>>
>>Sorry, you're right.  (I've been thinking of interpreting a None
>>result as "keep the input object" but the code generation would be too
>>messy.
>>
>
>Cool.  And now that I have my pedantic hat on, I may as well go all 
>out.  First,
>why call it func_attrs, when staticmethod and classmethod are underscoreless?
>Second, I know it is effectively the same, but shouldn't the .update line use
>vars(funcobj) instead of funcobj.__dict__?   This is something that I am asked
>(often!) by my Python students.  I use vars(obj) since it looks less magical.

For that matter, updating the dictionary is nice as a quick trick, but what 
if the object doesn't *have* a dictionary?  Using setattr would be safer, 
if this is intended to be subclassed.  Suppose, for example, that 
'synchronized' actually returns a callable wrapper written in C, rather 
than a function object, and that wrapper then delegates setattr to the 
thing it wraps?

So, I think using setattr would be safer, and __dict__.update() is 
premature optimization, given that function object creation and decorator 
invocation isn't likely to be a performance bottleneck.


From barry at python.org  Fri Apr  2 15:40:42 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr  2 15:41:00 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404022032.i32KWG821938@guido.python.org>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
	<406DC703.8050708@theopalgroup.com>
	<200404022008.i32K83F21842@guido.python.org>
	<406DC9D8.8040206@theopalgroup.com>
	<200404022032.i32KWG821938@guido.python.org>
Message-ID: <1080938441.5481.306.camel@anthem.wooz.org>

On Fri, 2004-04-02 at 15:32, Guido van Rossum wrote:

> We had a discussion about naming conventions recently, and I believe
> the outcome was that most folks like underscores in their names.  OTOH
> if I was writing this just for *me*, I would indeed make them
> underscoreless.

Actually, for multi-word method and function names, we preferred
underscores to camelCasing.  But I still wish has_key() were haskey()
and since neither "func" nor "attr" are actual words, funcattrs() is
just fine. :)

I'm torn on whether new decorators should go in as built-ins or not. 
It's darn convenient to stick them there, but I worry that the
documentation for the builtin module is getting pretty hefty now (and
no, I really don't want to do something like an os module split -- man,
I hate those pages ;).

-Barry



From dave at boost-consulting.com  Fri Apr  2 15:45:12 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Apr  2 15:46:23 2004
Subject: [Python-Dev] Re: Vote No on 318
References: <200404020313.i323DxL7024193@cosc353.cosc.canterbury.ac.nz>
	<008201c4186c$a9c96820$7329cb97@oemcomputer>
Message-ID: <ufzbmf6sn.fsf@boost-consulting.com>

"Raymond Hettinger" <python@rcn.com> writes:

>> > A failing common to all of the proposals is that they turn action
>> > words (like classmethod) into adjectives with implicit actions.
>
> [Greg Ewing]
>> Action words? I would have said "classmethod" is a noun, and as such
>> it's a perfectly good name for a decorator.
>
> Sure, the current name happens to be a noun instead of "classmethodify",
> but what it does is action oriented.   So, the real issue is declarative
> versus imperative.  Both are great, but don't mix well.

I disagree, especially where Python is concerned.  Declarations in
Python, if that even has meaning, are imperative by nature.  Try
running this:

  def f():
      g()

  f()

  def g():
      pass

The highest level languages are often declarative.  The fact that the
declarations may be executed is, as Greg says, an implementation
detail.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From tim.one at comcast.net  Fri Apr  2 15:48:37 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr  2 15:48:42 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404021950.i32JoZN21741@guido.python.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEKOJPAB.tim.one@comcast.net>

[Guido]
> ...
> doctest_script(multi_line_string) -- specifies a doctest script, for
> use by the doctest module (if this is absent, it will continue to look
> in the doc string).  I like to separate the doctest script from the
> actual documentation string because, when used for rigorous unit
> testing (as in Jim&Tim's talk at PyCON), the doctest script is too
> long to double as reasonable documentation.  Maybe this should be
> imported from the doctest module.

Let me try to squash this one before it gains traction from people who don't
use doctest anyway <wink>.  Burying a large triple-quoted string inside a
function call inside an indented "list" isn't going to help the readability
of anything.  Zope use is moving toward putting large doctests into their
own .py file, or, more recently, into their own .txt file.  An example of
the former:

    http://cvs.zope.org/ZODB/src/ZODB/tests/testmvcc.py

The idea that "too long" makes for unreasonable documentation doesn't make
sense to me:  these long doctests read like tutorials liberally sprinkled
with Python examples, and are the best docs some of these subsystems will
ever have.  Indeed, the Python Tutorial is a prime example of something that
"should be" a doctest, and I think that's exceptionally good documentation.

Of course this doesn't preclude putting a judiciously few examples in
docstrings either (also for doctest chewing), but I'd rather encourage
people who want more than that to go all the way, and write a tutorial-style
large doctest in its own file.


From dave at boost-consulting.com  Fri Apr  2 15:48:07 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Apr  2 15:50:31 2004
Subject: [Python-Dev] Re: PEP 318: Let's not give in to FUD
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<2m1xn6gwzp.fsf@starship.python.net>
	<200404021159.47419.fdrake@acm.org>
Message-ID: <u7jwyf6ns.fsf@boost-consulting.com>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> Barry Warsaw <barry@python.org> writes:
>  > At least with decorator-before-colon trying to run the code in older
>  > interpreters will barf loudly.
>
> On Friday 02 April 2004 11:34 am, Michael Hudson wrote:
>  > I think this is a good point that hadn't occurred to me: silent
>  > failures suck.
>
> There's also the little matter that using syntax only, without new keywords, 
> allows us to avoid __future__ imports and all the implementation cruft that 
> entails.
>
> Decorator-before-colon wins out big time for me.

+1

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From rowen at cesmail.net  Fri Apr  2 16:01:03 2004
From: rowen at cesmail.net (Russell E. Owen)
Date: Fri Apr  2 16:01:11 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
	<20040402105535.0526c27d.casey@zope.com>
Message-ID: <rowen-35CE89.13010302042004@sea.gmane.org>

In article <20040402105535.0526c27d.casey@zope.com>,
 Casey Duncan <casey@zope.com> wrote:

> Or how about as?
> 
>   as classmethod:
> 
>      def jinkies(cls):
>          pass
> 
> Which seems nice in the face of other declarations:
> 
>   as protected(some_permission):
> 
>      def zoinks(self, scooby, snack):
>          pass
> 
>      def jinkies(self):
>          pass
> 
> or
> 
>   [protected_by(some_permission)]:
> 
>       def dangerous(self, risk):
>           pass

I like this a lot. The effort to avoid new keywords, though laudable in 
many ways, comes at the cost of increased use of punctuation.

I generally find punctuation harder to read than keywords. Also, it is 
usually incomprehensible to someone unfamiliar with the language. The 
relative lack of punctuation in python as compared, say, to C is one if 
its great strengths, to my mind. It is one reason Python reads so much 
like pseudocode (or did before some of the recent changes such as list 
comprehensions -- very useful but a classic example of the problem of 
overloading punctuation).

I'd be very sad to see [] used for yet some other purpose.

-- Russell


From pedronis at bluewin.ch  Fri Apr  2 16:16:05 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Fri Apr  2 16:11:09 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <20040402105535.0526c27d.casey@zope.com>
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
Message-ID: <5.2.1.1.0.20040402231022.030c0058@pop.bluewin.ch>

At 10:55 02.04.2004 -0500, Casey Duncan wrote:

>   [protected_by(some_permission)]:
>
>       def dangerous(self, risk):
>           pass

I think the syntax (of which the proposed one is a subcase)

expr:
   suite

is better reserved for even more mischievous endeavors, even if just 
fantasized.



From bob at redivi.com  Fri Apr  2 16:22:13 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Apr  2 16:18:10 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <5.2.1.1.0.20040402231022.030c0058@pop.bluewin.ch>
References: <E1B8j8V-0006Mm-FD@mail.python.org>
	<1080759573.2043.49.camel@heinlein>
	<97B52378-834E-11D8-8201-000A95686CD8@redivi.com>
	<20040331234312.GA14993@mems-exchange.org>
	<2mekr7hp57.fsf@starship.python.net>
	<1080833042.22892.122.camel@localhost.localdomain>
	<u65cjpc8f.fsf@boost-consulting.com>
	<5.2.1.1.0.20040402231022.030c0058@pop.bluewin.ch>
Message-ID: <CF2FF471-84EB-11D8-8B4D-000A95686CD8@redivi.com>


On Apr 2, 2004, at 4:16 PM, Samuele Pedroni wrote:

> At 10:55 02.04.2004 -0500, Casey Duncan wrote:
>
>>   [protected_by(some_permission)]:
>>
>>       def dangerous(self, risk):
>>           pass
>
> I think the syntax (of which the proposed one is a subcase)
>
> expr:
>   suite
>
> is better reserved for even more mischievous endeavors, even if just 
> fantasized.

+1  :)


From aahz at pythoncraft.com  Fri Apr  2 20:16:47 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Apr  2 20:17:00 2004
Subject: [Python-Dev] PEP 328 -- relative and multi-line import
In-Reply-To: <200403302259.i2UMxLi10184@guido.python.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<20040330225324.GA1947@panix.com>
	<200403302259.i2UMxLi10184@guido.python.org>
Message-ID: <20040403011647.GA8816@panix.com>

On Tue, Mar 30, 2004, Guido van Rossum wrote:
> 
> Consider it Pronounced.

Okay, I've filed the updated PEP.  Who wants to implement it?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From dave at boost-consulting.com  Fri Apr  2 13:16:46 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Apr  2 20:38:55 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon --
	another new syntax idea
In-Reply-To: <007501c418c0$655eb6f0$6402a8c0@arkdesktop> (Andrew Koenig's
	message of "Fri, 2 Apr 2004 09:40:08 -0500")
References: <007501c418c0$655eb6f0$6402a8c0@arkdesktop>
Message-ID: <usmfmjldd.fsf@boost-consulting.com>

"Andrew Koenig" <ark-mlist@att.net> writes:

>> Has something along these lines been discussed?
>> 
>>   with [staticmethod, classmethod]:
>> 
>>      def foo(x):
>>          pass
>> 
>>      def bar(x):
>>          pass
>> 
>> IIUC, the PyObjC application needs whole swathes of functions with
>> the same decoration, but this syntax isn't much worse for one
>> function than for many.
>
> If I understand it correctly, the motivation for this idea is to try to
> avoid cluttering the definitions with large numbers of decorators.

Actually no.  It was that other syntaxes such as

as: [whatever]
def foo():
    pass

seemed to lack the scoping that needed to make the "as" part apply to
the definition.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From pythondev at bitfurnace.com  Fri Apr  2 15:22:02 2004
From: pythondev at bitfurnace.com (pythondev@bitfurnace.com)
Date: Fri Apr  2 20:38:58 2004
Subject: [Python-Dev] PEP 318: Is decorators the right term for these things?
Message-ID: <26182.216.254.103.139.1080937322.squirrel@my.modwest.com>

I know this discussion has focussed on the term "decorator", but I think
the term is a misnomer. These things arent just passive, static
decorations, but are rather more dynamic active things; possibly
"transformations" or "transforms".

<ducks>



From bob at redivi.com  Fri Apr  2 20:51:35 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Apr  2 20:47:30 2004
Subject: [Python-Dev] PEP 318: Is decorators the right term for these
	things?
In-Reply-To: <26182.216.254.103.139.1080937322.squirrel@my.modwest.com>
References: <26182.216.254.103.139.1080937322.squirrel@my.modwest.com>
Message-ID: <706B841B-8511-11D8-AE8C-000A95686CD8@redivi.com>

On Apr 2, 2004, at 3:22 PM, <pythondev@bitfurnace.com> wrote:

> I know this discussion has focussed on the term "decorator", but I 
> think
> the term is a misnomer. These things arent just passive, static
> decorations, but are rather more dynamic active things; possibly
> "transformations" or "transforms".
>
> <ducks>

Sometimes they are, sometimes they aren't.  Many of the useful cases 
actually are transforms (classmethod, staticmethod, objc.selector), not 
merely decoration (as an interface declaration may be, it could return 
the original function unchanged, it really depends on the adaptation 
framework).

I do pretty much agree with you though.

-bob


From edloper at gradient.cis.upenn.edu  Fri Apr  2 21:19:17 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Fri Apr  2 21:17:41 2004
Subject: [Python-Dev] Re: PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <E1B9VOO-0007uv-Ck@mail.python.org>
References: <E1B9VOO-0007uv-Ck@mail.python.org>
Message-ID: <406E1F25.9090702@gradient.cis.upenn.edu>

> deprecated(comment_string) -- mostly for documentation, but would set
> the deprecated attribute.  The comment_string argument is a string
> explaining why it is deprecated.  Maybe this should also have
> arguments specifying the first Python release in which it was
> considered deprecated and the last Python release (if known) where it
> will be available.

Should this be a python-specific decorator?  Or should it also be used 
to mark deprecated objects in external libraries?  (The answer will 
affect the arguments specifing when it became depreciated and when it 
will go  away.)

If we add this, we might want to also define a standard way of marking 
classes and modules as depreciated.  (Perhaps by setting the 
__deprecated__ variable/property in all 3 cases.)

> overrides -- indicates that this overrides a base class method.  Maybe
> the default metaclass could check that if this is used there actually
> is a corresponding base class method, and we might have a "strict"
> metaclass that checks this is set for all overriding methods.

Alternatively, it could be checked by an external inspection tool like 
pychecker.  (Since metaclasses are so difficult to mix, it's probably 
better not to use them when we don't need to.)

-Edward


From python at rcn.com  Sat Apr  3 04:27:52 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Apr  3 04:28:31 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404021950.i32JoZN21741@guido.python.org>
Message-ID: <004601c4195d$f03255a0$9433c797@oemcomputer>

> While I ponder the decorator syntax, let's propose some built-in
> decorators.

Okay, I'll play (despite my thumbs down on the pep and especially the
syntax using a list prefixing the function).

Several people have emphasized that the timing of decorator application
is only an implementation detail.  So, that leaves open the possibility
of using function decorators as compilation pragmas.

One possible pragma tells the compiler that the function may assume that
specified global variables are constant and already known.  This allows
the globals to be loaded in the constant table and accessed with
LOAD_GLOBAL instead of LOAD_CONSTANT.

Another pragma tells the compiler to assume that specified attribute
lookup targets won't change outside the function.  This allows the
lookups to be precomputed as soon as the root is available or whenever
it changes.


Example
-------
X = 20

[earlybind(int, X),
 cacheattr("mylist.append", "result.append", "random.random",
"self.score")]
def meth(self, data) 
    result = []
    for sub in data:
        mylist = []
        for elem in sub:
            if int(elem) == X:
                mylist.append(elem)
            else:
                elem += random.random()
                self.score(elem)
        result.append(mylist)
    return result



Transformation
--------------
def meth(self, data, int=int, X=20)
    _rr = random.random
    _ss = self.score
    result = []
    _ra = result.append
    for sub in data:
        mylist = []
        _ma = mylist.append
        for elem in sub:
            if int(elem) == X:
                _ma(elem)
            else:
                elem += _rr()
                _ss(elem)
        _ra(mylist)
    return result


Adding the decorator line is sufficient to localize everything in the
function, eliminate all lookups from the inner loop, and still keep the
code clean and readable.


with-this-much-weirdness-who-needs-to-go-pysco-ly yours,



Raymond Hettinger


From mwh at python.net  Sat Apr  3 05:58:57 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Apr  3 05:59:02 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404021950.i32JoZN21741@guido.python.org> (Guido van
	Rossum's message of "Fri, 02 Apr 2004 11:50:35 -0800")
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
Message-ID: <2mwu4xfhu6.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> While I ponder the decorator syntax, let's propose some built-in
> decorators.

[...]

> We could also add a standard implementation of synchronized.  Or
> perhaps that should be imported from threading.  (But is that really
> a good thing to copy from Java?)

I don't want to sound FUDdy, but I was under the impression that
people didn't think this is something we actually want...

> Other possibilities (all these are pretty much thinking aloud and very
> much up for votes; and not original, I've seen these proposed
> before!):

[...]

> overrides -- indicates that this overrides a base class method.  Maybe
> the default metaclass could check that if this is used there actually
> is a corresponding base class method, and we might have a "strict"
> metaclass that checks this is set for all overriding methods.

I don't get the point of this.

> I'm still torn whether to promote defining properties this way:
>
>   [propget]
>   def x(self):
>       "Doc string for x"
>       return self.__x
>
>   [propset]
>   def x(self, newx):
>       self.__x = newx
>
>   [propdel]
>   def x(self):
>       del self.__x
>
> but if people like this (whatever the decorator syntax :) we might as
> well make this the recommended way to define properties.

- "a bit"

I'm not sure I like the idea of promoting something that uses
sys._getframe.

> Should there be a a separate module from which all those decorators
> are imported, or should we make them built-ins, following the trend
> set by classmethod etc.?

I'm not sure any of the above are important enough to be builtins.
funcattrs, maybe, but there's sod all reason to implement that in C,
and getting Python implementations into the builtins seems likely to
be pain.  Could do it in site.py, I guess.

Cheers,
mwh

-- 
  Also, remember to put the galaxy back when you've finished, or an
  angry mob of astronomers will come round and kneecap you with a
  small telescope for littering. 
       -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year

From goodger at python.org  Sat Apr  3 07:52:03 2004
From: goodger at python.org (David Goodger)
Date: Sat Apr  3 07:52:15 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200403302218.i2UMIEV10058@guido.python.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
Message-ID: <406EB373.2030401@python.org>

Guido van Rossum wrote:
 > At least "from . import X" and "from .. import X" are completely
 > clear and more levels up are not likely to occur in practice...

I have Aahz's revision ready to check in, but I'm unclear on one
point.  Is the accepted/pronounced relative import syntax dots only,
or can there be a branch too?  In other words, is the "Y" here
acceptable (where Y is a sibling to the current module's package)?

     from ..Y import X

I think the PEP needs a clear statement (and examples) of which syntax
**and semantics** have been chosen.

-- David Goodger

From srichter at cosmos.phy.tufts.edu  Sat Apr  3 08:35:20 2004
From: srichter at cosmos.phy.tufts.edu (Stephan Richter)
Date: Sat Apr  3 08:35:42 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <406EB373.2030401@python.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org>
Message-ID: <200404030835.20989.srichter@cosmos.phy.tufts.edu>

On Saturday 03 April 2004 07:52, David Goodger wrote:
> Guido van Rossum wrote:
> ?> At least "from . import X" and "from .. import X" are completely
> ?> clear and more levels up are not likely to occur in practice...
>
> I have Aahz's revision ready to check in, but I'm unclear on one
> point. ?Is the accepted/pronounced relative import syntax dots only,
> or can there be a branch too? ?In other words, is the "Y" here
> acceptable (where Y is a sibling to the current module's package)?
>
> ? ? ?from ..Y import X
>
> I think the PEP needs a clear statement (and examples) of which syntax
> **and semantics** have been chosen.

If Y would not be allowed, the new syntax would be cumbersome and only half as 
productive. In Zope 3 we often have the following case:

packageX
? utilities.py (contains a bunch of helper functions)
? browser (package containing browser presentation code)
? ? ?someview.py (a specific browser view)

We often want to reach utilities from someview. So the ideal import for me 
would be:

from .utilities import somefunc

Also, as mentioned by Jim before, we have used this type of syntax for 
specifying Python objects in ZCML for a while, and it works well for us.

Regards,
Stephan
-- 
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training

From aahz at pythoncraft.com  Sat Apr  3 09:50:51 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 09:51:01 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <406EB373.2030401@python.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org>
Message-ID: <20040403145050.GA8626@panix.com>

On Sat, Apr 03, 2004, David Goodger wrote:
> 
> I have Aahz's revision ready to check in, but I'm unclear on one
> point.  Is the accepted/pronounced relative import syntax dots only,
> or can there be a branch too?  In other words, is the "Y" here
> acceptable (where Y is a sibling to the current module's package)?
> 
>     from ..Y import X
> 
> I think the PEP needs a clear statement (and examples) of which syntax
> **and semantics** have been chosen.

Good point.  Here's what I *think* the semantics are; as soon as I get
agreement, I'll update the PEP.  Given a package layout::

    package
        subpackage1
            moduleX
            moduleY
        subpackage2
            moduleZ
        moduleA

Each leading "." refers to one level of parent.  Assuming that the
current file is ``moduleX.py``, following are correct usages of the new
syntax::

    from .moduleY import spam
    from ..subpackage1 import moduleY
    from ..subpackage2.moduleZ import eggs
    from ..moduleA import foo
    from ...package import bar

Actually, I believe that this is the only possible useful semantic; I
would welcome clear reasoning for another semantic.

Here's a question.  Should the following be legal? ::

    from ...sys import path

If not, do we test for it?  What kind of error do we raise?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From dave at boost-consulting.com  Sat Apr  3 09:57:50 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Sat Apr  3 09:58:37 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org>
Message-ID: <ullldf6s1.fsf@boost-consulting.com>

David Goodger <goodger@python.org> writes:

> Guido van Rossum wrote:
>  > At least "from . import X" and "from .. import X" are completely
>  > clear and more levels up are not likely to occur in practice...
>
> I have Aahz's revision ready to check in, but I'm unclear on one
> point.  Is the accepted/pronounced relative import syntax dots only,
> or can there be a branch too?  In other words, is the "Y" here
> acceptable (where Y is a sibling to the current module's package)?
>
>      from ..Y import X

Wouldn't that have to be

       from ...Y import X
              ^-------------separator, as in A.B
??

Or was that what you intended, David?

I'm thinking some other syntax is needed to represent upward travel
in the hierarchy, e.g.

       from ^.Y import X
       from -.Y import X

<ducks>

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From goodger at python.org  Sat Apr  3 10:44:13 2004
From: goodger at python.org (David Goodger)
Date: Sat Apr  3 10:44:18 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <ullldf6s1.fsf@boost-consulting.com>
References: <200403302218.i2UMIEV10058@guido.python.org>	<406EB373.2030401@python.org>
	<ullldf6s1.fsf@boost-consulting.com>
Message-ID: <406EDBCD.30603@python.org>

> David Goodger <goodger@python.org> writes:
>> In other words, is the "Y" here acceptable
>> (where Y is a sibling to the current module's package)?
>>
>>     from ..Y import X

David Abrahams wrote:
> Wouldn't that have to be
> 
>        from ...Y import X
>               ^-------------separator, as in A.B
> ??

Aha!  My confusion is confirmed.

I hadn't thought of requiring a separator "."; that seems like
overloading of meaning.  I understood the syntax as Aahz
describes it, so a single leading "." means "current package"
(i.e., sibling of current module), ".." means "parent package"
or "up one level", etc.

> I'm thinking some other syntax is needed to represent upward travel
> in the hierarchy, e.g.
> 
>        from ^.Y import X

That has a certain elegance to it.  So ".Y" would mean import
from current package, "^.Y" would mean import from parent
package, "^^.Y" from parent's parent package, etc.

I don't want to reopen debate needlessly, but the overloading
of the meaning of "." *is* a bit confusing here.

Whatever makes sense to Guido will eventually make sense to me,
so I'm not worried.

-- David Goodger

From martin at v.loewis.de  Sat Apr  3 11:15:40 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Apr  3 11:15:47 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <200404021950.i32JoZN21741@guido.python.org>
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>	<1080880130.22892.158.camel@localhost.localdomain>	<1080912282.5481.158.camel@anthem.wooz.org>	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
Message-ID: <406EE32C.9000207@v.loewis.de>

Guido van Rossum wrote:
> I'm still torn whether to promote defining properties this way:
> 
>   [propget]
>   def x(self):
>       "Doc string for x"
>       return self.__x
> 
>   [propset]
>   def x(self, newx):
>       self.__x = newx
> 
>   [propdel]
>   def x(self):
>       del self.__x
> 
> but if people like this (whatever the decorator syntax :) we might as
> well make this the recommended way to define properties.

Does that actually work? I.e. is there an implementation of propget,
propset, propdel so that this code introduces a property x?

My understanding is that above syntax would be short for>   [propget]
     def x(self):
        "Doc string for x"
        return self.__x
     x = propget(x)

     def x(self, newx):
        self.__x = newx
     x = propset(x)

     def x(self):
        del self.__x
     x = propdel(x)

Later assignments to x would override earlier ones, so that only
the propdel survives.

Regards,
Martin



From pje at telecommunity.com  Sat Apr  3 11:32:45 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Apr  3 11:27:09 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <406EE32C.9000207@v.loewis.de>
References: <200404021950.i32JoZN21741@guido.python.org>
	<200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
Message-ID: <5.1.0.14.0.20040403112915.038069e0@mail.telecommunity.com>

At 06:15 PM 4/3/04 +0200, Martin v. L?wis wrote:
>Guido van Rossum wrote:
>>I'm still torn whether to promote defining properties this way:
>>   [propget]
>>   def x(self):
>>       "Doc string for x"
>>       return self.__x
>>   [propset]
>>   def x(self, newx):
>>       self.__x = newx
>>   [propdel]
>>   def x(self):
>>       del self.__x
>>but if people like this (whatever the decorator syntax :) we might as
>>well make this the recommended way to define properties.
>
>Does that actually work? I.e. is there an implementation of propget,
>propset, propdel so that this code introduces a property x?

Yes, using sys._getframe().f_locals[function.func_name].  Someone posted a 
link to an implementation earlier this week.


>My understanding is that above syntax would be short for>   [propget]
>     def x(self):
>        "Doc string for x"
>        return self.__x
>     x = propget(x)
>
>     def x(self, newx):
>        self.__x = newx
>     x = propset(x)
>
>     def x(self):
>        del self.__x
>     x = propdel(x)
>
>Later assignments to x would override earlier ones, so that only
>the propdel survives.

Technically, what you show is not the actual expansion of the new 
syntax.  The new syntax applies decorators before binding 'x' to the new 
function.  So, the old value of 'x' is available to a decorator via 
sys._getframe().f_locals.  This technique is also useful for implementing 
generic functions and/or multimethods, signature-based overloading, etc.


From guido at python.org  Sat Apr  3 11:56:52 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 11:56:58 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 07:52:03 EST."
	<406EB373.2030401@python.org> 
References: <200403302218.i2UMIEV10058@guido.python.org>  
	<406EB373.2030401@python.org> 
Message-ID: <200404031656.i33Guq524029@guido.python.org>

> Guido van Rossum wrote:
>  > At least "from . import X" and "from .. import X" are completely
>  > clear and more levels up are not likely to occur in practice...
> 
> I have Aahz's revision ready to check in, but I'm unclear on one
> point.  Is the accepted/pronounced relative import syntax dots only,
> or can there be a branch too?  In other words, is the "Y" here
> acceptable (where Y is a sibling to the current module's package)?
> 
>      from ..Y import X

Yes.

> I think the PEP needs a clear statement (and examples) of which syntax
> **and semantics** have been chosen.

Ah, I haven't had the time to read the PEP carefully.  Apparently it
can use a better specification (like so many PEPs these days... :-( ).

Don't wait for me; it's clear that someone needs to do some more work
on this (if just to spell out what seems obvious).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr  3 11:57:59 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 11:58:05 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 09:50:51 EST."
	<20040403145050.GA8626@panix.com> 
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <20040403145050.GA8626@panix.com> 
Message-ID: <200404031657.i33Gvxl24044@guido.python.org>

> Good point.  Here's what I *think* the semantics are; as soon as I get
> agreement, I'll update the PEP.  Given a package layout::
> 
>     package
>         subpackage1
>             moduleX
>             moduleY
>         subpackage2
>             moduleZ
>         moduleA
> 
> Each leading "." refers to one level of parent.  Assuming that the
> current file is ``moduleX.py``, following are correct usages of the new
> syntax::
> 
>     from .moduleY import spam
>     from ..subpackage1 import moduleY
>     from ..subpackage2.moduleZ import eggs
>     from ..moduleA import foo
>     from ...package import bar
> 
> Actually, I believe that this is the only possible useful semantic; I
> would welcome clear reasoning for another semantic.

All correct.

> Here's a question.  Should the following be legal? ::
> 
>     from ...sys import path
> 
> If not, do we test for it?  What kind of error do we raise?

Legal but insane.  Not an error.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sat Apr  3 11:58:26 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 11:58:31 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <ullldf6s1.fsf@boost-consulting.com>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <ullldf6s1.fsf@boost-consulting.com>
Message-ID: <20040403165826.GA8803@panix.com>

On Sat, Apr 03, 2004, David Abrahams wrote:
>
> I'm thinking some other syntax is needed to represent upward travel
> in the hierarchy, e.g.
> 
>        from ^.Y import X
>        from -.Y import X
> 
> <ducks>

That's been suggested, and it'll show up in the revised PEP as soon as
it's pushed out to python.org.  I'm not Uncle Timmy, but I'll channel
Guido: no chance.  I don't particularly like dots-only mechanism, but I
have to agree that it works well enough for one-to-three levels.  More
than that is a strong indication of a design that needs reworking.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From guido at python.org  Sat Apr  3 11:58:51 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 11:58:57 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 09:57:50 EST."
	<ullldf6s1.fsf@boost-consulting.com> 
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> 
	<ullldf6s1.fsf@boost-consulting.com> 
Message-ID: <200404031658.i33Gwps24056@guido.python.org>

> >      from ..Y import X
> 
> Wouldn't that have to be
> 
>        from ...Y import X
>               ^-------------separator, as in A.B

NO, NO, NO!  Each dot is one uplevel.  Exactly like Mac OS 9 paths do
with ':'.  (Read macpath.py.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr  3 11:59:35 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 11:59:41 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 10:44:13 EST."
	<406EDBCD.30603@python.org> 
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <ullldf6s1.fsf@boost-consulting.com> 
	<406EDBCD.30603@python.org> 
Message-ID: <200404031659.i33GxZV24069@guido.python.org>

> > I'm thinking some other syntax is needed to represent upward travel
> > in the hierarchy, e.g.
> > 
> >        from ^.Y import X
> 
> That has a certain elegance to it.

Let me nip that one in the bud.  NO WAY.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From Scott.Daniels at Acm.Org  Sat Apr  3 11:59:38 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sat Apr  3 11:59:47 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <E1B9o04-0007ZI-Tu@mail.python.org>
References: <E1B9o04-0007ZI-Tu@mail.python.org>
Message-ID: <406EED7A.5090908@Acm.Org>

One more (possibly unclear) case to consider:
     import . as package
Is this a way to name your parent package?

-- 
-Scott David Daniels
Scott.Daniels@Acm.Org

From guido at python.org  Sat Apr  3 12:01:27 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 12:01:33 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: Your message of "Sat, 03 Apr 2004 18:15:40 +0200."
	<406EE32C.9000207@v.loewis.de> 
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org> 
	<406EE32C.9000207@v.loewis.de> 
Message-ID: <200404031701.i33H1Ra24093@guido.python.org>

> > I'm still torn whether to promote defining properties this way:
> > 
> >   [propget]
> >   def x(self):
> >       "Doc string for x"
> >       return self.__x
> > 
> >   [propset]
> >   def x(self, newx):
> >       self.__x = newx
> > 
> >   [propdel]
> >   def x(self):
> >       del self.__x
> > 
> > but if people like this (whatever the decorator syntax :) we might as
> > well make this the recommended way to define properties.
> 
> Does that actually work? I.e. is there an implementation of propget,
> propset, propdel so that this code introduces a property x?

Yes, but it involves sys._getframe().

> My understanding is that above syntax would be short for>   [propget]
>      def x(self):
>         "Doc string for x"
>         return self.__x
>      x = propget(x)
> 
>      def x(self, newx):
>         self.__x = newx
>      x = propset(x)
> 
>      def x(self):
>         del self.__x
>      x = propdel(x)
> 
> Later assignments to x would override earlier ones, so that only
> the propdel survives.

No, the semantics of decorators are that the function name isn't
actually assigned to for the first time until all decorators have been
called.  So the previous binding of that name is available to devious
decorators like these.  (This wasn't my idea -- someone else proposed
it here first. :)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sat Apr  3 12:02:27 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 12:02:31 2004
Subject: [Python-Dev] python.sf.net
Message-ID: <20040403170227.GA9380@panix.com>

Could someone with admin privs update python.sf.net to include a link to
the Python project on SF?

http://www.sf.net/projects/python/

Just seems like a useful convenience.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From dave at boost-consulting.com  Sat Apr  3 12:02:05 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Sat Apr  3 12:02:51 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404031658.i33Gwps24056@guido.python.org> (Guido van
	Rossum's message of "Sat, 03 Apr 2004 08:58:51 -0800")
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <ullldf6s1.fsf@boost-consulting.com>
	<200404031658.i33Gwps24056@guido.python.org>
Message-ID: <uekr5dmgi.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> >      from ..Y import X
>> 
>> Wouldn't that have to be
>> 
>>        from ...Y import X
>>               ^-------------separator, as in A.B
>
> NO, NO, NO!  Each dot is one uplevel.  Exactly like Mac OS 9 paths do
> with ':'.  (Read macpath.py.)

That's a fine convention; I'm familiar with MacOS paths.  
The fact that David Goodger and I were both confused might indicate
that the proposed syntax might not be as intuitive as you'd like, but
I don't think it'll take long to get used to.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From guido at python.org  Sat Apr  3 12:07:14 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 12:07:31 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: Your message of "Sat, 03 Apr 2004 11:58:57 +0100."
	<2mwu4xfhu6.fsf@starship.python.net> 
References: <200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org> 
	<2mwu4xfhu6.fsf@starship.python.net> 
Message-ID: <200404031707.i33H7El24139@guido.python.org>

> > We could also add a standard implementation of synchronized.  Or
> > perhaps that should be imported from threading.  (But is that really
> > a good thing to copy from Java?)
> 
> I don't want to sound FUDdy, but I was under the impression that
> people didn't think this is something we actually want...

Hence my parenthetical remark.

But given that it is quite simple to do using decorators, surely
*someone* will implement it.  How useful it will be remains to be
seen.  Maybe someone should take this idea, run with it, and report
how it enhanced their project?

> > overrides -- indicates that this overrides a base class method.
> > Maybe the default metaclass could check that if this is used there
> > actually is a corresponding base class method, and we might have a
> > "strict" metaclass that checks this is set for all overriding
> > methods.
> 
> I don't get the point of this.

First, it's sometimes useful to know when subclassing which methods
you are overriding and which you are adding -- this may help the
reader understanding what's going on.  And then of course it would be
useful to have automatic verification of this information so that the
reader can actually believe it.

> - "a bit"
> 
> I'm not sure I like the idea of promoting something that uses
> sys._getframe.

Me neither.

> > Should there be a a separate module from which all those decorators
> > are imported, or should we make them built-ins, following the trend
> > set by classmethod etc.?
> 
> I'm not sure any of the above are important enough to be builtins.
> funcattrs, maybe, but there's sod all reason to implement that in C,
> and getting Python implementations into the builtins seems likely to
> be pain.  Could do it in site.py, I guess.

It would be simple enough in C.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr  3 12:11:15 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 12:11:24 2004
Subject: [Python-Dev] Re: PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: Your message of "Fri, 02 Apr 2004 21:19:17 EST."
	<406E1F25.9090702@gradient.cis.upenn.edu> 
References: <E1B9VOO-0007uv-Ck@mail.python.org>  
	<406E1F25.9090702@gradient.cis.upenn.edu> 
Message-ID: <200404031711.i33HBGX24158@guido.python.org>

> > deprecated(comment_string) -- mostly for documentation, but would set
> > the deprecated attribute.  The comment_string argument is a string
> > explaining why it is deprecated.  Maybe this should also have
> > arguments specifying the first Python release in which it was
> > considered deprecated and the last Python release (if known) where it
> > will be available.
> 
> Should this be a python-specific decorator?  Or should it also be used 
> to mark deprecated objects in external libraries?  (The answer will 
> affect the arguments specifing when it became depreciated and when it 
> will go away.)

I guess there could be a C-level equivalent way of getting the same
attributes set, so that introspecting tools might be able to
understand those, too.  The aguments would probably not be machine
interpretable, so considerably liberty should be allowed as long as
some conventions are established.  I'm not sufficiently interested in
pursueing this to see it through as a spec though.

> If we add this, we might want to also define a standard way of marking 
> classes and modules as depreciated.  (Perhaps by setting the 
> __deprecated__ variable/property in all 3 cases.)

If you or someone else is interested in doing this, please consider
writing a separate PEP.

> > overrides -- indicates that this overrides a base class method.
> > Maybe the default metaclass could check that if this is used there
> > actually is a corresponding base class method, and we might have a
> > "strict" metaclass that checks this is set for all overriding
> > methods.
> 
> Alternatively, it could be checked by an external inspection tool like 
> pychecker.  (Since metaclasses are so difficult to mix, it's probably 
> better not to use them when we don't need to.)

Of course.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sat Apr  3 12:12:48 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 12:12:54 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <406EED7A.5090908@Acm.Org>
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>
Message-ID: <20040403171247.GC8803@panix.com>

On Sat, Apr 03, 2004, Scott David Daniels wrote:
>
> One more (possibly unclear) case to consider:
>     import . as package
> Is this a way to name your parent package?

Nope.  All uses of relative imports require ``from <something> import``
(I think the PEP makes that clear).  You could, however, do

    from ... import package [as foo]

if Guido permits it; I want a Pronouncement on that before adding it to
the PEP.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From pje at telecommunity.com  Sat Apr  3 12:32:37 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Apr  3 12:27:03 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <200404031707.i33H7El24139@guido.python.org>
References: <Your message of "Sat,
	03 Apr 2004 11:58:57 +0100." <2mwu4xfhu6.fsf@starship.python.net>
	<200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<200404021950.i32JoZN21741@guido.python.org>
	<2mwu4xfhu6.fsf@starship.python.net>
Message-ID: <5.1.0.14.0.20040403122854.0281cec0@mail.telecommunity.com>

At 09:07 AM 4/3/04 -0800, Guido van Rossum wrote:
> > > We could also add a standard implementation of synchronized.  Or
> > > perhaps that should be imported from threading.  (But is that really
> > > a good thing to copy from Java?)
> >
> > I don't want to sound FUDdy, but I was under the impression that
> > people didn't think this is something we actually want...
>
>Hence my parenthetical remark.
>
>But given that it is quite simple to do using decorators, surely
>*someone* will implement it.  How useful it will be remains to be
>seen.  Maybe someone should take this idea, run with it, and report
>how it enhanced their project?

For what it's worth, Jim Fulton's ExtensionClass package has had a 
"synchronized" metaclass implemented in C since about...  1997?  I don't 
know of anybody actually using it, though.

For my few dabblings into objects being shared across threads, though, I've 
typically needed rather precise control over when things get locked and 
unlocked and how, so I've never bothered implementing a synchronizing 
decorator.


From bh at intevation.de  Sat Apr  3 13:24:24 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Sat Apr  3 13:24:31 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
Message-ID: <s9zwu4wdinb.fsf@salmakis.intevation.de>



I have some comments on PEP 328 (absolute/relative imports).  A brief
summary would be "-1", mostly because it unnecessarily breaks backwards
compatibility and will cause extra work just to keep up with Python.


Missing Rationale for the Incompatibility
-----------------------------------------

First of all, I think the rationale for the absolute imports is missing
some important points.

>From the pep:

    In current Python, if you're reading a module located inside a
    package, it is not clear whether ::

       import foo

    refers to a top-level module or to another module inside the package.

I don't think this is a good description of the actual problem.  The
problem is that the modules in the package shadow global ones and
there's no easy way to bypass the local package.  FWIW, though, I don't
recall ever running into a problem with that.

    To resolve the ambiguity, it is proposed that ``foo`` will always be
    a module or package reachable from ``sys.path``.


What I'm missing here is an explanation *why* this is the proposed
solution.  From my point of view the problem happens too rarely to
justify breaking existing code.  It's not even difficult to come up with
a backwards compatible solution.  In the few cases where a local module
shadows a global one, the import could flaged as absolute with the
global keyword, for instance (as has been suggested in one of the
threads referenced in the pep):

    import global time
    from global foo import bar

Unlike the proposed solution, this not only retains compatibility, it
would also be similar to how conflicts between local and global scopes
are handled elsewhere in Python.  For instance, local variables hide
globals and builtins.  If you later need to access a shadowed global you
need to e.g. rename your local variables or resort to some other way to
get at the global.  Why, then should local, i.e. relative, imports not
shadow global ones by default unless a globel, i.e. absolute, import is
explicitly requested?


Python-Version Independent Code
-------------------------------

I like relative imports and it's good to see that the pep proposes to
keep them and even to extend their capabilities.  However, I also like
to write code that runs on a range of Python versions, e.g. 2.2 and
later.  Unfortunately, if, as is hinted at in the pep, the new syntax
becomes the default in 2.5, there will no way to write relative imports
that will work in both 2.3 and 2.5.  So, for me the pep effectively
abolishes relative imports completely.  It also means that I will have
to go through all my code and change relative imports to absolute
imports, costing me at least a few hours that I would rather spend
enhancing my code instead of just keeping up with Python.


Deprecation Warnings
--------------------

Another thing missing is a note about DeprecationWarnings.  Since the
change will break exisiting code, DeprecationWarnings should be given
for relative imports at some point before the new semantics become the
default.


Leading Dots
------------

Leading dots were also recently mentioned to be used for a
with-statement.  Won't two different uses for leading dots be confusing
for newbies?  E.g. in

    with some_object:
	.a = 1
	from .magic import xyzzy

the leading dot in the import statement won't have anything to do with
the with statement.  I guess code like the above would be rare, but some
people expressed concern about the use of "as" for decorators for
similar reasons.


   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From tim.one at comcast.net  Sat Apr  3 13:36:42 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sat Apr  3 13:36:45 2004
Subject: [Python-Dev] python.sf.net
In-Reply-To: <20040403170227.GA9380@panix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEOHJPAB.tim.one@comcast.net>

[Aahz]
> Could someone with admin privs update python.sf.net to include a link
> to the Python project on SF?
>
> http://www.sf.net/projects/python/
>
> Just seems like a useful convenience.

I'm not exactly sure what you're after here.  Is this something not already
covered by the "Outside links" section on the left-side navigation bar
(which includes to "SF Project Page" (the URL you gave above), and links to
the SF bug tracker, SF patch tracker, SF CVS instructions, etc).


From aahz at pythoncraft.com  Sat Apr  3 13:56:49 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 13:56:57 2004
Subject: [Python-Dev] python.sf.net
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEOHJPAB.tim.one@comcast.net>
References: <20040403170227.GA9380@panix.com>
	<LNBBLJKPBEHFEDALKOLCGEOHJPAB.tim.one@comcast.net>
Message-ID: <20040403185649.GA1379@panix.com>

On Sat, Apr 03, 2004, Tim Peters wrote:
> [Aahz]
>>
>> Could someone with admin privs update python.sf.net to include a link
>> to the Python project on SF?
>>
>> http://www.sf.net/projects/python/
>>
>> Just seems like a useful convenience.
> 
> I'm not exactly sure what you're after here.  Is this something not
> already covered by the "Outside links" section on the left-side
> navigation bar (which includes to "SF Project Page" (the URL you gave
> above), and links to the SF bug tracker, SF patch tracker, SF CVS
> instructions, etc).

Ah, you're looking at the redirect page of http://www.python.org/dev/
Someone who turns redirects off just sees a link to /dev/
I figured that someone hitting python.sf.net would be likely interested
specifically in the Python page on SF.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From aahz at pythoncraft.com  Sat Apr  3 14:11:06 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 14:11:14 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: <s9zwu4wdinb.fsf@salmakis.intevation.de>
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
Message-ID: <20040403191106.GA4254@panix.com>

On Sat, Apr 03, 2004, Bernhard Herzog wrote:
>
> What I'm missing here is an explanation *why* this is the proposed
> solution.  From my point of view the problem happens too rarely to
> justify breaking existing code.  It's not even difficult to come up with
> a backwards compatible solution.  

Barry, would you please write up something for the PEP here?  (You're
the person who pushed this the most.)  I've already updated the PEP to
include clarification of the shadowing issue.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From arigo at tunes.org  Sat Apr  3 14:59:57 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sat Apr  3 15:03:28 2004
Subject: [Python-Dev] Python is faster than C
Message-ID: <20040403195957.GA30613@vicky.ecs.soton.ac.uk>

Hi!

This is a rant against the optimization trend of the Python interpreter.

Sorting a list of 100000 integers in random order takes:

* 0.75 seconds in Python 2.1
* 0.51 seconds in Python 2.2
* 0.46 seconds in Python 2.3

Tim Peters did a great job optimizing list.sort().  If I try with a
simple, non-stable pure Python quicksort implementation, in Python 2.3:

* 4.83 seconds
* 0.21 seconds with Psyco

First step towards world domination of high-level languages :-)

The reason that Psyco manages to outperform the C implementation is not
that gcc is a bad compiler (it is about 10 times better than Psyco's).
The reason is that the C implementation must use a generic '<' operator
to compare elements, while the Psyco version quickly figures out that it
can expect to find ints in the list; it still has to check this
assumption, but this is cheap and then the comparison is done with a
single machine instruction.

Similarily, here are some results about the heapq module, which is
rewritten in C in the CVS tree for Python 2.4:

    l = [random.random() for x in range(200000)]
    heapq.heapify(l)

This code executes on my laptop in:

* 1.96 seconds on Python 2.3 (pure Python)
* 0.18 seconds on Python 2.4cvs (rewritten in C)
* 0.16 seconds on Python 2.3 with Psyco

So this is not so much a plug for Psyco as a rant against the current
trend of rewriting standard modules in C.  Premature optimization and
all that.

Worse, and more importantly, the optimization starts to become visible
to the programmer.  Iterators, for example, are great in limited cases
but I consider their introduction a significant complication in the
language; before, you could expect that some function from which you
would expect a sequence returned a list.  Python was all lists and
dicts, with dicts used as namespaces here and there.  Nowadays you have
to be careful.  Moreover, it is harder to explain:

>>> zip([1,2,3], [4,5,6])     # easy to understand and explain
[(1, 4), (2, 5), (3, 6)]

>>> enumerate([6,7,8,9])         # uh ?
<enumerate object at 0x401a102c>

I know you can always do list(_).  My point is that this is a
user-visible optimization.  enumerate() should return a normal list, and
it should be someone else's job to ensure that it is correctly optimized
away if possible (and I'm not even talking about Psyco, it could be done
in the current Python implementation with a reasonable amount of
effort).


Protesting-ly yours,

Armin


From guido at python.org  Sat Apr  3 16:22:33 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 16:22:40 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 12:02:05 EST."
	<uekr5dmgi.fsf@boost-consulting.com> 
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <ullldf6s1.fsf@boost-consulting.com>
	<200404031658.i33Gwps24056@guido.python.org> 
	<uekr5dmgi.fsf@boost-consulting.com> 
Message-ID: <200404032122.i33LMXg24564@guido.python.org>

> > NO, NO, NO!  Each dot is one uplevel.  Exactly like Mac OS 9 paths do
> > with ':'.  (Read macpath.py.)
> 
> That's a fine convention; I'm familiar with MacOS paths.  
> The fact that David Goodger and I were both confused might indicate
> that the proposed syntax might not be as intuitive as you'd like, but
> I don't think it'll take long to get used to.

I think the only problem was that it might not have been explained
clearly enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Sat Apr  3 16:24:57 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sat Apr  3 16:25:04 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <5.1.0.14.0.20040403112915.038069e0@mail.telecommunity.com>
References: <200404021950.i32JoZN21741@guido.python.org>
	<200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<5.1.0.14.0.20040403112915.038069e0@mail.telecommunity.com>
Message-ID: <16495.11177.951402.175926@montanaro.dyndns.org>


    Phillip> Technically, what you show is not the actual expansion of the
    Phillip> new syntax.  The new syntax applies decorators before binding
    Phillip> 'x' to the new function.  So, the old value of 'x' is available
    Phillip> to a decorator via sys._getframe().f_locals.  This technique is
    Phillip> also useful for implementing generic functions and/or
    Phillip> multimethods, signature-based overloading, etc.

How would this be interpreted?

    x = 42

   def x(self) [propget]:
       "Doc string for x"
       return self.__x

   def x(self, newx) [propset]:
       self.__x = newx

   def x(self) [propdel]:
       del self.__x

That is, there is already an (otherwise invalid) 'x' in the calling scope
when propget() is called.  Do the property doodads just need to be
bulletproofed or should an exception be raised?

Skip

From guido at python.org  Sat Apr  3 16:25:30 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 16:25:38 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 12:12:48 EST."
	<20040403171247.GC8803@panix.com> 
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>  
	<20040403171247.GC8803@panix.com> 
Message-ID: <200404032125.i33LPU024587@guido.python.org>

> > One more (possibly unclear) case to consider:
> >     import . as package
> > Is this a way to name your parent package?
> 
> Nope.  All uses of relative imports require ``from <something> import``

Right.

> (I think the PEP makes that clear).  You could, however, do
> 
>     from ... import package [as foo]
> 
> if Guido permits it; I want a Pronouncement on that before adding it
> to the PEP.

Yes, even if ... refers to the root package (but not if it tries to
reach beyond that).  But this doesn't handle his case: what he wanted
(if it were allowed) would rename the current package to 'package' no
matter what its real name.  Not that I care about that use case. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr  3 16:50:45 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 16:50:53 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: Your message of "Sat, 03 Apr 2004 15:24:57 CST."
	<16495.11177.951402.175926@montanaro.dyndns.org> 
References: <200404021950.i32JoZN21741@guido.python.org>
	<200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<5.1.0.14.0.20040403112915.038069e0@mail.telecommunity.com> 
	<16495.11177.951402.175926@montanaro.dyndns.org> 
Message-ID: <200404032150.i33Lojb24741@guido.python.org>

> How would this be interpreted?
> 
>     x = 42
> 
>    def x(self) [propget]:
>        "Doc string for x"
>        return self.__x
> 
>    def x(self, newx) [propset]:
>        self.__x = newx
> 
>    def x(self) [propdel]:
>        del self.__x
> 
> That is, there is already an (otherwise invalid) 'x' in the calling scope
> when propget() is called.  Do the property doodads just need to be
> bulletproofed or should an exception be raised?

It's broken.  I expect this to raise an exception at some point.
Beyond that, who cares?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dave at boost-consulting.com  Sat Apr  3 16:55:05 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Sat Apr  3 16:55:50 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404032122.i33LMXg24564@guido.python.org> (Guido van
	Rossum's message of "Sat, 03 Apr 2004 13:22:33 -0800")
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <ullldf6s1.fsf@boost-consulting.com>
	<200404031658.i33Gwps24056@guido.python.org>
	<uekr5dmgi.fsf@boost-consulting.com>
	<200404032122.i33LMXg24564@guido.python.org>
Message-ID: <u3c7kbubq.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> > NO, NO, NO!  Each dot is one uplevel.  Exactly like Mac OS 9 paths do
>> > with ':'.  (Read macpath.py.)
>> 
>> That's a fine convention; I'm familiar with MacOS paths.  
>> The fact that David Goodger and I were both confused might indicate
>> that the proposed syntax might not be as intuitive as you'd like, but
>> I don't think it'll take long to get used to.
>
> I think the only problem was that it might not have been explained
> clearly enough.

Sure, that would handle it.  Fails to meet a high standard set by
other elements of Python that can be grasped immediately, without any
explanation, but of course not everything can be that simple.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From pje at telecommunity.com  Sat Apr  3 17:42:54 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Apr  3 17:36:49 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <16495.11177.951402.175926@montanaro.dyndns.org>
References: <5.1.0.14.0.20040403112915.038069e0@mail.telecommunity.com>
	<200404021950.i32JoZN21741@guido.python.org>
	<200404020334.i323YAGZ024235@cosc353.cosc.canterbury.ac.nz>
	<1080880130.22892.158.camel@localhost.localdomain>
	<1080912282.5481.158.camel@anthem.wooz.org>
	<1080925513.22892.173.camel@localhost.localdomain>
	<1080932935.5481.282.camel@anthem.wooz.org>
	<5.1.0.14.0.20040403112915.038069e0@mail.telecommunity.com>
Message-ID: <5.1.0.14.0.20040403174138.0209f020@mail.telecommunity.com>

At 03:24 PM 4/3/04 -0600, Skip Montanaro wrote:

>That is, there is already an (otherwise invalid) 'x' in the calling scope
>when propget() is called.  Do the property doodads just need to be
>bulletproofed or should an exception be raised?

If I recall correctly, the implementation someone contributed earlier this 
week (sorry I don't remember his name) raised an exception if the previous 
binding of the function name was not an instance of 'property'.


From ishnigarrab at earthlink.net  Sat Apr  3 18:08:28 2004
From: ishnigarrab at earthlink.net (foo)
Date: Sat Apr  3 18:08:32 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
Message-ID: <12499735.1081033709200.JavaMail.root@kermit.psp.pas.earthlink.net>

Here are some examples I've come up with...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: functionals_dec.py
Type: text/x-python
Size: 4809 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040403/7283ebd1/functionals_dec.py
From jcarlson at uci.edu  Sat Apr  3 19:37:55 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat Apr  3 19:42:20 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <12499735.1081033709200.JavaMail.root@kermit.psp.pas.earthlink.net>
References: <12499735.1081033709200.JavaMail.root@kermit.psp.pas.earthlink.net>
Message-ID: <20040403163615.FA19.JCARLSON@uci.edu>

> ##########
> #
> # Decorations
> #
> class Memoized(dict):
>     """ A function wrapper to cache the results of functions that take a long
>         time to complete, like database querries or intensive mathematical
>         computations.
> 
>         To wipe the cache, just call m.clear()
>     """
>     def __init__(self, func, init={}):
>         """ Accepts the function to be memoized and an optional initial 
>             dictionary of known results.
>         """
>         dict.__init__(self, init)
>         self.func = func
> 
>     def __call__(self, *args, **kwds):
>         key = (args, tuple(kwds.items())) # hope everything's hashable...
>         return ( self.get(key)
>                  or self.setdefault(key, self.func(*args, **kwds)) )

I believe that with standard dictionaries, kwds is not guaraneed to have
any particular order.  Perhaps sorting kwds.items() makes sense?

 - Josiah


From skip at pobox.com  Sat Apr  3 20:01:16 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sat Apr  3 20:01:32 2004
Subject: [Python-Dev] Help with PEP 318
Message-ID: <16495.24156.587861.468059@montanaro.dyndns.org>


There's been way too much PEP 318 material on the mailing list during the
past week for me to absorb (*).  I'm tied up with a new job and can't really
devote much time to keeping the PEP updated.  If someone could step up and
summarize some of the new material (or better yet, update the file directly)
it would be much appreciated.

Thx,

Skip

(*) Much of the recent discussion with "PEP 318" in the subject has actually
been about a syntax for setting function attributes which requires its own
PEP.

From edloper at gradient.cis.upenn.edu  Sat Apr  3 21:17:16 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Sat Apr  3 21:15:52 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <E1B9w1p-0008Jq-Ad@mail.python.org>
References: <E1B9w1p-0008Jq-Ad@mail.python.org>
Message-ID: <406F702C.8070700@gradient.cis.upenn.edu>

> How would this be interpreted?
> 
>     x = 42
> 
>    def x(self) [propget]:
>        "Doc string for x"
>        return self.__x

It should raise an exception here.  See the implementation that I posted 
a few days ago: <http://www.cis.upenn.edu/~edloper/pydecorators.html>. 
The same should apply for the generic decorator, and any other decorator 
that "combines" the new value with the old.

Errors-should-never-pass-silently-yours,

Edward


From aahz at pythoncraft.com  Sat Apr  3 21:18:00 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr  3 21:18:14 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404032125.i33LPU024587@guido.python.org>
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>
	<20040403171247.GC8803@panix.com>
	<200404032125.i33LPU024587@guido.python.org>
Message-ID: <20040404021800.GA25210@panix.com>

On Sat, Apr 03, 2004, Guido van Rossum wrote:
>Aahz:
>>
>> (I think the PEP makes that clear).  You could, however, do
>> 
>>     from ... import package [as foo]
>> 
>> if Guido permits it; I want a Pronouncement on that before adding it
>> to the PEP.
> 
> Yes, even if ... refers to the root package (but not if it tries to
> reach beyond that).  

Great!  Then PEP 328 is almost set except for Bernhard's wish for more
clarification of the rationale for making the default import absolute.
I've gone ahead and updated the PEP, and I think we should move forward
on implementation even without the clarification, but I've stuck a note
in my calendar to bug people regularly until the rationale is finished.

I've added the following:

Timeline
========

In Python 2.4, you must enable the new absolute import behavior with ::

    from __future__ import absolute_import

You may use relative imports freely.  In Python 2.5, any ``import``
statement that results in an intra-package import will generate a
``PendingDeprecation`` warning (this also applies to ``from <> import``
that fails to use the relative import syntax).  In Python 2.6, ``import``
will always be an absolute import.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From guido at python.org  Sat Apr  3 21:58:35 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr  3 21:58:42 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Sat, 03 Apr 2004 21:18:00 EST."
	<20040404021800.GA25210@panix.com> 
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>
	<20040403171247.GC8803@panix.com>
	<200404032125.i33LPU024587@guido.python.org> 
	<20040404021800.GA25210@panix.com> 
Message-ID: <200404040258.i342wZs25281@guido.python.org>

> Great!  Then PEP 328 is almost set except for Bernhard's wish for more
> clarification of the rationale for making the default import absolute.
> I've gone ahead and updated the PEP, and I think we should move forward
> on implementation even without the clarification, but I've stuck a note
> in my calendar to bug people regularly until the rationale is finished.
> 
> I've added the following:
> 
> Timeline
> ========
> 
> In Python 2.4, you must enable the new absolute import behavior with ::
> 
>     from __future__ import absolute_import
> 
> You may use relative imports freely.  In Python 2.5, any ``import``
> statement that results in an intra-package import will generate a
> ``PendingDeprecation`` warning (this also applies to ``from <> import``
> that fails to use the relative import syntax).  In Python 2.6, ``import``
> will always be an absolute import.

Perfect!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ishnigarrab at earthlink.net  Sat Apr  3 19:00:38 2004
From: ishnigarrab at earthlink.net (Isaac Freeman)
Date: Sat Apr  3 23:58:55 2004
Subject: [Python-Dev] Re: PEP 318: Let's propose some useful built-in
	decorators
In-Reply-To: <E1B9w1r-0008My-2k@mail.python.org>
References: <E1B9w1r-0008My-2k@mail.python.org>
Message-ID: <406F5026.6080509@earthlink.net>

Here are a few ideas (including a meta-decoration! :)

Of course, I'm biased to first-before-colon style. ;)

--- decorators.py ---

import inspect
class decorator:
    """ An obscenely gratuitous example of decorator usage. Meta-decoration?
        "One mans use is another man's abuse"
    """

    class decoration:
        """ The object that represents the decoration itself. """
        def __init__(decor, func, **state):
            self.func = func
            self.state = {'func': func.__name__}

        def __call__(self, *args, **kwds):
            """ Called when the function that is decorated by this 
decoration is
                called. (whew)
            """
            self.state.update(inspect.getargvalues(sys._getframe(1))[3]) 
# better way?           
            return self.decor(self.func, self.state, *args, **kwds)

        def options(self, opts=None, **kwd_opts):
            self.state.update(opts or kwd_opts)
            return self

    def __init__(self, decor):
        self.decor = decor

    def __call__(self, func):
        """ Called when the decorator is called to decorate a desired 
function. """
        init = self.init
        init.update(kwds)
        return self.decoration(decor, func)

    def fromstate(cls, state=None, **kwd_state) [classmethod]:
        """ Make a decoration with a specified initial state """
        return lambda fn: cls(fn, state or kwd_state)

# or...

# Smaller, but can be confusing. (But I guess the whole
# meta-function-generator thing is to begin with!)
def decorator2(decor=None, **state):
    """ Decoration factory. """

    def decoration2(func=None, **opts):
        """ Decorating function wrapper. """
        state.update(opts)
        state['func'] = func.__name__
       
        def callit(*args, **kwds):
            """ Function replacement. """
            state.update(inspect.getargvalues(sys._getframe(1))[3]) # 
better way?           
            decor(func, state, *args, **kwds)

        if func:
            return callit
        else:
            return lambda fn: decoration2(fn, **opts)

    if decor:
        return decoration
    else:
        return lambda dec: decorator2(dec, state)


def memoized(func, state, *args, **kwds) [decorator]:
    """ Function wrapper to cache the results of previously
        evaluated arguments.
    """
    items = kwds.items
    items.sort()
    key = (args, tuple(items)) # hope everything is hashable!
    return ( state.get(key)
             or state.setdefault(key, func(*args, **kwds)) )

def memoized2(func, state, *args, **kwds) [decorator2]:
    key = (args, tuple(kwds.items()))
    return ( state.get(key)
             or state.setdefault(key, func(*args, **kwds)) )

import sys, time
def logged(func, state, *args, **kwds) [decorator.fromstate(
                                            output=sys.stdout.write,
                                            prefix='%(func)s: ')]:
    """ Usage for default formatting and output:
   
            def func(*args) [logged]:
                pass

        Usage for custom formatting and output:
       
            def func(a, b) [logged.options(output=output_func,
                                           prefix="%(func)s %(a)s %(b)i")]:
                pass

        String replacements include 'func' which is the function's name, 
as well
        as all arguments to the function such that "%(a)s" would become 
the value
        of the argument named 'a'.
    """
    p = '[%s] %s' % (time.time(), state['prefix'] % state)
    state['output'](p)

def logged2(func, state, *args, **kwds) [decorator2(
                                            output=sys.stdout.write,
                                            prefix='%(func)s: ')]:
    """ Usage for default formatting and output:
   
            def func(*args) [logged2]:
                pass

        Usage for custom formatting and output:
       
            def func(a, b) [logged2(output=output_func,
                                   prefix="%(func)s %(a)s %(b)i")]:
                pass

        String replacements include 'func' which is the function's name, 
as well
        as all arguments to the function such that "%(a)s" would become 
the value
        of the argument named 'a'.
    """
    p = '[%s] %s' % (time.time(), state['prefix'] % state)
    state['output'](p)

# demo function
def factorial(n) [memoized,
                  logged.options(output=sys.stdout.write,
                                 prefix='factorial for %(n)s: ')]:
    """ Computes the factorial of n """
    print "fact(%i)" % n
    if n == 1:
        return 1
    else:
        return n * fact(n - 1) # make tail_recursive decorator(?)

def factorial2(n) [memoized,
                   logged2(output=sys.stdout.write,
                           prefix='factorial for %(s): ')]:
    """ Computes the factorial of n """
    print "fact(%i)" % n
    if n == 1:
        return 1
    else:
        return n * fact(n - 1)



From marktrussell at btopenworld.com  Sun Apr  4 06:18:56 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Sun Apr  4 06:19:02 2004
Subject: [Python-Dev] Re: PEP 318 bake-off?
In-Reply-To: <20040403163615.FA19.JCARLSON@uci.edu>
References: <12499735.1081033709200.JavaMail.root@kermit.psp.pas.earthlink.net>
	<20040403163615.FA19.JCARLSON@uci.edu>
Message-ID: <1081073936.2509.6.camel@localhost>

On Sun, 2004-04-04 at 01:37, Josiah Carlson wrote:
> >     def __call__(self, *args, **kwds):
> >         key = (args, tuple(kwds.items())) # hope everything's hashable...
> >         return ( self.get(key)
> >                  or self.setdefault(key, self.func(*args, **kwds)) )
> 
> I believe that with standard dictionaries, kwds is not guaraneed to have
> any particular order.  Perhaps sorting kwds.items() makes sense?

Also the call to self.func() is needlessly repeated if the return value
is 0, None, or anything else that is treated as False.  I realise that
this is off the point slightly, but using "e1 or e2" like this is such
an attractive nuisance it warrants being stomped on wherever possible.

Mark Russell

From aahz at pythoncraft.com  Sun Apr  4 12:21:00 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr  4 12:21:05 2004
Subject: [Python-Dev] PEP 328: __path__
Message-ID: <20040404162100.GA6059@panix.com>

I just realized that the new relative import semantics are likely to
interact oddly with uses of ``__path__``.  I therefore suggest that the
following text be added to PEP 328, although I'm perfectly happy to
adopt some other course:

Between the addition of ``as`` in Python 2.0 and the new syntax &
semantics for ``import``, there is no longer a need for ``__path__``, and
it is now officially deprecated.  In Python 2.4 and 2.5, changing
``__path__`` raises ``PendingDeprecationWarning``.  In Python 2.6,
``__path__`` becomes a read-only attribute.  In the presence of
``absolute_import`` or with a relative import, ``__path__`` is ignored.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From pedronis at bluewin.ch  Sun Apr  4 12:35:28 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sun Apr  4 12:30:31 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: <20040404162100.GA6059@panix.com>
Message-ID: <5.2.1.1.0.20040404183116.031c83a8@pop.bluewin.ch>

At 12:21 04.04.2004 -0400, Aahz wrote:
>I just realized that the new relative import semantics are likely to
>interact oddly with uses of ``__path__``.  I therefore suggest that the
>following text be added to PEP 328, although I'm perfectly happy to
>adopt some other course:
>
>Between the addition of ``as`` in Python 2.0 and the new syntax &
>semantics for ``import``, there is no longer a need for ``__path__``, and
>it is now officially deprecated.  In Python 2.4 and 2.5, changing
>``__path__`` raises ``PendingDeprecationWarning``.  In Python 2.6,
>``__path__`` becomes a read-only attribute.  In the presence of
>``absolute_import`` or with a relative import, ``__path__`` is ignored.
>--

that's an issue because people use __path__ to spread a package
over many directories and not just in ther relative

Btw I thought (I admit I have not read the PEP) that relative imports are 
an operation over package names

import .c .# in a.b -> a.c

not about concrete directories.

regards. 


From pedronis at bluewin.ch  Sun Apr  4 12:38:14 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sun Apr  4 12:33:14 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: <5.2.1.1.0.20040404183116.031c83a8@pop.bluewin.ch>
References: <20040404162100.GA6059@panix.com>
Message-ID: <5.2.1.1.0.20040404183738.02854208@pop.bluewin.ch>

sorry too fast.

At 18:35 04.04.2004 +0200, Samuele Pedroni wrote:
that's an issue because people use __path__ to spread a package
>over many directories and not just in ther relative

... in the relative case.


>Btw I thought (I admit I have not read the PEP) that relative imports are 
>an operation over package names
>
>import .c .# in a.b -> a.c
>
>not about concrete directories.
>
>regards.


From guido at python.org  Sun Apr  4 12:43:26 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Apr  4 12:43:33 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: Your message of "Sun, 04 Apr 2004 12:21:00 EDT."
	<20040404162100.GA6059@panix.com> 
References: <20040404162100.GA6059@panix.com> 
Message-ID: <200404041643.i34GhQY02513@guido.python.org>

> I just realized that the new relative import semantics are likely to
> interact oddly with uses of ``__path__``.  I therefore suggest that the
> following text be added to PEP 328, although I'm perfectly happy to
> adopt some other course:
> 
> Between the addition of ``as`` in Python 2.0 and the new syntax &
> semantics for ``import``, there is no longer a need for ``__path__``, and
> it is now officially deprecated.  In Python 2.4 and 2.5, changing
> ``__path__`` raises ``PendingDeprecationWarning``.  In Python 2.6,
> ``__path__`` becomes a read-only attribute.  In the presence of
> ``absolute_import`` or with a relative import, ``__path__`` is ignored.

Uh?  Maybe you should have some coffee and then try again.

__path__ has nothing to do with relative import.  When I say
"import foo.bar" then bar is searched for in foo.__path__.  __path__
defines which directories comprise the package contents, nothing more
or less.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sun Apr  4 15:57:56 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr  4 15:58:04 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: <200404041643.i34GhQY02513@guido.python.org>
References: <20040404162100.GA6059@panix.com>
	<200404041643.i34GhQY02513@guido.python.org>
Message-ID: <20040404195756.GA20803@panix.com>

On Sun, Apr 04, 2004, Guido van Rossum wrote:
>Aahz:
>>
>> I just realized that the new relative import semantics are likely to
>> interact oddly with uses of ``__path__``.  I therefore suggest that the
>> following text be added to PEP 328, although I'm perfectly happy to
>> adopt some other course:
>> 
>> Between the addition of ``as`` in Python 2.0 and the new syntax &
>> semantics for ``import``, there is no longer a need for ``__path__``, and
>> it is now officially deprecated.  In Python 2.4 and 2.5, changing
>> ``__path__`` raises ``PendingDeprecationWarning``.  In Python 2.6,
>> ``__path__`` becomes a read-only attribute.  In the presence of
>> ``absolute_import`` or with a relative import, ``__path__`` is ignored.
> 
> Uh?  Maybe you should have some coffee and then try again.

Don't drink coffee, remember?  ;-)

> __path__ has nothing to do with relative import.  When I say
> "import foo.bar" then bar is searched for in foo.__path__.  __path__
> defines which directories comprise the package contents, nothing more
> or less.

Yeah, I know; I think that ``as`` combined with relative imports can
obviate most uses for ``__path__``, but I'm now dealing with a family
emergency, so I'm going to just drop it.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From guido at python.org  Sun Apr  4 18:27:19 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Apr  4 18:27:29 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: Your message of "Sun, 04 Apr 2004 15:57:56 EDT."
	<20040404195756.GA20803@panix.com> 
References: <20040404162100.GA6059@panix.com>
	<200404041643.i34GhQY02513@guido.python.org> 
	<20040404195756.GA20803@panix.com> 
Message-ID: <200404042227.i34MRJG02985@guido.python.org>

> > Uh?  Maybe you should have some coffee and then try again.
> 
> Don't drink coffee, remember?  ;-)

Well, whatever you do take to improve your brain function.

> > __path__ has nothing to do with relative import.  When I say
> > "import foo.bar" then bar is searched for in foo.__path__.  __path__
> > defines which directories comprise the package contents, nothing more
> > or less.
> 
> Yeah, I know; I think that ``as`` combined with relative imports can
> obviate most uses for ``__path__``, but I'm now dealing with a family
> emergency, so I'm going to just drop it.

'as' has nothing to do with __path__ either.

__path__ is for distributing a package over multiple directories, and
has nothing to do with how you import it.

So, yes, please drop it.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jim.jewett at eds.com  Sun Apr  4 19:24:23 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Sun Apr  4 19:25:04 2004
Subject: [Python-Dev] FuncAttrs or funcattrs or func_attrs
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D436@USAHM010.amer.corp.eds.com>


> I propose this one to set function attributes:

>   class func_attrs(objects):


Aren't classes supposed to use UpperCase?
If this is a more subtle distinction, it should probably be 
added to PEP 8.

FWIW, I think this should go in a decorate module (particularly
since this use really is just decoration) rather than a builtin.

-jJ

From jim.jewett at eds.com  Sun Apr  4 19:30:01 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Sun Apr  4 19:31:05 2004
Subject: [Python-Dev] from ...sys import path
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D437@USAHM010.amer.corp.eds.com>

What is wrong with 

from ...sys import path?

There are several CVS notes about reordering startup 
so that users can point to a custom sys or os.

...sys

Sure looks like a request for a custom sys.  Whether
to raise ImportError or use the default sys when the
custom version isn't there ... I'm not sure.  

The PEP should decide though, since this will probably
happen with common addons.  (Ensure that I have *some*
PIL.)

-jJ

From bob at redivi.com  Sun Apr  4 19:42:44 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Apr  4 19:38:38 2004
Subject: [Python-Dev] FuncAttrs or funcattrs or func_attrs
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D436@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D436@USAHM010.amer.corp.eds.com>
Message-ID: <C538D4EF-8691-11D8-B16A-000A95686CD8@redivi.com>

On Apr 4, 2004, at 7:24 PM, Jewett, Jim J wrote:

>
>> I propose this one to set function attributes:
>
>>   class func_attrs(objects):
>
>
> Aren't classes supposed to use UpperCase?
> If this is a more subtle distinction, it should probably be
> added to PEP 8.
>
> FWIW, I think this should go in a decorate module (particularly
> since this use really is just decoration) rather than a builtin.

The fact that it's a class is really just an implementation deal.  It 
acts more like a "type", such as int, list, itertools.chain, property, 
etc.

-bob


From jim.jewett at eds.com  Sun Apr  4 19:45:16 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Sun Apr  4 19:46:03 2004
Subject: [Python-Dev] PEP 318: Properties
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D438@USAHM010.amer.corp.eds.com>

Skip:

>> How would this be interpreted?
 
>>     x = 42
 
>>    def x(self) [propget]:
>>        "Doc string for x"
>>        return self.__x

>> That is, there is already an (otherwise invalid) 'x' in the calling scope
>> when propget() is called.  Do the property doodads just need to be
>> bulletproofed or should an exception be raised?

Guido:

> It's broken.  I expect this to raise an exception at some point.
> Beyond that, who cares?

If so, then this decorator is a bad idea.

I would expect 42 to be the initial value of the property x.

I could learn that properties are different, and this won't work, but...
this would be a very unexpected result.  At the very least, an explicit
exception should be raised at definition time.

-jJ



From aahz at pythoncraft.com  Sun Apr  4 19:52:12 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr  4 19:52:22 2004
Subject: [Python-Dev] from ...sys import path
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D437@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D437@USAHM010.amer.corp.eds.com>
Message-ID: <20040404235212.GA29398@panix.com>

On Sun, Apr 04, 2004, Jewett, Jim J wrote:
>
> What is wrong with 
> 
> from ...sys import path?

Your question is lacking sufficient context to answer, assuming that
this refers to PEP 328.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"usenet imitates usenet"  --Darkhawk

From skip at pobox.com  Sun Apr  4 20:13:45 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sun Apr  4 20:13:58 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D438@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D438@USAHM010.amer.corp.eds.com>
Message-ID: <16496.42169.731352.91397@montanaro.dyndns.org>


    Skip> x = 42
 
    Skip> def x(self) [propget]:
    Skip>   "Doc string for x"
    Skip>   return self.__x

    Guido> It's broken.  I expect this to raise an exception at some point.
    Guido> Beyond that, who cares?

    Jim> If so, then this decorator is a bad idea.

    Jim> I would expect 42 to be the initial value of the property x.

Then it makes sense to put "x = 42" after the three property definitions
(before the end of the class definition).  Would that work?

Skip

From guido at python.org  Sun Apr  4 20:19:30 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Apr  4 20:19:40 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: Your message of "Sun, 04 Apr 2004 19:45:16 EDT."
	<B8CDFB11BB44D411B8E600508BDF076C1E96D438@USAHM010.amer.corp.eds.com> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D438@USAHM010.amer.corp.eds.com>
Message-ID: <200404050019.i350JU903188@guido.python.org>

> >> How would this be interpreted?
>  
> >>     x = 42
>  
> >>    def x(self) [propget]:
> >>        "Doc string for x"
> >>        return self.__x
> 
> >> That is, there is already an (otherwise invalid) 'x' in the calling scope
> >> when propget() is called.  Do the property doodads just need to be
> >> bulletproofed or should an exception be raised?
> 
> Guido:
> 
> > It's broken.  I expect this to raise an exception at some point.
> > Beyond that, who cares?
> 
> If so, then this decorator is a bad idea.
> 
> I would expect 42 to be the initial value of the property x.

Then I think you have much to learn about properties.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Apr  4 20:22:44 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Apr  4 20:22:52 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: Your message of "Sun, 04 Apr 2004 19:13:45 CDT."
	<16496.42169.731352.91397@montanaro.dyndns.org> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D438@USAHM010.amer.corp.eds.com>
	<16496.42169.731352.91397@montanaro.dyndns.org> 
Message-ID: <200404050022.i350MiG03211@guido.python.org>

> Then it makes sense to put "x = 42" after the three property definitions
> (before the end of the class definition).  Would that work?

No, for deep reasons that have to do with the implementation of
descriptors.  This has nothing to do with decorators and not a lot
with properties -- it's the same as

  class C:
    def x(self): return self.__x
    x = 42

today.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From edloper at gradient.cis.upenn.edu  Sun Apr  4 20:33:57 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Sun Apr  4 20:32:16 2004
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 9, Issue 16
In-Reply-To: <E1BAHql-0001BO-ET@mail.python.org>
References: <E1BAHql-0001BO-ET@mail.python.org>
Message-ID: <4070A975.3020505@gradient.cis.upenn.edu>

>>>How would this be interpreted?
>>>    x = 42
>>>    def x(self) [propget]:
>>>        "Doc string for x"
>>>        return self.__x
> 
>>>That is, there is already an (otherwise invalid) 'x' in the calling scope
>>>when propget() is called.  Do the property doodads just need to be
>>>bulletproofed or should an exception be raised?

[GvR said:]
>>It's broken.  I expect this to raise an exception at some point.
>>Beyond that, who cares?

[Jim said:]
> If so, then this decorator is a bad idea.
> 
> I would expect 42 to be the initial value of the property x.

That doesn't make sense.  If you want to set a default value, you'd say:

     __x = 42

A property's getter/setter/deleter can perform arbitrary actions, and in 
the general case there's no sensible way to define an "initial value" 
for a property.  E.g., consider:

     class A:
         def getX(self):
             return random.choice(['apple', 'orange', 'grape'])
         x = property(getX)

If this still doesn't make sense, go read the docs for properties:
<http://www.python.org/2.2.2/descrintro.html#property>

[Skip said:]
> Then it makes sense to put "x = 42" after the three property definitions
> (before the end of the class definition).  Would that work?

No, that would just overwrite x with the value 42 (throwing away the 
property object).

-Edward


From tim.one at comcast.net  Sun Apr  4 20:50:05 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sun Apr  4 20:50:11 2004
Subject: [Python-Dev] genexps slow?
In-Reply-To: <2my8phi9bu.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEEDKAAB.tim.one@comcast.net>

[Tim Peters]
>> Resuming a generator function is a lot cheaper than calling a
>> function, but there's still a non-trivial amount of code to get in
>> and out of eval_frame() each time, which the listcomp version gets
>> to skip.

[Michael Hudson.
> This is something that occurred to me a while ago: how many opcodes
> does a typical invocation of eval_frame actually execute?  A little
> script told me that the median length of functions in Lib/*.py was 38
> instructions (or 52 bytes) IIRC, but obviously a dynamic count is far
> more interesting.  If the number is fairly small (and honestly, I have
> no idea), the set up and tear down code becomes much more significant
> than one might think.

And whatever it is, I expect it will be significantly smaller for functions
synthesized to implement generator expressions.

> I didn't think much about the code to get *out* of eval_frame.

There's a twist there:  the functions synthesized for generator expressions
get to take the quicker-than-general "fast_yield" exit path.  Hmm!  It looks
like tests against WHY_YIELD before the fast_yield label can no longer
succeed, but are still there to slow down non-generator cases.


From jim.jewett at eds.com  Sun Apr  4 21:07:32 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Sun Apr  4 21:08:06 2004
Subject: [Python-Dev] PEP 318: Properties
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D43A@USAHM010.amer.corp.eds.com>

    Skip> x = 42
 
    Skip> def x(self) [propget]:
    Skip>   "Doc string for x"
    Skip>   return self.__x

    Guido> It's broken.  I expect this to raise an exception at some point.
    Guido> Beyond that, who cares?

    Jim> If so, then this decorator is a bad idea.

    Jim> I would expect 42 to be the initial value of the property x.

    Skip> Then it makes sense to put "x = 42" after the three property 
    Skip> definitions (before the end of the class definition).  
    Skip> Would that work?

Not really.  The documentation says 

	"Properties are a neat way to implement attributes whose usage 
	resembles attribute access, but whose implementation uses method 
	calls."

This suggests that creating a property creates a manager.  In this case,
the manager is created for an attribute that already exists.  If it uses 
the previous value as the initial current value, that is what I expect.  
If it raises an exception ("I can't manage that!  Someone beat me to it!"), 
I'll be annoyed, but live with the limitation.  If it silently throws the 
value away, or raises an exception at run time -- that is bad. 

Yes, this is a variation on the "Properties do not work for classic 
classes, but you don't get a clear error when you try this." wart.  
But the code this decorator replaces::

    x = property(getx, setx)

does explicitly replace x, so I think *this* confusion may be specific
to the decorator.

-jJ

From bob at redivi.com  Sun Apr  4 21:22:09 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Apr  4 21:18:00 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D43A@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D43A@USAHM010.amer.corp.eds.com>
Message-ID: <A8A816BC-869F-11D8-B16A-000A95686CD8@redivi.com>


On Apr 4, 2004, at 9:07 PM, Jewett, Jim J wrote:

>     Skip> x = 42
>
>     Skip> def x(self) [propget]:
>     Skip>   "Doc string for x"
>     Skip>   return self.__x
>
>     Guido> It's broken.  I expect this to raise an exception at some 
> point.
>     Guido> Beyond that, who cares?
>
>     Jim> If so, then this decorator is a bad idea.
>
>     Jim> I would expect 42 to be the initial value of the property x.
>
>     Skip> Then it makes sense to put "x = 42" after the three property
>     Skip> definitions (before the end of the class definition).
>     Skip> Would that work?
>
> Not really.  The documentation says
>
> 	"Properties are a neat way to implement attributes whose usage
> 	resembles attribute access, but whose implementation uses method
> 	calls."
>
> This suggests that creating a property creates a manager.  In this 
> case,
> the manager is created for an attribute that already exists.  If it 
> uses
> the previous value as the initial current value, that is what I expect.
> If it raises an exception ("I can't manage that!  Someone beat me to 
> it!"),
> I'll be annoyed, but live with the limitation.  If it silently throws 
> the
> value away, or raises an exception at run time -- that is bad.
>
> Yes, this is a variation on the "Properties do not work for classic
> classes, but you don't get a clear error when you try this." wart.
> But the code this decorator replaces::
>
>     x = property(getx, setx)
>
> does explicitly replace x, so I think *this* confusion may be specific
> to the decorator.

... have you ever used descriptors before?

Property descriptors are only functionally applicable to *instances* of 
the class where the descriptor lives.  FooClass.x = 42 will replace the 
descriptor.  It's possible to make a descriptor do something special 
when you fetch it from its class, but in this case FooClass doesn't 
even really exist when you are replacing the descriptor with something 
else!

-bob


From greg at cosc.canterbury.ac.nz  Sun Apr  4 21:23:53 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Apr  4 21:24:18 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
Message-ID: <200404050123.i351Nr9R029066@cosc353.cosc.canterbury.ac.nz>

Ka-Ping Yee <python-dev@zesty.ca>:

> Putting the [decorator] on a separate line before the function
> changes the stakes entirely.  It sets aside real functional issues
> in favour of aesthetics.

Not only that, but I think the aesthetics of this version are *worse*
than anything that's been considered before.  It strikes me as deeply
wrong and un-Pythonic -- so much so that I can't understand why Guido
is even considering it.

Was he abducted by aliens the other night and given a personality
transplant or something? It's the only plausible explanation I can
think of.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From pje at telecommunity.com  Sun Apr  4 23:34:44 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Apr  4 23:34:49 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D43A@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.0.14.0.20040404233010.03b05740@mail.telecommunity.com>

At 09:07 PM 4/4/04 -0400, Jewett, Jim J wrote:
>Not really.  The documentation says
>
>         "Properties are a neat way to implement attributes whose usage
>         resembles attribute access, but whose implementation uses method
>         calls."
>
>This suggests that creating a property creates a manager.  In this case,
>the manager is created for an attribute that already exists.  If it uses
>the previous value as the initial current value, that is what I expect.
>If it raises an exception ("I can't manage that!  Someone beat me to it!"),
>I'll be annoyed, but live with the limitation.  If it silently throws the
>value away, or raises an exception at run time -- that is bad.
>
>Yes, this is a variation on the "Properties do not work for classic
>classes, but you don't get a clear error when you try this." wart.
>But the code this decorator replaces::
>
>     x = property(getx, setx)
>
>does explicitly replace x, so I think *this* confusion may be specific
>to the decorator.

Are you saying this is what you think, or what you think that newbies will 
think?  If this is what you think, you probably need to read the descriptor 
tutorial (and perhaps PEP 252) again.  Descriptors don't work anything like 
what you're talking about.

As for the implementation of propget and propset, if you look at Ed Loper's 
draft implementation, you'll see that an error is raised at function 
definition time if there's an existing non-property binding for the named 
function.


From pje at telecommunity.com  Sun Apr  4 23:39:49 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Apr  4 23:40:02 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404050123.i351Nr9R029066@cosc353.cosc.canterbury.ac.nz>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
Message-ID: <5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>

At 01:23 PM 4/5/04 +1200, Greg Ewing wrote:
>Ka-Ping Yee <python-dev@zesty.ca>:
>
> > Putting the [decorator] on a separate line before the function
> > changes the stakes entirely.  It sets aside real functional issues
> > in favour of aesthetics.
>
>Not only that, but I think the aesthetics of this version are *worse*
>than anything that's been considered before.  It strikes me as deeply
>wrong and un-Pythonic -- so much so that I can't understand why Guido
>is even considering it.

There are different kinds of aesthetics.  Guido's proposal has grown on me 
from a *visual* aesthetics point of view.  After I worked with it a little 
bit, I realized it really is much prettier than decorators-before-colon.

However, from more intellectual aesthetics (consistency, predictability, 
etc.) I still dislike it, and don't really see how to sanely reconcile it 
with the Python syntax of today.  I wish that I did, because it really does 
*look* better for simple decorators.

On the other hand, *no* syntax proposed so far has been really that nice to 
look at when used for multiple decorators.


From guido at python.org  Mon Apr  5 00:33:38 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 00:33:53 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Sun, 04 Apr 2004 23:39:49 EDT."
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com> 
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>  
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com> 
Message-ID: <200404050433.i354XcF03700@guido.python.org>

> There are different kinds of aesthetics.  Guido's proposal has grown
> on me from a *visual* aesthetics point of view.  After I worked with
> it a little bit, I realized it really is much prettier than
> decorators-before-colon.

Right.  It really is.

> However, from more intellectual aesthetics (consistency,
> predictability, etc.) I still dislike it, and don't really see how
> to sanely reconcile it with the Python syntax of today.  I wish that
> I did, because it really does *look* better for simple decorators.

I, too, wish that there were a way to make it work with current
expectations.  The * prefix looks so arbitrary: why not /, why not @,
etc...

> On the other hand, *no* syntax proposed so far has been really that
> nice to look at when used for multiple decorators.

Yet, decorator-before-def does better there, too, because it gives you
more horizontal space to work with.  This is useful for decorators
that take argument lists, like the ObjC decorator and I believe some
of PEAK's decorators.  (I don't want to use funcattrs() as an argument
here, because I believe that it is simply an inevitable by-product of
introducing any kind of decorator syntax -- while far from perfect, in
terms of readability setting function attributes in a decorator is so
much better than setting them after the function has been defined,
that I believe we have no choice but to provide it.  (Similar for
synchronized, except I feel less pressure to add it; I'd much rather
introduce some kind of general block wrapper feature like we have
discussed many times here.))

I also note that accepting decorator-before-colon now would make it
harder to come up with a decent syntax for declaring the return type,
which I still want to do in some future version of Python with
optional (!) static typing.  But not impossible -- there's enough
punctuation available besides '[' and ':'.

I also note that the proposed variants that prefix the 'def clause'
with another (optional) indented clause, started e.g. by 'with:' or
'decorate:', look much worse.  They violate another strong expectation
in Python: that a suite is a sequence of statements.  (And it really
can't be a sequence of statements -- e.g. assignments just really
don't make sense as decorators, nor do many other forms of statement.)

One worry about every syntax that supports multiple decorators: as
soon as there's a transformation among the decorators, they stop being
commutative.  And that could cause a load of problems with beginners
trying their hand at using complicated decorators (e.g. trying to
define a class property by combining classmethod and property).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Mon Apr  5 01:16:26 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 01:16:53 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: <20040404162100.GA6059@panix.com>
Message-ID: <200404050516.i355GQes002218@cosc353.cosc.canterbury.ac.nz>

Aahz <aahz@pythoncraft.com>:

> Between the addition of ``as`` in Python 2.0 and the new syntax &
> semantics for ``import``, there is no longer a need for ``__path__``, and
> it is now officially deprecated.

NO! WAIT!

I have a project in which I'm modifying a module's __path__,
and it's *not* to work around any deficiencies in the relative
import mechanism. Rather it's to make a set of directories
behave like a single package for import purposes, with the
members of the set determined at run time.

Will there be any easy way to do this under the new scheme?
If not, can you please try to find some way to make __path__
work sensibly with the new rules.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon Apr  5 01:44:34 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 01:44:45 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <16496.42169.731352.91397@montanaro.dyndns.org>
Message-ID: <200404050544.i355iYF6002268@cosc353.cosc.canterbury.ac.nz>

Skip Montanaro <skip@pobox.com>:

>     Jim> I would expect 42 to be the initial value of the property x.
> 
> Then it makes sense to put "x = 42" after the three property definitions
> (before the end of the class definition).  Would that work?

No, it would wipe out the property descriptor that you'd just
carefully set up.

I don't think this makes sense, anyway. Being a property implies that
its value is computed somehow, so how to establish an initial value,
if that concept even applies, will depend entirely on the details of
how it's being computed.  Sounds like a job for a piece of code in an
__init__ method to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon Apr  5 02:01:04 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 02:01:15 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
Message-ID: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> There are different kinds of aesthetics.  Guido's proposal has grown on me 
> from a *visual* aesthetics point of view.  After I worked with it a little 
> bit, I realized it really is much prettier than decorators-before-colon.

Your tastes must be different from mine, then, because
it doesn't strike me as any prettier visually, either.

I don't think I can fully separate these different kinds of
aesthetics in my mind, anyway. To me, a piece of syntax
isn't just something to look at -- it has a grammar, and
it has a meaning, and if the grammar and the meaning and
the way it looks on the page don't all agree with each
other, it strikes a discordant note.

When I read the syntax

  def foo(args) [classmethod]:
    ...

it says to me "Define a function foo, with these args, and
which happens to be a classmethod. Here's the body..."

On the other hand, when I see

  [classmethod]
  def foo(args):
    ...

the little voice in my head doesn't really say anything
coherent at all.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From python-dev at zesty.ca  Mon Apr  5 02:14:42 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Mon Apr  5 02:14:21 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
Message-ID: <Pine.LNX.4.58.0404050114120.18028@server1.LFW.org>

On Mon, 5 Apr 2004, Greg Ewing wrote:
> the little voice in my head doesn't really say anything
> coherent at all.

Funny.  I experience that most days.


-- ?!ng

From fumanchu at amor.org  Mon Apr  5 02:37:39 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon Apr  5 02:39:46 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561FB0@opus.amorhq.net>

Guido van Rossum wrote:
> ...while far from perfect, in
> terms of readability setting function attributes in a decorator is so
> much better than setting them after the function has been defined,
> that I believe we have no choice but to provide it.

The more I read statements like this, the more I think of replacing:

def foo(arg1, arg2):
    pass
foo = classmethod(foo)
foo.attrA = 3

...with code like:

foo = classmethod()
foo.attrA = 3
def foo(arg1, arg2):
    pass

...the meaning of "def" changes from "make a function" to "bind this
suite to foo". OBVIOUSLY, this would be a huge change as I've written
it. But perhaps, having seen this ideal, a middle ground might become
more visible.

> I also note that accepting decorator-before-colon now would make it
> harder to come up with a decent syntax for declaring the return type,
> which I still want to do in some future version of Python with
> optional (!) static typing.  But not impossible -- there's enough
> punctuation available besides '[' and ':'.

Why would such additional features not be standard attributes of
function objects?

foo = function()
foo.returns(int)
def foo(arg1):
    return int(arg1)

All in all, I'd prefer that function decorators and attributes use
simple, known techniques like assignment and callables (as shown above),
than become monolithic, crystallized all-in-one statements like:

def foo(arg1 as str, arg2 as int) returns str [classmethod] {attrA: 3}:

I understand the desire to keep all those bits near each other on the
page, but stronger is my desire to make them independent statements. If
we could bind a "blank" function object and supply its actual codeblock
in a later statement, the rest falls magically into place, IMO.


Just some thoughts.

Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From shane.holloway at ieee.org  Mon Apr  5 03:07:10 2004
From: shane.holloway at ieee.org (Shane Holloway (IEEE))
Date: Mon Apr  5 03:07:38 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404031657.i33Gvxl24044@guido.python.org>
References: <200403302218.i2UMIEV10058@guido.python.org>	<406EB373.2030401@python.org>
	<20040403145050.GA8626@panix.com>
	<200404031657.i33Gvxl24044@guido.python.org>
Message-ID: <4071059E.2060807@ieee.org>

 >>Good point.  Here's what I *think* the semantics are; as soon as I get
 >>agreement, I'll update the PEP.  Given a package layout::
 >>
 >>    package
 >>        subpackage1
 >>            moduleX
 >>            moduleY
 >>        subpackage2
 >>            moduleZ
 >>        moduleA
 >>
 >>Each leading "." refers to one level of parent.  Assuming that the
 >>current file is ``moduleX.py``, following are correct usages of the new
 >>syntax::
 >>
 >>    from .moduleY import spam
 >>    from ..subpackage1 import moduleY
 >>    from ..subpackage2.moduleZ import eggs
 >>    from ..moduleA import foo
 >>    from ...package import bar

What about names inside package __init__?

     from .. import bar

Is this also valid?

From dave at boost-consulting.com  Mon Apr  5 03:49:32 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Mon Apr  5 03:50:28 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
Message-ID: <uzn9q50fn.fsf@boost-consulting.com>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

>> There are different kinds of aesthetics.  Guido's proposal has grown on me 
>> from a *visual* aesthetics point of view.  After I worked with it a little 
>> bit, I realized it really is much prettier than decorators-before-colon.
>
> Your tastes must be different from mine, then, because
> it doesn't strike me as any prettier visually, either.
>
> I don't think I can fully separate these different kinds of
> aesthetics in my mind, anyway. To me, a piece of syntax
> isn't just something to look at -- it has a grammar, and
> it has a meaning, and if the grammar and the meaning and
> the way it looks on the page don't all agree with each
> other, it strikes a discordant note.
>
> When I read the syntax
>
>   def foo(args) [classmethod]:
>     ...
>
> it says to me "Define a function foo, with these args, and
> which happens to be a classmethod. Here's the body..."
>
> On the other hand, when I see
>
>   [classmethod]
>   def foo(args):
>     ...
>
> the little voice in my head doesn't really say anything
> coherent at all.

Lest we get the idea that there's any absolute measure of aesthetics:

  +1

restoring-nature's-equilibrium-ly,
Dave

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From paul at prescod.net  Mon Apr  5 04:17:17 2004
From: paul at prescod.net (Paul Prescod)
Date: Mon Apr  5 04:22:57 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
Message-ID: <4071160D.5060204@prescod.net>

Greg Ewing wrote:

> "Phillip J. Eby" <pje@telecommunity.com>:
> 
> 
>>There are different kinds of aesthetics.  Guido's proposal has grown on me 
>>from a *visual* aesthetics point of view.  After I worked with it a little 
>>bit, I realized it really is much prettier than decorators-before-colon.
> 
> Your tastes must be different from mine, then, because
> it doesn't strike me as any prettier visually, either.

What people find readable depends mostly on what they are used to. The 
reason Python is "so readable" on first read is because it rips off so 
much syntax directly from C, the language that is the ancestor for all 
of the other languages people use today.

C doesn't have function decorators so people don't have expectations for 
them. But they will when they start coming over from C# and Java 1.5 in 
two or three years. One virtue of Guido's proposal is that it is 
basically what C# does. Java uses a pretty different syntax but it is 
also a prefix syntax. If Python uses a postfix syntax it will probably 
be alone in making that choice.

I'm not saying that Python has to do what the other languages do because 
they do it, but all else equal, being familiar is better than being 
idiosyncratic (i.e. different for no good reason). So I would rank "like 
C# and Java" higher than "fits my personal aesthetics in early 2004" 
because aesthetics are likely to drift towards C# and Java over time.

  Paul Prescod



From bh at intevation.de  Mon Apr  5 05:05:08 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Mon Apr  5 05:05:18 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040404021800.GA25210@panix.com> (aahz@pythoncraft.com's
	message of "Sat, 3 Apr 2004 21:18:00 -0500")
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>
	<20040403171247.GC8803@panix.com>
	<200404032125.i33LPU024587@guido.python.org>
	<20040404021800.GA25210@panix.com>
Message-ID: <s9zn05qx0aj.fsf@salmakis.intevation.de>

Aahz <aahz@pythoncraft.com> writes:

> Timeline
> ========
>
> In Python 2.4, you must enable the new absolute import behavior with ::
>
>     from __future__ import absolute_import
>
> You may use relative imports freely.  In Python 2.5, any ``import``
> statement that results in an intra-package import will generate a
> ``PendingDeprecation`` warning (this also applies to ``from <> import``
> that fails to use the relative import syntax).  In Python 2.6, ``import``
> will always be an absolute import.

Why PendingDeprecation?  If the import semantics change in incompatible
ways in 2.6, 2.5 should generate a "normal" DeprecationWarning, I think.

  Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From marktrussell at btopenworld.com  Mon Apr  5 05:16:51 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Mon Apr  5 05:17:02 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404050433.i354XcF03700@guido.python.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
Message-ID: <1081156611.2377.6.camel@localhost>

On Mon, 2004-04-05 at 05:33, Guido van Rossum wrote:
> I, too, wish that there were a way to make it work with current
> expectations.  The * prefix looks so arbitrary: why not /, why not @,
> etc...

What about:

	[as classmethod]
	def foo(bar, baz):
	    pass

To me this is more obvious (and less like p*rl) that *[classmethod], and
it is also currently a syntax error so won't break existing code.

Mark Russell

From dave at boost-consulting.com  Mon Apr  5 06:38:41 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Mon Apr  5 06:39:41 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
Message-ID: <uoeq6iua6.fsf@boost-consulting.com>

Mark Russell <marktrussell@btopenworld.com> writes:

> On Mon, 2004-04-05 at 05:33, Guido van Rossum wrote:
>> I, too, wish that there were a way to make it work with current
>> expectations.  The * prefix looks so arbitrary: why not /, why not @,
>> etc...
>
> What about:
>
> 	[as classmethod]
> 	def foo(bar, baz):
> 	    pass
>
> To me this is more obvious (and less like p*rl) that *[classmethod], and
> it is also currently a syntax error so won't break existing code.

Not bad.  Also tolerable:

  as [classmethod]
  def foo(bar, baz):
     pass

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From marktrussell at btopenworld.com  Mon Apr  5 06:54:31 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Mon Apr  5 06:54:37 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <uoeq6iua6.fsf@boost-consulting.com>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<uoeq6iua6.fsf@boost-consulting.com>
Message-ID: <1081162471.3004.4.camel@localhost>

On Mon, 2004-04-05 at 11:38, David Abrahams wrote:
> Not bad.  Also tolerable:
> 
>   as [classmethod]
>   def foo(bar, baz):
>      pass

The trouble with that is that its less clearly an error at the moment
(e.g.:

    as = [ 'pathological' ]
    classmethod = 0

    as [classmethod]
    def foo(bar, baz):
        pass

is actually legal right now).  The nice thing about "[as xxx]" is that
there's no way you can confuse it with a subscript expression.

Mark

From mwh at python.net  Mon Apr  5 06:58:15 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Apr  5 06:58:19 2004
Subject: [Python-Dev] genexps slow?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEEDKAAB.tim.one@comcast.net> (Tim
	Peters's message of "Sun, 4 Apr 2004 20:50:05 -0400")
References: <LNBBLJKPBEHFEDALKOLCEEEDKAAB.tim.one@comcast.net>
Message-ID: <2mptamg08o.fsf@starship.python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> [Tim Peters]
>>> Resuming a generator function is a lot cheaper than calling a
>>> function, but there's still a non-trivial amount of code to get in
>>> and out of eval_frame() each time, which the listcomp version gets
>>> to skip.
>
> [Michael Hudson.
>> This is something that occurred to me a while ago: how many opcodes
>> does a typical invocation of eval_frame actually execute?  A little
>> script told me that the median length of functions in Lib/*.py was 38
>> instructions (or 52 bytes) IIRC, but obviously a dynamic count is far
>> more interesting.  If the number is fairly small (and honestly, I have
>> no idea), the set up and tear down code becomes much more significant
>> than one might think.
>
> And whatever it is, I expect it will be significantly smaller for functions
> synthesized to implement generator expressions.

I expect invocations of most generators (not necessarily synthetic
genexp ones) probably execute *very* few instructions -- ~5, maybe?

>> I didn't think much about the code to get *out* of eval_frame.
>
> There's a twist there:  the functions synthesized for generator expressions
> get to take the quicker-than-general "fast_yield" exit path.  Hmm!  It looks
> like tests against WHY_YIELD before the fast_yield label can no longer
> succeed, but are still there to slow down non-generator cases.

Hmm.  I *think* I agree with you.  I also think the tests against
WHY_YIELD in the code for END_FINALLY are unnecessary, as we
(syntactically) don't allow yields when there are finally: statements
in play.  If I'm wrong about that, then you're wrong too :-) It's all
a bit subtle, to be sure.

Once again, I wonder if there was someway we could have two
eval_frames: one with tracing support, one without.  Unfortunately, I
don't think this can fly (how do you *start* tracing?).  Maybe we
could have a '-notrace' command line flag?  But I suspect that this is
a silly over-complication.

Cheers,
mwh

-- 
 "An infinite number of monkeys at an infinite number of keyboards
  could produce something like Usenet."
 "They could do a better job of it."
              -- the corollaries to Gene Spafford's Axiom #2 of Usenet

From pyth at devel.trillke.net  Mon Apr  5 06:58:09 2004
From: pyth at devel.trillke.net (Holger Krekel)
Date: Mon Apr  5 06:59:05 2004
Subject: Multiple Decorators? (was: Re: [Python-Dev] Re: PEP 318: Decorators
	last before colon)
In-Reply-To: <5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
Message-ID: <20040405105809.GE6361@solar.trillke>

Phillip J. Eby wrote:
> On the other hand, *no* syntax proposed so far has been really that nice to 
> look at when used for multiple decorators.

I agree completly here.  Add to it that understanding the semantics 
of applying multiple decorators might not always be easy in case of
a problem (which is a very common sitatuation for a programmer). 

Maybe this hints at thinking about syntaxes that don't neccesarily
work for multiple decorators but work well for one decorator?  

After all the main point of introducing descriptor syntax seems to be
that they should visually come closer to the 'def' statement line and
not that multiple decorators are a common problem. 

cheers,

    holger

From gmccaughan at synaptics-uk.com  Mon Apr  5 07:07:45 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Mon Apr  5 07:07:49 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <20040403195957.GA30613@vicky.ecs.soton.ac.uk>
References: <20040403195957.GA30613@vicky.ecs.soton.ac.uk>
Message-ID: <200404051207.45195.gmccaughan@synaptics-uk.com>

On Saturday 2004-04-03 20:59, Armin Rigo wrote:
> Hi!
> 
> This is a rant against the optimization trend of the Python interpreter.
> 
> Sorting a list of 100000 integers in random order takes:
> 
> * 0.75 seconds in Python 2.1
> * 0.51 seconds in Python 2.2
> * 0.46 seconds in Python 2.3
> 
> Tim Peters did a great job optimizing list.sort().  If I try with a
> simple, non-stable pure Python quicksort implementation, in Python 2.3:
> 
> * 4.83 seconds
> * 0.21 seconds with Psyco
> 
> First step towards world domination of high-level languages :-)

Mhm. And what happens if you want to sort something other
than integers? Say, Python objects with their own comparison
operators? Or even just (int,int) tuples?

-- 
g


From Paul.Moore at atosorigin.com  Mon Apr  5 08:54:38 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Mon Apr  5 08:54:38 2004
Subject: [Python-Dev] Python is faster than C
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>

From: Gareth McCaughan
> Mhm. And what happens if you want to sort something other
> than integers? Say, Python objects with their own comparison
> operators? Or even just (int,int) tuples?

I got the impression that Armin's point was that Psyco can optimise
the integer case *if* the code is in Python, but not if it's in C.

I suspect that sort isn't a good example here, but heapq may be. By
recoding it in C, the chance of getting of type-specific specialised
speedups is lost, in favour of a (possibly) smaller speedup across
the board. To get a feel for whether the right tradeoff has been
made, you'd need to analyze real use cases in some detail (which
may not be worth it).

[Armin, how does Psyco + a Python sort compare against timsort on
more general objects?]

Hmm. I'm torn. On the one hand, it's really cool that Python+Psyco can
out-perform C (another argument against the knee-jerk "recode bits in
C" reaction). But my gut feel is that not enough people use Psyco to
make this a good answer yet. On the third hand, if we recode chunks of
the stdlib in C, do we kill too many chances for Psyco to work its
magic?

Time for Psyco to go into the stdlib, maybe? :-)

Paul.

From scav at blueyonder.co.uk  Mon Apr  5 09:21:52 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Mon Apr  5 09:14:46 2004
Subject: [Python-Dev] Re: PEP309 re-written
In-Reply-To: <E1B8ngt-0005Wt-2e@mail.python.org>
References: <E1B8ngt-0005Wt-2e@mail.python.org>
Message-ID: <40715D70.1010402@blueyonder.co.uk>

David Abrams wrote:

>> If anyone can think of any really elegant hacks that are naturally
>> expressed by partial function application I'd like to see them
>  
>
>
>There are millions, but AFAICT they're all more natural with lambda,
>so...
>
>        "I agree that lambda is usually good enough, just not
>        always."
>
>Examples, please?
>  
>
Well, you can use partial to special-case classes as shorthand object 
factories.  So:

C = partial(Canvas,my_window,width=100,height=100,bg='white')

...gives you a callable C that is a factory for Tkinter Canvases 
parented to my_window,
100 pixels on a side, and with a white background.  How this differs 
from lambda is that
you can override keyword parameters from the supplied defaults:

c1 = C()

c2 = C(width=200)

c3 = C(bg='red')

>        "And I want the possibility of useful introspection and
>        subclassing"
>
>Can you elaborate on that, please?
>
>  
>
You could maybe sub-class partial to override __call__ , in case you 
want to do anything fancy like pre-supplying arguments at arbitrary 
positions.

You can inspect or indeed change C.fn, C.kw or C.args in the example 
above. I'm not speculating on why you might want to, but there's nothing 
stopping you. :)

Peter Harris

From jim.jewett at eds.com  Mon Apr  5 09:20:24 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Apr  5 09:21:09 2004
Subject: [Python-Dev] PEP 318: Properties
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D43B@USAHM010.amer.corp.eds.com>

>>Yes, this is a variation on the "Properties do not work for classic
>>classes, but you don't get a clear error when you try this." wart.
>>But the code this decorator replaces::

>>     x = property(getx, setx)

>>does explicitly replace x, so I think *this* confusion may be specific
>>to the decorator.

> Are you saying this is what you think, or what you think that newbies will

> think? 

I'm saying that it reads that way.  This does means that newbies will think
that, but it also means *I* will think that whenever I haven't *recently* 
read the docs or been burned.  It also means that when I'm debugging, I 
have to look more places to see why my value disappeared, just in case
someone used properties.

The reason to use a high-level language is that you can concentrate on your 
problem instead of worrying about implementation details.  Python normally
supports this very well.  It keeps most things explicit and it doesn't 
usually make you write much in the way of infrastructure/glue.  (Remember
the complaints about logging?  No one has suggested that it doesn't work 
well, only that it is too hard to use casually.)

What python does under the covers doesn't matter, *unless* the details
are so broken that I need to remember them.

	x=asdlfjk

looks like x is now set to something.  I expect it it to stay that way until
the next x=??? statement.  The fact that properties change it in a different
way under the covers doesn't matter; what matters is that it changes when
(and only when) I expect it to change.  By getting rid of the x=???
statement,
you force the programmer to keep more information in his own head.

> As for the implementation of propget and propset, if you look at Ed
Loper's 
> draft implementation, you'll see that an error is raised at function 
> definition time if there's an existing non-property binding for the named 
> function.

Which is an acceptable solution.  I would prefer that it keep the current
value, but I understand his objection that a particular getter might make 
"current value" meaningless.

What is not acceptable is a silent failure, planning to have it raise an
exception "sometime".  That would move the burden of implementation details 
back from the code that implements properties onto everyone who uses the
object.

-jJ

From mikepy-0404 at mike.de  Mon Apr  5 02:50:55 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Mon Apr  5 09:25:26 2004
Subject: [Python-Dev] The Need for a Declarative Syntax Element (aka Getting
	PEP 318 back on track)
Message-ID: <20040405065055.GA1485@mike.de>

Hi,

I wanted to share my thoughts about the issue and wrote the following essay.

Please note: all of this is IMHO of course. Nothing is cast in stone.
I'm very willing to change it to whatever decisions the discussion leads to.

Sorry about the length of this posting. :)

Any and all comments are appreciated. Please send your comments to
the list only, to avoid duplicate copies to my mailbox. Thank you!

Bye,
     Mike


The Need for a Declarative Syntax Element in Python
---------------------------------------------------

(aka "Getting PEP 318 back on track")

Author: Mike Pall
Release: 2004-04-05


1. Introduction
1.1 Clean and Lean
1.2 Sugar is Mean
1.3 Pep up Your Life

2. Getting it Straight
2.1 Terminology (n); cf. lack of ~
2.2 I Hereby Declare ...
2.3 Definitions
2.4 Now, what?

3. Foreign Territories
3.1 Big Brother in Action: C# Attributes
3.1.1 Syntax
3.1.2 Semantics
3.1.3 Use Cases
3.2 Catch Up, Baby: Java Annotations
3.2.1 Syntax
3.2.2 Semantics
3.2.3 Use Cases With JSR-175 Annotations
3.2.4 Use Cases With javadoc Annotations

4. Pythonic (Ab)use Cases
4.1 Assorted Attributes
4.2 Roaring Registries
4.3 Witty Wrappers
4.4 Proper Properties
4.5 To Sync or Not to Sync
4.6 Lexical Liberty

5. Semantic Wonderland
5.1 What?
5.2 How?
5.2.1 Calling the DO
5.2.2 Using __declare__
5.2.3 Passing The Context
5.3 When?
5.4 What else?

6. Syntax, Syntax on the Wall
6.1 Wishful Thinking
6.2 Round Up The Candidates
6.3 Narrowing the Candidate Set
6.4 And The Winner is ...
6.5 ASCII Art
6.6 Finally


1. Introduction
---------------

1.1 Clean and Lean

Python is an imperative language and has very few declarative elements
embedded down in the language specification. And most of them are in the
form of explicit statements (like "def", "class" or "global"). Only a few
are implicit (that's why some people have mixed feelings about "yield").
There are only two declarative delimiters ("*" and "**" in parameter lists)
and there are no non-statement declarative keywords (owing to the fact
that Python has pure dynamic typing).

This is in fact good. That's why we all love Python so much: the syntax
is clean and lean!

Just compare what other languages have embedded deep down in their
lexical analyzer and their grammar: type definitions in C, the massive
lexical and grammatical overloading of "*" and "&" in C++, scope-sensitive
keywords like "static", "volatile" or "private" in C, C++ or Java.
Some of that is pure necessity in languages with static typing, to save
some typing (no pun). But most of it can be described with just one
word: yuck!

Python has (so far) successfully avoided the syntax inflation. Even
classic OO stuff like "classmethod" or "staticmethod" are just builtin
types. You have to invoke (instantiate) them yourself in the body of
a class definition and modify the binding to the method name by assigning
to the class dictionary. Alas that works only *after* the method definition
is done, i.e. it has to be after the method body.


1.2 Sugar is Mean

Unfortunately simplicity comes at a price: a severe shortage of syntactic
sugar.

This is not necessarily a bad thing as the instance reference dilemma shows:
Some languages use implicit scoping (C++, Java) and provide overrides
("this"). Others use explicit delimiters ("@" in Ruby).

Python does neither and leaves the name of the identifier for the instance
reference up to the programmer (though "self" is a well established
convention). In this case most of us are happy about it, because "Explicit
is better than Implicit" holds here (you may object that "@" is explicit
enough, but well ... then it's implicit in the method definition).

However other issues leave something to be desired. The pressure to
get a nicer syntax for "classmethod" and "staticmethod" has been
traditionally rather low. Probably because neither is *that* common
(though this may depend on your programming style). And this is where
PEP 318 comes into play ...


1.3 Pep up Your Life

The proposal of PEP 318 has resulted in a flurry of activity at the
python-dev mailing list culminating during February and March 2004.
Even shouting and mutual accusations have been reported. Most everone
would be happy to get back to real work, as the subject has been
creating an enormous amount of traffic but little consent.

PEP 318 goes a bit further than simply proposing a nicer syntax for
"classmethod" and "staticmethod". It proposes a generic 'decorator'
syntax that allows you to write your own decorator functions. And you
get to see the decorator definition *before* the method body.

Most of the discussion has centered around what the most desirable syntax
would be. However everyone's definition of 'desirable' differs remarkably.
And the lack of convincing use cases hasn't helped keeping the discussion
on track either.

The early syntax proposals have some similarity with the C# language
feature "[Attribute] definition". The feature is called C# attributes but
cannot be compared directly to decorators nor Python attributes.

That of course sparked some discussion about using decorators for defining
attributes, but this is an entirely different matter.

A few regularly scheduled syntactic duels later, we are still where we
started: a solution in search of its problem. Knowing we want 'something',
but not spelling it out.

My humble hope is that this document helps to improve the quality of
the ongoing discussion.


2. Getting it Straight
----------------------

2.1 Terminology (n); cf. lack of ~

All this talk about decorators and attributes is really missing the point.

Well, the word 'attribute' suffers severe proliferation in computer science.
Everyone seems to have a different opinion on its meaning. The Pythonic
definition is at best misleading when it comes to our discussion. And
a 'decorator' is a software design pattern -- this is not a language issue.

But if we want to extend syntax we _have_ to talk about language. Computer
language design, that is. So let me rephrase our desire in proper terms:

==> We want to define a new kind of DECLARATIVE SYNTAX ELEMENT for Python.

Now that it's out of the hat, you can gain new appreciation at the
preceeding discussion. Part of the problem: naming conveys meaning.
Not having a proper name for something will lead you astray.


2.2 I Hereby Declare ...

I sure don't want to patronize the language cracks out there. You all know
very well what declarative vs. imperative means.

However giving some examples in context is always useful. So here are
a few of the declarative notions we would like to express, phrased
in natural language statements:

  "This is a classmethod."
  "This function needs to be run at exit."
  "This is an abstract class."
  "This is the documentation for a class attribute."
  "This method has been written by John Doe."
  "This class implements the Foo interface."
  "This method implements the Visitor pattern for this class."
  "This parameter must be an integral type."
  "This compound block shall by synchronized by a lock."
  "This statement should be supressed when DEBUG is not defined at runtime."

All of these statements _declare_ that some other language element
(class, method, function, class attribute ...) has a specific property.

Usually they tell you something about the _effect_ of that declaration,
but most of this knowledge is implied (you ought to know what happens
when you use such a statement).

However in general they do NOT tell you _how_ this effect is accomplished
(i.e. implemented). Nor should they. Declarative syntax owes much of its
power to the effect that most goes on behind the scenes. Someone else
has of course written up in glorious detail how that is to happen.
But you (the user of such a declaration) just get to say what you _want_.
Not more, but no less.

BTW: the word 'statement' in this context means 'natural language statement'.
     In computer languages we have more degrees of freedom, like keywords
     or delimiters. That's why I called it a declarative syntax _element_.
     The discussion about syntactic alternatives deserves an entire section
     of its own (see below).

Ok ... back to our high tech adventure.


2.3 Definitions

Declarative Syntax Element (DSE)
  The syntax element includes the Declaration itself and any lexical tokens
  that are required to make it recognizable as such by the parser. It can
  be either a grammatical production or a compound lexical token. This may
  be a minor issue from a visual point of view, but has subtle implications
  when it comes to applicability of the syntax element.

Declaration
  The content of the DSE. As far as the grammar goes it could be anything
  from an identifier up to a suite. It could even be in a language of its
  own (don't do that). However looking ahead to the section on semantics
  it makes most sense to use an expression.
  The term 'Declaration' is a bit too generic and would apply to other
  syntactic elements, too (e.g. a method definition includes a declaration).
  Thus it is suggested that the more specific term be used everywhere.

Declarative Expression (DE)
  The most likely syntactical choice for a Declaration. Any standard Python
  expression is possible here.

Declarative Object (DO)
  The result of evaluating a Declarative Expression.

Target of the DSE
  The grammatical element that is the target of the DSE. This could be
  anything from a method definition down to a statement or even an
  expression. The exact range of grammatical variety that could or should
  be permissible is discussed in the sections on syntax and semantics.

Processing of the DSE
  Processing of the DSE involves several steps: parsing the DSE,
  Compilation of the DE, Binding to the Target, Evaluating the DE and
  Applying the DO. These steps may or may not occur in that order.
  Each step may be assigned to a different processing phase.
  The important decision to make is the exact point in time _when_ each
  step is done.

Binding to the Target
  The process of binding either the DE or the DO to the Target.

Evaluation of the DE
  The process of evaluating a DE gives us a DO. This however does not (yet)
  affect the Target.

Applying the DO
  The process of applying the DO to the Target of the DSE.


2.4 Now, what?

Ok, now that we have set things straight its time to get our hands dirty.

I suggest we first take a peek over the fence, have a closer look at
use cases, take a refreshing detour to semantic wonderland and
then visit syntax hell again.

Gee, like all good things, the fun part is at the end.


3. Foreign Territories
----------------------

3.1 Big Brother in Action: C# Attributes

Ignoring C#, just because it comes from the company who brought you EDLIN
and BlueScreens is NOT a wise move.

You gotta keep up with the news or you loose track. What the heck, the
language spec (ECMA 334 -- freely available for download) goes so far as
to explicitly spell out that you should NOT use Hungarian notation for
identifiers. Whoa! Maybe they made their mind up?!?

And for our little discussion C# has quite something to offer. Not only
does it have an extensible declarative element. The same company brings
you to new heights of enjoyment with their .NET and Longhorn initiatives.
And part of that all-new lifestyle is using C# in every imaginable corner.
So we get some desperately needed use cases to fuel our discussion.
Even overly contorted ones -- promise!

So let's explain the way C# attributes work in our terms.


3.1.1 Syntax

An attribute section consists of one or more C# DSEs prepended to
the target.

A C# DSE is an optional attribute-target plus a colon and a list of
attributes enclosed in brackets:

DSE ::= "[" [attribute-target ":" ] attribute ("," attribute)* [","] "]"

A C# attribute cannot be an arbitrary expression. It looks like a call
to a constructor, but without "new":

attribute ::= type [ "(" arguments ")" ]

However the arguments look like any regular call and allow for arbitrary
constant expressions. Both positional and named arguments are allowed.
I'll save you the gory details of the syntax.

Most of the time the target of the DSE is implicitly derived from context.
The attribute-target allows you to specify the target explicitly.
The permissible targets are:

- An assembly or module.
- A type declaration.
- A method or operator declaration.
- A parameter or return value declaration.
- A field declaration.
- An accessor declaration.
- An event declaration.
- An enum declaration.

Defining an attribute is simple: write a class that derives from the
abstract class System.Attribute or any subclass of it.


3.1.2 Semantics

No big surprises so far. The semantics however are a bit tricky:

During compilation the argument expressions are evaluated. They must
resolve to constant arguments. An instance constructor is derived from
the type and the type of the arguments. The type, the constructor and
the arguments are bound to the target in the image for the module.

During runtime when an attribute is accessed its constructor is called
and returns the attribute instance. This is available with the
GetCustomAttributes() reflection method.

C# has static typing so compilation of one module requires access to
all referenced modules. Several 'magic' attributes assigned to members
of the referenced modules are evaluated at compile time to achieve
interesting effects:

- [AttributeUsage] specifies the scope of an attribute definition.
  I.e. which target(s) it applies to, whether it is inherited and if
  multiple applications to the same target are allowed.

- [Conditional] allows conditional compilation. This allows you to omit
  the call to conditional methods defined in other modules depending on
  defines set while compiling the module containing the call. So basically
  it is sufficient to tag the target once and get the effect everywhere
  else whenever it is used.
  Confused? Please read section 24.4.2 of the spec.

- [Obsolete] marks some elements as obsolete. When the compiler encounters
  it on a referenced element it emits a warning.

- The standard mentions a [Flags] attribute that applies to enums and
  redefines it to automatically assign bitfield masks instead of values to
  the enum members. However I have not found enough information to guess
  the implementation details.

There are a couple of other attributes that directly influence the
runtime. E.g. the JIT compiler takes hints from the [StructLayout]
or the [MarshalAs] attributes. And automatic permission elevation
(including permission checks) is performed when code security attributes
are present.

C# has another interesting feature: it allows you to omit the 'Attribute'
suffix from say [FooAttribute]. So when the compiler encounters [Foo]
and finds a type named Foo that is not derived from the Attribute base
class it searches again for FooAttribute. This is useful for interfaces
where you can get both declarative and imperative behaviour using the
same name.

Ok, so far so good. But you cannot achieve the same kind of effects with
user defined attributes in C# since that would either require extending
the compiler or force compile time evaluation of attributes.

This is where XC# comes in: this extension to the C# compiler opens up
the compilation entity tree and allows you to operate on that. It comes
packaged with some basic attributes: declarative assertions (contracts),
code coverage analysis, design rule verification, spell checking and
(oh well) code obfuscation. You can write your own custom attributes
and you can inspect and modify every aspect of the grammar on the fly.


3.1.3 Use Cases

The following use cases have been culled from various documents at MSDN
and other sources. I've stripped the implementations and just left the
bare declarations and definitions to avoid clutter.

*** An attribute definition uses attribute declarations, too:

[AttributeUsage(AttributeTargets.All)]
public class HelpAttribute: Attribute {
    public HelpAttribute(string url) {
        this.url = url;
    }
    public string Topic = null;
    private string url;
    public string Url {
        get { return url; }
    }
}

The attribute we just defined can be used like this:

[Help("http://.../Class1.html")]
public class Class1 {
    [Help("http://.../Class1.html", Topic = "Method1")]
    public void Method1() {}
}

*** A DEBUG Conditional:

[Conditional("DEBUG")]
public static void DebugMessage(string msg)
{
    ...
}

// Somewhere else, even in a different module
...
    // The *call* to the method is omitted if DEBUG is not defined while
    // compiling *this* module.
    DebugMessage("foo")
...

*** The enum magic:

[Flags]
public enum FileModes {
    UserRead,
    UserWrite,
    UserExecute,
    ...
}

This one gets you 1, 2, 4, ... instead of 0, 1, 2, ...

*** Custom serialization (marshaling, pickling) made easy:

[Serializable()]
public class PartiallySerializableClass
{
 public string serializeMe;
 public int serializeMeToo;
 [NonSerialized()] public string leaveMeAlone;
}

Not only is this easier to specify than imperative serialization support.
It is also less error prone because the serializer and deserializer is
autogenerated from the type information.

*** An XC# example of a declarative assertion (contract) that dynamically
    adds assertion checking code at compile time:

void WriteHashCode([NotNull] object o)
{
    Console.WriteLine(o.GetHashCode());
}

... generates the following code with an imperative assertion:

void WriteHashCode(object o)
{
    Debug.Assert(o != null);
    Console.WriteLine(o.GetHashCode());
}

But the really interesting consequences of a declarative assertion are:
- It allows for runtime introspection.
- If you include it in the interface declaration, it automatically
  applies to any implementation. This allows for inheritance of
  declarative aspects.

*** Calls to unmanaged code:

class NativeMethod
{
    [DllImport("msvcrt.dll", EntryPoint="_getch")]
    public static extern int GetCh();
}

*** Defining security behaviour:

[SecurityPermission(SecurityAction.Deny, Flags =
    SecurityPermissionFlag.UnmanagedCode)]
private static void CallUnmanagedCodeWithoutPermission()
{
    ...
}

[SuppressUnmanagedCodeSecurityAttribute()]
[DllImport("msvcrt.dll")]
internal static extern int puts(string str);

*** WinFS extensions:

[Folder(Uri="#System/Drive")]
public class Drive
{
    ...
    [Key]
    public string DriveName
    {
        [Probe] get { return drive; }
        set { ; }
    }
    ...
    [Probe(Uri="Name=_")]
    public static Drive GetLogicalDrive(string name)
    {
        ...
    }
    ...
    [Probe(Uri="GetAll", ResultType=typeof(Drive))]
    public static Drive[] GetLogicalDrives()
    {
        ...
    }
    ...
}

*** Indigo (Longhorn remote messaging framework):

URL: http://msdn.microsoft.com/longhorn/default.aspx?pull=/library/en-us/dnlong/html/indigoattrprog.asp

[System.MessageBus.Services.PortTypeChannelAttribute
    (UsingName="IndigoServiceClient")]
public interface IHelloChannel :
System.MessageBus.Services.IDatagramPortTypeChannel {

    [System.MessageBus.Services.WrappedMessageAttribute
        (Namespace="http://tempuri.org/")]
    [System.MessageBus.Services.ServiceMethodAttribute()]
    [return: System.MessageBus.Services.WrappedMessageAttribute
        (Namespace="http://tempuri.org/")]
    string Greeting(string name);
}

[DatagramPortType (Name = "Hello", Namespace = "http://tempuri.org/")]
public class IndigoService
{
    [ServiceMethod]
    public string Greeting (string name)
    {
        ...
    }
}


3.2 Catch Up, Baby: Java Annotations

Java has plenty of declarative syntax elements. Some people argue that
Java is overly declarative. Some like it. Some don't.

But up until recently it had no support for an extensible DSE.
Ok, ok ... Java always had javadoc tags embedded in comments:

/**
 * An abstract class with foo-like behaviour.
 *
 * @author  John Q. Doe
 * @see     Bar
 */
public abstract class Foo { ... }

These are processed by an extra tool (javadoc) to autogenerate the docs
from the sources. Calling this a DSE is ok, but it is not part of the
language itself and as such has no access to its structure (you cannot
pass constant expressions).

Some say it was a quick hack initially. But it stuck. And it got abused.
A lot. Examples below.

It seems now we have a plethora of tools that scan for javadoc tags and
then do some pre- or post-processing of your source and class files.

But then came JSR-175 (public review ended November 2003) ...


3.2.1 Syntax

JSR-175 introduces a new syntax for 'Annotations'.

An annotation definition starts with the "@interface" keyword. A severly
restricted variant of the interface definition syntax is used. It has
implicit and final inheritance from java.lang.annotation.Annotation.

Well ... to me it looks just like a glorified struct declaration anyway.

An annotation can be applied to any declaration: classes, interfaces,
fields, methods, parameters, constructors, enums, local variables and
enum constants (implied field declarations). However the same annotation
type can be applied at most once to any entity. Application of an
annotation uses one of three syntactic variants:

"@" type "(" member-name "=" member-val ["," member-name "=" member-val]* ")"
"@" type
"@" type "(" single-member-value ")"

The latter two are just shorthands for variants of the first one with
an empty list or for a single member type. This muddies the distinction
between types and constructors quite a bit though. The values must be
constant expressions of course.


3.2.2 Semantics

During compilation the constant expression arguments for annotations
are evaluated and combined into a set of initializers for the structure
defined by the annotation interface.

There are three retention policies for this set of initializers:

- Source-only annotations are dropped at compile time. This is supposed
  to replace/augment javadoc tags in source comments. Local-variable
  annotations are always source-only.

- Class-only annotations are stored in the class file but not loaded
  into the runtime.

- Runtime annotations are stored in the class file and are loaded
  by the runtime.

Reflection support via java.lang.reflect.AnnotatedElement works only
for runtime annotations. Reflection is easy to use because only a single
annotation for each type may be present for each element. The classes
used to represent annotations are created at runtime using dynamic proxies.

Owing to static typing the Java compiler has to read referenced class files
and understands a few special annotations (excerpt):

- @Target specifies the allowable targets for an annotation definition.

- @RetentionPolicy() specifies the retention policy for an annotation
  definition.

- @Inherited allows for inheritance of annotations from superclasses
  (this is NOT inheritance for annotation definitions themselves).

To summarize: nothing spectacular and not very dynamic. It shows that
the feature has been added to the language as an afterthought. The key
improvement over javadoc tags is that external tools do not have to
parse Java source files (unless pre-processing is required) and that
annotations can be stored in the class file. This however does not
obviate the need for some post-processors either.


3.2.3 Use Cases With JSR-175 Annotations

Since the spec is pretty young you won't find many use cases right now:

*** Giving a new tune to javadoc stuff:

public @interface RequestForEnhancement {
    int    id();
    String synopsis();
    String engineer();
    String date();
}

@RequestForEnhancement(
    id       = 2868724,
    synopsis = "Provide time-travel functionality",
    engineer = "Mr. Peabody",
    date     = "4/1/2004"
)
public static void travelThroughTime(Date destination) { ... }

*** This is the definition for the @Retention meta-annotation:

public enum RetentionPolicy {
    SOURCE,
    CLASS,
    RUNTIME
}

@Documented @Retention(RUNTIME) @Target(ANNOTATION_TYPE)
public @interface Retention {
    RetentionPolicy value();
}


3.2.4 Use Cases With javadoc Annotations

*** Some stuff for BEA WebLogic using Java web services annotations:

/**
 * @jws:location http-url="http://localhost:7001/webapp/Bank.jws"
 * @jws:protocol http-soap="true"
 */
public interface BankControl extends ServiceControl
{
}

*** Security annotations and conversational support for web services:

/**
 * @common:security single-principal="false"
 */
public class PurchaseSupplies implements com.bea.jws.WebService
{
    /**
     * @common:operation
     * @jws:conversation phase="start"
     */
    public void requestPurchase()
    {
    }

    /**
     * @common:operation
     * @jws:conversation phase="continue"
     */
    public void approvePurchase()
    {
    }

    /**
     * @common:operation
     * @jws:conversation phase="finish"
     */
    public void executePurchase()
    {
    }
}

*** Embedded SQL for Java:

/**
 * @jc:sql statement::
 *     SELECT name
 *     FROM employees
 *     WHERE name LIKE {partialName}
 * ::
 */
public String[] partialNameSearch(String partialName);


4. Pythonic (Ab)use Cases
-------------------------

Leaving the exact choice for the syntax aside I used my personal favourite
everywhere. I suggest you substitute it with your favourite choice just
to see the visual effect.

I'm sorry, but I gave up trying to identify the original contributor
for each of these examples. Let's say it's a true community effort.
A big thank you to everyone for your input!


4.1 Assorted Attributes

Attribute setters modify the target attributes but not the target itself.

*** The first one is generic, the second one could inherit from it:

<| funcattrs(foo = 1, bar = "baz") |>
def foobar(x):
    ...

<| rstattrs(
       arguments = (1, 0, 1),
       options   = {'class': directives.class_option},
       content   = 1)
 |>
def admonition(*args):
    return make_admonition(nodes.admonition, *args)

<| rstattrs(content = 1) |>
def attention(*args):
    return make_admonition(nodes.attention, *args)

*** Here is a nice example to show how neatly the vertical bars line up:

    <| Author("Joe Q. Random") |>
    <| Copyright("ACME Inc.")  |>
    <| Release("2004-04-01")   |>
    <| Version(0, 0, 1)        |>

    def industrialStrengthMethod(self):
        raise BlueScreenError(random.Random())

*** Hints for mad metaclasses with magic mangling motives:

    <|override|>
    def overrideMe(self, foo):
        ...

*** With clever definition of the DO application process (see semantics)
    we get docstrings *before* the corresponding definition. I guess
    nobody will like the idea, but anyway:

<|"""This is a new style docstring for a class.

     Bla bla bla ...
     Bla bla bla ...
     Bla bla bla ...
  """|>
class HeavilyDocumented(object):

    <|"Class attribute docstrings now have the proper position."|>
    BUFSIZE = 8192

    <|"This is a new style docstring for a method."|>
    def method1():
        pass

    <|"Dynamic %s [%s]." % ("docstring" % time.ctime()) |>
    def thisReallyWorks():
        pass


4.2 Roaring Registries

Registry functions add a reference to the target in some other object.
Often combined with attribute setters.

*** This one shows where it would be convenient to have a declarative
    type with the same name as an inheritable imperative type:

<| WebService("soap://localhost:8888/mywebservice") |>
class MyWebService(WebService):

    <| ServiceMethod(None, str) |>
    def setName(self, name):
        self.name = name

    <| ServiceMethod(str) |>
    def sayHello(self):
        return "Hello %s!" % self.name;

*** For more complicated interfaces to environments with static typing
    it would be nice to have declarations that apply to parameters
    and/or the return value:

    <| ServiceMethod |>
    def <|str|> method(self, <|str|> foo, <|int|> bar):
        ....


4.3 Witty Wrappers

Wrappers generally wrap functions or methods with their own functions.
The latter is then stored in the current dictionary to replace the
original function.

Oh BTW: the 'synchronized' wrapper has a section of its own. See below.

*** We know these two well enough by now:

    <|classmethod|>
    def AClassMethod(cls, foo):
        pass

    <|staticmethod|>
    def AStaticMethod(foo):
        pass

*** Using multiple decorators:

    <| Decorator1(), Decorator2("Just a random string") |>
    def someMethod(...):
        ...

    # which is equivalent to:

    <| Decorator1() |>
    <| Decorator2("Just a random string") |>
    def someMethod(...):
        ...

*** Generics for Python? Well, if you must:

<|generic(int)|>
def f(x):
    print x, 'is an int'

<|generic(str)|>
def f(x):
    print x, 'is a string'

*** A variation of the theme: runtime type checking:

<|CheckSignature(int, int)|>
def f(x, y):
    return x+y+1

*** Interface evolution does not need to be messy:

    <|Context("version", 1)|>
    def movePlayerMessage(self, arg1):
        ....

    <|Context("version", 2)|>
    def movePlayerMessage(self, arg1, arg2):
        ....

Yes, the second declaration needs to be able to get at the dict entry for
the first one (see semantics).

*** Defining a native method interface:

# The first one should be applicable to the module. TODO: But how?
<|native.Library("gtk-x11-2.0")|>
<|native.ClassMap()|>
class GtkScrolledWindow(GtkWindow):

    # Yow, the great renaming ... the GNOME people would love it ... :)

    <|native.EntryPoint("gtk_scrolled_window_set_policy")|>
    def setPolicy(self,
                  <|GtkPolicyType|> hScrollbarPolicy,
                  <|GtkPolicyType|> vScrollbarPolicy): pass

    <|native.EntryPoint("gtk_scrolled_window_get_shadow_type")|>
    def <|GtkShadowType|> getShadowType(self): pass


4.4 Proper Properties

The following has been proposed to make property definition easier:

class Foo(object):

    <|propget|>
    def x(self): return self.__x

    <|propset|>
    def x(self, newx): self.__x = val

    <|propdel|>
    def x(self): del self.__x

The problem with this is that propxxx needs to go through some hoops to
update the previous definition of x. This includes using sys._getframe()
and then searching the local and global dicts of the frame. Oh dear!

This is a good argument why the DO should get the target object _and_
the dict where the TO is to be stored.

A different idea would be to follow the C# precedent which provides an
extra nesting level for properties:

class Foo
{
    private int x_;

    public int x {           // <-- !!
        get { return x_; }
        set { x_ = value; }  // value is an implicit identifier
    }
}

Foo foo = new Foo()
foo.x = 1
int y = foo.x

However I don't know how to do this in Python. Declaring a method
as a property and using nested functions for the getter/setter does
not work because the locals (i.e. the nested functions) are lost:

class Foo(object):

    <|property|>
    def x(self):                          # Does not work!
        def get(): return self._x
        def set(value): self._x = value

Doing the same thing with an inner class might work somehow, but I'm not
sure. We could get inheritance for properties, too!?

TODO: I haven't explored this any further. More input is welcome.


4.5 To Sync or Not to Sync

Java has special declarative syntax for synchronized methods and blocks:

public abstract class FancyControlStream extends FancyStream {

    public synchronized void putMsg(byte[] data)
    {
    }

    public int kickBack()
    {
        ...
        synchronized (obj) {
            obj += len;
            obj.notify();
        }
        ...
    }
}

This would be a good use case for our DSE, too. But ... being only
applicable to a method definition just doesn't cut it:

    <|Synchronized(lock)|>
    def put(self, item):
        ...

This is way too tedious for some stuff. So, what about:

<|Synchronized()|>
class ThreadSafeQueue(Queue):
    pass                          # Yes, this is all that's needed.

On the other hand you really want to use fine grained locking for more
ambitious endeavors. And this is where you need to use locks at the
compound statement level (called suites in Python and blocks elsewhere):

    def stopStream(self):

        <| Synchronized(self.recv_lock) |>
        if self.recv_len<=0:
            if self.recv_thread:
                self.recv_thread.interrupt()
            self.recv_thread=None

        self.send_close=True

        <| Synchronized(self.send_lock) |>
        while self.send_len>=0 && self.send_close:
            self.send_lock.notify()

Getting it at the statement level would be nice, too. But you could work
around this with 'if 1: SUITE'.

How you would _implement_ this is another story. And this is where the
bad reputation from Java's 'synchronized' comes from: there have been some
initial design flaws. However just because you have one bad precedent
does not mean we can't do it better. IMHO declaring some parts of a program
to be regions with synchronized access is inherently useful.


4.6 Lexical Liberty

To get at the lexical level you may need a different delimiter which is
evaluated at compile time. I have opted not to replace it with a different
one in the following examples.

Please don't flame me. I'm just documenting some strange ideas:

class Whatever(object):

  # Uh, imports and assignments in a declarative compile-time context?
  <| from declarative.lexical import * |>
  <| T = Conditional("TRACE") |>

  # These statements are optimized away at module load time.
  def run(self):
      <|T|> self.tracePrint("Startup")
      ...
      <|T|> self.tracePrint("Shutdown")


  # Combining the power of Python with the syntax of SQL. Or vice versa?

  def query1(self, start, end)
      return <|SQL|> "SELECT * FROM table WHERE name BETWEEN :start AND :end"

  def query2(self, start, end)
      return <|SQLPy|> SELECT * FROM table WHERE name BETWEEN :start AND :end

  def query3(self, table, a, b)
      return <|PySQL|> SELECT "*" FROM table WHERE "f1" == a*2 AND "f2" >= a+b


  # C has inline assembler. Python has inline C.

  def fast1(self):
      <|C|> "{ for (int i=0; i<100; i++) somefunc(i) }"

  def fast2(self):
      <|C|> { for (int i=0; i<100; i++) somefunc(i) }

  def fast3(self, a, b):
      x = self.foo + <|C|>
      int dummy(int a, int b)
      {
          for (int i=0; i<1000; i++) { a=a+3*i+(b&15); b+=a; }
          return b;
      }
      <||>
      return x

  <|C|>
  int fast4(int a, int b)
  {
      for (int i=0; i<1000; i++) { a=a+3*i+(b&15); b+=a; }
      return b;
  }
  <||>


# Sssshhh! Don't mention the r-word!

<|LexicalSubstitute("@", ("self", "."))|>
<|Grammar("parameter_list").afterParsing( lambda p: p.insert(0, "self") )|>
class Wherever(object):
    def __init__():
        @flags=[]

    def setFlag(flag):
        @flags.append(flag)


5. Semantic Wonderland
----------------------

5.1 What?

First we need to specify what kind of grammatical element a DSE may
contain. I could go on an reason about all the possibilities. But
nobody ever mentioned anything else than an expression. It seems to be
the most straightforward approach.

This document is already far too long. A DSE contains a DE. So be it. :)

Expressions evaluate to a single object. This is the DO in our case.

For multiple declarations in a single DSE there are two possibilities:
- Allow a plain expression and add the commas to the DSE grammar.
- Allow an expression_list and add tuple handling.

Multiple DOs from multiple adjacent DSEs result in a DO list (in the
given order).

The target the DSE applies to is to be specified by the syntax.
To be able to bind the DSE to the target it needs to be bundled into
an object. We call this the Target Object (TO).

What the TO needs to contain depends on the target. The issues with
arbitrary grammatical elements are discussed below (see 'What Else?').

So let's restrict the target to functions, methods and classes for now.
Each one of them already bundles its contents into an object.

But _defining_ one of them involves adding its name to a dictionary, too:

- The Target Dictionary (TD) is required by the DO for some use cases.

- The name to be defined is accessible from the TO for all three types.
  But it is either read-only (for functions) and/or modifying it may not
  get you the desired effect. It is unclear whether we have any use case
  that requires storing a different name in the TD instead of the one used
  in the target. Being able to supress the store might be desirable, too.
  TODO: Ideas?


5.2 How?

[My bet is that this section will be the most controversial.]

We need to specify how each of the required steps is to be performed.
First the straightforward stuff:

- Parsing the DSE: since this is a plain Python expression it is to
  be performed by the compiler.

- Compilation of the DE: dito.

- Binding to the target: the Pythonic way to do this is to emit code
  for the module or class initialization code objects. The code gets
  to mangle the compiled DE, the target, the target name and the TD.

  [
  It is an implementation detail whether ...
  - a call to each DO is explicitly emitted
  OR
  - whether we call a builtin function or emit a new bytecode that
    gets passed the TO and all DOs.
  The latter may have some advantages for future extensibility.
  ]

  TODO: Code generation needs to be specified in detail.

- Evaluating the DE: this is done by the code compiled from the DE.

Now the troublesome part: applying the DO.


5.2.1 Calling the DO

The original proposal was derived from the way stuff like classmethod
works:

*** TO is a function:

def func(x):
    ...
func=callable(func)

*** TO is a method:

class Foo(object):
    def method(args):
        ...
    method=callable(method)

So we require the DO to be a callable and apply it by calling it,
passing the TO. The return value is then used as the new TO and stored
under the target name in the TD. I.e.: TO = DO(TO)

[
Subtle side effect: The TD may contain a previous element named like
the target before the target definition is performed. This is available
to the DO because the TO is not stored until all DOs have been applied.

This cannot be achieved without some renaming games by doing it the
classic way.
]

There are three common cases for DEs:

*** The DE is a function call that evaluates to an inner function. The DO
    is the inner function. It may be bound to derivatives of some arguments
    of the DE. Applying the DO means calling the inner function.

def funcattrs(attrs):
    def inner(func):
        for name, value in attrs.iteritems():
            setattr(func, name, value)
        return func
    return inner

*** The DE is a class constructor and evaluates to an instance of the
    class. The DO is an instance. It may hold derivatives of some arguments
    of the DE. Applying the DO means calling the instance (which only works
    if you define a __call__ method).

class funcattrs(object):
    def __init__(self, **kwds):
        self.attrs = kwds
    def __call__(self, func):
        for name, value in self.attrs.iteritems():
            setattr(func, name, value)
        return func

*** The DE is a class and evaluates to a class. The DO is a class.
    Applying the DO means to call the constructor of the class.
    A side effect of this is that the resulting TO is an instance of
    the class.

    The primary example for this are classmethod and staticmethod which
    are builtin types.


5.2.2 Using __declare__

Reusing the callable attribute of objects for declarator application
has a few drawbacks:

- Any callable may be used as a DO. There is no error checking. Mistaking
  imperative classes for declarative classes, dropping empty braces,
  scoping mismatches or just plain carelessness will NOT be caught.

  This may be very hard to track down since the resulting TO is not
  checked for validity. Errors may pop up very late or never.
  Any error message you might get will probably be misleading.

- A class cannot have both declarative and imperative behaviour.
  However as the precedent given by C# shows, this might be quite useful.

- Inheritance is a useful concept for declarative behaviour, too.
  Thus it is most likely that DE is a class constructor. This requires
  the use of __call__ which is neither self-documenting nor exactly
  easy to explain to newbies.

I propose to use a new kind of slot with the name "__declare__".

The DO must define this slot and it must be bound to a callable.
Applying the DO means calling the slot and passing the TO. The return
value is used as the new TO and stored under the target name in the TD.
I.e.: TO = DO.__declare__(TO)

This has the following advantages:

- Existing objects cannot be accidentially used as declarative expressions.

- Error checking is done at module load time and misuse will abort
  the load with a unique error message exactly pointing out the problem.

- Defining a method with the name __declare__ makes your intention
  crystal clear. Newbies can find out about the meaning easily by
  searching for __declare__ in the documentation. Compare this with
  the value the documentation for __call__ would have to a newbie.

- Whenever you see a declaration you know exactly where to look in the
  class definition to find out what it does.

- Requiring a __declare__ slot strongly encourages to use classes instead
  of functions for declarative behaviour. Since classes are inheritable,
  this will over time improve the quality of the code base.

  Using functions is still possible but one has to set the __declare__
  slot explicitly in the function dictionary. You can even avoid defining
  an inner function if you don't need bound DE arguments (otherwise
  a class would be the cleaner solution).

- We can easily support the old and the new way to apply classmethod
  and staticmethod. We also can adapt the new calling sequence to
  our needs without breaking compatibility with existing uses of
  classmethod and staticmethod (see the next section).

- We may set __declare__ slots on internal types if we want to give
  them declarative behaviour without necessarily making them callables.
  E.g. you can have strings behave like docstrings if used declaratively.

I know very well that adding a new slot requires changes in several
places. But I think the benefit outweighs the required work by far.

TODO: Write up what needs to be done to add a new slot.


5.2.3 Passing The Context

As you can see from the use cases, some of them need access to the TD.
The current workaround involves sys._getframe() and is ugly beyond belief.

We can avoid this problem by passing the TO *and* the TD to the DO.
I.e.: TO = DO.__declare__(TD, TO)

This may be required, too for some future extensions that involve targets
other than functions, methods and classes. In general it is not a bad
idea to pass the context where the target is defined.

Since applying the DOs is usually done at module initialization there
is no performance penalty for passing two arguments instead of one.
Code generation would be pretty easy, too.

Summary: we gain flexibility with minimal cost.

TODO: Check the issue mentioned above about modifying the target name
      before it gets stored in the TD. We could pass a modifiable
      reference to the name somehow but that would complicate the code
      generation for this case quite a bit. Ideas?


5.3 When?

There are basically two choices for the temporal behaviour of the DSE:

- A DSE with access to the lexical level needs compile time evaluated DEs.
  As indicated in other sections this is not within the current scope
  of this document.

- Any other imaginable DSE is only useful if the DE is evaluated and
  applied at module initialization time. Any runtime-only behaviour
  can be implemented on top of it.

This implies that all other processing steps need to be done at compilation
time (parsing the DSE, compilation of the DE and binding to the target).

Phew. That was easy.

A subtle issue that has been discussed previously is the order of
application of the DEs. Previously you had to write things backwards
to get the correct order of application (d first, then e):

def f(x):
    ...
f=e(d(f))

Thus the question arised, what the order might be with the new syntax:

<|d,e|>
def f(x):
    ...

The natural interpretation depends a little bit on the syntax chosen
(whether the DSE is before or after the function name). But with my
favourite syntax there is no ambiguity: d is applied before e.
Also <|d,e|> is equivalent to <|d|> <|e|>.

It helps to remember that Pythons existing declarative statements
(such as "def") are implemented in an imperative way. And this is
clearly top-to-bottom and left-to-right.

If the order of application is important, then it is the duty of the user
to get the order of the DEs right. It is however deemed unlikely that
this is a problem in reality.

TODO: I cannot find a way for the DO class author to specify the
      preferred order of application (other than through documentation).
      Priorities would not help since they would need to be attached
      to the DSE, which is pretty useless (remember: the compiler does not
      see the DO class). Comments?


5.4 What else?

The 'synchronized' issue highlights the need for having declarative
syntax that is applicable to a wider range of syntactic constructs.

But there is some trouble ahead with Python: methods and classes are first
class objects. Compound statements are not.

Passing a method or a class to a function to modify them is easy.
Inserting the returned object (possibly a wrapper) into the module or
class dictionary is easy, too. Getting the same level of support for
compound statements or even individual statements is way more difficult.

You basically have two choices for this:

a) Make statements first class objects and somehow optimize this artifact
   away by using late code combining.

b) Allow compile time evaluation of a declarative element giving it access
   to the lexical level.

Ok, so a) is more akin to the way statements inside a class definition
incrementally build up the class. You'd have to plan ahead a bit and
think how this could be extended to declarations for parameters and
other stuff, too.

But b) is a really a much more powerful tool. Forget about CPP macros.
Forget about SQL preprocessors. You can do all of this yourself now just
by writing some Python functions. C has inline assembler, Python could
have inline C! Embedding foreign languages, extending the grammer, even
redefining your favourite lexical tokens: you get it, if you ask for it.

[
Just a thought: you could do b) 'the language way' and open up the
compiler or 'the text processing way' and open up the input stream.
The latter would be pretty easy to do, I think. But then you may
not get to see or modify all the compiler context you need.
]

Both issues need A LOT MORE discussion and are certainly NOT something
for the 2.4 time frame or for PEP 318.

However we should take a mental note about two things:

- A new declarative element SHOULD have the potential to be applied
  to almost any other syntactic element. This must be reflected in
  the initial syntactic _specification_ but not necessarily in the
  initial _implementation_.

- Having a declarative element evaluated at compile time holds big
  potential for the future. Although this could emulate any other more
  traditional kind of declarative element, the framework for this
  is just not ready today.
  So in fact I do NOT propose to specify such an element right now.
  However I propose to either
  a) allow for a later extension of the declarative element that we
     are defining now
  OR
  b) define a second declarative element later that shares most
     of the syntax but has different temporal evaluation behaviour.


6. Syntax, Syntax on the Wall
-----------------------------

Hey, the first Python beauty contest in history. Quick, place your bets.


6.1 Wishful Thinking

The Syntax ...

Requirement  Requirement Definition:
Mnemnonic    The Syntax ...
------------------------------------------------------------------------------
|LEXICAL|    MUST fit into the existing lexical structure.

|PARSER|     SHOULD put no extra burden on the existing parser.

|GRAMMAR|    MUST fit into the existing grammar.

|GENERIC|    SHOULD allow for the declarative element to be applied to
             almost any other syntactic element.

|EXPR|       SHOULD allow for a general expression as the content of the
             declarative element.

|SPLIT|      SHOULD allow for lengthy element content that may need to
             be split across several lines.

|SEQ|        SHOULD allow for a convenient notation for a sequence of
             declarative elements.

|CONCISE|    MUST be concise.

|VISUAL|     MUST stand out visually.

|DISTINCT|   SHOULD be immediately recognizable as a distinct language
             feature.

|MIMIC|      SHOULD NOT mimic completely unrelated features of other
             common programming languages.

|NEWBIE|     SHOULD NOT overly confuse newbies. :)

There are a lot more issues, but I think I covered the most important ones
from a language perspective. Feedback is appreciated of course.

>From a language perspective not much attention has been given to |GENERIC|.
This is what worried me most and got me to write this essay.


6.2 Round Up The Candidates

The existing lexical and grammatical structure allows ...

{STATEMENT} a statement with a (possibly new) keyword:
            as DECL

{COMPOUND}  a compound statement with a (possibly new) keyword:
            as:
                DECL
                ...

{EXTENSION} to extend an existing grammatical element:
            def f(x) DECL:

{CONTEXT}   an existing lexical element in a different context:
            def [DECL] f(x):

{REDEFINE}  redefining the meaning of an existing grammatical construct:
            [DECL]
            def f(x):

{UNARY}     a new introducing delimiter:
            @DECL
            .DECL

{ASYM}      a new asymmetric paired delimiter:
            *[DECL]
            @[DECL]

{SYM}       a new symmetric paired delimiter:
            [|DECL|]
            <|DECL|>

I guess there are some more variations on this theme, but none too relevant.


6.3 Narrowing the Candidate Set

Well our BDFL has already spoken out and narrowed the set considerably.
But I think it's still worthwile to discuss this in detail:


{STATEMENT} has not been received well, because it would almost certainly
necessitate a new keyword. And neither has a good one been proposed nor
should the introduction of a new keyword be taken lightly. Getting
|GENERIC| straight might prove to be difficult, too. And don't forget
about |EXPR| and |VISUAL| while you are at it.

Even if we manage to find a good name, it's doubtful that everyone seeing
such a construct knows that it is evaluated in a different context
than almost any other statement (violating |NEWBIE|).


{COMPOUND} looks strange to me because it does not fit well into the
lexical structure (putting aside the variants that violate |LEXICAL|).
Compound blocks usually contain statements and not expressions. And of
course it only works if the target of the declarative element is a block
(violating |GENERIC|).

Prepending a new kind of block to an existing target block introducer is
confusing because one may expect another level of nesting here (which
would violate |CONCISE|).

Adding a new kind of block just _after_ the target block introducer destroys
the visual link between the introducer and the body of the block.

The compact variant (putting a single declarative element right after
the colon) shares the concerns about |NEWBIE| with {STATEMENT}.

In short I cannot find a compelling reason to add a new kind of compound
statement to the language. And a compound declarative statement is ...
well ... awkward (considering the lack of precedent set by other languages).


{EXTENSION} has been thrown out early because it violates |CONCISE|,
|VISUAL| and |MIMIC|. But I think far more important are the violation
of |SEQ| and |EXPR|.


{CONTEXT} has been discussed to death but this is just because the initial
discussion focused on variants of this syntax. Other than that it has
few merits. Since it is an existing lexical construct (list displays)
it violates |GENERIC|, |DISTINCT| and |NEWBIE|. Depending on the position
it may also violate |PARSER|, |GRAMMAR| and |VISUAL|.

But I think the most compelling reason against it, is that it hurts the
eyes with |SPLIT|. The existing use cases for C# indicate that this would
be pretty common.


{REDEFINE} is problematic because it redefines an existing (though useless)
grammatical construct and as such violates |GRAMMAR| and |DISTINCT|.

The grammatical meaning could be redefined, but it would only work just
before a block introducer (violating |GENERIC|). Just because this syntax
mimics C# somewhat is not a good enough reason to introduce the same syntax
to Python. It cannot mimic it completely anyway due to lexical differences
(break-on-white-space vs. break-on-lines-or-sequence-delimiters).

It also violates the principle of least surprise: running a program using
this construct in an older version of Python won't get you a syntax error.
And the desired effect is dropped silently -- which can be a very dangerous
thing to do.


{UNARY} fails on |VISUAL| for |SPLIT| because the end may be hard to match
to the start. It may be hard for |PARSER| in intra-line contexts because
of |EXPR|. Restricting the construct to be used only on extra lines
would violate |GENERIC| and maybe |CONCISE|, too.

I do not think we should waste one of the three remaining unused
non-alphanumeric ASCII characters (@, $, ?) for this purpose. It would
miserably fail on |MIMIC| anyway (except for the Java precedent).

Prepending a dot is not an option because it would not conform to |GENERIC|,
|VISUAL|, |DISTINCT|, |MIMIC| and |NEWBIE|. Also GvR indicated that he
wanted to reserve this construct for a future "with" statement.


{ASYM} doesn't look bad if the construct is used on extra lines. It may be
a bit harder to spot in an intra-line context.


{SYM} looks good in any context. It would be natural to define it like
any other paired sequence delimiter ((), [] and {}). This allows for
flexible line breaks and broad grammatical applicability.


6.4 And The Winner is ...

Beware: everything I wrote is IMHO of course, but this applies especially
to this section!

{ASYM} saves a single character to type over {SYM} (which is not much of
a gain). OTOH the asymmetry spoils the visual effect quite a bit.

So my personal winner is: {SYM}

Your mileage may vary though. Even if you disagree with my reasoning, the
best the preceeding section buys you is a systematic way to discuss all
alternatives now. Go ahead!


6.5 ASCII Art

If (BIG IF!) there is a concluding discussion that indeed supports my
reasoning about the lexical alternatives we should decide/vote/pronounce
on a delimiter pair.

Since @, $ and ? usually do not come in pairs, the shortest possible
delimiter pair is two characters long each.

We should include one of the four visual pairs that ASCII has proudly
brought to your home for the past decades ((), [], {} or <>). For best
visual effect the pair should be used for the outer characters. The inner
characters may be a single non-paired character or another set of paired
characters.

Of course neither the opening nor the closing delimiter should collide
with an existing delimiter. Nor should the inner character be mistaken
for the start of an expression or (less important) for the end.

That leaves us with:

(< DECL >)   [< DECL >]   {< DECL >}

($ DECL $)   (% DECL %)   (& DECL &)   (/ DECL /)   (: DECL :)
(= DECL =)   (? DECL ?)   (@ DECL @)   (^ DECL ^)   (| DECL |)

[$ DECL $]   [% DECL %]   [& DECL &]   [/ DECL /]   [: DECL :]
[= DECL =]   [? DECL ?]   [@ DECL @]   [^ DECL ^]   [| DECL |]

{$ DECL $}   {% DECL %}   {& DECL &}   {/ DECL /}   {: DECL :}
{= DECL =}   {? DECL ?}   {@ DECL @}   {^ DECL ^}   {| DECL |}

<$ DECL $>   <% DECL %>   <& DECL &>   </ DECL />   <: DECL :>
<= DECL =>   <? DECL ?>   <@ DECL @>   <^ DECL ^>   <| DECL |>

Now, before you choose, consider that you can use it without any
spaces between the delimiter and the declarative expression. I guess
this would be common for e.g. classmethod. That's why my personal
favourites (in descending order of preference) are:

<|DECL|>   <== I like this one most.
<:DECL:>
(|DECL|)
[|DECL|]
<%DECL%>
<?DECL?>

The vertical bars line up neatly if you use them on successive lines.
And it visually separates the content well, even without spaces.

But back to a more scientific analysis: conveying meaning with delimiters
is hard. Only precedent may help us here:

So <? ?> is used in XML for processing instructions (PIs). But it's up to
the language used for the PI whether the content of the PI has declarative
or imperative meaning. We'd fall prey to |MIMIC| I guess.

Choosing </ /> would not help to clear up matters, either.

Most other delimiters have no common precedent (that I know of). But the
inner character has. That makes $, @ and = seem awkward choices.

Oh and when picking one, we should think about reserving another one for
a future compile time evaluated variant, too (IMHO <:DECL:> has some merit
here).

BTW: choosing smileys as lexical tokens would sure help to entertain
     slashdot for weeks! :)


6.6 Finally

Still with me? Good, because here is the syntax I propose:

Lexical definitions:

DECL_ELEMENT_START ::= "<|"
DECL_ELEMENT_END   ::= "|>"

The tokens should be defined so they work like the existing paired
delimiters (i.e. allowing NEWLINE).

Grammar definitions:

decl_element ::= "<|" [expression] ("," expression)* [","] "|>"
OR
decl_element ::= "<|" [expression_list] "|>"

The expressions should yield declarative objects of course (see the section
on semantics).

The DSE should be placed immediately *before* the lexical element to which
it applies. Multiple declarative elements may stack up and all apply to
the following element.

The use cases indicate the following possible applications in descending
order of importance:

- Method and function definitions ("def").
- Class definitions ("class").
- Related to the module (there is no good place to put it though ...).
- Class attributes, i.e. assignments in a class definition.
- All compound statements ("if", "while", "for", "try").
- Parameter definitions.
- Related to the return value of a method or function (either by applying
  it to "return" or by putting it between "def" and the function name).
- All statements, i.e. assignments, too.

The sheer variety indicates that we should not restrict the DSE to be
usable only on a line of its own.

We might as well make the DSE applicable to *any* grammatical element.
>From a language perspective that would certainly be pretty orthogonal.
>From an implementation perspective we may disallow it or ignore it for
some cases. The exact way to pass the target of the DSE to the
declarative object(s) needs to be figured out, though.

A more radical thought would be to treat the DSE as a compound lexical
token. This could then be used before ANY token. It would work like
a C comment block. Yes, this smells a bit like a macro, but it doesn't
behave like one (unless we do compile time evaluation). And no, I wouldn't
place my bets on this one.

TODO: More input is required. Please go ahead.

-------- End of Document --------


From arigo at tunes.org  Mon Apr  5 09:23:16 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon Apr  5 09:26:54 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
Message-ID: <20040405132316.GA21940@vicky.ecs.soton.ac.uk>

Hello Paul,

On Mon, Apr 05, 2004 at 01:54:38PM +0100, Moore, Paul wrote:
> I got the impression that Armin's point was that Psyco can optimise
> the integer case *if* the code is in Python, but not if it's in C.

Yes.

> [Armin, how does Psyco + a Python sort compare against timsort on
> more general objects?]

Psyco suffers its current general problem: if you are using an operation that
hasn't been explicitely recoded in Psyco (like tuple comparison) then the
optimization wanishes.  Sorting a list of (integer, complex_object), which is
more common in practice, is 1.5x slower with Python+Psyco.  I am sure that
with 30 minutes of effort in the proper code I can make it faster than Timsort
again.  But Psyco has the same general problem everywhere.

I think that Psyco isn't ready for real uses yet because you just can't
predict the resulting performance unless you know its internals.  My hope with
PyPy is that it will change that, by giving a way to generate all these
missing Psyco parts automatically from the PyPy source, but that's still a
long way.  I won't even mention that Psyco works well only on 386.  (It has
got an almost-ready platform-independent back-end, but that's slower, so it is
only a win in more extreme cases.)

> Hmm. I'm torn. On the one hand, it's really cool that Python+Psyco can
> out-perform C (another argument against the knee-jerk "recode bits in
> C" reaction). But my gut feel is that not enough people use Psyco to
> make this a good answer yet. On the third hand, if we recode chunks of
> the stdlib in C, do we kill too many chances for Psyco to work its
> magic?

I think that (re)coding in C is very positive right now.  And if we don't
actually loose the original Python source (if there is one) then we don't
really loose the ability to revert to it if at some point it is consistently
better.

Essentially, I wanted to rant about the fact that some performance
considerations are driving the language's evolution, and this is much harder
to revert to if later the performance reasons vanish.


A bientot,

Armin.


From dave at boost-consulting.com  Mon Apr  5 09:55:38 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Mon Apr  5 09:56:37 2004
Subject: [Python-Dev] Re: PEP309 re-written
References: <E1B8ngt-0005Wt-2e@mail.python.org>
	<40715D70.1010402@blueyonder.co.uk>
Message-ID: <u65ceil5x.fsf@boost-consulting.com>

Peter Harris <scav@blueyonder.co.uk> writes:

> David Abrams wrote:
>
>>> If anyone can think of any really elegant hacks that are naturally
>>> expressed by partial function application I'd like to see them
>>  
>>
>>
>>There are millions, but AFAICT they're all more natural with lambda,
>>so...
>>
>>        "I agree that lambda is usually good enough, just not
>>        always."
>>
>>Examples, please?
>>  
>>
> Well, you can use partial to special-case classes as shorthand object
> factories.  So:
>
> C = partial(Canvas,my_window,width=100,height=100,bg='white')
>
> ...gives you a callable C that is a factory for Tkinter Canvases
> parented to my_window,
> 100 pixels on a side, and with a white background.  How this differs
> from lambda is that
> you can override keyword parameters from the supplied defaults:
>
> c1 = C()
>
> c2 = C(width=200)
>
> c3 = C(bg='red')

C = lambda *args, **kw: \
        Canvas(my_window, width=100, height=100, bg='white', *args, **kw)

OK, partial is a little shorter in that case.  I'm not sure that
example justifies a PEP, though.

    def _merge(d1, d2):
        d1.update(d2)
        return d1

    def partial(f, *args, **kw):
         return lambda *args2, **kw2: f(args + args2, merge(kw2,kw))

??

>>        "And I want the possibility of useful introspection and
>>        subclassing"
>>
>>Can you elaborate on that, please?
>>
>>  
>>
> You could maybe sub-class partial to override __call__ , in case you
> want to do anything fancy like pre-supplying arguments at arbitrary
> positions.

A *perfect* example of what lambda is already good at.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From mcherm at mcherm.com  Mon Apr  5 10:03:46 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Apr  5 10:03:43 2004
Subject: [Python-Dev] PEP 318: Let's propose some useful built-in
	decorators
Message-ID: <1081173826.40716742a1265@mcherm.com>

Guido writes:
> While I ponder the decorator syntax, let's propose some built-in
> decorators.
   [...]
> Proposals for other standard decorators are welcome!

memoized -- Encloses the function in a wrapper which provides automatic
   memoization. Provides two different uses. The simple use is just this:

       def fibonacci(n) [memoized()]:
           if n == 0: return 0
           elif n == 1: return 1
           else return fibonacci(n-1) + fibonacci(n-2)


   ...which automatically memoizes all results. For advanced users who
   want to limit memory use, there's this:


       def fibonacci(n) [memoized(cache_size=25)]:
           [...]

   ... which will save only the 25 most-recently-used values.



Here... I'll try to implement it, since I want to play around with
writing a decorator which takes arguments. I'll see if I can do so in
a highly readable manner:

    # WARNING... Untested code follows!

    def memoized(cache_size=None):
        if cache_size is None:
            # No cache size limits, so just store the values in a hash
            def memoize_decorator(f):
                memo_cache = {}
                def wrapper_function(*args, **kwargs):
                    if memo_cache.has_key((args, kwargs)):
                        return memo_cache[(args, kwargs)]
                    else:
                        value = f(*args, **kwargs)
                        memo_cache[(args, kwargs)] = value
                        elif not memo_cache_full:
                            
                        return value
                return wrapper_function

        else:
            # cache_size is given, so we need a LRU cache.
            #   This currently uses a dirt-simple LRU cache... a list
            #   of (key, value) pairs with new items added to the end.
            #   Performance could be improved with a fancier data
            #   structure.
            def memoize_decorator(f):
                memo_cache = []
                def wrapper_function(*args, **kwargs):
                    for key, value in memo_cache:
                        if key == (args, kwargs):
                            # Found it in the cache! Move to end of list.
                            memo_cache.remove( ((args, kwargs), value) )
                            memo_cache.append( ((args, kwargs), value) )
                    # Not found in cache... must calculate
                    value = f(*args, **kwargs)
                    memo_cache.append( ((args, kwargs), value) )
                    if len(memo_cache) > cache_size:
                        memo_cache.pop(0)
                    return value
                return wrapper_function

        return memoize_decorator


Hmm.... somewhat awkward, but readable. Nested function definitions and
nested scoping rules make this much easier than otherwise. Nevertheless,
a class-based approach might have worked better.

-- Michael Chermside


From tree at basistech.com  Mon Apr  5 10:36:26 2004
From: tree at basistech.com (Tom Emerson)
Date: Mon Apr  5 10:41:09 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
Message-ID: <16497.28394.944929.536518@magrathea.basistech.com>

Moore, Paul writes:
> Hmm. I'm torn. On the one hand, it's really cool that Python+Psyco can
> out-perform C (another argument against the knee-jerk "recode bits in
> C" reaction). But my gut feel is that not enough people use Psyco to
> make this a good answer yet. On the third hand, if we recode chunks of
> the stdlib in C, do we kill too many chances for Psyco to work its
> magic?

If I understand correctly, the issue is that when stdlib functions are
rewritten in C, they end up being faster but are still slower than
Psyco optimized code because they are still polymorphic while the
Psyco'ized generates multiple paths.

If that is the case then one could certainly hand-hack the C
implementations to expand common code paths in an optimal way.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Software Architect                                 http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"

From ark-mlist at att.net  Mon Apr  5 10:42:40 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Mon Apr  5 10:42:29 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1081162471.3004.4.camel@localhost>
Message-ID: <003901c41b1c$3fd9f760$6402a8c0@arkdesktop>

[I've mentioned this idea before, but don't recall seeing any responses to
it; so I'm mentioning it again in case it got lost the first time]

I am concerned about putting decorators before the function definition
because of the possibility of such code quietly sneaking through existing
compilers:

	[classmethod]
	def foo(bar, baz):
		pass

Guido is concerned about putting decorators after the function definition
because there may be a lot of them and it may be hard to read:

	def foo(bar, baz) \
		[staticmethod, classmethod, otherattributes(debug=True}) }:
	    pass

Here's what I don't understand.

I imagine that most of the time, when someone decorates a function with lots
of attributes, there is at least the possibility of decorating more than one
function with the same set of attributes.

In such a case, doesn't it make sense to bind a variable to the attributes
and use it instead?

	Attributes = [staticmethod, classmethod,
otherattributes(debug=True)]

	def foo(bar, baz) Attributes:
		pass

And doesn't this idea answer the objection that too many attributes after
the parameter list make the definition hard to read?

For that matter, why do we need the brackets if there is only one attribute:

	def foo(bar, baz) staticmethod:
		pass

I am suggesting that what comes between the ) and the : should be an
expression, which must evaluate to either a callable or a sequence of
callables.  For that matter, why not allow a tuple expression without
parentheses:

	def foo(bar, baz) staticmethod, classmethod:
		pass

Whatever sequence you put there, I think the semantics are clear:  Before
binding a name to the function, pass it to the callable or in turn to each
element of the sequence.


From pje at telecommunity.com  Mon Apr  5 11:18:34 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr  5 11:18:54 2004
Subject: [Python-Dev] PEP 318: Properties
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D43B@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.1.6.0.20040405111547.0242bc50@telecommunity.com>

At 09:20 AM 4/5/04 -0400, Jewett, Jim J wrote:

>         x=asdlfjk
>
>looks like x is now set to something.  I expect it it to stay that way until
>the next x=??? statement.

Or the next 'def x' statement, or 'class x' statement.  Assignment is not 
the only way to bind a name in Python.

Even if you know nothing about decorators, it should be apparent that in 
today's Python, this:

x = 42

def x(): pass

does *not* leave x equal to 42.


From jim.jewett at eds.com  Mon Apr  5 11:40:14 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Apr  5 11:41:07 2004
Subject: [Python-Dev] The Need for a Declarative Syntax Element (aka
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D43C@USAHM010.amer.corp.eds.com>

Mike Pall:

>	It is unclear whether we have any use case that 
>	requires storing a different name ...

When adapting another module or library, it is often 
desirable to expose both the original name (for people
familiar with the original) and a cleaned up name (so
that your module can be used with internally consistent 
naming).

The current syntax is to bind a second name to the 
object after it is defined; that second binding could 
reasonably move into a decorator.  Ensuring that it 
points to a final-enough version of the object is
probably the programmer's responsibility.

>	Using __declare__

[See the original for more detail - basically, decorators
 must have a __declare__ slot which will be called.]

I think this is the first non-arbitrary restriction we have
seen on decorators.  It does largely solve the old-code-
might-run-differently issue, though not the old-interpreters-
might-silently-run-code-undecorated issue.

Your suggested markers of [| |] or <| |> also solve that,
as would a new keyword.  A new word has the advantage of
stating explicitly what it means, which allows additional
extensions (some of which you already suggested).

Defining a decorator API also allows more freedom for the
decorators; the __declare__ API could accept more than 
just the target object.  For example, it could take keywords 
giving the dictionary where the object will eventually be 
defined, or the name that will be used.  (These would reduce 
the need for _getframe.)

-jJ

From shane at zope.com  Mon Apr  5 11:50:57 2004
From: shane at zope.com (Shane Hathaway)
Date: Mon Apr  5 11:51:00 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1081156611.2377.6.camel@localhost>
References: <200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
Message-ID: <40718061.2040607@zope.com>

Mark Russell wrote:
> What about:
> 
> 	[as classmethod]
> 	def foo(bar, baz):
> 	    pass
> 
> To me this is more obvious (and less like p*rl) that *[classmethod], and
> it is also currently a syntax error so won't break existing code.

I agree, it's nice.  You could even use this syntax at the interactive 
prompt.  The "is" keyword is another choice, too.

Shane

From barry at python.org  Mon Apr  5 12:00:56 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr  5 12:01:24 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <40718061.2040607@zope.com>
References: <200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>  <40718061.2040607@zope.com>
Message-ID: <1081180856.4556.33.camel@anthem.wooz.org>

On Mon, 2004-04-05 at 11:50, Shane Hathaway wrote:
> Mark Russell wrote:
> > What about:
> > 
> > 	[as classmethod]
> > 	def foo(bar, baz):
> > 	    pass
> > 
> > To me this is more obvious (and less like p*rl) that *[classmethod], and
> > it is also currently a syntax error so won't break existing code.
> 
> I agree, it's nice.  You could even use this syntax at the interactive 
> prompt.  The "is" keyword is another choice, too.

Since that is illegal syntax today, it would alleviate my primary
concern with decorator-before-def syntax.  And the 'as' keyword does
help tie it to the following def.

-Barry



From aahz at pythoncraft.com  Mon Apr  5 12:03:09 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 12:07:40 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <4071059E.2060807@ieee.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <20040403145050.GA8626@panix.com>
	<200404031657.i33Gvxl24044@guido.python.org>
	<4071059E.2060807@ieee.org>
Message-ID: <20040405160309.GA10904@panix.com>

On Mon, Apr 05, 2004, Shane Holloway (IEEE) wrote:
>>Aahz:
>>>
>>>Good point.  Here's what I *think* the semantics are; as soon as I get
>>>agreement, I'll update the PEP.  Given a package layout::
>>>
>>>    package
>>>        subpackage1
>>>            moduleX
>>>            moduleY
>>>        subpackage2
>>>            moduleZ
>>>        moduleA
>>>
>>>Each leading "." refers to one level of parent.  Assuming that the
>>>current file is ``moduleX.py``, following are correct usages of the new
>>>syntax::
>>>
>>>    from .moduleY import spam
>>>    from ..subpackage1 import moduleY
>>>    from ..subpackage2.moduleZ import eggs
>>>    from ..moduleA import foo
>>>    from ...package import bar
> 
> What about names inside package __init__?
> 
>     from .. import bar
> 
> Is this also valid?

Do you mean subpackage1's __init__.py?  If so, yes.  For most purposes,
a package's __init__.py is treated as if it were a module with the name
of the package.

Hmmmm...  I think I see what you're getting at: how do you access
subpackage1 from package's __init__.py using relative imports?  Seems to
me that you'd have to do ::

    from .package import subpackage1

Normally, though, subpackages import from their parents, not the other
way around.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From hernan at orgmf.com.ar  Mon Apr  5 12:09:23 2004
From: hernan at orgmf.com.ar (Hernan Foffani)
Date: Mon Apr  5 12:10:27 2004
Subject: [Python-Dev]  PEP 318: Decorators last before colon
In-Reply-To: <E1BAMUq-0005AF-Bp@mail.python.org>
Message-ID: <LNEGJOMPJEMNJDOPCDMLKEMMCDAA.hernan@orgmf.com.ar>

> ... The * prefix looks so arbitrary: why not /, why not @, etc...

It's curious that '@' was brought.  I bet it won't be included
in the proposal but it should be mentioned that there's a notorious
corner use case using a symetrical syntax with a prefix-postfix pair
of characters.
With proper semantics and honoring lowercase plus underscore name
style, it may look like this:

    o_o = [wow, it, reminds, me, ol, uncle, Pep]
    class Giuseppe(object):


        @[o_o]@


        def pepper_peps_up_the_pep_thread(self):
            pass      # heh... where's the "pess" reserved werd?

(BIG :-D if it wasn't clear enough)
Now seriously...
Don't know if it was mentioned before or even if it's relevant but
I strongly believe that most .NET programmers will understand its
meaning (even no-C# ones) and also java ones will (they might
not reckon it though ;-) yet still considering the fact that
python's decorators will be more powerful.

FWIW:  For reasons already given in previous messages I *do*
prefer decorator-before-def-plus-star syntax (prefix only ;-)

Regards,
-Hernan

do i send this or not? yes, no, yes, no, yes...


From gmccaughan at synaptics-uk.com  Mon Apr  5 12:10:42 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Mon Apr  5 12:10:48 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <003901c41b1c$3fd9f760$6402a8c0@arkdesktop>
References: <003901c41b1c$3fd9f760$6402a8c0@arkdesktop>
Message-ID: <200404051710.42704.gmccaughan@synaptics-uk.com>

On Monday 2004-04-05 at 15:42, Andrew Koenig wrote:

> Here's what I don't understand.
> 
> I imagine that most of the time, when someone decorates a function with lots
> of attributes, there is at least the possibility of decorating more than one
> function with the same set of attributes.
> 
> In such a case, doesn't it make sense to bind a variable to the attributes
> and use it instead?
> 
> 	Attributes = [staticmethod, classmethod,
> otherattributes(debug=True)]
> 
> 	def foo(bar, baz) Attributes:
> 		pass
> 
> And doesn't this idea answer the objection that too many attributes after
> the parameter list make the definition hard to read?

It certainly answers that objection, but it has (so it seems to me)
other problems of its own; see below.

> For that matter, why do we need the brackets if there is only one attribute:
> 
> 	def foo(bar, baz) staticmethod:
> 		pass
> 
> I am suggesting that what comes between the ) and the : should be an
> expression, which must evaluate to either a callable or a sequence of
> callables.  For that matter, why not allow a tuple expression without
> parentheses:
> 
> 	def foo(bar, baz) staticmethod, classmethod:
> 		pass
> 
> Whatever sequence you put there, I think the semantics are clear:  Before
> binding a name to the function, pass it to the callable or in turn to each
> element of the sequence.

The brackets serve multiple purposes.

  - They make the decoration *look*, even to novice eyes, like
    an annotation: "by the way, please make this a static method".

  - They may possibly help the parser a little. (I'm showing
    my ignorance here; I haven't really looked at the Python
    parser at all.)

  - They give a hint to the user that what goes in between them
    may be a sequence, not just a single item.

  - They ensure that all decorated definitions have a somewhat
    consistent appearance.

Omitting them breaks all these things, especially if an arbitrary
expression is allowed there. And do we really want to allow

    def foo(bar, baz) quux(wibble, spong):
        pass

That's not a hideous pathological case; it's what a simple
decoration using a parameterized decorator will look like
without the brackets. With good decorator names I suppose
it becomes somewhat comprehensible:

    def foo(bar, baz) memoized(25):
        pass

but surely it's still much clearer when written as

    def foo(bar, baz) [memoized(25)]:
        pass

or even (though I'm a little tempted to agree with whoever it was
that was wondering whether Guido has been abducted by aliens)

    [memoized(25)]
    def foo(bar, baz):
        pass

We can keep the syntactic distinctiveness while still allowing
multiple decorators to be combined, by having a function
(a builtin, perhaps, but it's not hard to write) that
composes decorators:

    def compose(*decorators):
        def _(f):
            for d in decorators: f = d(f)
            return f

    our_attributes = compose(staticmethod, classmethod,
      otherattributes(debug=True))

    def foo(bar, baz) [our_attributes]:
        pass

-- 
Gareth McCaughan



From aahz at pythoncraft.com  Mon Apr  5 12:11:09 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 12:11:33 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <s9zn05qx0aj.fsf@salmakis.intevation.de>
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>
	<20040403171247.GC8803@panix.com>
	<200404032125.i33LPU024587@guido.python.org>
	<20040404021800.GA25210@panix.com>
	<s9zn05qx0aj.fsf@salmakis.intevation.de>
Message-ID: <20040405161109.GB10904@panix.com>

On Mon, Apr 05, 2004, Bernhard Herzog wrote:
> Aahz <aahz@pythoncraft.com> writes:
>>
>> You may use relative imports freely.  In Python 2.5, any ``import``
>> statement that results in an intra-package import will generate a
>> ``PendingDeprecation`` warning (this also applies to ``from <> import``
>> that fails to use the relative import syntax).  In Python 2.6, ``import``
>> will always be an absolute import.
> 
> Why PendingDeprecation?  If the import semantics change in incompatible
> ways in 2.6, 2.5 should generate a "normal" DeprecationWarning, I think.

Guido can overrule this, but my take is that there will be few enough
cases where this is a significant issue.  People who want to write
cross-version libraries and applications shouldn't have to deal with
suppressing the DeprecationWarning.  If we were serious about issuing a
DeprecationWarning, I think that would need to be in Python 2.6, pushing
the default of absolute imports to Python 2.7.  That seems too long a
timeline to me.

Other opinions?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From guido at python.org  Mon Apr  5 12:43:30 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 12:43:38 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Mon, 05 Apr 2004 01:17:17 PDT."
	<4071160D.5060204@prescod.net> 
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>  
	<4071160D.5060204@prescod.net> 
Message-ID: <200404051643.i35GhU005203@guido.python.org>

> What people find readable depends mostly on what they are used to. The 
> reason Python is "so readable" on first read is because it rips off so 
> much syntax directly from C, the language that is the ancestor for all 
> of the other languages people use today.

Bizarre.  I think most of the readability actually comes from places
where I *didn't* rip off C, in particular Python's syntax for block
structure, lists and dicts, and the lack of anything resembling C's
abomination passing for type declarations (an mistake admitted by its
author).

Just goes to show that taste differs. :)

> C doesn't have function decorators so people don't have expectations
> for them. But they will when they start coming over from C# and Java
> 1.5 in two or three years. One virtue of Guido's proposal is that it
> is basically what C# does. Java uses a pretty different syntax but
> it is also a prefix syntax. If Python uses a postfix syntax it will
> probably be alone in making that choice.

That's exactly Jim Hugunin's argument for this syntax.

> I'm not saying that Python has to do what the other languages do
> because they do it, but all else equal, being familiar is better
> than being idiosyncratic (i.e. different for no good reason). So I
> would rank "like C# and Java" higher than "fits my personal
> aesthetics in early 2004" because aesthetics are likely to drift
> towards C# and Java over time.

Anyway, personal aesthetics are learned behavior, like it or not.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr  5 12:44:20 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 12:44:28 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Mon, 05 Apr 2004 10:16:51 BST."
	<1081156611.2377.6.camel@localhost> 
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org> 
	<1081156611.2377.6.camel@localhost> 
Message-ID: <200404051644.i35GiKD05216@guido.python.org>

> What about:
> 
> 	[as classmethod]
> 	def foo(bar, baz):
> 	    pass
> 
> To me this is more obvious (and less like p*rl) that *[classmethod], and
> it is also currently a syntax error so won't break existing code.

I appreciate the suggestion, but the current parser can't handle that.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Mon Apr  5 12:49:53 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Mon Apr  5 12:49:41 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1081180856.4556.33.camel@anthem.wooz.org>
Message-ID: <000101c41b2e$0519f7d0$6402a8c0@arkdesktop>

> > > What about:
> > >
> > > 	[as classmethod]
> > > 	def foo(bar, baz):
> > > 	    pass

> Since that is illegal syntax today, it would alleviate my primary
> concern with decorator-before-def syntax.  And the 'as' keyword does
> help tie it to the following def.

Indeed.  And we could even extend it to allow a single set of decorators to
apply to multiple function definitions, as follows:

	[as classmethod]:

		def foo(bar, baz):
			pass

		def bar(foo, baz):
			pass

Of course, the colon would be required if and only if an indent follows it.

(I still prefer decorators after the parameters, but this version pretty
much solves the same problems and answers my non-aesthetic objections)


From skip at pobox.com  Mon Apr  5 12:58:22 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Apr  5 12:58:43 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404051643.i35GhU005203@guido.python.org>
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
	<4071160D.5060204@prescod.net>
	<200404051643.i35GhU005203@guido.python.org>
Message-ID: <16497.36910.88010.318537@montanaro.dyndns.org>


    >> One virtue of Guido's proposal is that it is basically what C#
    >> does. Java uses a pretty different syntax but it is also a prefix
    >> syntax. If Python uses a postfix syntax it will probably be alone in
    >> making that choice.

    Guido> That's exactly Jim Hugunin's argument for this syntax.

Since both C# and Java differ significantly in their typing from Python I
suspect what works well for those languages (they already have a lot of
declarative "baggage" because of their compile-time type checks, so what's a
few more declarations?) may not work as well for Python.

Second, Python has a strong tradition of borrowing what"works well" from
other languages.  I'm skeptical that C# has been around long enough to
suggest that its syntax "works well".  It's pretty clear that Microsoft is
going to ram that down most programmers' throats, so the C# user base is no
doubt going to be very large in a year or two.  If what you're looking for
is to provide a familiar syntactic base for C# refugees, then I suppose
that's fine, but from the examples I've seen, C# decorations (annotations?
attributes?) can be sprinkled quite liberally through the code (and serve as
much to obscure as to highlight what's going on).  A C# refugee might be
disappointed to see that Python's decorators are limited to class, function
and method declarations.  With that in mind, it doesn't seem to me that
partially mimicing C#'s decorator system is necessarily a good thing.

Skip

From guido at python.org  Mon Apr  5 12:59:13 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 12:59:22 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: Your message of "Mon, 05 Apr 2004 14:23:16 BST."
	<20040405132316.GA21940@vicky.ecs.soton.ac.uk> 
References: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
	<20040405132316.GA21940@vicky.ecs.soton.ac.uk> 
Message-ID: <200404051659.i35GxDN05308@guido.python.org>

> Essentially, I wanted to rant about the fact that some performance
> considerations are driving the language's evolution, and this is
> much harder to revert to if later the performance reasons vanish.

I have to admit that sometimes I too yearn for the days of lists and
dicts, when heapq was implemented in Python.

But iterators are here to stay, and generator expressions are coming,
so we'd better get used to them, and know how to deal with them.

And yes, we'll see lots of code using genexps "for performance" while
lists would perform just as well given the actual data set sizes.

But that's okay, since genexps are a pretty nice way of saying things
regardless of performance!  And they are rarely if ever *slower* than
lists, so we shouldn't fear their performance, but rather use whatever
gives the most readable solution.

Regarding heapq: once Psyco is part of the standard library, it's easy
enough to revive heapq.py and friends.  In the mean time, I have no
problem with providing practical performance for regular Python users.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From marktrussell at btopenworld.com  Mon Apr  5 13:00:34 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Mon Apr  5 13:00:41 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404051644.i35GiKD05216@guido.python.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<200404051644.i35GiKD05216@guido.python.org>
Message-ID: <1081184434.4045.3.camel@localhost>

On Mon, 2004-04-05 at 17:44, Guido van Rossum wrote:
> I appreciate the suggestion, but the current parser can't handle that.

Would it not be possible to cheat and make the lexer transform 

	"[" WHITESPACE "as" WHITESPACE IDENTIFIER 

into an LBRACE-AS token (which I assume would make the parser's job
simple).

Mark

From shane.holloway at ieee.org  Mon Apr  5 13:19:48 2004
From: shane.holloway at ieee.org (Shane Holloway (IEEE))
Date: Mon Apr  5 13:20:15 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040405160309.GA10904@panix.com>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <20040403145050.GA8626@panix.com>
	<200404031657.i33Gvxl24044@guido.python.org>
	<4071059E.2060807@ieee.org> <20040405160309.GA10904@panix.com>
Message-ID: <40719534.2040301@ieee.org>



Aahz wrote:

> On Mon, Apr 05, 2004, Shane Holloway (IEEE) wrote:
> 
>>>Aahz:
>>>
>>>>Good point.  Here's what I *think* the semantics are; as soon as I get
>>>>agreement, I'll update the PEP.  Given a package layout::
>>>>
>>>>   package
>>>>       subpackage1
>>>>           moduleX
>>>>           moduleY
>>>>       subpackage2
>>>>           moduleZ
>>>>       moduleA
>>>>
>>>>Each leading "." refers to one level of parent.  Assuming that the
>>>>current file is ``moduleX.py``, following are correct usages of the new
>>>>syntax::
>>>>
>>>>   from .moduleY import spam
>>>>   from ..subpackage1 import moduleY
>>>>   from ..subpackage2.moduleZ import eggs
>>>>   from ..moduleA import foo
>>>>   from ...package import bar
>>
>>What about names inside package __init__?
>>
>>    from .. import bar
>>
>>Is this also valid?
> 
> 
> Do you mean subpackage1's __init__.py?  If so, yes.  For most purposes,
> a package's __init__.py is treated as if it were a module with the name
> of the package.

Actually, no I meant package's __init__.py -- so that::

     from .. import bar as barA
     from ...package import bar as barB

     assert barA is barB

> Hmmmm...  I think I see what you're getting at: how do you access
> subpackage1 from package's __init__.py using relative imports?  Seems to
> me that you'd have to do ::
> 
>     from .package import subpackage1
> 
> Normally, though, subpackages import from their parents, not the other
> way around.

I would intuit that one would import subpackage1 from package as::

     from . import subpackage1

but that's not explicitly spelled out.  Is this correct?



From aahz at pythoncraft.com  Mon Apr  5 13:21:03 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 13:21:34 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <Pine.LNX.4.58.0404050114120.18028@server1.LFW.org>
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
	<Pine.LNX.4.58.0404050114120.18028@server1.LFW.org>
Message-ID: <20040405172103.GB11824@panix.com>

On Mon, Apr 05, 2004, Ka-Ping Yee wrote:
> On Mon, 5 Apr 2004, Greg Ewing wrote:
>>
>> the little voice in my head doesn't really say anything
>> coherent at all.
> 
> Funny.  I experience that most days.

"I do what the voices in my head tell me to."
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From aahz at pythoncraft.com  Mon Apr  5 13:27:19 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 13:27:32 2004
Subject: [Python-Dev] PEP 318: Python's readability
In-Reply-To: <4071160D.5060204@prescod.net>
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
	<4071160D.5060204@prescod.net>
Message-ID: <20040405172719.GC11824@panix.com>

On Mon, Apr 05, 2004, Paul Prescod wrote:
>
> What people find readable depends mostly on what they are used to. The 
> reason Python is "so readable" on first read is because it rips off so 
> much syntax directly from C, the language that is the ancestor for all 
> of the other languages people use today.

Speaking as someone whose first languages were BASIC and Pascal, both
before C became so hugely popular, my feelings about Python's readability
were more along the lines of it being different from C.  Yes, I knew
enough C to recognize where it was being ripped off -- but that's a far
cry from the elegance of the language as a whole.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From aahz at pythoncraft.com  Mon Apr  5 13:36:24 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 13:36:30 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <40719534.2040301@ieee.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <20040403145050.GA8626@panix.com>
	<200404031657.i33Gvxl24044@guido.python.org>
	<4071059E.2060807@ieee.org> <20040405160309.GA10904@panix.com>
	<40719534.2040301@ieee.org>
Message-ID: <20040405173624.GD11824@panix.com>

On Mon, Apr 05, 2004, Shane Holloway (IEEE) wrote:
> Aahz wrote:
>>On Mon, Apr 05, 2004, Shane Holloway (IEEE) wrote:
>>>>Aahz:
>>>>
>>>>>Good point.  Here's what I *think* the semantics are; as soon as I get
>>>>>agreement, I'll update the PEP.  Given a package layout::
>>>>>
>>>>>  package
>>>>>      subpackage1
>>>>>          moduleX
>>>>>          moduleY
>>>>>      subpackage2
>>>>>          moduleZ
>>>>>      moduleA
>>>>>
>>>>>Each leading "." refers to one level of parent.  Assuming that the
>>>>>current file is ``moduleX.py``, following are correct usages of the new
>>>>>syntax::
>>>>>
>>>>>  from .moduleY import spam
>>>>>  from ..subpackage1 import moduleY
>>>>>  from ..subpackage2.moduleZ import eggs
>>>>>  from ..moduleA import foo
>>>>>  from ...package import bar
>>>
>>>What about names inside package __init__?
>>>
>>>   from .. import bar
>>>
>>>Is this also valid?
>>
>>
>>Do you mean subpackage1's __init__.py?  If so, yes.  For most purposes,
>>a package's __init__.py is treated as if it were a module with the name
>>of the package.
> 
> Actually, no I meant package's __init__.py -- so that::
> 
>     from .. import bar as barA
>     from ...package import bar as barB
> 
>     assert barA is barB

That makes no sense at all; both of those imports are wrong from
package's __init__.py.

>>Hmmmm...  I think I see what you're getting at: how do you access
>>subpackage1 from package's __init__.py using relative imports?  Seems to
>>me that you'd have to do ::
>>
>>    from .package import subpackage1
>>
>>Normally, though, subpackages import from their parents, not the other
>>way around.
> 
> I would intuit that one would import subpackage1 from package as::
> 
>     from . import subpackage1
> 
> but that's not explicitly spelled out.  Is this correct?

Your intuition is wrong.  From package's __init__.py, ``from .`` refers
to package's parent; to get back into the package, you need to use
``from .package``.  It's precisely the same as the example above with
moduleX getting ``bar`` -- but because moduleX is two levels down, it
needs two more dots.  You're getting confused because you think of
__init__.py as being a level below the package that contains it;
however, __init__.py *IS* the package level.  It's the code *for* the
package.

If anyone else has been confused by this, please let me know and I'll
add clarification to the PEP, but I think these consequences are pretty
clear if you understand how packages work.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From nas-python at python.ca  Mon Apr  5 13:51:17 2004
From: nas-python at python.ca (Neil Schemenauer)
Date: Mon Apr  5 13:51:30 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040405173624.GD11824@panix.com>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <20040403145050.GA8626@panix.com>
	<200404031657.i33Gvxl24044@guido.python.org>
	<4071059E.2060807@ieee.org> <20040405160309.GA10904@panix.com>
	<40719534.2040301@ieee.org> <20040405173624.GD11824@panix.com>
Message-ID: <20040405175117.GA21587@mems-exchange.org>

On Mon, Apr 05, 2004 at 01:36:24PM -0400, Aahz wrote:
> Your intuition is wrong.  From package's __init__.py, ``from .`` refers
> to package's parent; to get back into the package, you need to use
> ``from .package``.

That's confusing.  I expected '.' to refer to the current package,
just as '.' refers to the current directory in most file systems.  I
think that meaning would be more useful in practice as well.
Imagine a package where '.' means the current package:

    # __init__.py ###################################
    from .a import A
    from .b import B
    #################################################


    # a.py ##########################################
    from .utils import x, y

    class A:
        ...
    #################################################


    # b.py ##########################################
    from .utils import x, z

    class B:
        ...
    #################################################

    # utils.py ######################################
    def x():
        ...
    def y():
        ...
    def z():
        ...
    #################################################


Notice that the modules that make up the package do not need to know
what the package is called.  That was one of the major benefits of
relative imports that I was looking forward to.

Perhaps I misunderstand the PEP.

  Neil

From guido at python.org  Mon Apr  5 13:58:21 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 13:58:29 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Mon, 05 Apr 2004 12:11:09 EDT."
	<20040405161109.GB10904@panix.com> 
References: <E1B9o04-0007ZI-Tu@mail.python.org> <406EED7A.5090908@Acm.Org>
	<20040403171247.GC8803@panix.com>
	<200404032125.i33LPU024587@guido.python.org>
	<20040404021800.GA25210@panix.com>
	<s9zn05qx0aj.fsf@salmakis.intevation.de> 
	<20040405161109.GB10904@panix.com> 
Message-ID: <200404051758.i35HwLK05504@guido.python.org>

> > Why PendingDeprecation?  If the import semantics change in incompatible
> > ways in 2.6, 2.5 should generate a "normal" DeprecationWarning, I think.
> 
> Guido can overrule this, but my take is that there will be few enough
> cases where this is a significant issue.  People who want to write
> cross-version libraries and applications shouldn't have to deal with
> suppressing the DeprecationWarning.  If we were serious about issuing a
> DeprecationWarning, I think that would need to be in Python 2.6, pushing
> the default of absolute imports to Python 2.7.  That seems too long a
> timeline to me.

I think we ought to start it as a normal Deprecation.  If there are
enough people who get bit by it during alpha testing we can switch it
to PendingDeprecation, but IMO PendingDeprecation is not very useful
(since it is ignored by default).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr  5 14:03:09 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 14:03:14 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Mon, 05 Apr 2004 18:00:34 BST."
	<1081184434.4045.3.camel@localhost> 
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<200404051644.i35GiKD05216@guido.python.org> 
	<1081184434.4045.3.camel@localhost> 
Message-ID: <200404051803.i35I39U05548@guido.python.org>

> > I appreciate the suggestion, but the current parser can't handle that.
> 
> Would it not be possible to cheat and make the lexer transform 
> 
> 	"[" WHITESPACE "as" WHITESPACE IDENTIFIER 
> 
> into an LBRACE-AS token (which I assume would make the parser's job
> simple).

I don't think without extreme hackery, but feel free to prove me wrong
by producing a patch.  (A problem is, there could be newlines and
comments inside the WHITESPACE.  Backtracking over that would require
the lexer to have an arbitrary-length buffer.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pinard at iro.umontreal.ca  Mon Apr  5 14:06:41 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Mon Apr  5 14:07:51 2004
Subject: [Python-Dev] PEP 328: __path__
In-Reply-To: <200404050516.i355GQes002218@cosc353.cosc.canterbury.ac.nz>
References: <20040404162100.GA6059@panix.com>
	<200404050516.i355GQes002218@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040405180641.GA5876@alcyon.progiciels-bpi.ca>

[Greg Ewing]
> Aahz <aahz@pythoncraft.com>:

> > Between the addition of ``as`` in Python 2.0 and the new syntax &
> > semantics for ``import``, there is no longer a need for ``__path__``, and
> > it is now officially deprecated.

> NO! WAIT!

> I have a project in which I'm modifying a module's __path__,
> and it's *not* to work around any deficiencies in the relative
> import mechanism. Rather it's to make a set of directories
> behave like a single package for import purposes, with the
> members of the set determined at run time.

Same here.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard

From tim.one at comcast.net  Mon Apr  5 14:45:51 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon Apr  5 14:46:04 2004
Subject: [Python-Dev] genexps slow?
In-Reply-To: <2mptamg08o.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEJPKAAB.tim.one@comcast.net>

[Tim Peters]
>> There's a twist there:  the functions synthesized for generator
>> expressions get to take the quicker-than-general "fast_yield" exit
>> path. Hmm!  It looks like tests against WHY_YIELD before the
>> fast_yield label can no longer succeed, but are still there to slow
>> down non-generator cases.

[Michael Hudson]
> Hmm.  I *think* I agree with you.  I also think the tests against
> WHY_YIELD in the code for END_FINALLY are unnecessary,

I meant that one too:  all tests against WHY_YIELD physically preceding the
fast_yield label appear unnecessary now.  The only obvious assignment of
WHY_YIELD to "why" appears in YIELD_VALUE now, and that jumps immediately to
fast_yield, from which there's no going backwards.  do_raise() can't return
WHY_YIELD either.

> as we (syntactically) don't allow yields when there are finally:
> statements in play.

Even if we did, a yield doesn't collapse any pending block structure.

>  If I'm wrong about that, then you're wrong too :-) It's all
> a bit subtle, to be sure.

Na <wink>.  I'll toss in some asserts and check it in if nothing blows up.

> Once again, I wonder if there was someway we could have two
> eval_frames: one with tracing support, one without.  Unfortunately, I
> don't think this can fly (how do you *start* tracing?).  Maybe we
> could have a '-notrace' command line flag?  But I suspect that this is
> a silly over-complication.

Me too.


From fdrake at acm.org  Mon Apr  5 14:59:09 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Apr  5 14:59:22 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404051803.i35I39U05548@guido.python.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
Message-ID: <200404051459.10263.fdrake@acm.org>

Mark Russell suggested:
 > Would it not be possible to cheat and make the lexer transform
 >
 > 	"[" WHITESPACE "as" WHITESPACE IDENTIFIER
 >
 > into an LBRACE-AS token (which I assume would make the parser's job
 > simple).

Guido said:
 > I don't think without extreme hackery, but feel free to prove me wrong
 > by producing a patch.  (A problem is, there could be newlines and
 > comments inside the WHITESPACE.  Backtracking over that would require
 > the lexer to have an arbitrary-length buffer.)

Another possibility might be to require the construct start without the 
whitespace:

"[as" WHITESPACE ... "]"

That avoids the buffering issue and matches what most programmer are more 
likely to write anyway.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From aahz at pythoncraft.com  Mon Apr  5 15:01:51 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 15:02:03 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040405175117.GA21587@mems-exchange.org>
References: <200403302218.i2UMIEV10058@guido.python.org>
	<406EB373.2030401@python.org> <20040403145050.GA8626@panix.com>
	<200404031657.i33Gvxl24044@guido.python.org>
	<4071059E.2060807@ieee.org> <20040405160309.GA10904@panix.com>
	<40719534.2040301@ieee.org> <20040405173624.GD11824@panix.com>
	<20040405175117.GA21587@mems-exchange.org>
Message-ID: <20040405190151.GA16139@panix.com>

On Mon, Apr 05, 2004, Neil Schemenauer wrote:
> On Mon, Apr 05, 2004 at 01:36:24PM -0400, Aahz wrote:
>>
>> Your intuition is wrong.  From package's __init__.py, ``from .`` refers
>> to package's parent; to get back into the package, you need to use
>> ``from .package``.
> 
> That's confusing.  I expected '.' to refer to the current package,
> just as '.' refers to the current directory in most file systems.  I
> think that meaning would be more useful in practice as well.
> Imagine a package where '.' means the current package:
> 
>     # __init__.py ###################################
>     from .a import A
>     from .b import B
>     #################################################
> 
>     # a.py ##########################################
>     from .utils import x, y
>     class A:
>         ...
>     #################################################
> 
>     # b.py ##########################################
>     from .utils import x, z
>     class B:
>         ...
>     #################################################
> 
>     # utils.py ######################################
>     def x():
>         ...
>     def y():
>         ...
>     def z():
>         ...
>     #################################################
> 
> Notice that the modules that make up the package do not need to know
> what the package is called.  That was one of the major benefits of
> relative imports that I was looking forward to.
> 
> Perhaps I misunderstand the PEP.

Not sure.  The only difference between your semantics and mine lies in
__init__.py; there -- and only there -- is where the need exists to know
the package name.  You're making the mistake of thinking that
__init__.py exists in the same namespace as a.py, b.py, and utils.py --
it doesn't.  Now, we could in theory use your semantics due to
"practicality beats purity", but it seems to me that it would lead to
more mistakes in understanding how packages work.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From ishnigarrab at earthlink.net  Mon Apr  5 15:37:32 2004
From: ishnigarrab at earthlink.net (Isaac)
Date: Mon Apr  5 15:38:09 2004
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 9, Issue 17
In-Reply-To: <E1BAMUc-0004uL-48@mail.python.org>
References: <E1BAMUc-0004uL-48@mail.python.org>
Message-ID: <4071B57C.60803@earthlink.net>


>From: Guido van Rossum <guido@python.org>
>
[...]

>I also note that accepting decorator-before-colon now would make it
>harder to come up with a decent syntax for declaring the return type,
>which I still want to do in some future version of Python with
>optional (!) static typing.  But not impossible -- there's enough
>punctuation available besides '[' and ':'.
>
[...]

>--Guido van Rossum (home page: http://www.python.org/~guido/)
>

One thing I was thinking about (aloud on #python) was a validator 
syntax, something resembling:

    def is_int(i):
        return isinstance(number, int):

    def is_hashable(h):
        try:
            hash(h)
            return True
        except TypeError:
            return False # or raise exception

    def func(is_int : number, hashable : key):
        # code...

which translates to:

    def func(number, key):
        if isinstance(number, int):
            raise ValueError, "first argument must be int"

        try:
            hash(key)
        except TypeError:
            raise ValueError, "second argument must be hashable"

        # code...
         

I don't know if it would be best to go the boolean or the exception 
route. Either way, you get the point. This way python retains it's 
dynamicy but allows for type checking as well as other form of 
validation (i.e. range checking, etc). Of course, the validator and the 
colon are optional

    'def' NAME '(' [validator ':'] v_name ',' ... ')' ':' suite

or some crap like that, I'm not a language lawyer, but you get the idea. 
And for people concerned with wasting the time with 10 function calls 
per function call, there could be built-in validators (like is_int, or 
whatever you want to call it) which of course are written an C, and 
maybe some shortcutting could be done in the interpreter (or in psyco, 
maybe) which would allow is_int to be called with out all the pythonic 
function calling crap (I'm not sure exactly how this all works...). 
Something like that.

Also as an extension of that, there can a

    def ret_type : func(arg, arg, ...): pass

kinda thing... although the colon might confuse things, there idea is 
there, just needs to be hammered out.

Anyways, just some random thoughts. I'd be happy to work with some 
people to make a PEP out of this, if it gains any traction, just e-mail me.

Isaac

From marktrussell at btopenworld.com  Mon Apr  5 16:08:19 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Mon Apr  5 16:08:24 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404051459.10263.fdrake@acm.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<200404051459.10263.fdrake@acm.org>
Message-ID: <1081195698.2377.9.camel@localhost>

On Mon, 2004-04-05 at 19:59, Fred L. Drake, Jr. wrote:
> Another possibility might be to require the construct start without the 
> whitespace:
> 
> "[as" WHITESPACE ... "]"
> 
> That avoids the buffering issue and matches what most programmer are more 
> likely to write anyway.

I think that's probably too arbitrary a restriction.  Another thought:
presumably the parser could handle this syntax if "as" were a keyword. 
Is that true, and if so, is making "as" a keyword out of the question? 
The change could presumably be enabled by a "from __future__" directive
to avoid breaking existing code.

Mark

From nhodgson at bigpond.net.au  Mon Apr  5 17:31:14 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Mon Apr  5 17:30:19 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
	<4071160D.5060204@prescod.net>
	<200404051643.i35GhU005203@guido.python.org>
	<16497.36910.88010.318537@montanaro.dyndns.org>
Message-ID: <00fc01c41b55$52ed9ad0$84bb8490@neil>

   Skip:

> Second, Python has a strong tradition of borrowing what"works well" from
> other languages.  I'm skeptical that C# has been around long enough to
> suggest that its syntax "works well".

   C# inherited this syntax from Microsoft IDL and its predecessor ODL.
Unsure of the age of MS IDL but a 1995 edition of "Inside OLE" uses it. I
can't recall any problems with the syntax although fewer people would have
worked with it when it was only for defining interfaces rather than as part
of a general purpose language.

   Neil


From guido at python.org  Mon Apr  5 18:09:48 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 18:09:57 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: Your message of "Mon, 05 Apr 2004 21:08:19 BST."
	<1081195698.2377.9.camel@localhost> 
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<200404051459.10263.fdrake@acm.org> 
	<1081195698.2377.9.camel@localhost> 
Message-ID: <200404052209.i35M9m606023@guido.python.org>

> presumably the parser could handle this syntax if "as" were a keyword. 

Not really.  The parser is really dumb, it han only handle one
alternative starting with a particular token at any point.  Since '['
can already start an expression and an expression is a valid
statement, anything else also starting with '[', even if it isn't a
valid expression, cannot be accepted.  You may be thinking of Yacc,
which has much more liberal rules.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jcarlson at uci.edu  Mon Apr  5 18:27:40 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Apr  5 18:31:50 2004
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 9, Issue 17
In-Reply-To: <4071B57C.60803@earthlink.net>
References: <E1BAMUc-0004uL-48@mail.python.org> <4071B57C.60803@earthlink.net>
Message-ID: <20040405152615.EC80.JCARLSON@uci.edu>

> One thing I was thinking about (aloud on #python) was a validator 
> syntax, something resembling:
> 
>     def is_int(i):
>         return isinstance(number, int):
> 
>     def is_hashable(h):
>         try:
>             hash(h)
>             return True
>         except TypeError:
>             return False # or raise exception
> 
>     def func(is_int : number, hashable : key):
>         # code...
> 
> which translates to:
> 
>     def func(number, key):
>         if isinstance(number, int):
>             raise ValueError, "first argument must be int"
> 
>         try:
>             hash(key)
>         except TypeError:
>             raise ValueError, "second argument must be hashable"
> 
>         # code...
>          
> 
> I don't know if it would be best to go the boolean or the exception 
> route. Either way, you get the point. This way python retains it's 
> dynamicy but allows for type checking as well as other form of 
> validation (i.e. range checking, etc). Of course, the validator and the 
> colon are optional
> 
>     'def' NAME '(' [validator ':'] v_name ',' ... ')' ':' suite
> 
> or some crap like that, I'm not a language lawyer, but you get the idea. 
> And for people concerned with wasting the time with 10 function calls 
> per function call, there could be built-in validators (like is_int, or 
> whatever you want to call it) which of course are written an C, and 
> maybe some shortcutting could be done in the interpreter (or in psyco, 
> maybe) which would allow is_int to be called with out all the pythonic 
> function calling crap (I'm not sure exactly how this all works...). 
> Something like that.
> 
> Also as an extension of that, there can a
> 
>     def ret_type : func(arg, arg, ...): pass
> 
> kinda thing... although the colon might confuse things, there idea is 
> there, just needs to be hammered out.
> 
> Anyways, just some random thoughts. I'd be happy to work with some 
> people to make a PEP out of this, if it gains any traction, just e-mail me.

Isaac,

Everything you want with "validators" is going to be available with
decorators from PEP 318.

 - Josiah


From djc at object-craft.com.au  Mon Apr  5 19:06:16 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Mon Apr  5 19:06:23 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <16497.28394.944929.536518@magrathea.basistech.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
	<16497.28394.944929.536518@magrathea.basistech.com>
Message-ID: <1081206375.2666.20.camel@echidna.object-craft.com.au>

On Tue, 2004-04-06 at 00:36, Tom Emerson wrote:
> Moore, Paul writes:
> > Hmm. I'm torn. On the one hand, it's really cool that Python+Psyco can
> > out-perform C (another argument against the knee-jerk "recode bits in
> > C" reaction). But my gut feel is that not enough people use Psyco to
> > make this a good answer yet. On the third hand, if we recode chunks of
> > the stdlib in C, do we kill too many chances for Psyco to work its
> > magic?
> 
> If I understand correctly, the issue is that when stdlib functions are
> rewritten in C, they end up being faster but are still slower than
> Psyco optimized code because they are still polymorphic while the
> Psyco'ized generates multiple paths.
> 
> If that is the case then one could certainly hand-hack the C
> implementations to expand common code paths in an optimal way.

I have not looked at Psyco so could be operating from a misunderstanding
of what it does...

Would it make sense to build specialisation into the base Python
interpreter so that it builds specialised versions of bytecode for
functions.  I am assuming that there are only a few types that can be
targeted by specialisation (int, float).  Bytecodes then could be added
to the interpreter which operate on those native machine types rather
than the Python objects.

This has the advantage of allowing specialisation on all currently
supported platforms and does not preclude the further optimisation of
translation to machine code.

If specialisation was in the base interpreter then I suppose it should
possible for the compiler to know when a native machine type can be
compiled directly.  For example:

def foo():
    i = 0
    while i < 100:
        bar()
        i += 1

While noone writes loops like that, it does illustrate the point.  It is
clear that the local i is always an integer and is not used outside the
function.

- Dave

-- 
http://www.object-craft.com.au


From scav at blueyonder.co.uk  Mon Apr  5 19:29:10 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Mon Apr  5 19:22:02 2004
Subject: [Python-Dev] Re: PEP309 re-written
In-Reply-To: <E1BAWY5-0006yl-LZ@mail.python.org>
References: <E1BAWY5-0006yl-LZ@mail.python.org>
Message-ID: <4071EBC6.4010103@blueyonder.co.uk>

David Abrams wrote:

>>    
>>
>C = lambda *args, **kw: \
>        Canvas(my_window, width=100, height=100, bg='white', *args, **kw)
>
>  
>
C(bg='red')

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in -toplevel-
    C(bg='red')
  File "<pyshell#3>", line 1, in <lambda>
    C = lambda *args, **kw: \
TypeError: Canvas constructor got multiple values for keyword argument 'bg'

>OK, partial is a little shorter in that case.  I'm not sure that
>example justifies a PEP, though.
>
>  
>
PEPs can be for the little things too.

>    def _merge(d1, d2):
>        d1.update(d2)
>        return d1
>
>    def partial(f, *args, **kw):
>         return lambda *args2, **kw2: f(args + args2, merge(kw2,kw))
>
>??
>  
>
def partial(f, *args, **kw):
    def _merge(d1, d2):
        d1.update(d2)
        return d1
    return lambda *args2, **kw2: f(*(args + args2), **_merge(kw2,kw))


A good implementation indeed.   I may be inclined to provide both a 
class called Partial (for people who want to mess with the internals, or 
for some other reason want a class) and a function called partial.

Would that be OK, everyone?

>>You could maybe sub-class partial to override __call__ , in case you
>>want to do anything fancy like pre-supplying arguments at arbitrary
>>positions.
>>    
>>
>
>A *perfect* example of what lambda is already good at.
>  
>
Yup, no argument from me there.  I can't foresee what others may one day 
want to do, however.

Peter Harris

From eppstein at ics.uci.edu  Mon Apr  5 19:24:14 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Mon Apr  5 19:24:25 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<200404051459.10263.fdrake@acm.org>
	<1081195698.2377.9.camel@localhost>
	<200404052209.i35M9m606023@guido.python.org>
Message-ID: <eppstein-B1A289.16241405042004@sea.gmane.org>

In article <200404052209.i35M9m606023@guido.python.org>,
 Guido van Rossum <guido@python.org> wrote:

> > presumably the parser could handle this syntax if "as" were a keyword. 
> 
> Not really.  The parser is really dumb, it han only handle one
> alternative starting with a particular token at any point.  Since '['
> can already start an expression and an expression is a valid
> statement, anything else also starting with '[', even if it isn't a
> valid expression, cannot be accepted.  You may be thinking of Yacc,
> which has much more liberal rules.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

Ok, then how about
    <decorator>
    def ...
?

'<' can't start an expression or statement currently, can it?

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From guido at python.org  Mon Apr  5 19:30:23 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr  5 19:30:30 2004
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 9, Issue 17
In-Reply-To: Your message of "Mon, 05 Apr 2004 15:37:32 EDT."
	<4071B57C.60803@earthlink.net> 
References: <E1BAMUc-0004uL-48@mail.python.org>  
	<4071B57C.60803@earthlink.net> 
Message-ID: <200404052330.i35NUND06228@guido.python.org>

> One thing I was thinking about (aloud on #python) was a validator 
> syntax, something resembling:
> 
>     def is_int(i):
>         return isinstance(number, int):
> 
>     def is_hashable(h):
>         try:
>             hash(h)
>             return True
>         except TypeError:
>             return False # or raise exception
> 
>     def func(is_int : number, hashable : key):
>         # code...

Please note that you seem to be using the syntax ``type: variable''.
We've settled on using ``variable: type'' instead.  (How to express
types is quite a different story...)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg at cosc.canterbury.ac.nz  Mon Apr  5 20:28:38 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 20:28:44 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <4071160D.5060204@prescod.net>
Message-ID: <200404060028.i360ScMU003594@cosc353.cosc.canterbury.ac.nz>

Paul Prescod <paul@prescod.net>:

> I'm not saying that Python has to do what the other languages do because 
> they do it, but all else equal,

But all else isn't equal, at least not for me, because I like one
better than the other. If I didn't care one way or the other, I might
look to another language for a tiebreaker. But I don't need to,
because I already know which one I prefer.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon Apr  5 20:32:06 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 20:32:17 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1081162471.3004.4.camel@localhost>
Message-ID: <200404060032.i360W6ao003613@cosc353.cosc.canterbury.ac.nz>

On Mon, 2004-04-05 at 11:38, David Abrahams wrote:
> Not bad.  Also tolerable:
> 
>   as [classmethod]
>   def foo(bar, baz):
>      pass

I thought we decided long ago that we can't use a keyword
because no single keyword could sound right in the context
of all possible use cases.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon Apr  5 20:38:00 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 20:38:13 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200404060038.i360c0Qo003623@cosc353.cosc.canterbury.ac.nz>

"Moore, Paul" <Paul.Moore@atosorigin.com>:

> Hmm. I'm torn. On the one hand, it's really cool that Python+Psyco can
> out-perform C (another argument against the knee-jerk "recode bits in
> C" reaction). But my gut feel is that not enough people use Psyco to
> make this a good answer yet.

If and when Psyco becomes an integral part of the core
Python distribution and works on all supported platforms,
then it may make sense to keep more stuff in Python as
a matter of course.

Until then, keeping pure-Python versions of some modules
around as an alternative for Psyco users might be
desirable.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From aahz at pythoncraft.com  Mon Apr  5 20:42:45 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr  5 20:42:52 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <200404060038.i360c0Qo003623@cosc353.cosc.canterbury.ac.nz>
References: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
	<200404060038.i360c0Qo003623@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040406004244.GA7899@panix.com>

On Tue, Apr 06, 2004, Greg Ewing wrote:
>
> If and when Psyco becomes an integral part of the core Python
> distribution and works on all supported platforms, then it may make
> sense to keep more stuff in Python as a matter of course.
>
> Until then, keeping pure-Python versions of some modules around as an
> alternative for Psyco users might be desirable.

No, that's not the way to look at it, IMO.  The point is to keep Python
versions around for everyone who isn't a C programmer.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From bob at redivi.com  Mon Apr  5 20:50:28 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Apr  5 20:46:22 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <200404060038.i360c0Qo003623@cosc353.cosc.canterbury.ac.nz>
References: <200404060038.i360c0Qo003623@cosc353.cosc.canterbury.ac.nz>
Message-ID: <66133DA1-8764-11D8-B3FA-000A95686CD8@redivi.com>


On Apr 5, 2004, at 8:38 PM, Greg Ewing wrote:

> "Moore, Paul" <Paul.Moore@atosorigin.com>:
>
>> Hmm. I'm torn. On the one hand, it's really cool that Python+Psyco can
>> out-perform C (another argument against the knee-jerk "recode bits in
>> C" reaction). But my gut feel is that not enough people use Psyco to
>> make this a good answer yet.
>
> If and when Psyco becomes an integral part of the core
> Python distribution and works on all supported platforms,
> then it may make sense to keep more stuff in Python as
> a matter of course.
>
> Until then, keeping pure-Python versions of some modules
> around as an alternative for Psyco users might be
> desirable.

If nothing else pure python versions of modules:
	(a) serve as great documentation
	(b) are good for psyco
	(c) are good for PyPy
	(d) are good for Pyrex
	(e) are good for Starkiller
	(f) are good for Jython
	(g) are good for IronPython
	....

I would recommend that all standard library C extensions that can have 
python implementations should have them somewhere that is easy to find, 
even if it's in a CVS sandbox somewhere.

-bob


From greg at cosc.canterbury.ac.nz  Mon Apr  5 21:14:27 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 21:14:37 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <003901c41b1c$3fd9f760$6402a8c0@arkdesktop>
Message-ID: <200404060114.i361ERiT003676@cosc353.cosc.canterbury.ac.nz>

Andrew Koenig <ark-mlist@att.net>:

> For that matter, why do we need the brackets if there is only one attribute:
> 
> 	def foo(bar, baz) staticmethod:
> 		pass

Because the decorator doesn't stand out enough that way.
It's a matter of opinion, but I think it looks better
with the brackets, even if there is only one decorator.

The brackets mean "here is something additional", not
"here is a list".

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From ishnigarrab at earthlink.net  Mon Apr  5 21:30:32 2004
From: ishnigarrab at earthlink.net (Isaac)
Date: Mon Apr  5 21:31:08 2004
Subject: [Python-Dev] 
	Validators (was Re: Python-Dev Digest, Vol 9, Issue 17)
Message-ID: <40720838.5050905@earthlink.net>

Guido:

>Please note that you seem to be using the syntax ``type: variable''.
>We've settled on using ``variable: type'' instead.  (How to express
>types is quite a different story...)
>

I haven't been following the thread on the subject much at all, I've 
just heard some rumbling about introducing static typing into python, 
and frankly I'm a bit wary about the whole thing. I was just thinking 
that in keeping with python's dynamic nature that something more like a 
generalized way of validating the _nature_ of something as opposed to 
it's underlying *type* would be more appropriate. It's just that it 
rings more pythonic to me (not that I have in any way graduated to the 
rank of Pythonist to make such judgments). As for the order of things, 
it's really not that important, "name: validator" would be just as 
useful. The only problem would be that

    def func(name: int): pass

works in terms of type checking, as in

    def func(name):
        if not isinstance(name, int):
            raise TypeError

where as translating it as a validator:

    def func(name):
        int(name)

doesn't make as much sense, int would have to be overloaded or something 
like is_int would have to be implemented.

But the advantages of validators would be numerous, IMO:

 * custom validators (equivalent to custom 'types' like in_range, or 
iterable)
 * still allows for the same functionality as typing systems while 
retaining dynamic capabilities.
 * possibly inline conversions (interface mutators, adaptors, I wonder 
if twisted might like something like this).

i.e.:

    def add(a: int, b: int): return a+b

    add('45', 5) => 50

if it were established that validators raise exceptions if the input is 
invalid, and return the value (possibly modified) if valid. Similar to 
implicit upcasting, i.e.:

    class MyInt(int):
       def __init__(self, i):
          self.i = i
       def add(self, other):
          self.i += other.i
          return self

    def add(a: MyInt, b: MyInt):
       return a.add(b)

    add(4, 4) => MyInt(8)

Again, I'm mostly just thinking aloud, hoping to be productive.

Isaac


From s.percivall at chello.se  Mon Apr  5 22:00:50 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Mon Apr  5 22:00:55 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <eppstein-B1A289.16241405042004@sea.gmane.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<200404051459.10263.fdrake@acm.org>
	<1081195698.2377.9.camel@localhost>
	<200404052209.i35M9m606023@guido.python.org>
	<eppstein-B1A289.16241405042004@sea.gmane.org>
Message-ID: <3AA96ED4-876E-11D8-A42F-0003934AD54A@chello.se>

On 2004-04-06, at 01.24, David Eppstein wrote:
>
> Ok, then how about
>     <decorator>
>     def ...
> ?
>
> '<' can't start an expression or statement currently, can it?

Yeah. I think that would look better. On the other hand ...

On 2004-03-31, at 17.42, Guido van Rossum wrote:
> Why does <...> look better than [...]?  To me, <...> just reminds me
> of XML, which is totally the wrong association.
>
> There are several parsing problems with <...>: the lexer doesn't see <
> and > as matched brackets, so you won't be able to break lines without
> using a backslash, and the closing > is ambiguous -- it might be a
> comparison operator.


From eppstein at ics.uci.edu  Mon Apr  5 22:06:58 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Mon Apr  5 22:07:04 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<200404051459.10263.fdrake@acm.org>
	<1081195698.2377.9.camel@localhost>
	<200404052209.i35M9m606023@guido.python.org>
	<eppstein-B1A289.16241405042004@sea.gmane.org>
	<3AA96ED4-876E-11D8-A42F-0003934AD54A@chello.se>
Message-ID: <eppstein-B58564.19065805042004@sea.gmane.org>

In article <3AA96ED4-876E-11D8-A42F-0003934AD54A@chello.se>,
 Simon Percivall <s.percivall@chello.se> wrote:

> On 2004-04-06, at 01.24, David Eppstein wrote:
> >
> > Ok, then how about
> >     <decorator>
> >     def ...
> > ?
> >
> > '<' can't start an expression or statement currently, can it?
> 
> Yeah. I think that would look better. On the other hand ...
> 
> On 2004-03-31, at 17.42, Guido van Rossum wrote:
> > Why does <...> look better than [...]?  To me, <...> just reminds me
> > of XML, which is totally the wrong association.
> >
> > There are several parsing problems with <...>: the lexer doesn't see <
> > and > as matched brackets, so you won't be able to break lines without
> > using a backslash, and the closing > is ambiguous -- it might be a
> > comparison operator.

Well, Mike Pall's long message "The Need for a Declarative Syntax" 
convinced me that some kind of bracketed prefix is the best location for 
these things, but I still don't like [...] because it already has a 
(useless) meaning, so declarators would pass silently in old Pythons.
And the same objection applies to all other existing brackets.
So, angle brackets seem like the cleanest remaining choice.

Re the lexer, I don't see this as a big problem as long as multiple 
declarators can be handled by multiple <...> pairs.  Re the ambiguity, I 
think comparisons shouldn't normally appear in declarators, so it's ok 
if they have to be parenthesized.

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From jcarlson at uci.edu  Mon Apr  5 22:06:47 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Apr  5 22:12:04 2004
Subject: [Python-Dev]  Validators (was Re: Python-Dev Digest, Vol 9,
	Issue 17)
In-Reply-To: <40720838.5050905@earthlink.net>
References: <40720838.5050905@earthlink.net>
Message-ID: <20040405184719.EC83.JCARLSON@uci.edu>

Isaac,

> >Please note that you seem to be using the syntax ``type: variable''.
> >We've settled on using ``variable: type'' instead.  (How to express
> >types is quite a different story...)
> >
> 
> I haven't been following the thread on the subject much at all, I've 
> just heard some rumbling about introducing static typing into python, 

Python is not going to have static typing as a part of the base language,
so worry not.

> and frankly I'm a bit wary about the whole thing. I was just thinking 
> that in keeping with python's dynamic nature that something more like a 
> generalized way of validating the _nature_ of something as opposed to 
> it's underlying *type* would be more appropriate. It's just that it 

How do you propose to get at the "nature" of what an object is? 
Currently in Python, we really only have types to determine what an
object is.  With that, we /can/ do per-call checks on the types of the
input, heck, if we're smart, we can even do polymorphism.  Such a
solution is doable in current Python with various approaches.  Making a
decorator that would handle type checking and polymorphism has been
provided here already, or we can go a class-based route to do the same
thing, though not nearly as neat.

Still, what do you mean by the "nature" of what something is?

 - Josiah


From greg at cosc.canterbury.ac.nz  Mon Apr  5 22:25:18 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 22:25:43 2004
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 9, Issue 17
In-Reply-To: <4071B57C.60803@earthlink.net>
Message-ID: <200404060225.i362PIju003785@cosc353.cosc.canterbury.ac.nz>

Isaac <ishnigarrab@earthlink.net>:

>    'def' NAME '(' [validator ':'] v_name ',' ... ')' ':' suite
>
>    def ret_type : func(arg, arg, ...): pass

That would be repeating C's mistake of putting the
type info in front of the name being declared.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From ishnigarrab at earthlink.net  Mon Apr  5 22:29:58 2004
From: ishnigarrab at earthlink.net (Isaac)
Date: Mon Apr  5 22:30:37 2004
Subject: [Python-Dev]  Validators (was Re: Python-Dev Digest, Vol 9, Issue
	17)
In-Reply-To: <20040405184719.EC83.JCARLSON@uci.edu>
References: <40720838.5050905@earthlink.net>
	<20040405184719.EC83.JCARLSON@uci.edu>
Message-ID: <40721626.8090906@earthlink.net>

Josiah Carlson wrote:

>Isaac,
>
>  
>
>>>Please note that you seem to be using the syntax ``type: variable''.
>>>We've settled on using ``variable: type'' instead.  (How to express
>>>types is quite a different story...)
>>>
>>>      
>>>
>>I haven't been following the thread on the subject much at all, I've 
>>just heard some rumbling about introducing static typing into python, 
>>    
>>
>
>Python is not going to have static typing as a part of the base language,
>so worry not.
>
>  
>
>>and frankly I'm a bit wary about the whole thing. I was just thinking 
>>that in keeping with python's dynamic nature that something more like a 
>>generalized way of validating the _nature_ of something as opposed to 
>>it's underlying *type* would be more appropriate. It's just that it 
>>    
>>
>
>How do you propose to get at the "nature" of what an object is? 
>Currently in Python, we really only have types to determine what an
>object is.  With that, we /can/ do per-call checks on the types of the
>input, heck, if we're smart, we can even do polymorphism.  Such a
>solution is doable in current Python with various approaches.  Making a
>decorator that would handle type checking and polymorphism has been
>provided here already, or we can go a class-based route to do the same
>thing, though not nearly as neat.
>
>Still, what do you mean by the "nature" of what something is?
>
> - Josiah
>  
>
Nature was a horrible word for me to use; what I was trying to convey 
was that there are so many mays to analyze an object besides just the 
type (which I assume directly refers to the return value of type(object) 
as opposed to some loose concept of 'type' as in style) of the object. 
Like some of the examples I gave earlier, in_range, i.e.:

    def in_range(*args):
       r = range(*args)
       def partial(n):
          if n in r:
             return n
          else:
             raise TypeError
       return partial

    def func(a: in_range(10)): pass

which is very different from pure type checking which I assume would be:

    def func(a: int): pass

which simply translates to an 'isinstance(a, int)' call, where 'a' is a 
variable and 'int' is a type object.

From eppstein at ics.uci.edu  Mon Apr  5 22:34:38 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Mon Apr  5 22:34:45 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<200404051459.10263.fdrake@acm.org>
	<1081195698.2377.9.camel@localhost>
	<200404052209.i35M9m606023@guido.python.org>
	<eppstein-B1A289.16241405042004@sea.gmane.org>
	<3AA96ED4-876E-11D8-A42F-0003934AD54A@chello.se>
	<eppstein-B58564.19065805042004@sea.gmane.org>
Message-ID: <eppstein-A86399.19343805042004@sea.gmane.org>

> > > There are several parsing problems with <...>: the lexer doesn't see <
> > > and > as matched brackets, so you won't be able to break lines without
> > > using a backslash, and the closing > is ambiguous -- it might be a
> > > comparison operator.
>
> Re the lexer, I don't see this as a big problem as long as multiple 
> declarators can be handled by multiple <...> pairs.

...besides, how hard can it be to convince the lexer to treat '<' at the 
start of a non-continuation line differently from other placements of 
the same char?

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From greg at cosc.canterbury.ac.nz  Mon Apr  5 22:49:58 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 22:50:19 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040405175117.GA21587@mems-exchange.org>
Message-ID: <200404060249.i362nwXn003846@cosc353.cosc.canterbury.ac.nz>

Neil Schemenauer <nas-python@python.ca>:

> On Mon, Apr 05, 2004 at 01:36:24PM -0400, Aahz wrote:
> > Your intuition is wrong.  From package's __init__.py, ``from .`` refers
> > to package's parent; to get back into the package, you need to use
> > ``from .package``.
> 
> That's confusing.  I expected '.' to refer to the current package,
> just as '.' refers to the current directory in most file systems.

Hmmm. I think people's intuition is likely to differ here depending on
whether the module doing the import is a plain module or a package.

In a plain module, the only sensible thing for .foo to mean is a
module foo in the same package as the importing module.

But in a package, it seems to me the most obvious meaning of .foo is a
module foo in the package. I would suggest that this is what it
*should* mean, since it's going to be very common for a package to
refer to submodules of itself, and requiring the name of the package
to be included in all such references would defeat one of the purposes
of having relative imports, which is to minimise the amount of
renaming needed when moving things around in the module hierarchy.

Would it be unreasonable to define "." as meaning the current
*package*, rather than the current *module* -- i.e. the package itself
if the importer is a package, and the containing package if it's a
plain module?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon Apr  5 22:55:54 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 22:56:04 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <66133DA1-8764-11D8-B3FA-000A95686CD8@redivi.com>
Message-ID: <200404060255.i362tsPG003858@cosc353.cosc.canterbury.ac.nz>

> If nothing else pure python versions of modules:
> 	(a) serve as great documentation
> 	(b) are good for psyco
> 	(c) are good for PyPy
> 	(d) are good for Pyrex
> 	(e) are good for Starkiller
> 	(f) are good for Jython
> 	(g) are good for IronPython

I've just thought of another one:

        (h) are useful for platforms where there isn't a compiled
            C version available yet and it's too much hassle to
            compile it yourself

a case I ran into the other day when I wanted to use the csv
module from 2.3 with a 2.2 installation on Windows (don't
ask why). I ended up writing my own Python version of a part
of it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon Apr  5 22:58:05 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr  5 22:58:17 2004
Subject: [Python-Dev]	Validators (was Re: Python-Dev Digest, Vol 9,
	Issue 17)
In-Reply-To: <40720838.5050905@earthlink.net>
Message-ID: <200404060258.i362w5ZE003864@cosc353.cosc.canterbury.ac.nz>

While we're on the topic, an idea I had once for type
declaration syntax went like

  def foo(i/int, s/str)/float:
    ...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From bob at redivi.com  Mon Apr  5 23:04:44 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Apr  5 23:00:39 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <200404060255.i362tsPG003858@cosc353.cosc.canterbury.ac.nz>
References: <200404060255.i362tsPG003858@cosc353.cosc.canterbury.ac.nz>
Message-ID: <27B0C115-8777-11D8-84F3-000A95686CD8@redivi.com>


On Apr 5, 2004, at 10:55 PM, Greg Ewing wrote:

>> If nothing else pure python versions of modules:
>> 	(a) serve as great documentation
>> 	(b) are good for psyco
>> 	(c) are good for PyPy
>> 	(d) are good for Pyrex
>> 	(e) are good for Starkiller
>> 	(f) are good for Jython
>> 	(g) are good for IronPython
>
> I've just thought of another one:
>
>         (h) are useful for platforms where there isn't a compiled
>             C version available yet and it's too much hassle to
>             compile it yourself
>
> a case I ran into the other day when I wanted to use the csv
> module from 2.3 with a 2.2 installation on Windows (don't
> ask why). I ended up writing my own Python version of a part
> of it.

Yeah, the backwards portability bit is key, I forgot about that one. 
I've personally ported (read: copied the documentation into a python 
file) itertools to pure python so I could use it (and the sets module) 
from Python 2.2.

-bob


From DavidA at activestate.com  Tue Apr  6 00:22:13 2004
From: DavidA at activestate.com (David Ascher)
Date: Tue Apr  6 00:25:29 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404051643.i35GhU005203@guido.python.org>
References: <200404050601.i35614f6002297@cosc353.cosc.canterbury.ac.nz>
	<4071160D.5060204@prescod.net>
	<200404051643.i35GhU005203@guido.python.org>
Message-ID: <40723075.3000808@ActiveState.com>

Guido van Rossum wrote:

> Anyway, personal aesthetics are learned behavior, like it or not.

Maybe yours.  Mine are god-given.  So there.

--david

From jcarlson at uci.edu  Tue Apr  6 03:04:54 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue Apr  6 03:09:05 2004
Subject: [Python-Dev]  Validators (was Re: Python-Dev Digest, Vol 9,
	Issue 17)
In-Reply-To: <40721626.8090906@earthlink.net>
References: <20040405184719.EC83.JCARLSON@uci.edu>
	<40721626.8090906@earthlink.net>
Message-ID: <20040405233102.EC87.JCARLSON@uci.edu>

> Nature was a horrible word for me to use; what I was trying to convey 
> was that there are so many mays to analyze an object besides just the 
> type (which I assume directly refers to the return value of type(object) 
> as opposed to some loose concept of 'type' as in style) of the object. 
> Like some of the examples I gave earlier, in_range, i.e.:

type(thing) and isinstance(thing, typ) are the only reliable methods we
have for determining the "nature" of what an object really is.

In terms of general validation of input, that is also solved by the
general decorator syntax.  You can wrap your function with any arbitrary
argument or return validation as you see fit, if I remember correctly,
such decorators are even planned to be in the standard library somewhere.


> which is very different from pure type checking which I assume would be:
> 
>     def func(a: int): pass
> 
> which simply translates to an 'isinstance(a, int)' call, where 'a' is a 
> variable and 'int' is a type object.

Aside from your "validator" syntax, which I hope /never/ makes it into
Python, we understand what you are saying, however, as I said before
(and as you didn't listen to), such runtime pre and post validation is
currently possible in Python.  Certainly it isn't part of the /syntax/
of Python, but I don't believe such type checking or input validation
should be a part of Python syntax.

Considering that what you desire is already possible to do, and will be
made easier with decorators, what else do you want?

 - Josiah


From python-dev at zesty.ca  Tue Apr  6 08:46:44 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Tue Apr  6 08:46:22 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEGDJOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEGDJOAB.tim.one@comcast.net>
Message-ID: <Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>

On Tue, 30 Mar 2004, Tim Peters wrote:
> I want marshaling of fp numbers to give exact (not approximate) round-trip
> equality on a single box, and across all boxes supporting the 754 standard
> where C maps "double" to a 754 double.

That is a valuable property.  I support it and support Python continuing
to have that property.

I hope it has been made quite clear by now that this property does not
constrain how numbers are displayed by the interpreter in human-readable
form.  The issue of choosing an appropriate string representation of a
number is unaffected by the desire for the above property.

> [Ping]
> > All right.  Maybe we can make some progress.
>
> Probably not -- we have indeed been thru all of this before.

I think we *have* made progress.  Now we can set aside the red-herring
issue of platform-independent serialization and focus on the real issue:
human-readable string representation.

So let's look at what you said about Python's accessibility:

> As I said before (again and again and again <wink>), I'm the one who has
> fielded most newbie questions about fp since Python's beginning, and I'm
> very happy with the results of changing repr() to produce 17 digits.  They
> get a little shock at the start now, but potentially save themselves from
> catastrophe by being forced to grow some *necessary* caution about fp
> results early.

Now you are pulling rank.  I cannot dispute your longer history and
greater experience with Python; it is something i greatly admire and
respect.  I also don't know your personal experiences teaching Python.

But i can tell you my experiences.  And i can tell you that i have
tried to teach Python to many people, individually and in groups.  I
taught a class in Python at UC Berkeley last spring to 22 people who
had never used Python before.  I maintained good communication with
the students and their feedback was very positive about the class.

How did the class react to floating-point?  Seeing behaviour like this:

    >>> 3.3
    3.2999999999999998
    >>>

confused and frightened them, and continues to confuse and frighten
almost everyone i teach.  (The rare exceptions are the people who have
done lots of computational work before and know how binary floating-point
representations work.)  Every time this happens, the teaching is
derailed and i am forced to go into an explanation of binary
floating-point to assuage their fears.

Remember, i am trying to teach basic programming skills.  How to solve
problems; how to break down problems into steps; what's a subroutine;
and so on.  Aside from this floating-point thing throwing them off,
Python is a great first language for new programmers.  This is not the
time to talk about internal number representation.

I am tired of making excuses for Python.  I love to tell people about
Python and show them what it can do for them.  But this floating-point
problem is embarrassing.  People are confused because no other system
they've seen behaves like this.  Other languages don't print their
numbers like this.  Accounting programs and spreadsheets don't print
their numbers like this.  Matlab and Maple and Mathematica don't print
their numbers like this.  Only Python insists on being this ugly.  And
it screws up the most common way that people first get to know Python
-- as a handy interactive calculator.

And for what?  For no gain at all -- because when you limit your focus
to the display issue, the only argument you're making is "People should
be frightened."  That's a pointless reason.

Everything in Python -- everything in computers, in fact -- is a *model*.
We don't expect the model to be perfectly accurate or to be completely
free of limitations.  IEEE 754 happens to be the prevalent model for
the real number line.  We don't print every string with a message after
it saying "WARNING: MAXIMUM LENGTH 4294967296", and we shouldn't do
the same for floats.

"3.2999999999999998" does not give you any more information than "3.3".
They both represent exactly the same value.  "3.3" is vastly easier to
read.  The only reason you seem to want to display "3.2999999999999998"
is to frighten people.  So why not display "3.3 DANGER DANGER!!"?
Even that would be much easier to read, but my point is that i hope it
exposes the problem with your argument.

I'm asking you to be more realistic here.  Not everyone runs into
floating-point corner cases.  In fact, very few people do.  I have never
encountered such a problem in my entire history of using Python.  And if
you surveyed the user community, i'm sure you would find that only a
small minority cares enough about the 17th decimal place for the
discrepancy to be an issue.

Now, why should that minority make it everyone else's problem?

Is this CP4E or CPFPFWBFP?  Computer Programming For Everybody?  Or
Computer Programming For People Familiar With Binary Floating-Point?

You say it's better for people to get "bitten early".  What's better:
everyone suffering for a problem that will never affect most of them,
or just those who care about the issue having to deal with it?

    Beautiful is better than ugly.
    Practicality beats purity.
    Readability counts.



-- ?!ng

From s.percivall at chello.se  Tue Apr  6 09:12:22 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Tue Apr  6 09:12:26 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>
References: <LNBBLJKPBEHFEDALKOLCEEGDJOAB.tim.one@comcast.net>
	<Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>
Message-ID: <0AAC5E0D-87CC-11D8-A42F-0003934AD54A@chello.se>

On 2004-04-06, at 14.46, Ka-Ping Yee wrote:
> I think we *have* made progress.  Now we can set aside the red-herring
> issue of platform-independent serialization and focus on the real 
> issue:
> human-readable string representation.
[Snip]
> I am tired of making excuses for Python.  I love to tell people about
> Python and show them what it can do for them.  But this floating-point
> problem is embarrassing.  People are confused because no other system
> they've seen behaves like this.  Other languages don't print their
> numbers like this.  Accounting programs and spreadsheets don't print
> their numbers like this.  Matlab and Maple and Mathematica don't print
> their numbers like this.  Only Python insists on being this ugly.  And
> it screws up the most common way that people first get to know Python
> -- as a handy interactive calculator.

So how should "2.2 - 1.2 - 1" be represented?

Matlab (Solaris 9):          2.22044604925031e-16
Octave (MacOS X 10.3):       2.22044604925031e-16
Python 2.3.3 (MacOS X 10.3): 2.2204460492503131e-16

Is this something you accept since Matlab does it?


From barry at python.org  Tue Apr  6 09:17:11 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Apr  6 09:17:22 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: <s9zwu4wdinb.fsf@salmakis.intevation.de>
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
Message-ID: <1081257431.30379.22.camel@anthem.wooz.org>

On Sat, 2004-04-03 at 13:24, Bernhard Herzog wrote:

> I don't think this is a good description of the actual problem.  The
> problem is that the modules in the package shadow global ones and
> there's no easy way to bypass the local package.  FWIW, though, I don't
> recall ever running into a problem with that.

It happens, and it makes for tricky choices.  I know to stay away from
internal modules like re and email, but as the standard library gets
fatter, there's more chance of collision.  The only remedy will be to
rename an internal module when I find that I suddenly want to use some
new standard module.  There should be more independence.

If all import paths were absolute, then I wouldn't have this problem,
and in fact, that's the rule my own code tries to adhere to.  See the
email package which uses absolute imports internally.  I know others
have use cases for relative imports, but I've never quite seen the need.

Ideally, IMO, all imports would be absolute.  If we have to throw a bone
to relative import proponents in order to get to absolute imports by
default, so be it.

-Barry



From barry at python.org  Tue Apr  6 09:25:16 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Apr  6 09:25:23 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: <20040403191106.GA4254@panix.com>
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
	<20040403191106.GA4254@panix.com>
Message-ID: <1081257915.30379.27.camel@anthem.wooz.org>

On Sat, 2004-04-03 at 14:11, Aahz wrote:

> Barry, would you please write up something for the PEP here?  (You're
> the person who pushed this the most.)  I've already updated the PEP to
> include clarification of the shadowing issue.

I added a few more sentences.

-Barry



From python-dev at zesty.ca  Tue Apr  6 09:49:05 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Tue Apr  6 09:48:44 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <0AAC5E0D-87CC-11D8-A42F-0003934AD54A@chello.se>
References: <LNBBLJKPBEHFEDALKOLCEEGDJOAB.tim.one@comcast.net>
	<Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>
	<0AAC5E0D-87CC-11D8-A42F-0003934AD54A@chello.se>
Message-ID: <Pine.LNX.4.58.0404060842330.18028@server1.LFW.org>

On Tue, 6 Apr 2004, Simon Percivall wrote:
> So how should "2.2 - 1.2 - 1" be represented?
>
> Matlab (Solaris 9):          2.22044604925031e-16
> Octave (MacOS X 10.3):       2.22044604925031e-16
> Python 2.3.3 (MacOS X 10.3): 2.2204460492503131e-16
>
> Is this something you accept since Matlab does it?

I accept it, but not primarily because Matlab does it.

It is fine to show a value different from zero in this case
because the result really is different from zero.  In this case
those digits are necessary to represent the machine number.
In other cases (such as 1.1) they are not.


-- ?!ng

From aahz at pythoncraft.com  Tue Apr  6 10:00:53 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr  6 10:00:57 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <Pine.LNX.4.58.0404060842330.18028@server1.LFW.org>
References: <LNBBLJKPBEHFEDALKOLCEEGDJOAB.tim.one@comcast.net>
	<Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>
	<0AAC5E0D-87CC-11D8-A42F-0003934AD54A@chello.se>
	<Pine.LNX.4.58.0404060842330.18028@server1.LFW.org>
Message-ID: <20040406140052.GB27004@panix.com>

On Tue, Apr 06, 2004, Ka-Ping Yee wrote:
> On Tue, 6 Apr 2004, Simon Percivall wrote:
>>
>> So how should "2.2 - 1.2 - 1" be represented?
>>
>> Matlab (Solaris 9):          2.22044604925031e-16
>> Octave (MacOS X 10.3):       2.22044604925031e-16
>> Python 2.3.3 (MacOS X 10.3): 2.2204460492503131e-16
>>
>> Is this something you accept since Matlab does it?
> 
> I accept it, but not primarily because Matlab does it.
> 
> It is fine to show a value different from zero in this case
> because the result really is different from zero.  In this case
> those digits are necessary to represent the machine number.
> In other cases (such as 1.1) they are not.

The point is that anyone who relies on floating point will almost
certainly end up with the equivalent of Simon's case at some point.  The
most common *really* ugly case that's hard for new programmers to deal
with is:

    section_size = length / num_sections
    tmp = 0
    while tmp < length:
        process(section)
        tmp += section_size

and variations thereof.  Just because every other programming language
hides this essential difficulty is no reason to follow along.  I don't
have Tim's experience, but until we added that section to the tutorial,
I was one of the regular "first responders" on c.l.py who was always
dealing with this issue.  I stand with Uncle Timmy -- it's been much
easier now.

If you want to teach Python to complete newcomers, you need to do one of
two things:

* Avoid floating point

* Start by explaining what's going on (use Simon's example above for
emphasis)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From FBatista at uniFON.com.ar  Tue Apr  6 10:13:43 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr  6 10:15:04 2004
Subject: [Python-Dev] PEP 327: Decimal data type
Message-ID: <A128D751272CD411BC9200508BC2194D03383841@escpl.tcp.com.ar>

[Aahz]
#- > where ``value1`` can be int, long, string, tuple or Decimal,
#- > ``value1`` can be only float, and ``decimal_digits`` is an optional
#- > int.
#- 
#- You mean value2 in the second case, right?

Yes, thank you. This and other typos has been fixed by Tim Peters in the
process of acceptance of the PEP.

So, you can expect a more typo-free PEP in the future, :)

.	Facundo

From aahz at pythoncraft.com  Tue Apr  6 10:16:15 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr  6 10:16:19 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: <1081257431.30379.22.camel@anthem.wooz.org>
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
	<1081257431.30379.22.camel@anthem.wooz.org>
Message-ID: <20040406141615.GD27004@panix.com>

On Tue, Apr 06, 2004, Barry Warsaw wrote:
> On Sat, 2004-04-03 at 13:24, Bernhard Herzog wrote:
>> 
>> I don't think this is a good description of the actual problem.  The
>> problem is that the modules in the package shadow global ones and
>> there's no easy way to bypass the local package.  FWIW, though, I don't
>> recall ever running into a problem with that.
> 
> It happens, and it makes for tricky choices.  I know to stay away from
> internal modules like re and email, but as the standard library gets
> fatter, there's more chance of collision.  The only remedy will be to
> rename an internal module when I find that I suddenly want to use some
> new standard module.  There should be more independence.

Thanks!  Here's my rewrite; comments, anyone?

Rationale for Absolute Imports
==============================

In Python 2.3 and earlier, if you're reading a module located inside a
package, it is not clear whether ::

    import foo

refers to a top-level module or to another module inside the package.
As Python's library expands, more and more old modules suddenly shadow
builtin modules by accident.  It's a particularly difficult problem
inside packages because there's no way to specify which module is meant.
To resolve the ambiguity, it is proposed that ``foo`` will always be a
module or package reachable from ``sys.path``.  This is called an
absolute import.

The python-dev community chose absolute imports as the default because
they're the more common use case and because absolute imports can provide
all the functionality of relative (intra-package) imports  -- albeit at
the cost of difficulty when renaming package pieces higher up in the
hierarchy or when moving one package inside another.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From jim.jewett at eds.com  Tue Apr  6 10:29:10 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue Apr  6 10:30:04 2004
Subject: [Python-Dev] Expert floats
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D442@USAHM010.amer.corp.eds.com>

Ka-Ping Yee:

>    >>> 3.3
>    3.2999999999999998

> confused and frightened them, and continues to confuse and frighten
> almost everyone i teach. 

It bothered me when I first saw it (not in Python).

It was still better than trying to figure out why my code was doing
the wrong thing, when my math and logic were both clearly correct.  

To steal Simon Percivall's example, I assumed 2.20 - 1.20 == 1.00.
It didn't even occur to me to check something as obviously true as 
that -- until I had learned and remembered that floats were not 
decimals.

-jJ

From arigo at tunes.org  Tue Apr  6 10:33:56 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Apr  6 10:37:26 2004
Subject: [Python-Dev] Python is faster than C
In-Reply-To: <1081206375.2666.20.camel@echidna.object-craft.com.au>
References: <16E1010E4581B049ABC51D4975CEDB8803060E52@UKDCX001.uk.int.atosorigin.com>
	<16497.28394.944929.536518@magrathea.basistech.com>
	<1081206375.2666.20.camel@echidna.object-craft.com.au>
Message-ID: <20040406143356.GA21775@vicky.ecs.soton.ac.uk>

Hello Dave,

On Tue, Apr 06, 2004 at 09:06:16AM +1000, Dave Cole wrote:
> Would it make sense to build specialisation into the base Python
> interpreter so that it builds specialised versions of bytecode for
> functions.  I am assuming that there are only a few types that can be
> targeted by specialisation (int, float).  Bytecodes then could be added
> to the interpreter which operate on those native machine types rather
> than the Python objects.

This is a kind of project that would give a good speed-up with limited changes
in the Python interpreter, though it is still relatively involved.  Whether
someone shows up to actually implement and maintain it is another matter.  
Moreover there are a few dark spots that are easily overlooked; for example,
any operation between ints could actually return a long.  Psyco itself is very
different and copes well with this problem (and can do more than only
easy-typed locals).


A bientot,

Armin.


From jim.jewett at eds.com  Tue Apr  6 10:41:44 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue Apr  6 10:42:04 2004
Subject: [Python-Dev] Python is faster than C
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D443@USAHM010.amer.corp.eds.com>

Dave Cole:

> Would it make sense to build specialisation into the base Python
> interpreter so that it builds specialised versions of bytecode for
> functions.  I am assuming that there are only a few types that can be
> targeted by specialisation (int, float).  Bytecodes then could be added
> to the interpreter which operate on those native machine types rather
> than the Python objects.

If you add too many specializations, you blow the cache.
If you add not enough, then you help only certain types of programming.
If you're willing to do that, you might consider going even farther, as in 
http://www.foretec.com/python/workshops/1998-11/proceedings/papers/aycock-21
1/aycock211.html

Basically, he creates a large number of pseudo-codes that just mean
"do this, then this, then this" without going back to the eval loop
in between.  Because he knows what is coming next, there are additional
chances for optimizations.

-jJ

From tim.one at comcast.net  Tue Apr  6 11:00:27 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Apr  6 11:00:37 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D442@USAHM010.amer.corp.eds.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEADKBAB.tim.one@comcast.net>

[Ka-Ping Yee]
>>    >>> 3.3
>>    3.2999999999999998
>
>> confused and frightened them, and continues to confuse and frighten
>> almost everyone i teach.

[Jim Jewitt]
> It bothered me when I first saw it (not in Python).
>
> It was still better than trying to figure out why my code was doing
> the wrong thing, when my math and logic were both clearly correct.
>
> To steal Simon Percivall's example, I assumed 2.20 - 1.20 == 1.00.
> It didn't even occur to me to check something as obviously true as
> that -- until I had learned and remembered that floats were not
> decimals.

I believe Ping would have

>>> 2.20 - 1.20
1.0000000000000002

because "1.0000000000000002" is the shortest string that evals to the true
machine result.  OTOH, he'd have

>> 2.20 - 1.10
1.1

The true machine result isn't decimal 1.1, but "1.1" is again the shortest
string that evals to the true machine result.  Presumably newbies aren't
confused or frightened by 1.0000000000000002, because their intuition about
shortest-possible reproducing decimal representations of binary floats is
finely honed <wink>.

Most people should use Decimal:

>>> print Decimal("2.20") - Decimal("1.20")
1.00
>>> print Decimal("2.20") - Decimal("1.1")
1.10

The output is "what they expect" then, 100% truthful (the internal results
are exactly 1. and decimal 1.1, respectively), and even preserves a useful
notion of significant trailing zeroes.


From ark-mlist at att.net  Tue Apr  6 11:09:37 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue Apr  6 11:09:22 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEADKBAB.tim.one@comcast.net>
Message-ID: <002301c41be9$2dbedeb0$6402a8c0@arkdesktop>

> I believe Ping would have
> 
> >>> 2.20 - 1.20
> 1.0000000000000002

As would I.

> because "1.0000000000000002" is the shortest string that evals to the true
> machine result.  OTOH, he'd have
> 
> >> 2.20 - 1.10
> 1.1

Believe it or not, this apparent anomaly doesn't bother me, but

>>> 1.1
1.1000000000000001

does.  The reason is that the first example does arithmetic, and it is easy
to explain that floating-point arithmetic is not completely accurate.



From barry at python.org  Tue Apr  6 11:38:24 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Apr  6 11:38:33 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: <20040406141615.GD27004@panix.com>
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
	<1081257431.30379.22.camel@anthem.wooz.org>
	<20040406141615.GD27004@panix.com>
Message-ID: <1081265903.30379.92.camel@anthem.wooz.org>

On Tue, 2004-04-06 at 10:16, Aahz wrote:

> Thanks!  Here's my rewrite; comments, anyone?

Some minor rephrasing suggestion...

> Rationale for Absolute Imports
> ==============================
> 
> In Python 2.3 and earlier, if you're reading a module located inside a
> package, it is not clear whether ::
> 
>     import foo
> 
> refers to a top-level module or to another module inside the package.
> As Python's library expands, more and more old modules suddenly shadow
> builtin modules by accident.  

As Python's library expands, more and more existing package internal
modules suddenly shadow standard library modules by accident.

-Barry



From tim.one at comcast.net  Tue Apr  6 12:23:09 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Apr  6 12:23:18 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <002301c41be9$2dbedeb0$6402a8c0@arkdesktop>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEAOKBAB.tim.one@comcast.net>

[Tim]
>> I believe Ping would have
>>
>> >>> 2.20 - 1.20
>> 1.0000000000000002

[Andrew Koenig]
> As would I.

Only sometimes, right?  You earlier gave a list of 4 behaviors you sometimes
want, and that particular output isn't suitable for all of them.

>> OTOH, he'd have
>>
>> >>> 2.20 - 1.10
>> 1.1

> Believe it or not, this apparent anomaly doesn't bother me, but
>
> >>> 1.1
> 1.1000000000000001
>
> does.  The reason is that the first example does arithmetic, and it
> is easy to explain that floating-point arithmetic is not completely
> accurate.

Wouldn't it bother you to give a wrong explanation?  It's simply impossible
to explain why 2.2-1.2 is inaccurate without explaining that "2.2" and/or
"1.2" are already inaccurate on their own.  The *arithmetic* in both
examples is exactly correct (if you watched the hardware's inexact flag
across the subtractions, you'd see that it doesn't get set; x-y is always
exact in 754 binary floating point when x and y are within a factor of 2 of
each other).  The only cause of "funny results" in these examples is
representation error.

So trying to "explain" this by telling a newbie that arithmetic is
inaccurate may pacify them, but at the cost of increasing their confusion
(the explanation they're given then doesn't actually explain anything about
what they're seeing, and the true cause has been left untold).

OTOH,

>>> 2.2
2.2000000000000002
>>> 1.1
1.1000000000000001
>>>

points to the true cause, and the Tutorial Appendix on binary fp issues has
been successful in bringing newbies to a correct understanding of that.


From bv at artima.com  Tue Apr  6 16:14:18 2004
From: bv at artima.com (Bill Venners)
Date: Tue Apr  6 16:16:25 2004
Subject: [Python-Dev] Python Interest Group
Message-ID: <FBE9B4A2-8806-11D8-A283-00039301D446@artima.com>

Hello All,

I am the editor of Artima.com. Guido and others suggested that I send 
this query to python-dev.

I am about to release a new feature on Artima called Interest Groups, 
and I am looking for a few people who would like to moderate a Python 
Interest Group. I am wondering if any of you might like to serve as a 
moderator, or could recommend others from the Python community who 
might want to act as moderators. Here is the low down on Interest 
Groups:

Artima Interest Groups is a free service that allows communities to 
keep informed and discuss issues. Each interest group is essentially a 
news feed that serves a specific community. The news feed is moderated 
by one or more volunteers from the community. Anyone can submit news 
items for consideration. The moderators check the accuracy of the 
submissions, and decide which news items to approve. Approved items 
appear on web pages at Artima.com, and in two Artima RSS feeds. Each 
approved item can be discussed by the community in its own Artima 
Forums topic.

Basically, the moderators would get a short email notification when 
someone submits an item, then they can edit, approve, or reject the 
item via the web. It is like having a little theserverside.com or 
slashdot.org for your community. Another way you might think of it is 
as a web version of python-announce, except that only the topics are 
moderated. Any discussion that follows each topic is a free-for-all. 
One benefit of having an Interest Group at Artima is that people 
outside the Python community proper will also be exposed to the Python 
community's news.

If anyone is interested, or has a recommendation of someone I can 
approach about moderating a Python Interest Group, please email me.

Thanks.

Bill
--
Bill Venners
Artima Software, Inc.
http://www.artima.com


From lists at hugunin.net  Tue Apr  6 18:08:54 2004
From: lists at hugunin.net (Jim Hugunin)
Date: Tue Apr  6 18:21:38 2004
Subject: [Python-Dev] PEP 318: More examples of decorator use
Message-ID: <E1BAykW-000K2A-AP@smtp.npsis.com>

I'm a little scared of getting involved in the discussion of PEP 318.
Nevertheless, I think any language design discussion can benefit from more
examples.  Here's a list of ways that I would use decorators in my Python
code if I had simpler syntax than is available today.

*Interoperability with other languages and systems

jythonc parses method docstrings looking for a special directive to generate
Java-compatible method signatures, i.e.

def meth(self, a, b):
    "@sig public void meth(int a, int b)"
    pass

If Python had structured metadata, this would change to

[java_sig("public void meth(int a, int b)")]
def meth(self, a, b):
    pass

C# uses this mechanism to provide interoperability with C, COM, and web
services.  Here's a translation of a web services example from the .NET
framework docs.

[WebService(Description="Common Server Variables", 
            Namespace="http://www.contoso.com/")]
class ServerVariables:

    [WebMethod(Description="Obtains the Server Computer Name",
               EnableSession=false)]
    def GetMachineName(self):
       return self.Server.MachineName


    [SoapDocumentMethod(Action="http://www.contoso.com/Sample", 
           RequestNamespace="http://www.contoso.com/Request",
           RequestElementName="GetUserNameRequest",
           ResponseNamespace="http://www.contoso.com/Response",
           ResponseElementName="GetUserNameResponse"),
     WebMethod(Description="Obtains the User Name")]
    def GetUserName(self): 
        ...

This is the same functionality motivating Bob Ipolito's pleas for this
feature to support pyobjc, and it would obviously also help IronPython.


*Tracing and other global policies

Aspect-oriented programming has had great success capturing global policies
for things like tracing in a single module.  For a language like Python that
already has strong meta-programming support it's possible to program in an
AOP style without any language changes (see the tracing implementation in
parrotbench/b0.py for an example).

However, one lesson learned from actually applying AOP to large systems
(http://hugunin.net/papers/pra03-colyer.pdf) was that there was always a
need to capture a few local exceptions.  Current AOP systems can do this in
a clumsy way, but almost everyone agrees that C#/Java attributes are the
best way to specify these kinds of exceptions.

Here's an example of a tracing policy with a local exception.

from tracing import notrace, add_tracing

class C:
    def m1(self): ...
    def m2(self): ...

    [notrace]
    def m0(self): ...

if __DEBUG__: add_tracing(globals())

This code will recursively walk all the declarations in a module and wrap
all methods and functions with tracing code.  Attributes are used here to
indicate special handling for specific methods.  In this case, m0 should not
be traced.


*Unit testing

The nunit test framework uses attributes instead of naming conventions to
label test methods.  My favorite part of this is the additional attributes,
for example 'expected_exception' to indicate that the correct behavior for a
test is to throw a particular exception, i.e.

[test, expected_exception(InsufficientFundsError)]
def transfer2(self):
    self.source.transfer(100000000000)

vs.

def test_transfer2(self):
    self.assertRaises(InsufficientFundsError,
                      self.source.transfer, 1000000000)  

This example is extremely compelling for C# or Java code where functions
aren't first class and a complicated try/catch block is required instead of
assertRaises.  It's less compelling than I'd expected for Python; however, I
still find the version with attributes a little more clear.


*Parsing

There are many simple parser implementations for Python that embed grammar
rules in doc strings.  This would be nicer to have in explicit metadata.
Here's an example from Brian Sabbey's dparser for python.

[action("exp",  "exp '+' exp")]
def d_add(t):
    return t[0] + t[2]
    
[action("exp", '"[0-9]+"')]
def d_number(t):
    return int(t[0])


*Synchronization

I think a synchronized decorator would be useful, but more sophisticated
locking policies are even more interesting.  Here's an example of
reader/writer synchronization assuming a metaclass ReadWriteSynchro that
cooperates with attributes on specific methods.

class DataStore(ReadWriteSynchro):
    [reader]
    def read(self, key): ....

    [writer]
    def write(self, key, value): ...

    ...


*Other examples that I'd probably use

memoized from Michael Chermside to automatically cache the results of
idempotent functions

accepts and returns type checking wrappers from the PEP until/unless Python
gets a better answer for true optional type declarations

generic from Edward Loper to support generic functions.  I both really like
and really hate this decorator.  I'd love to have tolerable syntax for
generic functions and this gives me something tolerable.  On the other hand,
I really dislike code that uses sys._getframe() so that might be enough to
scare me off of this.


*Some examples that I'd like to see in an anti-patterns list

onexit from the PEP.  I don't think decorators should have side-effects
visible outside of the class/function itself.  This seems MUCH better
accomplished with the explicit call to the registration function.

singleton from the PEP.  Whenever possible, decorators should return the
same kind of thing that they are given.  This code seems much less clear
than explicitly creating a singleton instance.

propget, propset, propdel from Edward Loper via Guido.  To me, these violate
the principle of modularity.  I much prefer the explicit property
declaration that puts all of the parts together.

  class C(object):
     def getX(self): return self.__x
     def setX(self, x): self.__x = x
     x = property(getX, setX)


-Jim



From lists at hugunin.net  Tue Apr  6 18:09:37 2004
From: lists at hugunin.net (Jim Hugunin)
Date: Tue Apr  6 18:21:42 2004
Subject: [Python-Dev] PEP 318: How I would implement decorators 
Message-ID: <E1BAylD-000KKC-7j@smtp.npsis.com>

I've spent a couple days reading the decorators discussion and thinking
about the issues.  That's not a lot of time, but I still thought I'd share
my opinions on implementing this feature.  This all is in my (mostly) humble
opinion.

When I started looking at PEP 318, I thought that the most important issue
was deciding on the correct syntax.  After a while I became much more
worried about semantics.

* Semantics, composition and understandable failures

One of the nice properties of C# and Java attributes is that they are
declarative rather than transformative so there are no conflicts between
different attributes placed on the same object.  This is probably too strong
of a restriction for Python; nevertheless, it's a valuable reference point.

Using the definition of returns from the PEP, I get the following behavior
if I confuse its order with classmethod, i.e.

class C:
    [classmethod(m), returns(str)]
    def m(cls): return repr(cls)

C.m()
--> TypeError: unbound method new_f() must be called with C instance as
first argument (got nothing instead)

This is a hard message to understand, but at least it does occur at the
first instance of calling the method.  Even worse is where you forget to
create an instance of a class or to call a function like returns, i.e.

class C:
    [returns]
    def m(self):
        return "hello"

c = C()
print c.m() #there's no error for this call
--> function new_f at 0x01278BB0>

This kind of error scares me a lot.  It can be extremely hard to track down
because it doesn't occur either at the point of declaration or the point of
calling the method, but only at some later stage where the result from the
method is used.  I'm also concerned that this error is caused by the person
using the attribute not the person writing it and that person may be less
sophisticated and more easily confused.

A less critical, but important issue is the propagation of doc strings and
other function attributes, i.e.

class C:
    [returns(str)]
    def m(self):
        "a friendly greeting"
        return "hello"

print C.m.__doc__
--> None

This will really mess up programs that rely on doc strings or other function
attributes.

After thinking about these issues, my first inclination was to sign up for
Raymond Hettinger's NO on 318 campaign.  Unfortunately, as my previous
message showed there are a number of ways that I'd really like to use
decorators in my Python code.

My next thought was to propose that Python's decorators be restricted to be
declarative like C# and Java attributes.  They could be used to add meta
information to functions, but other meta-tools would be needed for any
actual transformations.  I'm still open to that idea, but I've come to
believe that's not a very pythonic approach.

My current idea is that we should design this feature and any documentation
around it to strongly encourage people to write safe, composable decorators.
Specifically, it should be easier to write safe and composable decorators
than to write bad ones.  After that, we can rely on decorator developers to
do the right thing.

1. Use a "decorate" method instead of a call to avoid the worst problems
like [returns].  This has the added benefit in my eyes of making it a little
harder to write small decorators and thus encouraging people to think a
little more before writing these meta-programs.

2. Provide a small number of helper classes that make it easy to write safe
and composable decorators.  Here are my top two candidates:

class attribute:
    def __init__(self, **kwds):
          self.attrs = kwds

    def decorate(self, decl):
        if not hasattr(decl, __attrs__):
            decl.__attrs__ = []
        decl.__attrs__.append(self)
        return decl

    def __repr__(self): ...

class func_wrapper:
    """subclass this and implement a wrap function"""
    def decorate(self, func):
        assert iscallable(func)
        new_func = self.wrap(func)
        assert iscallable(new_func)
        self.copy_attributes(func, new_func)
        return new_func

    def copy_attributes(self, old_func, new_func):
        ...

func_wrapper would be used for things like synchronized and memoize, and
attribute would be used for most of the other examples in my previous
message.

3. Provide a clear set of guidelines for good decorators, i.e.
  a. decorators should not have side effects outside of the class/function
  b. decorators should return the same kind of thing they are given
  c. decorators should be commutative whenever possible (requires b)


+ sys._getframe()

There should be a consensus on whether or not decorators like 'generic' are
a good or a bad idea.  If they're a good idea then the decorate method
should get an extra argument exposing the local namespace so that
sys._getframe() won't be required.  I'd be in favor of adding that argument
to decorate.


+ classmethod and staticmethod

These two decorators are a big problem because they don't return the same
kind of thing as they are given.  This means they always have to be the last
decorator applied in any list.  That kind of restriction is never a good
thing.

Personally, I'd like to see these changed from returning a new object to
instead setting an attribute on the existing function/method object.  Then
the method object could use this attribute to implement its __get__ method
correctly.  This would make these decorators compose very nicely with all of
the other decorators; however, it would be a fairly major change to how
Python handles static and class methods and I'm doubtful that would fly.


* Syntax

I've come to see syntax as less crucial than I originally thought.  In
particular, the examples I've read on this list of using temporary variables
showed me that even if the syntax is too confining there are tolerable
solutions.  Nevertheless, I'm still strongly in favor of putting the
decorator list in front of the def/class.

Decorators/attributes are an exciting new technique that's gaining
increasing use in different programming languages.  They're important enough
to be one of the key ideas that Java-1.5 is stealing from C# (turn-about is
always fair play).  The AOP community often has vehement debates not about
the value of attributes, but about how they can be most effectively used.

Because decorators are still a relatively new idea, it's hard to know how
much room they will need.  Certainly if we're only interested in classmethod
and staticmethod it's not much.  However, I think it's important to give
decorators as much room as possible to be prepared for the future.

I have some sympathy with the arguments that a bare prefix list is already
legal Python syntax and is certainly found in existing code.  I'm quite sure
that using a bare list would break existing Python programs that have
dangling lists left over as they evolved.

I like Sam's proposal to add a '*' in front of the list.  I could also be
persuaded to go along with any of the new grouping constructing, like <| |>;
however, those all seem like too much of a change for the value.  I'm fairly
confident that the *[] form would allow for attributes to optionally be put
on the same line as the def.  This would be LL1, but perhaps the Python
parser has other restrictions that I'm unaware of, i.e.

*[java_sig("public void meth(int a, int b)"), staticmethod]
def meth(self, a, b):
    pass

or for simple cases

*[staticmethod] def meth(self, a, b):
    pass

-Jim



From pje at telecommunity.com  Tue Apr  6 19:32:53 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr  6 19:35:04 2004
Subject: [Python-Dev] PEP 318: How I would implement decorators 
In-Reply-To: <E1BAylD-000KKC-7j@smtp.npsis.com>
Message-ID: <5.1.1.6.0.20040406191201.02228370@telecommunity.com>

At 03:09 PM 4/6/04 -0700, Jim Hugunin wrote:

>1. Use a "decorate" method instead of a call to avoid the worst problems
>like [returns].

Mike Pall actually suggested this recently, although he called the method 
'__decorate__' and proposed that it use a type slot.


>   This has the added benefit in my eyes of making it a little
>harder to write small decorators and thus encouraging people to think a
>little more before writing these meta-programs.

Actually, it's pretty trivial to write a 'decorator_factory' decorator that 
you could then use to make it easy again, i.e.:

[decorator_factory]
def returns(typ):
     ...

So, the approach won't actually discourage people from writing small 
decorators.  :)



>3. Provide a clear set of guidelines for good decorators, i.e.
>   a. decorators should not have side effects outside of the class/function

This is a use case that people have explicitly requested.  IIRC, there have 
been much better examples than 'atexit' that have been posted here.


>   b. decorators should return the same kind of thing they are given

I think it's probably more appropriate to say that decorators should verify 
that their input is something useful to them, and return a helpful error 
message if not.



>+ sys._getframe()
>
>There should be a consensus on whether or not decorators like 'generic' are
>a good or a bad idea.  If they're a good idea then the decorate method
>should get an extra argument exposing the local namespace so that
>sys._getframe() won't be required.  I'd be in favor of adding that argument
>to decorate.

I'm +1 on adding the argument, if there's a decoration method, be it 
'decorate' or '__decorate__'.  As long as decorators are plain functions, 
though, I think the extra argument makes interoperability harder.  By the 
way, it's not clear whether decorators in the general case might not also 
require the active globals, in addition to the active locals.


>+ classmethod and staticmethod
>
>These two decorators are a big problem because they don't return the same
>kind of thing as they are given.  This means they always have to be the last
>decorator applied in any list.  That kind of restriction is never a good
>thing.
>
>Personally, I'd like to see these changed from returning a new object to
>instead setting an attribute on the existing function/method object.  Then
>the method object could use this attribute to implement its __get__ method
>correctly.  This would make these decorators compose very nicely with all of
>the other decorators; however, it would be a fairly major change to how
>Python handles static and class methods and I'm doubtful that would fly.

This issue will apply to any decorator that creates a descriptor, so 
kludging functions for this aren't going to help any other kinds of 
descriptor-making decorators.  But, descriptors (other than functions) 
aren't usually callable, so as long as decorators that expect callables 
check for callability, I don't think this is too big of a deal.


From pje at telecommunity.com  Tue Apr  6 19:43:08 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr  6 19:43:25 2004
Subject: [Python-Dev] PEP 318: More examples of decorator use
In-Reply-To: <E1BAykW-000K2A-AP@smtp.npsis.com>
Message-ID: <5.1.1.6.0.20040406193816.0223b7c0@telecommunity.com>

At 03:08 PM 4/6/04 -0700, Jim Hugunin wrote:
>*Some examples that I'd like to see in an anti-patterns list
>
>onexit from the PEP.  I don't think decorators should have side-effects
>visible outside of the class/function itself.  This seems MUCH better
>accomplished with the explicit call to the registration function.
>
>singleton from the PEP.  Whenever possible, decorators should return the
>same kind of thing that they are given.  This code seems much less clear
>than explicitly creating a singleton instance.
>
>propget, propset, propdel from Edward Loper via Guido.  To me, these violate
>the principle of modularity.  I much prefer the explicit property
>declaration that puts all of the parts together.
>
>   class C(object):
>      def getX(self): return self.__x
>      def setX(self, x): self.__x = x
>      x = property(getX, setX)

Hm.  IIRC, Visual Basic defines "property set", "property let", and 
"property get" methods in a similar fashion to the propget/propset/propdel 
mechanism.  What does C# use?  Doesn't it have such methods also?

Also, most languages that I've encountered that offer generic functions, 
signature overloading, or other similar dispatch mechanisms use variations 
of defining the same function/method over and over again, each variation 
with the same name, just a different signature, matching pattern, etc.

A bigger problem with propget et al is that they don't follow a sensible 
inheritance mechanism.  That is, you can't inherit the get method while 
overriding the set method.  But this is really more a limitation of the way 
Python descriptors work, than any issue with decorators.


From raymond.hettinger at verizon.net  Tue Apr  6 21:22:19 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Apr  6 21:23:25 2004
Subject: [Python-Dev] More imformative iterator representations
Message-ID: <005101c41c3e$cdded760$e841fea9@oemcomputer>

Armin's note reminded me that the only thing I do not like about using
iterators is not being able to see their contents at the interactive
prompt.  So, I posted an idea about changing the print part of the
read-eval-print loop to display more information:

>>> enumerate('abcdefgh') 
<enumerate object at 0x401a102c:  (0, 'a') (1, 'b') (2, 'c') ...>

There a couple of problems with the approach.  One is how to identify an
iterator object -- the presence of __iter__() and next() is a good but
not perfect indicator.

The other issue was trying to leave the iterator undisturbed while
fetching the first three entries.  The least imperfect solution I found
was to replace the iterator with an equivalent object:
firstfew=list(islice(it, 3)); it=chain(firstfew, it).  Unfortunately,
now the "it" variable is a chain object rather than the original
iterator and the object id is different.

An alternative to the read-eval-print approach is providing a custom
__repr__() method for the builtin iterators.  The good news is that this
approach is very clean.  The bad news is that it won't help user defined
iterators or builtin iterators like enumerate() that do not know their
future until next() is called.  So, this approach is only applicable to
a few iterators; it is no problem for lists, tuples, dictionaries, sets,
sequence iterators, xrange iterator objects, reversed objects,
itertools.repeat(), itertools.count(), and collections.deque()
iterators.

Eventhough this solution cannot be applied uniformly for all iterators,
I think is worth considering for places where it does apply:

    >>> reversed('abcdefgh')
    reversed(['h', 'g', 'f', ...])

This could make iterators more pleasant to use interactively.  What do
you guys think?



Raymond Hettinger



Q&A:
---

Q:  What if .next() is labor intensive to compute?
A:  It is only being proposed for objects where the entries are
immediately accessible.


Q:  Are there any other candidate output formats?
A:  Yes, we could add length information for a more mathematical style
output:

    reversed(['h', 'g', 'f', ... to 8 objects])


Q:  What if .next() returns three objects whose representations consume
a lot of visual space?
A:  This is not unique to iterators.  The same issue occurs to a larger
extent when displaying lists.  It is sometimes a PITA but usually not a
problem.


Q:  What prevents enumerate(), imap(), izip(), etc. from implementing
the more informative representations?
A:  They could, but it is messy to go through copy=list(islice(it,3))
and it=chain(copy,it) for every input.


Q:  What if calling next() initiates some sort of activity that we
wished to defer until later?
A:  This would be a rare situation for interactive work, but if known to
be an issue, printing the repr() is easily avoided by making an
assignment:  _ = launch1000ships().  


Q:  Do you really want this?
A:  It is nice to have but not essential.


From greg at cosc.canterbury.ac.nz  Tue Apr  6 21:33:21 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Apr  6 21:33:41 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>
Message-ID: <200404070133.i371XL8d005666@cosc353.cosc.canterbury.ac.nz>

> How did the class react to floating-point?  Seeing behaviour like this:
> 
>     >>> 3.3
>     3.2999999999999998
>     >>>
> 
> confused and frightened them, and continues to confuse and frighten
> almost everyone i teach. ... Every time this happens, the teaching is
> derailed and i am forced to go into an explanation of binary
> floating-point to assuage their fears.

This is an argument for making some form of decimal floating
point the default in Python, with binary FP an option for
advanced users -- not an argument for trying to hide the fact
that binary FP is being used.

Because you *can't* hide that fact completely, and as has
been pointed out, it *will* rear up and bite these people
eventually. It's much better if that happens early, while there
is someone on hand who understands the issues and can guide
them gently through the shock-and-awe phase.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From aahz at pythoncraft.com  Tue Apr  6 21:52:51 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr  6 21:52:55 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: <1081265903.30379.92.camel@anthem.wooz.org>
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
	<1081257431.30379.22.camel@anthem.wooz.org>
	<20040406141615.GD27004@panix.com>
	<1081265903.30379.92.camel@anthem.wooz.org>
Message-ID: <20040407015251.GA15148@panix.com>

On Tue, Apr 06, 2004, Barry Warsaw wrote:
> On Tue, 2004-04-06 at 10:16, Aahz wrote:
>> 
>> Rationale for Absolute Imports
>> ==============================
>> 
>> In Python 2.3 and earlier, if you're reading a module located inside a
>> package, it is not clear whether ::
>> 
>>     import foo
>> 
>> refers to a top-level module or to another module inside the package.
>> As Python's library expands, more and more old modules suddenly shadow
>> builtin modules by accident.  
> 
> As Python's library expands, more and more existing package internal
> modules suddenly shadow standard library modules by accident.

Done.  Seems that the final remaining issue is the meaning of "." inside
__init__.py -- does it refer to the current package or the parent?
Guido, if you don't want to Pronounce, I'll post to c.l.py requesting
feedback.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From greg at cosc.canterbury.ac.nz  Tue Apr  6 21:58:33 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Apr  6 21:58:59 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <002301c41be9$2dbedeb0$6402a8c0@arkdesktop>
Message-ID: <200404070158.i371wXKk005694@cosc353.cosc.canterbury.ac.nz>

Andrew Koenig <ark-mlist@att.net>:

> The reason is that the first example does arithmetic, and it is easy
> to explain that floating-point arithmetic is not completely accurate.

Then how would you explain this to them:

>>> 1.1 < 1.1000000000000001
False

There's no arithmetic being done there.

And if you're thinking of something like "floats don't have that
many digits of precision", you then need to explain

>>> 1.2 < 1.2000000000000001
True

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From fumanchu at amor.org  Tue Apr  6 22:00:31 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Tue Apr  6 22:02:39 2004
Subject: [Python-Dev] More imformative iterator representations
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561FC3@opus.amorhq.net>

Raymond Hettinger wrote:
> Armin's note reminded me that the only thing I do not like about using
> iterators is not being able to see their contents at the interactive
> prompt.  So, I posted an idea about changing the print part of the
> read-eval-print loop to display more information:
> 
> >>> enumerate('abcdefgh') 
> <enumerate object at 0x401a102c:  (0, 'a') (1, 'b') (2, 'c') ...>
> 
> There a couple of problems with the approach.  One is how to 
> identify an
> iterator object -- the presence of __iter__() and next() is a good but
> not perfect indicator.

Oh, I don't know. The Library Ref makes it pretty clear:

"The iterator objects themselves are required to support the following
two methods, which together form the iterator protocol: 
__iter__() ...
next() "

If not perfect, it's still a reasonable assumption. Since there's no
Iterator superclass from which to inherit, you're going to have to
rewrite each repr by hand anyway, and therefore can decide each one
independently. Are there reasonable iterator objects which don't follow
the "iterator protocol"?

> An alternative to the read-eval-print approach is providing a custom
> __repr__() method for the builtin iterators.

+1.

> Q:  Are there any other candidate output formats?
> A:  Yes, we could add length information for a more mathematical style
> output:
> 
>     reversed(['h', 'g', 'f', ... to 8 objects])

The very presence of __len__ might be a strong argument for providing
the new repr, and its absence an indicator to avoid such.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From bob at redivi.com  Tue Apr  6 22:13:40 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Apr  6 22:09:45 2004
Subject: [Python-Dev] More imformative iterator representations
In-Reply-To: <DE1CF2B4FEC4A342BF62B6B2B334601E561FC3@opus.amorhq.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E561FC3@opus.amorhq.net>
Message-ID: <2FAA1373-8839-11D8-84F3-000A95686CD8@redivi.com>


On Apr 6, 2004, at 10:00 PM, Robert Brewer wrote:

> Raymond Hettinger wrote:
>> Q:  Are there any other candidate output formats?
>> A:  Yes, we could add length information for a more mathematical style
>> output:
>>
>>     reversed(['h', 'g', 'f', ... to 8 objects])
>
> The very presence of __len__ might be a strong argument for providing
> the new repr, and its absence an indicator to avoid such.

This is essentially getting back to what Armin was saying.. if 
iterators act more like lists, they will be easier to user and will 
open up some new doors for optimization potential.  Adding a way to get 
the length of some iterators essentially makes them list-like or 
tuple-like.

-bob


From aahz at pythoncraft.com  Tue Apr  6 22:28:02 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr  6 22:28:05 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404060249.i362nwXn003846@cosc353.cosc.canterbury.ac.nz>
References: <20040405175117.GA21587@mems-exchange.org>
	<200404060249.i362nwXn003846@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040407022802.GA17603@panix.com>

On Tue, Apr 06, 2004, Greg Ewing wrote:
>
> Would it be unreasonable to define "." as meaning the current
> *package*, rather than the current *module* -- i.e. the package itself
> if the importer is a package, and the containing package if it's a
> plain module?

Not unreasonable, but I think it complicates the explanation.  I'm
basically -0 on having "." in __init__.py refer to the current package
instead of the parent; we'll see what Guido says.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From tim.one at comcast.net  Tue Apr  6 23:10:16 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Apr  6 23:10:35 2004
Subject: [Python-Dev] Expert floats
In-Reply-To: <Pine.LNX.4.58.0404060704470.18028@server1.LFW.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEDLKBAB.tim.one@comcast.net>

[Tim]
>> I want marshaling of fp numbers to give exact (not approximate)
>> round-trip equality on a single box, and across all boxes supporting
>> the 754 standard where C maps "double" to a 754 double.

[Ping]
> That is a valuable property.  I support it and support Python
> continuing to have that property.

That's good, since nobody opposes it <wink>.

> I hope it has been made quite clear by now that this property does not
> constrain how numbers are displayed by the interpreter in
> human-readable form.  The issue of choosing an appropriate string
> representation of a number is unaffected by the desire for the above
> property.

...

> I think we *have* made progress.  Now we can set aside the red-herring
> issue of platform-independent serialization and focus on the real
> issue: human-readable string representation.

I don't think that's "the" real issue, but it is one of several.

...

>> I'm the one who> has fielded most newbie questions about fp since
>> Python's beginning, and I'm very happy with the results of changing
>> repr() to produce 17 digits.

> Now you are pulling rank.

I'm relating my experience, which informs my beliefs about these issues more
than any "head argument".

> I cannot dispute your longer history and greater experience with Python;
> it is something i greatly admire and respect.  I also don't know your
> personal experiences teaching Python.

In person, mostly to hardware geeks and other hardcore software geeks.  On
mailing lists and newsgroups, to all comers, although I've had decreasing
time for that as the years drag on.

> But i can tell you my experiences.  And i can tell you that i have
> tried to teach Python to many people, individually and in groups.  I
> taught a class in Python at UC Berkeley last spring to 22 people who
> had never used Python before.  I maintained good communication with
> the students and their feedback was very positive about the class.
>
> How did the class react to floating-point?  Seeing behaviour like
> this:
>
>     >>> 3.3
>     3.2999999999999998
>     >>>
>
> confused and frightened them, and continues to confuse and frighten
> almost everyone i teach.

Sorry, but so long as they stick to binary fp, stuff like that can't be
avoided, even using "your rule" (other examples of that were posted today,
and I won't repeat them again here).  I liked Python's *former* rule
myself(repr rounds to 12 significant digits), and would like it much better
than "shortest possible" (which still shows tons of crap I usually don't
care about) most days for my own uses.

That's a real problem Python hasn't addressed:  its format decisions are
often inappropriate and/or undesirable (both can differ by app and by
audience and by object type), and there are insufficient hooks for
overriding these decisions.  sys.displayhook goes a bit in that direction,
but not far enough.

BTW, if your students *remain* confused & frightened, it could be you're not
really trying to explain binary fp reality to them.

> (The rare exceptions are the people who have done lots of computational
> work before and know how binary floating-point representations work.)
> Every time this happens, the teaching is derailed and i am forced to go
> into an explanation of binary floating-point to assuage their fears.

Then they *don't* remain confused & frightened?  Great.  Then they've been
educated.  How long can it take to read the Tutorial Appendix?  It's well
worth however many years it takes <wink>.

> Remember, i am trying to teach basic programming skills.  How to solve
> problems; how to break down problems into steps; what's a subroutine;
> and so on.  Aside from this floating-point thing throwing them off,
> Python is a great first language for new programmers.  This is not the
> time to talk about internal number representation.

Use Decimal instead.  That's always been the best idea for newbies (and for
most casual users of floating-point, newbie or not).

> I am tired of making excuses for Python.  I love to tell people about
> Python and show them what it can do for them.  But this floating-point
> problem is embarrassing.  People are confused because no other system
> they've seen behaves like this.

If you're teaching "basic programming skills", what other systems have they
seen?  Hand calculators for sure -- which is why they should use Decimal
instead.  Virually nothing about it will surprise them, except the
liberating ability to crank up the precision.

> Other languages don't print their numbers like this.  Accounting
> programs and spreadsheets don't print their numbers like this.

I don't care -- really.  I'm thoroughly in agreement with Kahan on this;
see, e.g., section "QPRO 4.0 and QPRO for Windows" in

    http://www.cs.berkeley.edu/~wkahan/MktgMath.pdf

    ... the reader can too easily misinterpret a few references to 15
    or 16 sig. dec of precision as indications that no more need be said
    about QPRO's arithmetic.  Actually much more needs to be said because
    some of it is bizarre.

    Decimal displays of Binary nonintegers cannot always be WYSIWYG.

    Trying to pretend otherwise afflicts both customers and implementors
    with bugs that go mostly misdiagnosed, so ?fixing? one bug merely
    spawns others.

    ...

    The correct cure for the @ROUND and @INT (and some other) bugs is not
    to fudge their argument but to increase from 15 to 17 the maximum
    number of sig. dec. that users of QPRO may see displayed.

    But no such cure can be liberated from little annoyances:
    [snip things that make Ping's skin crawl about Python today]

    ...

    For Quattro?s intended market, mostly small businesses with little
    numerical expertise, a mathematically competent marketing follow-
    through would have chosen either to educate customers about binary
    floating-point or, more likely, to adopt decimal floating-point
    arithmetic even if it runs benchmarks slower.

The same cures are appropriate for Python.

> Matlab and Maple and Mathematica don't print their numbers like this.

Those are designed for experts (although Mathematica pretends not to be).

> Only Python insists on being this ugly.  And it screws up the most common
> way that people first get to know Python -- as a handy interactive
> calculator.
>
> And for what?  For no gain at all -- because when you limit your focus
> to the display issue, the only argument you're making is "People
> should be frightened."  That's a pointless reason.

Sorry, that's an absurd recharacterization, and I won't bother responding to
it.  If you really can't see any more to "my side" of the argument than that
yet, then repeating it another time isn't going to help.

So enough of this.  In what time I can make for "stuff like this", I'm going
to try to help the Decimal module along instead.  Do what you want with
interactive display of non-decimal floats, but do try to make it flexible
instead of fighting tooth and nail just to replace one often-hated fixed
behavior with another to-be-often-hated fixed behavior.

...

> Not everyone runs into floating-point corner cases.  In fact, very few
> people do.

Heh.  I like to think that part of that has to do with the change to repr()!
As I've said many times before, we *used* to get reports of a great variety
of relatively *subtle* problems due to binary fp behavior from newbies; we
generally get only one now, and the same one every time.  They're not
stupid, Ping, they just need the bit of education it takes to learn
something about that expensive fp hardware they bought.

> I have never encountered such a problem in my entire history of using
> Python.

Pride goeth before the fall ...

> And if you surveyed the user community, i'm sure you would find that
> only a small minority cares enough about the 17th decimal place for the
> discrepancy to be an issue.

The result of int() can change by 1 when the last bit changes, and the
difference between 2 and 3 can be a disaster -- see Kahan (op. cit.) for a
tale of compounded woe following from that one.  Aahz's recent example of a
loop going around one time more or less "than expected" used to be very
common, and is the same thing in a different guise.  It's like security that
way:  nobody gives a shit before they get burned, and then they get livid
about it.  If a user believes 0.1 is one tenth, they're going to get burned
by it.

> ...
> You say it's better for people to get "bitten early".  What's better:
> everyone suffering for a problem that will never affect most of them,
> or just those who care about the issue having to deal with it?

The force of this is lost because you don't have a way to spare users from
"unexpected extra digits" either.  It comes with the territory!  It's
inherit in using binary fp in a decimal world.  All you're really going on
about is showing "funny extra digits" less often -- which will make them all
the more mysterious when they show up.  I liked the former
round-to-12-digits behavior much better on that count.  I expect to like
Decimal mounds better on all counts except speed.


From greg at cosc.canterbury.ac.nz  Tue Apr  6 23:19:39 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Apr  6 23:19:55 2004
Subject: [Python-Dev] PEP 318: How I would implement decorators
In-Reply-To: <E1BAylD-000KKC-7j@smtp.npsis.com>
Message-ID: <200404070319.i373JdQF005829@cosc353.cosc.canterbury.ac.nz>

Jim Hugunin <lists@hugunin.net>:

> I like Sam's proposal to add a '*' in front of the list.
> 
> *[java_sig("public void meth(int a, int b)"), staticmethod]
> def meth(self, a, b):
>     pass

Would this have any implications if we were to decide to
allow * in unpacking one day, i.e.

  a, b, *c = some_tuple

It might become legal for an expression to start with *,
then.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Tue Apr  6 23:52:29 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Apr  6 23:53:10 2004
Subject: [Python-Dev] More imformative iterator representations
In-Reply-To: <2FAA1373-8839-11D8-84F3-000A95686CD8@redivi.com>
Message-ID: <200404070352.i373qTwh005897@cosc353.cosc.canterbury.ac.nz>

Bob Ippolito <bob@redivi.com>:

> This is essentially getting back to what Armin was saying.. if
> iterators act more like lists, they will be easier to user and will
> open up some new doors for optimization potential.  Adding a way to
> get the length of some iterators essentially makes them list-like or
> tuple-like.

But... if an iterator is sufficiently random-access to be able to
report its length and produce selected items without doing anything
irreversible, wouldn't it make more sense for it to be a (possibly
read-only) sequence rather than an iterator in the first place?

In other words, instead of e.g. dict.iteritems() there should be
a dict.itemseq() or something that returns a sequence-like view
of the dict.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Tue Apr  6 23:58:56 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Apr  6 23:59:14 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040407022802.GA17603@panix.com>
Message-ID: <200404070358.i373wu6R005907@cosc353.cosc.canterbury.ac.nz>

Aahz <aahz@pythoncraft.com>:

> > Would it be unreasonable to define "." as meaning the current
> > *package*, rather than the current *module* --
> 
> Not unreasonable, but I think it complicates the explanation. 

Not if the explanation simply says "'.' refers to the
current package", and leaves the reader to deduce what
"current package" means when the module is not a
package. :-) [1]

My intuition is that this will actually cause people
less surprises even if they never think about this
subtlety before diving in and using the feature. In
fact they'll probably be more confused if they *do*
think about it, either way...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

Footnotes:
[1] A footnote explaining it would be allowed.

From guido at python.org  Wed Apr  7 00:14:21 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Apr  7 00:16:04 2004
Subject: [Python-Dev] Some comments on PEP 328 (absolute/relative imports)
In-Reply-To: Your message of "Tue, 06 Apr 2004 21:52:51 EDT."
	<20040407015251.GA15148@panix.com> 
References: <s9zwu4wdinb.fsf@salmakis.intevation.de>
	<1081257431.30379.22.camel@anthem.wooz.org>
	<20040406141615.GD27004@panix.com>
	<1081265903.30379.92.camel@anthem.wooz.org> 
	<20040407015251.GA15148@panix.com> 
Message-ID: <200404070414.i374ELc02448@guido.python.org>

> Done.  Seems that the final remaining issue is the meaning of "." 
> inside __init__.py -- does it refer to the current package or the
> parent?  Guido, if you don't want to Pronounce, I'll post to c.l.py
> requesting feedback.

It ought to refer to the current package.  After all, in __init__.py,
relative imports (today) also work the same way as inside other
modules of that package.

__init__.py is funny that way (funny strange, not funny ha ha :-): its
__dict__ namespace is that of the package, i.e. the parent of the
package's submodules; but its location in the package tree is
equivalent to that of a submodule of the package.  That's just how it
is, and once you think about it long enough that's the only way it
*can* be.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From anthony at interlink.com.au  Wed Apr  7 00:29:47 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Apr  7 00:29:56 2004
Subject: [Python-Dev] MS open sources WiX
Message-ID: <407383BB.6050907@interlink.com.au>

Relevant to MvL's new MSI installer work:

http://blogs.msdn.com/robmen/archive/2004/04/05/107709.aspx

Windows Installer XML (WiX) toolset has released as Open Source on
SourceForge.net

The Windows Installer Xml (WiX) toolset (pronounced ?wicks toolset?) 
appears to have finished propagating around the SourceForge.net CVS 
servers, so I can finally start writing this blog entry.  As promised in 
my blog here, here, here, here, and here the WiX toolset and all of its 
source code has been released so that you can build Windows Installer 
databases (MSI and MSM files) the same way most groups inside Microsoft 
do.  However, a funny thing happened on the way to the forum.  WiX 
became the first project from Microsoft to be released under an OSS 
approved license, namely the Common Public License.

Before everyone gets sidetracked by the Open Source implications, let?s 
talk about exactly what WiX is.  WiX is a toolset composed of a 
compiler, a linker, a lib tool and a decompiler.  The compiler, called 
candle, is used to compile XML source code into object files that 
contain symbols and references to symbols.  The linker, called light, is 
fed one or more object files and links the references in the object 
files to the appropriate symbols in other object files.  Light is also 
responsible for collecting all of the binaries, packaging them 
appropriately, and generating the final MSI or MSM file.  The lib tool, 
called lit, is an optional tool that can be used to combine multiple 
object files into libraries that can be consumed by light.  Finally, the 
decompiler, called dark, can take existing MSI and MSM files and 
generate XML source code that represents the package.
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From aahz at pythoncraft.com  Wed Apr  7 00:53:18 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Apr  7 00:53:25 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404070358.i373wu6R005907@cosc353.cosc.canterbury.ac.nz>
References: <20040407022802.GA17603@panix.com>
	<200404070358.i373wu6R005907@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040407045318.GB173@panix.com>

On Wed, Apr 07, 2004, Greg Ewing wrote:
> Aahz <aahz@pythoncraft.com>:
>> Greg:
>>>
>>> Would it be unreasonable to define "." as meaning the current
>>> *package*, rather than the current *module* --
>> 
>> Not unreasonable, but I think it complicates the explanation. 
> 
> Not if the explanation simply says "'.' refers to the current
> package", and leaves the reader to deduce what "current package" means
> when the module is not a package. :-) [1]

All right, let's try this:

    A leading dot indicates a relative import, starting with the current
    package.  Two or more leading dots point to the parent(s) of the
    current package, one level per dot after the first.

Then the examples should explain the meaning well enough.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From fumanchu at amor.org  Wed Apr  7 01:27:55 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Wed Apr  7 01:30:03 2004
Subject: [Python-Dev] More imformative iterator representations
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561FC4@opus.amorhq.net>

Bob Ippolito wrote:
> > Raymond Hettinger wrote:
> >> Q:  Are there any other candidate output formats?
> >> A:  Yes, we could add length information for a more 
> mathematical style
> >> output:
> >>
> >>     reversed(['h', 'g', 'f', ... to 8 objects])
> >
> > The very presence of __len__ might be a strong argument for 
> providing
> > the new repr, and its absence an indicator to avoid such.
> 
> This is essentially getting back to what Armin was saying.. if 
> iterators act more like lists, they will be easier to user and will 
> open up some new doors for optimization potential.  Adding a 
> way to get 
> the length of some iterators essentially makes them list-like or 
> tuple-like.

Agreed. I don't mind the crux of Armin's idea, I was reacting more to
applying such optimizations to enumerate() in particular. Given
sufficient clarification like the above, it'd be easier to swallow.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From fumanchu at amor.org  Wed Apr  7 01:50:00 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Wed Apr  7 01:52:07 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561FC5@opus.amorhq.net>

[Guido van Rossum]
> > ...while far from perfect, in
> > terms of readability setting function attributes in a 
> decorator is so
> > much better than setting them after the function has been defined,
> > that I believe we have no choice but to provide it.

[Me]
> The more I read statements like this, the more I think of replacing:
> 
> def foo(arg1, arg2):
>     pass
> foo = classmethod(foo)
> foo.attrA = 3
> 
> ...with code like:
> 
> foo = classmethod()
> foo.attrA = 3
> def foo(arg1, arg2):
>     pass
> 
> Why would such additional features not be standard attributes of
> function objects?
>8 
> All in all, I'd prefer that function decorators and attributes use
> simple, known techniques like assignment and callables (as 
> shown above),
> than become monolithic, crystallized all-in-one statements like:
> 
> def foo(arg1 as str, arg2 as int) returns str [classmethod] 
> {attrA: 3}:
> 
> I understand the desire to keep all those bits near each other on the
> page, but stronger is my desire to make them independent 
> statements. If
> we could bind a "blank" function object and supply its actual 
> codeblock
> in a later statement, the rest falls magically into place, IMO.

I sat down and banged out a metaclass solution, just to see if I *could*
get the syntax I wanted in the example Shoe class, and to see what might
be required behind the scenes. I put Shoe and all the decorator junk in
one module to make the syntax look as clean as if Decorable, etc. were
builtins.

Basically, Decorable (the metaclass) looks at all attributes of the Shoe
class; if there's an attribute "x" and an attribute "x_suite", then
Shoe.x gets rebound as x(x_suite). The example provides classmethod and
arbitrary function attributes. There are ways to clean it up, of course;
this was a quick and dirty proof-of-concept. Rather than playing with
"_suite", it should probably either reuse the same name "x" or use a
token other than def. I also arbitrarily chose to allow a tuple or list
for the decorator (but didn't demonstrate it).


----decorate.py----

import new

class Decorable(type):
    def __init__(cls, name, bases, dct):
        type.__init__(cls, name, bases, dct)
        for attr in dct:
            if attr + "_suite" in dct:
                # Wrap the attr_suite in the attr decorator.
                inner_func = getattr(cls, attr + "_suite")
                decorator = getattr(cls, attr)
                if isinstance(decorator, (tuple, list)):
                    wrapped_func = inner_func
                    for item in decorator:
                        wrapped_func = item.decorate(wrapped_func)
                else:
                    wrapped_func = decorator.decorate(inner_func)
                setattr(cls, attr, wrapped_func)
                
                # [Optional] Unbind the inner function from the class.
                delattr(cls, attr + "_suite")


class Decorator(object):
    def __init__(self):
        self.attributes = {}
    
    def __setattr__(self, key, value):
        if key == 'attributes':
            self.__dict__['attributes'] = value
        else:
            self.__dict__['attributes'][key] = value
    
    def decorate(self, function):
        for key, value in self.attributes.iteritems():
            function.__setattr__(key, value)
        return function


class clsmethod(Decorator):
    def decorate(self, function):
        newfunc = new.instancemethod(function, function.im_class(),
function.im_class())
        return super(clsmethod, self).decorate(newfunc)


class Shoe(object):
    __metaclass__ = Decorable
    
    sized_shoe = clsmethod()
    sized_shoe.author = "Robert E Brewer"
    def sized_shoe_suite(cls, size):
        """Make a new Shoe with a 'size' attribute."""
        newShoe = Shoe()
        newShoe.size = size
        return newShoe

----end decorate.py----


>>> import decorate
>>> dir(decorate.Shoe)
['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__',
'__hash__', '__init__', '__metaclass__', '__module__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__',
'__weakref__', 'sized_shoe']

>>> decorate.Shoe.sized_shoe.author
'Robert E Brewer'

>>> s = decorate.Shoe.sized_shoe(3)
>>> s
<decorate.Shoe object at 0x0115CAD0>
>>> s.size
3


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From python at rcn.com  Wed Apr  7 05:16:25 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Apr  7 05:17:20 2004
Subject: [Python-Dev] More imformative iterator representations
In-Reply-To: <200404070352.i373qTwh005897@cosc353.cosc.canterbury.ac.nz>
Message-ID: <00b001c41c81$00620800$e841fea9@oemcomputer>

> But... if an iterator is sufficiently random-access to be able to
> report its length and produce selected items without doing anything
> irreversible, wouldn't it make more sense for it to be a (possibly
> read-only) sequence rather than an iterator in the first place?
> 
> In other words, instead of e.g. dict.iteritems() there should be
> a dict.itemseq() or something that returns a sequence-like view
> of the dict.

Trees, dicts, and sets all know their length and can iterate
sequentially without having random access.  PyDict_Next() makes it easy
to find the first 3 keys, but there is no general purpose way of finding
the nth key without looping over them all.


Raymond


From mcherm at mcherm.com  Wed Apr  7 10:51:49 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Apr  7 10:51:39 2004
Subject: [Python-Dev] Expert floats
Message-ID: <1081349509.407415852f0c3@mcherm.com>

Ka-Ping writes:
> How did the class react to floating-point?  Seeing behaviour like this:
> 
>     >>> 3.3
>     3.2999999999999998
>     >>>
> 
> confused and frightened them, and continues to confuse and frighten
> almost everyone i teach.

As well it should, until they understand it. I'll come back to this
point in a moment.

> Everything in Python -- everything in computers, in fact -- is a *model*.
> We don't expect the model to be perfectly accurate or to be completely
> free of limitations.  IEEE 754 happens to be the prevalent model for
> the real number line.  We don't print every string with a message after
> it saying "WARNING: MAXIMUM LENGTH 4294967296", and we shouldn't do
> the same for floats.

Now here is where I disagree with you quite strongly. I am an experienced
programmer, and I *DO* expect my computer programs to be "perfectly 
accurate". If I calculate 217 + 349, I expect to get 566 each and every
time. If it sometimes came out to be 567 instead, I would throw a fit!

I also expect my program to be free of limitations. For instance, if
I calculate 2147483640 + 9, I EXPECT to get 2147483649. In Python, I
get 2147483649L instead... a minor wart that I can easily live with.
If I am coding in C, I will instead get -2147483646, which is WAY off.
I UNDERSTAND why this happens in C, and I am forced to live with it,
but I don't LIKE or EXPECT it, and to be honest, trying to work around
this limitation gives me headaches some days!

For another example, I realize that if I work with a large enough data
set it won't fit into my machine's memory. But I would be very
disapointed if someone designed a programming language that limited
data sizes because of this. When I have a problem, I expect to be able
to go out and buy more memory (or upgrade my machine, my OS, or
whatever component is the limiting factor) and have things "just work".

And it's not just me... computer newbies have an even STRONGER belief
that computers are "always accurate" than I do! They don't understand
the degree to which human error and bugs can influence results, and
most people figure that if the computer calculates some numbers, it's
answer is guaranteed to be right.

So how can I stand to use floating point numbers? Well, the answer is
that I am aware of the limitations of floating point calculations. I
would never hesitate to use floating point to plot a graph, and I would
never choose to use floating point to perform financial calculations.
I'm not sufficiently well-versed in the details of floating point
behavior to always know when floating point would be acceptable to use,
so I take a conservative approach and use it only when I know that it
is safe to use.

And I believe that this is exactly what your students ought to do.

> Not everyone runs into
> floating-point corner cases.  In fact, very few people do.  I have never
> encountered such a problem in my entire history of using Python.  And if
> you surveyed the user community, i'm sure you would find that only a
> small minority cares enough about the 17th decimal place for the
> discrepancy to be an issue.

That's simply untrue. Intel conducted (unintentionally) an experiment
in just this question in 1994: try googling for "pentium bug". The 
conditions under which that bug affected results were far more obscure 
than the "corner cases" for floating point. And yet there was a huge 
hue and cry, and Intel was forced to replace tens of thousands of CPUs.
Clearly, people DO care about tiny discrepencies.

So, to come back to my point at the beginning, let me tell you what *I*
say when teaching Python to beginners and they complain about the odd
display of floating point in Python. I tell them that when computers
work with decimals, they store things internally in Binary (everyone
"knows" that computers work only in 1s and 0s), and this causes tiny
rounding errors. I tell them that Python makes a point of SHOWING you
those rounding errors, because it doesn't want to lie to you.

Then, (depending on my audience) I usually follow up with a quick demo.
I launch Microsoft Excel, and in the top cell I enter "-1". Then I
enter "=1/9" (don't forget the "=" or Excel will treat it as a date)
in each of the next nine cells. In the next cell I type "=sum(A1:A10)"
but before pressing enter, I ask the class what the sum is. As soon
as they're convinced it will be zero, I display the result. After that
demo, people are usually much happier about Python which "looks funny",
but "doesn't lie about its own mistakes".

I firmly believe that ANYONE (even beginners) who uses floating point
needs to understand that its not exact. They don't need to know the
details, but they need to grasp the concept that a computer is NOT
an Oracle, and this demonstration helps.

-- Michael Chermside


From kbk at shore.net  Wed Apr  7 21:00:13 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Apr  7 21:00:17 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200404080100.i3810C18003532@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  251 open ( +8) /  2364 closed ( +2) /  2615 total (+10)
Bugs    :  738 open ( +2) /  3972 closed (+12) /  4710 total (+14)
RFE     :  132 open ( +2) /   124 closed ( +0) /   256 total ( +2)

New / Reopened Patches
______________________

Port tests to unittest (Part 2)  (2003-05-13)
       http://python.org/sf/736962  reopened by  doerwalter

PEP 309 unit tests  (2004-04-07)
       http://python.org/sf/931010  opened by  Peter Harris 

struct.pack() on 64bit architectures  (2004-03-30)
       http://python.org/sf/925932  opened by  Aleksander Piotrowski 

Patch to setup.py to run on x86_64 Linux  (2004-03-30)
       http://python.org/sf/926209  opened by  Christopher Petrilli 

unichr(wide) error on ucs2 build  (2004-03-31)
CLOSED http://python.org/sf/926375  opened by  Hye-Shik Chang 

Implementation for PEP 318 (Guido's version)  (2004-03-31)
       http://python.org/sf/926860  opened by  Guido van Rossum 

Option to propagate errors from pkgutil  (2004-03-31)
       http://python.org/sf/927232  opened by  Jp Calderone 

Improvements to Bluetooth support  (2004-04-04)
       http://python.org/sf/929192  opened by  Paul Clifford 

Remove ROT_FOUR  (2004-04-04)
       http://python.org/sf/929502  opened by  Michael Chermside 

PEP 309 reference implementation  (2004-04-07)
       http://python.org/sf/931005  opened by  Peter Harris 

PEP 309 LaTeX documentation  (2004-04-07)
       http://python.org/sf/931007  opened by  Peter Harris 

Patches Closed
______________

unichr(wide) error on ucs2 build  (2004-03-31)
       http://python.org/sf/926375  closed by  perky

Updates to the Misc/RPM spec file.  (2004-03-27)
       http://python.org/sf/924497  closed by  loewis

bugfix for [Bugs-850964](optparse)  (2003-12-01)
       http://python.org/sf/851902  closed by  fdrake

New / Reopened Bugs
___________________

re.sub on u'' gives NON-unicode ''  (2004-03-30)
       http://python.org/sf/926075  opened by  Alex Martelli 

pdb sometimes sets breakpoints in the wrong location  (2004-03-30)
       http://python.org/sf/926369  opened by  Ilya Sandler 

socket timeouts + Ctrl-C don't play nice  (2004-03-30)
       http://python.org/sf/926423  opened by  David M. Cooke 

OEM codepage chars in mbcs filenames are misinterpreted  (2004-03-30)
CLOSED http://python.org/sf/926427  opened by  Mike Brown 

(ref-manual) position docstrings in source not documented  (2004-03-31)
       http://python.org/sf/926501  opened by  a.hofkamp 

pdb sometimes sets breakpoints in the wrong location  (2004-03-31)
CLOSED http://python.org/sf/926807  opened by  Ilya Sandler 

Overenthusiastic check in Swap?  (2004-03-31)
       http://python.org/sf/926910  opened by  benson margulies 

Python segfaults when freeing "deep" objects  (2004-03-31)
       http://python.org/sf/927248  opened by  Jp Calderone 

platform.libc_ver() fails on Cygwin  (2004-04-02)
       http://python.org/sf/928297  opened by  George Yoshida 

Python interpreter stalled on _PyPclose.WaitForSingleObject  (2004-04-02)
       http://python.org/sf/928332  opened by  Popov 

Typo in 7.2.1: socket.connect_ex  (2004-04-02)
CLOSED http://python.org/sf/928751  opened by  PieOnCar 

Default encoding harmful  (2004-04-03)
CLOSED http://python.org/sf/928881  opened by  Toni Goeller 

many os.path functions bahave inconsistently  (2004-04-04)
       http://python.org/sf/929316  opened by  Simon Percivall 

Python 2.3.3: socketmodule doesn't build on UnixWare 7.1.3  (2004-04-05)
       http://python.org/sf/929689  opened by  Larry Rosenman 

os.path.realpath can't handle symlink loops  (2004-04-05)
       http://python.org/sf/930024  opened by  A.M. Kuchling 

RecenFilesList mysteriously containing str with NULL bytes  (2004-04-07)
       http://python.org/sf/931336  opened by  Gustavo Tabares 

Bugs Closed
___________

sdist ignores scripts argument in setup  (2003-08-27)
       http://python.org/sf/796042  closed by  fdrake

urllib.urlencode doesn't work for output from cgi.parse_qs  (2003-10-30)
       http://python.org/sf/833405  closed by  myers_carpenter

Python.h must be included first  (2003-11-06)
       http://python.org/sf/837228  closed by  fdrake

OEM codepage chars in mbcs filenames can be misinterpreted  (2004-03-31)
       http://python.org/sf/926427  closed by  loewis

pdb sometimes sets breakpoints in the wrong location  (2004-03-31)
       http://python.org/sf/926807  closed by  isandler

os.py uses #' - undocumented magic?  (2004-03-29)
       http://python.org/sf/925500  closed by  montanaro

Wrong reference for specific minidom methods  (2003-10-29)
       http://python.org/sf/832251  closed by  fdrake

urlopen object's read() doesn't read to EOF  (2003-04-21)
       http://python.org/sf/725265  closed by  fdrake

Section 13.1 HTMLParser documentation error  (2003-08-23)
       http://python.org/sf/793702  closed by  fdrake

optparse: OptionParser.__init__'s "prog" argument ignored  (2003-11-28)
       http://python.org/sf/850964  closed by  fdrake

Typo in 7.2.1: socket.connect_ex  (2004-04-03)
       http://python.org/sf/928751  closed by  nnorwitz

Default encoding harmful  (2004-04-03)
       http://python.org/sf/928881  closed by  lemburg

segmentation fault  (2004-03-03)
       http://python.org/sf/909295  closed by  tim_one

New / Reopened RFE
__________________

"with self:" statement  (2004-04-01)
       http://python.org/sf/927543  opened by  Thomas Dunne 


From janssen at parc.com  Wed Apr  7 21:33:10 2004
From: janssen at parc.com (Bill Janssen)
Date: Wed Apr  7 21:33:40 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: Your message of "Tue, 06 Apr 2004 21:53:18 PDT."
	<20040407045318.GB173@panix.com> 
Message-ID: <04Apr7.183319pdt."58612"@synergy1.parc.xerox.com>

I really wish I'd been paying more attention to 328, which I've just
read.  This proposal seems to break lots and lots and lots of existing
code.  Maybe I'm reading it wrong.

I've written many, many packages that use the form

     import foo

where "foo" is a sibling module in the package.  Or

      from foo import bar

where, again, foo is a sibling to the current module.  Packages that
work like this have been submitted to on-line libraries, printed in
books, etc., etc., etc.

I really hope I'm reading the PEP wrong!  Am I correct in thinking
that come Python 2.4 or 2.5, these will just break if the package
isn't at the top level (reachable from sys.path)?

I was thinking (some months ago) that y'all would come up with an
absolute form instead, something like

      from / import os

or

      import /os

or

      import /email.Message

and a relative form like

      from foo import bar

to do fuzzy and imperfect matching of the module (what we have now),
or

      from ./foo import bar

to do more explicit matching of siblings, or

      from ../baz.foo import bar

or

      from -/baz.foo import bar

to go up a level.  And that the fuzzy and imperfect form would be
eventually deprecated, but never removed.

Bill

From b.scharpf at tesionmail.de  Wed Apr  7 15:46:14 2004
From: b.scharpf at tesionmail.de (Bertram Scharpf)
Date: Wed Apr  7 21:40:17 2004
Subject: [Python-Dev] Python vs. chat (conversation with modem)
Message-ID: <slrnc78mk6.46p.b.scharpf@homer.bertram-scharpf>

Hi,

in simple serial communication, I found a problem that
doesn't arise, when I'm doing exactly the same with the
`chat' program.

I admit my hardware is a little bit special and may have
problems itself. It is an ISDN telephone, Elmeg C200d@ta.
With a normal modem, everything works fine.

For a speech call, my telephone wants me to send a command
'at&b30' before dialing (otherwise the modem functionality
is dialing).

When I send the commands with `chat', the device is
dialing, but when I send them from a Python script, the
communication hangs.

My code is (simplified):


class Modem:
    def __init__( self, device = '/dev/ttyS0'):
        self.modem = open( device, 'w+')
        self.command( 'atz')

    def send( self, cmd):
        print >>self.modem, cmd + '\r'

    def expect( self, patt):
        import select
        ...

    def command( self, cmd):
        self.send( cmd)
        self.expect( 'OK')

    def dial_orig( self, nr):
        self.command( 'at&b30')
        # On this request, the modem won't send an answer.
        self.command( 'atdt,%s;h0z' % nr)
        self.command( 'at&b31')

    def dial_chat( self, nr):
        "That's working!?"
        script += [ repr( ''), 'atz',
                    'OK',      r'at\&b30dt,\\T\;h0z',
                    'OK',      r'\\c']
        chatcmd = '/usr/sbin/chat -vs -T %s %s <%s >%s' % \
            (nr, ' '.join( script), self.modem.name, self.modem.name)
        import os
        os.system( chatcmd)

    dial = dial_chat


(The full script is available at
`http://www.bertram-scharpf.de/dial.py'.)

On the 'atz' an answer of 'OK' is sent back, but not on the
'at&b30' request. `chat' receives an OK. Setting the
termcaps exactly to what chat would have chosen didn't help.

I have found this behaviour in the versions 2.1, 2.2 and 2.3.

  $ uname -srmp
  Linux 2.4.18-bs1 i586 unknown

Do I have any chance to find out what happens?

Bertram


-- 
Bertram Scharpf
Stuttgart, Deutschland/Germany
http://www.bertram-scharpf.de


From tjreedy at udel.edu  Wed Apr  7 22:55:36 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Apr  7 22:55:40 2004
Subject: [Python-Dev] Re: Python vs. chat (conversation with modem)
References: <slrnc78mk6.46p.b.scharpf@homer.bertram-scharpf>
Message-ID: <c52ev6$tte$1@sea.gmane.org>


"Bertram Scharpf" <b.scharpf@tesionmail.de> wrote in message
news:slrnc78mk6.46p.b.scharpf@homer.bertram-scharpf...
> in simple serial communication, I found a problem that ...
> I have found this behaviour in the versions 2.1, 2.2 and 2.3. ...

This appears to be a usage question about current and past Python versions
rather than anything related to the development of future versions.  As
such, it would be better posted to comp.lang.python or the corresponding
mailing list.

tjr




From paul at prescod.net  Thu Apr  8 01:01:56 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Apr  8 01:07:26 2004
Subject: [Python-Dev] More imformative iterator representations
In-Reply-To: <005101c41c3e$cdded760$e841fea9@oemcomputer>
References: <005101c41c3e$cdded760$e841fea9@oemcomputer>
Message-ID: <4074DCC4.4070106@prescod.net>

Raymond Hettinger wrote:
>...
> An alternative to the read-eval-print approach is providing a custom
> __repr__() method for the builtin iterators.  The good news is that this
> approach is very clean. 

And worth doing!

A few questions:

  * will it be easy for end-users to write iterators that can repr 
nicely? Do you anticipate a base class or helper function to help?

  * might it make sense for enumerate to report something like:

<enumerate object <reversed(['h', 'g', 'f', ...])> ...>

If the underlying object's repr is helpful then enumerate at least might 
not discard the information.

  Paul Prescod


From b.scharpf at tesionmail.de  Thu Apr  8 05:04:51 2004
From: b.scharpf at tesionmail.de (Bertram Scharpf)
Date: Thu Apr  8 05:05:00 2004
Subject: [Python-Dev] Re: Python vs. chat (conversation with modem)
References: <slrnc78mk6.46p.b.scharpf@homer.bertram-scharpf>
	<c52ev6$tte$1@sea.gmane.org>
Message-ID: <slrnc7a5di.7fs.b.scharpf@homer.bertram-scharpf>

Hi Terry,

Terry Reedy <tjreedy@udel.edu> schrieb:
>
> "Bertram Scharpf" <b.scharpf@tesionmail.de> wrote in message
> news:slrnc78mk6.46p.b.scharpf@homer.bertram-scharpf...
>> in simple serial communication, I found a problem that ...
>> I have found this behaviour in the versions 2.1, 2.2 and 2.3. ...
>
> This appears to be a usage question ...

I thoroughly analysed the problem. I cannot totally exclude
it's a usage question, but it made me wonder if it were.

Sorry.

Bertram

-- 
Bertram Scharpf
Stuttgart, Deutschland/Germany
http://www.bertram-scharpf.de


From martin at v.loewis.de  Thu Apr  8 14:47:42 2004
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Apr  8 14:48:07 2004
Subject: [Python-Dev] MS open sources WiX
In-Reply-To: <407383BB.6050907@interlink.com.au>
References: <407383BB.6050907@interlink.com.au>
Message-ID: <40759E4E.6050302@v.loewis.de>

Anthony Baxter wrote:
> Windows Installer XML (WiX) toolset has released as Open Source on
> SourceForge.net

Very interesting indeed. The challenge to any MSI generator is the
ability to include user interface in a flexible yet easy-to-use
manner. I'll take a look; off-hand, my criteria list for packaging
tools evaluates like so:
- avoid proprietary packagers: being open source, this is free of
   charge, and we can adjust it if we need to
- support Win64: Don't know, probably not (but could be added given
   that it is open source)
- support non-interactive installation, group policy: all done,
   thanks to MSI

Regards,
Martin


From skip at pobox.com  Thu Apr  8 17:29:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  8 17:29:43 2004
Subject: [Python-Dev] PEP 318: Singleton decorator
In-Reply-To: <Pine.LNX.4.58.0403300124560.18028@server1.LFW.org>
References: <Pine.LNX.4.58.0403300124560.18028@server1.LFW.org>
Message-ID: <16501.50231.895012.334330@montanaro.dyndns.org>


(Still catching up.  I guess no help with the substance of the PEP will be
forthcoming, so I'll try to get to things as I can.)

       ... [snip] ...

    Ping> But i don't think this is a good way to write a singleton, because
    Ping> then the user of the class has to get instances by saying
    Ping> "MyClass" instead of "MyClass()".  Better would be:

       ... [snip] ...

    Ping> I respectfully suggest that the PEP use the latter implementation
    Ping> in its singleton example instead.

Done.

Skip

From skip at pobox.com  Thu Apr  8 17:31:07 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  8 17:31:22 2004
Subject: [Python-Dev] PEP 318: Singleton decorator
In-Reply-To: <5.1.0.14.0.20040330095542.030435e0@mail.telecommunity.com>
References: <Pine.LNX.4.58.0403300124560.18028@server1.LFW.org>
	<5.1.0.14.0.20040330095542.030435e0@mail.telecommunity.com>
Message-ID: <16501.50331.232161.858966@montanaro.dyndns.org>


    >> But i don't think this is a good way to write a singleton, because
    >> then the user of the class has to get instances by saying "MyClass"
    >> instead of "MyClass()".

    Phillip> That's a stylistic decision.  IMO, it's more pythonic to *not*
    Phillip> call a constructor.  One does not, after all, call modules in
    Phillip> order to "construct" them, and modules are a prime example of
    Phillip> singletons in Python.

I think Ping's point is that if the user thinks Foo is a class, he's going
to want to instantiate it like Foo().  Whether or not you get back a new
instance or an existing one is sort of beside the point.  It's simply a
consistency issue.

Skip


From skip at pobox.com  Thu Apr  8 17:34:55 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  8 17:35:02 2004
Subject: [Python-Dev] PEP 318: Security use case
In-Reply-To: <Pine.LNX.4.58.0403300133000.18028@server1.LFW.org>
References: <Pine.LNX.4.58.0403300133000.18028@server1.LFW.org>
Message-ID: <16501.50559.174096.437955@montanaro.dyndns.org>


    Ping> Here is an additional simple use case you could consider.

    Ping>     def private(cls):
    Ping>         def instantiate(*args, **kw):
    Ping>             return cls(*args, **kw)
    Ping>         return instantiate

    Ping>     class DontTouch [private]:
    Ping>         ...

    Ping> Inner scopes are one of the best places to hide things in Python;
    Ping> they are very difficult to get at.

Are you using "private" to mean "can't be subclassed"?  I'm not sure that's
the correct term to use here.  I guess I'm interpreting "private" in the C++
sense of the word.

Skip

From marktrussell at btopenworld.com  Thu Apr  8 19:51:24 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Thu Apr  8 19:51:32 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <200404051803.i35I39U05548@guido.python.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<200404051644.i35GiKD05216@guido.python.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
Message-ID: <1081468284.2390.12.camel@localhost>

On Mon, 2004-04-05 at 19:03, Guido van Rossum wrote:
> > Would it not be possible to cheat and make the lexer transform 
> > 
> > 	"[" WHITESPACE "as" WHITESPACE IDENTIFIER 
> > 
> > into an LBRACE-AS token (which I assume would make the parser's job
> > simple).
> 
> I don't think without extreme hackery, but feel free to prove me wrong
> by producing a patch.  (A problem is, there could be newlines and
> comments inside the WHITESPACE.  Backtracking over that would require
> the lexer to have an arbitrary-length buffer.)

Unless I'm missing something (very possible) there's no need to
backtrack over the whitespace - all you need is a two-entry stack for
the "[" and the token that follows.  There's no question of INDENT or
DEDENT because we're inside a [] pair.  I've put a patch at
python.org/sf/932100 which does this, plus changes to Grammar/Grammar,
compile.c etc - it implements the syntax:

	[as classmethod]
	def foo(cls): pass

The newline is optional:

	[as foo, bar, baz] def foo(cls): pass

is also legal.

The test suite passes as before (there are still two failures but they
are unrelated to this patch).  The patch also includes your
test_decorators.py, modified for the new syntax.

Mark


From skip at pobox.com  Thu Apr  8 20:36:01 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr  8 20:36:05 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1081468284.2390.12.camel@localhost>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<200404051644.i35GiKD05216@guido.python.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<1081468284.2390.12.camel@localhost>
Message-ID: <16501.61425.664065.876124@montanaro.dyndns.org>


    Mark> I've put a patch at python.org/sf/932100 which does this, plus
    Mark> changes to Grammar/Grammar, compile.c etc - it implements the
    Mark> syntax:

    Mark>       [as classmethod]
    Mark>       def foo(cls): pass

    Mark> The newline is optional:

    Mark>       [as foo, bar, baz] def foo(cls): pass

    Mark> is also legal.

Does it treat '[' and 'as' as separate tokens so whitespace can occur
between them?

Note that the second alternative is likely to break all sorts of auxiliary
tools which scan Python source.  etags/ctags and python-mode come to mind.
Clearly, python-mode is well within our domain, but other syntax coloring
tools aren't.

Skip

From python at rcn.com  Thu Apr  8 22:37:18 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Apr  8 22:38:22 2004
Subject: [Python-Dev] Proposed iterator representations
Message-ID: <000b01c41ddb$94296280$0507a044@oemcomputer>

I checked in two itertool representations that were clear-cut:

>>> from itertools import count, repeat
>>> count(20)
count(20)
>>> repeat(None, 12)
repeat(None, 12)



For xrange iterators, there are a few choices:

>>> it = iter(xrange(10, 20, 2))
iter(xrange(10, 20, 2))
     or
rangeiterator(10, 20, 2)
     or
<rangeiterator start=10, stop=20, step=2>

The first is my favorite.  Though odd looking, it can be run through
eval() and is completely self-documenting.  The second has the virtue of
using the actual type name but it cannot be run through eval().  The
third is closer to the current format but adds some information.


For dictionaries, lists, tuples, and reversed, there are similar choices
but the eval() version can only be supported with a full listing of the
parent object:

>>> iter(dict(a=1, b=2, c=3, d=4, e=5))
iter(dict.fromkeys(['a', 'c', 'b', 'e', 'd']))
     or
<dictionary-keyiterator object:  'a', 'c', 'b', ...>


>>> dict(a=1, b=2, c=3, d=4, e=5).iteritems()
dict([('a', 1), ('c', 3), ('b', 2), ('e', 5), ('d', 4)]).iteritems()
     or
<dictionary-itemiterator object:  ('a', 1), ('c', 3), ('b', 2) ...>


>>> reversed('abcde')
reversed(['a', 'b', 'c', 'd', 'e'])
     or
<reversed object:  'e', 'd', 'c', ...>


What do you guys think?



Raymond Hettinger


From bac at OCF.Berkeley.EDU  Fri Apr  9 02:18:28 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Apr  9 02:18:34 2004
Subject: [Python-Dev] Proposed iterator representations
In-Reply-To: <000b01c41ddb$94296280$0507a044@oemcomputer>
References: <000b01c41ddb$94296280$0507a044@oemcomputer>
Message-ID: <40764034.6000306@ocf.berkeley.edu>

Raymond Hettinger wrote:

> For xrange iterators, there are a few choices:
> 
> 
>>>>it = iter(xrange(10, 20, 2))
> 
> iter(xrange(10, 20, 2))
>      or
> rangeiterator(10, 20, 2)
>      or
> <rangeiterator start=10, stop=20, step=2>
> 
> The first is my favorite.  Though odd looking, it can be run through
> eval() and is completely self-documenting.  The second has the virtue of
> using the actual type name but it cannot be run through eval().  The
> third is closer to the current format but adds some information.
> 

Third.  If I am at the interpreter I want information presented to me in 
the best way possible.  If I need to type it in again I have readline 
and a command history for that.  =)

> 
> For dictionaries, lists, tuples, and reversed, there are similar choices
> but the eval() version can only be supported with a full listing of the
> parent object:
> 
<SNIP>
> 
> What do you guys think?
> 

As mentioned above, I want the information presented in the most 
informative way and not worry about the repr output.

-Brett

From marktrussell at btopenworld.com  Fri Apr  9 05:44:03 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Apr  9 05:44:39 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <16501.61425.664065.876124@montanaro.dyndns.org>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<200404051644.i35GiKD05216@guido.python.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<1081468284.2390.12.camel@localhost>
	<16501.61425.664065.876124@montanaro.dyndns.org>
Message-ID: <1081503842.2391.8.camel@localhost>

On Fri, 2004-04-09 at 01:36, Skip Montanaro wrote:
> Does it treat '[' and 'as' as separate tokens so whitespace can occur
> between them?

Yes.

> Note that the second alternative is likely to break all sorts of auxiliary
> tools which scan Python source.  etags/ctags and python-mode come to mind.
> Clearly, python-mode is well within our domain, but other syntax coloring
> tools aren't.

It's a trivial change to disallow the second form - I'm pretty much
50/50 on whether it's a good idea to allow it or not.  Probably "There's
only one way to do it" should apply.  (The change is trivial - in
Grammar/Grammar, just replace [NEWLINE] with NEWLINE in the decorators
entry).

Mark


From barry at python.org  Fri Apr  9 09:10:23 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr  9 09:10:32 2004
Subject: [Python-Dev] Re: PEP 318: Decorators last before colon
In-Reply-To: <1081503842.2391.8.camel@localhost>
References: <Pine.LNX.4.58.0403311602480.18028@server1.LFW.org>
	<5.1.0.14.0.20040404233506.03b08210@mail.telecommunity.com>
	<200404050433.i354XcF03700@guido.python.org>
	<1081156611.2377.6.camel@localhost>
	<200404051644.i35GiKD05216@guido.python.org>
	<1081184434.4045.3.camel@localhost>
	<200404051803.i35I39U05548@guido.python.org>
	<1081468284.2390.12.camel@localhost>
	<16501.61425.664065.876124@montanaro.dyndns.org>
	<1081503842.2391.8.camel@localhost>
Message-ID: <1081516222.5071.132.camel@anthem.wooz.org>

On Fri, 2004-04-09 at 05:44, Mark Russell wrote:

> It's a trivial change to disallow the second form - I'm pretty much
> 50/50 on whether it's a good idea to allow it or not.  Probably "There's
> only one way to do it" should apply.  (The change is trivial - in
> Grammar/Grammar, just replace [NEWLINE] with NEWLINE in the decorators
> entry).

Personally, I think it's fine to allow them to be on the same line.  I
wouldn't want to arbitrarily disallow it because we're worried about how
the tools will react.  If people gravitate toward this syntax because
it's more readable, then the tools will eventually adjust.

just-because-you-can-doesn't-mean-you-should-ly y'rs,
-Barry



From aahz at pythoncraft.com  Fri Apr  9 11:42:52 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Apr  9 11:42:58 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <04Apr7.183319pdt."58612"@synergy1.parc.xerox.com>
References: <20040407045318.GB173@panix.com>
	<04Apr7.183319pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <20040409154252.GB28808@panix.com>

On Wed, Apr 07, 2004, Bill Janssen wrote:
>
> I really wish I'd been paying more attention to 328, which I've just
> read.  This proposal seems to break lots and lots and lots of existing
> code.  Maybe I'm reading it wrong.
> 
> I've written many, many packages that use the form
> 
>      import foo
> 
> where "foo" is a sibling module in the package.  

You won't be able to do that anymore.

> Or
> 
>       from foo import bar
> 
> where, again, foo is a sibling to the current module.  Packages that
> work like this have been submitted to on-line libraries, printed in
> books, etc., etc., etc.

You'll have to change these to

    from .foo import bar

> I really hope I'm reading the PEP wrong!  Am I correct in thinking
> that come Python 2.4 or 2.5, these will just break if the package
> isn't at the top level (reachable from sys.path)?

They won't break until Python 2.6.  Python 2.5 will issue a
DeprecationWarning when it finds relative imports.

> I was thinking (some months ago) that y'all would come up with an
> absolute form instead, something like
> 
>       from / import os

Nope.  Absolute imports are the most common form, and we do want to
encourage absolute imports whenever possible (because of the conflicts
with the stdlib).  Relative imports should be "marked".
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From tjreedy at udel.edu  Fri Apr  9 14:23:15 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri Apr  9 14:23:19 2004
Subject: [Python-Dev] Re: Proposed iterator representations
References: <000b01c41ddb$94296280$0507a044@oemcomputer>
Message-ID: <c56pmg$ppf$1@sea.gmane.org>


"Raymond Hettinger" <python@rcn.com> wrote in message
news:000b01c41ddb$94296280$0507a044@oemcomputer...
> I checked in two itertool representations that were clear-cut:

;-)

> >>> from itertools import count, repeat
> >>> count(20)
> count(20)

Is the representation (of the dynamic object) static?  Does

it=count(20); it.next(), it.next(); it

still print 'count(20)' as it one was but no longer is?

This seems much like lt=iter([1,2,3]); lt.next(); lt.next(); lt printing
iter([1,2,3]) instead of iter([3]), or worse, lt=[1,2,3]; lt.pop();
lt.pop(); lt printing [1,2,3] instead of [1].

Terry J. Reedy




From python at rcn.com  Fri Apr  9 14:49:07 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Apr  9 14:50:19 2004
Subject: [Python-Dev] Re: Proposed iterator representations
In-Reply-To: <c56pmg$ppf$1@sea.gmane.org>
Message-ID: <012001c41e63$5799a860$e435c797@oemcomputer>

> Is the representation (of the dynamic object) static? 

No, it is dynamic.

> Does
> 
> it=count(20); it.next(), it.next(); it
> 
> still print 'count(20)' as it one was but no longer is?

That prints "count(22)".



Raymond


"... as a general rule of thumb, the first time a memory location is
accessed incurring a cache miss, it will be at least as expensive as a
divide operation, and additional accesses to the same location are
roughly free." -- The Software Optimization Cookbook, Intel Press


From eppstein at ics.uci.edu  Fri Apr  9 16:43:29 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Fri Apr  9 16:43:33 2004
Subject: [Python-Dev] Re: Proposed iterator representations
References: <c56pmg$ppf$1@sea.gmane.org>
	<012001c41e63$5799a860$e435c797@oemcomputer>
Message-ID: <eppstein-47841F.13432909042004@sea.gmane.org>

In article <012001c41e63$5799a860$e435c797@oemcomputer>,
 "Raymond Hettinger" <python@rcn.com> wrote:

> > Does
> > 
> > it=count(20); it.next(), it.next(); it
> > 
> > still print 'count(20)' as it one was but no longer is?
> 
> That prints "count(22)".

Speaking of itertools.count, where can I find the rationale for the 
"Does not currently support python long integers." restriction?

This bit me the last time I needed count (part of an implementation of 
the greedy algorithm for Egyptian fractions, so I really needed longs) 
and I ended up having to roll my own.  Not that it was especially 
difficult to roll my own, but it seems like it would have been harder to 
put that check in than to just allow longs to work and I don't 
understand why it was done that way.

I tried googling the python-dev archives but no luck.

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From python at rcn.com  Fri Apr  9 18:06:38 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Apr  9 18:07:44 2004
Subject: [Python-Dev] Re: Proposed iterator representations
In-Reply-To: <eppstein-47841F.13432909042004@sea.gmane.org>
Message-ID: <019a01c41e7e$ee96f680$e435c797@oemcomputer>

[David Eppstein]
> Speaking of itertools.count, where can I find the rationale for the
> "Does not currently support python long integers." restriction?

The rationale was writing "the simplest thing that could possibly work".
At the time, there were no use cases that warranted additional code
complexity.  I wanted this to be bug free right out of the box.  In the
docs, I included a cut and pastable pure python version that did support
long integers so that no one would be left in the cold.  As you said,
rolling your own is trivial.

Also, the design was also affected by trying to fit itertools.count()
into the tiny ecological niche between xrange() and itertools.repeat().
Its competitive advantage was being lightweight, fast, and infinite so
you could write things like izip(count(10), data).


Raymond


"... as a general rule of thumb, the first time a memory location is
accessed incurring a cache miss, it will be at least as expensive as a
divide operation, and additional accesses to the same location are
roughly free." -- The Software Optimization Cookbook, Intel Press



From arigo at tunes.org  Sat Apr 10 07:22:17 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sat Apr 10 07:25:45 2004
Subject: [Python-Dev] Proposed iterator representations
In-Reply-To: <000b01c41ddb$94296280$0507a044@oemcomputer>
References: <000b01c41ddb$94296280$0507a044@oemcomputer>
Message-ID: <20040410112217.GA3001@vicky.ecs.soton.ac.uk>

Hello Raymond,

On Thu, Apr 08, 2004 at 10:37:18PM -0400, Raymond Hettinger wrote:
> I checked in two itertool representations that were clear-cut:
> 
> >>> from itertools import count, repeat
> >>> count(20)
> count(20)
> >>> repeat(None, 12)
> repeat(None, 12)

Another point of view is that these representation are not very informative if
you don't already know count() and repeat(): someone will probably type the
above code in order to experiment with them, and the repr doesn't help in this
respect.  I think I'd prefer a more standard representation for all the
iterators for which it is reasonably possible to do so.

>>> count(20)
<itertools.count: 20, 21, 22, ...>
>>> repeat(None, 12)
<itertools.repeat: None, None, None, ... 12 elements>
>>> iter(xrange(10, 20, 2))
<rangeiterator: 10, 12, 14, ... 5 elements>

A bit of effort should be made at showing the nicest answer, e.g.:

>>> iter(xrange(0))
<rangeiterator: empty>
>>> iter(xrange(1))
<rangeiterator: 0>
>>> iter(xrange(2))
<rangeiterator: 0, 1>
>>> iter(xrange(3))
<rangeiterator: 0, 1, 2>
>>> iter(xrange(4))
<rangeiterator: 0, 1, 2, 3>
>>> iter(xrange(5))
<rangeiterator: 0, 1, 2, ... 5 elements>


Armin


From janssen at parc.com  Sat Apr 10 12:38:36 2004
From: janssen at parc.com (Bill Janssen)
Date: Sat Apr 10 12:39:01 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: Your message of "Fri, 09 Apr 2004 08:42:52 PDT."
	<20040409154252.GB28808@panix.com> 
Message-ID: <04Apr10.093843pdt."58612"@synergy1.parc.xerox.com>

> Nope.  Absolute imports are the most common form, and we do want to
> encourage absolute imports whenever possible (because of the conflicts
> with the stdlib).  Relative imports should be "marked".

Wow.  This seems amazingly user-hostile.  Users (not Python
developers) download some two or three year old code to do something
from an online library, like the Vaults of Parnassus, and it just
breaks because the meaning of "import" has changed.  I hope we aren't
developing the kind of attitude towards non-developer users that made
lispers call them "lusers".

Bill



From ark at acm.org  Sat Apr 10 12:57:04 2004
From: ark at acm.org (Andrew Koenig)
Date: Sat Apr 10 12:57:01 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: <04Apr10.093843pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>

> Wow.  This seems amazingly user-hostile.  Users (not Python
> developers) download some two or three year old code to do something
> from an online library, like the Vaults of Parnassus, and it just
> breaks because the meaning of "import" has changed.  I hope we aren't
> developing the kind of attitude towards non-developer users that made
> lispers call them "lusers".

This is a tougher issue than it appears, as I have learned from my long
experience with C++.

The trouble is that if you commit yourself to backward compatibility, you
wind up with a growing list of decisions for which the default is wrong but
you didn't want to break code by changing it.  Each time you make such a
decision, you make it more difficult the next time to do anything but go for
compatibility again.  Ultimately, everyone has to remember to override the
defaults in lots of everyday circumstances, a mental burden that hurts
everyone from novices to experts.
 
On the other hand, if you don't commit yourself to backward compatibility,
you wind up with people downloading old code that breaks because things have
changed.

The third possibility is to support all alternatives forever, perhaps with a
mechanism such as "from __past__ import foo".  That's possible in theory,
but if changes include changes in semantics, it may be impossible for old
and new to coexist.  Moreover, even if they can coexist, testing becomes a
nightmare.

It's a trilemma--a choice between three unpleasant alternatives.  I don't
know the right answer, but I do wish to caution against assuming the problem
is trivial.


From aahz at pythoncraft.com  Sat Apr 10 14:33:10 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 10 14:33:48 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <04Apr10.093843pdt."58612"@synergy1.parc.xerox.com>
References: <20040409154252.GB28808@panix.com>
	<04Apr10.093843pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <20040410183310.GA27946@panix.com>

On Sat, Apr 10, 2004, Bill Janssen wrote:
>Aahz:
>>
>> Nope.  Absolute imports are the most common form, and we do want to
>> encourage absolute imports whenever possible (because of the conflicts
>> with the stdlib).  Relative imports should be "marked".
> 
> Wow.  This seems amazingly user-hostile.  Users (not Python
> developers) download some two or three year old code to do something
> from an online library, like the Vaults of Parnassus, and it just
> breaks because the meaning of "import" has changed.  I hope we aren't
> developing the kind of attitude towards non-developer users that made
> lispers call them "lusers".

In addition to Andrew's excellent response, I want to point out that this
only affects packages; most of the distributed code that I've seen tends
to be in single modules.  It also only affects packages that rely on
relative imports: as Barry points out, there's a significant set of
developers that currently uses only absolute imports because of the
tricky semantics for relative imports.

Also, by the time imports start to fail, what is currently two or three
year-old code will be five or six years old.  Anyone who is writing code
now will be aware of this issue and should be able to future-proof their
code if they want.  Nobody should expect to download code that's five or
six years old and have it just work.

Do you still think this is a "user-hostile" attitude?  If yes, you'll
need to make your case in more detail.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From janssen at parc.com  Sat Apr 10 15:26:56 2004
From: janssen at parc.com (Bill Janssen)
Date: Sat Apr 10 15:27:16 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: Your message of "Sat, 10 Apr 2004 09:57:04 PDT."
	<00db01c41f1c$d9beec10$6402a8c0@arkdesktop> 
Message-ID: <04Apr10.122659pdt."58612"@synergy1.parc.xerox.com>

> I do wish to caution against assuming the problem is trivial.

I've been down this road myself, and I think I understand the problem.
However, whenever I've made changes of this break-itude, I've always
done it in a major release.  Changing things this way in a minor
release just seems hostile to me.  Remember that one of the major
reasons people use Python (instead of, say, Scheme) is the availability
of lots of packages written in Python to do various things.  Making
those packages buggy is a suboptimal way to win friends and influence
people.

Bill

From janssen at parc.com  Sat Apr 10 15:44:28 2004
From: janssen at parc.com (Bill Janssen)
Date: Sat Apr 10 15:44:52 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: Your message of "Sat, 10 Apr 2004 11:33:10 PDT."
	<20040410183310.GA27946@panix.com> 
Message-ID: <04Apr10.124434pdt."58612"@synergy1.parc.xerox.com>

> In addition to Andrew's excellent response, I want to point out that this
> only affects packages; most of the distributed code that I've seen tends
> to be in single modules.  It also only affects packages that rely on
> relative imports: as Barry points out, there's a significant set of
> developers that currently uses only absolute imports because of the
> tricky semantics for relative imports.

It would be interesting to have a survey.  You'd want to have usage
statistics in there, as well.  That is, xx% of packages use relative
imports; these packages account for xx% of downloads.

> Also, by the time imports start to fail, what is currently two or three
> year-old code will be five or six years old.  Anyone who is writing code
> now will be aware of this issue and should be able to future-proof their
> code if they want.  Nobody should expect to download code that's five or
> six years old and have it just work.

I expressed myself poorly; I was thinking of packages written in the
future, so they'd still be two or three years old.

"Anyone who is writing code now will be aware..." is an unfortunate
example of ivory-tower thinking.  Many people who are writing code
using a wonderful tool like Python have better things to do than keep
up with the vagaries of Python releases.  They rightly feel that if
they learned Python with 2.2, what they know should be pretty stable
at least until Python 3.0 comes out.  They don't think that in a minor
release, Python import semantics will change out from under them.
Every few years, when they find some time, they will check out some of
the new features, like generators, that the developers find so
delightful.  But in between those times, they won't be clueful.

Most Python developers may be Python users, but most Python users are
not Python developers.

> Nobody should expect to download code that's five or
> six years old and have it just work.

One of the attractive features of Python has always been that
5-year-old code *does* just work.  That's because the design was
largely right from the start, and the designers have been careful to
keep things backwards-compatible.

> Do you still think this is a "user-hostile" attitude?  If yes, you'll
> need to make your case in more detail.

I think users unfamiliar with the Python development process will
perceive Python as being hostile to their concerns if things like this
keep happening.  I wouldn't attribute any hostility to the developers;
it's just a mistake.

Bill

From fumanchu at amor.org  Sat Apr 10 18:14:17 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Sat Apr 10 18:16:31 2004
Subject: [Python-Dev] PEP 321 and Gustavo's DateUtil
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561FDF@opus.amorhq.net>

There was some discussion about Gustavo Niemeyer's DateUtil module back
in March:

http://aspn.activestate.com/ASPN/Mail/Message/python-dev/2022361

...and an immediate question was whether it fulfilled (or exceeded ;)
the requirements of PEP 321:

http://www.python.org/peps/pep-0321.html

I looked at the dateutil.parser.py module today in some detail; with a
few slight modifications, it could very well meet PEP 321. The issues as
I see them:

1. I am *not* proposing the whole dateutil module be accepted.
Recurrence and relativedelta brought up too many unresolved questions
among developers back in March. Therefore, if dateutil.parser were to be
considered, it should stand alone, and be named independently, for
example, "dateparse" or "dateparser", IMO a sibling of the datetime
module, not a child.

2. If it were to stand on its own, it only needs about a dozen lines
modified to reduce dependency on:

    dateutil.relativedelta (1 mention, line 304):
            ret = ret+relativedelta.relativedelta(weekday=res.weekday)

...which could be replaced inline by simply distilling the relevant
chunk of relativedelta.__radd__. The only other dependency is on:

    dateutil.tz (5 mentions, lines 314-326), for example:
            ret = ret.replace(tzinfo=tz.tzutc())

...every use of which simply returns a concrete subclass of
datetime.tzinfo. To resolve this, we could:

a) include dateutil.tz in the library as its own module, "timezones", or
b) include dateutil.tz as "datetime.timezones", or
c) drop the 5 necessary tz classes into the new "dateparse" module, or
d) strip that particular functionality from dateparse, making it
effectively tz-naive.

3. Locale-dependent data is nicely separated into a "parserinfo" class,
which might benefit from built-in subclasses for major languages;
currently, the default is English.

There are minor cleaning issues, but none of them are show-stoppers:

4. dateutil.parser imports os.path and sys but doesn't reference them.
5. _resultbase uses __slots__ for some odd reason.
6. There's a mix of old- and new-style classes. I haven't see any reason
in the code not to make them all new-style.
7. Some PEP 8 conventions, like CapWord class names, might be nice.

In all, I think we could finish up yet another PEP and get some solid,
long-desired functionality out to the Python community with a minimum of
effort and even less breakage, since Gustavo's done all the hard work
already. ;)

Thoughts?


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From paul at prescod.net  Sat Apr 10 18:55:15 2004
From: paul at prescod.net (Paul Prescod)
Date: Sat Apr 10 18:57:49 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <04Apr10.124434pdt."58612"@synergy1.parc.xerox.com>
References: <04Apr10.124434pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <40787B53.9070007@prescod.net>

Bill Janssen wrote:
>>Also, by the time imports start to fail, what is currently two or three
>>year-old code will be five or six years old.  Anyone who is writing code
>>now will be aware of this issue and should be able to future-proof their
>>code if they want.  Nobody should expect to download code that's five or
>>six years old and have it just work.
> 
> 
> I expressed myself poorly; I was thinking of packages written in the
> future, so they'd still be two or three years old.
> 
> "Anyone who is writing code now will be aware..." is an unfortunate
> example of ivory-tower thinking.  Many people who are writing code
> using a wonderful tool like Python have better things to do than keep
> up with the vagaries of Python releases.  They rightly feel that if
> they learned Python with 2.2, what they know should be pretty stable
> at least until Python 3.0 comes out.

Most people in the Python community know that Python doesn't really have 
"minor releases" other than bugfix releases. Just like Linux and Perl, 
the choice to switch to a big version number is more marketing than 
anything. (and it says something about our marketing saavy that we've 
done it exactly once)

> ...  They don't think that in a minor
> release, Python import semantics will change out from under them.
> Every few years, when they find some time, they will check out some of
> the new features, like generators, that the developers find so
> delightful.  But in between those times, they won't be clueful.

It really just boils down to a question of how much time you give 
people. First you try to publicize the change via books, tutorials, 
mailing lists etc. Then you add an annoying warning. Then eventually you 
break the old code. If you move slow enough then I claim you can make an 
arbitrarily deep and pervasive change without significant user pain. For 
example consider the shift over the years from K&R C to ANSI C to C++. 
It wasn't totally painful but it was never so painful as to make people 
switch languages on the basis of backwards incompatibility pain. And 
there were some pretty major changes in there...the deprecation of 
implicit casting of void * being one that pops to mind.

> One of the attractive features of Python has always been that
> 5-year-old code *does* just work.  That's because the design was
> largely right from the start, and the designers have been careful to
> keep things backwards-compatible.

5-year-old code OFTEN just works. But not always. Each version of Python 
has a document that describes incompatibilities.

http://www.python.org/1.6/

http://www.python.org/2.2.3/bugs.html

e.g. "The special attributes __members__ and __methods__ are no longer 
supported (for most built-in types). Use the new and improved dir() 
function instead."

Admittedly this relative import change is a big one compared to those 
from the past. My opinion is that that just means it should be done a 
very long time after the version with the warning ships.

My opinion is that the process is correct and it is just a question of 
whether the timelines are generous enough or not. Somewhere between 
"Python 2.6" and "never" there must be a point where people have had 
enough time to change over their code after getting nagged by the 
DeprecationWarning.

  Paul Prescod


From bac at OCF.Berkeley.EDU  Sat Apr 10 22:27:36 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Apr 10 22:27:52 2004
Subject: [Python-Dev] python-dev Summary for 2004-03-16 through 2004-03-31
	[rough draft]
Message-ID: <4078AD18.5070502@ocf.berkeley.edu>

Don't plan on sending this summary out for a while since it is so long 
(over 8 pages if TextEdit is to be believed) and because of the volume 
of emails I had to go based on memory on some of the summaries and thus 
may have missed some of the discussions in the large threads.

Anyway, reply with any corrections you have and such.

---------------------------------

=====================
Summary Announcements
=====================
Still looking for a summer job or internship.  Please let me know if 
have one at your company or happen to know of one somewhere.  And yes, 
this message will go away; either I will actually get a summer job or 
the summer will come around.  =)

PyCon happened during the timespan this summary covers.  Unlike last 
year when emails to python-dev actually went down, there was a good 
amount of traffic at the conference.  A good portion of this was due to 
the sprint on Python's core that took place.  A bunch of core developers 
showed up for the whole weekend and used part of that time to answer 
emails.  Great for python-dev but made my life a little difficult.  =) 
But the sprint overall was a definite gain with various tweaks done, 
improvements to the profiler, and a bunch of bugs and patches closed 
(actually had the number of open patches and bugs have a net loss of 7 
and 10 from that week, respectively; that has not happened in a while).

The release schedule (and thus what is definitely planned and what can 
go in if it gets finished in time) was set while at PyCon.  Read the 
summary entitled `Python 2.4 release schedule`_ below for details and links.

=========
Summaries
=========
----------------------------
"I want to be like Raymond!"
----------------------------
So, you want to speed up Python like Raymond Hettinger has?  Well, he 
posted an email listing various optimizations he has on his list to try 
out.  That specific email can be found at 
http://mail.python.org/pipermail/python-dev/2004-March/043276.html

This also brought back the debate of whether pure Python modules should 
be recoded in C for performance.  See previous threads for a longer 
discussion on the subject.

Contributing threads:
   - `Joys of Optimization 
<http://mail.python.org/pipermail/python-dev/2004-March/043276.html>`__
   - `todo (was: Joys of Optimization) 
<http://mail.python.org/pipermail/python-dev/2004-March/043335.html>`__


--------------
Decimal update
--------------
`PEP 327`_ was revised to make certain issues more explicit and to 
discuss possible function naming (summarized below).  This lead to a 
discussion about how to name various things.  Although there are 
historical violations of the coding standard, the names should follow 
`PEP 7`_ and `PEP 8`_.

The testing suite is also finished along with the PEP essentially being 
finished.

.. _PEP 327: http://www.python.org/peps/pep-0327.html


Contributing threads:
   - `Changes to PEP 327: Decimal data type 
<http://mail.python.org/pipermail/python-dev/2004-March/043155.html>`__
   - `PEP 327: Decimal data type 
<http://mail.python.org/pipermail/python-dev/2004-March/043809.html>`__


--------------------------------------------------------------------------
<in a stereotypical coach voice>Follow the coding standard, people</voice>
--------------------------------------------------------------------------
Discussing naming conventions came up while discussing PEP 327.  The 
basic conclusion is that older modules don't follow the coding standards 
laid out in `PEP 7`_ and `PEP 8`_, but that does not mean new code 
should be allowed to break the rules.

.. _PEP 7: http://www.python.org/peps/pep-0007.html
.. _PEP 8: http://www.python.org/peps/pep-0008.html

Contributing threads:
   - `Changes to PEP 327: Decimal data type 
<http://mail.python.org/pipermail/python-dev/2004-March/043155.html>`__
   - `(class) module names clarification 
<http://mail.python.org/pipermail/python-dev/2004-March/043265.html>`__
   - `(old) module names 
<http://mail.python.org/pipermail/python-dev/2004-March/043286.html>`__


----------------------------------------------------
"It depends on what the meaning of the word 'is' is"
----------------------------------------------------
The idea of redefining how 'is' operates when comparing immutable types 
came up.  It was suggested that when two immutable objects of the same 
type were being compared that it should be an equivalence test instead 
of an identity test.  For instance::

   x = 99
   y = 99
   x is y

is true while::

   x = 100
   y = 100
   x is y

is false.  The point was made, though, that unless you are comparing to 
None that 'is' should not be used for comparing immutables and instead 
the equality operator ('==') should be used instead.  There is 
backwards-compatibility issues with changing this.

Guido pronounced that 'is' ain't changing.

Contributing threads:
   - `A proposal has surfaced on comp.lang.python to redefine "is" 
<http://mail.python.org/pipermail/python-dev/2004-March/043181.html>`__
   - `redefining is 
<http://mail.python.org/pipermail/python-dev/2004-March/043222.html>`__
   - `(not) redefining is 
<http://mail.python.org/pipermail/python-dev/2004-March/043285.html>`__


---------------------------------
Getting PyChecker into the stdlib
---------------------------------
The idea of adding PyChecker_ to the stdlib came up once again.  Support 
seems to be there but the developers of PyChecker did not speak up. 
PyChecker 2 is under development and that changes how PyChecker does 
things significantly (analyzes bytecode directly) which might be why 
they were mum on the topic.

.. _PyChecker: http://pychecker.sf.net/


-------------------------------
Simplifying logging.basicConfig
-------------------------------
Simplifying basicConfig in the logging package to making logging to a 
file simpler was proposed.  One person suggested more examples in the 
documentation ala unittest.  Another person disagreed.

No change has been made at this point nor any specific statement as to 
whether any will be.

Contributing threads:
   - `Some changes to logging 
<http://mail.python.org/pipermail/python-dev/2004-March/043290.html>`__


--------------------------
A new committer amongst us
--------------------------
Phillip J. Eby gained CVS commit privileges to help with development of 
Distutils.  But Phillip has been active on python-dev for a while so I 
am sure he can be put to work on some other things as well.  =)

Contributing threads:
   - `new Python CVS committer 
<http://mail.python.org/pipermail/python-dev/2004-March/043338.html>`__


----------------
Working on HP-UX
----------------
Cameron Laird is still interested in working on the port of Python on 
HP-UX.  If you are interested in helping out please contact him.

Contributing threads:
   - `Portability:  update and oblique apology 
<http://mail.python.org/pipermail/python-dev/2004-March/043364.html>`__


-------------------------------------------
Binding variables for generator expressions
-------------------------------------------
What would you expect in this situation using generator expressions?::

   gen_exps = []
   for cnt in range(10):
     gen_exps.append(x**2 for x in range(cnt))

If you thought that gen_exps would contain 10 generators that each 
return a list longer than the one previous then you like capture binding 
for generator expressions (capturing the values of all free variables 
used in the generator expression at definition time).  If you though 
there would be 10 generators that all had the power of 2 from 0 to 9 
then you like late binding (using the value of the free variables in the 
generator expression at execution time of the generator).

Guido brought this up in his keynote for PyCon_ and said that he 
preferred late binding (don't remember if he pronounced; I could lie and 
say I was under the influence of a hang-over from a drunken stupor from 
the night before but I actually didn't go drinking once this year at 
PyCon and so my excuse is I just don't plain remember).  The reasoning 
is that there are no surprises in terms of corner cases and having more 
obvious values for the free variables.

The point about performance of generator expressions was also brought up 
using the now-standard ``sum(x**2 for range(1000))`` example.  Using a 
list comprehension was actually slightly faster (less than 10%) then the 
generator expression.  This is partially thanks to Raymond's tweaking of 
list comprehensions by adding the LIST_APPEND opcode.  The rest is due 
to generator expressions creating an actual generator and thus having to 
create eval frames and playing with them.  But it was pointed out that 
if the number passed into 'range' is raised then generator expressions 
quickly end up winning because the execution fits in the cache better, 
especially if xrange is used.

Contributing threads:
   - `An issue recently brought up in patch #872326 
<http://mail.python.org/pipermail/python-dev/2004-March/043396.html>`__
   - `Possible resolution of generator expression variable... 
<http://mail.python.org/pipermail/python-dev/2004-March/043443.html>`__
   - `genexps slow? 
<http://mail.python.org/pipermail/python-dev/2004-March/043777.html>`__


--------------------------------------------------
More than you ever wanted to know about decorators
--------------------------------------------------
Decorators were the topic for the latter half of March.  Initially the 
discussion focused on where to put the list of decorators (if Guido's 
patch as found at http://www.python.org/sf/926860 is any indication he 
has a preference with the decorators coming on the line above the 
function/method definition) should go.  With C# having a similar 
functionality as what is being proposed roughly following C#'s syntax 
was considered a good thing, albeit tweaked so that the decorator list 
goes on its own line (reason being the LL(1) parser is too dumb to 
handle it coming before the 'def' on the same line and it alleviates the 
issue of having a long decorator list that would a full line anyway.

With the winning syntax, though, there is the issue of preexisting code 
and the work at interpreter prompt.  The former is a problem for code 
that happens to have a list that is not assigned to anything before a 
function definition; why anyone would have a list defined that is not 
assigned is beyond me.  As for the latter issue, that is a slight 
problem.  To get around it you can stick the definition in an 'if' 
statement::

   if True:
       [decorate, this]
       def marthastewart(): pass

Slight pain, but it has been generally accepted that placing decorators 
on their own line is just so much easier to deal with aesthetically this 
minor inconvenience has been viewed as acceptable.

After PyCon Guido talked to some people who suggested having another way 
of tacking on metadata separate from using descriptors, partially since 
C#'s version of decorators is mostly for metadata.  But tacking on 
attributes to a function can be done using the proposed decorators by 
defining some auxiliary function that takes in named arguments that 
specify the name of the attribute and the value to set it and returns a 
callable that will take in a function and assign the attributes.

This also brought up the discussion of setting function attributes from 
within a function since decorators will most likely be used to set 
metadata.  The idea of having attributes set for the function if a name 
starts with '.' was brought up but Guido said he wanted to save that 
syntax for within a 'with' block.  The other suggestion was to do 
``func.attribute`` where "func" is the name of the function is being 
defined, but Guido gave that a -1000 vote so that ain't about to happen.

Contributing threads:
   - `order of decorator application? 
<http://mail.python.org/pipermail/python-dev/2004-March/043411.html>`__
   - `PEP 318 - posting draft 
<http://mail.python.org/pipermail/python-dev/2004-March/043414.html>`__
   - `PEP 318 
<http://mail.python.org/pipermail/python-dev/2004-March/043425.html>`__
   - `method decorators (PEP 318) 
<http://mail.python.org/pipermail/python-dev/2004-March/043528.html>`__
   - `PEP 318 and syntax coloring 
<http://mail.python.org/pipermail/python-dev/2004-March/043643.html>`__
   - `decorators (not transformations) 
<http://mail.python.org/pipermail/python-dev/2004-March/043540.html>`__
   - `PEP318 metaclass approach 
<http://mail.python.org/pipermail/python-dev/2004-March/043589.html>`__
   - `method decorators 
<http://mail.python.org/pipermail/python-dev/2004-March/043613.html>`__
   - `Yet Another Decorator Syntax Suggestion  (YADSS) 
<http://mail.python.org/pipermail/python-dev/2004-March/043640.html>`__
   - `PEP 318: Singleton decorator 
<http://mail.python.org/pipermail/python-dev/2004-March/043682.html>`__
   - `PEP 318: Preserve function signatures 
<http://mail.python.org/pipermail/python-dev/2004-March/043683.html>`__
   - `PEP 318: Security use case 
<http://mail.python.org/pipermail/python-dev/2004-March/043684.html>`__
   - `PEP 318: Decorators last before colon 
<http://mail.python.org/pipermail/python-dev/2004-March/043685.html>`__
   - `PEP 318: Set attribs with .name = value 
<http://mail.python.org/pipermail/python-dev/2004-March/043686.html>`__
   - `Re: Python-Dev Digest, Vol 8, Issue 91 
<http://mail.python.org/pipermail/python-dev/2004-March/043698.html>`__


---------------------------
Python 2.4 release schedule
---------------------------
If you read `PEP 320`_ you will notice an alpha is currently planned in 
July with a possible release in September if the alphas and betas go well.

To see what is planned for this release read the PEP.  Looks like some 
really cool new additions to Python (under the hood, stdlib, and to 
syntax) might make this release.

.. _PEP 320: http://python.org/peps/pep-0320.html

Contributing threads:
   - `Timing for Py2.4 
<http://mail.python.org/pipermail/python-dev/2004-March/043681.html>`__


------------------------------------------
Cutting down on string creation at startup
------------------------------------------
Martin v. Loewis discovered that Python in CVS created about 12,000 
strings while Python 2.2 created 8,000 strings.  Martin subsequently 
changed 'marshal' so as to share strings that are the same instead of 
having to constantly create strings, discover they have been interned, 
and then destroy the second copy.

Contributing threads:
   - `Python startup time: String objects 
<http://mail.python.org/pipermail/python-dev/2004-March/043475.html>`__


---------------------------------
f_tstate gone the way of the dodo
---------------------------------
In case you don't catch the meaning of the title, f_tstate has now been 
removed.

Contributing threads:
   - `Last chance! 
<http://mail.python.org/pipermail/python-dev/2004-March/043594.html>`__


---------------------------------------------------------------
Tentative fix for OpenVMS and its dislike of UNIVERSAL_NEWLINES
---------------------------------------------------------------
See http://www.python.org/sf/903339 for the proposed patch.

Contributing threads:
   - ` OpenVMS file system and UNIVERSAL_NEWLINES 
<http://mail.python.org/pipermail/python-dev/2004-March/043660.html>`__


----------------------------------------------
The hell that is floating point representation
----------------------------------------------
The discussion came up over the output of ``repr(1.1)``, which is 
'1.1000000000000001' (although, apparently, a few years back this was 
not the case).  The objection was risen that this can surprise people 
since repr is what 'marshal' uses for writing .pyc files.  But the 
counter-argument was that this forces people to deal with the issues 
inherent in binary floating point.  Plus it makes the value consistent 
across platforms if you pass around a .pyc file since the value the 
programmer used is explicitly stated.  The status quo is staying put.

It was also pointed out that decimal floating point has similar issues 
as binary floating point, but at least you get to specify the accuracy 
at the cost of speed.

Contributing threads:
   - `Expert floats 
<http://mail.python.org/pipermail/python-dev/2004-March/043703.html>`__
   - `repr(1.1) 
<http://mail.python.org/pipermail/python-dev/2004-March/043726.html>`__


----------------------------------------
Relative imports getting a bunch of dots
----------------------------------------
For `PEP 328`_ (which deals with relative imports of modules), Guido 
pronounced that he prefers the multiple dots to delineate going up a 
level in the directory hierarchy for handling relative imports.

.. _PEP 328: htttp://www.python.org/peps/pep-0328.html

Contributing threads:
   - `PEP 328 -- relative and multi-line import 
<http://mail.python.org/pipermail/python-dev/2004-March/043739.html>`__

From aahz at pythoncraft.com  Sun Apr 11 02:14:35 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr 11 02:14:45 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <04Apr10.122659pdt."58612"@synergy1.parc.xerox.com>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<04Apr10.122659pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <20040411061435.GA13165@panix.com>

On Sat, Apr 10, 2004, Bill Janssen wrote:
>Andrew Koenig:
>>
>> I do wish to caution against assuming the problem is trivial.
> 
> I've been down this road myself, and I think I understand the
> problem.  However, whenever I've made changes of this break-itude,
> I've always done it in a major release.  Changing things this way
> in a minor release just seems hostile to me.  Remember that one of
> the major reasons people use Python (instead of, say, Scheme) is the
> availability of lots of packages written in Python to do various
> things.  Making those packages buggy is a suboptimal way to win
> friends and influence people.

Python's version changes from 2.0 through 2.3 have all been considered
"major" releases, as will the upcoming 2.4 release.  Ditto 1.4 to 1.5
(haven't been involved with Python long enough to know about earlier
releases).  For that matter, 1.5 to 1.5.1 to 1.5.2 contained a fair
number of major changes, which led to the creation of PEP 6.  The jump
from 1.5 to 1.6 to 2.0 was overall driven more by politics than technical
issues.

Although we talk about plans for Python 3.0, that's very much a
pie-in-the-sky release at this point, and that will likely contain about
as much breakage as going from FORTRAN II to Fortran 77.

I don't see anyone else considering this a major issue in this case as
long as we allot a sufficiently long timeline (and if you search the
records, I'm generally one of the people more likely to raise objections
about causing old code to fail).  If you want to pursue this angle, I
think you should go to c.l.py to see if anyone agrees with you.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From mwh at python.net  Sun Apr 11 06:19:56 2004
From: mwh at python.net (Michael Hudson)
Date: Sun Apr 11 06:20:01 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <04Apr7.183319pdt."58612"@synergy1.parc.xerox.com> (Bill
	Janssen's message of "Wed, 7 Apr 2004 18:33:10 PDT")
References: <04Apr7.183319pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <2m8yh2byur.fsf@starship.python.net>

Bill Janssen <janssen@parc.com> writes:

> I really wish I'd been paying more attention to 328, which I've just
> read.  This proposal seems to break lots and lots and lots of existing
> code.

Lots of *your* code, maybe...  I think you might be one of the few
people who embraced packages as soon as they were available.

> Maybe I'm reading it wrong.
>
> I've written many, many packages that use the form
>
>      import foo
>
> where "foo" is a sibling module in the package.  Or
>
>       from foo import bar
>
> where, again, foo is a sibling to the current module. 

Nevertheless, do you agree or not that it was a mistake that relative
and absolute imports are spelt the same?  I you do, then surely the
longer we leave it, the worse the problem gets...

[...]

> I was thinking (some months ago) that y'all would come up with an
> absolute form instead, something like

Then you really haven't been paying attention!  Pretty much *everyone*
who has contributed to the discussion, and presumably plenty who
haven't (like me), has assumed that absolute imports are more common
and should the default (certainly, all the imports in my own code are
absolute...).

Cheers,
mwh

-- 
  > With Python you can start a thread, but you can't stop it.  Sorry.
  > You'll have to wait until reaches the end of execution.
  So, just the same as c.l.py, then?
                       -- Cliff Wells & Steve Holden, comp.lang.python

From mwh at python.net  Sun Apr 11 06:22:21 2004
From: mwh at python.net (Michael Hudson)
Date: Sun Apr 11 06:22:25 2004
Subject: [Python-Dev] python-dev Summary for 2004-03-16 through
	2004-03-31 [rough draft]
In-Reply-To: <4078AD18.5070502@ocf.berkeley.edu> (Brett C.'s message of
	"Sat, 10 Apr 2004 19:27:36 -0700")
References: <4078AD18.5070502@ocf.berkeley.edu>
Message-ID: <2m4qrqbyqq.fsf@starship.python.net>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> ---------------------------------
> f_tstate gone the way of the dodo
> ---------------------------------
> In case you don't catch the meaning of the title, f_tstate has now
> been removed.
>
> Contributing threads:
>    - `Last chance!
>      <http://mail.python.org/pipermail/python-dev/2004-March/043594.html>`__

Only in stackless Python!

Cheers,
mwh

-- 
  /* I'd just like to take this moment to point out that C has all
     the expressive power of two dixie cups and a string.
   */                       -- Jamie Zawinski from the xkeycaps source

From barry at python.org  Sun Apr 11 11:22:01 2004
From: barry at python.org (Barry Warsaw)
Date: Sun Apr 11 11:22:16 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
Message-ID: <1081696920.17920.76.camel@anthem.wooz.org>

On Sat, 2004-04-10 at 12:57, Andrew Koenig wrote:

> The trouble is that if you commit yourself to backward compatibility, you
> wind up with a growing list of decisions for which the default is wrong but
> you didn't want to break code by changing it.  Each time you make such a
> decision, you make it more difficult the next time to do anything but go for
> compatibility again.  Ultimately, everyone has to remember to override the
> defaults in lots of everyday circumstances, a mental burden that hurts
> everyone from novices to experts.
>  
> On the other hand, if you don't commit yourself to backward compatibility,
> you wind up with people downloading old code that breaks because things have
> changed.
> 
> The third possibility is to support all alternatives forever, perhaps with a
> mechanism such as "from __past__ import foo".  That's possible in theory,
> but if changes include changes in semantics, it may be impossible for old
> and new to coexist.  Moreover, even if they can coexist, testing becomes a
> nightmare.
> 
> It's a trilemma--a choice between three unpleasant alternatives.  I don't
> know the right answer, but I do wish to caution against assuming the problem
> is trivial.

This is one of the reasons I suggested early on that there be a way to
spell -- at the package level by the package developer -- "I'm using
absolute imports here".  Without that spelling, the package would still
use the old semantics, however broken they are <wink>.

E.g. if I could put in email/__init__.py something like:

from __future__ import i_am_absolutely_resolved_about_my_imports

then I'd expect PEP 328 semantics in package email and all subpackages. 
Older packages wouldn't have this declaration and would operate under
the old rules.  (I still don't have any need for relative imports. ;)

-Barry



From aahz at pythoncraft.com  Sun Apr 11 11:54:53 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr 11 11:54:56 2004
Subject: [Python-Dev] python-dev Summary for 2004-03-16 through 2004-03-31
	[rough draft]
In-Reply-To: <2m4qrqbyqq.fsf@starship.python.net>
References: <4078AD18.5070502@ocf.berkeley.edu>
	<2m4qrqbyqq.fsf@starship.python.net>
Message-ID: <20040411155453.GA18539@panix.com>

On Sun, Apr 11, 2004, Michael Hudson wrote:
> "Brett C." <bac@OCF.Berkeley.EDU> writes:
>> 
>> ---------------------------------
>> f_tstate gone the way of the dodo
>> ---------------------------------
>> In case you don't catch the meaning of the title, f_tstate has now
>> been removed.
>>
>> Contributing threads:
>>    - `Last chance!
>>      <http://mail.python.org/pipermail/python-dev/2004-March/043594.html>`__
> 
> Only in stackless Python!

Sure, but Guido said it should be removed from CPython, too.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From skip at pobox.com  Sun Apr 11 12:26:18 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sun Apr 11 12:26:26 2004
Subject: [Python-Dev] python-dev Summary for 2004-03-16 through 2004-03-31
	[rough draft]
In-Reply-To: <20040411155453.GA18539@panix.com>
References: <4078AD18.5070502@ocf.berkeley.edu>
	<2m4qrqbyqq.fsf@starship.python.net>
	<20040411155453.GA18539@panix.com>
Message-ID: <16505.29098.291755.625930@montanaro.dyndns.org>


    >>> In case you don't catch the meaning of the title, f_tstate has now
    >>> been removed.
    >> 
    >> Only in stackless Python!

    aahz> Sure, but Guido said it should be removed from CPython, too.

True, but at this point it still yet to actually be removed.

Skip

From aahz at pythoncraft.com  Sun Apr 11 13:23:51 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr 11 13:23:54 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <1081696920.17920.76.camel@anthem.wooz.org>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
Message-ID: <20040411172351.GA23114@panix.com>

On Sun, Apr 11, 2004, Barry Warsaw wrote:
>
> This is one of the reasons I suggested early on that there be a way to
> spell -- at the package level by the package developer -- "I'm using
> absolute imports here".  Without that spelling, the package would still
> use the old semantics, however broken they are <wink>.
> 
> E.g. if I could put in email/__init__.py something like:
> 
> from __future__ import i_am_absolutely_resolved_about_my_imports
> 
> then I'd expect PEP 328 semantics in package email and all subpackages. 
> Older packages wouldn't have this declaration and would operate under
> the old rules.  (I still don't have any need for relative imports. ;)

Anyone else favor having the ``__future__`` directive apply to the
package and all its subpackages?  (Currently it follows the standard
rules of applying only to each module.)  Note that this would make it
impossible to do what some people want, where a package becomes another
package's subpackage, if the new subpackage relies on relative imports
under the old rules.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From bac at OCF.Berkeley.EDU  Sun Apr 11 15:55:41 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Apr 11 15:55:58 2004
Subject: [Python-Dev] python-dev Summary for 2004-03-16 through 2004-03-31
	[rough draft]
In-Reply-To: <16505.29098.291755.625930@montanaro.dyndns.org>
References: <4078AD18.5070502@ocf.berkeley.edu>	<2m4qrqbyqq.fsf@starship.python.net>	<20040411155453.GA18539@panix.com>
	<16505.29098.291755.625930@montanaro.dyndns.org>
Message-ID: <4079A2BD.4030104@ocf.berkeley.edu>

Skip Montanaro wrote:

>     >>> In case you don't catch the meaning of the title, f_tstate has now
>     >>> been removed.
>     >> 
>     >> Only in stackless Python!
> 
>     aahz> Sure, but Guido said it should be removed from CPython, too.
> 
> True, but at this point it still yet to actually be removed.
> 

Sorry, wrong tense to use.  It's now future tense.

-Brett

From ajole-1 at gci.net  Sun Apr 11 17:59:14 2004
From: ajole-1 at gci.net (Patrick Stinson)
Date: Mon Apr 12 00:09:38 2004
Subject: [Python-Dev] pyc/embedded c++ efficiency wrt binary data
Message-ID: <200404111359.14011.ajole-1@gci.net>

I'm using pyuic, a tool that comes as part of pyqt, that reads images and 
creates a python file with the data embedded in it for easy loading. This is 
based on a similar tool called qembed that embeds image data in C++ header 
files. 
Well, I have found that upon loading the py (9.2MB) file, it takes quite a 
while, but a pyc file is created, and is approx. 1/4 the size. This gives me 
the indication that the binary encoding is quite good, and importing the pyc 
file will takes considerably less time. Does anyone have any idea how the 
binary encoding of pyc files is compared to something like this done in C++? 
Of course, I'm not asking about the ultimate efficiency including processing 
the binary data as pixmaps, but just the loading from disk bit.

Cheers!
-P

From aahz at pythoncraft.com  Mon Apr 12 00:54:49 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 12 00:54:55 2004
Subject: [Python-Dev] pyc/embedded c++ efficiency wrt binary data
In-Reply-To: <200404111359.14011.ajole-1@gci.net>
References: <200404111359.14011.ajole-1@gci.net>
Message-ID: <20040412045449.GA13027@panix.com>

On Sun, Apr 11, 2004, Patrick Stinson wrote:
>
> Well, I have found that upon loading the py (9.2MB) file, it takes
> quite a while, but a pyc file is created, and is approx. 1/4 the
> size. This gives me the indication that the binary encoding is
> quite good, and importing the pyc file will takes considerably less
> time. Does anyone have any idea how the binary encoding of pyc files
> is compared to something like this done in C++?  Of course, I'm not
> asking about the ultimate efficiency including processing the binary
> data as pixmaps, but just the loading from disk bit.

python-dev is for the work of the Python project, not for questions
about using Python.  Please post your question to comp.lang.python.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From chrism at plope.com  Mon Apr 12 00:18:29 2004
From: chrism at plope.com (Chris McDonough)
Date: Mon Apr 12 09:43:49 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040411172351.GA23114@panix.com>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com>
Message-ID: <1081743508.2987.118.camel@james>

What is the rationale given for not allowing:

import .moduleY

as a relative import statement meaning "from the package containing the
module running the import statement, import moduleY"?  The PEP suggests
that "relative imports will require listing specific names to import". 
I'm wondering what about "moduleY" in the above isn't specific?

FWIW, I read through much of the meanderings of the old threads but
never saw a specific damnation-with-cause of this.

- C


On Sun, 2004-04-11 at 13:23, Aahz wrote:
> On Sun, Apr 11, 2004, Barry Warsaw wrote:
> >
> > This is one of the reasons I suggested early on that there be a way to
> > spell -- at the package level by the package developer -- "I'm using
> > absolute imports here".  Without that spelling, the package would still
> > use the old semantics, however broken they are <wink>.
> > 
> > E.g. if I could put in email/__init__.py something like:
> > 
> > from __future__ import i_am_absolutely_resolved_about_my_imports
> > 
> > then I'd expect PEP 328 semantics in package email and all subpackages. 
> > Older packages wouldn't have this declaration and would operate under
> > the old rules.  (I still don't have any need for relative imports. ;)
> 
> Anyone else favor having the ``__future__`` directive apply to the
> package and all its subpackages?  (Currently it follows the standard
> rules of applying only to each module.)  Note that this would make it
> impossible to do what some people want, where a package becomes another
> package's subpackage, if the new subpackage relies on relative imports
> under the old rules.


From barry at python.org  Mon Apr 12 10:06:47 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 12 10:06:58 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040411172351.GA23114@panix.com>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com>
Message-ID: <1081778806.17920.300.camel@anthem.wooz.org>

On Sun, 2004-04-11 at 13:23, Aahz wrote:
> On Sun, Apr 11, 2004, Barry Warsaw wrote:
> >
> > This is one of the reasons I suggested early on that there be a way to
> > spell -- at the package level by the package developer -- "I'm using
> > absolute imports here".  Without that spelling, the package would still
> > use the old semantics, however broken they are <wink>.
> > 
> > E.g. if I could put in email/__init__.py something like:
> > 
> > from __future__ import i_am_absolutely_resolved_about_my_imports
> > 
> > then I'd expect PEP 328 semantics in package email and all subpackages. 
> > Older packages wouldn't have this declaration and would operate under
> > the old rules.  (I still don't have any need for relative imports. ;)
> 
> Anyone else favor having the ``__future__`` directive apply to the
> package and all its subpackages?  (Currently it follows the standard
> rules of applying only to each module.)  Note that this would make it
> impossible to do what some people want, where a package becomes another
> package's subpackage, if the new subpackage relies on relative imports
> under the old rules.

Would that be any different than if modules of the subpackage add the
__future__ in them?  I'm envisioning semantics such that a __future__ in
an __init__.py was the same as if that __future__ was explicitly added
to every module (i.e. it's a convenience).

-Barry



From jeremy at alum.mit.edu  Mon Apr 12 09:41:02 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon Apr 12 10:54:34 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <1081778806.17920.300.camel@anthem.wooz.org>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com>
	<1081778806.17920.300.camel@anthem.wooz.org>
Message-ID: <1081777262.2610.2.camel@localhost.localdomain>

On Mon, 2004-04-12 at 10:06, Barry Warsaw wrote:
> Would that be any different than if modules of the subpackage add the
> __future__ in them?  I'm envisioning semantics such that a __future__ in
> an __init__.py was the same as if that __future__ was explicitly added
> to every module (i.e. it's a convenience).

I think they belong in every module.  A future statement is a compiler
gimmick and modules are compiled separately, so each module ought to
have its own future statement.  If not, the compiler (and human reader)
have no idea whether a future statement is in effect.  Obviously, the
compiler and the reader could learn about some change to import
semantics by studying context, but that's extra complexity I'd like to
avoid (particular in the case of the compiler).

Jeremy



From guido at python.org  Mon Apr 12 10:55:10 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 12 10:55:25 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: Your message of "Mon, 12 Apr 2004 00:18:29 EDT."
	<1081743508.2987.118.camel@james> 
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com> 
	<1081743508.2987.118.camel@james> 
Message-ID: <200404121455.i3CEtAE19580@guido.python.org>

> What is the rationale given for not allowing:
> 
> import .moduleY

For all other forms of import, after

  import XXX.YYY.ZZZ

then

  XXX.YYY.ZZZ

is usable in an expression.  (Very different from Java!)  But

  .moduleY

is not usable in an expression.

(This might be worth adding to the PEP.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jim.jewett at eds.com  Mon Apr 12 12:11:00 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Apr 12 12:12:05 2004
Subject: [Python-Dev] backwards compatibility (was: PEP 328 -- relative and
	multi-line import)
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D45B@USAHM010.amer.corp.eds.com>

Bill Janssen:
> Wow.  This seems amazingly user-hostile.  Users (not Python
> developers) download some two or three year old code to do something
> from an online library, like the Vaults of Parnassus, and it just
> breaks because the meaning of "import" has changed.

Andrew Koenig:

> ... If you commit yourself to backward compatibility, you
> wind up with a growing list of decisions for which the 
> default is wrong ... everyone has to remember to override 
> the defaults in lots of everyday circumstances, a mental
> burden that hurts everyone from novices to experts.
 
> ... If you don't commit yourself to backward compatibility,
> you wind up with people downloading old code that breaks 

> The third possibility is to support all alternatives forever, 
> perhaps with a mechanism such as "from __past__ import foo". 

Most changes are for something that was possible but awkward
(inconvenient/ugly/inefficient) in the past.  Would it be 
reasonable to add a magic attribute like 

	__target_python__ = 2.3

Then the interpreter (or an externally maintained translator)
could translate the code automatically, in at least several 
cases.  Older code wouldn't have this, but that already 
suggests it is pre-(something), and the FAQ could say to that
if downloaded code doesn't work, try the automated translator.

In theory, the translator could also work backwards, so that 
maintainers could start to write True instead of 1.

I would be willing to work on such a translator, though I doubt
I could get it to full strength on my own.

-jJ

From aahz at pythoncraft.com  Mon Apr 12 12:52:49 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 12 12:53:50 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404121455.i3CEtAE19580@guido.python.org>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com>
	<1081743508.2987.118.camel@james>
	<200404121455.i3CEtAE19580@guido.python.org>
Message-ID: <20040412165249.GA4311@panix.com>

On Mon, Apr 12, 2004, Guido van Rossum wrote:
>
> For all other forms of import, after
> 
>   import XXX.YYY.ZZZ
> 
> then
> 
>   XXX.YYY.ZZZ
> 
> is usable in an expression.  (Very different from Java!)  But
> 
>   .moduleY
> 
> is not usable in an expression.
> 
> (This might be worth adding to the PEP.)

Done!
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From aahz at pythoncraft.com  Mon Apr 12 12:55:59 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 12 12:56:43 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <1081777262.2610.2.camel@localhost.localdomain>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com>
	<1081778806.17920.300.camel@anthem.wooz.org>
	<1081777262.2610.2.camel@localhost.localdomain>
Message-ID: <20040412165559.GB4311@panix.com>

On Mon, Apr 12, 2004, Jeremy Hylton wrote:
> On Mon, 2004-04-12 at 10:06, Barry Warsaw wrote:
>>
>> Would that be any different than if modules of the subpackage add the
>> __future__ in them?  I'm envisioning semantics such that a __future__ in
>> an __init__.py was the same as if that __future__ was explicitly added
>> to every module (i.e. it's a convenience).

Right.  And my point is that there's at least one corner case where this
will have definite drawbacks.

> I think they belong in every module.  A future statement is a compiler
> gimmick and modules are compiled separately, so each module ought to
> have its own future statement.  If not, the compiler (and human reader)
> have no idea whether a future statement is in effect.  Obviously, the
> compiler and the reader could learn about some change to import
> semantics by studying context, but that's extra complexity I'd like to
> avoid (particular in the case of the compiler).

That's what I expected the reaction to be from others.  Barry, if you
want your change, lobby Guido.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From barry at python.org  Mon Apr 12 13:35:27 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 12 13:35:37 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040412165559.GB4311@panix.com>
References: <00db01c41f1c$d9beec10$6402a8c0@arkdesktop>
	<1081696920.17920.76.camel@anthem.wooz.org>
	<20040411172351.GA23114@panix.com>
	<1081778806.17920.300.camel@anthem.wooz.org>
	<1081777262.2610.2.camel@localhost.localdomain>
	<20040412165559.GB4311@panix.com>
Message-ID: <1081791326.21906.32.camel@anthem.wooz.org>

On Mon, 2004-04-12 at 12:55, Aahz wrote:

> > I think they belong in every module.  A future statement is a compiler
> > gimmick and modules are compiled separately, so each module ought to
> > have its own future statement.  If not, the compiler (and human reader)
> > have no idea whether a future statement is in effect.  Obviously, the
> > compiler and the reader could learn about some change to import
> > semantics by studying context, but that's extra complexity I'd like to
> > avoid (particular in the case of the compiler).
> 
> That's what I expected the reaction to be from others.  Barry, if you
> want your change, lobby Guido.  ;-)

I might be crazy but I'm not insane. :)

-Barry



From janssen at parc.com  Mon Apr 12 18:03:49 2004
From: janssen at parc.com (Bill Janssen)
Date: Mon Apr 12 18:04:20 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: Your message of "Sat, 10 Apr 2004 15:55:15 PDT."
	<40787B53.9070007@prescod.net> 
Message-ID: <04Apr12.150359pdt."58612"@synergy1.parc.xerox.com>

> My opinion is that the process is correct and it is just a question of 
> whether the timelines are generous enough or not. Somewhere between 
> "Python 2.6" and "never" there must be a point where people have had 
> enough time to change over their code after getting nagged by the 
> DeprecationWarning.

I couldn't agree with you more, Paul.  I think that point is whenever
the major version number is bumped to 3.

> Most people in the Python community know that Python doesn't really have 
> "minor releases" other than bugfix releases.

This is python-dev thinking again.  There are lots of users of Python
who don't want to be "in the Python community", and feel that a
properly managed programming language is one that doesn't require
"membership" to get things right.

Bill

From janssen at parc.com  Mon Apr 12 18:05:58 2004
From: janssen at parc.com (Bill Janssen)
Date: Mon Apr 12 18:06:19 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import 
In-Reply-To: Your message of "Sun, 11 Apr 2004 08:22:01 PDT."
	<1081696920.17920.76.camel@anthem.wooz.org> 
Message-ID: <04Apr12.150604pdt."58612"@synergy1.parc.xerox.com>

> E.g. if I could put in email/__init__.py something like:
> 
> from __future__ import i_am_absolutely_resolved_about_my_imports
> 
> then I'd expect PEP 328 semantics in package email and all subpackages. 
> Older packages wouldn't have this declaration and would operate under
> the old rules.

Sounds like a good idea to me.

Bill

From raymond.hettinger at verizon.net  Tue Apr 13 07:56:27 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Apr 13 07:57:24 2004
Subject: [Python-Dev] Candidate Function Decorator
Message-ID: <000301c4214e$5a3e1400$e530cb97@oemcomputer>

I worked out the details for binding constants and posted a recipe to
ASPN:
 
      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940
 
Example:
 
 
[bind_constants]
def biased_dotprod(v1, v2):
    sum = 0
    for i in xrange(min(len(v1), len(v2)):
        sum += v1[i] * (v2[i] - bias)
    return sum
 
The decorate function binds globals that are known at compile time.  So,
"xrange", "min", and "len" are added to the constant table, but bias is
left as a global lookup (unless it was already defined somewhere before
the function definition).
 
The chief advantage of this construct to avoid code like:  _len=len,
while still getting all of the advantages.  The modules in the sre
package are prime candidates for taking advantage of constant binding.
 
 
Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040413/d8a1937f/attachment.html
From tws5 at mail.ru  Tue Apr 13 10:06:39 2004
From: tws5 at mail.ru (=?koi8-r?Q?=22?=Maxim Krikun=?koi8-r?Q?=22=20?=)
Date: Tue Apr 13 10:06:43 2004
Subject: [Python-Dev] pipe syntax for sequential data
Message-ID: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>

Helo python-dev.

Please take a look at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/276960

This recipe proposes an alternative syntax for sequential data processing functions, that reflects unix shell pipe syntax. Here is an example:

bash_users = ( file('/etc/passwd') | grep('/bin/bash') | aslist )

I don't know if this idea was already explored before; hope this can be useful. For details and sample code please see link above.


regards,

Maxim

From aahz at pythoncraft.com  Tue Apr 13 11:41:16 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr 13 11:41:28 2004
Subject: [Python-Dev] pipe syntax for sequential data
In-Reply-To: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
References: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
Message-ID: <20040413154116.GA12847@panix.com>

On Tue, Apr 13, 2004, "Maxim Krikun"  wrote:
> 
> This recipe proposes an alternative syntax for sequential data
> processing functions, that reflects unix shell pipe syntax. Here is an
> example:
>
> bash_users = ( file('/etc/passwd') | grep('/bin/bash') | aslist )
>
> I don't know if this idea was already explored before; hope this can
> be useful. For details and sample code please see link above.

If you're serious about this, you'll need to write up a PEP and push it
forward.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From jcarlson at uci.edu  Tue Apr 13 12:10:41 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue Apr 13 12:15:03 2004
Subject: [Python-Dev] pipe syntax for sequential data
In-Reply-To: <20040413154116.GA12847@panix.com>
References: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
	<20040413154116.GA12847@panix.com>
Message-ID: <20040413085533.0CF5.JCARLSON@uci.edu>

> > This recipe proposes an alternative syntax for sequential data
> > processing functions, that reflects unix shell pipe syntax. Here is an
> > example:
> >
> > bash_users = ( file('/etc/passwd') | grep('/bin/bash') | aslist )
> >
> > I don't know if this idea was already explored before; hope this can
> > be useful. For details and sample code please see link above.
> 
> If you're serious about this, you'll need to write up a PEP and push it
> forward.


While some would call Python a 'scripting language', I think that using
pipes in such a way in Python would be counter to the zen:
"There should be one-- and preferably only one --obvious way to do it."

We can already use list comprehensions, filter and friends (or ifilter as used by
MK), when such functionality is desired.  IMO, just because you can make
a language  behave like a command line, does not necessarily mean that
you should.

 - Josiah


From FBatista at uniFON.com.ar  Tue Apr 13 12:18:41 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 12:20:47 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>

People:

Tim Peters reviewed the PEP 327, and asked some questions.  Trying to
answer those questions, I found that some items needs to be addressed with
a comunity consensus.

So, the next items are to be included in the PEP when the discussion
finishes:


Exponent Maximum
----------------

The Decimal number composes of three elements: A sign that can be 0 or 1, a
tuple of digits where each can be 0..9, and an exponent. 

The exponent is an integer, and in the actual implementation exists a
maximum
value::

DEFAULT_MAX_EXPONENT = 999999999
DEFAULT_MIN_EXPONENT = -999999999
ABSOLUTE_MAX_EXP = 999999999
ABSOLUTE_MIN_EXP = -999999999

The issue is that this limit is artificial: As long it's a long, you should
be able to make it as big as your memory let you.

In the General Decimal Arithmetic Specification says:

  In the abstract, there is no upper limit on the absolute value of the
  exponent.  In practice there may be some upper limit, E_limit , on the
  absolute value of the exponent.
  
So, should we impose an artificial limit to the exponent?

This is important, as there're several cases where this maximums are checked
and exceptions raised and/or the numbers get changed.


New operations
--------------

Tim Peters found I missed three operations required by the standard.  Those
are:

a. ``to-scientific-string``: This operation converts a number to a string,
   using scientific notation if an exponent is needed. The operation is not
   affected by the context.

b. ``to-engineering-string``: This operation converts a number to a string,
   using engineering notation if an exponent is needed.

c. ``to-number``: This operation converts a string to a number, as defined
   by its abstract representation.

First we should agree the names of the methods, I propose:

a. to_sci_string
b. to_eng_string
c. from_string

The (a) and (b) methods are different from str, as this method just doesn't
adjust the exponent at all.

Regarding the method (c), the only difference with creating the decimal with
Decimal(string) is that method (c) honors the context (if the literal
contains more digits that the current precision the numbers get rounded, and
gets rounded according to the round method specified in context, etc). For
example, with a precision of 9 and with the name I proposed::

>>> Decimal('112233445566')
Decimal( (0, (1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6), 0) )
>>> Decimal.from_string('112233445566')
Decimal( (0, (1, 1, 2, 2, 3, 3, 4, 4, 6), 3L) )


Hash behaviour
--------------

This item actually wasn't raised by Tim, but I found it when implementing
the module.

In the PEP I wrote that Decimal must be hashable. But what hash should it
give?

Should the following be true?::

  hash(Decimal(25) == hash(25)
  hash(Decimal.from_float(25.35) == hash(25.35)
  hash(Decimal('-33.8')) == hash(-33.8)

I don't think so. I think that hash(Decimal(...)) just should return a
different value in each case, but no the same value that other data types.


Thank you all for your feedback.

.	Facundo

From bob at redivi.com  Tue Apr 13 12:39:55 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Apr 13 12:35:51 2004
Subject: [Python-Dev] pipe syntax for sequential data
In-Reply-To: <20040413085533.0CF5.JCARLSON@uci.edu>
References: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
	<20040413154116.GA12847@panix.com>
	<20040413085533.0CF5.JCARLSON@uci.edu>
Message-ID: <31A9B879-8D69-11D8-B460-000A95686CD8@redivi.com>


On Apr 13, 2004, at 12:10 PM, Josiah Carlson wrote:

>>> This recipe proposes an alternative syntax for sequential data
>>> processing functions, that reflects unix shell pipe syntax. Here is 
>>> an
>>> example:
>>>
>>> bash_users = ( file('/etc/passwd') | grep('/bin/bash') | aslist )
>>>
>>> I don't know if this idea was already explored before; hope this can
>>> be useful. For details and sample code please see link above.
>>
>> If you're serious about this, you'll need to write up a PEP and push 
>> it
>> forward.
>
>
> While some would call Python a 'scripting language', I think that using
> pipes in such a way in Python would be counter to the zen:
> "There should be one-- and preferably only one --obvious way to do it."

But there are others, such as "Although practicality beats purity".

This case is already pretty practical using existing syntax though:
[line for line in file('/etc/passwd') if '/bin/bash' in line]

Though it does pollute the locals..

> We can already use list comprehensions, filter and friends (or ifilter 
> as used by
> MK), when such functionality is desired.  IMO, just because you can 
> make
> a language  behave like a command line, does not necessarily mean that
> you should.

Of course you should, but that doesn't mean it should be __builtin__.

-bob


From skip at pobox.com  Tue Apr 13 12:58:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Apr 13 12:59:47 2004
Subject: [Python-Dev] pipe syntax for sequential data
In-Reply-To: <20040413154116.GA12847@panix.com>
References: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
	<20040413154116.GA12847@panix.com>
Message-ID: <16508.7219.539660.287802@montanaro.dyndns.org>


    >> This recipe proposes an alternative syntax for sequential data
    >> processing functions, that reflects unix shell pipe syntax. Here is
    >> an example:
    >> 
    >> bash_users = ( file('/etc/passwd') | grep('/bin/bash') | aslist )

    aahz> If you're serious about this, you'll need to write up a PEP and
    aahz> push it forward.

Also, you should release it as a standalone module (perhaps create a
SourceForge project for it) so it can "cook" for awhile and you can get
feedback and perhaps some new command objects for your pipe.

Skip


From tjreedy at udel.edu  Tue Apr 13 13:40:27 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Apr 13 13:40:35 2004
Subject: [Python-Dev] Re: Candidate Function Decorator
References: <000301c4214e$5a3e1400$e530cb97@oemcomputer>
Message-ID: <c5h8m6$r5n$1@sea.gmane.org>


"Raymond Hettinger" <raymond.hettinger@verizon.net>  [cc'ed]:

>I worked out the details for binding constants and posted a recipe
      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

I have the impression that it is proposed that the function f will *not*
yet be bound to its name (f.func_name) in its environment at the time the
decorator is called.  If so, f.func_name will not appear in env unless you
separately add it.  If you do add it, then I would think that recursive
functions would be bound to themselves and recursive calls would also go
faster, just as with calls to builtins, etc.

The only obvious downside to me is that the circular ref would keep
function around until gc'ed, but that is why gc was added ;-).

Terry J. Reedy




From python at rcn.com  Tue Apr 13 13:48:34 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Apr 13 13:50:34 2004
Subject: [Python-Dev] Re: Candidate Function Decorator
In-Reply-To: <c5h8m6$r5n$1@sea.gmane.org>
Message-ID: <007d01c4217f$8affce40$e530cb97@oemcomputer>

> I have the impression that it is proposed that the function f will
*not*
> yet be bound to its name (f.func_name) in its environment at the time
the
> decorator is called.

Hmm, that would be a change.  Currently, staticmethod() and
classmethod() take fully formed functions as inputs.  So, if your
impression is correct, there would be a small semantic difference
between the [classmethod] prefix and f=classmethod(f) postfix
decoration.


Raymond Hettinger


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From john at neggie.net  Tue Apr 13 13:52:32 2004
From: john at neggie.net (John Belmonte)
Date: Tue Apr 13 13:52:40 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
Message-ID: <407C28E0.7090808@neggie.net>

Hello,

I'd like to make some observations and proposals about pydoc regarding 
slots, properties, and descriptors in general.

Given this program:

     class CountingDescriptor(object):

         def __init__(self, doc=None):
             self.val = 0
             self.__doc__ = doc

         def __get__(self, obj, objtype):
             self.val += 1
             return self.val


     class Foo(object):
         __slots__ = {'bar': 'slot doc test'}

         counter = CountingDescriptor(doc='descriptor doc test')

         def _set_counter_multiple(self, val):
             print 'I hear you want to set me to %s?' % val

         counter_multiple = property(
             lambda self: self.counter*2,
             _set_counter_multiple,
             doc='property doc test'
         )

     print help(Foo)


The output is:

     Help on class Foo in module __main__:

     class Foo(__builtin__.object)
      |  Properties defined here:
      |
      |  counter_multiple
      |      property doc test
      |
      |      lambdaself
      |
      |      <set> = _set_counter_multiple(self, val)
      |
      |  ------------------------------------------------------------
      |  Data and other attributes defined here:
      |
      |  __slots__ = {'bar': 'slot doc test'}
      |
      |  bar = <member 'bar' of 'Foo' objects>
      |
      |  counter = 2


Observations (in order from least to most arguable :-):

     1) pydoc's property display doesn't handle lambda well

     2) pydoc doesn't show the doc string of general descriptors

     3) Using dictionary values as a way to set the doc string of slots, 
as suggested in Guido's "Unifying types and classes", has not been 
implemented.  Currently there is no way to set a doc string on a 
descriptor generated by __slots__.  Even if you could, pydoc would not 
display it, according to #2.

     4) pydoc is displaying implementation details of properties (set 
and get method names, etc.).  Such details should probably not be 
displayed, similar to underbar-prefixed method and data attributes.

Proposals:

     1) remove property-specific handling from pydoc

     2) extend pydoc with descriptor-specific handling.  In a section 
called "Descriptors defined here", list each attribute name, with 
associated doc string if it exists.

     3) extend the __slots__ handler to set the descriptor's doc string 
to dictionary values if the slot's value is a dictionary.  This is the 
only Python change being proposed.

     4) have pydoc make a special case for the display of __slots__. 
Alternatives:
         A) don't display __slots__ if the value is a dictionary.  The 
display of __slots__ is redundant, because all items are displayed 
according to #2.
         B) if for some reason distinguishing slots from other 
descriptors is essential, then always display __slots__, but normalize 
it to a list of attribute names (show only dictionary keys, etc.)


With these changes, the new output of the example would be:

     Help on class Foo in module __main__:

     class Foo(__builtin__.object)
      |  Descriptors defined here:
      |
      |  bar
      |      slot doc test
      |
      |  counter
      |      descriptor doc test
      |
      |  counter_multiple
      |      property doc test



I'd be willing to work on a PEP.  If these issues are covered in or 
conflict with an existing PEP, please let me know.

Thanks,
-John


-- 
http:// if  ile.org/


From mwh at python.net  Tue Apr 13 13:54:44 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Apr 13 13:54:47 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
	(Facundo Batista's message of "Tue, 13 Apr 2004 13:18:41 -0300")
References: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
Message-ID: <2m65c33grf.fsf@starship.python.net>

A comment from the peanut gallery...

"Batista, Facundo" <FBatista@uniFON.com.ar> writes:

> Should the following be true?::
>
>   hash(Decimal(25) == hash(25)
>   hash(Decimal.from_float(25.35) == hash(25.35)
>   hash(Decimal('-33.8')) == hash(-33.8)

Well, if Decimal(25) == 25, they'd better hash the same!  If not, then
what's the point?

Cheers,
mwh

-- 
  I think perhaps we should have electoral collages and construct
  our representatives entirely of little bits of cloth and papier 
  mache.          -- Owen Dunn, ucam.chat, from his review of the year

From pje at telecommunity.com  Tue Apr 13 13:54:58 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 13 13:55:33 2004
Subject: [Python-Dev] Re: Candidate Function Decorator
In-Reply-To: <c5h8m6$r5n$1@sea.gmane.org>
References: <000301c4214e$5a3e1400$e530cb97@oemcomputer>
Message-ID: <5.1.1.6.0.20040413134823.025b0870@telecommunity.com>

At 01:40 PM 4/13/04 -0400, Terry Reedy wrote:

>"Raymond Hettinger" <raymond.hettinger@verizon.net>  [cc'ed]:
>
> >I worked out the details for binding constants and posted a recipe
>       http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940
>
>I have the impression that it is proposed that the function f will *not*
>yet be bound to its name (f.func_name) in its environment at the time the
>decorator is called.  If so, f.func_name will not appear in env unless you
>separately add it.  If you do add it, then I would think that recursive
>functions would be bound to themselves and recursive calls would also go
>faster, just as with calls to builtins, etc.
>
>The only obvious downside to me is that the circular ref would keep
>function around until gc'ed, but that is why gc was added ;-).

You'd have to follow a more complex process to set that up.  First, you'd 
create a code object with the function-constant filled in as None, then 
create a function object from it.  Then, you create a second code object 
containing the new function object in its constants.  Finally, you assign 
the new code object to the new function's func_code attribute.


From pje at telecommunity.com  Tue Apr 13 13:58:08 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 13 13:58:30 2004
Subject: [Python-Dev] Re: Candidate Function Decorator
In-Reply-To: <007d01c4217f$8affce40$e530cb97@oemcomputer>
References: <c5h8m6$r5n$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20040413135528.025a9540@telecommunity.com>

At 01:48 PM 4/13/04 -0400, Raymond Hettinger wrote:
> > I have the impression that it is proposed that the function f will
>*not*
> > yet be bound to its name (f.func_name) in its environment at the time
>the
> > decorator is called.
>
>Hmm, that would be a change.  Currently, staticmethod() and
>classmethod() take fully formed functions as inputs.  So, if your
>impression is correct,

It is.  See PEP 318, under "Current Implementation".  Also, several 
proposed decorators such as propget/propset/propdel and "generic" rely on 
this distinction for their magic.


>there would be a small semantic difference
>between the [classmethod] prefix and f=classmethod(f) postfix
>decoration.

Yes, and that's the desired semantics, as it makes possible many things 
that are quite awkward with the existing syntax.


From FBatista at uniFON.com.ar  Tue Apr 13 14:10:33 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 14:12:42 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383868@escpl.tcp.com.ar>

[Michael Hudson]
#- > Should the following be true?::
#- >
#- >   hash(Decimal(25) == hash(25)
#- >   hash(Decimal.from_float(25.35) == hash(25.35)
#- >   hash(Decimal('-33.8')) == hash(-33.8)
#- 
#- Well, if Decimal(25) == 25, they'd better hash the same!  If 
#- not, then
#- what's the point?

Remember that you only can compare Decimal to ints or long, not float nor
string.

So, This way, hash() must *only* return the same value than the equivalent
integer.

I'm ok with this.

.	Facundo

From pf_moore at yahoo.co.uk  Tue Apr 13 15:20:14 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Apr 13 15:20:23 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
Message-ID: <k70j4rdd.fsf@yahoo.co.uk>

"Batista, Facundo" <FBatista@uniFON.com.ar> writes:

> Exponent Maximum
> ----------------

[...]

> So, should we impose an artificial limit to the exponent?
>
> This is important, as there're several cases where this maximums are
> checked and exceptions raised and/or the numbers get changed.

I don't have a strong opinion either way (as I'll never be likely to
hit a limit in practice) but my instinct says that you should not
impose arbitrary limits. Are there any benefits to having a limit? You
don't mention any in your message...

Paul.
-- 
This signature intentionally left blank


From john at neggie.net  Tue Apr 13 13:10:10 2004
From: john at neggie.net (John Belmonte)
Date: Tue Apr 13 15:21:47 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
Message-ID: <407C1EF2.9000301@neggie.net>

Hello,

I'd like to make some observations and proposals about pydoc regarding 
slots, properties, and descriptors in general.

Given this program:

     class CountingDescriptor(object):

         def __init__(self, doc=None):
             self.val = 0
             self.__doc__ = doc

         def __get__(self, obj, objtype):
             self.val += 1
             return self.val


     class Foo(object):
         __slots__ = {'bar': 'slot doc test'}

         counter = CountingDescriptor(doc='descriptor doc test')

         def _set_counter_multiple(self, val):
             print 'I hear you want to set me to %s?' % val

         counter_multiple = property(
             lambda self: self.counter*2,
             _set_counter_multiple,
             doc='property doc test'
         )

     print help(Foo)


The output is:

     Help on class Foo in module __main__:

     class Foo(__builtin__.object)
      |  Properties defined here:
      |
      |  counter_multiple
      |      property doc test
      |
      |      lambdaself
      |
      |      <set> = _set_counter_multiple(self, val)
      |
      |  ------------------------------------------------------------
      |  Data and other attributes defined here:
      |
      |  __slots__ = {'bar': 'slot doc test'}
      |
      |  bar = <member 'bar' of 'Foo' objects>
      |
      |  counter = 2


Observations (in order from least to most arguable :-):

     1) pydoc's property display doesn't handle lambda well

     2) pydoc doesn't show the doc string of general descriptors

     3) Using dictionary values as a way to set the doc string of slots, 
as suggested in Guido's "Unifying types and classes", has not been 
implemented.  Currently there is no way to set a doc string on a 
descriptor generated by __slots__.  Even if you could, pydoc would not 
display it, according to #2.

     4) pydoc is displaying implementation details of properties (set 
and get method names, etc.).  Such details should probably not be 
displayed, similar to underbar-prefixed method and data attributes.

Proposals:

     1) remove property-specific handling from pydoc

     2) extend pydoc with descriptor-specific handling.  In a section 
called "Descriptors defined here", list each attribute name, with 
associated doc string if it exists.

     3) extend the __slots__ handler to set the descriptor's doc string 
to dictionary values if the slot's value is a dictionary.  This is the 
only Python change being proposed.

     4) have pydoc make a special case for the display of __slots__. 
Alternatives:
         A) don't display __slots__ if the value is a dictionary.  The 
display of __slots__ is redundant, because all items are displayed 
according to #2.
         B) if for some reason distinguishing slots from other 
descriptors is essential, then always display __slots__, but normalize 
it to a list of attribute names (show only dictionary keys, etc.)


With these changes, the new output of the example would be:

     Help on class Foo in module __main__:

     class Foo(__builtin__.object)
      |  Descriptors defined here:
      |
      |  bar
      |      slot doc test
      |
      |  counter
      |      descriptor doc test
      |
      |  counter_multiple
      |      property doc test



I'd be willing to work on a PEP.


-John


-- 
http:// if  ile.org/

From mcherm at mcherm.com  Tue Apr 13 15:31:16 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Apr 13 15:31:20 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <1081884676.407c4004be701@mcherm.com>

Facundo Batista writes:
> Exponent Maximum
> ----------------
  [...]
> The exponent is an integer, and in the actual implementation exists a
> maximum
> value::
> 
> DEFAULT_MAX_EXPONENT = 999999999
> DEFAULT_MIN_EXPONENT = -999999999
> ABSOLUTE_MAX_EXP = 999999999
> ABSOLUTE_MIN_EXP = -999999999
> 
> The issue is that this limit is artificial:
     [...]
> So, should we impose an artificial limit to the exponent?
> 
> This is important, as there're several cases where this 
> maximums are checked
> and exceptions raised and/or the numbers get changed.

I strongly prefer for my programming language not to impose
aribtrary limits, however, I can't see myself using decimal
to represent numbers with 1 billion digits. So IMHO, I would
avoid the limit if possible, but if imposing the limit
permitted substantial code simplification or performance
improvements then I wouldn't complain. Much.

> New operations
> --------------
> 
> Tim Peters found I missed three operations required by the 
> standard.

Sure. Your proposal sounds good.

> Hash behaviour
> --------------
   [...]
> In the PEP I wrote that Decimal must be hashable. But what 
> hash should it
> give?
> 
> Should the following be true?::
> 
>   hash(Decimal(25) == hash(25)
>   hash(Decimal.from_float(25.35) == hash(25.35)
>   hash(Decimal('-33.8')) == hash(-33.8)
> 
> I don't think so. I think that hash(Decimal(...)) just should return a
> different value in each case, but no the same value that 
> other data types.

I disagree. If x == y, then that should imply that hash(x) == hash(y).
So hash(Decimal(25)) == hash(25) had better be true. On the other
hand, hash(Decimal('-33')) CANNOT equal hash(Decimal('-33')), since
the latter must equal hash(-33). And I'm not particularly concerned
about floats... if it's easy to make the hashes equal for those few
numbers which can be expressed EXACTLY as both a (binary) float and
a (decimal) Decimal, then that's be "kinda nice", but since there
are so few values which can be expressed exactly as both a float
and a Decimal, I don't see it as a big issue.

-- Michael Chermside


From FBatista at uniFON.com.ar  Tue Apr 13 15:54:49 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 15:56:51 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383869@escpl.tcp.com.ar>

[Paul Moore]
#- > So, should we impose an artificial limit to the exponent?
#- >
#- > This is important, as there're several cases where this 
#- maximums are
#- > checked and exceptions raised and/or the numbers get changed.
#- 
#- I don't have a strong opinion either way (as I'll never be likely to
#- hit a limit in practice) but my instinct says that you should not
#- impose arbitrary limits. Are there any benefits to having a 
#- limit? You
#- don't mention any in your message...

I don't find any benefit. But the implementation I took to finish imposed
the limits, and I'm searching community agreement before extract them.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040413/2fb70c63/attachment.html
From bob at redivi.com  Tue Apr 13 16:01:22 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Apr 13 15:57:14 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <1081884676.407c4004be701@mcherm.com>
References: <1081884676.407c4004be701@mcherm.com>
Message-ID: <563EA5F2-8D85-11D8-B460-000A95686CD8@redivi.com>


On Apr 13, 2004, at 3:31 PM, Michael Chermside wrote:

> Facundo Batista writes:
>> Hash behaviour
>> --------------
>    [...]
>> In the PEP I wrote that Decimal must be hashable. But what
>> hash should it
>> give?
>>
>> Should the following be true?::
>>
>>   hash(Decimal(25) == hash(25)
>>   hash(Decimal.from_float(25.35) == hash(25.35)
>>   hash(Decimal('-33.8')) == hash(-33.8)
>>
>> I don't think so. I think that hash(Decimal(...)) just should return a
>> different value in each case, but no the same value that
>> other data types.
>
> I disagree. If x == y, then that should imply that hash(x) == hash(y).
> So hash(Decimal(25)) == hash(25) had better be true. On the other
> hand, hash(Decimal('-33')) CANNOT equal hash(Decimal('-33')), since
> the latter must equal hash(-33). And I'm not particularly concerned
> about floats... if it's easy to make the hashes equal for those few
> numbers which can be expressed EXACTLY as both a (binary) float and
> a (decimal) Decimal, then that's be "kinda nice", but since there
> are so few values which can be expressed exactly as both a float
> and a Decimal, I don't see it as a big issue.

 >>> hash(float(1.0)) == hash(1) == hash(1L)
True

Since this is already true, hash(Decimal.from_float(1.0)), 
hash(Decimal(1)), hash(Decimal(1L)), hash(Decimal('1.0')), 
hash(Decimal('1')), hash(Decimal(u'1.0')), etc. better be equal as 
well, whatever that takes.

As far as arbitrary floats hashing equally, I don't really care, but 
the simple ones definitely should.  Hopefully at some point, "float 
constants" (as in coming from source code or user input) in Python will 
actually be Decimal representations, so you don't lose precision until 
you ask for it.

-bob


From fumanchu at amor.org  Tue Apr 13 16:05:25 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Tue Apr 13 16:07:42 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561FF1@opus.amorhq.net>

[Paul Moore] 
#- > So, should we impose an artificial limit to the exponent? 
#- > 
#- > This is important, as there're several cases where this 
#- maximums are 
#- > checked and exceptions raised and/or the numbers get changed. 
#- 
#- I don't have a strong opinion either way (as I'll never be likely to 
#- hit a limit in practice) but my instinct says that you should not 
#- impose arbitrary limits. Are there any benefits to having a 
#- limit? You don't mention any in your message... 

[Facundo Batista]
> I don't find any benefit. But the implementation I took to finish
> imposed the limits, and I'm searching community agreement
> before extract them.

My only "strong opinion" is that, IF there are limits, they should be
consumer-modifiable. Given that, I'd still prefer no limits. In
plusminusspeak:

No limit: +1
Modifiable limits: +0
Concrete limits: -1


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From FBatista at uniFON.com.ar  Tue Apr 13 16:04:02 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 16:08:19 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338386A@escpl.tcp.com.ar>

[Michael Chermside]

#- > The issue is that this limit is artificial:
#-      [...]
#- > So, should we impose an artificial limit to the exponent?
#- > 
#- > This is important, as there're several cases where this 
#- > maximums are checked
#- > and exceptions raised and/or the numbers get changed.
#- 
#- I strongly prefer for my programming language not to impose
#- aribtrary limits, however, I can't see myself using decimal
#- to represent numbers with 1 billion digits. So IMHO, I would
#- avoid the limit if possible, but if imposing the limit
#- permitted substantial code simplification or performance
#- improvements then I wouldn't complain. Much.

Actually, imposing the limit means more code and complexity, and I don't
find any benefit.  But as I answered to Paul, I'm searching for community
agreement before changing functionality that I found in the implementation


#- I disagree. If x == y, then that should imply that hash(x) 
#- == hash(y).
#- So hash(Decimal(25)) == hash(25) had better be true. On the other
#- hand, hash(Decimal('-33')) CANNOT equal hash(Decimal('-33')), since
#- the latter must equal hash(-33). And I'm not particularly concerned
#- about floats... if it's easy to make the hashes equal for those few
#- numbers which can be expressed EXACTLY as both a (binary) float and
#- a (decimal) Decimal, then that's be "kinda nice", but since there
#- are so few values which can be expressed exactly as both a float
#- and a Decimal, I don't see it as a big issue.

So, while Decimal(25) == 25 is True, hash(Decimal(25)) should be equal to
hash(25).

The detail is that you can NOT compare Decimal to floats or strings, so
maybe we should not worry about them to give the same hashes.

I'm OK to make hash(int or long) == hash(Decimal(int or long)). But only to
int or long, not float nor string.

Do you agree?

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040413/855b22a6/attachment-0001.html
From shane at zope.com  Tue Apr 13 16:09:35 2004
From: shane at zope.com (Shane Hathaway)
Date: Tue Apr 13 16:09:28 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <563EA5F2-8D85-11D8-B460-000A95686CD8@redivi.com>
References: <1081884676.407c4004be701@mcherm.com>
	<563EA5F2-8D85-11D8-B460-000A95686CD8@redivi.com>
Message-ID: <407C48FF.1070000@zope.com>

Bob Ippolito wrote:
> As far as arbitrary floats hashing equally, I don't really care, but the 
> simple ones definitely should.  Hopefully at some point, "float 
> constants" (as in coming from source code or user input) in Python will 
> actually be Decimal representations, so you don't lose precision until 
> you ask for it.

Should we expect this to work?

     d = {}
     d[Decimal('1.1')] = 1
     print d[float('1.1')]

Maybe pathological, maybe not.

Shane

From FBatista at uniFON.com.ar  Tue Apr 13 16:08:01 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 16:10:04 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338386B@escpl.tcp.com.ar>

[Robert Brewer]

#- My only "strong opinion" is that, IF there are limits, they should be
#- consumer-modifiable. Given that, I'd still prefer no limits. In
#- plusminusspeak:
#- 
#- No limit: +1
#- Modifiable limits: +0
#- Concrete limits: -1

Nice. Thank you.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040413/8e36e758/attachment.html
From FBatista at uniFON.com.ar  Tue Apr 13 16:15:51 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 16:17:54 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338386C@escpl.tcp.com.ar>

[Bob Ippolito]

#-  >>> hash(float(1.0)) == hash(1) == hash(1L)
#- True
#- 
#- Since this is already true, hash(Decimal.from_float(1.0)), 
#- hash(Decimal(1)), hash(Decimal(1L)), hash(Decimal('1.0')), 
#- hash(Decimal('1')), hash(Decimal(u'1.0')), etc. better be equal as 
#- well, whatever that takes.
#- 
#- As far as arbitrary floats hashing equally, I don't really care, but 
#- the simple ones definitely should.  Hopefully at some point, "float 
#- constants" (as in coming from source code or user input) in 
#- Python will 
#- actually be Decimal representations, so you don't lose 
#- precision until 
#- you ask for it.

I agree. The detail is that the "simple ones" means the same Decimal that
the ones constructed from ints:

>>> Decimal.from_float(1.0)
Decimal( (0, (1,), 0L) )
>>> Decimal(1)
Decimal( (0, (1,), 0) )
>>> hash(Decimal.Decimal(1))
1

So, you'll *always* have the same hash there. 


Regarding user input, treating them as strings,

>>> Decimal('1')
Decimal( (0, (1,), 0) )
>>> hash('1')
1977051568

There, you won't have the same hash that the string. But I don't think we
need this as long as you can not compare Decimal with strings...

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040413/1acbb44d/attachment.html
From FBatista at uniFON.com.ar  Tue Apr 13 16:20:00 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 16:22:16 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338386D@escpl.tcp.com.ar>

[Shane Hathaway]

#- Should we expect this to work?
#- 
#-      d = {}
#-      d[Decimal('1.1')] = 1
#-      print d[float('1.1')]
#- 

No, because Decimal('1.1') is not the same number than float('1.1')!

>>> d = {}
>>> d[Decimal('1.1')] = 1
>>> d
{Decimal( (0, (1, 1), -1) ): 1}
>>> d[float('1.1')]
Traceback (most recent call last):
  File "<pyshell#10>", line 1, in ?
    d[float('1.1')]
KeyError: 1.1000000000000001

In part, this is why you can not compare Decimal with float, and why you can
not create a Decimal directly from float. You can use the alternative
method, but you should know what you're doing:

>>> Decimal.from_float(1.1)
Decimal( (0, (1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 1,
7, 8, 4, 1, 9, 7, 0, 0, 1, 2, 5, 2, 3, 2, 3, 3, 8, 9, 0, 5, 3, 3, 4, 4, 7,
2, 6, 5, 6, 2, 5), -51L) )
>>> Decimal.Decimal.from_float(1.1, 1)
Decimal( (0, (1, 1), -1L) )


.	Facundo

From pf_moore at yahoo.co.uk  Tue Apr 13 16:34:51 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Apr 13 16:34:55 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D0338386A@escpl.tcp.com.ar>
Message-ID: <fzb74nx0.fsf@yahoo.co.uk>

"Batista, Facundo" <FBatista@uniFON.com.ar> writes:

[...]

> Actually, imposing the limit means more code and complexity, and I don't
> find any benefit.  But as I answered to Paul, I'm searching for community
> agreement before changing functionality that I found in the implementation

[...]

> So, while Decimal(25) == 25 is True, hash(Decimal(25)) should be equal to
> hash(25).
>
> The detail is that you can NOT compare Decimal to floats or strings, so
> maybe we should not worry about them to give the same hashes.
>
> I'm OK to make hash(int or long) == hash(Decimal(int or long)). But only to
> int or long, not float nor string.
>
> Do you agree?

So far, I've found your instincts to be pretty good. I'd say

1. Remove the limits.
2. Make hashes the same for int/long, but not str/float.
3. While I'm at it, I'm OK with the names you suggest for the 3 extra
   functions.

Paul.
-- 
This signature intentionally left blank


From FBatista at uniFON.com.ar  Tue Apr 13 16:48:56 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 16:51:04 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338386E@escpl.tcp.com.ar>

[Paul Moore]

#- So far, I've found your instincts to be pretty good. I'd say
#- 
#- 1. Remove the limits.
#- 2. Make hashes the same for int/long, but not str/float.
#- 3. While I'm at it, I'm OK with the names you suggest for the 3 extra
#-    functions.

Thank you very much.

.	Facundo

From bob at redivi.com  Tue Apr 13 17:13:22 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Apr 13 17:09:15 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <fzb74nx0.fsf@yahoo.co.uk>
References: <A128D751272CD411BC9200508BC2194D0338386A@escpl.tcp.com.ar>
	<fzb74nx0.fsf@yahoo.co.uk>
Message-ID: <65040708-8D8F-11D8-B460-000A95686CD8@redivi.com>


On Apr 13, 2004, at 4:34 PM, Paul Moore wrote:

> "Batista, Facundo" <FBatista@uniFON.com.ar> writes:
>
> [...]
>
>> Actually, imposing the limit means more code and complexity, and I 
>> don't
>> find any benefit.  But as I answered to Paul, I'm searching for 
>> community
>> agreement before changing functionality that I found in the 
>> implementation
>
> [...]
>
>> So, while Decimal(25) == 25 is True, hash(Decimal(25)) should be 
>> equal to
>> hash(25).
>>
>> The detail is that you can NOT compare Decimal to floats or strings, 
>> so
>> maybe we should not worry about them to give the same hashes.
>>
>> I'm OK to make hash(int or long) == hash(Decimal(int or long)). But 
>> only to
>> int or long, not float nor string.
>>
>> Do you agree?
>
> So far, I've found your instincts to be pretty good. I'd say
>
> 1. Remove the limits.
> 2. Make hashes the same for int/long, but not str/float.

Make hash(Decimal('123.1')) == hash(Decimal('123.1')) .. there is no 
reason not to.  What's the point of having a Decimal type if the only 
way to get an accurate representation of a non-integer is to do some 
garbage like Decimal(1231)/Decimal(10)?

hash(float(1.0)) == hash(Decimal(1)) comes for free, because 
hash(float(1.0)) == hash(1).  For the few other non-integer numbers 
that can be accurately represented as floating point, it might make 
sense to keep the same property?  Though I suppose that is pretty 
what-the-heck-does-C-do specific, so I wouldn't blame anyone if this 
property couldn't be maintained.

> 3. While I'm at it, I'm OK with the names you suggest for the 3 extra
>    functions.

-bob


From FBatista at uniFON.com.ar  Tue Apr 13 17:23:29 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 13 17:25:33 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383873@escpl.tcp.com.ar>

[Bob Ippolito]

#- Make hash(Decimal('123.1')) == hash(Decimal('123.1')) .. there is no 

I'm pretty sure that those two hash will return the same value, as long have
the same argument (?).


#- hash(float(1.0)) == hash(Decimal(1)) comes for free, because 
#- hash(float(1.0)) == hash(1).  For the few other non-integer numbers 
#- that can be accurately represented as floating point, it might make 
#- sense to keep the same property?  Though I suppose that is pretty 
#- what-the-heck-does-C-do specific, so I wouldn't blame anyone if this 
#- property couldn't be maintained.

OK, so I won't promise it.

.	Facundo

From martin at v.loewis.de  Tue Apr 13 17:26:40 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Apr 13 17:26:56 2004
Subject: [Python-Dev] pipe syntax for sequential data
In-Reply-To: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
References: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
Message-ID: <407C5B10.30103@v.loewis.de>

Maxim Krikun wrote:
> I don't know if this idea was already explored before; hope this can
> be useful. For details and sample code please see link above.

As you can see from the responses you got so far, readership of
python-dev is uncertain what to do with your message, which
primarily originates from the fact that you did not indicate
what you expect readership to do.

If you merely want to educate people to use your recipe,
python-dev might be the wrong forum: readers of that list
will quickly grasp your interesting approach, but may
then chose to ignore it. It may be more educating to people
who were unaware of __ror__ before; you would reach a
greater audience on news:comp.lang.python.

If you also want to release this technique as an actual
library, you should find some packaging, and announce
it to news:comp.lang.python.announce.

If you then also want this library to be included as a
part of the standard Python distribution, you should write
a PEP. You will probably find that people would complain
about the ad-hoc list of filter objects you provide, their
ad-hoc naming, and the relative youth of the entire project.

I personally find the approach entertaining, but likely
will never use it in a real project, because it is too
cute for my taste.

Regards,
Martin


From t-meyer at ihug.co.nz  Tue Apr 13 18:50:19 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue Apr 13 18:50:37 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <1ED4ECF91CDED24C8D012BCF2B034F1305E11770@its-xchg4.massey.ac.nz>
Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F13026F2B86@its-xchg4.massey.ac.nz>

> c. ``to-number``: This operation converts a string to a 
> number, as defined by its abstract representation.
[...]
> c. from_string
[...]
> Regarding the method (c), the only difference with creating 
> the decimal with Decimal(string) is that method (c) honors
> the context (if the literal contains more digits that the
> current precision the numbers get rounded, and gets rounded
> according to the round method specified in context, etc).
> For example, with a precision of 9 and with the name I proposed::
> 
> >>> Decimal('112233445566')
> Decimal( (0, (1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6), 0) )
> >>> Decimal.from_string('112233445566')
> Decimal( (0, (1, 1, 2, 2, 3, 3, 4, 4, 6), 3L) )

As a relative newbie, I think that it would be better if (c) had a name that
somehow indicated the difference; however, I can't think of one offhand that
does :)  It seems to me that it would be easy to assume that the two above
cases were the same (and given that they produce the same result in some
situations, might be missed by weak testing).  Certainly a quick look at
documentation would correct that, but if the name could also reflect the
difference, that would be better.

Even if from_string took a parameter "honour_context", defaulting to True,
that would, IMO, be more clear that there's a difference between the two,
and what it is.

So, 

(a) +1
(b) +1
(c) +0

=Tony Meyer


From raymond.hettinger at verizon.net  Tue Apr 13 19:59:53 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Apr 13 20:00:50 2004
Subject: [Python-Dev] String hash function multiplier
Message-ID: <001c01c421b3$6acd0460$e841fea9@oemcomputer>

Does anyone have any issues with changing the hash multiplier for the
string and Unicode hash functions?

Instead of 1000003, I would like to use Aho's 65599, a prime near 2**16
that is efficiently expressible as (x << 6) + (x << 16) - x.   This
replaces a multiply with fast adds and shifts (with the two shifts
potentially being executed in parallel).  

Googling for "hash 65599" shows a long history of widespread use and
testing without any problems.


Raymond Hettinger



From jepler at unpythonic.net  Tue Apr 13 21:09:54 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Apr 13 21:10:32 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <001c01c421b3$6acd0460$e841fea9@oemcomputer>
References: <001c01c421b3$6acd0460$e841fea9@oemcomputer>
Message-ID: <20040414010954.GA8795@unpythonic.net>

With -O2 -mcpu=i686 or newer, gcc uses "imul" for both 100003 and 65599,
rather than shifts and adds.

There may be a few people who care about some other processor, but I
wouldn't listen to them. (the only non-x86 CPU I program for on a weekly
basis doesn't have hardware multiply, but it's also much too small for
Python)

The current value goes back a long way:
http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Objects/stringobject.c#rev2.31
... all the way back to when Python did string haching instead of
hashing.

Other than some abstract beauty to 65599, are there some other practical
advantages I'm missing?

Jeff

From bob at redivi.com  Tue Apr 13 21:30:28 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Apr 13 21:26:16 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <20040414010954.GA8795@unpythonic.net>
References: <001c01c421b3$6acd0460$e841fea9@oemcomputer>
	<20040414010954.GA8795@unpythonic.net>
Message-ID: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>


On Apr 13, 2004, at 9:09 PM, Jeff Epler wrote:

> With -O2 -mcpu=i686 or newer, gcc uses "imul" for both 100003 and  
> 65599,
> rather than shifts and adds.
>
> There may be a few people who care about some other processor, but I
> wouldn't listen to them. (the only non-x86 CPU I program for on a  
> weekly
> basis doesn't have hardware multiply, but it's also much too small for
> Python)
>
> The current value goes back a long way:
> http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Objects/ 
> stringobject.c#rev2.31
> ... all the way back to when Python did string haching instead of
> hashing.
>
> Other than some abstract beauty to 65599, are there some other  
> practical
> advantages I'm missing?

It's not expected that GCC optimize an integer constant into shifts on  
its own.  Anyways, a practical advantage is that with a sane  
instruction set, like PPC, it saves you a memory access or some  
instructions (depending on the compiler I guess).  Both 100003 and  
65599 are too big to be immediate values in a PPC instruction, but the  
shift constants are not.

I guess the real question for Raymond is, does it really make a  
measurable difference?  And what effect does it have on pickled dicts  
(or other such hash-using data structures), if any?

-bob


From jepler at unpythonic.net  Tue Apr 13 22:04:10 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Apr 13 22:04:49 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
References: <001c01c421b3$6acd0460$e841fea9@oemcomputer>
	<20040414010954.GA8795@unpythonic.net>
	<4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
Message-ID: <20040414020410.GC8795@unpythonic.net>

On Tue, Apr 13, 2004 at 09:30:28PM -0400, Bob Ippolito wrote:
> It's not expected that GCC optimize an integer constant into shifts on  
> its own.

But gcc does! -- with -mcpu=i386, 65599 is optimized into some shifts,
and 100003 is optimized into some very painful use of "lea" (x is in
edx):
   lea    (%edx,%edx,4),%eax  //  eax = 5 * edx
   lea    (%eax,%eax,4),%eax  //  eax = 25 * edx
   lea    (%eax,%eax,4),%eax  //  eax = 125 * edx
   lea    (%eax,%eax,4),%eax  //  eax = 625 * edx
   lea    (%eax,%eax,4),%eax  //  eax = 3125 * edx
   lea    (%eax,%eax,4),%eax  //  eax = 15625 * edx
   shl    $0x5,%eax           //  eax = 50000 * edx
   add    %edx,%eax           //  eax = 50001 * edx
   lea    (%edx,%eax,2),%edx  //  edx = 100003 * edx
On the newer x86 CPUs (starting at i686 / k6) imul is preferred by the
optimizer.

Here's what 65599 gives, with -mcpu=i386 (x is in edx again):
   mov    %edx,%eax           //  eax = edx
   shl    $0xa,%eax           //  eax = edx * 1024
   add    %edx,%eax           //  eax = edx * 1025
   shl    $0x6,%eax           //  eax = edx * 65600
   sub    %edx,%eax           //  eax = edx * 65599
   mov    %eax,%edx           //  edx = eax

If gcc can emit these tortured instruction sequences, but chooses not
to, I have to suspect it knows the properties of the CPU better than me.

Jeff

From python at rcn.com  Tue Apr 13 22:16:16 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Apr 13 22:17:18 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
Message-ID: <001d01c421c6$77b4df00$e841fea9@oemcomputer>

[Jeff Epler]
> > With -O2 -mcpu=i686 or newer, gcc uses "imul" for both 100003 and
> > 65599,
> > rather than shifts and adds.


[Bob Ippolito]
> It's not expected that GCC optimize an integer constant into shifts on
> its own. 

Right, the actual diff is:

*** 1145,1151 ****
        p = (unsigned char *) a->ob_sval;
        x = *p << 7;
        while (--len >= 0)
!               x = (1000003*x) ^ *p++;
        x ^= a->ob_size;
        if (x == -1)
                x = -2;
--- 1152,1158 ----
        p = (unsigned char *) a->ob_sval;
        x = *p << 7;
        while (--len >= 0)
!               x = (x << 6) + (x << 16) - x + (long)*p++;
        x ^= a->ob_size;
        if (x == -1)
                x = -2;



> I guess the real question for Raymond is, does it really make a
> measurable difference?

Yes, the loop runs 20% faster on my Pentium III.  The speedup ought to
be much more dramatic on the Pentium IV (where the integer ALU
instructions speedup from 1 clock to 0.5 clocks while the latency on
integer multiplication slows down from 4 clocks to 14-18 clocks).


>  And what effect does it have on pickled dicts
> (or other such hash-using data structures), if any?

The test suite runs fine.  Dicts may display in a different order than
in previous pythons.  That may upset some doctests if the writer didn't
take Tim's documented advice about making tests that do not depend on
display order.


Raymond


From jepler at unpythonic.net  Tue Apr 13 23:09:38 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Apr 13 23:10:37 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <001d01c421c6$77b4df00$e841fea9@oemcomputer>
References: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
	<001d01c421c6$77b4df00$e841fea9@oemcomputer>
Message-ID: <20040414030938.GD8795@unpythonic.net>

On Tue, Apr 13, 2004 at 10:16:16PM -0400, Raymond Hettinger wrote:
> Yes, the loop runs 20% faster on my Pentium III.  The speedup ought to
> be much more dramatic on the Pentium IV (where the integer ALU
> instructions speedup from 1 clock to 0.5 clocks while the latency on
> integer multiplication slows down from 4 clocks to 14-18 clocks).

I got different results here, also on a Pentium III (but in a laptop)
I'm using gcc, I don't know what you're using.

I get the best results from -O3 -mcpu=pentium3 for both MUL values and
do worse with CLEVER in most cases.

In my test, PyStringObject is not quite the same as in Python, so this
could explain different results.  Use of a different compiler, or gcc
with the implied default -mcpu=i386 (?), might explain why you
benchmarked the shifts as better, too. (-O{2,3} -mcpu=i386)

[* = best alternative for these optimizer flags]
-O -mcpu=i386        -DMUL=100003         1.78
-O -mcpu=i386        -DMUL=65599          1.37    *
-O -mcpu=i386        -DCLEVER             1.40

-O -mcpu=pentium3    -DMUL=100003         1.27    *
-O -mcpu=pentium3    -DMUL=65599          1.27    *
-O -mcpu=pentium3    -DCLEVER             1.35

-O2 -mcpu=i386       -DMUL=100003         1.93
-O2 -mcpu=i386       -DMUL=65599          1.54
-O2 -mcpu=i386       -DCLEVER             1.28    *

-O2 -mcpu=pentium3   -DMUL=100003         1.11    *
-O2 -mcpu=pentium3   -DMUL=65599          1.12
-O2 -mcpu=pentium3   -DCLEVER             1.29

-O3 -mcpu=i386       -DMUL=100003         1.69
-O3 -mcpu=i386       -DMUL=65599          1.28
-O3 -mcpu=i386       -DCLEVER             1.10    *

-O3 -mcpu=pentium3   -DMUL=100003         0.90    *
-O3 -mcpu=pentium3   -DMUL=65599          0.90    *
-O3 -mcpu=pentium3   -DCLEVER             1.05

-Os -mcpu=i386       -DMUL=100003         1.16    *
-Os -mcpu=i386       -DMUL=65599          1.16    *
-Os -mcpu=i386       -DCLEVER             1.45

-Os -mcpu=pentium3   -DMUL=100003         1.05    *
-Os -mcpu=pentium3   -DMUL=65599          1.05    *
-Os -mcpu=pentium3   -DCLEVER             1.30

# alternatives.py
OPT = [
    '-O -mcpu=i386', '-O -mcpu=pentium3',
    '-O2 -mcpu=i386', '-O2 -mcpu=pentium3',
    '-O3 -mcpu=i386', '-O3 -mcpu=pentium3',
    '-Os -mcpu=i386', '-Os -mcpu=pentium3',
]

HASH = ['-DMUL=100003', '-DMUL=65599', '-DCLEVER']

import sys, os
for o in OPT:
    for h in HASH:
        sys.stdout.write("%-20s %-20s " % (o, h))
        sys.stdout.flush()
        os.system("gcc %s %s hashtest.c && TIME=%%U /usr/bin/time ./a.out"
                     % (o, h))
    print


// hashtest.c
#ifndef MUL
#define MUL 100003
#endif

typedef struct {
    long ob_shash;
    unsigned long ob_size;
    unsigned char ob_sval[16];
} PyStringObject;

static long
string_hash(PyStringObject *a)
{
        register int len;
        register unsigned char *p;
        register long x;

        if (a->ob_shash != -1)
                return a->ob_shash;
        len = a->ob_size;
        p = (unsigned char *) a->ob_sval;
        x = *p << 7;
        while (--len >= 0)
#ifdef CLEVER
                x = (x << 6) + (x << 16) - x + (long)*p++;
#else
                x = (MUL*x) ^ *p++;
#endif
        x ^= a->ob_size;
        if (x == -1)
                x = -2;
        a->ob_shash = x;
        return x;
}

int main(void) {
    PyStringObject s = {-1, 13, "hello raymond"};
    int i;
    for(i=0; i < 1<<23; i++) {
        string_hash(&s);
        s.ob_shash = -1;
    }
    return 0;
}

From guido at python.org  Tue Apr 13 23:25:03 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Apr 13 23:26:33 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: Your message of "Tue, 13 Apr 2004 19:59:53 EDT."
	<001c01c421b3$6acd0460$e841fea9@oemcomputer> 
References: <001c01c421b3$6acd0460$e841fea9@oemcomputer> 
Message-ID: <200404140325.i3E3P4722847@guido.python.org>

> Does anyone have any issues with changing the hash multiplier for the
> string and Unicode hash functions?
> 
> Instead of 1000003, I would like to use Aho's 65599, a prime near 2**16
> that is efficiently expressible as (x << 6) + (x << 16) - x.   This
> replaces a multiply with fast adds and shifts (with the two shifts
> potentially being executed in parallel).  
> 
> Googling for "hash 65599" shows a long history of widespread use and
> testing without any problems.

It would break the parrot benchmark, which has a re-implementation of
the hash in Python and assumes/checks at various points that the two
agree.

So I'd rather not see this changed before July 30.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python-dev at zesty.ca  Tue Apr 13 23:29:39 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Tue Apr 13 23:29:11 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <1ED4ECF91CDED24C8D012BCF2B034F13026F2B86@its-xchg4.massey.ac.nz>
References: <1ED4ECF91CDED24C8D012BCF2B034F13026F2B86@its-xchg4.massey.ac.nz>
Message-ID: <Pine.LNX.4.58.0404132157350.18028@server1.LFW.org>

Facundo Batista wrote:
> c. from_string
[...]
> Regarding the method (c), the only difference with creating
> the decimal with Decimal(string) is that method (c) honors
> the context (if the literal contains more digits that the
> current precision the numbers get rounded, and gets rounded
> according to the round method specified in context, etc).
> For example, with a precision of 9 and with the name I proposed::
>
> >>> Decimal('112233445566')
> Decimal( (0, (1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6), 0) )
> >>> Decimal.from_string('112233445566')
> Decimal( (0, (1, 1, 2, 2, 3, 3, 4, 4, 6), 3L) )

On Wed, 14 Apr 2004, Tony Meyer wrote:
> As a relative newbie, I think that it would be better if (c) had a name that
> somehow indicated the difference; however, I can't think of one offhand that
> does :)  It seems to me that it would be easy to assume that the two above
> cases were the same (and given that they produce the same result in some
> situations, might be missed by weak testing).  Certainly a quick look at
> documentation would correct that, but if the name could also reflect the
> difference, that would be better.
>
> Even if from_string took a parameter "honour_context", defaulting to True,
> that would, IMO, be more clear that there's a difference between the two,
> and what it is.

I agree with Tony.  Having both Decimal() and Decimal.from_string() work
in similar but subtly different ways seems likely to be misunderstood.

I also think his last suggestion could provide a nice, convenient solution
to the conversion-from-float issue, since it raises the same question (a
method named "Decimal.from_float" doesn't explain how or why it is different
from just constructing with a float).

So here's a more detailed suggestion.  Suppose s = '12345', f = 12.345,
the default context specifies a precision of 9 as given in the PEP, and
c is a context object with a precision of 6.

Then this:                      would yield this:

Decimal(s)                      Decimal((0, (1, 2, 3, 4, 5), 0))
Decimal(s, 2)                   Decimal((0, (1, 2), 4))
Decimal(s, context=c)           Decimal((0, (1, 2, 3, 4, 5, 0), -1))
Decimal(s, default_context=1)   Decimal((0, (1, 2, 3, 4, 5, 0, 0, 0, 0), -4))

Decimal(f)                      Decimal((0, (1, 2, 3, 4, 5, 0, 0, 0, 0, 0,
                                             0, 0, 0, 0, 0, 0, 0, 6, 3, 9,
                                             4, 8, 8, 4, 6, 2, 1, 8, 4, 0,
                                             9, 0, 1, 6, 7, 2, 8, 4, 0, 1,
                                             1, 8, 4, 0, 8, 2, 0, 3, 1, 2,
                                             5), -49))
Decimal(f, 2)                   Decimal((0, (1, 2), 0))
Decimal(f, context=c)           Decimal((0, (1, 2, 3, 4, 5, 0), -4))
Decimal(f, default_context=1)   Decimal((0, (1, 2, 3, 4, 5, 0, 0, 0, 0), -6))

How about that?

(Do I assume correctly that there will be context objects that wrap up
the precision and rounding mode, and there will be some sort of interface
for getting and setting the current context?)


-- ?!ng

From python-dev at zesty.ca  Tue Apr 13 23:35:19 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Tue Apr 13 23:34:48 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <fzb74nx0.fsf@yahoo.co.uk>
References: <A128D751272CD411BC9200508BC2194D0338386A@escpl.tcp.com.ar>
	<fzb74nx0.fsf@yahoo.co.uk>
Message-ID: <Pine.LNX.4.58.0404132229510.18028@server1.LFW.org>

Sorry if I missed this, but I didn't see a description in the PEP
of how Decimal numbers would convert to other types.  If d is a
Decimal number, I assume int(d) is obvious and float(d) does the
best it can with double precision, right?

How about str(d)?  I would hope that it uses exponential notation
only when necessary to preserve the number of significant digits.

Could we please have repr(d) use the string in the constructor
rather than the tuple form?  That would be much easier to read.

So, for example:

    >>> Decimal('12345')
    Decimal('12345')     # instead of Decimal((0, (1, 2, 3, 4, 5, 0))
    >>> _ * 10
    Decimal('1.2345e1')
    >>> Decimal('12345', 2)
    Decimal('1.2e3')
    >>> Decimal(1.1)
    Decimal('1.100000000000000088817841970012523233890533447265625')
    >>> Decimal(1.1, default_context=1)
    Decimal('1.10000000')


-- ?!ng

From tim_one at email.msn.com  Tue Apr 13 23:40:32 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Tue Apr 13 23:56:38 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <001c01c421b3$6acd0460$e841fea9@oemcomputer>
Message-ID: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>

[Raymond]
> Does anyone have any issues with changing the hash multiplier for the
> string and Unicode hash functions?

Don't touch it unless you can prove major benefits -- it's a remarkable fact
of life that the current multiplier hasn't resulted in any real-life (but
non-contrived) pathological cases.

> Instead of 1000003, I would like to use Aho's 65599, a prime near 2**16
> that is efficiently expressible as (x << 6) + (x << 16) - x.   This
> replaces a multiply with fast adds and shifts (with the two shifts
> potentially being executed in parallel).

It's unclear why that's a good thing.  Perhaps you think shifts and adds are
faster?  I wouldn't -- the imul instruction on modern Pentiums is very fast.

It is clear why it may be a bad thing:  that it *can* be expressed as just a
couple shifts and adds makes it suspect as a good scrambler (read Knuth).

> Googling for "hash 65599" shows a long history of widespread use and
> testing without any problems.

Testing in the context of Python string hashing?  I didn't think so <wink>.
The current multiplier has been studied extensively *in* Python, both via
real-life use and via large-scale focused statistical testing (we got some
money to do that during BeOpen.com's brief life -- a surprise that came out
of that was that CRC made a terrible string hash function as the number of
input strings got large).  The right thing to compare Python's string hash
to is "the standard" Fowler-Noll-Vo string hash, which was developed
independently, but which also ended up using a large multiplier.



From tim_one at email.msn.com  Wed Apr 14 00:53:57 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Wed Apr 14 00:54:16 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
Message-ID: <BAY0-SMTP06jo6BGAsA000078f7@BAY0-SMTP06.bay0.hotmail.com>

[Batista, Facundo]

...

> The exponent is an integer, and in the actual implementation exists a
> maximum value::
>
> DEFAULT_MAX_EXPONENT = 999999999
> DEFAULT_MIN_EXPONENT = -999999999
> ABSOLUTE_MAX_EXP = 999999999
> ABSOLUTE_MIN_EXP = -999999999
>
> The issue is that this limit is artificial: As long it's a long, you
> should be able to make it as big as your memory let you.
>
> In the General Decimal Arithmetic Specification says:
>
>   In the abstract, there is no upper limit on the absolute value of the
>   exponent.  In practice there may be some upper limit, E_limit , on the
>   absolute value of the exponent.
>
> So, should we impose an artificial limit to the exponent?

I expect everyone's initial reaction will be to say "no limit", but think
two or three times about that.  Eliminating any restriction on exponent
magnitude is the same as saying overflow can never happen.  But overflow is
usually a valuable safety net in real life, telling you that your algorithm
has gone insane.  OTOH, there's nothing with physical significance that
requires an exponent anywhere near as large as 999999999, and REXX has used
that exponent limit for years without problems.

Everyone should also note that a restriction on exponent size isn't also a
restriction on the number of significant digits:  if the max exponent is
restricted to 999999999, you can still have more than 10**999999999
significant digits (as soon as you buy enough disk space and a CPU with
unbounded addresses <wink>).


> a. to_sci_string
> b. to_eng_string
> c. from_string
> 
> The (a) and (b) methods are different from str, as this method just
> doesn't adjust the exponent at all.
> 
> Regarding the method (c), the only difference with creating the decimal
> with Decimal(string) is that method (c) honors the context (if the
> literal contains more digits that the current precision the numbers
> get rounded, and gets rounded according to the round method specified
> in context, etc).

(c) also specifies ways to spell special values, like NaNs and infinities.
Does Decimal(string)?  I don't believe the PEP covered that.

In any case, from_string() should be subsumed by Decimal(string), perhaps
with via an optional "use_context" flag argument.

It's quite arguable that str() should be exactly the standard's
to_sci_string() (and for those who haven't read the spec, do before you
complain about that -- no, you don't always get an exponent under
to_sci_string()).

> Hash behaviour
> --------------

Decimals that happen to be exact integers must return the same hash codes as
the hash codes of the ints or longs they're equal to.   All bets are off wrt
hash codes for non-integral Decimals (e.g., hash(Decimal('1.5')) may or may
not be the same as hash(1.5)); promising more than that is simply
impractical.

BTW, hash(Decimal('1.0')) must be the same as hash(Decimal('1.00')) must be
the same as hash(Decimal('0.001e3')) despite that they have different
internal representations.
 
...

> Should the following be true?::
>
>   hash(Decimal(25)) == hash(25)

Yes.

>   hash(Decimal.from_float(25.35) == hash(25.35)

Not defined.

>   hash(Decimal('-33.8')) == hash(-33.8)

Not defined.



From greg at cosc.canterbury.ac.nz  Wed Apr 14 02:24:40 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Apr 14 02:24:58 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040409154252.GB28808@panix.com>
Message-ID: <200404140624.i3E6OeMg017756@cosc353.cosc.canterbury.ac.nz>

Aahz <aahz@pythoncraft.com>:

> Nope.  Absolute imports are the most common form

I don't agree with that. In every major Python application I've
written, the vast majority of imports are from other modules of the
application, which are most sensibly referenced relatively.

I, too, am quite concerned about the imminent breaking of every
existing relative import, apparently because Guido has an ideological
objection to relative imports and wants to punish us for having used
them so freely over the years.

If he goes ahead with this, I suggest we hack his unix system and
remove the ability to use relative pathnames. Let's see how he likes
using that. :-)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From jepler at unpythonic.net  Wed Apr 14 09:08:36 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Apr 14 09:08:41 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <20040414030938.GD8795@unpythonic.net>
References: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
	<001d01c421c6$77b4df00$e841fea9@oemcomputer>
	<20040414030938.GD8795@unpythonic.net>
Message-ID: <20040414130834.GS6139@unpythonic.net>

I benchmarked this on a Pentium 4 and Raymond's hash does indeed come
out ahead, regardless of compiler flags.

Pentium IV, 2.4GHz:
-O2 -mcpu=i386       -DMUL=100003         1.56
-O2 -mcpu=i386       -DMUL=65599          0.78
-O2 -mcpu=i386       -DCLEVER             0.54  *

-O2 -mcpu=pentium3   -DMUL=100003         0.78
-O2 -mcpu=pentium3   -DMUL=65599          0.79
-O2 -mcpu=pentium3   -DCLEVER             0.53  *

-O2 -mcpu=pentium4   -DMUL=100003         0.63
-O2 -mcpu=pentium4   -DMUL=65599          0.65
-O2 -mcpu=pentium4   -DCLEVER             0.50  *

With AMD CPUs, the current multiplier beats both the new multipler and
the version expressed as shifts and adds/subtracts:

On a Duron, 1GHz:
-O2 -mcpu=i386       -DMUL=100003         2.03
-O2 -mcpu=i386       -DMUL=65599          1.04
-O2 -mcpu=i386       -DCLEVER             0.95 *

-O2 -mcpu=athlon     -DMUL=100003         0.92 *
-O2 -mcpu=athlon     -DMUL=65599          1.03
-O2 -mcpu=athlon     -DCLEVER             0.94

On an Athlon XP 2600+:
-O2 -mcpu=i386       -DMUL=100003         0.95
-O2 -mcpu=i386       -DMUL=65599          0.49
-O2 -mcpu=i386       -DCLEVER             0.44 *

-O2 -mcpu=athlon-xp  -DMUL=100003         0.43 *
-O2 -mcpu=athlon-xp  -DMUL=65599          0.48
-O2 -mcpu=athlon-xp  -DCLEVER             0.45

If we want a fast hash function, then we should write one that works a
"long" at a time.  This probably can't be done if hashes must be equal
on different machines, but aren't hashes already different on LP64
machines (because they're 64 bits instead of 32)? (I guess the low-order
bits would be identical)

Long-at-a-time hash, Duron, 1GHz:
-O2 -march=athlon-tbird -DMUL=100003         0.35
-O2 -march=athlon-tbird -DMUL=65599          0.41
-O2 -march=athlon-tbird -DCLEVER             0.42

With this code, it would be necessary to allocate strings "rounded up"
to 4 bytes, and zero the unused bytes.

Or we could do as Tim and Guido suggest: nothing.

Jeff

/* long-at-a-time hashing */
typedef struct {
    long ob_shash;
    unsigned long ob_size;
    union {
            unsigned char ob_sval[16];
            long ob_svall[4];
    };
} PyStringObject;

static long
string_hash(PyStringObject *a)
{
        register int len;
        register long *p;
        register long x;

        if (a->ob_shash != -1)
                return a->ob_shash;
        len = (a->ob_size+3) / 4;
        p = a->ob_svall;
        x = *p << 7;
        while (--len >= 0)
#ifdef CLEVER
                x = (x << 6) + (x << 16) - x + *p++;
#else
                x = (MUL*x) ^ *p++;
#endif
        x ^= a->ob_size;
        if (x == -1)
                x = -2;
        a->ob_shash = x;
        return x;
}


From aahz at pythoncraft.com  Wed Apr 14 10:39:35 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Apr 14 10:39:43 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
Message-ID: <20040414143935.GB5922@panix.com>

On Tue, Apr 13, 2004, Batista, Facundo wrote:
>
> Exponent Maximum
> ----------------
> 
> The Decimal number composes of three elements: A sign that can be 0 or 1, a
> tuple of digits where each can be 0..9, and an exponent. 
> 
> The exponent is an integer, and in the actual implementation exists a
> maximum
> value::
> 
> DEFAULT_MAX_EXPONENT = 999999999
> DEFAULT_MIN_EXPONENT = -999999999
> ABSOLUTE_MAX_EXP = 999999999
> ABSOLUTE_MIN_EXP = -999999999
> 
> The issue is that this limit is artificial: As long it's a long, you should
> be able to make it as big as your memory let you.

As long as it's in Python code, it's easy enough for a whiner to modify
it.  Leave it as-is.  Other than that, I agree with Tim.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From aahz at pythoncraft.com  Wed Apr 14 10:42:26 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Apr 14 10:42:32 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <Pine.LNX.4.58.0404132157350.18028@server1.LFW.org>
References: <1ED4ECF91CDED24C8D012BCF2B034F13026F2B86@its-xchg4.massey.ac.nz>
	<Pine.LNX.4.58.0404132157350.18028@server1.LFW.org>
Message-ID: <20040414144226.GC5922@panix.com>

On Tue, Apr 13, 2004, Ka-Ping Yee wrote:
>
> So here's a more detailed suggestion.  Suppose s = '12345', f = 12.345,
> the default context specifies a precision of 9 as given in the PEP, and
> c is a context object with a precision of 6.
> 
> Then this:                      would yield this:
>
> Decimal(f)                      Decimal((0, (1, 2, 3, 4, 5, 0, 0, 0, 0, 0,
>                                              0, 0, 0, 0, 0, 0, 0, 6, 3, 9,
>                                              4, 8, 8, 4, 6, 2, 1, 8, 4, 0,
>                                              9, 0, 1, 6, 7, 2, 8, 4, 0, 1,
>                                              1, 8, 4, 0, 8, 2, 0, 3, 1, 2,
>                                              5), -49))

I'll continue to oppose allowing Decimal() to take a float argument.
It's just asking for trouble.  Re-read the previous thread about repr()
of floats if you want to argue with me; I shan't say anything new.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From aahz at pythoncraft.com  Wed Apr 14 10:46:14 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Apr 14 10:46:21 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <Pine.LNX.4.58.0404132229510.18028@server1.LFW.org>
References: <A128D751272CD411BC9200508BC2194D0338386A@escpl.tcp.com.ar>
	<fzb74nx0.fsf@yahoo.co.uk>
	<Pine.LNX.4.58.0404132229510.18028@server1.LFW.org>
Message-ID: <20040414144614.GD5922@panix.com>

On Tue, Apr 13, 2004, Ka-Ping Yee wrote:
>
> Could we please have repr(d) use the string in the constructor
> rather than the tuple form?  That would be much easier to read.

I'm not dead-set on using a tuple form, but I think it's more useful for
someone trying to see the internals of a Decimal instance.  The string
form already contains all the necessary information to reconstruct a
Decimal object (unlike floats).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?

From jim.jewett at eds.com  Wed Apr 14 11:27:12 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed Apr 14 11:28:10 2004
Subject: [Python-Dev] PEP 318 - check for consensus
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D465@USAHM010.amer.corp.eds.com>

The decorator discussion has gotten lively again, 
and a summary would be a long list of options.

I'm hoping that at least some of the decisions are
non-controversial.  If you have an opinion on the 
following, please email me directly (to not clutter 
the list).  If several people disagree (or some
disagree strongly), I'll consider the issue still
undecided.

	Issue 1:  When is the name bound?

The current proposal is that the name does not get
bound until after the final decorator is applied.
This means that decorators do not have access to
the function by its name.  

They can learn the name, perhaps through sys._getframe, 
but the name will not yet be bound, unless it happens
to refer to an unrelated object.  (See issue two for
a better way to get the name.)

Does anyone feel strongly that the name should be 
bound to the original function before decorators are
applied?

Does anyone feel strongly that the name should be
bound to the result of each intermediate step, if
there are multiple decorators?

	Issue 2:  Restrictions on decorators

The working assumption had been "any callable, which
will be called with a single object as the argument"

	Issue 2a:  name of the entry point

Is there a strong feeling (in any direction) about 
calling the decorator's "decorate" or "__decorate__"
attribute, rather than its "__call__" attribute?

	Issue 2b:  context available to decorators

Would there be any objection to passing additional
context as keywords?  

decorate(object, name="myfunc")

The obvious keywords are the name and the context's
globals; decorators should probably accept arbitrary
keywords for forward-compatibility.

-jJ

From python at rcn.com  Wed Apr 14 12:05:50 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Apr 14 12:06:49 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
Message-ID: <001901c4223a$5bab0a40$e841fea9@oemcomputer>

> [Raymond]
> > Does anyone have any issues with changing the hash multiplier for
the
> > string and Unicode hash functions?

[Tim]
> Don't touch it unless you can prove major benefits -- it's a
remarkable
> fact
> of life that the current multiplier hasn't resulted in any real-life
(but
> non-contrived) pathological cases.

Will leave it alone. 


>  Perhaps you think shifts and adds
> are
> faster?  I wouldn't -- the imul instruction on modern Pentiums is very
> fast.

On the P4, the documented latency went up from 4 cycles to 14 cycles
while shifts and adds went down to 0.5 cycles and 1 cycle respectively.
Timings confirm the result.

It looks like the best bet is to try to speedup the code without
changing the multiplier.  Intel's software optimization cookbook
recommends a partial unrolling and elimination of data dependencies so
that a second multiply can start 4 cycles after the previous one
started.  If practice bears out the theory, the timings could show a
three or fourfold speedup without changing the multiplier.



> (read Knuth).

Of course, I already have :-)



>  The right thing to compare Python's string hash
> to is "the standard" Fowler-Noll-Vo string hash

Ditto.



Raymond


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From guido at python.org  Wed Apr 14 12:30:30 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Apr 14 12:30:38 2004
Subject: [Python-Dev] PEP 318 - check for consensus
In-Reply-To: Your message of "Wed, 14 Apr 2004 11:27:12 EDT."
	<B8CDFB11BB44D411B8E600508BDF076C1E96D465@USAHM010.amer.corp.eds.com> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D465@USAHM010.amer.corp.eds.com>
Message-ID: <200404141630.i3EGUUk24252@guido.python.org>

> The decorator discussion has gotten lively again, 
> and a summary would be a long list of options.

Really?  If feels pretty quiet here on python-dev.

> I'm hoping that at least some of the decisions are
> non-controversial.  If you have an opinion on the 
> following, please email me directly (to not clutter 
> the list).  If several people disagree (or some
> disagree strongly), I'll consider the issue still
> undecided.
> 
> 	Issue 1:  When is the name bound?
> 
> The current proposal is that the name does not get
> bound until after the final decorator is applied.
> This means that decorators do not have access to
> the function by its name.  
> 
> They can learn the name, perhaps through sys._getframe, 
> but the name will not yet be bound, unless it happens
> to refer to an unrelated object.  (See issue two for
> a better way to get the name.)

A function's name is accessible through its __name__ attribute and
also as func_name.  I think that's quite enough. ;-)

> Does anyone feel strongly that the name should be 
> bound to the original function before decorators are
> applied?
> 
> Does anyone feel strongly that the name should be
> bound to the result of each intermediate step, if
> there are multiple decorators?

No.  The function should not be bound until after all decorators have
been applied.

> 	Issue 2:  Restrictions on decorators
> 
> The working assumption had been "any callable, which
> will be called with a single object as the argument"
> 
> 	Issue 2a:  name of the entry point
> 
> Is there a strong feeling (in any direction) about 
> calling the decorator's "decorate" or "__decorate__"
> attribute, rather than its "__call__" attribute?

It should be __call__; that's the only backwards compatible way, and
makes it easy to create decorators as functions.

> 	Issue 2b:  context available to decorators
> 
> Would there be any objection to passing additional
> context as keywords?  
> 
> decorate(object, name="myfunc")
> 
> The obvious keywords are the name and the context's
> globals; decorators should probably accept arbitrary
> keywords for forward-compatibility.

That's unnecessary; see answer to #1.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Apr 14 12:46:35 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 14 12:47:06 2004
Subject: [Python-Dev] PEP 318 - check for consensus
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D465@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.1.6.0.20040414123205.021d0380@telecommunity.com>

(I'm replying to this on the list because there are a couple of errors in 
it regarding names and scopes, and you probably don't want the private 
votes you get to be based on misunderstandings.)


At 11:27 AM 4/14/04 -0400, Jewett, Jim J wrote:

>         Issue 1:  When is the name bound?
>
>The current proposal is that the name does not get
>bound until after the final decorator is applied.
>This means that decorators do not have access to
>the function by its name.
>
>They can learn the name, perhaps through sys._getframe,

Actually, they can learn the name via the 'func_name' or '__name__' 
attribute of the function.  sys._getframe is only needed to access any 
*previous* binding of the name, or to do other magic in the class or module 
namespace.


>but the name will not yet be bound, unless it happens
>to refer to an unrelated object.  (See issue two for
>a better way to get the name.)
>
>Does anyone feel strongly that the name should be
>bound to the original function before decorators are
>applied?
>
>Does anyone feel strongly that the name should be
>bound to the result of each intermediate step, if
>there are multiple decorators?

A strong -1 on both questions.  The name should not be bound until after 
all decorators have completed.  Nobody has presented any use cases for 
early binding of the name, but several use cases for late binding the 
name.  Also, all actual implementation patches to date implement the late 
binding.


>         Issue 2:  Restrictions on decorators
>
>The working assumption had been "any callable, which
>will be called with a single object as the argument"

Yes.  However, Mike Pall's suggestion of an e.g. '__decorate__' member has 
some very appealing characteristics, with the downside of requiring a more 
complex implementation.


>         Issue 2a:  name of the entry point
>
>Is there a strong feeling (in any direction) about
>calling the decorator's "decorate" or "__decorate__"
>attribute, rather than its "__call__" attribute?

+0.5.  I like it, but not enough to want to take the time to write a new 
patch to implement it, at least not this month.  :)


>         Issue 2b:  context available to decorators
>
>Would there be any objection to passing additional
>context as keywords?

Only that builtin decorators like classmethod and staticmethod would have 
to change to support it.


>decorate(object, name="myfunc")
>
>The obvious keywords are the name and the context's
>globals;

Actually, those are not the obvious keywords.  The function's name is an 
attribute of the function, so it's redundant.  For that matter, a 
function's globals are available as 'func_globals'.  It's the enclosing 
*local* scope that the function was defined in that's needed.  For a 
function defined at module level, the globals and locals are the same, but 
for functions defined in a class body, you need the class locals.  So, 
'locals' would be a more obvious keyword.


>  decorators should probably accept arbitrary
>keywords for forward-compatibility.

I'm not sure this is a good idea.  The only additional argument that anyone 
has proposed that makes sense is the local scope.  I can't imagine what 
kind of change to Python would result in there being anything else that one 
might want.  On the other hand, I *can* imagine accidentally calling a 
decorator (assuming __call__ is used) and passing it invalid keywords that 
are then ignored, silencing an otherwise useful error message.


From pje at telecommunity.com  Wed Apr 14 12:51:31 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 14 12:51:54 2004
Subject: [Python-Dev] PEP 318 - check for consensus
In-Reply-To: <200404141630.i3EGUUk24252@guido.python.org>
References: <Your message of "Wed, 14 Apr 2004 11:27:12 EDT."
	<B8CDFB11BB44D411B8E600508BDF076C1E96D465@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D465@USAHM010.amer.corp.eds.com>
Message-ID: <5.1.1.6.0.20040414124720.021d3260@telecommunity.com>

At 09:30 AM 4/14/04 -0700, Guido van Rossum wrote:
>[much more succinct presentation of the points I just wrote]

The time machine strikes again!  I'm really going to have to start waiting 
longer before I post; by the time I finish writing and editing and hit 
"Send", your reply to the same issue pops up in my mailbox, almost without 
fail.  OTOH, maybe I should just try writing shorter messages.  :)

(Sadly, writing shorter messages usually takes me longer, because of all 
the editing-down I have to do...)


From theller at python.net  Wed Apr 14 13:40:44 2004
From: theller at python.net (Thomas Heller)
Date: Wed Apr 14 13:40:50 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
Message-ID: <zn9esbj7.fsf@python.net>

I've made a short script (74 lines) for Inno Setup, which builds an
installer for the Python debug exes, dlls, and pyds.

The installer copies these debug images into the correct places when a
'normal' installation of Python is found in the registry.

I don't think it would be a good idea to publish this installer on
python.org, but it may be convenient for people who build python from
source, because they no longer have to copy the files in place manually.

Can I check this into the release-23maint branch in the PCBuild
directory, or are there any objections?

Thomas


From fdrake at acm.org  Wed Apr 14 14:00:05 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed Apr 14 14:00:26 2004
Subject: [Python-Dev] PEP 318 - check for consensus
In-Reply-To: <5.1.1.6.0.20040414123205.021d0380@telecommunity.com>
References: <5.1.1.6.0.20040414123205.021d0380@telecommunity.com>
Message-ID: <200404141400.05071.fdrake@acm.org>

On Wednesday 14 April 2004 12:46 pm, Phillip J. Eby wrote:
 > Actually, those are not the obvious keywords.  The function's name is an
 > attribute of the function, so it's redundant.  For that matter, a
 > function's globals are available as 'func_globals'.  It's the enclosing
 > *local* scope that the function was defined in that's needed.  For a
 > function defined at module level, the globals and locals are the same, but
 > for functions defined in a class body, you need the class locals.  So,
 > 'locals' would be a more obvious keyword.

Locals would certainly be nice to have.  What should be done about the case 
of:

    def func():
        [decorator]
        def f(): pass
        return f

I think the decorator would want some way of determining that the locals can't 
be modified so an appropriate error can be raised, if it's interested in 
modifying the namespace directly.  This could be done by returning a dict 
proxy that denies writes; we have one of those already.

I don't know that there are many examples where we want write access to the 
locals in practice; the most interesting case for me is the set of property 
decorators that have been discussed, and read access is sufficient for those.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From andymac at bullseye.apana.org.au  Wed Apr 14 09:34:13 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Wed Apr 14 15:23:23 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <20040414130834.GS6139@unpythonic.net>
References: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
	<001d01c421c6$77b4df00$e841fea9@oemcomputer>
	<20040414030938.GD8795@unpythonic.net>
	<20040414130834.GS6139@unpythonic.net>
Message-ID: <20040414232756.Q67555@bullseye.apana.org.au>

On Wed, 14 Apr 2004, Jeff Epler wrote:

> Pentium IV, 2.4GHz:
> -O2 -mcpu=i386       -DMUL=100003         1.56

{...}

> -O2 -mcpu=pentium4   -DMUL=100003         0.63

{...}

> With AMD CPUs, the current multiplier beats both the new multipler and
> the version expressed as shifts and adds/subtracts:

{...}

> On an Athlon XP 2600+:
> -O2 -mcpu=i386       -DMUL=100003         0.95

{...}

> -O2 -mcpu=athlon-xp  -DMUL=100003         0.43 *

{...}

> Long-at-a-time hash, Duron, 1GHz:
> -O2 -march=athlon-tbird -DMUL=100003         0.35

Ummm...  are you showing what you think you're showing here?  As I recall,
i386 gcc uses -mcpu and -march differently to most other architectures:
- -mcpu just sets scheduling parameters, but not instruction set;
- -march sets the instruction set.

So most of the timings you show are for the i386 instruction set, but with
different scheduling.  The exception is the long-at-a-time hash.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia

From martin at v.loewis.de  Wed Apr 14 15:32:11 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Apr 14 15:32:24 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
In-Reply-To: <zn9esbj7.fsf@python.net>
References: <zn9esbj7.fsf@python.net>
Message-ID: <407D91BB.2030603@v.loewis.de>

Thomas Heller wrote:
  > Can I check this into the release-23maint branch in the PCBuild
> directory, or are there any objections?

What about PCbuild/python.iss? Is it not sufficient?

Regards,
Martin


From jepler at unpythonic.net  Wed Apr 14 15:33:59 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Apr 14 15:35:07 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <20040414232756.Q67555@bullseye.apana.org.au>
References: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
	<001d01c421c6$77b4df00$e841fea9@oemcomputer>
	<20040414030938.GD8795@unpythonic.net>
	<20040414130834.GS6139@unpythonic.net>
	<20040414232756.Q67555@bullseye.apana.org.au>
Message-ID: <20040414193359.GW6139@unpythonic.net>


On Wed, Apr 14, 2004 at 11:34:13PM +1000, Andrew MacIntyre wrote:
> Ummm...  are you showing what you think you're showing here?  As I recall,
> i386 gcc uses -mcpu and -march differently to most other architectures:
> - -mcpu just sets scheduling parameters, but not instruction set;
> - -march sets the instruction set.

I'm pretty sure I am. (I'm not sure why I used -march on the
long-at-a-time version, though)

imul is on all x86 architectures, but whether to use it or not depends
on the characteristics of the target CPU.  With -mcpu=i386, imul is
considered quite slow and a shift sequence is (almost?) always preferred
when one operand is constant.  With -mcpu=i686 and newer, imul seems to
be preferred.

I did actually inspect the generated assembly with -mcpu=i386 and
-mcpu=i686, though I didn't look at it for all the combinations I
posted.

Jeff

From theller at python.net  Wed Apr 14 15:37:22 2004
From: theller at python.net (Thomas Heller)
Date: Wed Apr 14 15:37:30 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
In-Reply-To: <407D91BB.2030603@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed,
	14 Apr 2004 21:32:11 +0200")
References: <zn9esbj7.fsf@python.net> <407D91BB.2030603@v.loewis.de>
Message-ID: <u0zmqrkd.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Thomas Heller wrote:
>   > Can I check this into the release-23maint branch in the PCBuild
>> directory, or are there any objections?
>
> What about PCbuild/python.iss? Is it not sufficient?

PCBuild/python.iss is an (currently unused) alternative to the wise
setup script which installs the 'official' Python distribution, while
the script I mentioned only installs the debug compiled binaries (dll's,
pyd's, exe's, and lib's) into an already existing installation.

Thomas


From niemeyer at conectiva.com  Wed Apr 14 15:51:15 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Wed Apr 14 15:50:58 2004
Subject: [Python-Dev] pipe syntax for sequential data
In-Reply-To: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
References: <E1BDOYh-00049E-00.tws5-mail-ru@f23.mail.ru>
Message-ID: <20040414195114.GA27608@burma.localdomain>

> bash_users = ( file('/etc/passwd') | grep('/bin/bash') | aslist )

I don't think I'll be using this myself, but it's
undoubtfully clever! :-)

-- 
Gustavo Niemeyer
http://niemeyer.net

From martin at v.loewis.de  Wed Apr 14 16:10:12 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Apr 14 16:10:25 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
In-Reply-To: <u0zmqrkd.fsf@python.net>
References: <zn9esbj7.fsf@python.net> <407D91BB.2030603@v.loewis.de>
	<u0zmqrkd.fsf@python.net>
Message-ID: <407D9AA4.10108@v.loewis.de>

Thomas Heller wrote:
> PCBuild/python.iss is an (currently unused) alternative to the wise
> setup script which installs the 'official' Python distribution, while
> the script I mentioned only installs the debug compiled binaries (dll's,
> pyd's, exe's, and lib's) into an already existing installation.

It's not really unused: people do submit patches for it from time to
time. Also, it does instally pyd files. So I wonder whether you could
use it as-is.

OTOH, I don't mind having an additional file in the next 2.3 release.

Regards,
Martin


From FBatista at uniFON.com.ar  Wed Apr 14 17:29:53 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 14 17:32:02 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338387A@escpl.tcp.com.ar>

[Tony Meyer]

#- As a relative newbie, I think that it would be better if (c) 
#- had a name that
#- somehow indicated the difference; however, I can't think of 
#- one offhand that
#- does :)  It seems to me that it would be easy to assume that 

The name cames from the specification. A better name it's obviously more
useful. If you think one, propose it to the list and 
if get accepted, I'll change the code. It better be soon, because when the
interface get fixed, I don't think we all would want to change it.


#- Even if from_string took a parameter "honour_context", 
#- defaulting to True,
#- that would, IMO, be more clear that there's a difference 
#- between the two,
#- and what it is.

Don't like it, because the doc specify to honor the context. I don't want
the method to comply with the specification regarding the value of an
argument.

.	Facundo

From FBatista at uniFON.com.ar  Wed Apr 14 17:38:26 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 14 17:40:32 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338387B@escpl.tcp.com.ar>

[Ka-Ping Yee]

#- I agree with Tony.  Having both Decimal() and 
#- Decimal.from_string() work
#- in similar but subtly different ways seems likely to be 
#- misunderstood.

Decimal(string) works the way we want/need/desire.
Decimal.from_string(string) works the way the documentation specify.

I think we can only change the way the former works or the name of the
latter.


#- I also think his last suggestion could provide a nice, 
#- convenient solution
#- to the conversion-from-float issue, since it raises the same 
#- question (a
#- method named "Decimal.from_float" doesn't explain how or why 
#- it is different
#- from just constructing with a float).

You can NOT construct directly from a float. That's the difference.


#- So here's a more detailed suggestion.  Suppose s = '12345', 
#- f = 12.345,
#- the default context specifies a precision of 9 as given in 
#- the PEP, and
#- c is a context object with a precision of 6.
#- 
#- Then this:                      would yield this:
#- 
#- Decimal(s)                      Decimal((0, (1, 2, 3, 4, 5), 0))
#- ...
#- Decimal(s, context=c)           Decimal((0, (1, 2, 3, 4, 5, 0), -1))
#- ...
#- Decimal(f)                      Decimal((0, (1, 2, 3, 4, 5, 0, 0, 0, 0,
0,...
#- ...

The community agreed to not create directly from float. You have the
from_float() method instead.

Also, remember that the context doesn't affect the creation when you create
directly from string (only if you use from_string).  Your example means to
change the behaviour of the creation when creating directly from string? And
when create directly from int or long?


#- (Do I assume correctly that there will be context objects 
#- that wrap up
#- the precision and rounding mode, and there will be some sort 
#- of interface
#- for getting and setting the current context?)

Yes. You can get the context with getcontext() and then change it
attributes.

.	Facundo

From mikepy-0404 at mike.de  Wed Apr 14 17:49:45 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Wed Apr 14 17:50:24 2004
Subject: [Python-Dev] Optimization targets (was: String hash function
	multiplier)
In-Reply-To: <001901c4223a$5bab0a40$e841fea9@oemcomputer>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
Message-ID: <20040414214945.GA3443@mike.de>

Hi,

Raymond wrote:
> It looks like the best bet is to try to speedup the code without
> changing the multiplier.

Indeed. And while you are at it, there are other optimizations, that
seem more promising:

I compiled a recent CVS Python with profiling and here is a list of the
top CPU hogs (on a Pentium III, your mileage may vary):

pystone:

  CPU%  Function Name
----------------------------
 55.44  eval_frame
  7.30  lookdict_string
  4.34  PyFrame_New
  3.73  frame_dealloc
  1.73  vgetargs1
  1.65  PyDict_SetItem
  1.42  string_richcompare
  1.15  PyObject_GC_UnTrack
  1.11  PyObject_RichCompare
  1.08  PyInt_FromLong
  1.08  tupledealloc
  1.04  insertdict

parrotbench:

  CPU%  Function Name
----------------------------
 23.65  eval_frame
  8.68  l_divmod
  4.43  lookdict_string
  2.95  k_mul
  2.27  PyType_IsSubtype
  2.23  PyObject_Malloc
  2.09  x_add
  2.05  PyObject_Free
  2.05  tupledealloc

Arguably parrotbench is a bit unrepresentative here. And beware: due to
automatic inlining of static functions the real offender may be hidden
(x_divmod is the hog, not l_divmod).

Anyway, this just confirms that the most important optimization targets are:
1. eval_frame
2. string keyed dictionaries
3. frame handling

I think 3. needs an algorithmic approach (there was some discussion about
it a few weeks ago), while 1. and 2. look like a code generation issue.

So I took a look at the machine code that GCC generates on x86 with -O3
for these: uh oh, not a pretty sight. The main problems are bad branch
predictions and lack of inlining.

About branch predictions:

The worst offender is the main code path for eval_frame: it gets split
across half a dozen segments. The code path for the non-argument bytecodes
is in fact the slowest.

Similarly the code for lookdict_string branches around like crazy. The
most likely code path is not the fastest path.

One solution is to use likely()/unlikely() macros (using __builtin_expect).
This is a good solution for small, isolated and performance critical code.
It does not work well for eval_frame, though (I tried).

But GCC has more to offer: read the man page entries for -fprofile-arcs
and -fbranch-probabilities. Here is a short recipe:

Go to your Python source directory and do this:

$ mkdir build_profile
$ cd build_profile
$ ../configure         # add any options you may need
$ make
$ mv python python_orig

Edit the Makefile and add -fprofile-arcs to OPT.

$ rm Python/ceval.o
$ make
$ mv python python_profile

Run your favourite benchmark(s), but only invoke python *once*:

$ ./python_profile -c 'import test.pystone; test.pystone.main(loops=100000)'

Forget about the performance numbers the benchmark reports. Never use
an executable compiled with profiling for comparison. But ... you should
now have a new (binary) file called Python/ceval.da that contains the
profiled branch probabilities.

Edit the Makefile and replace -fprofile-arcs with -fbranch-probabilities

$ rm Python/ceval.o
$ make
$ mv python python_opt

Then compare the benchmarks:

$ ./python_orig -c 'import test.pystone; test.pystone.main(loops=100000)'
$ ./python_opt -c 'import test.pystone; test.pystone.main(loops=100000)'

On my machine I get a 10-15% speedup. But we only optimized ceval.c ...

So repeat the above steps, but now delete Python/ceval.o and Objects/*.o
each time. Now I get about 20-25% speedup for pystone and 10% speedup for
parrotbench! Not bad, huh?

Now the bad news: I don't know how to integrate this into the regular
build process. So this is not an option for everyone, but ambitious
packagers might want to take the trouble to do this by hand.

Oh and of course neither pystone nor parrotbench are representative
of the branch probabilities of any particular application. But for
predicting eval_frame and lookdict_string, they are probably good enough.

On a related note: GCC uses random branch probabilities with -O, when
no probability information is present (no __builtin_expect or *.da files).
Use -fno-guess-branch-probability if you want predictable timings on
recompiles.


About inlining:

- The opcode decoding with NEXTOP/HAS_ARG/NEXTARG does not compile well.
  GCC has trouble inferring the lifetimes for opcode and oparg, too.
  Ideas:
  - Use a special inline assembly variant for x86.
  OR
  - Move NEXTARG to each opcode that needs it and make oparg a local block
    variable.
  - Use NEXTOP directly for the switch and refetch opcode into a local
    block variable where necessary.

- The Py_Ticker check and the tracing support should be moved out of the
  core loop. The Py_Ticker check may be put into a subroutine and called
  from selected opcodes only (haven't checked if that works). I have no
  (good) idea about how to get rid of the tracing support, though.

- _PyString_Eq() is a good candidate to inline for lookdict_string().
  I.e. a new macro in stringobject.h. But wait ...

- GCC generates pretty slow code for _PyString_Eq() (at least on x86).
  This is mainly due to a bad branch prediction (we should use likely())
  but also due to the penalty for 8-bit partial register load/compare
  generated by the ob_sval dereferences. The latter is completely useless
  because memcmp() is inlined (with repz cmpsb) and duplicates that
  comparison. Omitting the *ob_sval comparison is faster.

- Create a macro that inlines the first part of PyObject_Hash().
  Useful for PyDict_SetItem() and others.

I bet there are more, but I'm running out of time right now -- sorry.

Bye,
     Mike

From FBatista at uniFON.com.ar  Wed Apr 14 17:51:08 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 14 17:53:18 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338387C@escpl.tcp.com.ar>

[Tim Peters]

#- I expect everyone's initial reaction will be to say "no 
#- limit", but think
#- two or three times about that.  Eliminating any restriction 
#- on exponent
#- magnitude is the same as saying overflow can never happen.  
#- But overflow is
#- usually a valuable safety net in real life, telling you that 
#- your algorithm
#- has gone insane.  OTOH, there's nothing with physical 
#- significance that
#- requires an exponent anywhere near as large as 999999999, 
#- and REXX has used
#- that exponent limit for years without problems.

But it's still an artificial restriction. We have no-limit longs, should
have limited Decimal?

And also implies more code and complexity in the module. But this is a side
effect and not the primary concern in the decision.


#- (c) also specifies ways to spell special values, like NaNs 
#- and infinities.
#- Does Decimal(string)?  I don't believe the PEP covered that.

Yes, it does. I missed mention it and an example in the PEP. Already fixed.


#- In any case, from_string() should be subsumed by 
#- Decimal(string), perhaps
#- with via an optional "use_context" flag argument.

You mean something like Decimal(string, use_context=True)? And when creating
from long, for example? Remember that the spec says that the context affect
the operations, not creations (with the exception of from_string).


#- It's quite arguable that str() should be exactly the standard's
#- to_sci_string() (and for those who haven't read the spec, do 
#- before you
#- complain about that -- no, you don't always get an exponent under
#- to_sci_string()).

I'm lost here. Are you saying that str() should have the same behaviour that
to_sci_string()?


#- > Hash behaviour
#- > --------------
#- 
#- Decimals that happen to be exact integers must return the 
#- same hash codes as
#- the hash codes of the ints or longs they're equal to.   All 
#- bets are off wrt
#- hash codes for non-integral Decimals (e.g., 
#- hash(Decimal('1.5')) may or may
#- not be the same as hash(1.5)); promising more than that is simply
#- impractical.
#- 
#- BTW, hash(Decimal('1.0')) must be the same as 
#- hash(Decimal('1.00')) must be
#- the same as hash(Decimal('0.001e3')) despite that they have different
#- internal representations.

I agree completely with this behaviour.


.	Facundo

From FBatista at uniFON.com.ar  Wed Apr 14 17:53:19 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 14 17:55:32 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338387D@escpl.tcp.com.ar>

[Aahz]

#- > Exponent Maximum
#- > ----------------
#- > 
#- > The Decimal number composes of three elements: A sign that 
#- can be 0 or 1, a
#- > tuple of digits where each can be 0..9, and an exponent. 
#- > 
#- > The exponent is an integer, and in the actual 
#- implementation exists a
#- > maximum
#- > value::
#- > 
#- > DEFAULT_MAX_EXPONENT = 999999999
#- > DEFAULT_MIN_EXPONENT = -999999999
#- > ABSOLUTE_MAX_EXP = 999999999
#- > ABSOLUTE_MIN_EXP = -999999999
#- > 
#- > The issue is that this limit is artificial: As long it's a 
#- long, you should
#- > be able to make it as big as your memory let you.
#- 
#- As long as it's in Python code, it's easy enough for a 
#- whiner to modify
#- it.  Leave it as-is.  Other than that, I agree with Tim.

That means that I should let the limit be, because can be easily modified? 

If I don't put the limit, nobody will need to modify it, :)

.	Facundo

From FBatista at uniFON.com.ar  Wed Apr 14 17:58:11 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 14 18:00:21 2004
Subject: [Python-Dev] Decimal conversion to string
Message-ID: <A128D751272CD411BC9200508BC2194D0338387E@escpl.tcp.com.ar>

[Ka-Ping Yee]

#- Could we please have repr(d) use the string in the constructor
#- rather than the tuple form?  That would be much easier to read.
#- 
#- So, for example:
#- 
#-     >>> Decimal('12345')
#-     Decimal('12345')     # instead of Decimal((0, (1, 2, 3, 4, 5, 0))
#-     >>> _ * 10
#-     Decimal('1.2345e1')
#-     >>> Decimal('12345', 2)
#-     Decimal('1.2e3')
#-     >>> Decimal(1.1)
#-     Decimal('1.100000000000000088817841970012523233890533447265625')
#-     >>> Decimal(1.1, default_context=1)
#-     Decimal('1.10000000')

To *me* is more natural the tuples. But that's only because that's what the
internal objects are.

An intermediate solution can be something like: 

>>> Decimal('12345')
Decimal(0, 12345, 0)

a tuple of three values: sign, coefficient, and exponent. The three of them
as integers.}

Anyway, if you want to change the repr() behaviour, you should push it hard
to the list. I'm -0 on it.

.	Facundo

From greg at cosc.canterbury.ac.nz  Wed Apr 14 19:51:08 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Apr 14 19:51:19 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <20040411172351.GA23114@panix.com>
Message-ID: <200404142351.i3ENp81h019114@cosc353.cosc.canterbury.ac.nz>

Aahz <aahz@pythoncraft.com>:

> Anyone else favor having the ``__future__`` directive apply to the
> package and all its subpackages?

-1. Future statements should only apply to the module
that contains them.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Wed Apr 14 20:14:21 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Apr 14 20:14:29 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <1081743508.2987.118.camel@james>
Message-ID: <200404150014.i3F0ELhf019167@cosc353.cosc.canterbury.ac.nz>

> What is the rationale given for not allowing:
> 
> import .moduleY
> 
> as a relative import statement meaning "from the package containing the
> module running the import statement, import moduleY"?

Probably because it's not clear what name to bind the
imported module to, and in what namespace.

With the current import syntax, you refer to the imported
module using exactly the same name (or dotted sequence of
names) that you used when importing, but that's not
possible in this case, because ".moduleY" is not a
valid Python expression.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Wed Apr 14 21:45:09 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Apr 14 21:45:58 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <20040414144614.GD5922@panix.com>
Message-ID: <200404150145.i3F1j9tl019325@cosc353.cosc.canterbury.ac.nz>

Aahz <aahz@pythoncraft.com>:

> I'm not dead-set on using a tuple form, but I think it's more useful for
> someone trying to see the internals of a Decimal instance. 

But who cares about the gory details of the internals?
The most important thing about a repr() is to unambiguously
indicate the object's type; the next is to show something
about its value, as far as reasonably possible. What's
actually inside the object doesn't matter much.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From edloper at gradient.cis.upenn.edu  Wed Apr 14 22:24:13 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Wed Apr 14 22:22:37 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <E1BDvxx-0007q7-0l@mail.python.org>
References: <E1BDvxx-0007q7-0l@mail.python.org>
Message-ID: <407DF24D.1000607@gradient.cis.upenn.edu>

[Facundo wrote:]
> Anyway, if you want to change the repr() behaviour, you should push it hard
> to the list. I'm -0 on it.

+1 on changing the repr behavior to show:

   Decimal('12345')

Instead of:

   Decimal(0, (1, 2, 3, 4, 5, 0))

Using the latter representation puts up an unnecessary barrier to using 
decimal, that I'm guessing many people (esp newbies) won't want to put 
up with.  In particular, it makes decimal much less friendly for 
interactive use.

-Edward


From tim_one at email.msn.com  Wed Apr 14 23:13:45 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Wed Apr 14 23:14:05 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <001901c4223a$5bab0a40$e841fea9@oemcomputer>
Message-ID: <BAY0-SMTP05tLamTtYD00007b38@BAY0-SMTP05.bay0.hotmail.com>

[Raymond]
> On the P4, the documented latency went up from 4 cycles to 14 cycles
> while shifts and adds went down to 0.5 cycles and 1 cycle respectively.
> Timings confirm the result.

What makes the current stepping of some flavor of P4 overwhelmingly
important <0.3 wink>?  Tradeoffs at this level change with every chip
generation, and within a generation vary across vendors.  In particular,
they can make int multiply as fast as they want just by throwing more highly
regular silicon at it (which is why its relative performance varies so much
across chip generations, and is usually worst in the earliest iterations of
a new generation).

My overwhelming concern isn't micro-efficiency of the generated code for the
string-hash loop, it's that the resulting hash have high quality.  I don't
know that the "little prime" is vulnerable to bad cases in non-contrived
real life, but we have years of evidence suggesting that the "big prime"
isn't.  If we can develop evidence that the little prime doesn't either,
great, then I lose all objections to using it.

> It looks like the best bet is to try to speedup the code without
> changing the multiplier.  Intel's software optimization cookbook
> recommends a partial unrolling and elimination of data dependencies so
> that a second multiply can start 4 cycles after the previous one
> started.  If practice bears out the theory, the timings could show a
> three or fourfold speedup without changing the multiplier.

Or leave it alone.  Hand unrolling a loop all but guarantees that the next
generation of compiler optimizer won't be able to reverse-engineer the
original "natural" loop structure, and so won't be able to do the code
transformation that works best for the next generation of hardware tricks.
IOW, once you start down the path of hand-optimizing for a specific compiler
+ HW combo, there are two outcomes:  (1) someone makes this their job
forever after; or, (2) the performance of the code actually gets worse over
time (compared to what it would have been if the original code had been left
dirt simple).

I happen to have an app (spambayes) that makes very heavy use of
string-keyed dicts, and where interning is impractical (so the string hashes
can't be avoided).  Still, speed up string hashing by a factor of 10, and it
won't make enough difference in that app's throughput to mention.  I also
have a hyper-threaded box today, and the imul stalls in this P4 give great
opportunities for other processes to get work done <0.9 wink>.



From tim_one at email.msn.com  Wed Apr 14 23:13:45 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Wed Apr 14 23:14:13 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D0338387C@escpl.tcp.com.ar>
Message-ID: <BAY0-SMTP05bftRe0qs00007b39@BAY0-SMTP05.bay0.hotmail.com>

[Facundo Batista, on max exponent]
> But it's still an artificial restriction.  We have no-limit longs,
> should have limited Decimal?

Nobody has a use for unbounded exponents, so it's a silly thing to support
if there are costs.  There are costs, including:

+ As mentioned last time, eliminating bounds on exponents effectively
  means overflow (and underflow) can never happen.  But overflow
  *is* a valuable safety net in real life fp use, like a canary
  in a coal mine, giving danger signs early when a program goes
  insane.  This is of no real interest to you (your real goal is a Money
  type), it's of interest to people who will use Decimal *as* a form
  of floating-point arithmetic (your use cases are more fixed-point).

+ Virtually all implementations of 854 use (and as IBM's standard even
  suggests) "forbidden" exponent values to encode non-finite
  numbers (infinities and NaNs).  A bounded exponent can do this at
  virtually no extra storage cost.  If the exponent is unbounded,
  then additional bits have to be used instead.  This cost remains
  hidden until more time- and space- efficient implementations are
  attempted.

+ Big as it is, the IBM standard is a tiny start at supplying a
  complete numeric facility.  Having no bound on exponent size will
  enormously complicate the implementations of, e.g., decimal sin()
  and cos() (there's then no a priori limit on how many digits of
  pi effectively need to be known in order to perform argument
  reduction).

> And also implies more code and complexity in the module.  But this
> is a side effect and not the primary concern in the decision.

I don't know how it's implemented now.  "The natural way" to check exponent
bounds is in a common routine called at the end of every operation; in that
case, the "more code and complexity" doesn't amount to anything worth
mentioning.

...

> #- In any case, from_string() should be subsumed by
> #- Decimal(string), perhaps with via an optional "use_context" flag
> #- argument.

> You mean something like Decimal(string, use_context=True)?

Yes.

> And when creating from long, for example?

Sure.

> Remember that the spec says that the context affect
> the operations,

Yes.

> not creations (with the exception of from_string).

I don't understand.  The only place the spec uses the word "creation" is in
regard to the creation of context objects.  What do you mean by "creations"?
Context does affect every way of creating a decimal number in the spec; the
only ways to create numbers covered by the spec are via operations (and
from_string isn't a special case here, from_string is just one of the
required operations in the spec, and all operations covered by the spec
respect context).

> #- It's quite arguable that str() should be exactly the standard's
> #- to_sci_string() (and for those who haven't read the spec, do
> #- before you complain about that -- no, you don't always get an
> #- exponent under to_sci_string()).

> I'm lost here. Are you saying that str() should have the same behaviour
> that to_sci_string()?

Yes, that str(Decimal) should produce exactly the string the spec specifies
as being the result of its abstract to-sci-string operation.  There's no
requirement to have a method *named* "to_sci_string", the only requirement
is that *some* way to spell to-sci-string's functionality be supplied.  The
meaning of to-sci-string is precisely specified by the standard, and is a
good choice for both str(Decimal) and repr(Decimal).  (It's both
human-readable and exact.)



From tim_one at email.msn.com  Wed Apr 14 23:13:45 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Wed Apr 14 23:14:17 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <A128D751272CD411BC9200508BC2194D0338387E@escpl.tcp.com.ar>
Message-ID: <BAY0-SMTP05BPo5X3a500007b3a@BAY0-SMTP05.bay0.hotmail.com>

[Ping]
>> Could we please have repr(d) use the string in the constructor
>> rather than the tuple form?  That would be much easier to read.

[Batista, Facundo]
> To *me* is more natural the tuples. But that's only because that's what
> the internal objects are.

The standard's to-scientific-string operation produces a human-readable
string with 1-to-1 correspondence between strings and internal
representations.  So it's suitable for both human use and as repr() output;
it's also suitable for str() output.  Tuples are (IMO) *only* interesting to
implementers.  That's OK -- give 'em an as_tuple() method instead.  The
repr() and str() forms really should be readable by end users.

> An intermediate solution can be something like:
>
> >>> Decimal('12345')
> Decimal(0, 12345, 0)
>
> a tuple of three values: sign, coefficient, and exponent. The three of
> them as integers.}

Then we've got quadratic-time requirements for eval(repr(Decimal)), because
the coeffecient takes time quadratic in the number of decimal digits to
convert to a Python integer, and then there's *another* quadratic-time
operation needed to break the Python integer back into its digits.  The
string and tuple forms are linear-time, so don't suffer this ugliness.  The
to-scientific-string form is also readable by end-users with no education on
the topic.

> Anyway, if you want to change the repr() behaviour, you should push it
> hard to the list. I'm -0 on it.

+1 here, for the reasons above.

Also +0 on adding an .as_tuple() method.  I can't be stronger than +0 on
that, because tuples of single decimal digits reveal *too* much about the
current implementation.  For example, a speedier implementation could use
base 100 or base 10000 (or ...) internally, and then its idea of a natural
tuple representation would be correspondingly different.  The standard
already goes to great pains to define a canonical and readable string
representation, so let's use it.



From bsder at allcaps.org  Thu Apr 15 02:05:27 2004
From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Thu Apr 15 02:01:32 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
Message-ID: <20040414225101.A4530@mail.allcaps.org>

On Tue, 13 Apr 2004, Batista, Facundo wrote:

> The issue is that this limit is artificial: As long it's a long, you should
> be able to make it as big as your memory let you.

I agree with Tim that underflow/overflow are useful flags of something
going wrong.  Very few people would connect Decimal overflow with "Gee,
why is Python allocating all of my system's memory?"  I would like to see
a soft default limit that can be modified.

As for the value of the default limits, I would actually choose both a
default precision and exponent limit which would be within the range
represented by a double precision floating point number.

This helps in two cases:

1) People will try to interconvert FP and Decimal at various points.
Having one with a significantly different range will certainly give some
surprises. (example: Decimal would get converted to double precision FP
before being used in OpenGL or Direct3D to draw a pie chart, graph, etc.)

2) Having a default range somewhere inside double precision means that
some mythical "efficient" implementation could actually use double
precision FP operations to get close and then use a cleanup step to check
the final digit.  (Very useful for exponentials)

-a

From python-dev at zesty.ca  Thu Apr 15 05:49:17 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Thu Apr 15 05:48:49 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D0338387B@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D0338387B@escpl.tcp.com.ar>
Message-ID: <Pine.LNX.4.58.0404150428060.18028@server1.LFW.org>

On Wed, 14 Apr 2004, Batista, Facundo wrote:
> Decimal(string) works the way we want/need/desire.
> Decimal.from_string(string) works the way the documentation specify.
>
> I think we can only change the way the former works or the name of the
> latter.

Or you can add an optional argument, which is what I suggested.
Aren't you going to need an optional argument anyway to let the
user specify a context?

> > I also think his last suggestion could provide a nice, convenient
> > solution to the conversion-from-float issue, since it raises the
> > same question (a method named "Decimal.from_float" doesn't explain
> > how or why it is different from just constructing with a float).
>
> You can NOT construct directly from a float. That's the difference.

I understand that you want to make it hard for people to construct
a Decimal from a float.  But what you just stated does not explain
anything.

I assume the purpose of from_float is to prevent people from being
confused about whether Decimal(1.1) produces exactly 1.1 or the machine
value of the double-precision rounded float closest to 1.1.  Is that
correct?

If so, then the real exceptional case is *exact* conversion, not float
conversion.  Decimal(1.1, precision=4) is unambiguous.  Therefore:

    Decimal(value, default_context=True)
    Decimal(value, context=c)
    Decimal(value, precision=p)

should all work for integer, float, string, or Decimal values.

To protect against the ambiguity of Decimal(1.1),

    Decimal(value)

should work for integer, string, or Decimal values, and raise an
exception for a float, such as "precision must be specified for
conversion from float to Decimal".

The extra method should be specifically for the exceptional case,
and named appropriately.

    Decimal.exact_float(value)

It would serve just one purpose: allow exact conversion from floats
when it's really desired.

This would both (a) separate the functionality between Decimal()
and Decimal.exact_float() in a more logical way; and (b) give
the extra method a more meaningful, useful name -- a name that
really explains why the extra method exists.

> Also, remember that the context doesn't affect the creation when you create
> directly from string (only if you use from_string).  Your example means to
> change the behaviour of the creation when creating directly from string? And
> when create directly from int or long?

That's what I'm suggesting: that you can supply a context object,
or ask for the default context, when constructing any Decimal value.


-- ?!ng

From aahz at pythoncraft.com  Thu Apr 15 06:06:36 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 15 06:06:42 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <20040414225101.A4530@mail.allcaps.org>
References: <A128D751272CD411BC9200508BC2194D03383864@escpl.tcp.com.ar>
	<20040414225101.A4530@mail.allcaps.org>
Message-ID: <20040415100636.GA16358@panix.com>

On Wed, Apr 14, 2004, Andrew P. Lentvorski, Jr. wrote:
>
> As for the value of the default limits, I would actually choose both a
> default precision and exponent limit which would be within the range
> represented by a double precision floating point number.

-1

(slow connection, no time to argue.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From mwh at python.net  Thu Apr 15 07:05:37 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Apr 15 07:05:42 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <20040414214945.GA3443@mike.de> (Mike Pall's message of "Wed,
	14 Apr 2004 23:49:45 +0200")
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de>
Message-ID: <2mekqp1oxq.fsf@starship.python.net>

Mike Pall <mikepy-0404@mike.de> writes:

> Hi,
>
> Raymond wrote:
>> It looks like the best bet is to try to speedup the code without
>> changing the multiplier.
>
> Indeed. And while you are at it, there are other optimizations, that
> seem more promising:
>
> I compiled a recent CVS Python with profiling and here is a list of the
> top CPU hogs (on a Pentium III, your mileage may vary):

I played this game recently, but on a G3 ibook, which is probably a
much more boring processor from a scheduling point of view.

> pystone:
>
>   CPU%  Function Name
> ----------------------------
>  55.44  eval_frame
>   7.30  lookdict_string
>   4.34  PyFrame_New
>   3.73  frame_dealloc
>   1.73  vgetargs1
>   1.65  PyDict_SetItem
>   1.42  string_richcompare
>   1.15  PyObject_GC_UnTrack
>   1.11  PyObject_RichCompare
>   1.08  PyInt_FromLong
>   1.08  tupledealloc
>   1.04  insertdict

I saw similar results to this, tho' I don't remember lookdict_string
being so high on the list.

> parrotbench:
>
>   CPU%  Function Name
> ----------------------------
>  23.65  eval_frame
>   8.68  l_divmod
>   4.43  lookdict_string
>   2.95  k_mul
>   2.27  PyType_IsSubtype
>   2.23  PyObject_Malloc
>   2.09  x_add
>   2.05  PyObject_Free
>   2.05  tupledealloc
>
> Arguably parrotbench is a bit unrepresentative here. And beware: due to
> automatic inlining of static functions the real offender may be hidden
> (x_divmod is the hog, not l_divmod).

Probably a fine candidate function for rewriting in assembly too...

> Anyway, this just confirms that the most important optimization targets are:
> 1. eval_frame
> 2. string keyed dictionaries
> 3. frame handling

[...]

> But GCC has more to offer: read the man page entries for -fprofile-arcs
> and -fbranch-probabilities. Here is a short recipe:

I tried this on the ibook and I found that it made a small difference
*on the program you ran to generate the profile data* (e.g. pystone),
but made naff all difference for something else.  I can well believe
that it makes more difference on a P4 or G5.

[snippety]

> I bet there are more, but I'm running out of time right now --
> sorry.

I certainly don't want to discourage people from optimizing Python's
current implementation, but...

Some months ago (just after I played with -fprofile-arcs, most likely)
I wrote a rant about improving Python's performance, which I've
finally got around to uploading:

    http://starship.python.net/crew/mwh/hacks/speeding-python.html

Tell me what you think!

Cheers,
mwh

-- 
  Hmmm... its Sunday afternoon: I could do my work, or I could do a
  Fourier analysis of my computer's fan noise.
       -- Amit Muthu, ucam.chat (from Owen Dunn's summary of the year)

From FBatista at uniFON.com.ar  Thu Apr 15 09:27:17 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 15 09:29:34 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383884@escpl.tcp.com.ar>

[Tim Peters]

#- [Facundo Batista, on max exponent]
#- > But it's still an artificial restriction.  We have no-limit longs,
#- > should have limited Decimal?
#- 
#- Nobody has a use for unbounded exponents, so it's a silly 
#- thing to support
#- if there are costs.  There are costs, including:
#- 
#- + As mentioned last time, eliminating bounds on exponents effectively
#-   ...
#- + Virtually all implementations of 854 use (and as IBM's 
#-   ...
#- + Big as it is, the IBM standard is a tiny start at supplying a
#-   ...

The feeling previous to this mail was that an artificial limit was silly if
you don't gain nothing.

Here are three valuable reasons for keep the limits.

So, if nobody opposes strongly, I'll put these reasons in the PEP and keep
the limit.



#- > not creations (with the exception of from_string).
#- 
#- I don't understand.  The only place the spec uses the word 
#- "creation" is in
#- regard to the creation of context objects.  What do you mean 
#- by "creations"?
#- Context does affect every way of creating a decimal number 
#- in the spec; the
#- only ways to create numbers covered by the spec are via 
#- operations (and
#- from_string isn't a special case here, from_string is just one of the
#- required operations in the spec, and all operations covered 
#- by the spec
#- respect context).

Until now, Decimal does not uses context in creation time (with creation I
mean when you create an object instantiating the class).

This allows to not lose any information at creation time, and only use the
context in operations.

Do you want to change this behaviour?



#- > I'm lost here. Are you saying that str() should have the 
#- same behaviour
#- > that to_sci_string()?
#- 
#- Yes, that str(Decimal) should produce exactly the string the 

Ok.


#- supplied.  The
#- meaning of to-sci-string is precisely specified by the 
#- standard, and is a
#- good choice for both str(Decimal) and repr(Decimal).  (It's both
#- human-readable and exact.)

The only issue *I* see here, is that you don't have a clear representation
of the internals of the object, but yes, you have an exact representation of
that number. Let me show an example:

>>> d = Decimal.Decimal((1, (1,2,3,4,0), 1))
>>> d
Decimal( (1, (1, 2, 3, 4, 0), 1L) )
>>> d.to_sci_string()
'-1.2340E+5'

There you have to count the decimals and use the exponent in the string to
know what the real exponent is (but I don't know how much important this
is).

.	Facundo

From mcherm at mcherm.com  Thu Apr 15 09:32:23 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Apr 15 09:32:29 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <1082035943.407e8ee7975ef@mcherm.com>

Andrew P. Lentvorski writes:
> As for the value of the default limits, I would actually choose both a
> default precision and exponent limit which would be within the range
> represented by a double precision floating point number.

I strongly object. I have one particular use case for decimal, and I
consider it to be a very important one.... to store and manipulate
monetary amounts. Putting a limit as small as that used for floats
would significantly harm my use case.

On the other hand, your argument seems to be that it will make it easy
to convert between floats and Decimals. I disagree... if there were
simple easy ways to convert between floats and Decimals (like a
Decimal constructor that took a float) then people who didn't know
better would use them and then start running into all sorts of errors
due to floating point imprecision -- the very thing that Decimal is
supposed to save us from. That's why we EXPLICITLY chose to make it
possible but somewhat awkward to convert to/from floats.

You can't hide the difficulties inherent in (binary) floating point
by just sweeping them under the rug. And Decimal can't make it easier
and better (at some cost of performance and additional complexity) if
you attempt to tie it to float.

As for the limits themselves, I remain -0 on them. Doesn't pose a
problem for me but why would you bother? Of course, Tim and others
have posted answers to this question, mostly just so people will
get overflow errors instead of out-of-memory errors (and possibly
some vague future advantage to implementers of a "complete 
numerical package" including things like sin() and cos()).

I-never-make-errors-anyway-lly yours,

-- Michael Chermside


From mikepy-0404 at mike.de  Thu Apr 15 09:36:13 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Thu Apr 15 09:36:32 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <2mekqp1oxq.fsf@starship.python.net>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net>
Message-ID: <20040415133613.GA4735@mike.de>

Hi,

mwh wrote:
> > (x_divmod is the hog, not l_divmod).
> 
> Probably a fine candidate function for rewriting in assembly too...

As a data point: I once had the doubtful pleasure to write a long-integer
library for cryptography. Hand-crafted x86 assembler outperforms plain
(but carefully optimized) C code by a factor of 2 to 3.

But Python's long-int code is a lot slower than e.g. gmp (factor 15-25
for mul/div, factor 100 for modular exponentiation).

I assume the difference between C and assembler is less pronounced with
other processors.

The register pressure issue may soon be a moot point with x86-64, though.
It has been shown that 64 bit pointers slow things down a bit, but compilers
just love the extra registers (R8-R15).

> > But GCC has more to offer: read the man page entries for -fprofile-arcs
> > and -fbranch-probabilities. Here is a short recipe:
> 
> I tried this on the ibook and I found that it made a small difference
> *on the program you ran to generate the profile data* (e.g. pystone),
> but made naff all difference for something else.  I can well believe
> that it makes more difference on a P4 or G5.

For x86 even profiling python -c 'pass' makes a major difference.
And the speed-ups are applicable to almost any program, since the
branch predictions for eval_frame and lookdict_string affect all
Python programs.

I'm currently engaged in a private e-mail conversation with Raymond
on how to convince GCC to generate good code on x86 without the help
of profiling.

> I wrote a rant about improving Python's performance, which I've
> finally got around to uploading:
> 
>     http://starship.python.net/crew/mwh/hacks/speeding-python.html
> 
> Tell me what you think!

About GC: yes, refcounting is the silent killer. But there's a lot to
optimize even without discarding refcounting. E.g. the code generated
for Py_DECREF is awful (spread across >3500 locations) and
PyObject_GC_UnTrack needs some work, too.

About psyco: I think it's wonderful. But, you are right, nobody is using it.
Why? Simple: It's not 'on' by default.

About type inference: maybe the right way to go with Python is lazy and
pure runtime type inference? Close to what psyco does.

About type declarations: this is too contrary to the Pythonic way of
thinking. And before we start to implement this, we should make sure
that it's a lot faster than a pure dynamic type inferencing approach.

About PyPy: very interesting, will take a closer look. But there's still
a long road ahead ...

Bye,
     Mike

From FBatista at uniFON.com.ar  Thu Apr 15 09:45:58 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 15 09:48:10 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383885@escpl.tcp.com.ar>

[Ka-Ping Yee]

#- Aren't you going to need an optional argument anyway to let the
#- user specify a context?

What for?


#- I assume the purpose of from_float is to prevent people from being
#- confused about whether Decimal(1.1) produces exactly 1.1 or 
#- the machine
#- value of the double-precision rounded float closest to 1.1.  Is that
#- correct?
#- 
#- If so, then the real exceptional case is *exact* conversion, 
#- not float
#- conversion.  Decimal(1.1, precision=4) is unambiguous.  Therefore:

Here I think you're confused. *Precision* is a parameter of the context, and
determines in how many digits you will get the coefficient after an
operation.

The issue when constructing from float, is how many *positions* after the
decimal point will be relevant when creating from a float.

This list agreed to have a special method to creating from float, to avoid
confusing users about the real behaviour there. And in that method only you
permit to specify the positions. When constructing from other data types,
positions does not have any meaning as those other data types are always
exact.


#- The extra method should be specifically for the exceptional case,
#- and named appropriately.
#- 
#-     Decimal.exact_float(value)
#- 
#- It would serve just one purpose: allow exact conversion from floats
#- when it's really desired.
#- 
#- This would both (a) separate the functionality between Decimal()
#- and Decimal.exact_float() in a more logical way; and (b) give
#- the extra method a more meaningful, useful name -- a name that
#- really explains why the extra method exists.

I think that *maybe* a better name is needed, but just that.


#- > Also, remember that the context doesn't affect the 
#- creation when you create
#- > directly from string (only if you use from_string).  Your 
#- example means to
#- > change the behaviour of the creation when creating 
#- directly from string? And
#- > when create directly from int or long?
#- 
#- That's what I'm suggesting: that you can supply a context object,
#- or ask for the default context, when constructing any Decimal value.

Do you want to supply a context that *will affect* the creation process, or
just supply one that the Decimal will keep to use in other operations?

Remember that you have one context per thread.  All the decimal instances in
a thread share the same context.  This was specified by Aahz, the list
agreed, and get into the PEP. Do you want to change it?

.	Facundo

From guido at python.org  Thu Apr 15 09:55:56 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr 15 09:56:02 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: Your message of "Thu, 15 Apr 2004 13:45:09 +1200."
	<200404150145.i3F1j9tl019325@cosc353.cosc.canterbury.ac.nz> 
References: <200404150145.i3F1j9tl019325@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200404151355.i3FDtuT26138@guido.python.org>

> But who cares about the gory details of the internals?
> The most important thing about a repr() is to unambiguously
> indicate the object's type; the next is to show something
> about its value, as far as reasonably possible. What's
> actually inside the object doesn't matter much.

I didn't see the start of this, but the guidelines for repr() are and
have always been this: if at all possible it should be an expression
that, when fed to eval(), (assuming the right imports are in scope),
returns an object with the same type and value.  In most cases, this
means it should resemble a call to the constructor with suitable
arguments.  If that isn't possible, a form surrounded with <...>
should be used, and it should at least show the object's type and some
stuff that can be used to distinguish different instances (either the
address or some representative property of the instance).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 15 10:27:17 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr 15 10:27:22 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: Your message of "Thu, 15 Apr 2004 15:36:13 +0200."
	<20040415133613.GA4735@mike.de> 
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net> 
	<20040415133613.GA4735@mike.de> 
Message-ID: <200404151427.i3FERHo26387@guido.python.org>

> >     http://starship.python.net/crew/mwh/hacks/speeding-python.html

I recently met Mario Wolczko who worked on the Self language, and
asked him if he had one sentence of advice for Python.  His response
was: Just-in-time compilation.

> About psyco: I think it's wonderful. But, you are right, nobody is
> using it.  Why? Simple: It's not 'on' by default.

I don't think that's the real reason.  Last week Jimmy Retzlaff gave a
talk at the Bay Piggies meeting on optimization, using a large
commercial application he's written as an example.  He mentioned that
he had tried to use Psyco, and that it had been turned on in some
distributions of the code, but that it was currently turned off again.

I don't recall the reason he stated for that, but I suspect that it
was mostly Psyco's memory bloat, plus the fact that his code was "fast
enough" and he was able to code things in C if needed.  (He also
showed a nice anecdote about Psyco being faster than his hand-coded
C in one case, but that didn't last long when he realized the C code
was using doubles and the Python code was using ints. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Thu Apr 15 10:29:41 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 15 10:36:58 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <200404151355.i3FDtuT26138@guido.python.org>
References: <200404150145.i3F1j9tl019325@cosc353.cosc.canterbury.ac.nz>
	<200404151355.i3FDtuT26138@guido.python.org>
Message-ID: <20040415142941.GA22303@panix.com>

On Thu, Apr 15, 2004, Guido van Rossum wrote:
>Greg Ewing:
>>
>> But who cares about the gory details of the internals?  The most
>> important thing about a repr() is to unambiguously indicate the
>> object's type; the next is to show something about its value, as far
>> as reasonably possible. What's actually inside the object doesn't
>> matter much.
>
> I didn't see the start of this, but the guidelines for repr() are and
> have always been this: if at all possible it should be an expression
> that, when fed to eval(), (assuming the right imports are in scope),
> returns an object with the same type and value.  In most cases, this
> means it should resemble a call to the constructor with suitable
> arguments.  If that isn't possible, a form surrounded with <...>
> should be used, and it should at least show the object's type and some
> stuff that can be used to distinguish different instances (either the
> address or some representative property of the instance).

Right.  Question is, to what extent does "human readable" factor in
after these criteria?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From jeremy at alum.mit.edu  Thu Apr 15 10:38:26 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Apr 15 10:38:46 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <200404151427.i3FERHo26387@guido.python.org>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de> <2mekqp1oxq.fsf@starship.python.net>
	<20040415133613.GA4735@mike.de>
	<200404151427.i3FERHo26387@guido.python.org>
Message-ID: <1082039905.2610.128.camel@localhost.localdomain>

On Thu, 2004-04-15 at 10:27, Guido van Rossum wrote:
> > >     http://starship.python.net/crew/mwh/hacks/speeding-python.html
> 
> I recently met Mario Wolczko who worked on the Self language, and
> asked him if he had one sentence of advice for Python.  His response
> was: Just-in-time compilation.

It seems like an obvious, but labor intensive solution to me.  I didn't
get a chance to pursue discussion much with Jim Hugunin or Mike Salib,
but it would be interesting to know why they focus on static analysis;
Self started out with static analysis but got better performance with
runtime type feedback
(http://www.cs.ucsb.edu/labs/oocsb/papers/toplas96.shtml).

Another question that I'd love to hear the answer to: What's the
difference between Pysco and something like this Self implementation or
the HotSpot Java implementation?  I've never read a clear explanation of
what Pysco does or how it compares to other systems that do dynamic
compilation.

Jeremy



From guido at python.org  Thu Apr 15 10:41:37 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr 15 10:41:44 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: Your message of "Thu, 15 Apr 2004 10:29:41 EDT."
	<20040415142941.GA22303@panix.com> 
References: <200404150145.i3F1j9tl019325@cosc353.cosc.canterbury.ac.nz>
	<200404151355.i3FDtuT26138@guido.python.org> 
	<20040415142941.GA22303@panix.com> 
Message-ID: <200404151441.i3FEfb226502@guido.python.org>

> Right.  Question is, to what extent does "human readable" factor in
> after these criteria?

If the choice is between a string and a tuple, I'd vote for the string
because it is (a) shorter and (b) more readable.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 15 10:42:31 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr 15 10:42:42 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: Your message of "Thu, 15 Apr 2004 10:38:26 EDT."
	<1082039905.2610.128.camel@localhost.localdomain> 
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net>
	<20040415133613.GA4735@mike.de>
	<200404151427.i3FERHo26387@guido.python.org> 
	<1082039905.2610.128.camel@localhost.localdomain> 
Message-ID: <200404151442.i3FEgVR26515@guido.python.org>

> It seems like an obvious, but labor intensive solution to me.

Well, all *other* proposals seem just as labor intensive...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Thu Apr 15 10:34:27 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 15 10:43:33 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383885@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383885@escpl.tcp.com.ar>
Message-ID: <20040415143427.GB22303@panix.com>

On Thu, Apr 15, 2004, Batista, Facundo wrote:
>
> Remember that you have one context per thread.  All the decimal
> instances in a thread share the same context.  This was specified by
> Aahz, the list agreed, and get into the PEP. Do you want to change it?

Correction for precision: all Decimal operations in a thread use the
same context until the context is changed.  Threads inherit context from
their starting thread (I think).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From FBatista at uniFON.com.ar  Thu Apr 15 10:53:15 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 15 10:55:30 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D03383889@escpl.tcp.com.ar>

[Aahz]

#- Correction for precision: all Decimal operations in a thread use the
#- same context until the context is changed.  Threads inherit 
#- context from
#- their starting thread (I think).

I'm lost. Help me with an example (that later I'll shamelessly stole to use
in a test case):

   d1 = Decimal('123')
   d2 = Decimal('456')

So, long, they have the same context, right?

   getcontext().prec = 46

The precision should change for both d1 and d2, right?

   d3 = Decimal('789')

d3 also has the precision in 46, right?

Thank you!

.	Facundo

From edloper at gradient.cis.upenn.edu  Thu Apr 15 11:17:56 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Thu Apr 15 11:16:17 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <E1BDvxx-0007q7-0l@mail.python.org>
References: <E1BDvxx-0007q7-0l@mail.python.org>
Message-ID: <407EA7A4.6070709@gradient.cis.upenn.edu>

> #- > DEFAULT_MAX_EXPONENT = 999999999
> #- > DEFAULT_MIN_EXPONENT = -999999999
> #- > ABSOLUTE_MAX_EXP = 999999999
> #- > ABSOLUTE_MIN_EXP = -999999999

I can think of at least one real use for exponents outside this range: 
probabilities.  E.g., if I'm using probabilistic models to estimate 
P(author|text), I end up multiplying together a large number of very low 
probabilities, and the total probability could easily get this small. 
Of course, people usually use tricks to avoid underflow: taking logs, 
and if necessary performing periodic re-normalization (which means that 
you don't have probabilities anymore, just some sort of likelihood 
measures).  But it might be nice to not have to do those tricks.

I'm +1 for following the lead set by Long, and not adding unnecessary 
limits.

-Edward


From FBatista at uniFON.com.ar  Thu Apr 15 11:18:58 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 15 11:21:15 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338388B@escpl.tcp.com.ar>

[Edward Loper]

#- > #- > DEFAULT_MAX_EXPONENT = 999999999
#- > #- > DEFAULT_MIN_EXPONENT = -999999999
#- > #- > ABSOLUTE_MAX_EXP = 999999999
#- > #- > ABSOLUTE_MIN_EXP = -999999999
#- 
#- I can think of at least one real use for exponents outside 
#- this range: 
#- probabilities.  E.g., if I'm using probabilistic models to estimate 

So you need a number smaller than 10**(-999999999)?

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040415/c50b02f5/attachment.html
From tim.one at comcast.net  Thu Apr 15 11:32:15 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Apr 15 11:32:25 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383889@escpl.tcp.com.ar>
Message-ID: <E1BE8qk-0008DG-PQ@mail.python.org>

[Batista, Facundo]
> I'm lost. Help me with an example (that later I'll shamelessly stole to
> use in a test case):
>
>    d1 = Decimal('123')
>    d2 = Decimal('456')
>
> So, long, they have the same context, right?

Context is associated with operations, not with instances, so d1 and d2
don't "have" any context.  It's unclear to me what use (if any) the
Decimal() constructor makes of context right now.  To the extent that the
Decimal() constructor makes of context, to exactly that same extent the
Decimal constructor is "an operation", so while context may affect
Decimal(), I don't know exactly how it does now.  In any case, context has
no effect on the instances d1 and d2 after they're constructed (although
context both affects and is affected by any operations d1 and d2 may later
take part in).

>    getcontext().prec = 46
>
> The precision should change for both d1 and d2, right?

Sorry, it simply doesn't make sense to talk about context wrt instances,
only wrt operations.  If, e.g., you do an operation now, like

    d4 = d1 + d2

then the result of the operation "d1+d2" has precision 46.

>    d3 = Decimal('789')
>
> d3 also has the precision in 46, right?

That depends on how Decimal() is implemented, and I don't know the answer to
that.



From FBatista at uniFON.com.ar  Thu Apr 15 11:38:54 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 15 11:41:04 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338388F@escpl.tcp.com.ar>

[Tim Peters]

#- Context is associated with operations, not with instances, 
#- so d1 and d2
#- don't "have" any context.  It's unclear to me what use (if any) the

You're right. My confusion. Sorry.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040415/57d38a90/attachment.html
From mikepy-0404 at mike.de  Thu Apr 15 11:52:30 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Thu Apr 15 11:56:29 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <1082039905.2610.128.camel@localhost.localdomain>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net>
	<20040415133613.GA4735@mike.de>
	<200404151427.i3FERHo26387@guido.python.org>
	<1082039905.2610.128.camel@localhost.localdomain>
Message-ID: <20040415155230.GA692@mike.de>

Hi,

Jeremy Hylton wrote:
> Another question that I'd love to hear the answer to: What's the
> difference between Pysco and something like this Self implementation or
> the HotSpot Java implementation?  I've never read a clear explanation of
> what Pysco does or how it compares to other systems that do dynamic
> compilation.

Here is a link for Java HotSpot:

http://java.sun.com/products/hotspot/docs/whitepaper/Java_Hotspot_v1.4.1/Java_HSpot_WP_v1.4.1_1002_4.html

It seems that many advantages gained by static typing are lost since the
Java codebase encourages strong polymorphism (in contrast to C++).

What can we learn from this for Python: static typing is probably not
all that helpful for a JIT compiler (in contrast to popular belief).


About the CLR (.NET and C# target) JIT:

http://www.artima.com/intv/choices.html

What I found most interesting: their bytecode is polymorphic because they
never targetted an interpreter and the JIT must do type inference anyway.
Well, Python bytecode doesn't look that bad now ...


I would be very interested to hear what Armin thinks about static vs.
dynamic type inferencing and about semi-static (CLR) vs. hotspot (Java)
compilation.

Bye,
     Mike

From mikepy-0404 at mike.de  Thu Apr 15 11:56:20 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Thu Apr 15 11:56:35 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <F4EAA8B9-8EE2-11D8-BBB8-003065E15634@amber.org>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net>
	<20040415133613.GA4735@mike.de>
	<F4EAA8B9-8EE2-11D8-BBB8-003065E15634@amber.org>
Message-ID: <20040415155620.GB692@mike.de>

Hi,

Christopher Petrilli wrote:
> Please do make sure that "better code on x86" doesn't equally worse 
> code on other platforms.  It may be the "majority" platform, but it is 
> not the only platform.

We are aware of that. In fact I gave up on trying to trick GCC into
producing good code by rearranging the C source. A single architecture
dependent #ifdef plus some inline assembly seems much cleaner. And it
can be removed quickly if it ever turns out to be a bad idea for newer x86
processors.

Bye,
     Mike

From skip at pobox.com  Thu Apr 15 11:59:25 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Apr 15 11:59:47 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <BAY0-SMTP05tLamTtYD00007b38@BAY0-SMTP05.bay0.hotmail.com>
References: <001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<BAY0-SMTP05tLamTtYD00007b38@BAY0-SMTP05.bay0.hotmail.com>
Message-ID: <16510.45405.16024.437304@montanaro.dyndns.org>


    Tim> What makes the current stepping of some flavor of P4 overwhelmingly
    Tim> important <0.3 wink>?

Unrelated to this thread, what does "stepping" mean?  A quick Google search
for "Pentium stepping glossary" yielded this page

  http://processorfinder.intel.com/scripts/help3.asp

but doesn't explain why Intel uses "stepping" instead of "version" or
"revision". 

Skip

From edloper at gradient.cis.upenn.edu  Thu Apr 15 12:49:51 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Thu Apr 15 12:48:31 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <E1BE9Ld-0008Q2-3Z@mail.python.org>
References: <E1BE9Ld-0008Q2-3Z@mail.python.org>
Message-ID: <407EBD2F.1080402@gradient.cis.upenn.edu>

> #- > #- > DEFAULT_MAX_EXPONENT = 999999999
> #- > #- > DEFAULT_MIN_EXPONENT = -999999999
> #- > #- > ABSOLUTE_MAX_EXP = 999999999
> #- > #- > ABSOLUTE_MIN_EXP = -999999999
> #-
> #- I can think of at least one real use for exponents outside
> #- this range:
> #- probabilities.  E.g., if I'm using probabilistic models to estimate
> 
> So you need a number smaller than 10**(-999999999)?

Not often.  And of course, "need" might be a bit strong, since I can use 
other tricks (logs & renormalization) to avoid underflow.  But yes, I 
can imagine cases where I would get underflow using naive algorithms. 
E.g., if I'm looking at probabilities for 1 billion words of text 
(computational linguists are starting to use corpora of this size), and 
for some reason I want to do language modelling on it, I could end up 
with numbers on the order of (1e-20)**(1,000,000,000).  I.e., 
10**(-20,000,000,000).

On the other hand, I can't think of a use, off hand, of calculating the 
probability of a billion word string of text.  But I just wanted to 
point out that there may be cases where extremely large/small numbers 
can be useful; and as computers get more powerful and space gets 
cheaper, the numbers people use will tend to grow in size.

I understand the argument that this *may* help catch errors, but I just 
don't think it *will* catch many errors.  How many people would have 
errors that would have been caught if long had an absolute min/max?

-Edward


From theller at python.net  Thu Apr 15 13:11:02 2004
From: theller at python.net (Thomas Heller)
Date: Thu Apr 15 13:11:46 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
In-Reply-To: <407D9AA4.10108@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed,
	14 Apr 2004 22:10:12 +0200")
References: <zn9esbj7.fsf@python.net> <407D91BB.2030603@v.loewis.de>
	<u0zmqrkd.fsf@python.net> <407D9AA4.10108@v.loewis.de>
Message-ID: <vfk1cgk9.fsf@python.net>

Most probably I was too short in describing what I want, and what this
script is and does, I also got a personal email asking for more, so I'll
try again:

Inno Setup is a free tool of superb quality to create executable windows
installers.  See http://www.jrsoftware.org/isinfo.php

The script I mention allows to create a windows installer containing
*only* the debugging binaries and libraries which are created when you
compile Python from the sources in debug mode.

When run, it copies these files into an existing, standard Python
installation, the location is read from the registry.  The standard
files are not overwritten, since the debug binaries on windows have an
additional _d appended to the filename (python_d.exe instead of
python.exe, for example).

This allows easy building of debug versions of your own extensions for
debugging.

The existing python.iss inno setup script is not used in the official
Python build, although without doubt it is useful to other people.

I want to run the official Python binary distribution (I must, because I
build them ;-), and also want to have the debug binaries in the same
location.  This debug build is also installed by my coworkers, who don't
want to build debug Python themselves (at least when the standard
extension modules requiring third party code, like zlib, have to be
included).

That is the reason that I don't want to include this into the existing
python.iss script.

Thomas


From mwh at python.net  Thu Apr 15 13:26:44 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Apr 15 13:27:25 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <1082039905.2610.128.camel@localhost.localdomain> (Jeremy
	Hylton's message of "Thu, 15 Apr 2004 10:38:26 -0400")
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de> <2mekqp1oxq.fsf@starship.python.net>
	<20040415133613.GA4735@mike.de>
	<200404151427.i3FERHo26387@guido.python.org>
	<1082039905.2610.128.camel@localhost.localdomain>
Message-ID: <2m65c117aj.fsf@starship.python.net>

Jeremy Hylton <jeremy@alum.mit.edu> writes:

> Another question that I'd love to hear the answer to: What's the
> difference between Pysco and something like this Self implementation
> or the HotSpot Java implementation?  I've never read a clear
> explanation of what Pysco does or how it compares to other systems
> that do dynamic compilation.

I don't know the answer to this... I suspect part of it might have to
do with 'abstract interpretation'.  I'll see Armin tomorrow (at
PythonUK) and beat him up until he answers this question himself :-)

Cheers,
mwh

-- 
  please realize that the Common  Lisp community is more than 40
  years old.  collectively, the community has already been where
  every clueless newbie  will be going for the next three years.
  so relax, please.                     -- Erik Naggum, comp.lang.lisp

From bsder at allcaps.org  Thu Apr 15 14:08:36 2004
From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Thu Apr 15 14:04:40 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <1082035943.407e8ee7975ef@mcherm.com>
References: <1082035943.407e8ee7975ef@mcherm.com>
Message-ID: <20040415093930.F5694@mail.allcaps.org>

On Thu, 15 Apr 2004, Michael Chermside wrote:

> I strongly object. I have one particular use case for decimal, and I
> consider it to be a very important one.... to store and manipulate
> monetary amounts. Putting a limit as small as that used for floats
> would significantly harm my use case.

I presume that you are objecting to only having a default limit of 15
digits of precision?  I can certainly understand your position.

However, why does having a soft default limit set at this level cause you
an issue?  If you need more, you can move it up.  You have to override the
default precision (currently 8)  anyway.

If you want a default limit much higher than this, then there needs to be
a way for people to determine when they leave the realm of "fast, but
narrower range" and enter the realm of "slow, but much greater range".

What I am trying to establish is a "sensible default" limit which allows
most people to use fast operations, does not limit those people who want
greater range (at the expense of slower operation), does not impose undue
constraints on an optimized version of the module (either software or
later hardware), and provides useful feedback for those who suddenly find
themselves moving from one group to another (either intentionally or
unintentionally).

Part of creating Decimal is to provide a useful module for currency and
other applications which require accurate Decimal arithmetic.

But part of creating Decimal is also to be newbie friendly.  Letting
people know when they pass outside of "sensible" regimes is friendly.

> get overflow errors instead of out-of-memory errors (and possibly
> some vague future advantage to implementers of a "complete
> numerical package" including things like sin() and cos()).

<return sarcasm>
And here I was thinking that someone manipulating financial amounts might
actually need speed on a silly operation known as "division".  My mistake.
</return sarcasm>

-a

From jim.jewett at eds.com  Thu Apr 15 14:08:34 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu Apr 15 14:09:13 2004
Subject: [Python-Dev] Optimization targets - refcount
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>

Mike Pall:

> About GC: yes, refcounting is the silent killer.
>... Py_DECREF is awful ... > 3500 locations

Is it always needed?

What if a few common (constant, singleton) objects 
(such as None, -1, 0, 1) were declared immortal at 
compile-time?  They would be created at initial 
load in a special untracked pool, and their 
tp_dealloc would do nothing.

The slot for tracking references would still be
there, but could be ignored -- even if it went
negative.

Since the reference count no longer has to be
correct (for these objects), the reference 
counting macros could optimize to nothing when 
they know at compile time that they'll have one 
of these constant objects.  

-jJ

From python at rcn.com  Thu Apr 15 14:20:34 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Apr 15 14:22:09 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <16510.45405.16024.437304@montanaro.dyndns.org>
Message-ID: <009001c42316$6a74aae0$e841fea9@oemcomputer>

>     Tim> What makes the current stepping of some flavor of P4
> overwhelmingly
>     Tim> important <0.3 wink>?
> 
> Unrelated to this thread, what does "stepping" mean?  A quick Google
> search
> for "Pentium stepping glossary" yielded this page
> 
>   http://processorfinder.intel.com/scripts/help3.asp
> 
> but doesn't explain why Intel uses "stepping" instead of "version" or
> "revision".

[Channeling Tim]

In this context, "stepping" means all of the idiosyncratic
implementation details of processor workflow.

The word "stepping" is natural after staring at Gantt charts showing
timings, data dependencies, speculative execution, and parallel
operations.

Tim is advising me to against focusing on one processor at one point in
history because the "stepping" is guaranteed to change over time.


Raymond Hettinger 


From theller at python.net  Thu Apr 15 14:31:16 2004
From: theller at python.net (Thomas Heller)
Date: Thu Apr 15 14:31:22 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <16510.45405.16024.437304@montanaro.dyndns.org> (Skip
	Montanaro's message of "Thu, 15 Apr 2004 10:59:25 -0500")
References: <001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<BAY0-SMTP05tLamTtYD00007b38@BAY0-SMTP05.bay0.hotmail.com>
	<16510.45405.16024.437304@montanaro.dyndns.org>
Message-ID: <k70hccuj.fsf@python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Tim> What makes the current stepping of some flavor of P4 overwhelmingly
>     Tim> important <0.3 wink>?
>
> Unrelated to this thread, what does "stepping" mean?  A quick Google search
> for "Pentium stepping glossary" yielded this page
>
>   http://processorfinder.intel.com/scripts/help3.asp
>
> but doesn't explain why Intel uses "stepping" instead of "version" or
> "revision". 

Here is a possible answer:

  A stepping is the processor hardware equivalent of a new version. In
  software, we refer to minor version changes as 1.0, 1.1, 1.2, and so
  on, while in processors we call these minor revisions steppings.

Found in
<http://www.informit.com/articles/article.asp?p=130978&seqNum=22>

Thomas


From martin at v.loewis.de  Thu Apr 15 15:07:12 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Apr 15 15:07:30 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <1082039905.2610.128.camel@localhost.localdomain>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>	<001901c4223a$5bab0a40$e841fea9@oemcomputer>	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net>	<20040415133613.GA4735@mike.de>	<200404151427.i3FERHo26387@guido.python.org>
	<1082039905.2610.128.camel@localhost.localdomain>
Message-ID: <407EDD60.8000101@v.loewis.de>

Jeremy Hylton wrote:
> Another question that I'd love to hear the answer to: What's the
> difference between Pysco and something like this Self implementation or
> the HotSpot Java implementation?  

Psyco is specializing, and that is the main difference compared
to Java Hotspot. If you have a program

def f(a, b):
   if a>b:
     return a
   else:
     return a-b

f(1,2)
f(1.0, 2.0)
f([1,2],[3,4])

then psyco generates machine code for *three* functions

int f_int(int a, int b){ // uses process int arithmethic throughout
   if(a>b)return a;
   else return a-b;
}

double f_double(double a, double b){ // uses FPU ops throughout
   if(a>b)return a;
   else return a-b;
}

list f_list(list a, list b){ // might invoke C functions
   if(list_gt(a, b))return a;
   else raise TypeError("unexpected operands");
}

(it actually generates different specializations)

In Hotspot, the type of f would already be defined in the Java
source code, and Hotspot generates native machine instructions
for it.

The changes over standard JIT appear to be inlining; it also
appears to do inlining of virtual functions, combined with
a type check to detect cases where the a different functions
should have been called compared to the last time the virtual
call was made.

Regards,
Martin


From op73418 at mail.telepac.pt  Thu Apr 15 15:22:00 2004
From: op73418 at mail.telepac.pt (Goncalo Rodrigues)
Date: Thu Apr 15 15:31:14 2004
Subject: [Python-Dev] Use case for class decorators
Message-ID: <loom.20040415T212046-954@post.gmane.org>

Here is a use case for class decorators.

I find myself writing a lot of proxies and adapters. Usually, they are just a 
subclass of something as simple as

class Proxy(object):
    """The Proxy base class."""

    def __init__(self, obj):
        """The initializer."""
        super(Proxy, self).__init__(obj)
        #Private attributes.
        self.__obj = obj

    #Read-only properties.
    __obj__ = property(lambda self: self.__obj,
                       doc = "The underlying object of the Proxy.")

    def __repr__(self):
        return "%s(%r)" % (self.__class__.__name__,
                           self.__obj__)

    def __getattr__(self, attrib):
        return getattr(self.__obj__, attrib)

This is all fine and dandy, except that no magic methods are trapped. Usually, 
I just inject them directly after the class statement via:

#Auxiliary getter function.
def getter(attrib):
    return lambda self, *args, **kwargs: getattr(self.__obj__, attrib)(*args, 
**kwargs)


def inject(cls, magicnames):
    """Inject magic methods directly in a class.

    Warning(s):
    - Only magic names are injected.
    - Does not overwrite already present magic names.
    """
    for magicname in magicnames:
        if magicname.startswith("__") and magicname.endswith("__"):
            if not hasattr(cls, magicname):
                setattr(cls, magicname, getter(magicname))

The function inject is a use case for a class decorator. If it's a good use 
case that's something I'll leave you to argue at will (<runs away>). I think 
it's fairly obvious that using a metaclass to do this simple operation is a 
clear case of the wrong tool for the job.

With my best regards,
G. Rodrigues

p.s: 
<comment on syntax>
The syntax for decorators:

[dec1, dec2, ...]
def foo(arg1, arg2, ...):
    pass

is probably the worst of all syntaxes (with appologies for my bluntness). It 
just doesn't parse well. What I see in the above proposed syntax is just a list 
literal and a definition statement. There's no indication that the two are 
somehow linked (I can't express this any better, English is not my native 
lingo).

The best, IMHO is

def foo(arg1, arg2, ...) [dec1, dec2, ...]:
    pass

<\ comment on syntax>


From tcdelaney at optusnet.com.au  Thu Apr 15 18:07:19 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Thu Apr 15 18:07:29 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
References: <zn9esbj7.fsf@python.net>
	<407D91BB.2030603@v.loewis.de><u0zmqrkd.fsf@python.net>
	<407D9AA4.10108@v.loewis.de> <vfk1cgk9.fsf@python.net>
Message-ID: <006301c42336$05459970$a100a8c0@ryoko>

Thomas Heller wrote:

> Most probably I was too short in describing what I want, and what this
> script is and does, I also got a personal email asking for more, so
> I'll try again:
>
> Inno Setup is a free tool of superb quality to create executable
> windows installers.  See http://www.jrsoftware.org/isinfo.php

I've found NSIS to be a much better choice for Windows installers. Produces
very small installers (adds about 30-40K overhead, can use LZMA), has
multiple language support, and are simple to write.

http://nsis.sourceforge.net/home/

I keep intending to write an installer for Python in it - maybe next week
(I'm on leave at the moment, but I've got the niece (6) and nephew (5)
visiting all this week). I've been gradually moving all our installers at
work away from WISE and into NSIS.

Tim Delaney




From barry at barrys-emacs.org  Thu Apr 15 19:19:19 2004
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu Apr 15 19:31:29 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
In-Reply-To: <006301c42336$05459970$a100a8c0@ryoko>
References: <zn9esbj7.fsf@python.net> <407D91BB.2030603@v.loewis.de>
	<u0zmqrkd.fsf@python.net> <407D9AA4.10108@v.loewis.de>
	<vfk1cgk9.fsf@python.net> <006301c42336$05459970$a100a8c0@ryoko>
Message-ID: <6.0.3.0.2.20040416000249.03208e50@torment.chelsea.private>


At 15-04-2004 23:07, Tim Delaney wrote:
>Thomas Heller wrote:
>
> > Most probably I was too short in describing what I want, and what this
> > script is and does, I also got a personal email asking for more, so
> > I'll try again:
> >
> > Inno Setup is a free tool of superb quality to create executable
> > windows installers.  See http://www.jrsoftware.org/isinfo.php
>
>I've found NSIS to be a much better choice for Windows installers. Produces
>very small installers (adds about 30-40K overhead, can use LZMA), has
>multiple language support, and are simple to write.

When I looked a while ago NSIS was overly complex compared to INNO
given the same task. I use INNO to install the pysvn extension and it
was easy to do. See 
http://svn.collab.net/repos/pysvn/trunk/pysvn/Extension/Kit/Win32/pysvn.iss

Barry



From greg at cosc.canterbury.ac.nz  Thu Apr 15 20:09:38 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Apr 15 20:09:56 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <Pine.LNX.4.58.0404150428060.18028@server1.LFW.org>
Message-ID: <200404160009.i3G09cAX020939@cosc353.cosc.canterbury.ac.nz>

Ka-Ping Yee <python-dev@zesty.ca>:

> The extra method should be specifically for the exceptional case,
> and named appropriately.
> 
>     Decimal.exact_float(value)

That's a confusing name, since it seems to be promising that the
conversion will be exact. While this might technically be true for
certain values of float, there's no way the implementation can know
whether the result is exactly what the user had in mind for the
meaning of that float.

I think the name from_float is fine, since it says "I know
constructing a Decimal from a float isn't necessarily exact, but I
want to do it anyway, and I'm prepared to accept the result,
whatever it is."

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Thu Apr 15 21:22:57 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Apr 15 21:23:09 2004
Subject: [Python-Dev] Decimal conversion to string
In-Reply-To: <20040415142941.GA22303@panix.com>
Message-ID: <200404160122.i3G1MvjO021056@cosc353.cosc.canterbury.ac.nz>

Aahz <aahz@pythoncraft.com>:

> Right.  Question is, to what extent does "human readable" factor in
> after these criteria?

In the case of Decimal, I think it's fairly clear that

  Decimal("123.45")

is the winner overall, since both the type and value are
unambiguous and easily readable, and it's eval-able
to boot.

On the other hand, I see absolutely zero utility in
displaying the tuple-form of the internals.

>From the point of view of using the interactive interpreter
as a desk calculator, it might be slightly more desirable
to see

  Decimal(123.45)

but that wouldn't be eval-able. For such use it might be
better to have a special calculator-mode interpreter that
took all numeric literals as Decimal and displayed them
unadorned.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From perky at i18n.org  Thu Apr 15 22:56:49 2004
From: perky at i18n.org (Hye-Shik Chang)
Date: Thu Apr 15 22:56:54 2004
Subject: [Python-Dev] Inno Setup script for Python debug runtime
In-Reply-To: <006301c42336$05459970$a100a8c0@ryoko>
References: <zn9esbj7.fsf@python.net> <407D9AA4.10108@v.loewis.de>
	<vfk1cgk9.fsf@python.net> <006301c42336$05459970$a100a8c0@ryoko>
Message-ID: <20040416025649.GA81432@i18n.org>

On Fri, Apr 16, 2004 at 08:07:19AM +1000, Tim Delaney wrote:
> Thomas Heller wrote:
> 
> > Most probably I was too short in describing what I want, and what this
> > script is and does, I also got a personal email asking for more, so
> > I'll try again:
> >
> > Inno Setup is a free tool of superb quality to create executable
> > windows installers.  See http://www.jrsoftware.org/isinfo.php
> 
> I've found NSIS to be a much better choice for Windows installers. Produces
> very small installers (adds about 30-40K overhead, can use LZMA), has
> multiple language support, and are simple to write.
> 
> http://nsis.sourceforge.net/home/
> 
> I keep intending to write an installer for Python in it - maybe next week
> (I'm on leave at the moment, but I've got the niece (6) and nephew (5)
> visiting all this week). I've been gradually moving all our installers at
> work away from WISE and into NSIS.
> 

CJKPython (http://cjkpython.i18n.org/#CJKPython) is based on NSIS.
And .nsi source is available at
http://cvs.berlios.de/cgi-bin/viewcvs.cgi/cjkpython/CJKPython/cjkpython.nsi?rev=1.4&content-type=text/vnd.viewcvs-markup

As you may find, lack of log-based uninstall feature makes .nsi
file too large.


Hye-Shik

From jeremy at alum.mit.edu  Thu Apr 15 23:25:56 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Apr 15 23:26:19 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <407EDD60.8000101@v.loewis.de>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>
	<20040414214945.GA3443@mike.de> <2mekqp1oxq.fsf@starship.python.net>
	<20040415133613.GA4735@mike.de>
	<200404151427.i3FERHo26387@guido.python.org>
	<1082039905.2610.128.camel@localhost.localdomain>
	<407EDD60.8000101@v.loewis.de>
Message-ID: <1082085955.4953.15.camel@localhost.localdomain>

On Thu, 2004-04-15 at 15:07, "Martin v. L?wis" wrote:
> Jeremy Hylton wrote:
> > Another question that I'd love to hear the answer to: What's the
> > difference between Pysco and something like this Self implementation or
> > the HotSpot Java implementation?  
> 
> Psyco is specializing, and that is the main difference compared
> to Java Hotspot. If you have a program

That's a helpful clarification, thanks.

> In Hotspot, the type of f would already be defined in the Java
> source code, and Hotspot generates native machine instructions
> for it.
> 
> The changes over standard JIT appear to be inlining; it also
> appears to do inlining of virtual functions, combined with
> a type check to detect cases where the a different functions
> should have been called compared to the last time the virtual
> call was made.

I'm not sure what you mean by the changes over standard JIT.  Do you
mean the difference between Hotspot and "standard JIT"?  Your
descriptions sounds a bit like the Deutsch and Schiffman inline method
cache, which doesn't inline the method body but does inline the method
lookup.  Or does Hotspot actually inline methods?

Jeremy



From tim_one at email.msn.com  Fri Apr 16 00:17:24 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Fri Apr 16 00:17:51 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <407EA7A4.6070709@gradient.cis.upenn.edu>
Message-ID: <BAY0-SMTP08LscJJGtp0000822b@BAY0-SMTP08.adinternal.hotmail.com>

>> #- > DEFAULT_MAX_EXPONENT = 999999999
>> #- > DEFAULT_MIN_EXPONENT = -999999999
>> #- > ABSOLUTE_MAX_EXP = 999999999
>> #- > ABSOLUTE_MIN_EXP = -999999999

[Edward Loper]
> I can think of at least one real use for exponents outside this range:
> probabilities.  E.g., if I'm using probabilistic models to estimate
> P(author|text), I end up multiplying together a large number of very low
> probabilities, and the total probability could easily get this small.

No, it couldn't -- and if you worked hard to contrive an example, a
probability in the end smaller than the probability that the universe will
spontaneously decide to run time backwards for a century is accurately
enough represented by 0 (which your example will underflow to).



From tim_one at email.msn.com  Fri Apr 16 00:17:24 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Fri Apr 16 00:17:58 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383884@escpl.tcp.com.ar>
Message-ID: <BAY0-SMTP08BS3hgRCw0000822c@BAY0-SMTP08.adinternal.hotmail.com>

[Facundo Batista, on whether to limit exponent magnitude]
> The feeling previous to this mail was that an artificial limit was silly
> if you don't gain nothing.
>
> Here are three valuable reasons for keep the limits.
>
> So, if nobody opposes strongly, I'll put these reasons in the PEP and
> keep the limit.

I'm only +0 on keeping the limits, btw -- I think they do more good than
harm, but it's not going to kill us either way.

...

> Until now, Decimal does not uses context in creation time (with creation
> I mean when you create an object instantiating the class).
> 
> This allows to not lose any information at creation time, and only use
> the context in operations.
>
> Do you want to change this behaviour?

No, I want to spell the spec's mandatory from-string *operation* (which does
use and affect context, in the specified ways) via an optional bool argument
to the Decimal constructor.  This isn't at all useless, although if you're
only thinking about a money type it may seem that way.  In general
number-crunching, literals may be given to high precision, but that
precision isn't free and *usually* isn't needed:  the operations in the spec
don't round back inputs to current context precision before computing, they
only round outputs after, and digits feeding into computation are expensive.
So, in general, a scientific user will usually *not* want all the digits in

    pi = Decimal("3.1415926535897932384626433832795", use_context=True)

to be used.  Instead they'll want to set the working precision once in
context, and have their literals automatically get rounded back to that
precision.  This pattern will be familiar to Fortran77 users, where Fortran
REAL ("single precision") literals are usually given to DOUBLE precision, so
that changing a single IMPLICIT (static typing) statement can make the same
code usable as-is under both REAL and DOUBLE precision.  Decimal supports an
unbounded number of precisions, and so the need for pleasant precision
control is that much more acute (and the spec's from-string operation is one
aid toward that end).

...

[on what repr() should do]
> The only issue *I* see here, is that you don't have a clear
> representation of the internals of the object,

But nobody cares except implementers.  I'm perfectly happy to introduce an
.as_tuple() method for them.  

More generally, Python should (eventually, not necessarily at the start)
implement an extension to context, specifying the desired str() and repr()
behavior.  One size doesn't fit all, and context is the natural place to
record individual preferences.

Against "one size doesn't fit all", everyone can at least *understand* the
to-sci-string format, while only a handful will ever be able to read the
tuples with ease.

> but yes, you have an exact representation of that number. Let me
> show an example:

There are more in the spec (for those interested).

> >>> d = Decimal.Decimal((1, (1,2,3,4,0), 1))
> >>> d
> Decimal( (1, (1, 2, 3, 4, 0), 1L) )
> >>> d.to_sci_string()
> '-1.2340E+5'
>
> There you have to count the decimals and use the exponent in the string
> to know what the real exponent is (but I don't know how much important
> this is).

It's not important to end users; indeed, trying to tell most of them that
"the real exponent" *isn't* 5 would be difficult.

For most people most of the time, after entering 12345.67, output of
Decimal("12345.67") will be much more understandable than output of 

    Decimal( (0, (1, 2, 3, 4, 5, 6, 7), -2) )



From tim_one at email.msn.com  Fri Apr 16 00:17:24 2004
From: tim_one at email.msn.com (Tim Peters)
Date: Fri Apr 16 00:18:03 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <16510.45405.16024.437304@montanaro.dyndns.org>
Message-ID: <BAY0-SMTP08UzavtVHX0000822d@BAY0-SMTP08.adinternal.hotmail.com>

[Skip]
> Unrelated to this thread, what does "stepping" mean?  A quick Google
> search for "Pentium stepping glossary" yielded this page
>
>   http://processorfinder.intel.com/scripts/help3.asp
>
> but doesn't explain why Intel uses "stepping" instead of "version" or
> "revision".

It means "minor revision".  Major revisions are usually given names (my
current P4 was/is known as "Northwood").  I don't know why minor revisions
are called steppings -- I've never been a HW guy, although the term has been
in common use among HW guys I've known.



From edloper at gradient.cis.upenn.edu  Fri Apr 16 00:52:04 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Fri Apr 16 00:51:05 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <BAY0-SMTP08LscJJGtp0000822b@BAY0-SMTP08.adinternal.hotmail.com>
References: <BAY0-SMTP08LscJJGtp0000822b@BAY0-SMTP08.adinternal.hotmail.com>
Message-ID: <407F6674.4070402@gradient.cis.upenn.edu>

Tim Peters wrote:
>>I can think of at least one real use for exponents outside this range:
>>probabilities.  E.g., if I'm using probabilistic models to estimate
>>P(author|text), I end up multiplying together a large number of very low
>>probabilities, and the total probability could easily get this small.
> 
> No, it couldn't -- and if you worked hard to contrive an example, a
> probability in the end smaller than the probability that the universe will
> spontaneously decide to run time backwards for a century is accurately
> enough represented by 0 (which your example will underflow to).

As long as the limits are soft (and can be changed), I won't object too 
strenuously, because someone who wanted extremely large/small numbers 
could use them.  But...

   - I do believe that there will be use cases for exponents outside this
     range.  Just because there's no physical quantities that are
     reasonably measured with these numbers, doesn't mean that there
     won't be abstract quantities that are reasonably measured with huge
     numbers.

   - I don't believe that signaling an error when a number goes outside
     this range will help catch any errors.  What type of error are we
     expecting to catch here?  If this is such a problem, then why
     doesn't long also have a max/min?

Basically, I don't see much loss from adding the constraints, but I also 
see very little gain.

-Edward

From martin at v.loewis.de  Fri Apr 16 02:00:12 2004
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Apr 16 02:00:42 2004
Subject: [Python-Dev] Optimization targets
In-Reply-To: <1082085955.4953.15.camel@localhost.localdomain>
References: <BAY0-SMTP082so7mbAf00007bae@BAY0-SMTP08.adinternal.hotmail.com>	
	<001901c4223a$5bab0a40$e841fea9@oemcomputer>	
	<20040414214945.GA3443@mike.de>
	<2mekqp1oxq.fsf@starship.python.net>	
	<20040415133613.GA4735@mike.de>	
	<200404151427.i3FERHo26387@guido.python.org>	
	<1082039905.2610.128.camel@localhost.localdomain>	
	<407EDD60.8000101@v.loewis.de>
	<1082085955.4953.15.camel@localhost.localdomain>
Message-ID: <407F766C.3070304@v.loewis.de>

Jeremy Hylton wrote:
> I'm not sure what you mean by the changes over standard JIT.  Do you
> mean the difference between Hotspot and "standard JIT"?  Your
> descriptions sounds a bit like the Deutsch and Schiffman inline method
> cache, which doesn't inline the method body but does inline the method
> lookup.  Or does Hotspot actually inline methods?

I lost the link, but on some page describing it, they actually claimed
they do inline methods. They went on explaining that you need
"deoptimization" in case a different method should be called; I assume
they generate a type check just before the inlined method body. I might
be that they keep collecting statistics on how often the type guess
was wrong to discard the inlining, and come up with a new one.

Hotspot operates by only compiling functions selectively, which have
been invoked a number of times, and they claim they collect data on
where virtual calls go to. So when they eventually do generate machine
code, they have the data to do the inlining.

A "standard" JIT might chose to compile everything the first time it
is invoked, and would then just replace the virtual-call bytecode with
the necessary memory fetches and an indirect call.

Regards,
Martin


From paul at prescod.net  Fri Apr 16 02:11:00 2004
From: paul at prescod.net (Paul Prescod)
Date: Fri Apr 16 02:13:21 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
Message-ID: <407F78F4.6000408@prescod.net>

Jewett, Jim J wrote:

> Mike Pall:
> 
> 
>>About GC: yes, refcounting is the silent killer.
>>... Py_DECREF is awful ... > 3500 locations
> 
> 
> Is it always needed?
> 
> What if a few common (constant, singleton) objects 
> (such as None, -1, 0, 1) were declared immortal at 
> compile-time?  They would be created at initial 
> load in a special untracked pool, and their 
> tp_dealloc would do nothing.
> 
> The slot for tracking references would still be
> there, but could be ignored -- even if it went
> negative.
> 
> Since the reference count no longer has to be
> correct (for these objects), the reference 
> counting macros could optimize to nothing when 
> they know at compile time that they'll have one 
> of these constant objects.  

So instead of writing Py_DECREF(foo) would we write code like

if(foo!=Py_None&&foo!=Py_Neg1&&foo!=Py_Zero&&foo!=PyOne){
	Py_DECREF(foo);
}

  Paul Prescod



From tcdelaney at optusnet.com.au  Fri Apr 16 04:01:22 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Fri Apr 16 04:01:38 2004
Subject: [Python-Dev] Decimal data type issues
References: <BAY0-SMTP08LscJJGtp0000822b@BAY0-SMTP08.adinternal.hotmail.com>
	<407F6674.4070402@gradient.cis.upenn.edu>
Message-ID: <001a01c42389$0252a980$a100a8c0@ryoko>

Edward Loper wrote:

> As long as the limits are soft (and can be changed), I won't object
> too
> strenuously, because someone who wanted extremely large/small numbers
> could use them.  But...
>
>    - I do believe that there will be use cases for exponents outside
>      this range.  Just because there's no physical quantities that are
>      reasonably measured with these numbers, doesn't mean that there
>      won't be abstract quantities that are reasonably measured with
>      huge numbers.

There's no reason that if these use cases become common enough to care
about, that a future version of Python couldn't increase or eliminate the
limits. It's a lot easier to remove limits (e.g. the int/long unification)
than to add them.

I'm +0 on having the limits there initially, even if they complicate the
implementation somewhat. I'm highly unlikely to ever encounter them, just as
I've never produced a long large enough to cause problems.

Tim Delaney




From python-dev at zesty.ca  Fri Apr 16 04:58:26 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri Apr 16 04:58:43 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <200404160009.i3G09cAX020939@cosc353.cosc.canterbury.ac.nz>
References: <200404160009.i3G09cAX020939@cosc353.cosc.canterbury.ac.nz>
Message-ID: <Pine.LNX.4.58.0404160349190.18028@server1.LFW.org>

On Fri, 16 Apr 2004, Greg Ewing wrote:
> Ka-Ping Yee <python-dev@zesty.ca>:
> > The extra method should be specifically for the exceptional case,
> > and named appropriately.
> >
> >     Decimal.exact_float(value)
>
> That's a confusing name, since it seems to be promising that the
> conversion will be exact. While this might technically be true for
> certain values of float, there's no way the implementation can know
> whether the result is exactly what the user had in mind for the
> meaning of that float.

I think you have misunderstood my suggestion.  The name promises that
the conversion will be exact precisely because it *is* exact.  The
exact_float() method would always yield a Decimal number with the
exact value of the given floating-point number.

My understanding of the reason for disallowing a float argument to the
Decimal constructor is that people may be confused by exact conversion:
they don't realize that the exact value may be slightly different
from the decimal number they entered.

My point is that since exact conversion is the confusing case, that's
what the special method should be for.

I'm arguing very much along the same lines as Tim: let's have an
optional argument to the Decimal constructor that specifies the
precision of the newly constructed Decimal value.  Tim is talking
about strings; i am generalizing and saying that, when the precision
is specified, all number types should be accepted.


-- ?!ng

From python-dev at zesty.ca  Fri Apr 16 05:14:36 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri Apr 16 05:14:11 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383885@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383885@escpl.tcp.com.ar>
Message-ID: <Pine.LNX.4.58.0404160358520.18028@server1.LFW.org>

On Thu, 15 Apr 2004, Batista, Facundo wrote:
> [Ka-Ping Yee]
>
> #- Aren't you going to need an optional argument anyway to let the
> #- user specify a context?
>
> What for?

So the user can specify how much precision the resulting Decimal value
should have.

> #- I assume the purpose of from_float is to prevent people from being
> #- confused about whether Decimal(1.1) produces exactly 1.1 or the machine
> #- value of the double-precision rounded float closest to 1.1.  Is that
> #- correct?
> #-
> #- If so, then the real exceptional case is *exact* conversion, not float
> #- conversion.  Decimal(1.1, precision=4) is unambiguous.  Therefore:
>
> Here I think you're confused. *Precision* is a parameter of the context, and
> determines in how many digits you will get the coefficient after an
> operation.
>
> The issue when constructing from float, is how many *positions* after the
> decimal point will be relevant when creating from a float.

I never mentioned positions, and i don't see any reason why it is
necessary to isolate these two ways of specifying precision.  Sometimes
it will be useful to specify a number of significant digits and other
times it will be useful to specify a number of digits after the decimal
point.  Both kinds of specification are useful as a context for arithmetic
operations and useful when constructing from any type including float.

So it would be nice if contexts were able to specify precision in either
fashion.

But that is a separate issue from the one i was trying to raise.  To
bring up this precision vs. position question is to miss my point.

Here is my point, again: Decimal(1.1, significant_digits=4) is unambiguous.

Converting from float is not confusing when the precision is specified
(either in terms of significant digits or decimal places).

Therefore, it is not necessary to make all float conversions inconvenient.
It is only necessary to make *exact* float conversions inconvenient.

Focusing on the true source of the problem also lets us use a more
descriptive name for this special case.

> #- That's what I'm suggesting: that you can supply a context object,
> #- or ask for the default context, when constructing any Decimal value.
>
> Do you want to supply a context that *will affect* the creation process, or
> just supply one that the Decimal will keep to use in other operations?

I want to supply a context for constructing the Decimal value.

It would not be persistent.  You would use some other method (presumably
something you have already conceived and named) for setting the Decimal
context for the entire thread.



-- ?!ng

From Paul.Moore at atosorigin.com  Fri Apr 16 05:19:53 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Fri Apr 16 05:19:55 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB054@UKDCX001.uk.int.atosorigin.com>

From: Ka-Ping Yee

> Here is my point, again: Decimal(1.1, significant_digits=4) is unambiguous.
>
> Converting from float is not confusing when the precision is specified
> (either in terms of significant digits or decimal places).

Decimal(1.1, significant_digits=34) is unambiguous, but confusing.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________

From python-dev at zesty.ca  Fri Apr 16 05:34:59 2004
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri Apr 16 05:34:39 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB88052CB054@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB88052CB054@UKDCX001.uk.int.atosorigin.com>
Message-ID: <Pine.LNX.4.58.0404160433070.18028@server1.LFW.org>

On Fri, 16 Apr 2004, Moore, Paul wrote:
> From: Ka-Ping Yee
> > Here is my point, again: Decimal(1.1, significant_digits=4) is unambiguous.
> >
> > Converting from float is not confusing when the precision is specified
> > (either in terms of significant digits or decimal places).
>
> Decimal(1.1, significant_digits=34) is unambiguous, but confusing.

It is no more confusing than '%.33f' % 1.1, which gives you exactly
the same thing.


-- ?!ng

From pje at telecommunity.com  Fri Apr 16 08:40:11 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Apr 16 08:39:55 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <407F78F4.6000408@prescod.net>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
Message-ID: <5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>

At 11:11 PM 4/15/04 -0700, Paul Prescod wrote:
>Jewett, Jim J wrote:
>
>>Mike Pall:
>>
>>>About GC: yes, refcounting is the silent killer.
>>>... Py_DECREF is awful ... > 3500 locations
>>
>>Is it always needed?
>>What if a few common (constant, singleton) objects (such as None, -1, 0, 
>>1) were declared immortal at compile-time?  They would be created at 
>>initial load in a special untracked pool, and their tp_dealloc would do 
>>nothing.
>>The slot for tracking references would still be
>>there, but could be ignored -- even if it went
>>negative.
>>Since the reference count no longer has to be
>>correct (for these objects), the reference counting macros could optimize 
>>to nothing when they know at compile time that they'll have one of these 
>>constant objects.
>
>So instead of writing Py_DECREF(foo) would we write code like
>
>if(foo!=Py_None&&foo!=Py_Neg1&&foo!=Py_Zero&&foo!=PyOne){
>         Py_DECREF(foo);
>}

I think the idea is more that you could skip the 'Py_INCREF(Py_None)', 
which is a fairly common prelude to 'return Py_None'.  You'd set their 
refcounts to the maximum possible value, and the deallocation function 
would simply reset the refcount to maximum again.

I'm not sure, however, that this would be common enough to be helpful.  It 
seems to me Py_INCREF should effectively translate to only a single machine 
instruction or two.  I mean, it's just incrementing an integer whose 
address is known at compile time.


From FBatista at uniFON.com.ar  Fri Apr 16 08:48:34 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Apr 16 08:50:44 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338389E@escpl.tcp.com.ar>

[Ka-Ping Yee]

#- My understanding of the reason for disallowing a float 
#- argument to the
#- Decimal constructor is that people may be confused by exact 
#- conversion:
#- they don't realize that the exact value may be slightly different
#- from the decimal number they entered.
#- 
#- My point is that since exact conversion is the confusing case, that's
#- what the special method should be for.

I think is deeper than that. For example, *I* don't understand what is
"exact conversion" to you:

  1.1 -> "1.1"
  1.1 -> "1.1000000000000001"
  1.1 -> "1.100000000000000088817841970012523233890533447265625"

So, what is the expectation that floating point newbies will understand this
safely enough as to let them create from:

  Decimal(float, extra_argument)
  Decimal.exact_float()

To me, is complicate the things to much for the newcomer. I think the
following reasoning is simpler:

  Do you want to create from string? Strings are exact, use Decimal(string)
  Do you want to create from int/long? Ints and longs are exact, use
Decimal(int/long)
  Do you want to create from float? Oops, floats are NOT exact, so you have
to use Decimal.from_float(float), but take note of this, this and this.

.	Facundo

From FBatista at uniFON.com.ar  Fri Apr 16 09:07:09 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Apr 16 09:09:28 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D0338389F@escpl.tcp.com.ar>

[Ka-Ping Yee]

#- On Thu, 15 Apr 2004, Batista, Facundo wrote:
#- >...
#- > Here I think you're confused. *Precision* is a parameter 
#- of the context, and
#- > determines in how many digits you will get the coefficient after an
#- > operation.
#- >
#- > The issue when constructing from float, is how many 
#- *positions* after the
#- > decimal point will be relevant when creating from a float.
#- 
#- I never mentioned positions, and i don't see any reason why it is
#- necessary to isolate these two ways of specifying precision. 

Because they are two totally different concepts. And they're not two ways of
specifying precision.

Precision is the maximum of significant digits the number can have. And
positions is the number of digits after the decimal point at which you want
to round the number you're creating from and discard the other digits.


#-  Sometimes
#- it will be useful to specify a number of significant digits and other
#- times it will be useful to specify a number of digits after 
#- the decimal
#- point.  Both kinds of specification are useful as a context 
#- for arithmetic
#- operations and useful when constructing from any type 
#- including float.
#- 
#- So it would be nice if contexts were able to specify 
#- precision in either
#- fashion.

Well. If you want the positions to be included in the context to be used in
arithmetic operations, you're out of spec. And the PEP is for implementing
the "General Decimal Arithmetic Specification", so the answer to that is NO.


#- But that is a separate issue from the one i was trying to raise.  To
#- bring up this precision vs. position question is to miss my point.
#- 
#- Here is my point, again: Decimal(1.1, significant_digits=4) 
#- is unambiguous.

I don't think so. For example: 

>>> d = Decimal.Decimal.from_float(25364.1254)
>>> str(d)
'25364.12540000000080908648669719696044921875'

What whould mean here "significant_digits=4"?

  To have '2536E2'? You use "precision" to achieve this.
  To have '25364.1254'? You use "position" in from_float method to achieve
this.


#- Converting from float is not confusing when the precision is 
#- specified
#- (either in terms of significant digits or decimal places).

Don't confuse the terminology. Precision means only one thing, and it's
specified in Cowlishaw work. You can not change that. Maybe all the
discussion doesn't make sense if we talk about different things.


#- I want to supply a context for constructing the Decimal value.

I'm +1 with this. But the context that you supply affect the PRECISION to be
used. It has nothing to do with POSITIONS.

.	Facundo

From casey at zope.com  Fri Apr 16 09:36:05 2004
From: casey at zope.com (Casey Duncan)
Date: Fri Apr 16 09:34:52 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D03383884@escpl.tcp.com.ar>
	<BAY0-SMTP08BS3hgRCw0000822c@BAY0-SMTP08.adinternal.hotmail.com>
Message-ID: <20040416093605.78034671.casey@zope.com>

On Fri, 16 Apr 2004 00:17:24 -0400
"Tim Peters" <tim_one@email.msn.com> wrote:

> [Facundo Batista, on whether to limit exponent magnitude]
[..]
> No, I want to spell the spec's mandatory from-string *operation*
> (which does use and affect context, in the specified ways) via an
> optional bool argument to the Decimal constructor.  This isn't at all
> useless, although if you're only thinking about a money type it may
> seem that way.  In general number-crunching, literals may be given to
> high precision, but that precision isn't free and *usually* isn't
> needed:  the operations in the spec don't round back inputs to current
> context precision before computing, they only round outputs after, and
> digits feeding into computation are expensive. So, in general, a
> scientific user will usually *not* want all the digits in
> 
>     pi = Decimal("3.1415926535897932384626433832795",
>     use_context=True)
> 
> to be used.  Instead they'll want to set the working precision once in
> context, and have their literals automatically get rounded back to
> that precision.  
[..]

I find boolean arguments a general anti-pattern, especially given we
have class methods. Why not use an alternate constructor like::

  pi = Decimal.rounded_to_context("3.1415926535897932384626433832795")

Spelling notwithstanding.

-Casey


From FBatista at uniFON.com.ar  Fri Apr 16 09:53:35 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Apr 16 09:55:51 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>

[Casey Duncan]

#- I find boolean arguments a general anti-pattern, especially given we
#- have class methods. Why not use an alternate constructor like::
#- 
#-   pi = 
#- Decimal.rounded_to_context("3.1415926535897932384626433832795")

This is a good idea to me.

We all must decide which of the following is better:

  a) Decimal(number)     # no context used
     Decimal(number, context)

  b) Decimal(number)     # no context used
     Decimal.using_context(number, context)

"number" can be string, int, etc., but NOT float. The issue with rounding at
a defined position has nothing to do with context.

.	Facundo

From skip at pobox.com  Fri Apr 16 10:18:38 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Apr 16 10:18:51 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
Message-ID: <16511.60222.516830.948141@montanaro.dyndns.org>

(I'm not sure where precisely to address this.  I'd send it directly to
Barry but figure that someone else might also know the answer and have
infinitessimally more time to respond.)

In my new job I nipped the Python project's syncmail script and added it to
our CVSROOT.  I then tweaked the loginfo file to run syncmail under the
proper conditions.  The messages go out just fine, but they come from
server@servername instead of from the person who executed the checkin
(e.g. skipm@servername).  Since that's the only indication in the syncmail
message who performed the checkin I'd like to get that corrected.  It
appears syncmail just sends out the message as the user it's running as.  Is
this a cvs server setting or some tweak to the SF version of CVS?  Locally
we're running cvs 1.11.2, which seems to match what SF runs on shell.sf.net.

Thanks,

Skip

From casey at zope.com  Fri Apr 16 10:32:14 2004
From: casey at zope.com (Casey Duncan)
Date: Fri Apr 16 10:30:47 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>
Message-ID: <20040416103214.6fdf808f.casey@zope.com>

On Fri, 16 Apr 2004 10:53:35 -0300
"Batista, Facundo" <FBatista@uniFON.com.ar> wrote:

> [Casey Duncan]
> 
> #- I find boolean arguments a general anti-pattern, especially given
> #we- have class methods. Why not use an alternate constructor like::
> #- 
> #-   pi = 
> #- Decimal.rounded_to_context("3.1415926535897932384626433832795")
> 
> This is a good idea to me.
> 
> We all must decide which of the following is better:
> 
>   a) Decimal(number)     # no context used
>      Decimal(number, context)
> 
>   b) Decimal(number)     # no context used
>      Decimal.using_context(number, context)

If the context must be explicitly provided as an argument (rather than
just a flag) I think a) is better. It depends also on what other
arguments the constructor has, and thus what other contracts it
fulfills. Are there other optional arguments?

-Casey


From jacobs at theopalgroup.com  Fri Apr 16 10:32:47 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Fri Apr 16 10:32:52 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>
Message-ID: <407FEE8F.2090402@theopalgroup.com>

Batista, Facundo wrote:

>[Casey Duncan]
>
>#- I find boolean arguments a general anti-pattern, especially given we
>#- have class methods. Why not use an alternate constructor like::
>#- 
>#-   pi = 
>#- Decimal.rounded_to_context("3.1415926535897932384626433832795")
>
>This is a good idea to me.
>
>We all must decide which of the following is better:
>
>  a) Decimal(number)     # no context used
>     Decimal(number, context)
>
>  b) Decimal(number)     # no context used
>     Decimal.using_context(number, context)
>
>"number" can be string, int, etc., but NOT float. The issue with rounding at
>a defined position has nothing to do with context.
>  
>

Don't forget that many financial applications use fixed scale and 
precision as
their primary mechanism for specifying Decimal types.  As such, it
would be very nice to have a constructor that took a literal representation
as well as scale and precision.  While using context is sufficient, it will
be unintuitive and may present an undue burden on many users.

So-many-people-to-make-happy-ly,
-Kevin


From guido at python.org  Fri Apr 16 10:58:17 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr 16 10:58:24 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: Your message of "Fri, 16 Apr 2004 08:40:11 EDT."
	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com> 
Message-ID: <200404161458.i3GEwH328527@guido.python.org>

> I think the idea is more that you could skip the 'Py_INCREF(Py_None)', 
> which is a fairly common prelude to 'return Py_None'.  You'd set their 
> refcounts to the maximum possible value, and the deallocation function 
> would simply reset the refcount to maximum again.
> 
> I'm not sure, however, that this would be common enough to be helpful.  It 
> seems to me Py_INCREF should effectively translate to only a single machine 
> instruction or two.  I mean, it's just incrementing an integer whose 
> address is known at compile time.

I vaguely recall that this was proposed and perhaps even tried before,
and not found to make a difference.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From sjoerd at acm.org  Fri Apr 16 11:26:04 2004
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Fri Apr 16 11:26:37 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <16511.60222.516830.948141@montanaro.dyndns.org>
References: <16511.60222.516830.948141@montanaro.dyndns.org>
Message-ID: <407FFB0C.6050200@acm.org>

Skip Montanaro wrote:
> (I'm not sure where precisely to address this.  I'd send it directly to
> Barry but figure that someone else might also know the answer and have
> infinitessimally more time to respond.)
> 
> In my new job I nipped the Python project's syncmail script and added it to
> our CVSROOT.  I then tweaked the loginfo file to run syncmail under the
> proper conditions.  The messages go out just fine, but they come from
> server@servername instead of from the person who executed the checkin
> (e.g. skipm@servername).  Since that's the only indication in the syncmail
> message who performed the checkin I'd like to get that corrected.  It
> appears syncmail just sends out the message as the user it's running as.  Is
> this a cvs server setting or some tweak to the SF version of CVS?  Locally
> we're running cvs 1.11.2, which seems to match what SF runs on shell.sf.net.

Are you using ssh or pserver to talk to the cvs server?
If using ssh, does every user have his/her own account on the server 
that they use?
When using pserver, the cvs server is run under a fixed user id, so 
that's where the mail will appear to come from.

-- 
Sjoerd Mullender <sjoerd@acm.org>

From FBatista at uniFON.com.ar  Fri Apr 16 11:32:19 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Apr 16 11:34:29 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838A5@escpl.tcp.com.ar>

[Casey Duncan]

#- If the context must be explicitly provided as an argument 
#- (rather than
#- just a flag) I think a) is better. It depends also on what other
#- arguments the constructor has, and thus what other contracts it
#- fulfills. Are there other optional arguments?

Until now, it receives just the number.

Community seems to agree to be optional to pass a context for
construction-time. There isn't other argument by now.

.	Facundo

From FBatista at uniFON.com.ar  Fri Apr 16 11:33:38 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Apr 16 11:36:43 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838A6@escpl.tcp.com.ar>

[Kevin Jacobs]
#- Don't forget that many financial applications use fixed scale and 
#- precision as
#- their primary mechanism for specifying Decimal types.  As such, it
#- would be very nice to have a constructor that took a literal 
#- representation
#- as well as scale and precision.  While using context is 

What is "fixed scale"?

Could you provide examples of the functionality that you're asking for?
Thanks.

.	Facundo

From skip at pobox.com  Fri Apr 16 11:43:43 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Apr 16 11:43:57 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <407FFB0C.6050200@acm.org>
References: <16511.60222.516830.948141@montanaro.dyndns.org>
	<407FFB0C.6050200@acm.org>
Message-ID: <16511.65327.705618.504369@montanaro.dyndns.org>


    Sjoerd> Are you using ssh or pserver to talk to the cvs server?

pserver

    Sjoerd> When using pserver, the cvs server is run under a fixed user id,
    Sjoerd> so that's where the mail will appear to come from.

Thanks, that must be it.  I'll have to scrounge around for a way to make
syncmail figure out the submitter.

Skip

From fdrake at acm.org  Fri Apr 16 12:20:19 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Apr 16 12:21:14 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <16511.65327.705618.504369@montanaro.dyndns.org>
References: <16511.60222.516830.948141@montanaro.dyndns.org>
	<407FFB0C.6050200@acm.org>
	<16511.65327.705618.504369@montanaro.dyndns.org>
Message-ID: <200404161220.19351.fdrake@acm.org>

On Friday 16 April 2004 11:43 am, Skip Montanaro wrote:
 > Thanks, that must be it.  I'll have to scrounge around for a way to make
 > syncmail figure out the submitter.

Hint:  the pserver puts the username used to authenticate in the CVS_USER 
environment variable when running scripts from the admin files (like 
loginfo).

See:  http://www.cvshome.org/docs/manual/cvs-1.11.14/cvs_18.html#SEC179


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jacobs at theopalgroup.com  Fri Apr 16 12:56:43 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Fri Apr 16 12:56:47 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838A6@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838A6@escpl.tcp.com.ar>
Message-ID: <4080104B.6000702@theopalgroup.com>

Batista, Facundo wrote:

>[Kevin Jacobs]
>#- Don't forget that many financial applications use fixed scale and 
>#- precision as
>#- their primary mechanism for specifying Decimal types.  As such, it
>#- would be very nice to have a constructor that took a literal 
>#- representation
>#- as well as scale and precision.  While using context is 
>
>What is "fixed scale"?
>
>Could you provide examples of the functionality that you're asking for?
>Thanks.
>  
>

See for example, the SQL Decimal and Numeric data types, which is one of 
many
applications that use scale and precision parameters for decimal 
representations:

[borrowed from the PostgreSQL manual @
    
http://www.postgresql.org/docs/7.4/interactive/datatype.html#DATATYPE-NUMERIC-DECIMAL]

> The /scale/ of a numeric is the count of decimal digits in the 
> fractional part, to the right
> of the decimal point. The /precision/ of a numeric is the total count 
> of significant digits
> in the whole number, that is, the number of digits to both sides of 
> the decimal point. So
> the number 23.5141 has a precision of 6 and a scale of 4. Integers can 
> be considered to
> have a scale of zero.
>
> Both the precision and the scale of the numeric type can be 
> configured. To declare a column
> of type numeric use the syntax
>
>  NUMERIC(precision, scale)
>
> The precision must be positive, the scale zero or positive. 
> Alternatively,
>
>  NUMERIC(precision)
>
> selects a scale of 0. Specifying
>
>  NUMERIC
>
> without any precision or scale creates a column in which numeric 
> values of any precision
> and scale can be stored, up to the implementation limit on precision. 
> A column of this kind
> will not coerce input values to any particular scale, whereas numeric 
> columns with a
> declared scale will coerce input values to that scale. (The SQL 
> standard requires a default
> scale of 0, i.e., coercion to integer precision. We find this a bit 
> useless. If you're concerned
> about portability, always specify the precision and scale explicitly.)
>
> If the precision or scale of a value is greater than the declared 
> precision or scale of a
> column, the system will attempt to round the value. If the value 
> cannot be rounded
> so as to satisfy the declared limits, an error is raised.
>

Python decimals would do well to support the creation of instances with 
fixed scale
and precision parameters, since this information will be what is 
provided by databases,
other financial and engineering applications, and schema.  i.e., these 
parameters override
the natural scale and precision found in the literal values used when 
constructing decimals.
e.g., hypothetically:

  Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
  Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')

Remember, these literals are frequently coming from an external source 
that must
be constrained to a given schema.  This issue doesn't come up with 
floating point
numbers nearly as often, because those schema use explicitly 
precision-qualified
types (i.e., single, double, quad precision).

-Kevin


From jim.jewett at eds.com  Fri Apr 16 13:05:20 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Fri Apr 16 13:06:06 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D472@USAHM010.amer.corp.eds.com>

Paul Moore:
> Decimal(1.1, significant_digits=34) is unambiguous, 
> but confusing.

What would the preferred solution be?

Requiring the user to explicitly type zeros so as to 
have 34 digits?

Implicitly doing a Decimal(str(1.1)) and then adding
32 zeros to the internal representation?  (Note: in
at least my environment, str(1.1) = '1.1'; it is only
repr that adds the extra digits.)

Living with the fact that it will have some confusing
non-zero digits?

Facundo Batista:
> ... what is "exact conversion" to you:

>  1.1 -> "1.1"
>  1.1 -> "1.1000000000000001"
>  1.1 -> "1.100000000000000088817841970012523233890533447265625"

machine precision of floats, whatever that happens to be.  
That implementation-dependent part is one reason to use a
longer name, like exact_float.

But if I say 

  Decimal(1.1, precision=4)
  Decimal(1.1, positions=3)

Those are unambiguous.  If the context is 43, then later 
calculations will effectively fill in the extra digits;
I have just said to fill them in with zeros, rather than 
whatever the float approximations were.


Ka-Ping Yee:
>> I never mentioned positions, and i don't see any reason 
>> why it is necessary to isolate these two ways of specifying 
>> precision. 

Facundo Batista:
> Because they are two totally different concepts. And they're 
> not two ways of specifying precision.

Sure they are; the reason to specify positions is that the
underlying data wasn't really floating point -- it was an
integer which by convention is written in a larger unit.
Example with money:

	Assume meat is 0.987USD/pound.  There are three
	price digits to multiply by the weight, but the
	final cost is rounded to an integer number of
	pennies.

	10 pounds cost 9.87, but
	1 pound costs 0.99

I don't want to say that my prices have three digits of 
precision, or I'll keep fractions of a penny.  I don't
want to say that they have only two, or I'll drop the
pennies on expensive items.  I want to say that the
precision varies depending on the actual price.

> What whould mean here "significant_digits=4"?

It means precision.  In many US classrooms, 
"precision" applies to the original measurement.  
Once you start doing calculations, it gets called 
"significant digits", as a reminder not to let your 
answer be more precise than your inputs.

Ka-Ping Yee:
>> Converting from float is not confusing when the 
>> precision is specified (either in terms of significant 
>> digits or decimal places).

> Don't confuse the terminology. Precision means only one
> thing, and it's specified in Cowlishaw work.

But for any *specific* value, specifying either the total 
number of valid digits (signficant figures, precision) 
or the number of fractional digits (position) is enough
to determine both.

> "number" can be string, int, etc., but NOT float.
> The issue with rounding at a defined position has 
> nothing to do with context.

I assume that a new Decimal would normally be created
with as much precision as the context would need for 
calculations.  By passing a context/precision/position,
the user is saying "yeah, but this measurement wasn't
that precise in the first place.  Use zeros for the
rest, no matter what this number claims."

-jJ

From skip at pobox.com  Fri Apr 16 14:09:57 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Apr 16 14:10:07 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <r01050400-1026-E9ACD2668FBD11D8BBEE003065D5E7E4@[10.0.0.23]>
References: <16511.65327.705618.504369@montanaro.dyndns.org>
	<r01050400-1026-E9ACD2668FBD11D8BBEE003065D5E7E4@[10.0.0.23]>
Message-ID: <16512.8565.21289.392919@montanaro.dyndns.org>


    >> Thanks, that must be it.  I'll have to scrounge around for a way to
    >> make syncmail figure out the submitter.

    Just> If the submitters all have "regular" accounts on the machine
    Just> running pserver, you can easily have each submitting process run
    Just> as the proper user, by adding the user to the CVSROOT/passwd file,
    Just> like so:

    Just> Just:xxxxxxxx:just

    Just> this is:

    Just> <pserver user name>:<crypt'ed password>:<run as this user>

Hot damn!  That might be just the ticket.  I'll try it with my own account
first.

Thanks,

Skip


From mikepy-0404 at mike.de  Fri Apr 16 14:26:05 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Fri Apr 16 14:26:10 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
Message-ID: <20040416182605.GA3590@mike.de>

Hi,

pje wrote:
> I think the idea is more that you could skip the 'Py_INCREF(Py_None)', 
> which is a fairly common prelude to 'return Py_None'.  You'd set their 
> refcounts to the maximum possible value, and the deallocation function 
> would simply reset the refcount to maximum again.
> 
> I'm not sure, however, that this would be common enough to be helpful.  It 
> seems to me Py_INCREF should effectively translate to only a single machine 
> instruction or two.  I mean, it's just incrementing an integer whose 
> address is known at compile time.

I don't think it would matter much performance-wise. A quick scan does
not reveal a single Py_INCREF(Py_None) in any hot code section. But with
more than 800 occurences it would reduce the code size a bit (5-10K).
So that alone is not a good reason to throw it out.

Checking for Py_None in Py_DECREF is counter-productive because that would
need another branch. The existing branch is always taken with Py_None since
the refcount never goes zero. I.e. this is not an issue.

But let's look at the general case. Here's a common fragment of inlined
x86 code for Py_DECREF:

mov    -something(%ebp),%esi	; odep %esi
mov    (%esi),%eax		; idep %esi, odep %eax
dec    %eax 			; idep %eax, odep %eax
test   %eax,%eax		; [idep %eax], odep flags
mov    %eax,(%esi)		; [idep %eax %esi]
jne    .L1 			; idep flags, predict

sub    $0xc,%esp
mov    0x4(%esi),%eax
push   %esi
call   *0x18(%eax)
add    $0x10,%esp
.L1:

There are at least 2 to 3 data dependencies that are hard to untangle.
Tough luck.

The other problem is branch prediction. I've done a little analysis with
gcov on ceval.c and found that the situation is not that bad. The Py_DECREF
occurences that account for 90% of the time are almost exclusively 0/100
or 100/0 cases. The remaining 10% have a few 30/70 or 50/50 cases.
The folded and weighted average is around 4/96. The branch misprediction
penalty accounts for less than 0.1% of the runtime.

But the deallocation itself is costly. On average the deallocator is
called 15% of the time (I checked ceval.c only). This is a lot and
accounts for up to 5% of the runtime (this is based on an overall profile).
My guess is that allocation is a bit cheaper, but adds another 2-3%.
I think tuples are the worst offender (again only a good guess from my
profiling experiments).

To summarize: I think we can optimize the GC a little more (have not done
an in-depth analysis yet), but we really should focus on making Python
less malloc-happy. Suggestions welcome!

Bye,
     Mike

From jepler at unpythonic.net  Fri Apr 16 15:06:22 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Fri Apr 16 15:06:28 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
Message-ID: <20040416190622.GC3078@unpythonic.net>

Maybe something like this:

#ifdef __GCC__
#define constant_p(x) __builtin_constant_p(x)
#else
#define constant_p(x) 0
#endif

#define Py_INCREF(x) \
    if( ! constant_p(x) || (x != Py_None && x != other immortals)) \
        actual Py_INCREF body(x)

this should allow the compiler, by constant propagation, to skip actual
Py_INCREF body when it sees
    Py_INCREF(Py_None)
and when it's
    Py_INCREF(nonconstant)
to have no test for equality with the immortal values.

does this actually work?  I didn't check.  I think that on some
important systems there may be no equivalent to gcc's
__builtin_constant_p(), and that Py_None is not a compile-time constant
(*cough* DLLs *cough*)

Jeff

From skip at pobox.com  Fri Apr 16 17:30:48 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Apr 16 17:30:56 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <16512.8565.21289.392919@montanaro.dyndns.org>
References: <16511.65327.705618.504369@montanaro.dyndns.org>
	<r01050400-1026-E9ACD2668FBD11D8BBEE003065D5E7E4@[10.0.0.23]>
	<16512.8565.21289.392919@montanaro.dyndns.org>
Message-ID: <16512.20616.41799.327583@montanaro.dyndns.org>


    Just> <pserver user name>:<crypt'ed password>:<run as this user>

    Skip> Hot damn!  That might be just the ticket.  I'll try it with my own
    Skip> account first.

Alas, I was told that wouldn't work in our environment.  Permission problems
or something.  (IOW, more administrative struggle than I was prepared to
deal with. ;-)

    Fred> Hint: the pserver puts the username used to authenticate in the
    Fred> CVS_USER environment variable when running scripts from the admin
    Fred> files (like loginfo).

This worked perfectly.

Thanks for the help...

Skip

From rowen at cesmail.net  Fri Apr 16 19:52:43 2004
From: rowen at cesmail.net (Russell E. Owen)
Date: Fri Apr 16 19:52:49 2004
Subject: [Python-Dev] PEP 318, a different approach
Message-ID: <rowen-0BB15C.16524216042004@sea.gmane.org>

I had a strange idea for decorator syntax.

def.classmethod func(cls,...):

Multiple decorators are going to need the usual [] thing (or the current 
clumsy notation), but the strange thought is allowing

def.iter

so one could be more explicit about defining iterator functions w/out 
adding a keyword.

Sorry to beat that dead horse. Back to the real notation.

-- Russell

P.S. one could imagine allowing these, as well,, but I can't imagine 
users typing them:
def.func
def.instancemethod


From guido at python.org  Fri Apr 16 20:31:07 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr 16 20:31:15 2004
Subject: [Python-Dev] PEP 318, a different approach
In-Reply-To: Your message of "Fri, 16 Apr 2004 16:52:43 PDT."
	<rowen-0BB15C.16524216042004@sea.gmane.org> 
References: <rowen-0BB15C.16524216042004@sea.gmane.org> 
Message-ID: <200404170031.i3H0V7h29228@guido.python.org>

> I had a strange idea for decorator syntax.
> 
> def.classmethod func(cls,...):

This would violate the 'greppability' requirement.  It's too nice to
have the name always be the token immediately following 'def' to allow
anything in between the two.

Basically, the only two positions that are still in the race are
before the def, and between the argument list and the colon.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From andymac at bullseye.apana.org.au  Fri Apr 16 19:36:52 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Fri Apr 16 21:14:00 2004
Subject: [Python-Dev] String hash function multiplier
In-Reply-To: <20040414193359.GW6139@unpythonic.net>
References: <4FCB1278-8DB3-11D8-B460-000A95686CD8@redivi.com>
	<001d01c421c6$77b4df00$e841fea9@oemcomputer>
	<20040414030938.GD8795@unpythonic.net>
	<20040414130834.GS6139@unpythonic.net>
	<20040414232756.Q67555@bullseye.apana.org.au>
	<20040414193359.GW6139@unpythonic.net>
Message-ID: <20040417093341.V89016@bullseye.apana.org.au>

On Wed, 14 Apr 2004, Jeff Epler wrote:

> imul is on all x86 architectures, but whether to use it or not depends
> on the characteristics of the target CPU.  With -mcpu=i386, imul is
> considered quite slow and a shift sequence is (almost?) always preferred
> when one operand is constant.  With -mcpu=i686 and newer, imul seems to
> be preferred.

My misunderstanding. I hadn't really comprehended the full extent of
"instruction scheduling" in this context.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia

From john at neggie.net  Sat Apr 17 01:29:43 2004
From: john at neggie.net (John Belmonte)
Date: Sat Apr 17 01:29:53 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <407C28E0.7090808@neggie.net>
References: <407C28E0.7090808@neggie.net>
Message-ID: <4080C0C7.7070602@neggie.net>

I've implemented the proposal, as far as the pydoc modifications.  See 
<http://sourceforge.net/tracker/index.php?func=detail&aid=936774&group_id=5470&atid=305470>.

>     3) extend the __slots__ handler to set the descriptor's doc string 
> to dictionary values if the slot's value is a dictionary.

I haven't implemented this, because I don't know a thing about Python's 
C interface.  If someone more able would knock off a prototype, I would 
be grateful.

-John


-- 
http:// if  ile.org/

From aahz at pythoncraft.com  Sat Apr 17 09:58:34 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 17 09:58:40 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
Message-ID: <20040417135834.GA8542@panix.com>

On Thu, Apr 15, 2004, Jewett, Jim J wrote:
>
> What if a few common (constant, singleton) objects (such as None, -1,
> 0, 1) were declared immortal at compile-time?  They would be created
> at initial load in a special untracked pool, and their tp_dealloc
> would do nothing.
>
> The slot for tracking references would still be there, but could be
> ignored -- even if it went negative.
>
> Since the reference count no longer has to be correct (for these
> objects), the reference counting macros could optimize to nothing when
> they know at compile time that they'll have one of these constant
> objects.

http://mail.python.org/pipermail/python-list/2003-September/thread.html#183710
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From aahz at pythoncraft.com  Sat Apr 17 10:01:13 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 17 10:01:16 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <20040416182605.GA3590@mike.de>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
	<20040416182605.GA3590@mike.de>
Message-ID: <20040417140113.GB8542@panix.com>

On Fri, Apr 16, 2004, Mike Pall wrote:
>
> To summarize: I think we can optimize the GC a little more (have not done
> an in-depth analysis yet), but we really should focus on making Python
> less malloc-happy. Suggestions welcome!

Could you expand on this?  What do you mean by "less malloc-happy"?
What you were talking about earlier was ``None``, which doesn't go
through malloc hoops.

I think you might find it useful to at least read old threads about
refcounting vs. GC.  (I managed to dig up the old thread about getting
rid of refcounting for ``None``; don't have time for more.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From aahz at pythoncraft.com  Sat Apr 17 10:03:46 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 17 10:03:49 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <407FEE8F.2090402@theopalgroup.com>
References: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>
	<407FEE8F.2090402@theopalgroup.com>
Message-ID: <20040417140346.GC8542@panix.com>

On Fri, Apr 16, 2004, Kevin Jacobs wrote:
>
> Don't forget that many financial applications use fixed scale and
> precision as their primary mechanism for specifying Decimal types.  As
> such, it would be very nice to have a constructor that took a literal
> representation as well as scale and precision.  While using context is
> sufficient, it will be unintuitive and may present an undue burden on
> many users.

Could you use an example to explain what you mean?  This doesn't make
sense to me given that there are already two mechanisms for precisely
specifying the input to Decimal().
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From aahz at pythoncraft.com  Sat Apr 17 10:07:13 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 17 10:07:22 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <4080104B.6000702@theopalgroup.com>
References: <A128D751272CD411BC9200508BC2194D033838A6@escpl.tcp.com.ar>
	<4080104B.6000702@theopalgroup.com>
Message-ID: <20040417140712.GD8542@panix.com>

[That'll teach me to respond before reading a thread. ;-) ]

On Fri, Apr 16, 2004, Kevin Jacobs wrote:
>
> Python decimals would do well to support the creation of instances
> with fixed scale and precision parameters, since this information will
> be what is provided by databases, other financial and engineering
> applications, and schema. i.e., these parameters override the natural
> scale and precision found in the literal values used when constructing
> decimals. e.g., hypothetically:
>
>  Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
>  Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')

I'd say it makes more sense for someone else to implement these features
as helper functions or classes.  They're not directly part of the
Decimal system.  Facundo got roped into this because he wanted a Money
class and was persuaded to do the finishing work for getting Decimal
into Python as the basis.  Seems to me this is in the same category of
building some additional capabilities on top of Decimal.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From aahz at pythoncraft.com  Sat Apr 17 10:09:51 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 17 10:09:54 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838A4@escpl.tcp.com.ar>
Message-ID: <20040417140951.GE8542@panix.com>

On Fri, Apr 16, 2004, Batista, Facundo wrote:
>
> We all must decide which of the following is better:
> 
>   a) Decimal(number)     # no context used
>      Decimal(number, context)
> 
>   b) Decimal(number)     # no context used
>      Decimal.using_context(number, context)

Based on my reading of Tim's posts, there's also

    c) Decimal(number, [context])   # context always used, with 
                                    # optional different from default
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From guido at python.org  Sat Apr 17 13:33:04 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr 17 13:33:11 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: Your message of "Sat, 17 Apr 2004 01:29:43 EDT."
	<4080C0C7.7070602@neggie.net> 
References: <407C28E0.7090808@neggie.net>  <4080C0C7.7070602@neggie.net> 
Message-ID: <200404171733.i3HHX4B30513@guido.python.org>

> >     3) extend the __slots__ handler to set the descriptor's doc string 
> > to dictionary values if the slot's value is a dictionary.
> 
> I haven't implemented this, because I don't know a thing about Python's 
> C interface.  If someone more able would knock off a prototype, I would 
> be grateful.

If I understand this proposal correctly (the dict values would provide
the docstrings for the slots) I'd rather not make this default
behavior; different metaclasses may want to use the dict value for
different purposes, and subsuming it by default for the doc string
would make that harder to do.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fumanchu at amor.org  Sat Apr 17 15:17:36 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Sat Apr 17 15:19:58 2004
Subject: [Python-Dev] dict.sortedkeys()
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E562011@opus.amorhq.net>

I'm willing to pay money for someone to write {}.sortedkeys() and/or
sets.Set().sorteditems() (since IANACH (I am not a C hacker)). I'm just
*really* tired of writing what should be a one-liner in three lines:

From fumanchu at amor.org  Sat Apr 17 15:21:18 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Sat Apr 17 15:23:39 2004
Subject: [Python-Dev] dict.sortedkeys()
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E562012@opus.amorhq.net>

I wrote:
> I'm willing to pay money for someone to write {}.sortedkeys() and/or
> sets.Set().sorteditems() (since IANACH (I am not a C 
> hacker)). I'm just
> *really* tired of writing what should be a one-liner in three lines:

... e.g.

    dkeys = d.keys()
    dkeys.sort()
    for key in dkeys:

instead of:

    for key in d.sortedkeys():


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From jepler at unpythonic.net  Sat Apr 17 16:08:17 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Sat Apr 17 16:08:31 2004
Subject: [Python-Dev] dict.sortedkeys()
In-Reply-To: <DE1CF2B4FEC4A342BF62B6B2B334601E562011@opus.amorhq.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E562011@opus.amorhq.net>
Message-ID: <20040417200817.GB18480@unpythonic.net>

Factor sorted into a function that returns a sorted copy:
    def sorted(seq):
        seq = list(seq)
        seq.sort()
        return seq

or one that sorts in place:
    def sorted(seq):
        seq.sort()
        return seq

Now you can write your one-liner:
for i in sorted({}.keys()):
    ...

Jeff

From bob at redivi.com  Sat Apr 17 16:23:35 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Apr 17 16:19:21 2004
Subject: [Python-Dev] dict.sortedkeys()
In-Reply-To: <DE1CF2B4FEC4A342BF62B6B2B334601E562011@opus.amorhq.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E562011@opus.amorhq.net>
Message-ID: <1A2EBB04-90AD-11D8-8EF5-000A95686CD8@redivi.com>

On Apr 17, 2004, at 3:17 PM, Robert Brewer wrote:

> I'm willing to pay money for someone to write {}.sortedkeys() and/or
> sets.Set().sorteditems() (since IANACH (I am not a C hacker)). I'm just
> *really* tired of writing what should be a one-liner in three lines:

I'm pretty sure that in Python 2.4 you will be able to say sorted(seq).

-bob


From fumanchu at amor.org  Sat Apr 17 16:21:26 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Sat Apr 17 16:23:48 2004
Subject: [Python-Dev] dict.sortedkeys()
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E562014@opus.amorhq.net>

[Jeff Epler]
> Factor sorted into a function that returns a sorted copy:
>     def sorted(seq):
>         seq = list(seq)
>         seq.sort()
>         return seq
> 
> or one that sorts in place:
>     def sorted(seq):
>         seq.sort()
>         return seq
> 
> Now you can write your one-liner:
> for i in sorted({}.keys()):
>     ...

Yes, of course. But I'm working on a large business app framework. That
means either:

1. Writing sorted into every module where it's needed.
2. Putting sorted into a package somewhere, which makes it longer and
uglier with absolute referencing.
3. Customizing consumers' Python install.

...none of which are good options.


[Bob Ippolito]
> I'm pretty sure that in Python 2.4 you will be able to say
sorted(seq).

Excellent! That'll do nicely. :)


-another bob

From fdrake at acm.org  Sat Apr 17 16:46:26 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sat Apr 17 16:46:36 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <16512.20616.41799.327583@montanaro.dyndns.org>
References: <16511.65327.705618.504369@montanaro.dyndns.org>
	<16512.8565.21289.392919@montanaro.dyndns.org>
	<16512.20616.41799.327583@montanaro.dyndns.org>
Message-ID: <200404171646.26885.fdrake@acm.org>

On Friday 16 April 2004 05:30 pm, Skip Montanaro wrote:
 > Alas, I was told that wouldn't work in our environment.  Permission
 > problems or something.  (IOW, more administrative struggle than I was
 > prepared to deal with. ;-)

There's the little matter that CVS users don't necessarily map one-to-one with 
user accounts on the host with pserver to deal with as well.

 >     Fred> Hint: the pserver puts the username used to authenticate in the
 >     Fred> CVS_USER environment variable when running scripts from the
 > admin Fred> files (like loginfo).
 >
 > This worked perfectly.

This is something syncmail should probably support out of the box.  If you'd 
like to submit a patch, we'd love to have it:

    http://sourceforge.net/projects/cvs-syncmail/


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one at comcast.net  Sat Apr 17 17:37:23 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sat Apr 17 17:37:30 2004
Subject: [Python-Dev] No-cost optimizing VC 7.1
Message-ID: <E1BExVA-0004Be-3Z@mail.python.org>

Thanks to David Abrahams for passing on this info:  a free optimizing
cmdline VC 7.1 has been released by MS:

    http://msdn.microsoft.com/visualc/vctoolkit2003/

The 32MB download appears to contain much of the meat of Visual Studio .NET
2003 Professional, including the compiler, linker, static-link libraries for
the C, C++ & STL runtimes, and the .NET Common Language Runtime.



From mikepy-0404 at mike.de  Sat Apr 17 17:50:00 2004
From: mikepy-0404 at mike.de (Mike Pall)
Date: Sat Apr 17 17:50:28 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <20040417140113.GB8542@panix.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
	<20040416182605.GA3590@mike.de> <20040417140113.GB8542@panix.com>
Message-ID: <20040417215000.GA5263@mike.de>

Hi,

Aahz wrote:
> On Fri, Apr 16, 2004, Mike Pall wrote:
> > To summarize: I think we can optimize the GC a little more (have not done
> > an in-depth analysis yet), but we really should focus on making Python
> > less malloc-happy. Suggestions welcome!
> 
> Could you expand on this?  What do you mean by "less malloc-happy"?
> What you were talking about earlier was ``None``, which doesn't go
> through malloc hoops.
> 
> I think you might find it useful to at least read old threads about
> refcounting vs. GC.  (I managed to dig up the old thread about getting
> rid of refcounting for ``None``; don't have time for more.)

I'm very well aware of the tradeoffs between refcounting and pure GC
schemes. I do *not* suggest that we change CPython's approach. And I do
not intend to start another refcount-vs-GC flamewar. You've got me wrong.

Also please check the thread -- I never suggested modifying anything
about Py_None. I just presented a short quantitative analysis that it's
useless to do so (the thread you referenced contains another good reason).

What I meant with 'less malloc-happy' is that we should try to reduce
the number of allocations/deallocations for internal (!) objects (I'm not
talking about objects explicitly managed by the application). Many of them
have a very short time-to-live. I guess some of them could be recycled or
even optimized away.

E.g. global profiling indicates that tuple allocation/deallocation has a
noticeable effect on performance. There are probably other objects that
have prohibitive setup/teardown cost (but are less pronounced on allocation
costs).

But since this is only #4 on my list I have not given it a closer look.
That's why I said 'suggestions welcome'.

Bye,
     Mike

From martin at v.loewis.de  Sat Apr 17 18:18:33 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Apr 17 18:18:45 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <20040417215000.GA5263@mike.de>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>	<B8CDFB11BB44D411B8E600508BDF076C1E96D46A@USAHM010.amer.corp.eds.com>	<5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>	<20040416182605.GA3590@mike.de>
	<20040417140113.GB8542@panix.com> <20040417215000.GA5263@mike.de>
Message-ID: <4081AD39.8090404@v.loewis.de>

Mike Pall wrote:
> What I meant with 'less malloc-happy' is that we should try to reduce
> the number of allocations/deallocations for internal (!) objects (I'm not
> talking about objects explicitly managed by the application). Many of them
> have a very short time-to-live. I guess some of them could be recycled or
> even optimized away.
> 
> E.g. global profiling indicates that tuple allocation/deallocation has a
> noticeable effect on performance. There are probably other objects that
> have prohibitive setup/teardown cost (but are less pronounced on allocation
> costs).

The problem is that this has already been done in the past. Anybody
contributing to the Python core is certainly aware that avoiding memory
allocations, where possible, should be done, and indeed, special cases
have been added (e.g. my addition of METH_NONE and METH_O to avoid
tuple creation).

So unless a specific proposal is made, I'm doubtful that much
optimization is possible without breaking core semantic aspects
of the language.

Regards,
Martin


From aahz at pythoncraft.com  Sat Apr 17 20:53:56 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Apr 17 20:54:01 2004
Subject: [Python-Dev] Re: PEP 328 -- relative and multi-line import
In-Reply-To: <200404140624.i3E6OeMg017756@cosc353.cosc.canterbury.ac.nz>
References: <20040409154252.GB28808@panix.com>
	<200404140624.i3E6OeMg017756@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040418005356.GA27203@panix.com>

On Wed, Apr 14, 2004, Greg Ewing wrote:
> Aahz <aahz@pythoncraft.com>:
>> 
>> Nope.  Absolute imports are the most common form
> 
> I don't agree with that. In every major Python application I've
> written, the vast majority of imports are from other modules of the
> application, which are most sensibly referenced relatively.

So you're saying that your applications are all put together in a single
package?  Or that there isn't much in the way of inter-package imports?
Could you show us the header of one of your modules as an example?

> I, too, am quite concerned about the imminent breaking of every
> existing relative import, apparently because Guido has an ideological
> objection to relative imports and wants to punish us for having used
> them so freely over the years.

Actually, it's Barry's hobbyhorse.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From guido at python.org  Sat Apr 17 21:04:43 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Apr 17 21:05:07 2004
Subject: [Python-Dev] dict.sortedkeys()
In-Reply-To: Your message of "Sat, 17 Apr 2004 13:21:26 PDT."
	<DE1CF2B4FEC4A342BF62B6B2B334601E562014@opus.amorhq.net> 
References: <DE1CF2B4FEC4A342BF62B6B2B334601E562014@opus.amorhq.net> 
Message-ID: <200404180104.i3I14ha31011@guido.python.org>

> [Bob Ippolito]
> > I'm pretty sure that in Python 2.4 you will be able to say
> sorted(seq).
> 
> Excellent! That'll do nicely. :)

Even better, you'll be able to say sorted(iterable), and a dict is an
iterable (yielding its keys), so you can say sorted(dict).

$ ./python 
Python 2.4a0 (#93, Mar 31 2004, 10:08:22) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-113)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sorted(sys.modules)
['UserDict', '__builtin__', '__main__', '_codecs', '_sre', 'codecs', 'copy_reg', 'distutils', 'distutils.dep_util', 'distutils.distutils', 'distutils.errors', 'distutils.log', 'distutils.os', 'distutils.re', 'distutils.spawn', 'distutils.string', 'distutils.sys', 'distutils.util', 'encodings', 'encodings.aliases', 'encodings.codecs', 'encodings.exceptions', 'encodings.iso8859_15', 'encodings.types', 'exceptions', 'linecache', 'os', 'os.path', 'posix', 'posixpath', 're', 'readline', 'rlcompleter', 'signal', 'site', 'sre', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'string', 'sys', 'types', 'warnings', 'zipimport']
>>> 

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Sun Apr 18 05:23:15 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Apr 18 05:23:41 2004
Subject: [Python-Dev] Optimization targets - refcount
In-Reply-To: <5.1.1.6.0.20040416083351.02166020@mail.telecommunity.com>
Message-ID: <200404180923.i3I9NFiq025149@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> I'm not sure, however, that this would be common enough to be
> helpful.  It seems to me Py_INCREF should effectively translate to
> only a single machine instruction or two.

On a RISC architecture, probably 3 instructions -- load,
increment, store.

It's the memory accesses that will be the most expensive
part of this. But then, if increfing Py_None is really
that common, its refcount will likely be in cache.

I doubt whether it really is very common, though, compared
to increfs of all other objects.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun Apr 18 05:40:41 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Apr 18 05:40:49 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D0338389F@escpl.tcp.com.ar>
Message-ID: <200404180940.i3I9efOM025212@cosc353.cosc.canterbury.ac.nz>

"Batista, Facundo" <FBatista@uniFON.com.ar>:

> And positions is the number of digits after the decimal point at
> which you want to round the number you're creating from and discard
> the other digits.

Most people call that "decimal places".

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From john at neggie.net  Sun Apr 18 08:45:29 2004
From: john at neggie.net (John Belmonte)
Date: Sun Apr 18 08:45:52 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <200404171733.i3HHX4B30513@guido.python.org>
References: <407C28E0.7090808@neggie.net> <4080C0C7.7070602@neggie.net>
	<200404171733.i3HHX4B30513@guido.python.org>
Message-ID: <40827869.2030300@neggie.net>

Guido van Rossum wrote:
>>>    3) extend the __slots__ handler to set the descriptor's doc string 
>>>to dictionary values if the slot's value is a dictionary.
>>
>>I haven't implemented this, because I don't know a thing about Python's 
>>C interface.  If someone more able would knock off a prototype, I would 
>>be grateful.
> 
> 
> If I understand this proposal correctly (the dict values would provide
> the docstrings for the slots) I'd rather not make this default
> behavior; different metaclasses may want to use the dict value for
> different purposes, and subsuming it by default for the doc string
> would make that harder to do.

Ok.  Can anyone suggest an unobtrusive way to set docstrings on slots?

Perhaps this (but how could it be implemented?):

     class Foo(object):
         __slots__ = docslots(
             ('slot1': 'description'),
             ('slot2':
                 """description
                 ...continued
                 """),
             'slot3',
         )


-John


-- 
http:// if  ile.org/

From fdrake at acm.org  Sun Apr 18 10:13:19 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun Apr 18 10:13:38 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <40827869.2030300@neggie.net>
References: <407C28E0.7090808@neggie.net>
	<200404171733.i3HHX4B30513@guido.python.org>
	<40827869.2030300@neggie.net>
Message-ID: <200404181013.20045.fdrake@acm.org>

On Sunday 18 April 2004 08:45 am, John Belmonte wrote:
 > Perhaps this (but how could it be implemented?):
 >
 >      class Foo(object):
 >          __slots__ = docslots(
 >              ('slot1': 'description'),
 >              ('slot2':

How about:

    class Foo(object):
       docslots(slot1='description',
                slot2=("Some really long\n"
                       "multi-line description."),
                ...)

The implementation for docslots() would either provide or update __slots__, 
and defining properities for slots with munged names.  This wouldn't actually 
be difficult to implement, though is would require a sys._getframe() hack.  
In fact, you could crib from the attached sample.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: docslots.py
Type: text/x-python
Size: 1169 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040418/6aaee156/docslots.py
From skip at pobox.com  Sun Apr 18 08:53:19 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sun Apr 18 11:34:40 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <200404171646.26885.fdrake@acm.org>
References: <16511.65327.705618.504369@montanaro.dyndns.org>
	<16512.8565.21289.392919@montanaro.dyndns.org>
	<16512.20616.41799.327583@montanaro.dyndns.org>
	<200404171646.26885.fdrake@acm.org>
Message-ID: <16514.31295.926740.594739@montanaro.dyndns.org>


    Fred> Hint: the pserver puts the username used to authenticate in the
    Fred> CVS_USER environment variable when running scripts from the
    >> admin Fred> files (like loginfo).
    >> 
    >> This worked perfectly.

    Fred> This is something syncmail should probably support out of the box.
    Fred> If you'd like to submit a patch, we'd love to have it:

    Fred>     http://sourceforge.net/projects/cvs-syncmail/

Will do when I'm back at work Monday.  I added a new command line flag to
enable CVS_USER usage.  Were you thinking that should be the default?  If
so, I could delete about 75% of my changes.

Skip

From john at neggie.net  Sun Apr 18 11:35:50 2004
From: john at neggie.net (John Belmonte)
Date: Sun Apr 18 11:36:17 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <200404181013.20045.fdrake@acm.org>
References: <407C28E0.7090808@neggie.net>
	<200404171733.i3HHX4B30513@guido.python.org>
	<40827869.2030300@neggie.net> <200404181013.20045.fdrake@acm.org>
Message-ID: <4082A056.8030804@neggie.net>

Fred L. Drake, Jr. wrote:
> """Sample implementation of the docslots idea."""
> 
> import sys
> 
> 
> class slotproperty(object):
>     def __init__(self, name, docstring):
>         self.__name__ = name
>         self.__doc__ = docstring
>         self._slotname = "_slotproperty_" + name
> 
>     def __get__(self, inst, cls=None):
>         return getattr(inst, self._slotname)
> 
>     def __set__(self, inst, value):
>         setattr(inst, self._slotname, value)
> 
> 
> def docslots(**kw):
>     namespace = sys._getframe(1).f_locals
>     __slots__ = namespace.get("__slots__", ())
>     __slots__ = list(__slots__)
>     for name, docstring in kw.iteritems():
>         prop = slotproperty(name, docstring)
>         if name in __slots__:
>             __slots__.remove(name)
>         __slots__.append(prop._slotname)
>         namespace[name] = prop
>     namespace["__slots__"] = tuple(__slots__)

Thanks, that at least gives me a framework to try out some ideas.

I'm concerned about something though.  Doesn't this implementation 
impose an overhead on access of slots with doc strings, or can a C 
implementation be made just as efficient as normal slots?

I'm also wondering about Guido's comment.  Even if the __slots__ handler 
were extended to handle docstrings directly via dict values, wouldn't 
metaclasses still be free to intercept the dict for other uses?

-John


-- 
http:// if  ile.org/

From fdrake at acm.org  Sun Apr 18 11:55:30 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun Apr 18 11:55:38 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <4082A056.8030804@neggie.net>
References: <407C28E0.7090808@neggie.net> <200404181013.20045.fdrake@acm.org>
	<4082A056.8030804@neggie.net>
Message-ID: <200404181155.30957.fdrake@acm.org>

On Sunday 18 April 2004 11:35 am, John Belmonte wrote:
 > I'm concerned about something though.  Doesn't this implementation
 > impose an overhead on access of slots with doc strings, or can a C
 > implementation be made just as efficient as normal slots?

This implementation does, but a C implementation in the core could avoid that 
easily enough.  It may even be that this could be avoided using a Python 
implementation by generating an __slots__ that isn't munged so heavily and 
then playing metaclass tricks to cause the docstrings to be inserted directly 
on the descriptors generated by the current __slots__ machinery.

I wasn't too worried about that for the example code, but you're certainly 
free to construct something that suits you.  I think it would be more 
interesting to create slot-based properties without the renaming currently 
needed, but I'm not sure how to do that off-hand.  If the slot property type 
were available for subclassing it would be pretty easy, I'd hope.

 > I'm also wondering about Guido's comment.  Even if the __slots__ handler
 > were extended to handle docstrings directly via dict values, wouldn't
 > metaclasses still be free to intercept the dict for other uses?

Metaclasses can do what they want to affect the new type.  That's what they're 
for.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From pf_moore at yahoo.co.uk  Sun Apr 18 12:13:56 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Apr 18 12:14:31 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
References: <E1BExVA-0004Be-3Z@mail.python.org>
Message-ID: <n059mfgb.fsf@yahoo.co.uk>

"Tim Peters" <tim.one@comcast.net> writes:

> Thanks to David Abrahams for passing on this info:  a free optimizing
> cmdline VC 7.1 has been released by MS:
>
>     http://msdn.microsoft.com/visualc/vctoolkit2003/
>
> The 32MB download appears to contain much of the meat of Visual Studio .NET
> 2003 Professional, including the compiler, linker, static-link libraries for
> the C, C++ & STL runtimes, and the .NET Common Language Runtime.

Sadly, a test seems to show that the /MD option links with msvcrt.dll,
and not with msvcr71.dll. So this still isn't of immediate use in
building Python extensions which will run against a standard Python
binary distribution.

This would tie in with the information given on the MS website: "C
Runtime Library and the C++ Standard Library, including the Standard
Template Library. These are the same static-link libraries included
with Visual Studio."

Note this bit........................^^^^^^^^^^^

I just checked my hard disk (I have MSVC6SP6, a recent platform SDK,
.NET framework and SDK, and the VC Toolkit 2003 installed) and I can't
find any occurrences of msvcr71.lib, implying that none of these tools
offers the possibility of creating code which links against
msvcr71.dll...

Another oddity - apparently a cast from float to long generates a call
to an internal function _ftol2, where it called _ftol in earlier
versions. So code that links with /MD fails to link if it contains
such a cast (as the compiler generates code to call support functions
which are not in the CRT DLL that is being used...)

Don't get me wrong - it's great that MS have done this, it's just not
the end of the story.

Paul.
-- 
This signature intentionally left blank


From guido at python.org  Sun Apr 18 13:21:29 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Apr 18 13:21:36 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: Your message of "Sun, 18 Apr 2004 08:45:29 EDT."
	<40827869.2030300@neggie.net> 
References: <407C28E0.7090808@neggie.net> <4080C0C7.7070602@neggie.net>
	<200404171733.i3HHX4B30513@guido.python.org> 
	<40827869.2030300@neggie.net> 
Message-ID: <200404181721.i3IHLTp07934@guido.python.org>

> Ok.  Can anyone suggest an unobtrusive way to set docstrings on slots?
> 
> Perhaps this (but how could it be implemented?):
> 
>      class Foo(object):
>          __slots__ = docslots(
>              ('slot1': 'description'),
>              ('slot2':
>                  """description
>                  ...continued
>                  """),
>              'slot3',
>          )

Anything can be done using metaclasses.  __slots__ is not special
once the class exists -- it is a set of instructions for the default
metaclass to create a specific set of descriptors (and associated
storage).  Another metaclass could use a different convention
(although it may have to set __slots__ to let the base metaclass
create the associated storage slots).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg at cosc.canterbury.ac.nz  Sun Apr 18 20:09:55 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Apr 18 20:10:44 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <n059mfgb.fsf@yahoo.co.uk>
Message-ID: <200404190009.i3J09tfN026216@cosc353.cosc.canterbury.ac.nz>

Paul Moore <pf_moore@yahoo.co.uk>:

> Sadly, a test seems to show that the /MD option links with msvcrt.dll,
> and not with msvcr71.dll. So this still isn't of immediate use in
> building Python extensions which will run against a standard Python
> binary distribution.

Can I just ask at this point whether it's possible to
use MinGW to compile extensions for the Python version
under discussion here?

If not, I think you're making a big mistake basing the
standard distribution on this compiler.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From python-dev-list at cstork.org  Sun Apr 18 21:53:48 2004
From: python-dev-list at cstork.org (Christian Stork)
Date: Sun Apr 18 21:53:52 2004
Subject: [Python-Dev] iter.index()
Message-ID: <20040419015348.GB6807@ics.uci.edu>

Hi,

I wanted to do something like

    i = itertools.chain(list1, list2).index(elem)

but that gave me 

    ...
    AttributeError: 'itertools.chain' object has no attribute 'index'

If I use the operator module it works just fine.

    i = operator.indexOf(itertools.chain(list1, list2), elem)

Why not add the index method to iterator objects?  Of course, internally
only next() is used by the default implementation.

-- 
Chris Stork   <>  Support eff.org!  <>   http://www.ics.uci.edu/~cstork/
OpenPGP fingerprint:  B08B 602C C806 C492 D069  021E 41F3 8C8D 50F9 CA2F

From bob at redivi.com  Sun Apr 18 22:08:25 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Apr 18 22:04:12 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <20040419015348.GB6807@ics.uci.edu>
References: <20040419015348.GB6807@ics.uci.edu>
Message-ID: <70D3F6CF-91A6-11D8-A48C-000A95686CD8@redivi.com>


On Apr 18, 2004, at 9:53 PM, Christian Stork wrote:

> Hi,
>
> I wanted to do something like
>
>     i = itertools.chain(list1, list2).index(elem)
>
> but that gave me
>
>     ...
>     AttributeError: 'itertools.chain' object has no attribute 'index'
>
> If I use the operator module it works just fine.
>
>     i = operator.indexOf(itertools.chain(list1, list2), elem)
>
> Why not add the index method to iterator objects?  Of course, 
> internally
> only next() is used by the default implementation.

An iterator mutates each time you call its next().  Your call to 
indexOf does one of three things: exhaust some of your iterator and 
return a useless integer, exhaust all of your iterator and raise an 
exception, or never return.

If you want an object that acts like a list, you should use a list.

-bob


From python-dev-list at cstork.org  Sun Apr 18 22:21:07 2004
From: python-dev-list at cstork.org (Christian Stork)
Date: Sun Apr 18 22:21:12 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <70D3F6CF-91A6-11D8-A48C-000A95686CD8@redivi.com>
References: <20040419015348.GB6807@ics.uci.edu>
	<70D3F6CF-91A6-11D8-A48C-000A95686CD8@redivi.com>
Message-ID: <20040419022107.GC6807@ics.uci.edu>

On Sun, Apr 18, 2004 at 10:08:25PM -0400, Bob Ippolito wrote:
> 
> On Apr 18, 2004, at 9:53 PM, Christian Stork wrote:
> 
> >Hi,
> >
> >I wanted to do something like
> >
> >    i = itertools.chain(list1, list2).index(elem)
> >
> >but that gave me
> >
> >    ...
> >    AttributeError: 'itertools.chain' object has no attribute 'index'
> >
> >If I use the operator module it works just fine.
> >
> >    i = operator.indexOf(itertools.chain(list1, list2), elem)
> >
> >Why not add the index method to iterator objects?  Of course, 
> >internally
> >only next() is used by the default implementation.
> 
> An iterator mutates each time you call its next().  Your call to 
> indexOf does one of three things: exhaust some of your iterator and 
> return a useless integer, 

Why useless?  It returns the index.  Why do I have to build a new
concatinated list (or my own special index function) in order to get
that information?

> exhaust all of your iterator and raise an exception, 

Yep, just as it would for lists.

> or never return.

Well, that's the risk we take with iterators, but in my case I'm certain
of two sufficient termination criteria: the iterator is finite and I
know that elem is in it.

> If you want an object that acts like a list, you should use a list.

I don't see anything inherently "listy" about index().  It just counts
how many elements there are to reach elem.  And obviously the
functionality is already there in the operator module.  I'm just
proposing a little convenience.

-- 
Chris Stork   <>  Support eff.org!  <>   http://www.ics.uci.edu/~cstork/
OpenPGP fingerprint:  B08B 602C C806 C492 D069  021E 41F3 8C8D 50F9 CA2F

From john at neggie.net  Sun Apr 18 22:29:08 2004
From: john at neggie.net (John Belmonte)
Date: Sun Apr 18 22:29:21 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <200404181721.i3IHLTp07934@guido.python.org>
References: <407C28E0.7090808@neggie.net> <4080C0C7.7070602@neggie.net>
	<200404171733.i3HHX4B30513@guido.python.org>
	<40827869.2030300@neggie.net>
	<200404181721.i3IHLTp07934@guido.python.org>
Message-ID: <40833974.10800@neggie.net>

Guido van Rossum wrote:
> Anything can be done using metaclasses.  __slots__ is not special
> once the class exists -- it is a set of instructions for the default
> metaclass to create a specific set of descriptors (and associated
> storage).  Another metaclass could use a different convention
> (although it may have to set __slots__ to let the base metaclass
> create the associated storage slots).

My original proposal was to use __slots__ dict values for docstrings in 
the default metaclass.  You said you'd rather not do that because 
different metaclasses may want to use the dict value for different 
purposes.  But from what you've explained, metaclasses are free to 
interpret the value of __slots__ in any way they choose.  Metaclasses 
built on top of the default metaclass could translate their __slots__ 
value to the __slots__ I proposed.

Are optional tuples any better?  This wouldn't preclude use of dict 
values for something else.

class Foo(object):
     __slots__ = [
         'slot1',
         ('slot2', 'description'),
         ('slot3', """description
             ...continued"""),
     ]


-John


-- 
http:// if  ile.org/

From python at rcn.com  Sun Apr 18 22:37:28 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun Apr 18 22:38:35 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <20040419015348.GB6807@ics.uci.edu>
Message-ID: <000b01c425b7$42592d80$e841fea9@oemcomputer>

[Christian Stork]
> I wanted to do something like
> 
>     i = itertools.chain(list1, list2).index(elem)

I'm curious about what application needed to do this.


> If I use the operator module it works just fine.
> 
>     i = operator.indexOf(itertools.chain(list1, list2), elem)

Nice solution.  It is general purpose, self-documenting, and efficient.



Raymond Hettinger


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From pje at telecommunity.com  Sun Apr 18 22:44:11 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Apr 18 22:43:53 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <200404190009.i3J09tfN026216@cosc353.cosc.canterbury.ac.nz>
References: <n059mfgb.fsf@yahoo.co.uk>
Message-ID: <5.1.1.6.0.20040418224118.0219d470@mail.telecommunity.com>

At 12:09 PM 4/19/04 +1200, Greg Ewing wrote:
>Paul Moore <pf_moore@yahoo.co.uk>:
>
> > Sadly, a test seems to show that the /MD option links with msvcrt.dll,
> > and not with msvcr71.dll. So this still isn't of immediate use in
> > building Python extensions which will run against a standard Python
> > binary distribution.
>
>Can I just ask at this point whether it's possible to
>use MinGW to compile extensions for the Python version
>under discussion here?

Yes.  See:

http://mail.python.org/pipermail/python-dev/2004-January/041650.html



From bob at redivi.com  Sun Apr 18 22:50:13 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Apr 18 22:45:54 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <20040419022107.GC6807@ics.uci.edu>
References: <20040419015348.GB6807@ics.uci.edu>
	<70D3F6CF-91A6-11D8-A48C-000A95686CD8@redivi.com>
	<20040419022107.GC6807@ics.uci.edu>
Message-ID: <482D62ED-91AC-11D8-A48C-000A95686CD8@redivi.com>

On Apr 18, 2004, at 10:21 PM, Christian Stork wrote:

> On Sun, Apr 18, 2004 at 10:08:25PM -0400, Bob Ippolito wrote:
>>
>> On Apr 18, 2004, at 9:53 PM, Christian Stork wrote:
>>
>>> Hi,
>>>
>>> I wanted to do something like
>>>
>>>    i = itertools.chain(list1, list2).index(elem)
>>>
>>> but that gave me
>>>
>>>    ...
>>>    AttributeError: 'itertools.chain' object has no attribute 'index'
>>>
>>> If I use the operator module it works just fine.
>>>
>>>    i = operator.indexOf(itertools.chain(list1, list2), elem)
>>>
>>> Why not add the index method to iterator objects?  Of course,
>>> internally
>>> only next() is used by the default implementation.
>>
>> An iterator mutates each time you call its next().  Your call to
>> indexOf does one of three things: exhaust some of your iterator and
>> return a useless integer,
>
> Why useless?  It returns the index.  Why do I have to build a new
> concatinated list (or my own special index function) in order to get
> that information?

It returns an index into some sequence that no longer exists.  What 
good is that?

>> exhaust all of your iterator and raise an exception,
>
> Yep, just as it would for lists.

The difference is that the iterator's life is more or less over, 
there's probably nothing useful you can do with it, but a list is not 
changed by this operation.

>> If you want an object that acts like a list, you should use a list.
>
> I don't see anything inherently "listy" about index().  It just counts
> how many elements there are to reach elem.  And obviously the
> functionality is already there in the operator module.  I'm just
> proposing a little convenience.

I think it's pretty rare that you would want to know this information 
at the cost of exhausting some/all of your iterator... and if that 
really is what you want, then you should just use operator.indexOf.  
There are MANY iterable types, it's not reasonable to change them all.

-bob


From tim.one at comcast.net  Sun Apr 18 22:51:12 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sun Apr 18 22:51:21 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <200404190009.i3J09tfN026216@cosc353.cosc.canterbury.ac.nz>
Message-ID: <E1BFOsQ-00031q-JK@mail.python.org>

[Greg Ewing]
> Can I just ask at this point whether it's possible to
> use MinGW to compile extensions for the Python version
> under discussion here?

Don't know -- someone who uses MinGW would need to answer that.

> If not, I think you're making a big mistake basing the
> standard distribution on this compiler.

Why is that?  If it doesn't play nice with the latest MSVC, MinGW doesn't
have a future on Windows.

The precompiled Python Windows distro has always been built with the flavor
of MSVC current at the time.  Nothing has changed in that respect, except
the specific MSVC flavor in use -- and that MS has made some versions of
this flavor available without cost.



From python-dev-list at cstork.org  Sun Apr 18 23:12:47 2004
From: python-dev-list at cstork.org ('Christian Stork')
Date: Sun Apr 18 23:12:52 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <000b01c425b7$42592d80$e841fea9@oemcomputer>
References: <20040419015348.GB6807@ics.uci.edu>
	<000b01c425b7$42592d80$e841fea9@oemcomputer>
Message-ID: <20040419031247.GA7895@ics.uci.edu>

On Sun, Apr 18, 2004 at 10:37:28PM -0400, Raymond Hettinger wrote:
> [Christian Stork]
> > I wanted to do something like
> > 
> >     i = itertools.chain(list1, list2).index(elem)
> 
> I'm curious about what application needed to do this.
 
It's for a simulation of a peer-to-peer algorithm I'm working on.  The
lists are lists of peers and could potentially become _very_ large.
Therefore concatenation might not be cheap.
 
> > If I use the operator module it works just fine.
> > 
> >     i = operator.indexOf(itertools.chain(list1, list2), elem)
> 
> Nice solution.  It is general purpose, self-documenting, and efficient.

Thanks. :-)

-- 
Chris Stork   <>  Support eff.org!  <>   http://www.ics.uci.edu/~cstork/
OpenPGP fingerprint:  B08B 602C C806 C492 D069  021E 41F3 8C8D 50F9 CA2F

From john at neggie.net  Sun Apr 18 23:16:17 2004
From: john at neggie.net (John Belmonte)
Date: Sun Apr 18 23:16:47 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <200404181013.20045.fdrake@acm.org>
References: <407C28E0.7090808@neggie.net>	<200404171733.i3HHX4B30513@guido.python.org>	<40827869.2030300@neggie.net>
	<200404181013.20045.fdrake@acm.org>
Message-ID: <40834481.7000002@neggie.net>

Fred L. Drake, Jr. wrote:
> How about:
> 
>     class Foo(object):
>        docslots(slot1='description',
>                 slot2=("Some really long\n"
>                        "multi-line description."),
>                 ...)

My intention was to make something that can completely replace the 
current __slots__ pattern.  Your proposal doesn't support slots that 
don't have a docstring very naturally (you'd use slot='').

Attached is another strategy that uses optional tuples.  Combined with 
my pydoc patch,

     class Foo(object):
         slots(
             ('slot1', 'description'),
             ('slot2', """description
                 ...continued"""),
             'slot3',
             ('_slot4', 'hidden'),
         )

yields this help:

     class Foo(__builtin__.object)
      |  Data descriptors defined here:
      |
      |  slot1
      |      description
      |  slot2
      |      description
      |      ...continued
      |  slot3


-John


-- 
http:// if  ile.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: slots.py
Type: text/x-python
Size: 2050 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040418/d4400ae9/slots.py
From python-dev-list at cstork.org  Sun Apr 18 23:26:32 2004
From: python-dev-list at cstork.org (Christian Stork)
Date: Sun Apr 18 23:26:36 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <482D62ED-91AC-11D8-A48C-000A95686CD8@redivi.com>
References: <20040419015348.GB6807@ics.uci.edu>
	<70D3F6CF-91A6-11D8-A48C-000A95686CD8@redivi.com>
	<20040419022107.GC6807@ics.uci.edu>
	<482D62ED-91AC-11D8-A48C-000A95686CD8@redivi.com>
Message-ID: <20040419032632.GB7895@ics.uci.edu>

On Sun, Apr 18, 2004 at 10:50:13PM -0400, Bob Ippolito wrote:
...
> >Why useless?  It returns the index.  Why do I have to build a new
> >concatinated list (or my own special index function) in order to get
> >that information?
> 
> It returns an index into some sequence that no longer exists.  What 
> good is that?

In my special case, it is essentially a ranking.  I don't need to access
the lists via the index.

> >>exhaust all of your iterator and raise an exception,
> >
> >Yep, just as it would for lists.
> 
> The difference is that the iterator's life is more or less over, 
> there's probably nothing useful you can do with it, but a list is not 
> changed by this operation.

Note that the mathematical function 

    index: Elems -> Integer

has a right to live on its own with the application of list accesses.
;-)

> >>If you want an object that acts like a list, you should use a list.
> >
> >I don't see anything inherently "listy" about index().  It just counts
> >how many elements there are to reach elem.  And obviously the
> >functionality is already there in the operator module.  I'm just
> >proposing a little convenience.
> 
> I think it's pretty rare that you would want to know this information 
> at the cost of exhausting some/all of your iterator... and if that 
> really is what you want, then you should just use operator.indexOf.  
> There are MANY iterable types, it's not reasonable to change them all.

If that is the consensus, so be it.  :-)  I'm just proposing it as it's
in the spirit of itertools.  It also eliminates an asymmetry between lists
and iterators as demonstrated by operator.index() which is supposed to
implement the intrinsic operators.  Admittedly, it might not be very
common.

-- 
Chris Stork   <>  Support eff.org!  <>   http://www.ics.uci.edu/~cstork/
OpenPGP fingerprint:  B08B 602C C806 C492 D069  021E 41F3 8C8D 50F9 CA2F

From aahz at pythoncraft.com  Sun Apr 18 23:52:15 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun Apr 18 23:52:19 2004
Subject: [Python-Dev] iter.index()
In-Reply-To: <20040419015348.GB6807@ics.uci.edu>
References: <20040419015348.GB6807@ics.uci.edu>
Message-ID: <20040419035215.GA7157@panix.com>

On Sun, Apr 18, 2004, Christian Stork wrote:
>
> Why not add the index method to iterator objects?  Of course, internally
> only next() is used by the default implementation.

While not entirely off-topic for python-dev, you'll probably get better
discussion by going to c.l.py.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From greg at cosc.canterbury.ac.nz  Mon Apr 19 00:19:44 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr 19 00:22:31 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <E1BFOsQ-00031q-JK@mail.python.org>
Message-ID: <200404190419.i3J4Ji2H026578@cosc353.cosc.canterbury.ac.nz>

> > If not, I think you're making a big mistake basing the
> > standard distribution on this compiler.
> 
> Why is that?  If it doesn't play nice with the latest MSVC, MinGW doesn't
> have a future on Windows.

I got scared by an earlier message that said something like "there is
currently no free compiler that will generate extension modules for
Python 2.4".

Fortunately it seems that meant no free *Microsoft* compiler, and
doesn't include gcc.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From fdrake at acm.org  Mon Apr 19 00:29:57 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Apr 19 00:30:09 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <40834481.7000002@neggie.net>
References: <407C28E0.7090808@neggie.net> <200404181013.20045.fdrake@acm.org>
	<40834481.7000002@neggie.net>
Message-ID: <200404190029.57021.fdrake@acm.org>

On Sunday 18 April 2004 11:16 pm, John Belmonte wrote:
 > My intention was to make something that can completely replace the
 > current __slots__ pattern.  Your proposal doesn't support slots that
 > don't have a docstring very naturally (you'd use slot='').

Please don't misunderstand; I wasn't making a proposal, but was just 
responding to your question about what can be done.  I've no doubt something 
better can be done, and if anything is done as part of stock Python, it would 
need to be.

That said, docless slots would not be handled any differently with my sample 
code than they are now; just name them in __slots__.  My docslots() function 
was very careful not to blindly overwrite an existing __slots__.  So this:

    class Sample(object):
        __slots__ = "slot1",
        docslots(slot2="description")

would work just fine.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From martin at v.loewis.de  Mon Apr 19 01:04:21 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Apr 19 01:04:29 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <200404190419.i3J4Ji2H026578@cosc353.cosc.canterbury.ac.nz>
References: <200404190419.i3J4Ji2H026578@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40835DD5.5050008@v.loewis.de>

Greg Ewing wrote:
> I got scared by an earlier message that said something like "there is
> currently no free compiler that will generate extension modules for
> Python 2.4".
> 
> Fortunately it seems that meant no free *Microsoft* compiler, and
> doesn't include gcc.

That statement would have been false in any case. It would
have talked only about the official binary distribution; if you
build Python yourself with a different compiler (say, Borland,
or cygwin), you can certainly also build extensions with that
compiler.

Regards,
Martin


From bac at OCF.Berkeley.EDU  Mon Apr 19 02:56:34 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Apr 19 02:56:42 2004
Subject: [Python-Dev] Possible modules that could use docs
Message-ID: <40837822.6060906@ocf.berkeley.edu>

Discovering the 'opcode' module thanks to the new PEP 329 caused me to 
wonder how many modules are undocumented and yet deserve to possibly be 
(I also was procrastinating from doing schoolwork and felt like doing 
some light coding =).

Below is a list of modules that might deserve some docs.  Any with 
numbers by them mean that a bug report has been filed to have them 
documented.

If any of the modules below should not be documented, please speak up 
(especially about the ones with SF bug reports since those could then be 
closed; but they seem like modules that should get some docs). 
Otherwise I would like to at least mention this list in the next 
python-dev Summary in hopes someone will want to step forward and 
document them.  I would also like to file a bug report for each one that 
doesn't have one yet so it at least gets recorded somewhere which 
modules still need docs beyond just the mailing list archives.

DocXMLRPCServer.py
ihooks.py
imputil.py 515751
modulefinder.py 914375
opcode.py
pickletools.py
smtpd.py 450803
symtable.py
toaiff.py
trace.py 779976 737734
clmodule.c
puremodule.c
symtablemodule.c 896052
timingmodule.c
zipimport.c
imp (built-in)

-Brett

From t-meyer at ihug.co.nz  Mon Apr 19 03:12:00 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Mon Apr 19 03:12:09 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <1ED4ECF91CDED24C8D012BCF2B034F1305E92F19@its-xchg4.massey.ac.nz>
Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F13026F2BAC@its-xchg4.massey.ac.nz>

> Otherwise I would like to at least mention this list in the next 
> python-dev Summary in hopes someone will want to step forward and 
> document them.

+1 to this, although also noting those that have a patch for documentation
already would be good (this includes smtpd.py/450803 and
zipimport.c/853800), so that people know that they should not bother with
those, or modify the existing patch.

You also missed one report :)

> zipimport.c 853800

=Tony Meyer


From Paul.Moore at atosorigin.com  Mon Apr 19 04:30:26 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Mon Apr 19 04:30:38 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060E63@UKDCX001.uk.int.atosorigin.com>

From: Tim Peters

>[Greg Ewing]
>> Can I just ask at this point whether it's possible to
>> use MinGW to compile extensions for the Python version
>> under discussion here?
>
> Don't know -- someone who uses MinGW would need to answer that.

Yes, it is. However, that's for your own extensions. Grabbing an
arbitrary 3rd party extension from the web and trying to compile
it yourself isn't always easy. (MSVC-isms in the code, use of
3rd party commercial libraries which don't support mingw, etc)

I don't know if anyone has stats on how many of the 3rd party
extension authors who currently provide Windows binaries have
access to MSVC7, and so can make the switch. It may be "all of
them". I don't have MSVC7, although I do have MSVC6 (my company,
or at least my group, never upgraded) but I've no idea how unusual
my situation is. A heads-up on c.l.p would probably be worth it,
but it's also likely to generate a huge amount of FUD, and anti-
Microsoft rants.

>> If not, I think you're making a big mistake basing the
>> standard distribution on this compiler.
>
> Why is that?  If it doesn't play nice with the latest MSVC, MinGW
> doesn't have a future on Windows.

Mingw is fine in this regard.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________

From aleaxit at yahoo.com  Mon Apr 19 05:20:25 2004
From: aleaxit at yahoo.com (Alex Martelli)
Date: Mon Apr 19 05:21:04 2004
Subject: [Python-Dev] dict.sortedkeys()
In-Reply-To: <DE1CF2B4FEC4A342BF62B6B2B334601E562014@opus.amorhq.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E562014@opus.amorhq.net>
Message-ID: <200404191120.25839.aleaxit@yahoo.com>

On Saturday 17 April 2004 10:21 pm, Robert Brewer wrote:
   ...
> Yes, of course. But I'm working on a large business app framework. That
> means either:
>
> 1. Writing sorted into every module where it's needed.
> 2. Putting sorted into a package somewhere, which makes it longer and
> uglier with absolute referencing.
> 3. Customizing consumers' Python install.
>
> ...none of which are good options.

Is sorted the ONLY "utility" function your LARGE business app framework needs 
in addition to what comes with Python 2.3?  That would really be utterly 
astounding to me.  In my experience, when coding a large framework I always 
end up needing somewhere between 0.5 and 2 "utility functions" per tens of 
thousands of SLOCs (standardized lines of code -- logical lines, ignoring 
comments, blank lines, docstrings, &c).  So, in a moderately large framework 
of 100,000 lines, I would expect to need somewhere between 5 and 20 utility 
functions (in addition to Python builtins, of course).  I never expect 
Python's built-ins to swell to accomodate _every_ such minute need, of 
course.  The only truly unacceptable alternative, in my opinion, is to repeat 
the coding for the utility functions more than once: as long as they're coded 
_once, and only once_, I'm a pretty happy camper.

When I was relatively new as a Pythonista, I used to deal with "the utiliities 
issue" by injecting names in __builtin__ during the start-up of my framework:

import __builtin__
import utils
for u in utils.__all__:
    setattr(__builtin__, u, getattr(utils, u))

or the like.  I have since decided that this is not at all a good idea -- as 
the number of utilities grows, there may be clashes with builtins, names 
being shadowed, etc, etc -- and the opacity of seeing a barename used
and having to guess where it comes from is particularly unacceptable.  These 
days, I far prefer to code, more simply:

import __builtin__
# get error msg in case of conflict
assert not hasattr(__builtin__, 'ut')
import utils as ut
__builtin__.ut = ut

and access the utilities from everywhere as ut.sorted, ut.scrambled, etc, etc.
ONE relatively mysterious barename is OK, it's always 'ut' or 'util' after 
all, and with nested packages etc it may be worth it to avoid the hassle of 
importing it repeatedly from different places in the hierarchy.  
Alternatively,
from mywondrousframework_toplevel import utils as ut
may be quite acceptable (even though this line _is_ needed in every module of 
the framework, I don't think it counts as "repetition" -- and it does 
simplify things if two frameworks are in use at the same time...).


> [Bob Ippolito]
>
> > I'm pretty sure that in Python 2.4 you will be able to say
>
> sorted(seq).
>
> Excellent! That'll do nicely. :)

Yes, for this one specific need, but surely there will be other tiny utilities 
you desire to have available.  Moreover, I don't think you can expect to 
release production applications relying on Python 2.4 anytime before this 
coming fall (optimistically, september -- but I wouldn't base my business on 
the assumption that this date can't possibly shift to, say, october...) -- 
and that's only if your customers are uniformly "early adopters" (for many 
corporations with a large installed base of Python code, it's _always_ a 
hassle to get them to upgrade their reference Python version, even though 
backwards compatibility IS targeted and a lot of effort is put into it...).

So, I think you should still consider what "utility-functions access strategy" 
is most appropriate for your needs.  The "trickle" of neat utility functions 
into Python's built-ins, by itself, will not entirely solve the problem for 
you, IMHO.


Alex


From raymond.hettinger at verizon.net  Mon Apr 19 08:54:41 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon Apr 19 08:55:55 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
Message-ID: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>

Comments are invited on a new pep:

	http://www.python.org/peps/pep-0329.html



Based on feedback received so far, there were several changes to the
original draft:

* The module is public

* The module name reflects its nature as a bytecode hack

* There is a flag to disable binding

* Added summaries of how it works, when it is called, and the effect on
startup time (near zero).

* Included a reference to Jython FAQs supporting the means for
automatically doing nothing in environments without bytecodes.


From FBatista at uniFON.com.ar  Mon Apr 19 09:09:36 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Apr 19 09:11:52 2004
Subject: [Python-Dev] Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838AE@escpl.tcp.com.ar>

[Greg Ewing]
#- > And positions is the number of digits after the decimal point at
#- > which you want to round the number you're creating from and discard
#- > the other digits.
#- 
#- Most people call that "decimal places".

Don't know enough english as to know which way most people call something,
and I thought that decimal places could get confused with decimal (the
module).

But, if you all think that "decimal places" is more clear than "positions",
I'll fix the PEP.

Thanks!

From FBatista at uniFON.com.ar  Mon Apr 19 09:11:33 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Apr 19 09:13:56 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838AF@escpl.tcp.com.ar>

[Kevin Jacobs]

#- Python decimals would do well to support the creation of 
#- instances with 
#- fixed scale
#- and precision parameters, since this information will be what is 
#- provided by databases,
#- other financial and engineering applications, and schema.  
#- i.e., these 
#- parameters override
#- the natural scale and precision found in the literal values 
#- used when 
#- constructing decimals.
#- e.g., hypothetically:
#- 
#-   Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
#-   Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')
#- 
#- Remember, these literals are frequently coming from an 
#- external source 
#- that must
#- be constrained to a given schema.  This issue doesn't come up with 
#- floating point
#- numbers nearly as often, because those schema use explicitly 
#- precision-qualified
#- types (i.e., single, double, quad precision).

I like it a lot, but not for Decimal.

Money maybe will get something like this.

.	Facundo

From FBatista at uniFON.com.ar  Mon Apr 19 09:22:18 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Apr 19 09:24:38 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838B0@escpl.tcp.com.ar>

[Aahz]

#- On Fri, Apr 16, 2004, Batista, Facundo wrote:
#- >
#- > We all must decide which of the following is better:
#- > 
#- >   a) Decimal(number)     # no context used
#- >      Decimal(number, context)
#- > 
#- >   b) Decimal(number)     # no context used
#- >      Decimal.using_context(number, context)
#- 
#- Based on my reading of Tim's posts, there's also
#- 
#-     c) Decimal(number, [context])   # context always used, with 
#-                                     # optional different from default


Got it. But I think there're issues:

  a) You can specifiy the context as an optional argument.

     Decimal(number, [context])

     You can pass context as an argument, and this context will
     be used at creation time. If you don't pass the context,
     ?no context is used or is used the "default" one? (with default
     I mean the context that it's on the thread at that time).


With another specific method, that's avoided:

  b)  You have to specify the context in other method.

      b.1) Decimal(number)

           This way you don't use the context at creation time.

      b.2) Decimal.using_context(number, [context])

           Always a context is used at creation time. If you specify
           the context, that's the one used.  If you don't, the
           "default" context is used. 


I think the (b) syntax is more clear and less error prone. If you all agree,
this can be the way to create from Decimal:

  - Lets you create with no context
  - Lets you create with default context
  - Lets you create with an specific context

The issue is how to mix this syntax with the from_float one. What do you
propose?

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040419/373101ac/attachment.html
From fdrake at acm.org  Mon Apr 19 09:28:23 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Apr 19 09:28:31 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <40837822.6060906@ocf.berkeley.edu>
References: <40837822.6060906@ocf.berkeley.edu>
Message-ID: <200404190928.23248.fdrake@acm.org>

On Monday 19 April 2004 02:56 am, Brett C. wrote:
 > pickletools.py

SF bug report 873146

 > imp (built-in)

This one is documented; check the module index.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From pje at telecommunity.com  Mon Apr 19 09:30:55 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr 19 09:30:45 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the Standard Library
In-Reply-To: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>
Message-ID: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>

At 08:54 AM 4/19/04 -0400, Raymond Hettinger wrote:
>Comments are invited on a new pep:
>
>         http://www.python.org/peps/pep-0329.html

I'm not sure I understand the reasoning for implementing this via a 
bytecode hack, rather than in the compiler or loading mechanisms.  I think 
I'd rather see something like:

    from __future__ import fast_globals

which would mean that globals and builtins could be considered constants 
unless declared with 'global' at the module level.  Then, the compiler 
could optimize any undeclared builtins, and the 'MAKE_FUNCTION' opcode 
could bind any constants defined as of the function's declaration.

Finally, the module object thus created would ban any __setattr__ on any 
constant that has been bound into a function.  (Since these are the only 
setattrs that could cause harm.)


From FBatista at uniFON.com.ar  Mon Apr 19 09:39:20 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Apr 19 09:41:38 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838B1@escpl.tcp.com.ar>

[Jewett, Jim J]

#- Facundo Batista:
#- > ... what is "exact conversion" to you:
#- 
#- >  1.1 -> "1.1"
#- >  1.1 -> "1.1000000000000001"
#- >  1.1 -> "1.100000000000000088817841970012523233890533447265625"
#- 
#- machine precision of floats, whatever that happens to be.  

That is not very portable...


#- That implementation-dependent part is one reason to use a
#- longer name, like exact_float.
#- 
#- But if I say 
#- 
#-   Decimal(1.1, precision=4)
#-   Decimal(1.1, positions=3)
#- 
#- Those are unambiguous.  If the context is 43, then later 
#- calculations will effectively fill in the extra digits;
#- I have just said to fill them in with zeros, rather than 
#- whatever the float approximations were.

And why you want to fill them with zeros?  I think this is a presentation
issue and doesn't got nothing to do about how to build the number internals.

I want my number to get showed as "1.100", but that is on Money, not on
Decimal.


#- Sure they are; the reason to specify positions is that the
#- underlying data wasn't really floating point -- it was an
#- integer which by convention is written in a larger unit.
#- Example with money:
#- 
#- 	Assume meat is 0.987USD/pound.  There are three
#- 	price digits to multiply by the weight, but the
#- 	final cost is rounded to an integer number of
#- 	pennies.
#- 
#- 	10 pounds cost 9.87, but
#- 	1 pound costs 0.99
#- 
#- I don't want to say that my prices have three digits of 
#- precision, or I'll keep fractions of a penny.  I don't
#- want to say that they have only two, or I'll drop the
#- pennies on expensive items.  I want to say that the
#- precision varies depending on the actual price.

I think that you should have all the math with three decimal places, and
then round to two when you have to charge your costumer (how you say "get
the money and put it inside the cash register" in english?).

But, this is an issue of Money, not Decimal.


#- But for any *specific* value, specifying either the total 
#- number of valid digits (signficant figures, precision) 
#- or the number of fractional digits (position) is enough
#- to determine both.

According to your words, sticking with the specification will do your
purpose. Why should we add another parameter to the ecuation? Decimal is
*now* enough complex.


#- > "number" can be string, int, etc., but NOT float.
#- > The issue with rounding at a defined position has 
#- > nothing to do with context.
#- 
#- I assume that a new Decimal would normally be created
#- with as much precision as the context would need for 
#- calculations.  By passing a context/precision/position,
#- the user is saying "yeah, but this measurement wasn't
#- that precise in the first place.  Use zeros for the
#- rest, no matter what this number claims."

I don't get to understand you, and I'm not sure if you have the right
concept. Several examples may help:

>>> getcontext().prec = 5
>>> Decimal(124)
Decimal( (0, (1, 2, 4), 0) )
>>> +Decimal(124)
Decimal( (0, (1, 2, 4), 0) )
>>> Decimal('258547.368')
Decimal( (0, (2, 5, 8, 5, 4, 7, 3, 6, 8), -3) )
>>> +Decimal('258547.368')
Decimal( (0, (2, 5, 8, 5, 5), 1L) )
>>> Decimal.from_float(1.1)
Decimal( (0, (1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 1,
7, 8, 4, 1, 9, 7, 0, 0, 1, 2, 5, 2, 3, 2, 3, 3, 8, 9, 0, 5, 3, 3, 4, 4, 7,
2, 6, 5, 6, 2, 5), -51L) )
>>> +Decimal.from_float(1.1)
Decimal( (0, (1, 1, 0, 0, 0), -4L) )

.	Facundo

From pedronis at bluewin.ch  Mon Apr 19 09:49:25 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon Apr 19 09:44:31 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the Standard Library
In-Reply-To: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>
Message-ID: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>

At 08:54 19.04.2004 -0400, Raymond Hettinger wrote:
>Comments are invited on a new pep:
>
>         http://www.python.org/peps/pep-0329.html
>
>
>
>Based on feedback received so far, there were several changes to the
>original draft:
>
>* The module is public
>
>* The module name reflects its nature as a bytecode hack
>
>* There is a flag to disable binding
>
>* Added summaries of how it works, when it is called, and the effect on
>startup time (near zero).
>
>* Included a reference to Jython FAQs supporting the means for
>automatically doing nothing in environments without bytecodes.

Raymond, could you  please try to tame your speed obsession without 
devising even more hacks
and pissing in other people's pool.

Thanks. 


From gmccaughan at synaptics-uk.com  Mon Apr 19 09:56:14 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Mon Apr 19 09:56:31 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
References: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
Message-ID: <200404191456.14865.gmccaughan@synaptics-uk.com>

On Monday 2004-04-19 14:49, Samuele Pedroni wrote:

> Raymond, could you  please try to tame your speed obsession without 
> devising even more hacks and pissing in other people's pool.

What a gratuitously obnoxious and pointless reply. If you have
reason to think Raymond's proposal, if implemented, will cause
trouble, then show it. If not, then why shouldn't he work on
making Python faster if he wants to?

The only drawback I can see to Raymond's proposal is that if
you change the values of builtins and expect that to influence
the behaviour of the standard library, then that may not work
any more. I'd be more inclined to call *that* "pissing in other
people's pools" than Raymond's proposal.

-- 
g



From aahz at pythoncraft.com  Mon Apr 19 10:11:44 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 19 10:11:50 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838B0@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838B0@escpl.tcp.com.ar>
Message-ID: <20040419141144.GB6494@panix.com>

On Mon, Apr 19, 2004, Batista, Facundo wrote:
> [Aahz]
> 
> #- Based on my reading of Tim's posts, there's also
> #- 
> #-     c) Decimal(number, [context])   # context always used, with 
> #-                                     # optional different from default
> 
> Got it. But I think there're issues:
> 
>   a) You can specifiy the context as an optional argument.
> 
>      Decimal(number, [context])
> 
>      You can pass context as an argument, and this context will
>      be used at creation time. If you don't pass the context,
>      ?no context is used or is used the "default" one? (with default
>      I mean the context that it's on the thread at that time).
> 
> With another specific method, that's avoided:
> 
>   b)  You have to specify the context in other method.
> 
>       b.1) Decimal(number)
> 
>            This way you don't use the context at creation time.
> 
>       b.2) Decimal.using_context(number, [context])
> 
>            Always a context is used at creation time. If you specify
>            the context, that's the one used.  If you don't, the
>            "default" context is used. 

I really don't care much about what decision gets made here; I just
wanted to make sure all options were clearly specified.

> The issue is how to mix this syntax with the from_float one. What do you
> propose?

This depends on what we choose as the mechanism for Decimal().  If the
default is contextless, then from_float() should also be contextless
with maximum precision.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From jack at performancedrivers.com  Mon Apr 19 10:15:28 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Mon Apr 19 10:15:33 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>
References: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>
Message-ID: <20040419141528.GA23507@performancedrivers.com>

On Mon, Apr 19, 2004 at 08:54:41AM -0400, Raymond Hettinger wrote:
> Comments are invited on a new pep:
> 
> 	http://www.python.org/peps/pep-0329.html
> 
> Based on feedback received so far, there were several changes to the
> original draft:
> 
> * The module is public
> 
> * The module name reflects its nature as a bytecode hack
> 
I dread saying it, but is 'pragma.py' a better name than 'codetweaks.py'?
They both rhyme with magic but pragma is more familiar (I think?).

-jackdied


From aahz at pythoncraft.com  Mon Apr 19 10:19:01 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 19 10:19:09 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <200404191456.14865.gmccaughan@synaptics-uk.com>
References: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
	<200404191456.14865.gmccaughan@synaptics-uk.com>
Message-ID: <20040419141901.GC6494@panix.com>

On Mon, Apr 19, 2004, Gareth McCaughan wrote:
> On Monday 2004-04-19 14:49, Samuele Pedroni wrote:
>> 
>> Raymond, could you  please try to tame your speed obsession without 
>> devising even more hacks and pissing in other people's pool.

Samuele, please remember that this is a "professional" environment and
phrase your posts appropriately.

> What a gratuitously obnoxious and pointless reply. If you have reason
> to think Raymond's proposal, if implemented, will cause trouble, then
> show it. If not, then why shouldn't he work on making Python faster if
> he wants to?

It's gratuitously insulting, but it's certainly not pointless in
context: remember that Samuele is responsible for maintaining Jython,
and I can certainly understand his bitterness at seeing a proposal that
mucks with Python internals like this.  I had a similar strong reaction
to Raymond's proposal even without Samuele's reasons.

Keep in mind that one of the goals of Python development has always been
to keep the implementation as simple as possible.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From aahz at pythoncraft.com  Mon Apr 19 10:20:44 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 19 10:20:49 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
References: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>
	<5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
Message-ID: <20040419142043.GD6494@panix.com>

On Mon, Apr 19, 2004, Phillip J. Eby wrote:
> 
> I'm not sure I understand the reasoning for implementing this via a 
> bytecode hack, rather than in the compiler or loading mechanisms.  I think 
> I'd rather see something like:
> 
>    from __future__ import fast_globals

+1
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From pje at telecommunity.com  Mon Apr 19 10:39:06 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr 19 10:38:47 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in 
	the Standard Library
In-Reply-To: <000201c42615$fb979300$e841fea9@oemcomputer>
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040419103223.02a51be0@mail.telecommunity.com>

At 09:55 AM 4/19/04 -0400, Raymond Hettinger wrote:
> > I'm not sure I understand the reasoning for implementing this via a
> > bytecode hack, rather than in the compiler or loading mechanisms.
>
>The reason is that it is not always possible to know (in general) what
>things constants.  Take a look at the ASPN recipe example, it ought to
>clarify for you how this works.

I don't need clarification about the mechanism; I independently implemented 
*exactly* this hack myself a couple of years ago.  (I just didn't propose 
it become part of standard Python.)  See 'bind_func()' in:

http://cvs.eby-sarna.com/PEAK/src/peak/util/Code.py?rev=1.1.2.1&content-type=text/vnd.viewcvs-markup



> >     from __future__ import fast_globals
>
>The __future__ mechanism is a transitional approach for things that
>would break existing code.

Right, and the slightly broader proposal I made *would* break existing 
code, but in a direction that Guido has said before he'd like to move.


>Also, __future__ statements go at the beginning of a script and the
>proposed bind_all goes at the end.

Which is a good argument for why the __future__ statement is a much better 
idea.  :)


> > which would mean that globals and builtins could be considered
>constants
> > unless declared with 'global' at the module level.  Then, the compiler
> > could optimize any undeclared builtins, and the 'MAKE_FUNCTION' opcode
> > could bind any constants defined as of the function's declaration.
> >
> > Finally, the module object thus created would ban any __setattr__ on
>any
> > constant that has been bound into a function.  (Since these are the
>only
> > setattrs that could cause harm.)
>
>
>IMO, this is making something hard out of something  simple.

No, it's trying to advance the *language*, rather than merely its 
implementation, and thus make it easier for Jython, IronPython, et al to cope.


>Try applying the working code in the PEP to a few modules.

Given that I've had this hack in my CVS repository for a few years now, you 
will probably realize that I already have.  :)


From pedronis at bluewin.ch  Mon Apr 19 10:45:37 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon Apr 19 10:40:52 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the Standard Library
In-Reply-To: <200404191456.14865.gmccaughan@synaptics-uk.com>
References: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
	<5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20040419161441.02d12688@pop.bluewin.ch>

At 13:56 19.04.2004 +0000, Gareth McCaughan wrote:
>On Monday 2004-04-19 14:49, Samuele Pedroni wrote:
>
> > Raymond, could you  please try to tame your speed obsession without
> > devising even more hacks and pissing in other people's pool.
>
>What a gratuitously obnoxious and pointless reply. If you have
>reason to think Raymond's proposal, if implemented, will cause
>trouble, then show it. If not, then why shouldn't he work on
>making Python faster if he wants to?

he's proposing an hack that would speed up CPython and slow down Jython
(because it would result in a no-op but OTOH he would like to remove the code
when some globals are bound to locals for speed reason. Of course that's 
not nice too
but replacing a hack with another is just sort of a win. Honestly there are 
probably
very convoluted ways to make this, in this form, a non no-op on Jython too, 
not that I would like to
implement them or that Raymond cared).

if he has an elegant proposal that works for all of Python that would be a 
proposal, this
seems just a hack.

[Not that I particularly like this kind of semi-overt mud slinging] Other 
people do not completely share Raymond's design taste, so to say, in fact 
they have politely responded to single proposals, OTOH although it was a 
bit over-rudely formulated, I think it was the case to bring to the day-light
that that is creating some real tension.

PS: I'm starting to be really annoyed by the fact that Jython is often 
cited good PR
but a 2nd class citizen for all the rest. I'm aware of the practical 
circumstances of why that's the case  in some occassions, OTOH we are 
mostly all investing our free time on Python activities, it would be nice 
if we showed respect beyond just the forms and politeness. I'm back from 
the ACCU which was fun but tiring and also an occasion of some related heat 
taking. All I was looking forward is restarting
to finish the new-style class work (started in December and January) in 
Jython,after the stop because of working on the negotations for the PyPy EU 
proposal together with the others PyPy people, which we hope will conclude 
positively for the people working on it and looking forward to restart 
making PyPy a real concrete contribution to Python future. 


From tim.one at comcast.net  Mon Apr 19 10:57:52 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon Apr 19 10:57:58 2004
Subject: [Python-Dev] Decimal data type issues
In-Reply-To: <407F6674.4070402@gradient.cis.upenn.edu>
Message-ID: <LNBBLJKPBEHFEDALKOLCOENPKFAB.tim.one@comcast.net>

[Edward Loper]
> ...
>    - I do believe that there will be use cases for exponents outside
>      this range.  Just because there's no physical quantities that are
>      reasonably measured with these numbers, doesn't mean that there
>      won't be abstract quantities that are reasonably measured with
>      huge numbers.

You don't have a real use case, Edward (if you did, you would have given it
by now <wink>).  YAGNI.  Note that GNU GMP's arbitrary-precision float types
also have bounds on exponent magnitude (but not on precision) -- this is
conventional.

>    - I don't believe that signaling an error when a number goes
>      outside this range will help catch any errors.  What type of
>      error are we expecting to catch here?

Overflow, and possibly underflow.  Typical when a correct iterative
algorithm is fed an input outside its radius of convergence, or an incorrect
iterative algorithm is fed anything.

>      If this is such a problem, then why doesn't long also have
>      a max/min?

For the same reason Decimal doesn't have a bound on precision:  exact
calculations can require any number of digits.  Exponents kick in when the
calculation becomes approximate (when precision is exceeded).


From guido at python.org  Mon Apr 19 11:10:59 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 19 11:11:44 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: Your message of "Sun, 18 Apr 2004 22:29:08 EDT."
	<40833974.10800@neggie.net> 
References: <407C28E0.7090808@neggie.net> <4080C0C7.7070602@neggie.net>
	<200404171733.i3HHX4B30513@guido.python.org>
	<40827869.2030300@neggie.net>
	<200404181721.i3IHLTp07934@guido.python.org> 
	<40833974.10800@neggie.net> 
Message-ID: <200404191510.i3JFAxX09635@guido.python.org>

> Guido van Rossum wrote:
> > Anything can be done using metaclasses.  __slots__ is not special
> > once the class exists -- it is a set of instructions for the default
> > metaclass to create a specific set of descriptors (and associated
> > storage).  Another metaclass could use a different convention
> > (although it may have to set __slots__ to let the base metaclass
> > create the associated storage slots).
> 
> My original proposal was to use __slots__ dict values for docstrings in 
> the default metaclass.  You said you'd rather not do that because 
> different metaclasses may want to use the dict value for different 
> purposes.  But from what you've explained, metaclasses are free to 
> interpret the value of __slots__ in any way they choose.  Metaclasses 
> built on top of the default metaclass could translate their __slots__ 
> value to the __slots__ I proposed.

Yes, but *if* the default metaclass assumed a dict contained only
docstrings, this would be the standard usage, and it would be
confusing (and sometimes incompatible) if a custom metaclass had a
different interpretation.  As long as the default metaclass doesn't
have any particular interpretation for the values in the dict, custom
metaclasses can do what they like.

> Are optional tuples any better?  This wouldn't preclude use of dict 
> values for something else.
> 
> class Foo(object):
>      __slots__ = [
>          'slot1',
>          ('slot2', 'description'),
>          ('slot3', """description
>              ...continued"""),
>      ]

But that currently doesn't work.  Tbe most future-proof solution would
be to put some kind of object in the dict values whose attributes can
guide various metaclasses.  Perhaps:

class slotprop(object):
    def __init__(self, **kwds):
        self.__dict__.update(kwds)

class C(object):
    __slots__ = {'slot1': slotprop(doc="this is the docstring"),
                 'slit2': slotprop('doc="another one")}

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 19 11:17:15 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 19 11:17:33 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: Your message of "Sun, 18 Apr 2004 23:56:34 PDT."
	<40837822.6060906@ocf.berkeley.edu> 
References: <40837822.6060906@ocf.berkeley.edu> 
Message-ID: <200404191517.i3JFHFa09689@guido.python.org>

> DocXMLRPCServer.py
> ihooks.py
> imputil.py 515751
> modulefinder.py 914375
> opcode.py
> pickletools.py
> smtpd.py 450803
> symtable.py
> toaiff.py
> trace.py 779976 737734
> clmodule.c
> puremodule.c
> symtablemodule.c 896052
> timingmodule.c
> zipimport.c
> imp (built-in)

I think we're trying to get the timing module to be deprecated.  The
others indeed ought to be documented IMO.  Maybe imp is documented in
the docs about import?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Mon Apr 19 11:23:34 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Apr 19 11:23:39 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <20040419142043.GD6494@panix.com>
References: <00cb01c4260d$80b9b120$e841fea9@oemcomputer>
	<5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<20040419142043.GD6494@panix.com>
Message-ID: <16515.61174.247974.888236@montanaro.dyndns.org>


Phillip wrote:

    > I'm not sure I understand the reasoning for implementing this via a
    > bytecode hack, rather than in the compiler or loading mechanisms.  

This sort of stuff is a lot easier to experiment with in Python.  If it
stabilizes and you figure out a way to decide whether to transform functions
automatically then sure, it belongs in the compiler or an optimizer of some
sort.

Skip

From guido at python.org  Mon Apr 19 11:25:26 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 19 11:27:27 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: Your message of "Mon, 19 Apr 2004 09:30:55 EDT."
	<5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com> 
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com> 
Message-ID: <200404191525.i3JFPW009731@guido.python.org>

> I'd rather see something like:
> 
>     from __future__ import fast_globals
> 
> which would mean that globals and builtins could be considered
> constants unless declared with 'global' at the module level.

Don't you think that this should be flagged with syntax that doesn't
permanently require the use of the word "future"?  And I think that
reusing the global statement at the global level is hardly the best
way to do this.

I do think that explicitly flagging "volatile" globals somehow might
be the right way to go eventually, but it should only be required for
those globals for which the compiler can't tell whether they may be
modified (i.e. anything that is assigned to more than once or or from
inside a loop or conditional or function is automatically volatile).

> Finally, the module object thus created would ban any __setattr__ on
> any constant that has been bound into a function.  (Since these are
> the only setattrs that could cause harm.)

Huh?  If the object's identity is constant (which is what we're
talking about) why should its *contents* be constant?  And who says
setattr is the only way to modify an object?  Or am I misunderstanding
what you're trying to say?  (I thought that "a constant bound into a
function" would be something like a global/builtin name binding.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Mon Apr 19 11:36:51 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 19 11:37:02 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <200404191517.i3JFHFa09689@guido.python.org>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
Message-ID: <1082389011.7365.123.camel@anthem.wooz.org>

On Mon, 2004-04-19 at 11:17, Guido van Rossum wrote:
> > DocXMLRPCServer.py
> > ihooks.py
> > imputil.py 515751
> > modulefinder.py 914375
> > opcode.py
> > pickletools.py
> > smtpd.py 450803
> > symtable.py
> > toaiff.py
> > trace.py 779976 737734
> > clmodule.c
> > puremodule.c
> > symtablemodule.c 896052
> > timingmodule.c
> > zipimport.c
> > imp (built-in)
> 
> I think we're trying to get the timing module to be deprecated.  The
> others indeed ought to be documented IMO.  Maybe imp is documented in
> the docs about import?

puremodule.c could probably be deprecated.  Does anybody use Purify
anymore -- and if so, have you ever used puremodule?  Does it still
compile?  Are the APIs it uses still up-to-date?

I haven't had a license for Purify in years.

-Barry



From guido at python.org  Mon Apr 19 11:36:53 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 19 11:37:07 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: Your message of "Mon, 19 Apr 2004 16:45:37 +0200."
	<5.2.1.1.0.20040419161441.02d12688@pop.bluewin.ch> 
References: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
	<5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch> 
	<5.2.1.1.0.20040419161441.02d12688@pop.bluewin.ch> 
Message-ID: <200404191536.i3JFasX09776@guido.python.org>

[Samuele]
> he's proposing an hack that would speed up CPython and slow down Jython
> (because it would result in a no-op but OTOH he would like to remove the code
> when some globals are bound to locals for speed reason. Of course that's 
> not nice too
> but replacing a hack with another is just sort of a win. Honestly there are 
> probably
> very convoluted ways to make this, in this form, a non no-op on Jython too, 
> not that I would like to
> implement them or that Raymond cared).
> 
> if he has an elegant proposal that works for all of Python that would be a 
> proposal, this
> seems just a hack.
> 
> [Not that I particularly like this kind of semi-overt mud slinging] Other 
> people do not completely share Raymond's design taste, so to say, in fact 
> they have politely responded to single proposals, OTOH although it was a 
> bit over-rudely formulated, I think it was the case to bring to the day-light
> that that is creating some real tension.

While Samuele's words were inappropriately rude, I'm glad this was
brought up (and even to some extent I'm glad about some of the words
that Raymond used).  Raymond *does* seem to have a "speed obsession",
and while I'm sure he has Python's best interests at heart, I'm not
sure that accepting everything he proposes will actually be good for
Python.

I am happy with tweaking of existing internals (like speeding up
append) and the addition of new interfaces built on iterators (I don't
quite share Armin's views of "iterators considered harmful") but I'm
worried that e.g. replacing heapq.py with C code is not useful (raise
your hand if you have used heapq.py -- now raise your hand if it was
too slow for your purpose -- I expect to see very few hands) and
introducing speed-up hacks that will invite many people with less
judgement to use them all the time and hence reduce the code quality
of Python code samples available to the world with useless
encrustations.

> PS: I'm starting to be really annoyed by the fact that Jython is
> often cited good PR but a 2nd class citizen for all the rest. I'm
> aware of the practical circumstances of why that's the case in some
> occassions, OTOH we are mostly all investing our free time on Python
> activities, it would be nice if we showed respect beyond just the
> forms and politeness. I'm back from the ACCU which was fun but
> tiring and also an occasion of some related heat taking. All I was
> looking forward is restarting to finish the new-style class work
> (started in December and January) in Jython,after the stop because
> of working on the negotations for the PyPy EU proposal together with
> the others PyPy people, which we hope will conclude positively for
> the people working on it and looking forward to restart making PyPy
> a real concrete contribution to Python future.

I hope you don't think *everybody* sees Jython as a 2nd class citizen.
I personally see it as very important.  Certainly the PSF is
considering funding it, whether via the general grants committee (just
formed) or via a special grant.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From neal at metaslash.com  Mon Apr 19 11:42:38 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Apr 19 11:42:46 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <1082389011.7365.123.camel@anthem.wooz.org>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
Message-ID: <20040419154238.GC22725@epoch.metaslash.com>

On Mon, Apr 19, 2004 at 11:36:51AM -0400, Barry Warsaw wrote:
>
> puremodule.c could probably be deprecated.  Does anybody use Purify
> anymore -- and if so, have you ever used puremodule?  Does it still
> compile?  Are the APIs it uses still up-to-date?

Although I did use Purify on python years ago, I never use it now.
Though, I never used the puremodule.  IMO, Purify is not useful
compared to valgrind.  +1 on deprecating puremodule.c.

Neal

From nbastin at opnet.com  Mon Apr 19 11:47:08 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Mon Apr 19 11:47:19 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <20040419154238.GC22725@epoch.metaslash.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
Message-ID: <D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>


On Apr 19, 2004, at 11:42 AM, Neal Norwitz wrote:

> On Mon, Apr 19, 2004 at 11:36:51AM -0400, Barry Warsaw wrote:
>>
>> puremodule.c could probably be deprecated.  Does anybody use Purify
>> anymore -- and if so, have you ever used puremodule?  Does it still
>> compile?  Are the APIs it uses still up-to-date?
>
> Although I did use Purify on python years ago, I never use it now.
> Though, I never used the puremodule.  IMO, Purify is not useful
> compared to valgrind.  +1 on deprecating puremodule.c.

I think it's useful to point out that Purify runs on a lot more 
platforms than valgrind does, so we should probably keep puremodule 
around, and update it if necessary.

--
Nick


From tcdelaney at optusnet.com.au  Mon Apr 19 11:56:41 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Mon Apr 19 11:56:48 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants inthe
	Standard Library
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
Message-ID: <004301c42626$e8634890$a100a8c0@ryoko>

Phillip J. Eby wrote:

> At 08:54 AM 4/19/04 -0400, Raymond Hettinger wrote:
>> Comments are invited on a new pep:
>>
>>         http://www.python.org/peps/pep-0329.html
>
> I'm not sure I understand the reasoning for implementing this via a
> bytecode hack, rather than in the compiler or loading mechanisms.  I
> think
> I'd rather see something like:
>
>     from __future__ import fast_globals


+1. The semantics would have to be that globals *may* be optimised when the
above is present (and code should assume that they will be) but there is no
guarantee i.e. code should work both in the presence and absence of the
above. This allows Jython (for example) to ignore it entirely, partially
implement it, or completely implement it as desired.

An additional bonus is that this puts it up front, so someone reading the
code knows that the author is expecting optimisations to occur - when it's
at the end it's much more likely to be missed, and also much less likely to
be taken into account when reading the code.

> which would mean that globals and builtins could be considered
> constants unless declared with 'global' at the module level.

This would be a backwards-incompatible change (and hence definitely
warranting the __future__) but I presume you mean considered constant within
the module that they are defined - hence essentially a non-issue.

> Then, the compiler
> could optimize any undeclared builtins, and the 'MAKE_FUNCTION' opcode
> could bind any constants defined as of the function's declaration.

I know it's been suggested before, but I believe the idea of module-level
locals - an indexed lookup rather than a dict lookup - might be worthwhile
considering as part of this. That would also give improved performance for
external modules using module-level names.

> Finally, the module object thus created would ban any __setattr__ on
> any constant that has been bound into a function.  (Since these are the
> only setattrs that could cause harm.)

It's late (2am), so I'm not entirely sure I understand this - do you mean
something like:

    class X:
        pass

    x = X()

    def func():
        print x

    x.a = 1

would throw an exception? Why would this cause a problem? The above would
become almost equivalent to:

    class X:
        pass

    x = X()

    def func():
        global x
        _x = x
        print _x

    x.a = 1

and I don't see a problem with that.

Tim Delaney




From neal at metaslash.com  Mon Apr 19 12:01:49 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Apr 19 12:01:56 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
Message-ID: <20040419160149.GD22725@epoch.metaslash.com>

On Mon, Apr 19, 2004 at 11:47:08AM -0400, Nick Bastin wrote:
> 
> On Apr 19, 2004, at 11:42 AM, Neal Norwitz wrote:
> 
> >On Mon, Apr 19, 2004 at 11:36:51AM -0400, Barry Warsaw wrote:
> >>
> >>puremodule.c could probably be deprecated.  Does anybody use Purify
> >>anymore -- and if so, have you ever used puremodule?  Does it still
> >>compile?  Are the APIs it uses still up-to-date?
> >
> >Although I did use Purify on python years ago, I never use it now.
> >Though, I never used the puremodule.  IMO, Purify is not useful
> >compared to valgrind.  +1 on deprecating puremodule.c.
> 
> I think it's useful to point out that Purify runs on a lot more 
> platforms than valgrind does, so we should probably keep puremodule 
> around, and update it if necessary.

Valgrind runs on x86 and PPC (still experimental).
IIRC, Purify "runs" on Solaris, HPUX, Windows (x86), and SGI?
(I tried to verify but the IBM site isn't responding.)
IMO that's not a lot.

Where Purify "running" is relative to any particular patch you have
installed on the system and if you can download a beta.  I stopped using
Purify because Rational was so awful to deal with.

Neal

From barry at python.org  Mon Apr 19 12:09:46 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 19 12:09:55 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <20040419160149.GD22725@epoch.metaslash.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
	<20040419160149.GD22725@epoch.metaslash.com>
Message-ID: <1082390985.7365.155.camel@anthem.wooz.org>

On Mon, 2004-04-19 at 12:01, Neal Norwitz wrote:
> On Mon, Apr 19, 2004 at 11:47:08AM -0400, Nick Bastin wrote:
> > 
> > On Apr 19, 2004, at 11:42 AM, Neal Norwitz wrote:
> > 
> > >On Mon, Apr 19, 2004 at 11:36:51AM -0400, Barry Warsaw wrote:
> > >>
> > >>puremodule.c could probably be deprecated.  Does anybody use Purify
> > >>anymore -- and if so, have you ever used puremodule?  Does it still
> > >>compile?  Are the APIs it uses still up-to-date?
> > >
> > >Although I did use Purify on python years ago, I never use it now.
> > >Though, I never used the puremodule.  IMO, Purify is not useful
> > >compared to valgrind.  +1 on deprecating puremodule.c.
> > 
> > I think it's useful to point out that Purify runs on a lot more 
> > platforms than valgrind does, so we should probably keep puremodule 
> > around, and update it if necessary.
> 
> Valgrind runs on x86 and PPC (still experimental).
> IIRC, Purify "runs" on Solaris, HPUX, Windows (x86), and SGI?
> (I tried to verify but the IBM site isn't responding.)
> IMO that's not a lot.

I was able to get to this page:

http://www-306.ibm.com/software/awdtools/purifyplus/unix/

and it looks like the suite of tools runs on "Linux" now too.

> Where Purify "running" is relative to any particular patch you have
> installed on the system and if you can download a beta.  I stopped using
> Purify because Rational was so awful to deal with.

-Barry



From pje at telecommunity.com  Mon Apr 19 12:21:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr 19 12:21:32 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the Standard Library
In-Reply-To: <200404191525.i3JFPW009731@guido.python.org>
References: <Your message of "Mon, 19 Apr 2004 09:30:55 EDT."
	<5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040419121625.02fd9190@telecommunity.com>

At 08:25 AM 4/19/04 -0700, Guido van Rossum wrote:
> > I'd rather see something like:
> >
> >     from __future__ import fast_globals
> >
> > which would mean that globals and builtins could be considered
> > constants unless declared with 'global' at the module level.
>
>Don't you think that this should be flagged with syntax that doesn't
>permanently require the use of the word "future"?

I thought that the goal was to ultimately make at least builtins 
non-rebindable by default, without having flagged them as such.


>And I think that
>reusing the global statement at the global level is hardly the best
>way to do this.

Okay.



>I do think that explicitly flagging "volatile" globals somehow might
>be the right way to go eventually, but it should only be required for
>those globals for which the compiler can't tell whether they may be
>modified (i.e. anything that is assigned to more than once or or from
>inside a loop or conditional or function is automatically volatile).

The issue I was addressing was whether another module is allowed to rebind 
them via setattr.



> > Finally, the module object thus created would ban any __setattr__ on
> > any constant that has been bound into a function.  (Since these are
> > the only setattrs that could cause harm.)
>
>Huh?  If the object's identity is constant (which is what we're
>talking about) why should its *contents* be constant?  And who says
>setattr is the only way to modify an object?  Or am I misunderstanding
>what you're trying to say?  (I thought that "a constant bound into a
>function" would be something like a global/builtin name binding.)

Sorry, I was unclear.  I meant that the module object could disallow 
__setattr__(name,value) on *itself* for any name that was considered a 
"constant", because that name's binding had been copied into a function for 
use as a constant.

I wasn't trying to say that "constants" should be immutable; in fact there 
are plenty of use cases for mutable (but not rebindable) globals, such as 
module-level caches and registries.


From nbastin at opnet.com  Mon Apr 19 12:25:12 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Mon Apr 19 12:25:31 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <20040419160149.GD22725@epoch.metaslash.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
	<20040419160149.GD22725@epoch.metaslash.com>
Message-ID: <21CC4191-921E-11D8-BC61-000393CBDF94@opnet.com>


On Apr 19, 2004, at 12:01 PM, Neal Norwitz wrote:

> Valgrind runs on x86 and PPC (still experimental).
> IIRC, Purify "runs" on Solaris, HPUX, Windows (x86), and SGI?
> (I tried to verify but the IBM site isn't responding.)
> IMO that's not a lot.

That's more than valgrind.  Specifically, it includes solaris, which is 
where we happen to run Purify most often (better results than win32).

> Where Purify "running" is relative to any particular patch you have
> installed on the system and if you can download a beta.  I stopped 
> using
> Purify because Rational was so awful to deal with.

While I agree that Rational was terrible to deal with, Purify is the 
only option available to some users, and it seems like a bad idea to 
remove something from the distribution while it provides benefit to 
some people, while not affecting those who don't wish to use it.

--
Nick


From pje at telecommunity.com  Mon Apr 19 12:27:20 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr 19 12:27:30 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants inthe
	Standard Library
In-Reply-To: <004301c42626$e8634890$a100a8c0@ryoko>
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040419122230.02fdab40@telecommunity.com>

At 01:56 AM 4/20/04 +1000, Tim Delaney wrote:
>Phillip J. Eby wrote:
>
> > which would mean that globals and builtins could be considered
> > constants unless declared with 'global' at the module level.
>
>This would be a backwards-incompatible change (and hence definitely
>warranting the __future__) but I presume you mean considered constant within
>the module that they are defined - hence essentially a non-issue.

It's already backwards-incompatible.  Currently, you can change the 
behavior of a module by modifying its attributes.  A module that uses this 
binding approach is immune to changes in its globals.  This is a 
*seriously* backward incompatible change, as it essentially disallows 
monkeypatching.


> > Then, the compiler
> > could optimize any undeclared builtins, and the 'MAKE_FUNCTION' opcode
> > could bind any constants defined as of the function's declaration.
>
>I know it's been suggested before, but I believe the idea of module-level
>locals - an indexed lookup rather than a dict lookup - might be worthwhile
>considering as part of this. That would also give improved performance for
>external modules using module-level names.

Yes, I gather IronPython uses this technique as one of three alternative 
approaches to module globals.



> > Finally, the module object thus created would ban any __setattr__ on
> > any constant that has been bound into a function.  (Since these are the
> > only setattrs that could cause harm.)
>
>It's late (2am), so I'm not entirely sure I understand this - do you mean
>something like:
>
>     class X:
>         pass
>
>     x = X()
>
>     def func():
>         print x
>
>     x.a = 1
>
>would throw an exception? Why would this cause a problem?

No.  I meant that if your module above were named 'foo', then setting 
'foo.x' would be disallowed.


From fumanchu at amor.org  Mon Apr 19 12:43:07 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon Apr 19 12:46:49 2004
Subject: [Python-Dev] dict.sortedkeys()
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E56201A@opus.amorhq.net>

Alex Martelli wrote:
> On Saturday 17 April 2004 10:21 pm, Robert Brewer wrote:
>    ...
> > Yes, of course. But I'm working on a large business app 
> framework. That
> > means either:
> >
> > 1. Writing sorted into every module where it's needed.
> > 2. Putting sorted into a package somewhere, which makes it 
> longer and
> > uglier with absolute referencing.
> > 3. Customizing consumers' Python install.
> >
> > ...none of which are good options.
> 
> Is sorted the ONLY "utility" function your LARGE business app 
> framework needs in addition to what comes with Python 2.3?  
> That would really be utterly astounding to me.

That would astound me, too. :) However, I will make the following points
(if only for posterity--this is getting off-topic):

1. Despite what I just said, Python 2.3 is astoundingly complete.

2. Although I suck at actual implementation, I can be a pretty good
designer. I'd say 95% of the "utility" functions I write end up being
specific to one of the (sub)packages in such a framework, so it's
logical to shove them in that package.

3. I just checked; I have 4 "utility" modules that the framework uses:
one consists of 3 dict subclasses, another has a couple date tools, one
provides first-class expressions, and finally, there's a small import
helper.

Now that I've written that out, I wonder how many other people have
similar, generic, yet custom-built code. Which leads me to wonder if a
survey on c.l.p. would help hone direction for some standard lib
development.

> The only truly unacceptable alternative, in my opinion,
> is to repeat the coding for the utility functions more
> than once: as long as they're coded _once, and only once_,
> I'm a pretty happy camper.

Yep. There's nothing like forgetting to update both copies to make you
wish you had only done it once the first time. :)

> These days, I far prefer to code, more simply:
> 
> import __builtin__
> # get error msg in case of conflict
> assert not hasattr(__builtin__, 'ut')
> import utils as ut
> __builtin__.ut = ut
> 
> and access the utilities from everywhere as ut.sorted, 
> ut.scrambled, etc, etc.
> ONE relatively mysterious barename is OK...

That runs counter to my sensibilities as a Modern. I have to have clean
containers; a module named "utils" starts to cross the line between
practicality and purity. I much prefer to group similar code into
multiple modules which then get placed in site-packages. Again, if they
are app-specific, they go in the __init__.py for the app. The leap from
ZERO mysterious barenames to ONE is larger than the leap from one to
two, for me at least. ;)

> The "trickle" of neat utility functions into Python's
> built-ins, by itself, will not entirely solve the
> problem for you, IMHO.

No; I don't expect it to. I use the alternatives all the time. But there
comes a point when the cost of the neat function (implemented in any way
other than a builtin) outweighs the cost of the normal "boilerplate"
method of doing it.

sorted() is just such a case--*extremely* generic and also greatly
useful in making code clear and succinct. If you could sort a
dictionary's keys in three lines of code by saying:

keys
sort
iterate

there wouldn't be a problem, because the cost to the brain and fingers
of the programmer is low. But in pre-2.4 one must write:

dkeys = mydict.keys()
dkeys.sort()
for key in dkeys:

...which is extremely costly in terms of usability. Compare to:

for key in sorted(mydict):

Wow. What a difference. Design of a language is not that different from
any other design task: you want to make a better product the easiest
choice for the user. Small changes like the above end up returning many
times their investment.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From barry at python.org  Mon Apr 19 13:07:35 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 19 13:07:46 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <21CC4191-921E-11D8-BC61-000393CBDF94@opnet.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
	<20040419160149.GD22725@epoch.metaslash.com>
	<21CC4191-921E-11D8-BC61-000393CBDF94@opnet.com>
Message-ID: <1082394455.7365.175.camel@anthem.wooz.org>

On Mon, 2004-04-19 at 12:25, Nick Bastin wrote:

> While I agree that Rational was terrible to deal with, Purify is the 
> only option available to some users, and it seems like a bad idea to 
> remove something from the distribution while it provides benefit to 
> some people, while not affecting those who don't wish to use it.

Deprecation isn't the same as removal.  What puremodule is really
lacking at the moment is a champion; have you actually used it recently?

-Barry



From nbastin at opnet.com  Mon Apr 19 13:41:56 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Mon Apr 19 13:42:17 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <1082394455.7365.175.camel@anthem.wooz.org>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
	<20040419160149.GD22725@epoch.metaslash.com>
	<21CC4191-921E-11D8-BC61-000393CBDF94@opnet.com>
	<1082394455.7365.175.camel@anthem.wooz.org>
Message-ID: <DA64D487-9228-11D8-BC61-000393CBDF94@opnet.com>


On Apr 19, 2004, at 1:07 PM, Barry Warsaw wrote:

> On Mon, 2004-04-19 at 12:25, Nick Bastin wrote:
>
>> While I agree that Rational was terrible to deal with, Purify is the
>> only option available to some users, and it seems like a bad idea to
>> remove something from the distribution while it provides benefit to
>> some people, while not affecting those who don't wish to use it.
>
> Deprecation isn't the same as removal.  What puremodule is really
> lacking at the moment is a champion; have you actually used it 
> recently?

Ah, the moment of truth.. :-)  Well, uh, no.  I didn't know puremodule 
existed, so I didn't give it a try.  It seems like something that would 
be of use, though, so I'd hate to see it go away.  A cursory look at 
the code indicates that it may need some updating (have to try...the 
API may not have actually changed significantly).  I'm actually more 
interested in it from the Quantify side of things from the standpoint 
of being able to provide the user a single look at their application 
profiling data from both a C and Python perspective.

--
Nick


From barry at python.org  Mon Apr 19 13:52:40 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 19 13:52:49 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <DA64D487-9228-11D8-BC61-000393CBDF94@opnet.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
	<20040419160149.GD22725@epoch.metaslash.com>
	<21CC4191-921E-11D8-BC61-000393CBDF94@opnet.com>
	<1082394455.7365.175.camel@anthem.wooz.org>
	<DA64D487-9228-11D8-BC61-000393CBDF94@opnet.com>
Message-ID: <1082397159.7365.182.camel@anthem.wooz.org>

On Mon, 2004-04-19 at 13:41, Nick Bastin wrote:

> Ah, the moment of truth.. :-)  Well, uh, no.  I didn't know puremodule 
> existed, so I didn't give it a try.  It seems like something that would 
> be of use, though, so I'd hate to see it go away.  A cursory look at 
> the code indicates that it may need some updating (have to try...the 
> API may not have actually changed significantly).  I'm actually more 
> interested in it from the Quantify side of things from the standpoint 
> of being able to provide the user a single look at their application 
> profiling data from both a C and Python perspective.

It's been years since I looked at the stuff, but IIRC, the module did
work for Quantify too.  Well, feel free to update it!  Lacking a
champion though I still think it should be deprecated (but not removed
at least for now).

-Barry



From nbastin at opnet.com  Mon Apr 19 13:56:48 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Mon Apr 19 13:57:02 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <1082397159.7365.182.camel@anthem.wooz.org>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
	<20040419160149.GD22725@epoch.metaslash.com>
	<21CC4191-921E-11D8-BC61-000393CBDF94@opnet.com>
	<1082394455.7365.175.camel@anthem.wooz.org>
	<DA64D487-9228-11D8-BC61-000393CBDF94@opnet.com>
	<1082397159.7365.182.camel@anthem.wooz.org>
Message-ID: <EE073933-922A-11D8-BC61-000393CBDF94@opnet.com>


On Apr 19, 2004, at 1:52 PM, Barry Warsaw wrote:

> On Mon, 2004-04-19 at 13:41, Nick Bastin wrote:
>
>> Ah, the moment of truth.. :-)  Well, uh, no.  I didn't know puremodule
>> existed, so I didn't give it a try.  It seems like something that 
>> would
>> be of use, though, so I'd hate to see it go away.  A cursory look at
>> the code indicates that it may need some updating (have to try...the
>> API may not have actually changed significantly).  I'm actually more
>> interested in it from the Quantify side of things from the standpoint
>> of being able to provide the user a single look at their application
>> profiling data from both a C and Python perspective.
>
> It's been years since I looked at the stuff, but IIRC, the module did
> work for Quantify too.  Well, feel free to update it!  Lacking a
> champion though I still think it should be deprecated (but not removed
> at least for now).

That seems fine.  I may pick it up and take a look after I'm done with 
my current pass of profiler enhancements, but probably not before, so 
it'll be a little while.  It also appears that no one else on 
python-dev actually uses it... :-)

--
Nick


From bac at OCF.Berkeley.EDU  Mon Apr 19 14:06:04 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Apr 19 14:06:16 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <200404190928.23248.fdrake@acm.org>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404190928.23248.fdrake@acm.org>
Message-ID: <4084150C.3060908@ocf.berkeley.edu>

Fred L. Drake, Jr. wrote:

> On Monday 19 April 2004 02:56 am, Brett C. wrote:
>  > pickletools.py
> 
> SF bug report 873146
> 
>  > imp (built-in)
> 
> This one is documented; check the module index.
> 

Oops.  My bad.

-Brett

From bac at OCF.Berkeley.EDU  Mon Apr 19 14:24:00 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Apr 19 14:24:13 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the	Standard Library
In-Reply-To: <200404191525.i3JFPW009731@guido.python.org>
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<200404191525.i3JFPW009731@guido.python.org>
Message-ID: <40841940.4010305@ocf.berkeley.edu>

Guido van Rossum wrote:

>>I'd rather see something like:
>>
>>    from __future__ import fast_globals
>>
>>which would mean that globals and builtins could be considered
>>constants unless declared with 'global' at the module level.
> 
> 
> Don't you think that this should be flagged with syntax that doesn't
> permanently require the use of the word "future"?  And I think that
> reusing the global statement at the global level is hardly the best
> way to do this.
> 
> I do think that explicitly flagging "volatile" globals somehow might
> be the right way to go eventually, but it should only be required for
> those globals for which the compiler can't tell whether they may be
> modified (i.e. anything that is assigned to more than once or or from
> inside a loop or conditional or function is automatically volatile).
> 

Just to make sure I am understanding this, are you suggesting a possible 
statement like ``volatile len`` to flag that len may be changed?  That 
way all "volatile"-flagged globals and one that are redefined in the 
module use LOAD_GLOBAL while all other built-ins either get them set to 
locals or use a LOAD_BUILTIN opcode?

Or am I getting the use of volatile reversed (which would make more 
backwards-compatible)?

If we go with the former interpretation I take it we would still need to 
get that warning Neil worked on a while back for warning people when 
someone is injecting into a module's global namespace.

-Brett

From skip at pobox.com  Mon Apr 19 12:46:53 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Apr 19 14:34:43 2004
Subject: [Python-Dev] Possible modules that could use docs
In-Reply-To: <D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
References: <40837822.6060906@ocf.berkeley.edu>
	<200404191517.i3JFHFa09689@guido.python.org>
	<1082389011.7365.123.camel@anthem.wooz.org>
	<20040419154238.GC22725@epoch.metaslash.com>
	<D0B335D1-9218-11D8-BC61-000393CBDF94@opnet.com>
Message-ID: <16516.637.719565.642835@montanaro.dyndns.org>

    >>> puremodule.c could probably be deprecated.  Does anybody use Purify
    >>> anymore -- and if so, have you ever used puremodule?  Does it still
    >>> compile?  Are the APIs it uses still up-to-date?
    >> 
    >> Although I did use Purify on python years ago, I never use it now.
    >> Though, I never used the puremodule.  IMO, Purify is not useful
    >> compared to valgrind.  +1 on deprecating puremodule.c.

    Nick> I think it's useful to point out that Purify runs on a lot more
    Nick> platforms than valgrind does, so we should probably keep
    Nick> puremodule around, and update it if necessary.

I don't think there's any question that Purify (and perhaps the pure module)
are of use to a certain segment of the Python user base.  The question is
more whether or not it can be properly maintained as part of the
distribution.  Barry and perhaps Neal are the only two people I remember
ever using it, and it appears that neither of them to anymore.  Deprecating
it doesn't mean it has to go away, just that support has to come from
outside the core group of maintainers.

Does anyone with checkin privileges:

    1. have the necessary Purify license which would allow them to build and
       exercise the pure module?

    2. have the time/desire to support it?

If the answer is "no" to either question, I think deprecation is the way to
go.

Skip

From jim.jewett at EDS.COM  Mon Apr 19 14:34:10 2004
From: jim.jewett at EDS.COM (Jewett, Jim J)
Date: Mon Apr 19 14:35:06 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D47C@USAHM010.amer.corp.eds.com>

Kevin Jacobs:
#-   Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
#-   Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')
#- 
#- Remember, these literals are frequently coming from an 
#- external source that must be constrained to a given schema.

Facundo Batista:

> I like it a lot, but not for Decimal.

This is another face of "what to do with float"

Even if your default context is n digits, there is no reason
to assume that all your inputs will be measured that precisely.
If someone sends me a list of weights:

	PartA	1105 kg
	PartB	   3 kg

then I don't want to pretend that the 3kg part was weighed
any more precisely just because the total is smaller.

On the other hand, if the weights are:

	PartA	3 kg
	PartB 3.0 kg

then I do want to assume that the second weight is more precise.

As an example

	3000 g	3001 g
	+  2 g	+  1 g
      ------      ------
	   3kg	3002 g

#- I assume that a new Decimal would normally be created
#- with as much precision as the context would need for 
#- calculations.  By passing a context/precision/position,
#- the user is saying "yeah, but this measurement wasn't
#- that precise in the first place.  Use zeros for the
#- rest, no matter what this number claims."

I don't get to understand you, and I'm not sure if you have the right
concept. Several examples may help:

>>> getcontext().prec = 5
>>> Decimal(124)
Decimal( (0, (1, 2, 4), 0) )
>>> +Decimal(124)
Decimal( (0, (1, 2, 4), 0) )
>>> Decimal('258547.368')
Decimal( (0, (2, 5, 8, 5, 4, 7, 3, 6, 8), -3) )
>>> +Decimal('258547.368')
Decimal( (0, (2, 5, 8, 5, 5), 1L) )
>>> Decimal.from_float(1.1)
Decimal( (0, (1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 1,
7, 8, 4, 1, 9, 7, 0, 0, 1, 2, 5, 2, 3, 2, 3, 3, 8, 9, 0, 5, 3, 3, 4, 4, 7,
2, 6, 5, 6, 2, 5), -51L) )
>>> +Decimal.from_float(1.1)
Decimal( (0, (1, 1, 0, 0, 0), -4L) )

.	Facundo


From guido at python.org  Mon Apr 19 14:35:10 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 19 14:35:39 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: Your message of "Mon, 19 Apr 2004 11:24:00 PDT."
	<40841940.4010305@ocf.berkeley.edu> 
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<200404191525.i3JFPW009731@guido.python.org> 
	<40841940.4010305@ocf.berkeley.edu> 
Message-ID: <200404191835.i3JIZAm10204@guido.python.org>

[Phillip Eby]
> >>I'd rather see something like:
> >>
> >>    from __future__ import fast_globals
> >>
> >>which would mean that globals and builtins could be considered
> >>constants unless declared with 'global' at the module level.

[Guido van Rossum]
> > Don't you think that this should be flagged with syntax that doesn't
> > permanently require the use of the word "future"?  And I think that
> > reusing the global statement at the global level is hardly the best
> > way to do this.
> > 
> > I do think that explicitly flagging "volatile" globals somehow might
> > be the right way to go eventually, but it should only be required for
> > those globals for which the compiler can't tell whether they may be
> > modified (i.e. anything that is assigned to more than once or or from
> > inside a loop or conditional or function is automatically volatile).

[Brett Cannon]
> Just to make sure I am understanding this, are you suggesting a possible 
> statement like ``volatile len`` to flag that len may be changed?

Yeah, but not that syntax.  (I'm not proposing any syntax at this point.)

> That way all "volatile"-flagged globals and one that are redefined
> in the module use LOAD_GLOBAL while all other built-ins either get
> them set to locals or use a LOAD_BUILTIN opcode?

Sort of, although I think you meant STORE_GLOBAL rather than
LOAD_GLOBAL.  Exactly how this would be implemented is up to the
bytecode compiler; I can see that some cheap builtins (e.g. len) may
be turned into special opcodes so that no lookup is used at all.

> Or am I getting the use of volatile reversed (which would make more 
> backwards-compatible)?

You're not getting it reversed.  Having to flag the ones that are not
volatile would indeed be more backward compatible, but it would
require everybody to put lots of 'nonvolatile' statements in all the
time just in the hope to get faster code.  That's what I'm trying to
avoid, and that's why I'm not psyched about Raymond's (or anybody's!) 
proposed "make this function faster" API.  Given that "volatile"
globals are very rare, it makes more sense to put the burden on the
programmer who needs his seemingly-constant globals to be reloaded.

> If we go with the former interpretation I take it we would still
> need to get that warning Neil worked on a while back for warning
> people when someone is injecting into a module's global namespace.

Yeah, that warning is still the right thing to do, but it's
implementation was a nuisance because there were some legit cases that
it couldn't know about.  I don't recall -- did we turn it into a
silent deprecation or did we just delete the warning?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From nbastin at opnet.com  Mon Apr 19 14:37:34 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Mon Apr 19 14:37:56 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the	Standard Library
In-Reply-To: <40841940.4010305@ocf.berkeley.edu>
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<200404191525.i3JFPW009731@guido.python.org>
	<40841940.4010305@ocf.berkeley.edu>
Message-ID: <A016DB92-9230-11D8-BC61-000393CBDF94@opnet.com>


On Apr 19, 2004, at 2:24 PM, Brett C. wrote:

> Guido van Rossum wrote:
>
>>> I'd rather see something like:
>>>
>>>    from __future__ import fast_globals
>>>
>>> which would mean that globals and builtins could be considered
>>> constants unless declared with 'global' at the module level.
>> Don't you think that this should be flagged with syntax that doesn't
>> permanently require the use of the word "future"?  And I think that
>> reusing the global statement at the global level is hardly the best
>> way to do this.
>> I do think that explicitly flagging "volatile" globals somehow might
>> be the right way to go eventually, but it should only be required for
>> those globals for which the compiler can't tell whether they may be
>> modified (i.e. anything that is assigned to more than once or or from
>> inside a loop or conditional or function is automatically volatile).
>
> Just to make sure I am understanding this, are you suggesting a 
> possible statement like ``volatile len`` to flag that len may be 
> changed?  That way all "volatile"-flagged globals and one that are 
> redefined in the module use LOAD_GLOBAL while all other built-ins 
> either get them set to locals or use a LOAD_BUILTIN opcode?
>
> Or am I getting the use of volatile reversed (which would make more 
> backwards-compatible)?

You could, of course, create a statement like "const len" to flag that 
len will NOT be changed, thus creating true backwards compatibility, 
but you'd like to believe that const is the 95% use case, and thus it 
should be "volatile foo" when you want to change foo, but that's not 
all that backwards compatible.  Of course, you could add something like 
"from __past__ import volatile_globals", and "from __future__ import 
const_globals" for a release.. ;-)

--
Nick


From jim.jewett at eds.com  Mon Apr 19 14:39:22 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Apr 19 14:40:08 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D47D@USAHM010.amer.corp.eds.com>

[Kevin Jacobs]
#-   Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
#-   Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')
#- 
#- Remember, these literals are frequently coming from an 
#- external source that must be constrained to a given schema.

Facundo
> I like it a lot, but not for Decimal.

This is another face of "what do do with float"

Just because your default context has n digits of precision
doesn't mean that all your input will.

If my input has 2.4000 (but the extra zeros were measured)
then I want to keep that information.

If my input has 2.4 (and I didn't measure beyond those digits)
I want to know that for error analysis.

If my input has 1.1 (only measured to two digits) but my 
context is 25, I don't want to assume that the machine's
float-rounding is correct; the fairest "I don't know" 
estimate is still zero, even out to digit 100.

-jJ

From aahz at pythoncraft.com  Mon Apr 19 14:51:49 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Apr 19 14:51:54 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D47D@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D47D@USAHM010.amer.corp.eds.com>
Message-ID: <20040419185149.GA23690@panix.com>

On Mon, Apr 19, 2004, Jewett, Jim J wrote:
>
> [Kevin Jacobs]
> #-   Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
> #-   Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')
> #- 
> #- Remember, these literals are frequently coming from an 
> #- external source that must be constrained to a given schema.
> 
> Facundo
>> I like it a lot, but not for Decimal.
> 
> This is another face of "what do do with float"

This has nothing to do with binary floating point.  These are strings
that map to exact numbers in Decimal; if you want to do something
different with them, it's your responsibility as a programmer.  Decimal
should provide a solid foundation, not every bell & whistle.  If you
disagree, go read Cowlishaw first.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From tjreedy at udel.edu  Mon Apr 19 15:00:56 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon Apr 19 15:00:58 2004
Subject: [Python-Dev] Re: Possible modules that could use docs
References: <40837822.6060906@ocf.berkeley.edu>
Message-ID: <c617l2$m00$1@sea.gmane.org>


"Brett C." <bac@OCF.Berkeley.EDU> wrote in message
news:40837822.6060906@ocf.berkeley.edu...
> Discovering the 'opcode' module thanks to the new PEP 329 caused me to
> wonder how many modules are undocumented and yet deserve to possibly be
> (I also was procrastinating from doing schoolwork and felt like doing
> some light coding =).
...
[list snipped]

This 'bug' (and those already filed) could be minimally/temporarily
disposed of by adding a page 'Modules Not Otherwise Documented' with the
list and one sentence for each (plus status if discouraged/deprecated/etc)
and then adding index entries pointing to this page.  Also add
indication/suggestion to try interactive help('modname') for the modules
for which one will get something.  Then people would know that they had not
missed anything to be found in the Lib Ref.

tjr




From pje at telecommunity.com  Mon Apr 19 15:25:00 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr 19 15:25:12 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the	Standard Library
In-Reply-To: <A016DB92-9230-11D8-BC61-000393CBDF94@opnet.com>
References: <40841940.4010305@ocf.berkeley.edu>
	<5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<200404191525.i3JFPW009731@guido.python.org>
	<40841940.4010305@ocf.berkeley.edu>
Message-ID: <5.1.1.6.0.20040419150808.02b70560@telecommunity.com>

At 02:37 PM 4/19/04 -0400, Nick Bastin wrote:

>You could, of course, create a statement like "const len" to flag that len 
>will NOT be changed, thus creating true backwards compatibility, but you'd 
>like to believe that const is the 95% use case, and thus it should be 
>"volatile foo" when you want to change foo, but that's not all that 
>backwards compatible.

For names that are never bound by a module's code (i.e. no 'foo=', 'import 
foo', 'global foo'), it should be safe to make them "constant", since there 
is nothing that can change them now except tinkering with the builtins, or 
assigning them from outside modules.  (The big hurdle here is 'from 
somewhere import *' if 'somewhere' defines any names that shadow 
builtins.  Of course, one could simply say that using 'import *' disables 
this optimization.)

I'm more cautious wrt module globals, however.  As I've been thinking about 
this today, I keep finding things that will not quite work as expected if 
too many globals become "constants".  Optimizing only builtins, however, 
would probably give plenty of speed improvement, and not require anything 
more complex than:

len = len

to re-enable substitution of a given builtin.  But I think module objects 
would need to change in a way that allows a loaded module to flag what 
builtins are changeable, so that warnings could be generated during the 
transition period.

IOW, module objects need to know what names they should reject for 
__setattr__, and whether the rejection should be a warning or 
error.  Modules with 'from __future__ import fast_builtins' would generate 
errors when other code tried to modify their builtins, and the rest would 
issue warnings indicating that you must specifically enable that by 
assigning to the builtin in the target module.

Hm.  That means, by the way, that the compiler would need to detect these 
issues and track modifiable builtins in *all* modules, not just those using 
the future statement.  But only the ones using the future statement would 
be able to take advantage of faster access to builtins (or builtin 
replacements like a "len" opcode, or compiler tricks to optimize away 
builtins).


From bac at OCF.Berkeley.EDU  Mon Apr 19 15:27:54 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Apr 19 15:28:07 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <200404191835.i3JIZAm10204@guido.python.org>
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<200404191525.i3JFPW009731@guido.python.org>
	<40841940.4010305@ocf.berkeley.edu>
	<200404191835.i3JIZAm10204@guido.python.org>
Message-ID: <4084283A.8050108@ocf.berkeley.edu>

Guido van Rossum wrote:

> [Brett Cannon]
> 
>>Just to make sure I am understanding this, are you suggesting a possible 
>>statement like ``volatile len`` to flag that len may be changed?
> 
> 
> Yeah, but not that syntax.  (I'm not proposing any syntax at this point.)
> 

Right.  I just wanted to write it out in some form.  Didn't mean to 
imply that is what you were suggesting.

> 
>>That way all "volatile"-flagged globals and one that are redefined
>>in the module use LOAD_GLOBAL while all other built-ins either get
>>them set to locals or use a LOAD_BUILTIN opcode?
> 
> 
> Sort of, although I think you meant STORE_GLOBAL rather than
> LOAD_GLOBAL.  Exactly how this would be implemented is up to the
> bytecode compiler; I can see that some cheap builtins (e.g. len) may
> be turned into special opcodes so that no lookup is used at all.
> 

I actually was thinking of LOAD_GLOBAL; I was thinking of ``foo = 
len(x)`` which loads len (global), loads x (local), calls len, and then 
stores into foo (local).  That is what Raymond's code is optimizing; 
storing built-ins into the local namespace to skip having to try load 
from the global namespace, fail, and then check the built-in namespace 
(which are both slower than local since they are dictionaries and not an 
array like locals) (I just realized how much "talking out load" text I 
put in my emails; I'm a talker =).

As for the cheap opcodes, it seems reasonable for the frequently used 
built-ins.

And as for implementation, while I am thinking about it, storing 
built-ins into the local namespace would be the fastest, but using a 
LOAD_BUILTIN opcode would allow for universal overriding of built-ins if 
so desired.  That is going to be an interesting thread if this ends up 
coming to fruition.  =)

> 
>>Or am I getting the use of volatile reversed (which would make more 
>>backwards-compatible)?
> 
> 
> You're not getting it reversed.  Having to flag the ones that are not
> volatile would indeed be more backward compatible, but it would
> require everybody to put lots of 'nonvolatile' statements in all the
> time just in the hope to get faster code.  That's what I'm trying to
> avoid, and that's why I'm not psyched about Raymond's (or anybody's!) 
> proposed "make this function faster" API.  Given that "volatile"
> globals are very rare, it makes more sense to put the burden on the
> programmer who needs his seemingly-constant globals to be reloaded.
> 

Right.  I know one of the reasons I enjoy programming in Python is that 
I don't have to think about optimizations when I code.  If I want some 
non-standard use of the language I have no issue of jumping through some 
hoops (such as defining something as volatile) but the base case 
(programming for "cases 90% of the time" as Neal and Jeremy pointed out 
to me at PyCon) is simple and does not require extra thought.

> 
>>If we go with the former interpretation I take it we would still
>>need to get that warning Neil worked on a while back for warning
>>people when someone is injecting into a module's global namespace.
> 
> 
> Yeah, that warning is still the right thing to do, but it's
> implementation was a nuisance because there were some legit cases that
> it couldn't know about.  I don't recall -- did we turn it into a
> silent deprecation or did we just delete the warning?
> 

I thought we just ripped it out since it never quite worked perfectly, 
but I could be wrong.

-Brett

From mcherm at mcherm.com  Mon Apr 19 15:33:10 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Apr 19 15:33:11 2004
Subject: [Python-Dev] Re: Possible modules that could use docs
Message-ID: <1082403190.4084297622d8b@mcherm.com>

Terry Reedy writes:
> This 'bug' (and those already filed) could be minimally/temporarily
> disposed of by adding a page 'Modules Not Otherwise Documented' with the
> list and one sentence for each (plus status if discouraged/deprecated/etc)
> and then adding index entries pointing to this page.  Also add
> indication/suggestion to try interactive help('modname') for the modules
> for which one will get something.  Then people would know that they had not
> missed anything to be found in the Lib Ref.

But let's not jump into doing that just yet... perhaps Brett's listing
will attract some champions willing to document these orphan modules.
I'd really rather not give up on documenting a module unless we're
forced to... it's tantamount to giving up on the module itself!

In the future, of course, no new modules should be accepted without docs.

-- Michael Chermside


From bac at OCF.Berkeley.EDU  Mon Apr 19 15:39:31 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Apr 19 15:39:47 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
	the	Standard Library
In-Reply-To: <200404191536.i3JFasX09776@guido.python.org>
References: <5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>	<5.2.1.1.0.20040419154633.02d12be8@pop.bluewin.ch>
	<5.2.1.1.0.20040419161441.02d12688@pop.bluewin.ch>
	<200404191536.i3JFasX09776@guido.python.org>
Message-ID: <40842AF3.8070600@ocf.berkeley.edu>

Guido van Rossum wrote:
[SNIP]
> 
> While Samuele's words were inappropriately rude, I'm glad this was
> brought up (and even to some extent I'm glad about some of the words
> that Raymond used).  Raymond *does* seem to have a "speed obsession",
> and while I'm sure he has Python's best interests at heart, I'm not
> sure that accepting everything he proposes will actually be good for
> Python.
> 

While I was thinking about this whole thread I realized that this is 
just a PEP and can easily be rejected by Guido.  This is not like 
Raymond went ahead and added the module along with changing the stdlib 
to use it without proposing it to python-dev first (not going to get 
into that and the whole heapq thing here).  It looks like this thread is 
not going to explode into a heated argument and I for one am glad it 
isn't since I don't think it needs to since this is just a PEP and 
nothing else.  This needs to be viewed as just another PEP and not as 
"Raymond's speed obsession has acted up again!" or something of that ilk.

[SNIP]
> 
>>PS: I'm starting to be really annoyed by the fact that Jython is
>>often cited good PR but a 2nd class citizen for all the rest. I'm
>>aware of the practical circumstances of why that's the case in some
>>occassions, OTOH we are mostly all investing our free time on Python
>>activities, it would be nice if we showed respect beyond just the
>>forms and politeness. I'm back from the ACCU which was fun but
>>tiring and also an occasion of some related heat taking. All I was
>>looking forward is restarting to finish the new-style class work
>>(started in December and January) in Jython,after the stop because
>>of working on the negotations for the PyPy EU proposal together with
>>the others PyPy people, which we hope will conclude positively for
>>the people working on it and looking forward to restart making PyPy
>>a real concrete contribution to Python future.
> 
> 
> I hope you don't think *everybody* sees Jython as a 2nd class citizen.
> I personally see it as very important.  Certainly the PSF is
> considering funding it, whether via the general grants committee (just
> formed) or via a special grant.
> 

I know I appreciate Jython as well.  I don't know about everyone else, 
but I know I view CPython as an implementation of the Python language, 
just like Jython is an implementation, and thus on the same playing 
field in that regard.  While CPython might be used as the testbed and 
reference implementation, it is still just an implementation of the 
language which is the most important thing in all of this, not the 
specifics of how we implement it, more or less.

-Brett

From jim.jewett at eds.com  Mon Apr 19 15:46:31 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Apr 19 15:47:06 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D47E@USAHM010.amer.corp.eds.com>


On Mon, Apr 19, 2004, Jewett, Jim J wrote:

> [Kevin Jacobs]
> #-   Decimal('2.4000', precision=2, scale=1) == Decimal('2.4')
> #-   Decimal('2.4', precision=5, scale=4) == Decimal('2.4000')
> #- 
> #- Remember, these literals are frequently coming from an 
> #- external source that must be constrained to a given schema.
> 
> Facundo
>> I like it a lot, but not for Decimal.
 
> This is another face of "what do do with float"

Aahz:
> This has nothing to do with binary floating point.  These are 
> strings that map to exact numbers in Decimal; 

Under the current implementation:

	(0, (2, 4, 0, 0, 0), -4) 

is not quite the same as

	(0, (2, 4) -1)

Given this, is should be possible for the user to specify 
(at creation) which is desired.

They do normalize to the same thing, but this requires 
extending the second number with zeros.

	(0, (1,), -17)	
	(0, (1, 1), -1)

The float discussions make it clear that "extend by zero"
may not always be the most accurate, and the python docs
do not yet prohibit extending by some other digit (that
rounds the same).

I agree that the programmer should specify if they want 
something odd -- and that is precisely why the constructor 
should take an optional context argument which allows the 
user to specify precision.

Given that float input may well look like string (saved to a
file by another program) or even an integer (C double may have 
more precision than C long) there is no particular reason to
say "context is only allowed when constructing from a float"

-jJ

From mcherm at mcherm.com  Mon Apr 19 15:53:08 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Apr 19 15:53:13 2004
Subject: [Python-Dev] Possible modules that could use docs
Message-ID: <1082404388.40842e247bc8a@mcherm.com>

See http://www.python.org/cgi-bin/moinmoin/ModulesThatNeedDocs

-- Michael Chermside


From fdrake at acm.org  Mon Apr 19 16:01:46 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Apr 19 16:01:58 2004
Subject: [Python-Dev] Re: Possible modules that could use docs
In-Reply-To: <c617l2$m00$1@sea.gmane.org>
References: <40837822.6060906@ocf.berkeley.edu> <c617l2$m00$1@sea.gmane.org>
Message-ID: <200404191601.46411.fdrake@acm.org>

On Monday 19 April 2004 03:00 pm, Terry Reedy wrote:
 > This 'bug' (and those already filed) could be minimally/temporarily
 > disposed of by adding a page 'Modules Not Otherwise Documented' with the
 > list and one sentence for each (plus status if discouraged/deprecated/etc)

There's already a whole chapter like that; I'd really rather the modules get 
documented than addiing to that, but listing them there is better than 
releasing them without any docs.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From martin at v.loewis.de  Mon Apr 19 16:03:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Apr 19 16:03:35 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060E63@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060E63@UKDCX001.uk.int.atosorigin.com>
Message-ID: <40843086.9000705@v.loewis.de>

Moore, Paul wrote:
> I don't know if anyone has stats on how many of the 3rd party
> extension authors who currently provide Windows binaries have
> access to MSVC7, and so can make the switch. It may be "all of
> them". I don't have MSVC7, although I do have MSVC6 (my company,
> or at least my group, never upgraded) but I've no idea how unusual
> my situation is. A heads-up on c.l.p would probably be worth it,
> but it's also likely to generate a huge amount of FUD, and anti-
> Microsoft rants.

Given that the Python release is still several months ahead,
I'd advise against such a posting. Personally, I don't know
a single VC6 user who doesn't also have a copy of VC.NET 2003
available (except for you, whom I only know via email :-)

In any case, the only possible change out of this discussion
is that Python 2.4 would be built with VC6. I personally know
a few people which have VC.NET 2003, but not VC6, because you
cannot purchase the latter one, anymore. So, standardizing
on .NET 2003 is a good thing: it means that .NET (2002) gets
skipped (and widely so, beyond Python); it may also mean
that .NET 2005 (aka Whidbey) might get skipped, in favour
of then-.NET 2006.

Regards,
Martin



From FBatista at uniFON.com.ar  Mon Apr 19 16:08:03 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Apr 19 16:10:30 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838B6@escpl.tcp.com.ar>

[Jewett, Jim J]
#- Under the current implementation:
#- 
#- 	(0, (2, 4, 0, 0, 0), -4) 
#- 
#- is not quite the same as
#- 
#- 	(0, (2, 4) -1)
#- 
#- Given this, is should be possible for the user to specify 
#- (at creation) which is desired.

It *is* posible:

>>> Decimal('2.4000')
Decimal( (0, (2, 4, 0, 0, 0), -4) )
>>> Decimal('2.4')
Decimal( (0, (2, 4), -1) )
>>>


#- I agree that the programmer should specify if they want 
#- something odd -- and that is precisely why the constructor 
#- should take an optional context argument which allows the 
#- user to specify precision.

If you construct using precision, and the precision is smaller than the
quantity of digits you provide, you'll get rounded, but if the precision is
greater than the quantity of digits you provide, you don't get filled with
zeros.


#- Given that float input may well look like string (saved to a
#- file by another program) or even an integer (C double may have 
#- more precision than C long) there is no particular reason to
#- say "context is only allowed when constructing from a float"

No. Why we should do that? 

.	Facundo

From skip at pobox.com  Mon Apr 19 17:46:18 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Apr 19 17:46:26 2004
Subject: [Python-Dev] syncmail question - how to get submitter as From:?
In-Reply-To: <16514.31295.926740.594739@montanaro.dyndns.org>
References: <16511.65327.705618.504369@montanaro.dyndns.org>
	<16512.8565.21289.392919@montanaro.dyndns.org>
	<16512.20616.41799.327583@montanaro.dyndns.org>
	<200404171646.26885.fdrake@acm.org>
	<16514.31295.926740.594739@montanaro.dyndns.org>
Message-ID: <16516.18602.256777.317163@montanaro.dyndns.org>


    Fred> This is something syncmail should probably support out of the box.
    Fred> If you'd like to submit a patch, we'd love to have it:

    Fred> http://sourceforge.net/projects/cvs-syncmail/

    Skip> Will do when I'm back at work Monday.

http://sourceforge.net/tracker/index.php?func=detail&aid=938197&group_id=47611&atid=450021

This is the simplest possible patch (no command-line flag).  Just grab
CVS_USER and fall back to the user running syncmail if that's not available.

Skip

From nas at arctrix.com  Mon Apr 19 19:02:18 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon Apr 19 19:02:23 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in the
	Standard Library
In-Reply-To: <200404191835.i3JIZAm10204@guido.python.org>
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<200404191525.i3JFPW009731@guido.python.org>
	<40841940.4010305@ocf.berkeley.edu>
	<200404191835.i3JIZAm10204@guido.python.org>
Message-ID: <20040419230218.GA11314@mems-exchange.org>

On Mon, Apr 19, 2004 at 11:35:10AM -0700, Guido van Rossum wrote:
> Yeah, that warning is still the right thing to do, but it's
> implementation was a nuisance because there were some legit cases that
> it couldn't know about.  I don't recall -- did we turn it into a
> silent deprecation or did we just delete the warning?

We ripped it out (too many spurious warnings).

  Neil

From greg at cosc.canterbury.ac.nz  Mon Apr 19 20:13:54 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Apr 19 20:14:30 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants inthe
	Standard Library
In-Reply-To: <5.1.1.6.0.20040419122230.02fdab40@telecommunity.com>
Message-ID: <200404200013.i3K0DsrZ028600@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> It's already backwards-incompatible.  Currently, you can change the 
> behavior of a module by modifying its attributes.  A module that uses this 
> binding approach is immune to changes in its globals.  This is a 
> *seriously* backward incompatible change, as it essentially disallows 
> monkeypatching.

This suggests that there should be some way of disabling
the optimisation from *outside* the module. Not sure what
that would be, though.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Mon Apr 19 20:23:18 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Apr 19 20:23:25 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants inthe
	Standard Library
In-Reply-To: Your message of "Tue, 20 Apr 2004 12:13:54 +1200."
	<200404200013.i3K0DsrZ028600@cosc353.cosc.canterbury.ac.nz> 
References: <200404200013.i3K0DsrZ028600@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200404200023.i3K0NIu10904@guido.python.org>

> This suggests that there should be some way of disabling
> the optimisation from *outside* the module. Not sure what
> that would be, though.

A command line flag?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Mon Apr 19 20:43:49 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Apr 19 20:43:32 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants inthe
	Standard Library
In-Reply-To: <200404200023.i3K0NIu10904@guido.python.org>
References: <Your message of "Tue, 20 Apr 2004 12:13:54 +1200."
	<200404200013.i3K0DsrZ028600@cosc353.cosc.canterbury.ac.nz>
	<200404200013.i3K0DsrZ028600@cosc353.cosc.canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20040419203338.0339fe60@mail.telecommunity.com>

At 05:23 PM 4/19/04 -0700, Guido van Rossum wrote:
> > This suggests that there should be some way of disabling
> > the optimisation from *outside* the module. Not sure what
> > that would be, though.
>
>A command line flag?

For that to work, there'd need to be some place to store the names of the 
modules to which one wished to apply this "pessimization".  That being the 
case, one might as well expose an API to Python for that, rather than 
forcing the issue up to the command line and thereby reducing modularity, 
as well as possibly being an issue for embedders, Jython, et al.

Anyway, the API would give you a function to call if you want to disable 
optimization for a module in order to monkeypatch it.  That function would 
then raise an error if the module in question was already imported with 
optimization enabled, or if the runtime environment disallows such changes 
(e.g. if Jython, IronPython, or various hypothetical 
Python-to-static-language translators want to disallow monkeypatching).

One possibly interesting use case is Pyrex, which would like to be able to 
disallow modifications to built-ins in order to translate e.g. 'len()' into 
C.  It would be nice if there's a way for C modules to "play nice" with the 
mechanisms under discussion.  Specifically, if there's a way for them to 
declare metadata of the same sort that's contemplated for indicating what 
attributes of a module are modifiable from outside.


From pedronis at bluewin.ch  Tue Apr 20 06:49:34 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Tue Apr 20 06:44:29 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants inthe
	Standard Library
In-Reply-To: <006f01c426c2$56978ce0$27b2958d@oemcomputer>
References: <5.2.1.1.0.20040419161441.02d12688@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20040420124302.02a7b2e0@pop.bluewin.ch>

At 06:29 20.04.2004 -0400, Raymond Hettinger wrote:
> > he's proposing an hack that would speed up CPython and slow down
>Jython
> > (because it would result in a no-op but OTOH he would like to remove
>the
> > code
> > when some globals are bound to locals for speed reason. Of course
>that's
> > not nice too
> > but replacing a hack with another is just sort of a win. Honestly
>there
> > are
> > probably
> > very convoluted ways to make this, in this form, a non no-op on Jython
> > too,
> > not that I would like to
> > implement them or that Raymond cared).
>
>Factually false on both counts.
>
>I do care. Jython was a key design consideration from the beginning.
>Guido and I specifically went over the non-CPython implications before
>the PEP was submitted.
>
>The no-op is automatic and makes bind_all equal to lambda *args, **kwds:
>0.  It is invoked only the first time a module is imported.   The time
>difference is unmeasurably miniscule.

from the PEP

"""
The library contains code such as _len=len which is intended to create fast 
local references instead of slower global lookups. Though necessary for 
performance, these constructs clutter the code and are usually incomplete 
(missing many opportunities).

If the proposal is adopted, those constructs could be eliminated from the 
code base and at the same time improve upon their results in terms of 
performance.
"""

if they are eliminated (_len = len), then CPython will win because of the 
binding at load time
and the fact that time is not lost at each function call  for binding them, 
Jython will loose something
because if they were there, there was likely a reason.









From mwh at python.net  Tue Apr 20 06:59:28 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Apr 20 06:59:31 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in  the
	Standard Library
In-Reply-To: <5.1.1.6.0.20040419103223.02a51be0@mail.telecommunity.com>
	(Phillip J. Eby's message of "Mon, 19 Apr 2004 10:39:06 -0400")
References: <5.1.1.6.0.20040419091544.02cc6ce0@mail.telecommunity.com>
	<5.1.1.6.0.20040419103223.02a51be0@mail.telecommunity.com>
Message-ID: <2mr7uiykxb.fsf@starship.python.net>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 09:55 AM 4/19/04 -0400, Raymond Hettinger wrote:
>> > I'm not sure I understand the reasoning for implementing this via a
>> > bytecode hack, rather than in the compiler or loading mechanisms.
>>
>>The reason is that it is not always possible to know (in general) what
>>things constants.  Take a look at the ASPN recipe example, it ought to
>>clarify for you how this works.
>
> I don't need clarification about the mechanism; I independently
> implemented *exactly* this hack myself a couple of years ago.  (I just
> didn't propose it become part of standard Python.)  See 'bind_func()'
> in:
>
> http://cvs.eby-sarna.com/PEAK/src/peak/util/Code.py?rev=1.1.2.1&content-type=text/vnd.viewcvs-markup

Here was me thinking I was the only one nuts enough to implement the
bytecodehacks, and I find everyone's at it!

Cheers,
mwh

-- 
  Academic politics is the most vicious and bitter form of politics,
  because the stakes are so low.                      -- Wallace Sayre

From jacobs at theopalgroup.com  Tue Apr 20 08:04:16 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Tue Apr 20 08:04:20 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838B6@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838B6@escpl.tcp.com.ar>
Message-ID: <408511C0.8000101@theopalgroup.com>

Batista, Facundo wrote:

>[Jewett, Jim J]
>#- Under the current implementation:
>#- 
>#- 	(0, (2, 4, 0, 0, 0), -4) 
>#- 
>#- is not quite the same as
>#- 
>#- 	(0, (2, 4) -1)
>#- 
>#- Given this, is should be possible for the user to specify 
>#- (at creation) which is desired.
>
>It *is* posible:
>  
>
>>>>Decimal('2.4000')
>>>>        
>>>>
>Decimal( (0, (2, 4, 0, 0, 0), -4) )
>  
>
>>>>Decimal('2.4')
>>>>        
>>>>
>Decimal( (0, (2, 4), -1) )
>  
>

<sarcasm>Great!</sarcasm>.   One of my previous posts specifically listed
that I didn't want to have to pre-parse and reformulate string literals to
achieve the desired precision and scale.  The "external" library solution
will either have to do that, or create a Decimal from the string literal,
look at the tuple length and exponent and form a new Decimal instance,
rounding if necessary.  Hmmm... sounds like something low-level
enough that Decimal should be doing it.  And remember, SQL and
many financial applications will be doing this operation for every
datum they load.

>#- I agree that the programmer should specify if they want 
>#- something odd -- and that is precisely why the constructor 
>#- should take an optional context argument which allows the 
>#- user to specify precision.
>
>If you construct using precision, and the precision is smaller than the
>quantity of digits you provide, you'll get rounded, but if the precision is
>greater than the quantity of digits you provide, you don't get filled with
>zeros.
>  
>

Rounding is exactly what should be done if one exceeds the desired 
precision.  Using
less that the desired precision (i.e., not filling in zeros) may be okay 
for many applications.
This is because any operations on the value will have to be performed 
with the precision
defined in the decimal context.  Thus, the results will be identical, 
other than that the
Decimal instance may not store the maximum precision available by the 
schema.

Thanks,
-Kevin


From aleaxit at yahoo.com  Tue Apr 20 08:29:37 2004
From: aleaxit at yahoo.com (Alex Martelli)
Date: Tue Apr 20 08:29:44 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <40843086.9000705@v.loewis.de>
References: <16E1010E4581B049ABC51D4975CEDB8803060E63@UKDCX001.uk.int.atosorigin.com>
	<40843086.9000705@v.loewis.de>
Message-ID: <200404201429.37605.aleaxit@yahoo.com>

On Monday 19 April 2004 10:03 pm, Martin v. L?wis wrote:
> Moore, Paul wrote:
> > I don't know if anyone has stats on how many of the 3rd party
> > extension authors who currently provide Windows binaries have
> > access to MSVC7, and so can make the switch. It may be "all of
> > them". I don't have MSVC7, although I do have MSVC6 (my company,
> > or at least my group, never upgraded) but I've no idea how unusual
> > my situation is. A heads-up on c.l.p would probably be worth it,
> > but it's also likely to generate a huge amount of FUD, and anti-
> > Microsoft rants.
>
> Given that the Python release is still several months ahead,
> I'd advise against such a posting. Personally, I don't know
> a single VC6 user who doesn't also have a copy of VC.NET 2003
> available (except for you, whom I only know via email :-)

Add me to the set of VC6 users who don't have VC.NET 2003 (basically because 
the Windows I run is still Win98 -- as I can run it under Linux with cheap 
win4lin -- and VC.NET doesn't support that old Win version).  Still, I 
suspect my particular reason is rare -- and from this thread, if I understand 
it correctly, I gather that if you do run Win/XP, as most do these days of 
course, it doesn't cost extra money to get a compiler able to build Python 
extensions.


> In any case, the only possible change out of this discussion
> is that Python 2.4 would be built with VC6. I personally know
> a few people which have VC.NET 2003, but not VC6, because you
> cannot purchase the latter one, anymore. So, standardizing
> on .NET 2003 is a good thing: it means that .NET (2002) gets

Yes, if VC6 cannot be purchased then it would be too backwards-looking for 
Python 2.4 to require it from extension-authors.  Presumably, for a Python 
release coming out in 2004, the fact that you cannot build extensions if the 
only Windows version you run is win98 is acceptable (sigh).


Alex


From theller at python.net  Tue Apr 20 08:49:20 2004
From: theller at python.net (Thomas Heller)
Date: Tue Apr 20 08:49:26 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <200404201429.37605.aleaxit@yahoo.com> (Alex Martelli's message
	of "Tue, 20 Apr 2004 14:29:37 +0200")
References: <16E1010E4581B049ABC51D4975CEDB8803060E63@UKDCX001.uk.int.atosorigin.com>
	<40843086.9000705@v.loewis.de> <200404201429.37605.aleaxit@yahoo.com>
Message-ID: <smeyizlb.fsf@python.net>

>> In any case, the only possible change out of this discussion
>> is that Python 2.4 would be built with VC6. I personally know
>> a few people which have VC.NET 2003, but not VC6, because you
>> cannot purchase the latter one, anymore. So, standardizing
>> on .NET 2003 is a good thing: it means that .NET (2002) gets
>
> Yes, if VC6 cannot be purchased then it would be too backwards-looking for 
> Python 2.4 to require it from extension-authors.

It may be that it's no longer possible to buy MSVC6 separately, but it
is still on the MSDN professional DVD set (even 1.5.2, the 16-bit
compiler).

Thomas


From theller at python.net  Tue Apr 20 08:54:54 2004
From: theller at python.net (Thomas Heller)
Date: Tue Apr 20 08:54:59 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <n059mfgb.fsf@yahoo.co.uk> (Paul Moore's message of "Sun, 18
	Apr 2004 17:13:56 +0100")
References: <E1BExVA-0004Be-3Z@mail.python.org> <n059mfgb.fsf@yahoo.co.uk>
Message-ID: <oepmizc1.fsf@python.net>

Paul Moore <pf_moore@yahoo.co.uk> writes:

> "Tim Peters" <tim.one@comcast.net> writes:
>
>> Thanks to David Abrahams for passing on this info:  a free optimizing
>> cmdline VC 7.1 has been released by MS:
>>
>>     http://msdn.microsoft.com/visualc/vctoolkit2003/
>>
>> The 32MB download appears to contain much of the meat of Visual Studio .NET
>> 2003 Professional, including the compiler, linker, static-link libraries for
>> the C, C++ & STL runtimes, and the .NET Common Language Runtime.
>
> Sadly, a test seems to show that the /MD option links with msvcrt.dll,
> and not with msvcr71.dll.

Although it seems *not even* msvcrt.lib is included. The lib directory
of my vctoolkit installation only contains:

  10.08.2002  05:00           191.866 kernel32.lib
  21.02.2003  02:58         2.707.332 libc.lib
  18.03.2003  19:58         3.023.122 libcd.lib
  18.03.2003  19:58            94.208 libcd.pdb
  21.02.2003  03:01         2.937.240 libcmt.lib
  21.02.2003  02:58         3.604.302 libcp.lib
  18.03.2003  19:58         4.433.724 libcpd.lib
  18.03.2003  19:58           225.280 libcpd.pdb
  21.02.2003  03:02         3.654.594 libcpmt.lib
  20.02.2003  19:38            18.618 mscoree.lib
  21.02.2003  02:53            69.512 oldnames.lib

Thomas


From FBatista at uniFON.com.ar  Tue Apr 20 09:28:48 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 20 09:31:13 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>

[Kevin Jacobs]

#- >[Jewett, Jim J]
#- >#- Under the current implementation:
#- >#- 	(0, (2, 4, 0, 0, 0), -4) 
#- >#- is not quite the same as
#- >#- 	(0, (2, 4) -1)
#- >#- Given this, is should be possible for the user to specify 
#- >#- (at creation) which is desired.
#- >
#- >It *is* posible:
#- >
#- >>>>Decimal('2.4000')
#- >Decimal( (0, (2, 4, 0, 0, 0), -4) )
#- >
#- >>>>Decimal('2.4')
#- >Decimal( (0, (2, 4), -1) )
#- >  
#- 
#- <sarcasm>Great!</sarcasm>.   One of my previous posts 
#- specifically listed
#- that I didn't want to have to pre-parse and reformulate 
#- string literals to
#- achieve the desired precision and scale.  The "external" 

<lost> what? </lost> :p

I still don't understand why do you want that.


#- >If you construct using precision, and the precision is 
#- smaller than the
#- >quantity of digits you provide, you'll get rounded, but if 
#- the precision is
#- >greater than the quantity of digits you provide, you don't 
#- get filled with
#- >zeros.
#- 
#- Rounding is exactly what should be done if one exceeds the desired 
#- precision.  Using
#- less that the desired precision (i.e., not filling in zeros) 
#- may be okay 
#- for many applications.
#- This is because any operations on the value will have to be 
#- performed 
#- with the precision
#- defined in the decimal context.  Thus, the results will be 
#- identical, 
#- other than that the
#- Decimal instance may not store the maximum precision 
#- available by the 
#- schema.

If I don't misunderstand, you're saying that store additional zeroes is
important to your future operations?

Let's make an example.

If I have '2.4000', I go into decimal and get:

>>>Decimal('2.4000')
Decimal( (0, (2, 4, 0, 0, 0), -4) )

If I have '2.4', I go into decimal and get:

>>>Decimal('2.4')
Decimal( (0, (2, 4), -1) )

Are you trying to say that you want Decimal to fill up that number with
zeroes...

>>>Decimal('2.4', scale=4)      # behaviour don't intended, just an example
Decimal( (0, (2, 4, 0, 0, 0), -4) )

...just to represent that you have that precision in your measurements and
reflect that in future arithmetic operations?

If yes, I think that: a) '2.4' and '2.4000' will behaviour identically in
future operations;  b) why do you need to represent in the number the
precision of your measurement?

.	Facundo

From jacobs at theopalgroup.com  Tue Apr 20 10:00:40 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Tue Apr 20 10:00:48 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>
Message-ID: <40852D08.2030801@theopalgroup.com>

Batista, Facundo wrote:

>[Kevin Jacobs]
>#- <sarcasm>Great!</sarcasm>.   One of my previous posts 
>#- specifically listed
>#- that I didn't want to have to pre-parse and reformulate 
>#- string literals to
>#- achieve the desired precision and scale.  The "external" 
>
><lost> what? </lost> :p
>
>I still don't understand why do you want that.
>
>  
>
It seems that Jim and I want to be able to easily create Decimal instances
that conform to a pre-specified (maximum) scale and (maximum) precision.
The motivation for this is clearly explained in that section of the 
PostgeSQL
manual that I sent the other day.  i.e., numeric and decimal values in SQL
are specified in terms of scale and precision parameters.  Thus, I would
like to create decimal instances that conform to those schema -- i.e., they
would be rounded appropriately and overflow errors generated if they
exceeded either the maximum precision or scale.  e.g.:

  Decimal('20000.001', precision=4, scale=0) === Decimal('20000')
  Decimal('20000.001', precision=4, scale=0) raises an overflow exception
  Decimal('20000.001', precision=5, scale=3) raises an overflow exception
  Decimal('200.001', precision=6, scale=3) === Decimal('200.001')
  Decimal('200.000', precision=6, scale=3) === Decimal('200') or 
Decimal('200.000')
     (depending on if precision and scale are interpreted as absolutes or
      maximums)

In order to be able to accomplish this behavior in an "external" library,
either the literals would have to be pre-parsed and manipulated, OR
an intermediate Decimal value would be created using the raw literal,
which would then used to detect overflows and then apply the
necessary rounding criteria based on the desired (maximum) scale.

Hopefully this is somewhat clearer.

>If I don't misunderstand, you're saying that store additional zeroes is
>important to your future operations?
>
>  
>
Not for mine.  I would be content with interpreting the scale and 
precision parameters
as maximums rather than absolutes.  However, it is important to poll 
other users,
since their applications may be less forgiving.

>>>>Decimal('2.4', scale=4)      # behaviour don't intended, just an example
>>>>        
>>>>
>Decimal( (0, (2, 4, 0, 0, 0), -4) )
>
>...just to represent that you have that precision in your measurements and
>reflect that in future arithmetic operations?
>
>If yes, I think that: a) '2.4' and '2.4000' will behaviour identically in
>future operations;  b) why do you need to represent in the number the
>precision of your measurement?
>
>  
>
Neither.  It is well understood that operations on Decimal instances 
must rely on the
context.  The idea here is to overflow and round correctly upon instance 
creation without
going through a great deal of additional effort.

Thanks,
-Kevin


From aahz at pythoncraft.com  Tue Apr 20 10:42:35 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr 20 10:42:40 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <40852D08.2030801@theopalgroup.com>
References: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>
	<40852D08.2030801@theopalgroup.com>
Message-ID: <20040420144235.GB8327@panix.com>

On Tue, Apr 20, 2004, Kevin Jacobs wrote:
>
> Neither.  It is well understood that operations on Decimal instances
> must rely on the context.  The idea here is to overflow and round
> correctly upon instance creation without going through a great deal of
> additional effort.

Why do you think this is a "great deal" of effort?  I still have some
trouble understanding why you think this should go into Decimal rather
than being an add-on.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From FBatista at uniFON.com.ar  Tue Apr 20 11:23:05 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Apr 20 11:25:27 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838C3@escpl.tcp.com.ar>

[Kevin Jacobs]

#- Thus, I would
#- like to create decimal instances that conform to those 
#- schema -- i.e., they
#- would be rounded appropriately and overflow errors generated if they
#- exceeded either the maximum precision or scale.  e.g.:
#- 
#-   Decimal('20000.001', precision=4, scale=0) === Decimal('20000')
#-   Decimal('20000.001', precision=4, scale=0) raises an 
#- overflow exception
#-   Decimal('20000.001', precision=5, scale=3) raises an 
#- overflow exception
#-   Decimal('200.001', precision=6, scale=3) === Decimal('200.001')
#-   Decimal('200.000', precision=6, scale=3) === Decimal('200') or 
#- Decimal('200.000')
#-      (depending on if precision and scale are interpreted as 
#- absolutes or
#-       maximums)
#- 
#- In order to be able to accomplish this behavior in an 
#- "external" library,
#- either the literals would have to be pre-parsed and manipulated, OR
#- an intermediate Decimal value would be created using the raw literal,
#- which would then used to detect overflows and then apply the
#- necessary rounding criteria based on the desired (maximum) scale.

I think that is better to include this functionality outside Decimal, by
using it or inheritating it (the same way I'll include all money specific
uses in other class).  Remember that Decimal is a module to implement the
Arithmetic behaviour of decimal floating point.

All other specific use cases should be outside it.


#- Hopefully this is somewhat clearer.

Very, thank you.


#- Neither.  It is well understood that operations on Decimal instances 
#- must rely on the
#- context.  The idea here is to overflow and round correctly 
#- upon instance 
#- creation without
#- going through a great deal of additional effort.

Don't know how much effort it will be. But I prefer that before bloating
Decimal. 

In the PEP I said:  "It is not the purpose of this PEP to have a data type
that can be used as Money without further effort". This is applicable to
other specific uses.

.	Facundo






. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040420/557b709f/attachment.html
From jacobs at theopalgroup.com  Tue Apr 20 11:28:04 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Tue Apr 20 11:28:09 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <20040420144235.GB8327@panix.com>
References: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>	<40852D08.2030801@theopalgroup.com>
	<20040420144235.GB8327@panix.com>
Message-ID: <40854184.7040009@theopalgroup.com>

Aahz wrote:

>On Tue, Apr 20, 2004, Kevin Jacobs wrote:
>  
>
>>Neither.  It is well understood that operations on Decimal instances
>>must rely on the context.  The idea here is to overflow and round
>>correctly upon instance creation without going through a great deal of
>>additional effort.
>>    
>>
>
>Why do you think this is a "great deal" of effort?  I still have some
>trouble understanding why you think this should go into Decimal rather
>than being an add-on.
>  
>

It could be an add-on, but it seems a common and fundamental enough 
operation
that it should be well supported by the core library.  External 
implementations
may be less efficient, as they cannot take advantage of the internal 
implementation
details that a better integrated solution would offer.

This isn't something I am willing to go to war on, but at the same time, I'm
willing to expend some effort to lobby for inclusion.  Either way, I will
have the necessary infrastructure to accomplish my aims, though my
goal is for everyone to have it without re-inventing the wheel.  Silence on
this topic benefits nobody.

Thanks,
-Kevin

From guido at python.org  Tue Apr 20 11:31:25 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Apr 20 11:31:41 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/nondist/peps pep-0329.txt, 1.2, 1.3
In-Reply-To: Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net> 
References: <E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200404201531.i3KFVP612202@guido.python.org>

> Adopt Jack Diederich's suggested module name.

I think pragma.py is a poor name for this, because (a) pragma is a
candidate keyword (it has keyword status in most languages that have
it) and (b) the word pragma implies compiler directives of any kind,
not just the specific function proposed in this PEP.

Also, a heads up: unless this PEP gets a lot more support from folks
whose first name isn't Raymond, I'm going to reject it.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Tue Apr 20 11:33:48 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue Apr 20 11:33:42 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants
	inthe	Standard Library
In-Reply-To: <A016DB92-9230-11D8-BC61-000393CBDF94@opnet.com>
Message-ID: <001901c426ec$e00e7e00$6402a8c0@arkdesktop>

> You could, of course, create a statement like "const len" to flag that
> len will NOT be changed, thus creating true backwards compatibility,

Somehow this idea is getting tangled in my mind with the distinction between
mutable and immutable objects.  When you use an object as a dict key, it
must not change, in order to allow the optimization that keys can be sought
through hashing rather than by sequential search.  Similarly, making a name
such as True immutable allows the optimization that "while True:" can be
determined during compilation to be unconditional.

I understand that there is a difference between the kinds of immutability,
but still there seems to be a strong connection here.


From pje at telecommunity.com  Tue Apr 20 11:58:21 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 20 11:58:32 2004
Subject: [Python-Dev] Re: No-cost optimizing VC 7.1
In-Reply-To: <200404201429.37605.aleaxit@yahoo.com>
References: <40843086.9000705@v.loewis.de>
	<16E1010E4581B049ABC51D4975CEDB8803060E63@UKDCX001.uk.int.atosorigin.com>
	<40843086.9000705@v.loewis.de>
Message-ID: <5.1.1.6.0.20040420115711.02463c70@telecommunity.com>

At 02:29 PM 4/20/04 +0200, Alex Martelli wrote:

>Yes, if VC6 cannot be purchased then it would be too backwards-looking for
>Python 2.4 to require it from extension-authors.  Presumably, for a Python
>release coming out in 2004, the fact that you cannot build extensions if the
>only Windows version you run is win98 is acceptable (sigh).

I'm on Win98SE, and I built extensions with MinGW for Python 2.4, so this 
should not be an issue.  Please see my previous posts.


From jim.jewett at eds.com  Tue Apr 20 11:58:44 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue Apr 20 11:59:04 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants in
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D481@USAHM010.amer.corp.eds.com>

Brett Cannon wrote:

> ``foo = len(x)`` which loads len (global), loads x (local), 
> calls len, and then stores into foo (local).  ...
> storing built-ins into the local namespace to skip having 
> to try load from the global namespace, fail, and then 
> check the built-in namespace 

An in-between option would be to steal an idea from Jython,
and to make (some?) namespace dictionaries different from 
regular dictionaries. Not only do we know what sorts of keys
are possible, we almost know what every key is at compile time.

-jJ

From pje at telecommunity.com  Tue Apr 20 12:01:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 20 12:01:16 2004
Subject: [Python-Dev] PEP 329: Treating Builtins as Constants
	inthe	Standard Library
In-Reply-To: <001901c426ec$e00e7e00$6402a8c0@arkdesktop>
References: <A016DB92-9230-11D8-BC61-000393CBDF94@opnet.com>
Message-ID: <5.1.1.6.0.20040420115907.046a51f0@telecommunity.com>

At 11:33 AM 4/20/04 -0400, Andrew Koenig wrote:
> > You could, of course, create a statement like "const len" to flag that
> > len will NOT be changed, thus creating true backwards compatibility,
>
>Somehow this idea is getting tangled in my mind with the distinction between
>mutable and immutable objects.  When you use an object as a dict key, it
>must not change, in order to allow the optimization that keys can be sought
>through hashing rather than by sequential search.  Similarly, making a name
>such as True immutable allows the optimization that "while True:" can be
>determined during compilation to be unconditional.
>
>I understand that there is a difference between the kinds of immutability,
>but still there seems to be a strong connection here.

I think it's a bad idea to confuse a read-only name binding, and the 
concept of an immutable object.  They aren't the same thing, although the 
former can be implemented by a namespace that is the latter.


From pje at telecommunity.com  Tue Apr 20 12:16:01 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 20 12:16:12 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <200404201531.i3KFVP612202@guido.python.org>
References: <Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
Message-ID: <5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>

At 08:31 AM 4/20/04 -0700, Guido van Rossum wrote:
> > Adopt Jack Diederich's suggested module name.
>
>I think pragma.py is a poor name for this, because (a) pragma is a
>candidate keyword (it has keyword status in most languages that have
>it) and (b) the word pragma implies compiler directives of any kind,
>not just the specific function proposed in this PEP.
>
>Also, a heads up: unless this PEP gets a lot more support from folks
>whose first name isn't Raymond, I'm going to reject it.

Would it be salvageable if it were changed to:

* Get rid of bytecode hacking, in favor of a change to the compiler

* Optimize builtins *only*, and only those that are never assigned to by 
the module

* use a __future__ statement to enable the behavior initially, before 
making it the default in a future release

* have module.__setattr__ warn when shadowing a previously unshadowed 
builtin (unless the module uses the __future__ statement, in which case 
it's an error)

Would this be acceptable?  It seems to me that this approach would allow 
Jython and IronPython the option in future of replacing lookups of builtins 
with static field accesses and/or method calls, which would give them quite 
a potential performance boost.


From guido at python.org  Tue Apr 20 12:50:19 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Apr 20 12:50:27 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: Your message of "Tue, 20 Apr 2004 12:16:01 EDT."
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com> 
References: <Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net> 
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com> 
Message-ID: <200404201650.i3KGoKL12450@guido.python.org>

> Would it be salvageable if it were changed to:
> 
> * Get rid of bytecode hacking, in favor of a change to the compiler
> 
> * Optimize builtins *only*, and only those that are never assigned to by 
> the module
> 
> * use a __future__ statement to enable the behavior initially, before 
> making it the default in a future release
> 
> * have module.__setattr__ warn when shadowing a previously unshadowed 
> builtin (unless the module uses the __future__ statement, in which case 
> it's an error)
> 
> Would this be acceptable?  It seems to me that this approach would
> allow Jython and IronPython the option in future of replacing
> lookups of builtins with static field accesses and/or method calls,
> which would give them quite a potential performance boost.

It is quite the opposite of the PEP!  The PEP proposes a quick, very
visible hack that works only for one implementation; your proposal
here lays the foundation for changing the language to enable the same
kind of optimizations.

I like that much better, but I doubt that it is doable in the
timeframe for 2.4, nor do I think it is needed.  Also, your 4th bullet
proposes exactly (except for the __future__ statement) what was
implemented in moduleobject.c in rev 2.46 and then withdrawn in rev
2.47; it is not feasible for a number of reasons (see python-dev for
the gory details; I don't recall what they were, just that they were
convincing).

The __future__ statement sounds like an excellent idea to me, as it
enables experimentation with the new feature.  One thing: we need to
specify the future behavior very carefully so that other Python
implementations will be able to do the right thing without having to
reverse-engineer CPython.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jim.jewett at eds.com  Tue Apr 20 13:04:44 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue Apr 20 13:05:13 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D484@USAHM010.amer.corp.eds.com>


Kevin Jacobs:
> It is well understood that operations on Decimal instances
> must rely on the context.  The idea here is to overflow and 
> round correctly upon instance creation without going through 
> a great deal of additional effort.

Aahz:

> Why do you think this is a "great deal" of effort? 

(1)
If your data comes from an external source, it makes sense to say
"create a decimal from this" instead of "create a string from this,
then a decimal from that, then another decimal from that with the
right rounding."

(2)
If this is done by inheritance, there will quickly be diamond
inheritance.  With a mixin, that is probably OK, but still not
desirable.

(3)
Is there any reason *not* to honor an optional context when
creating a Decimal?  

The machinery is already there when creating one as the result 
of a calculation.  The machinery is even there to change context 
at arbitrary times -- why not allow it on input/output, which is 
when you are most likely to be dealing with an external data source?

-jJ

From mcherm at mcherm.com  Tue Apr 20 13:23:26 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Apr 20 13:23:31 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <1082481806.40855c8e821a7@mcherm.com>

Kevin Jacobs writes:
> It could be an add-on, but it seems a common and fundamental enough
> operation
> that it should be well supported by the core library.  External
> implementations
> may be less efficient, as they cannot take advantage of the internal
> implementation
> details that a better integrated solution would offer.
>
> This isn't something I am willing to go to war on, but at the same time, I'm
> willing to expend some effort to lobby for inclusion.  Either way, I will
> have the necessary infrastructure to accomplish my aims, though my
> goal is for everyone to have it without re-inventing the wheel.  Silence on
> this topic benefits nobody.

Okay, then I'll speak up too. To me it seems that a way of specifying
the precision INDEPENDENT of the string representation is an unusual
enough need that it does not merit inclusion in Decimal so long as it
is easy to implement as an add-on.

If I receive a bunch of numbers as strings and the precision is
already correct, I can use Decimal(s) to create it. If I receive a
bunch of numbers as fixed position strings and need to set a fixed
precision which doesn't match what the strings contain, then simple
string manipulation will work:

    nums = ["3.14159", "2.71828", "0.57721"]
    # Pad with zeros
    high_precision = [Decimal(num + "000") for num in nums]
    low_precision = [Decimal(num[0:4]) for num in nums]

If I receive a bunch of numbers as variable-length or variable-
precision strings or I receive precision information separately,
then I may need some additional logic to create the Decimal
objects. I consider that to be reasonable.

Again... just one opinion, and I don't mind if you disagree.

-- Michael Chermside





From aahz at pythoncraft.com  Tue Apr 20 13:36:34 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Apr 20 13:36:55 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <40854184.7040009@theopalgroup.com>
References: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>
	<40852D08.2030801@theopalgroup.com>
	<20040420144235.GB8327@panix.com>
	<40854184.7040009@theopalgroup.com>
Message-ID: <20040420173633.GB29268@panix.com>

On Tue, Apr 20, 2004, Kevin Jacobs wrote:
> Aahz wrote:
>>On Tue, Apr 20, 2004, Kevin Jacobs wrote:
>>>
>>>Neither.  It is well understood that operations on Decimal instances
>>>must rely on the context.  The idea here is to overflow and round
>>>correctly upon instance creation without going through a great deal of
>>>additional effort.
>>
>>Why do you think this is a "great deal" of effort?  I still have some
>>trouble understanding why you think this should go into Decimal rather
>>than being an add-on.
> 
> It could be an add-on, but it seems a common and fundamental enough
> operation that it should be well supported by the core library.
> External implementations may be less efficient, as they cannot take
> advantage of the internal implementation details that a better
> integrated solution would offer.
>
> This isn't something I am willing to go to war on, but at the same
> time, I'm willing to expend some effort to lobby for inclusion.
> Either way, I will have the necessary infrastructure to accomplish my
> aims, though my goal is for everyone to have it without re-inventing
> the wheel.  Silence on this topic benefits nobody.

How is your need here more common and fundamental than Money?  I see
here a repeat of the discussions around the new datetime module, where
the decision was made to keep the core implementation dirt-simple, with
enough hooks for people to add functionality.  What makes this case
different?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From pje at telecommunity.com  Tue Apr 20 14:05:44 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 20 14:06:04 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <200404201650.i3KGoKL12450@guido.python.org>
References: <Your message of "Tue, 20 Apr 2004 12:16:01 EDT."
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
Message-ID: <5.1.1.6.0.20040420131828.04232080@telecommunity.com>

At 09:50 AM 4/20/04 -0700, Guido van Rossum wrote:

>It is quite the opposite of the PEP!  The PEP proposes a quick, very
>visible hack that works only for one implementation; your proposal
>here lays the foundation for changing the language to enable the same
>kind of optimizations.
>
>I like that much better, but I doubt that it is doable in the
>timeframe for 2.4, nor do I think it is needed.  Also, your 4th bullet
>proposes exactly (except for the __future__ statement) what was
>implemented in moduleobject.c in rev 2.46 and then withdrawn in rev
>2.47; it is not feasible for a number of reasons (see python-dev for
>the gory details; I don't recall what they were, just that they were
>convincing).

Reviewing the problems I see that the issues are with:

1) extension modules have a problem during initialization if they use setattr

2) modules and subpackages of package, that have names that shadow builtins

#1 is fixable, as long as there is some kind of flag for the module's 
state, which is needed in order to support the __future__ statement 
anyway.  I don't know how this would affect Jython or IronPython, but I 
don't think they really have "extension modules" as such.

#2 is harder, because it needs sane rules.  I don't think the parser should 
have to know about other modules.  But banning modules named after builtins 
isn't appropriate either.

OTOH, the only time this causes ambiguity is when the following conditions 
*all* apply:

  * The module is a package __init__

  * The module contains functions or methods that reference the builtin name

  * The module does not directly import the name

I'm tempted to say that this is broken code, except that it's possible for 
the module to import a module that then imports the module that has the 
conflicting name.

But I believe I have a solution.  See below.


>The __future__ statement sounds like an excellent idea to me, as it
>enables experimentation with the new feature.  One thing: we need to
>specify the future behavior very carefully so that other Python
>implementations will be able to do the right thing without having to
>reverse-engineer CPython.

Here is my proposal for the semantics of "optimized builtins":

* The compiler identifies names in a module that are builtin names (as 
defined by the language version), but are never assigned to (or otherwise 
declared global) in the module.  It adds code at the beginning of the 
module that sets a module-level variable, let's say '__builtins_used__', to 
list those names whose use it *may* be able to optimize.  (Note that this 
step applies to all modules compiled, not just those with the __future__ 
statement, and there is some additional complexity needed in the generated 
code to correctly handle being 'exec'-d in an existing namespace.)

* If the __future__ statement is in effect, also set a 
'__builtins_optimized__' flag in the module dictionary, and actually 
implement any desired optimizations.

* module.__setattr__ either warns or issues an error when setting a name 
listed in '__builtins_used__', depending on the status of 
'__builtins_optimized__'.  If either is missing, the current 
(backward-compatible) semantics of setattr should apply.

Note that if an extension module uses setattr to initialize itself, it will 
not break, because it does not have a '__builtins_used__' attribute.  Also 
note that mere container packages will not break because they contain 
modules or packages named after builtins.  Only packages which actually do 
something with the contained module, while also failing to bind that name, 
will receive warnings.

Such modules can then simply add an explicit e.g.  'import .list' or 
'global list' in the appropriate function(s), or use some similar approach 
to clarify that the named item is a module.  There would be some potential 
pain here when new builtins are added, however, since previously-working 
code could break.

There's a way to fix that too, but it may be a bit harsh.  Issue a warning 
for ambiguous use of global names that are *not* builtin, but are not 
explicitly bound by the module.  That is, if I use the name 'foo' in a 
function, and it is not a local, and is not declared 'global' or explicitly 
bound by module level code (i.e. because I am hacking globals() or because 
the name is a submodule), I should be warned that I should explicitly 
declare my intended usage.  E.g. "Name 'foo' is never assigned a value: 
perhaps you're missing a 'global' declaration or 'import' statement?"

The warning could be introduced as a PendingDeprecationWarning, upgraded to 
a warning for modules using the __future__ statement.  This would then 
discourage writing such ambiguous code in future.  (Oh, and speaking of 
ambiguity, use of 'import *' would either have to be forbidden in optimized 
modules, disable all optimization, or else use setattr and thus break at 
runtime if there's a conflict with an optimized name.)

Whew.  That's quite a list of things that would have to be done, but 
presumably we'll have to pay the piper (pyper?) sometime if we want to get 
to optimized builtin land someday.


From kbk at shore.net  Tue Apr 20 15:39:37 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Apr 20 15:39:48 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200404201939.i3KJdb2H017799@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  265 open (+14) /  2365 closed ( +1) /  2630 total (+15)
Bugs    :  752 open (+14) /  3983 closed (+11) /  4735 total (+25)
RFE     :  134 open ( +2) /   124 closed ( +0) /   258 total ( +2)

New / Reopened Patches
______________________

prefix and exec_prefix as root dir bug  (2004-04-08)
       http://python.org/sf/931938  opened by  Daniel Goertzen 

Implementation for PEP 318 ([as classmethod] version)  (2004-04-08)
       http://python.org/sf/932100  opened by  Mark Russell 

Error caused by patch #852334  (2004-04-10)
       http://python.org/sf/932796  opened by  George Yoshida 

doctest: suggest the use of rawstrings for backslashes  (2004-04-10)
       http://python.org/sf/932930  opened by  Edward Loper 

doctest: Add Tester params to DocTestSuite  (2004-04-10)
       http://python.org/sf/932932  opened by  Edward Loper 

doctest: add an option to end examples w/ dedent  (2004-04-10)
       http://python.org/sf/932933  opened by  Edward Loper 

doctest: allow custom matchers for testing if got==want  (2004-04-10)
       http://python.org/sf/932935  opened by  Edward Loper 

doctest: add a special (dedented) marker for blank lines  (2004-04-11)
       http://python.org/sf/933238  opened by  Edward Loper 

help on re-exported names (bug 925628)  (2004-04-13)
       http://python.org/sf/934356  opened by  Jim Jewett 

platform-specific entropy  (2004-04-13)
       http://python.org/sf/934711  opened by  Trevor Perrin 

trace.py and CR at EOL  (2004-04-14)
CLOSED http://python.org/sf/934971  opened by  Felix Wiemann 

sha256 module  (2004-04-14)
       http://python.org/sf/935454  opened by  Trevor Perrin 

CodeContext - an extension to show you where you are  (2004-04-16)
       http://python.org/sf/936169  opened by  Noam Raphael 

pydoc data descriptor unification  (2004-04-17)
       http://python.org/sf/936774  opened by  John Belmonte 

fast modular exponentiation  (2004-04-17)
       http://python.org/sf/936813  opened by  Trevor Perrin 

Patches Closed
______________

trace.py and CR at EOL  (2004-04-14)
       http://python.org/sf/934971  closed by  montanaro

New / Reopened Bugs
___________________

re seems broken on 64-bit machines/linux  (2004-04-08)
       http://python.org/sf/931848  opened by  Mihai Ibanescu 

Segfault in object_reduce_ex  (2004-04-08)
       http://python.org/sf/931877  opened by  Tim Peters 

f.flush() fails on FreeBSD 5.2  (2004-04-08)
CLOSED http://python.org/sf/931902  opened by  Radim Kolar 

logging: need a way to discard Logger objects  (2004-04-09)
       http://python.org/sf/932563  opened by  Fred L. Drake, Jr. 

#!/usr/bin/python can find wrong python  (2004-04-10)
       http://python.org/sf/932977  opened by  Jack Jansen 

Different behaviour using super() and explicit base class.  (2004-04-10)
       http://python.org/sf/933068  opened by  Lawrence Allan Jones 

pty.fork() leaves slave fd's open on Solaris  (2004-04-12)
       http://python.org/sf/933670  opened by  Scott Lowrey 

configure not able to find ncurses/curses in Solaris   (2004-04-12)
       http://python.org/sf/933795  opened by  stefanparvu 

pydoc.stripid doesn't strip ID  (2004-04-13)
       http://python.org/sf/934282  opened by  Jim Jewett 

nametowidget throws TypeError for Tcl_Objs  (2004-04-13)
       http://python.org/sf/934418  opened by  Greg Couch 

configure fails at getaddrinfo() check on KAME  (2004-04-14)
CLOSED http://python.org/sf/934635  opened by  suz@kame.net 

Strange behavior with '08' and '09'  (2004-04-14)
CLOSED http://python.org/sf/934989  opened by  Pascal 

bdist_winist post-install script fails on non-admin Python  (2004-04-14)
       http://python.org/sf/935091  opened by  David R. Handy 

pkgutil doesn't understand case-senseless filesystems  (2004-04-14)
       http://python.org/sf/935117  opened by  Fred L. Drake, Jr. 

locale dependency of string methods undocumented  (2004-04-16)
       http://python.org/sf/935749  opened by  George Yoshida 

Tix not usable on windows  (2004-04-15)
CLOSED http://python.org/sf/935836  opened by  David Stoner 

os.nullfilename  (2004-04-16)
       http://python.org/sf/935915  opened by  John Belmonte 

XMLFilterBase has bogus startElementNS  (2004-04-17)
       http://python.org/sf/936637  opened by  Magnus Lie Hetland 

PyNumber_And() 's description  (2004-04-17)
CLOSED http://python.org/sf/936827  opened by  [N/A] 

PyNumber_InPlaceDivide()'s description  (2004-04-17)
       http://python.org/sf/936837  opened by  [N/A] 

Py_FilesystemDefaultEncoding leaks  (2004-04-17)
       http://python.org/sf/936915  opened by  Laszlo Toth 

Random.choice doesn't work with sets.  (2004-04-17)
       http://python.org/sf/936988  opened by  Jeremy Fincher 

configure ignores --without-pth  (2004-04-17)
CLOSED http://python.org/sf/937043  opened by  Thomas Linden 

cannot find or replace umlauts  (2004-04-18)
CLOSED http://python.org/sf/937263  opened by  Tünde Kriegl 

minidom.py writes TEXT_NODE wrong  (2004-04-18)
       http://python.org/sf/937282  opened by  Bernd Preusing 

Slicing changes item-attributes of an instance of a UserList  (2004-04-19)
       http://python.org/sf/937700  opened by  Hans Flori 

XMLGenerator ignores encoding in output  (2004-04-19)
       http://python.org/sf/938076  opened by  Magnus Lie Hetland 

Bugs Closed
___________

RecenFilesList mysteriously containing str with NULL bytes  (2004-04-07)
       http://python.org/sf/931336  closed by  kbk

f.flush() fails on FreeBSD 5.2  (2004-04-08)
       http://python.org/sf/931902  closed by  loewis

Make popen raise an exception if cmd doesn't exist  (2004-01-12)
       http://python.org/sf/875471  closed by  nidoizo

configure fails at getaddrinfo() check on KAME  (2004-04-14)
       http://python.org/sf/934635  closed by  perky

Strange behavior with '08' and '09'  (2004-04-14)
       http://python.org/sf/934989  closed by  sjoerd

Tix not usable on windows  (2004-04-15)
       http://python.org/sf/935836  closed by  loewis

Malformed HTML in the FAQ Wizard  (2003-07-27)
       http://python.org/sf/778558  closed by  montanaro

csv.DictReader parms inconsistent with docs  (2003-08-21)
       http://python.org/sf/792558  closed by  montanaro

PyNumber_And() 's description  (2004-04-17)
       http://python.org/sf/936827  closed by  rhettinger

configure ignores --without-pth  (2004-04-17)
       http://python.org/sf/937043  closed by  zarahg

unicodeobjct: bad argument to internal function  (2004-03-27)
       http://python.org/sf/924361  closed by  loewis

cannot find or replace umlauts  (2004-04-18)
       http://python.org/sf/937263  closed by  doerwalter

New / Reopened RFE
__________________

Py_INCREF/DECREF available as macros only  (2004-04-19)
       http://python.org/sf/938302  opened by  Bob Ippolito 


From jeremy at alum.mit.edu  Tue Apr 20 16:50:35 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Apr 20 16:51:28 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
References: <Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
Message-ID: <1082494234.13726.365.camel@localhost.localdomain>

On Tue, 2004-04-20 at 12:16, Phillip J. Eby wrote:
> Would this be acceptable?  It seems to me that this approach would allow 
> Jython and IronPython the option in future of replacing lookups of builtins 
> with static field accesses and/or method calls, which would give them quite 
> a potential performance boost.

This is basically what IronPython does already.  A sufficiently clever
implementation can make the current semantics go fast enough.  Thus,
we'd only need changes to the compiler, not even changes to the
language.  This is the most attractive option to me (see PEP 267).

I don't like the PEP 329 approach because it adds extra complexity to
each module to work around a limitation of the current implementation
that will almost surely disappear one day.

Jeremy



From pje at telecommunity.com  Tue Apr 20 17:21:00 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Apr 20 17:21:14 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <1082494234.13726.365.camel@localhost.localdomain>
References: <5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
Message-ID: <5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>

At 04:50 PM 4/20/04 -0400, Jeremy Hylton wrote:
>On Tue, 2004-04-20 at 12:16, Phillip J. Eby wrote:
> > Would this be acceptable?  It seems to me that this approach would allow
> > Jython and IronPython the option in future of replacing lookups of 
> builtins
> > with static field accesses and/or method calls, which would give them 
> quite
> > a potential performance boost.
>
>This is basically what IronPython does already.  A sufficiently clever
>implementation can make the current semantics go fast enough.

Actually, I suspect the cleverness of implementation required is what has 
kept anybody from actually implementing it for CPython at least.  :)


>   Thus,
>we'd only need changes to the compiler, not even changes to the
>language.  This is the most attractive option to me (see PEP 267).

PEP 267 isn't backwards compatible either, since it requires that "Each 
code object would then be bound irrevocably to the module it was defined 
in."  (And yes, there are lots of not-so-subtle problems with this, such as 
'exec codeobject in dict'.)

I think it would be helpful for the language spec to include the rules for 
potential optimizing of built-ins, and cover what happens if you attempt to 
replace a builtin whose dynamic nature has been optimized away.


>I don't like the PEP 329 approach because it adds extra complexity to
>each module to work around a limitation of the current implementation
>that will almost surely disappear one day.

Agreed.  The thing I'm proposing really needs another PEP.  Unfortunately, 
I need authorship or co-authorship of *another* PEP like I need another 
hole in my head.  :)  (I've been asked to help w/246 and 318 already, and 
have had a web container API pre-PEP on the back burner for a few months now.)


From goodger at python.org  Tue Apr 20 17:22:15 2004
From: goodger at python.org (David Goodger)
Date: Tue Apr 20 17:22:23 2004
Subject: [Python-Dev] Re: python/nondist/peps pep-0000.txt, 1.270,
 1.271 pep-0329.txt, 1.3, 1.4
In-Reply-To: <E1BFzPO-0004OJ-PD@sc8-pr-cvs1.sourceforge.net>
References: <E1BFzPO-0004OJ-PD@sc8-pr-cvs1.sourceforge.net>
Message-ID: <40859487.6050803@python.org>

rhettinger@users.sourceforge.net wrote:
> + Status
> + ======
> + 
> + The PEP is self rejected by the author due to the unprecedented
> + level of enmity it generated from developers.

Oh, there's precedent.  I seem to recall Guido's PEP 259, "Omit
printing newline after newline", getting a lot of flak.  Plenty
of negative feedback from a lot of other PEPs too.  So don't
be so hard on the poor PEP.  :-)

-- David Goodger

From pf_moore at yahoo.co.uk  Tue Apr 20 18:49:51 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Apr 20 18:49:53 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D033838BF@escpl.tcp.com.ar>
	<40852D08.2030801@theopalgroup.com>
	<20040420144235.GB8327@panix.com> <40854184.7040009@theopalgroup.com>
Message-ID: <zn96w9gw.fsf@yahoo.co.uk>

Kevin Jacobs <jacobs@theopalgroup.com> writes:

> This isn't something I am willing to go to war on, but at the same
> time, I'm willing to expend some effort to lobby for inclusion.
> Either way, I will have the necessary infrastructure to accomplish
> my aims, though my goal is for everyone to have it without
> re-inventing the wheel. Silence on this topic benefits nobody.

I thought I'd try to comment, based on this. After all, I do have an
interest in the issue (I'm an Oracle user), and so if there is an
issue, it could well affect me, so I should at least be sure I
understand the point.

... and I discovered that I understand Decimal far less than I thought
I did. But after some experimentation, and reading of the spec, I
think that I've hit on a key point:

    The internal representation of a Decimal instance, and
    specifically the number of digits of precision stored internally,
    has no impact on anything, *except when the instance is converted
    to a string*

The reason for this is that every possible operation on a Decimal
instance uses context, with the sole exception of the "convert to
string" operations (sections 4 and 5 of the spec).

As a result of this, I'm not sure that it's valid to care about the
internal representation of a Decimal instance.

> It seems that Jim and I want to be able to easily create Decimal
> instances that conform to a pre-specified (maximum) scale and
> (maximum) precision.

But here we have that same point - Decimal instances do not "conform
to" a scale/precision.

> The motivation for this is clearly explained in that section of the
> PostgeSQL manual that I sent the other day. i.e., numeric and
> decimal values in SQL are specified in terms of scale and precision
> parameters. Thus, I would like to create decimal instances that
> conform to those schema -- i.e., they would be rounded appropriately
> and overflow errors generated if they exceeded either the maximum
> precision or scale.

OK, so what you are talking about is rounding during construction. Or
is it? Hang on, and let's look at your examples.

> e.g.:

>   Decimal('20000.001', precision=4, scale=0) === Decimal('20000')

This works fine with the current Decimal:

>>> Decimal("20000.001").round(prec=4) == Decimal("20000")
True

Do you dislike the need to construct an exact Decimal, and then round
it? On what grounds? I got the impression that you thought it would be
"hard", but I don't think the round() method is too hard to use...
(Although I would say that the documentation in the PEP is currently
very lacking in its coverage of how to use the type - I found the
round() method after a lot of experimentation. Before the Decimal
module is ready for prime time, it needs some serious documentation
effort).

>   Decimal('20000.001', precision=4, scale=0) raises an overflow
>   exception

Hang on - this example is the same as the previous one, but you want a
different result! In any case, the General Decimal Arithmetic spec
doesn't have a concept of overflow when a precision is exceeded (only
when the implementation-defined maximum exponent is exceeded), so I'm
not sure what you want to happen here in the context of the spec.

>   Decimal('20000.001', precision=5, scale=3) raises an overflow
>   exception

A similar comment abut overflow applies here. I can imagine that you
want to know if information has been lost, but that's no problem -
check like this:

>>> Decimal("20000.001").round(prec=5) == Decimal("20000.001")
False

>   Decimal('200.001', precision=6, scale=3) === Decimal('200.001')

Again, not an issue:

>>> Decimal("200.001").round(prec=6) == Decimal("200.001")
True

>   Decimal('200.000', precision=6, scale=3) === Decimal('200') or
>   Decimal('200.000')
>      (depending on if precision and scale are interpreted as absolutes or
>       maximums)

This doesn't make sense, given that Decimal("200") ==
Decimal("200.000"). Unless your use of === is meant to imply "has the
same internal representation as", in which case I don't believe that
you have a right to care what the internal representation is.

I've avoided considering scale too much here - Decimal has no concept
of scale, only precision. But that's effectively just a matter of
multiplying by the appropriate power of 10, so shouldn't be a major
issue.

Apologies if I've completely misunderstood or misrepresented your
problem here. If it's any consolation, I've learned a lot in the
process of attempting to comment.

Paul.
-- 
This signature intentionally left blank


From jeremy at alum.mit.edu  Tue Apr 20 21:31:13 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Apr 20 21:32:02 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
References: <5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
Message-ID: <1082511073.13726.538.camel@localhost.localdomain>

On Tue, 2004-04-20 at 17:21, Phillip J. Eby wrote:
> At 04:50 PM 4/20/04 -0400, Jeremy Hylton wrote:
> >On Tue, 2004-04-20 at 12:16, Phillip J. Eby wrote:
> > > Would this be acceptable?  It seems to me that this approach would allow
> > > Jython and IronPython the option in future of replacing lookups of 
> > builtins
> > > with static field accesses and/or method calls, which would give them 
> > quite
> > > a potential performance boost.
> >
> >This is basically what IronPython does already.  A sufficiently clever
> >implementation can make the current semantics go fast enough.
> 
> Actually, I suspect the cleverness of implementation required is what has 
> kept anybody from actually implementing it for CPython at least.  :)

I'll grant you it's easier in a language like C#, but I don't think such
code would be excessively complicated.  The implementation needs a few
levels of indirection, but not much more.

> >   Thus,
> >we'd only need changes to the compiler, not even changes to the
> >language.  This is the most attractive option to me (see PEP 267).
> 
> PEP 267 isn't backwards compatible either, since it requires that "Each 
> code object would then be bound irrevocably to the module it was defined 
> in."  (And yes, there are lots of not-so-subtle problems with this, such as 
> 'exec codeobject in dict'.)

I haven't read the text of PEP 267 in a quite a while ;-).  At some
point, we worked out a scheme that was completely backwards compatible. 
There are a couple of realistic options.  One is to update the code at
function definition time to insert the offsets for globals that are
appropriate for the module.  Another option is to initialize a table in
the frame that is used for globals bindings.

> I think it would be helpful for the language spec to include the rules for 
> potential optimizing of built-ins, and cover what happens if you attempt to 
> replace a builtin whose dynamic nature has been optimized away.

I'm not opposed to that, but it isn't required to get good performance
for globals.

> >I don't like the PEP 329 approach because it adds extra complexity to
> >each module to work around a limitation of the current implementation
> >that will almost surely disappear one day.
> 
> Agreed.  The thing I'm proposing really needs another PEP.  Unfortunately, 
> I need authorship or co-authorship of *another* PEP like I need another 
> hole in my head.  :)  (I've been asked to help w/246 and 318 already, and 
> have had a web container API pre-PEP on the back burner for a few months now.)

I know how you feel, even though I've only got one PEP that I'm
currently responsible for.

Jeremy



From tim.one at comcast.net  Tue Apr 20 22:30:21 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Apr 20 22:30:29 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <40852D08.2030801@theopalgroup.com>
Message-ID: <E1BG7VL-0002Cp-Uc@mail.python.org>

[Kevin Jacobs]
...
> Hopefully this is somewhat clearer.

Sorry, it really isn't to me.  When you extract "a number" from one of your
databases, what do you get from it, concretely?  A triple of (decimal string
with embedded decimal point, integer precision, integer scale)?  An integer
value with an integer scale?  A decimal string w/o embedded decimal point
and an integer scale?  Etc.  I gave up then after the first two examples:

> ...
> Thus, I would like to create decimal instances that conform to those
> schema -- i.e., they would be rounded appropriately and overflow errors
> generated if they exceeded either the maximum precision or scale.  e.g.:
> 
>   Decimal('20000.001', precision=4, scale=0) === Decimal('20000')
>   Decimal('20000.001', precision=4, scale=0) raises an overflow exception

The inputs on those two lines look identical to me, so I'm left more lost
than before -- you can't really want Decimal('20000.001', precision=4,
scale=0) *both* to return 20000 *and* raise an overflow exception.

In any case, that's not what the IBM standard supports.  Context must be
respected in its abstract from-string operation, and maximum precision is a
component of context.  If context's precision is 4, then

    from-string('20000.001')

would round to the most-significant 4 digits (according to the rounding mode
specified in context), and signal both the "inexact" and "rounded"
conditions.  What "signal" means:  if the trap-enable flags are set in
context for either or both of those conditions, an exception will be raised;
if the trap-enable flags for both of those conditions are clear, then the
inexact-happened and rounded-happened status flags in context are set, and
you can inspect them or not (as you please).

That's what the standard provides.  More than that would be extensions to
the standard.  The standard is precise about semantics, and it's plenty to
implement (just) all of that at the start.



From tim.one at comcast.net  Tue Apr 20 23:11:13 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Apr 20 23:11:21 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <zn96w9gw.fsf@yahoo.co.uk>
Message-ID: <E1BG88s-0003WB-27@mail.python.org>

[Paul Moore]
> ... and I discovered that I understand Decimal far less than I thought
> I did. But after some experimentation, and reading of the spec, I
> think that I've hit on a key point:
>
>     The internal representation of a Decimal instance, and
>     specifically the number of digits of precision stored internally,
>     has no impact on anything, *except when the instance is converted
>     to a string*
>
> The reason for this is that every possible operation on a Decimal
> instance uses context, with the sole exception of the "convert to
> string" operations (sections 4 and 5 of the spec).
>
> As a result of this, I'm not sure that it's valid to care about the
> internal representation of a Decimal instance.

It's not quite right.  While this is a floating-point arithmetic, it was
designed to "preserve scale" so long as precision isn't exceeded.  Because
you can set precision to anything, this is more powerful than it sounds at
first.  For example, Decimal("20.32") - Decimal("8.02") is displayed as
Decimal("12.30").  That final trailing zero is effectively inherited from
"the scales" of the inputs, and is entirely thanks to that the internal
arithmetic is unnormalized.  More info can be found here:

    http://www2.hursley.ibm.com/decimal/IEEE-cowlishaw-arith16.pdf

    However, if unnormalized floating-point is used with
    sufficient precision to ensure that rounding does not
    occur during simple calculations, then exact scale-preserving
    (type-preserving) arithmetic is possible, and the performance
    and other overheads of normalization are avoided.

and in Cowlishaw's FAQ:

    http://www2.hursley.ibm.com/decimal/decifaq4.html#unnari

    Why is decimal arithmetic unnormalized?
 
> ...
> I've avoided considering scale too much here - Decimal has no concept
> of scale, only precision.

As above, it was carefully designed to *support* apps that need to preserve
scale, but as a property that falls out of a more powerful and more general
arithmetic.

Note that the idea that various legacy apps *agree* on what "preserve scale"
means to begin with is silly -- there's a large variety of mutually
incompatible rules in use (e.g., for multiplication there's at least "the
scale of the result is the sum of the scales of the multiplicands", "is the
larger of the multiplicands' scales", "is the smaller of the multiplicands'
scales", and "is a fixed value independent of the multiplicands' scales").
Decimal provides a single arithmetic capable of emulating all those (and
more), but it's up to the app to use enough precision to begin with, and
rescale according its own bizarre rules.

> But that's effectively just a matter of multiplying by the
> appropriate power of 10, so shouldn't be a major issue.

It can also require rounding, so it's not wholly trivial.  For example,
what's 2.5 * 2.5?  Under the "larger (or smaller) input scale" rules, it's
6.2 under "banker's rounding" or 6.3 under "European, and American tax
rounding" rules.  Decimal won't give either of those directly (unless
precision is set to the ridiculously low 2), but it's easy to get either of
them *using* Decimal (or to get 6.25 directly, which is the least surprising
result to people).



From pf_moore at yahoo.co.uk  Wed Apr 21 03:50:12 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Wed Apr 21 03:50:20 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <zn96w9gw.fsf@yahoo.co.uk> <E1BG88s-0003WB-27@mail.python.org>
Message-ID: <brllhirv.fsf@yahoo.co.uk>

"Tim Peters" <tim.one@comcast.net> writes:

> [Paul Moore]

>> As a result of this, I'm not sure that it's valid to care about the
>> internal representation of a Decimal instance.
>
> It's not quite right.  While this is a floating-point arithmetic, it was
> designed to "preserve scale" so long as precision isn't exceeded.
[...]

Tim,
Thanks for taking the time to clarify this. I'm going to need to think
about this some more to grasp the implications, but I see where it's
intended to apply.

FWIW, my feeling now is that Kevin's requirement is something that can
be handled by a subclass of Decimal, or a class which contains a
Decimal. I'm not convinced by Kevin's suggestion that the operations
needed are "hard" - code complexity can (and should) be encapsulated
inside the subclass, and I don't see the need for runtime
inefficiency. Specifically, I can't see why, if you can first get an
(effectively, according to whatever rules you want to apply) exact
Decimal representation of your "number", you can't do any further
scaling and changing of precision, etc, entirely with Decimal
instances, and with minimal loss of runtime efficiency.

Maybe a concrete example of what Kevin is after (I have a database
background, so I'm happy if it's based around SQL NUMBER datatypes)
would clarify his concerns.

Paul.
-- 
This signature intentionally left blank


From pf_moore at yahoo.co.uk  Wed Apr 21 03:56:02 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Wed Apr 21 03:56:05 2004
Subject: [Python-Dev] Decimal - context manipulation
Message-ID: <7jw9hii5.fsf@yahoo.co.uk>

One thing I couldn't find an obvious way of doing with Decimal is to
obtain a copy of the current context, with a different (say)
precision. The Context constructor defaults from DefaultContext rather
than from the current context, and I can't see a way of modifying the
precision of a Context object (is the "prec" attribute intended to be
publicly writeable?) I assume that copy.copy works to copy a context,
although maybe an explicit copy method would be useful?

Paul.
-- 
This signature intentionally left blank


From FBatista at uniFON.com.ar  Wed Apr 21 09:09:54 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 21 09:12:18 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838D1@escpl.tcp.com.ar>

[Paul Moore]

#- (Although I would say that the documentation in the PEP is currently
#- very lacking in its coverage of how to use the type - I found the
#- round() method after a lot of experimentation. Before the Decimal
#- module is ready for prime time, it needs some serious documentation
#- effort).

This is my fault, maybe:

  - Methods like round() don't need to be discussed: the Spec defines how
them work, and the PEP is for implementing the PEP.

  - The PEP encourages to read the Spec, and round() is detailed in the
Spec.

  - Both code and documentation will be provided when the PEP requests to be
"Final".

Because of these three reasons, there's a lack of definitions yet.

Of course, all of that will be perfectly documented (I'll do my best) when
the PEP champions to Final.

.	Facundo

From pf_moore at yahoo.co.uk  Wed Apr 21 09:56:43 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Wed Apr 21 09:56:46 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D033838D1@escpl.tcp.com.ar>
Message-ID: <8ygpe8o4.fsf@yahoo.co.uk>

"Batista, Facundo" <FBatista@uniFON.com.ar> writes:

> This is my fault, maybe:

I wasn't trying to lay blame! It was more an observation about why I
suspect some of these discussions get bogged down in theory rather
than providing concrete code examples.

[...]

> Of course, all of that will be perfectly documented (I'll do my best) when
> the PEP champions to Final.

If I get the time (sadly, quite unlikely) I'll do what I can to help.

Paul.
-- 
This signature intentionally left blank


From FBatista at uniFON.com.ar  Wed Apr 21 10:20:38 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 21 10:23:05 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838D4@escpl.tcp.com.ar>

[Paul Moore]

#- > This is my fault, maybe:
#- 
#- I wasn't trying to lay blame! It was more an observation about why I
#- suspect some of these discussions get bogged down in theory rather
#- than providing concrete code examples.

I wasn't trying to take the blame neither, ;)

But as an unexperienced python community participant, sometimes don't know
the exact steps to accomplish some things. Anyway, I'll keep asking, and
asking, and asking.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040421/6658111f/attachment.html
From pje at telecommunity.com  Wed Apr 21 10:50:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 21 10:49:46 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <1082511073.13726.538.camel@localhost.localdomain>
References: <5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
Message-ID: <5.1.1.6.0.20040421104017.03c8aec0@mail.telecommunity.com>

At 09:31 PM 4/20/04 -0400, Jeremy Hylton wrote:
>On Tue, 2004-04-20 at 17:21, Phillip J. Eby wrote:
> > I think it would be helpful for the language spec to include the rules for
> > potential optimizing of built-ins, and cover what happens if you 
> attempt to
> > replace a builtin whose dynamic nature has been optimized away.
>
>I'm not opposed to that, but it isn't required to get good performance
>for globals.

I could be wrong, but it seems to me that globals shouldn't be nearly as 
bad for performance as builtins.  A global only does one dict lookup, while 
builtins do two.  Also, builtins can potentially be optimized away 
altogether (e.g. 'while True:') or converted to fast LOAD_CONST, or perhaps 
even a new CALL_BUILTIN opcode, assuming that adding the opcode doesn't 
blow the cacheability of the eval loop.


From FBatista at uniFON.com.ar  Wed Apr 21 11:55:00 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 21 11:57:28 2004
Subject: [Python-Dev] Decimal - context manipulation
Message-ID: <A128D751272CD411BC9200508BC2194D033838D5@escpl.tcp.com.ar>

[Paul Moore]

#- One thing I couldn't find an obvious way of doing with Decimal is to
#- obtain a copy of the current context, with a different (say)
#- precision. The Context constructor defaults from 
#- DefaultContext rather
#- than from the current context, and I can't see a way of modifying the
#- precision of a Context object (is the "prec" attribute intended to be
#- publicly writeable?) I assume that copy.copy works to copy a context,
#- although maybe an explicit copy method would be useful?

>>> from Decimal import *
>>> actual_context = getcontext()
>>> copied_context = copy.copy(actual_context)
>>> copied_context.prec = 5
>>> +Decimal('123456789')
Decimal( (0, (1, 2, 3, 4, 5, 6, 7, 8, 9), 0) )
>>> actual_context.prec = 5
>>> +Decimal('123456789')
Decimal( (0, (1, 2, 3, 4, 6), 4L) )

Note the + in front of the creation of Decimal: that implies an operation
and that's why the context get involved.

Regarding a copy() method in Context class, don't know. It's really easy to
implement, but don't know if the standard preference is to get a copy method
inside each data type (I'm confused about dictionaries having a copy method
and lists, for example, not having it).

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040421/025b75b3/attachment.html
From tim.one at comcast.net  Wed Apr 21 12:13:16 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 12:13:20 2004
Subject: [Python-Dev] Decimal - context manipulation
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838D5@escpl.tcp.com.ar>
Message-ID: <LNBBLJKPBEHFEDALKOLCCELGKGAB.tim.one@comcast.net>

[Batista, Facundo]
> ...
> Regarding a copy() method in Context class, don't know. It's really
> easy to implement, but don't know if the standard preference is to
> get a copy method inside each data type (I'm confused about
> dictionaries having a copy method and lists, for example, not having
> it).

There have always been several succinct ways to get a (shallow) copy of a
list, like

    copied_list = list(original)

and

    copied_list = original[:]

For dicts, while you can do

    copied_dict = dict(original)

today, at the time dict.copy() was introduced you could not do that.

I usually add a .copy() method to my mutable types, like so:

class MyClass:
    ...
    def __copy__(self):
        return build_a_shallow_copy_of(self)
    copy = __copy__

Note that __copy__ is a special name, and the standard copy.copy(thing)
automatically invokes thing.__copy__() if thing has a __copy__() method.
The "copy = __copy__" line then supplies that method directly to class users
too under a conventional name.  So, after the above, copy.copy(thing) and
thing.copy() both work.


From jim.jewett at eds.com  Wed Apr 21 12:28:48 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed Apr 21 12:29:06 2004
Subject: [Python-Dev] peps 329, 266, 267
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D488@USAHM010.amer.corp.eds.com>

Jeremy Hylton:

> I haven't read the text of PEP 267 in a quite a while ;-).  
> At some point, we worked out a scheme that was completely 
> backwards compatible. 

Alas, it wasn't recorded in the PEP.

There are quite a few ways to change shadowing (exec
object in dict, change a base class, etc).  

About all they have in common is that they should be rare.

How about

	from __future__ import noshadow

	<==>

	If a name (or its immediately enclosing class) 
	is explicitly declared dynamic, semantics are 
	the same as today.

	For all other names, the compiler may assume that
	the nearest enclosing binding of this name will 
	always be in the same namespace.  (Names need not
	all be in a single namespace, but once a particular
	name is found, that namespace will always be the 
	correct place to look for that name.)

	Results are undefined if the name is later unbound 
	in that namespace or shadowed by a nearer enclosing
	namespace.

Question:  Should the compiler be able to assume the
same *object* (=can bind as local), or only the same 
namespace (=can do with a single indirection, perhaps
sped up with a variant of DLict).

Question:  Is there any reason that this should apply
only to builtins, rather than to any namespace?

Objection:  Users can do something undefined and get 
"normal" results instead of a warning -- on their own 
platform.  They can even do this strictly through 
changes to other modules which do not themselves 
import noshadow.  How serious is this objection?

If a warning is required, will the bookkeeping be
needed even in release mode, or will it be an option
depending on compiler settings?  Should the compiler
decline to take advantage of its freedom if it finds
a namespace without noshadow somewhere in the lookup
chain?

-jJ

From jim.jewett at eds.com  Wed Apr 21 12:47:27 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed Apr 21 12:48:10 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D489@USAHM010.amer.corp.eds.com>

[Paul Moore]

> I can't see why, if you can first get an (effectively, 
> according to whatever rules you want to apply) exact
> Decimal representation of your "number", you can't do 
> any further scaling and changing of precision, etc, 
> entirely with Decimal instances, and with minimal loss
> of runtime efficiency.

This is effectively saying

(1)  Create a decimal using the default context.
(2)  Change the context to my custom context.
(3)  Perform various rounding and scaling operations.
(4)  Change the context back to the default.

vs

(1)  Create a decimal using my custom context.

The four-step procedure may (or may not) be done just as
efficiently under the covers, but it is ugly.

Is there any reason why input and output should be the only
operations that do not honor an optional local context?

-jJ

From FBatista at uniFON.com.ar  Wed Apr 21 12:54:14 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 21 12:56:43 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838D9@escpl.tcp.com.ar>

[Jewett, Jim J]

#- This is effectively saying
#- 
#- (1)  Create a decimal using the default context.
#- (2)  Change the context to my custom context.
#- (3)  Perform various rounding and scaling operations.
#- (4)  Change the context back to the default.
#- 
#- vs
#- 
#- (1)  Create a decimal using my custom context.
#- 
#- The four-step procedure may (or may not) be done just as
#- efficiently under the covers, but it is ugly.
#- 
#- Is there any reason why input and output should be the only
#- operations that do not honor an optional local context?

I didn't reviewed all the mails to write down the community will, but as far
I recall, you *could* use context in creation time.

I think that still is not clear if to use...

  Decimal(number, [context])

or

  Decimal(number)
  Decimal.using_context(number, [context])

...(I prefer the laters) but I think you could use it.

But: There is no such thing as "scale" in the context.

.	Facundo

From jacobs at theopalgroup.com  Wed Apr 21 12:56:35 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Wed Apr 21 12:56:55 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <E1BG7VL-0002Cp-Uc@mail.python.org>
References: <E1BG7VL-0002Cp-Uc@mail.python.org>
Message-ID: <4086A7C3.90300@theopalgroup.com>

Tim Peters wrote:

>[Kevin Jacobs]
>...
>  
>
>>Hopefully this is somewhat clearer.
>>    
>>
>
>Sorry, it really isn't to me.  When you extract "a number" from one of your
>databases, what do you get from it, concretely?  A triple of (decimal string
>with embedded decimal point, integer precision, integer scale)?  An integer
>value with an integer scale?  A decimal string w/o embedded decimal point
>and an integer scale?  Etc.
>

Sorry for all of the unnecessary confusion.  I am in and out of meetings 
all of this
week and have been trying to keep up several technical conversations in 
5 minute
breaks between sessions.  As such, my examples were flawed.  However, I now
have _10_ minutes, to answer some of your questions, so hopefully I can
explain slightly better.

First, I get decimal numbers from many database adapters, flat files, 
XML files, in
a variety of string formats, mainly.  Virtually all are decimal string 
representations
(i.e., a string of numbers with an option decimal point thrown in 
somewhere).  Not
all of them encode scale explicitly by adding trailing zeros, though 
most of the time
do they conform to a given maximum precision.  A few sources do provide 
decimals
as an integer with an explicit decimal scale exponent.

>>Thus, I would like to create decimal instances that conform to those
>>schema -- i.e., they would be rounded appropriately and overflow errors
>>generated if they exceeded either the maximum precision or scale.  e.g.:
>>
>>  Decimal('20000.001', precision=4, scale=0) === Decimal('20000')
>>  Decimal('20000.001', precision=4, scale=0) raises an overflow exception
>>    
>>
>
>The inputs on those two lines look identical to me, so I'm left more lost
>than before -- you can't really want Decimal('20000.001', precision=4,
>scale=0) *both* to return 20000 *and* raise an overflow exception.
>  
>

Clearly not.  The first example was supposed to have a precision of 5:

   Decimal('20000.001', precision=5, scale=0) === Decimal('20000')


>In any case, that's not what the IBM standard supports.  Context must be
>respected in its abstract from-string operation, and maximum precision is a
>component of context.  If context's precision is 4, then
>
>    from-string('20000.001')
>
>would round to the most-significant 4 digits (according to the rounding mode
>specified in context), and signal both the "inexact" and "rounded"
>conditions.  What "signal" means:  if the trap-enable flags are set in
>context for either or both of those conditions, an exception will be raised;
>if the trap-enable flags for both of those conditions are clear, then the
>inexact-happened and rounded-happened status flags in context are set, and
>you can inspect them or not (as you please).
>  
>

Yes -- this is what I would like to have happen, but with a short-cut to 
support this
common operation.  My previous comment about "great difficulty" was not 
in terms
of the implementation, but rather the number of times it would have to 
be developed
independently, if not readily available.

However, I am still not aware of a trivial way to enforce a given scale 
when creating
decimal instances.  As you point out in a separate e-mail, there are 
many operations
that in effect preserve scale due to unnormalized arithmetic operations.

However, this conversation is somewhat academic since there does not seem
to be a consensus that adding support for construction with scale and 
precision
parameters are of general use.  So I will create my own decimal subclass 
and/or
utility function and be on my merry way.

Thanks,
-Kevin


From FBatista at uniFON.com.ar  Wed Apr 21 12:55:50 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 21 12:58:16 2004
Subject: [Python-Dev] Decimal - context manipulation
Message-ID: <A128D751272CD411BC9200508BC2194D033838DA@escpl.tcp.com.ar>

[Tim Peters]

#- Note that __copy__ is a special name, and the standard 
#- copy.copy(thing)
#- automatically invokes thing.__copy__() if thing has a 
#- __copy__() method.
#- The "copy = __copy__" line then supplies that method 
#- directly to class users
#- too under a conventional name.  So, after the above, 
#- copy.copy(thing) and
#- thing.copy() both work.

I'm +1 on introducing this behaviour to Context.

.	Facundo

From tim.one at comcast.net  Wed Apr 21 13:03:56 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 13:04:03 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838D1@escpl.tcp.com.ar>
Message-ID: <LNBBLJKPBEHFEDALKOLCEELNKGAB.tim.one@comcast.net>

[Batista, Facundo]
> ...
>   - Methods like round() don't need to be discussed: the Spec defines
> how them work, and the PEP is for implementing the PEP.

Actually, there is no round() operation in the spec.  I don't remember
whether there used to be, but there definitely isn't now.  Doesn't mean we
can't supply .round(), does mean we have to spell out what it does.  I
assume decimal.round(whatever) acts the same as the spec's decimal.plus()
would act if "whatever" were temporarily (for the duration of plus()) folded
into context.  If so, that's all it needs to say.


From tim.one at comcast.net  Wed Apr 21 13:12:07 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 13:12:12 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D489@USAHM010.amer.corp.eds.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELOKGAB.tim.one@comcast.net>

[Jewett, Jim J]
> ...
> Is there any reason why input and output should be the only
> operations that do not honor an optional local context?

While all operations in the spec except for the two to-string operations use
context, no operations in the spec support an optional local context.  That
the Decimal() constructor ignores context by default is an extension to the
spec.  We must supply a context-honoring from-string operation to meet the
spec.  I recommend against any concept of "local context" in any
operation -- it complicates the model and isn't necessary.  Piles of
"convenience features" should wait until people actually use this in real
life, so can judge what's truly clumsy based on experience instead of
speculation.


From pje at telecommunity.com  Wed Apr 21 13:48:33 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 21 13:48:49 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D488@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.1.6.0.20040421125924.03199ec0@telecommunity.com>

At 12:28 PM 4/21/04 -0400, Jewett, Jim J wrote:

>How about
>
>         from __future__ import noshadow
>
>         <==>
>
>         If a name (or its immediately enclosing class)
>         is explicitly declared dynamic, semantics are
>         the same as today.

How do you declare that something is dynamic?


>         For all other names, the compiler may assume that
>         the nearest enclosing binding of this name will
>         always be in the same namespace.  (Names need not
>         all be in a single namespace, but once a particular
>         name is found, that namespace will always be the
>         correct place to look for that name.)
>
>         Results are undefined if the name is later unbound
>         in that namespace or shadowed by a nearer enclosing
>         namespace.

This actually isn't that different from my proposal for builtins, except 
that my proposal doesn't have "undefined results".  Instead, names that are 
determined to be builtin are not allowed to be bound via __setattr__, and 
are never looked up in the globals dictionary.



>Question:  Should the compiler be able to assume the
>same *object* (=can bind as local), or only the same
>namespace (=can do with a single indirection, perhaps
>sped up with a variant of DLict).
>
>Question:  Is there any reason that this should apply
>only to builtins, rather than to any namespace?

Simplicity.  Functions today do only three kinds of lookups: LOAD_CELL(?), 
LOAD_FAST and LOAD_GLOBAL.  LOAD_CELL is an indirect load from a known, 
specific nested scope.  LOAD_FAST loads from an array offset into the 
current frame object.  LOAD_GLOBAL checks globals and then 
builtins.  Module-level and class body code use LOAD_NAME, which looks in 
locals, globals, and builtins.

Module and class body code is almost never important for application 
performance, certainly not enough to justify effort on optimization, I think.

So that leaves LOAD_GLOBAL, which I suggest leaving alone in favor of 
adding LOAD_BUILTIN and/or CALL_BUILTIN.  In other words, at that point all 
lookups made by functions *would* be explicit as to scope, since 
LOAD_GLOBAL could only fail its first lookup if the global in question was 
a not-yet-shadowed-but-expected-to-be builtin.



>Objection:  Users can do something undefined and get
>"normal" results instead of a warning -- on their own
>platform.  They can even do this strictly through
>changes to other modules which do not themselves
>import noshadow.  How serious is this objection?

I think my proposal adequate deals with this by disallowing shadowing at 
runtime.  And, if it was desired to disallow shadowing by modifiying 
globals(), then perhaps globals() and module.__dict__ could simply return a 
dictionary proxy that prevented modification of the disallowed 
values.  (But the bare dictionary would still be used by the eval loop.)


>If a warning is required, will the bookkeeping be
>needed even in release mode, or will it be an option
>depending on compiler settings?

The runtime checking in my proposal only requires a bit of work at module 
setup, and checking when a module attribute is set.  Neither of these occur 
frequently enough in general to be concerned about.


From theller at python.net  Wed Apr 21 14:52:34 2004
From: theller at python.net (Thomas Heller)
Date: Wed Apr 21 14:52:43 2004
Subject: [Python-Dev] CVS problems?
Message-ID: <u0zdcgel.fsf@python.net>

Is anyone else having CVS problems, or is it just me?

The problems are: cvs update is very, very slow, or unreliable.  I have
to enter cvs update several times, most of the time I only get 'end of
file from server: see above messages if any' without any messages.

It may be related to the new network connection I have, there have been
some problems with the MTU.  It may be related to that I uploaded a new
public ssh key to sourceforge.  It may be related to that I installed
plink.exe from putty additionally to the ssh.exe I have.  Or is it SF?

Thanks,

Thomas


From jim.jewett at eds.com  Wed Apr 21 15:02:23 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed Apr 21 15:03:11 2004
Subject: [Python-Dev] peps 329, 266, 267
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>

>> For all other names, the compiler may assume that
>> the nearest enclosing binding of this name will
>> always be in the same namespace.  (Names need not
>> all be in a single namespace, but once a particular
>> name is found, that namespace will always be the
>> correct place to look for that name.)

> This actually isn't that different from my proposal
> for builtins, 

I had been assuming that class (and instance) attribute
resolution would be subject to the same speedup.

If this is really only about globals and builtins,
then you can just initialize each module's dictionary
with a copy of builtins.  (Or cache them in the module
__dict__ on the first lookup, since you know where it
would have gone.)  This still won't catch updates to 
builtins, but it will eliminate the failed lookup and 
the second dictionary lookup.  

If you really want to track changes to builtin, it is
still faster to echo builtin changes across each module
than it would be to track every name's referrers in
every module (as in PEP 266.)

> Instead, names that are determined to be builtin are
> not allowed to be bound via __setattr__, and 
> are never looked up in the globals dictionary.

Some of the bugs that got the global tracking backed
out involved changing __builtins__.  If you only add 
to it, then I suppose the current method (which allows 
shadowing) is a reasonable fallback.  It doesn't work 
so well if you want to remove names from builtin.  

In fairness, the language spec does warn that a new 
builtin dict may contain more than you expect, and I
suppose it could be created with extra names pointed
to NotImplemented instead of just raising a NameError.

>>Question:  Is there any reason that this should apply
>>only to builtins, rather than to any namespace?

> Simplicity.  Functions today do only three kinds of 
> lookups: LOAD_CELL(?), LOAD_FAST and LOAD_GLOBAL.  
> LOAD_CELL is an indirect load from a known, specific 
> nested scope.  LOAD_FAST loads from an array offset 
> into the current frame object.  LOAD_GLOBAL checks
> globals and then builtins.  

It could be converted to LOAD_CELL (or perhaps even
LOAD_FAST) if the compiler were allowed to assume no 
changes in shadowing.  (Including an assumption that 
the same dictionaries will continue to represent the 
globals and builtin namespaces for this code object.)

> if it was desired to disallow shadowing by modifiying 
> globals(), then perhaps globals() and module.__dict__ 
> could simply return a dictionary proxy that prevented 
> modification of the disallowed values.  (But the bare
> dictionary would still be used by the eval loop.)

Why not just make NameDict a subclass that has a different
__setitem__?  The times when eval cares should be exactly 
the times when you need to do extra checks.  This also lets 
you use something like DLict (PEP 267) to move most lookup
overhead to compile-time.

-jJ

From pje at telecommunity.com  Wed Apr 21 15:27:01 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 21 15:27:16 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.1.6.0.20040421151250.0308db30@telecommunity.com>

At 03:02 PM 4/21/04 -0400, Jewett, Jim J wrote:

>I had been assuming that class (and instance) attribute
>resolution would be subject to the same speedup.

Nope, sorry, they're entirely unrelated lookups.


>If this is really only about globals and builtins,
>then you can just initialize each module's dictionary
>with a copy of builtins.  (Or cache them in the module
>__dict__ on the first lookup, since you know where it
>would have gone.)

Interesting thought.  The same process that currently loads the 
__builtins__ member could instead update the namespace directly.

There's only one problem with this idea, and it's a big one: 'import *' 
would now include all the builtins, causing one module's builtins (or 
changes thereto) to propagate to other modules.  Yuck.

It's a pity, because I think this would otherwise be an immediately usable 
speedup.  Indeed, if today you were to use something like:

globals().update(
     dict([(k,v) for k,v in __builtins__.__dict__.items() if not 
k.startswith('_')])
)

at the top of a module, you'd get a lot of the speedup benefit.  But you 
really better have an '__all__' definition, then.


>   This still won't catch updates to
>builtins, but it will eliminate the failed lookup and
>the second dictionary lookup.

Actually, I think the language could easily live with the concept that a 
module's builtins are determined at module import time, given that I'm 
pushing for them to be determined at *compile* time.


> > Instead, names that are determined to be builtin are
> > not allowed to be bound via __setattr__, and
> > are never looked up in the globals dictionary.
>
>Some of the bugs that got the global tracking backed
>out involved changing __builtins__.  If you only add
>to it, then I suppose the current method (which allows
>shadowing) is a reasonable fallback.  It doesn't work
>so well if you want to remove names from builtin.

The idea is to simply declare that the any builtin used in a module that's 
known to be a builtin, is allowed to be optimized to the meaning of that 
builtin.  This isn't just for CPython's benefit: Pyrex for example would 
greatly benefit from knowing whether it's safe to consider e.g. 'len()' a 
builtin.

In effect, '__builtins__' should be considered an implementation detail, 
not part of the language, and screwing with it is off limits.  In practice, 
CPython 2.x versions will need to provide backward compatibility because 
there is code out there that "adds new builtins".


> > Simplicity.  Functions today do only three kinds of
> > lookups: LOAD_CELL(?), LOAD_FAST and LOAD_GLOBAL.
> > LOAD_CELL is an indirect load from a known, specific
> > nested scope.  LOAD_FAST loads from an array offset
> > into the current frame object.  LOAD_GLOBAL checks
> > globals and then builtins.
>
>It could be converted to LOAD_CELL (or perhaps even
>LOAD_FAST) if the compiler were allowed to assume no
>changes in shadowing.  (Including an assumption that
>the same dictionaries will continue to represent the
>globals and builtin namespaces for this code object.)

Not without also changing the dictionary type as per PEP 267 or PEP 
280.  LOAD_CELL and LOAD_FAST don't use dictionaries today.


From jack at performancedrivers.com  Wed Apr 21 15:29:41 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Wed Apr 21 15:29:50 2004
Subject: [Python-Dev] CVS problems?
In-Reply-To: <u0zdcgel.fsf@python.net>
References: <u0zdcgel.fsf@python.net>
Message-ID: <20040421192941.GB9062@performancedrivers.com>

On Wed, Apr 21, 2004 at 08:52:34PM +0200, Thomas Heller wrote:
> Is anyone else having CVS problems, or is it just me?
> 
> The problems are: cvs update is very, very slow, or unreliable.  I have
> to enter cvs update several times, most of the time I only get 'end of
> file from server: see above messages if any' without any messages.
>
[snip reasons why it might not be SF]
> Or is it SF?

Its SF.  I get slow & intermittent failures for any project via SSH,
I assume it is worse for pserver but I'm used to that.

-jackdied

From FBatista at uniFON.com.ar  Wed Apr 21 16:03:35 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Apr 21 16:06:00 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838DD@escpl.tcp.com.ar>

[Tim Peters]

#- [Batista, Facundo]
#- > ...
#- >   - Methods like round() don't need to be discussed: the 
#- Spec defines
#- > how them work, and the PEP is for implementing the PEP.
#- 
#- Actually, there is no round() operation in the spec.  I 
#- don't remember
#- whether there used to be, but there definitely isn't now.  

You're right. My fault. 


#- Doesn't mean we
#- can't supply .round(), does mean we have to spell out what 
#- it does.  I
#- assume decimal.round(whatever) acts the same as the spec's 
#- decimal.plus()
#- would act if "whatever" were temporarily (for the duration 
#- of plus()) folded
#- into context.  If so, that's all it needs to say.

Well, I think we must decide how it works

With:

>>> d = Decimal('12345.678')
>>> d
Decimal( (0, (1, 2, 3, 4, 5, 6, 7, 8), -3) )
>>> str(d)
'12345.678'

And being the syntax Decimal.round(n), we have the
following options:

  a) n is the quantity of relevant digits of the
     final number (must be non negative).

     >>> d.round(4)
     Decimal( (0, (1, 2, 3, 5), 1L) )
     >>> str(d.round(4))
     '1.235E+4'

  b) n has the same behaviour that in the built in round().

     >>> d.round(1)
     Decimal( (0, (1, 2, 3, 4, 5, 7), -1L) )
     >>> str(d.round(1))
     '12345.7'
     >>> d.round(-1)
     Decimal( (0, (1, 2, 3, 5), 1L) )
     >>> str(d.round(-1))
     '1.235E+4'

What option do you all like more?

.	Facundo

From barry at python.org  Wed Apr 21 16:14:00 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Apr 21 16:14:05 2004
Subject: [Python-Dev] CVS problems?
In-Reply-To: <u0zdcgel.fsf@python.net>
References: <u0zdcgel.fsf@python.net>
Message-ID: <1082578440.17274.104.camel@geddy.wooz.org>

On Wed, 2004-04-21 at 14:52, Thomas Heller wrote:

> It may be related to the new network connection I have, there have been
> some problems with the MTU.  It may be related to that I uploaded a new
> public ssh key to sourceforge.  It may be related to that I installed
> plink.exe from putty additionally to the ssh.exe I have.  Or is it SF?

It's pretty much SF.

-Barry



From skip at pobox.com  Wed Apr 21 15:59:24 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Apr 21 16:34:58 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>
Message-ID: <16518.53916.22531.235406@montanaro.dyndns.org>


    Jim> If you really want to track changes to builtin, it is still faster
    Jim> to echo builtin changes across each module than it would be to
    Jim> track every name's referrers in every module (as in PEP 266.)

This is an interesting idea but still has a problem (which I think can be
worked around).  Most python apps will rarely create module-level shadows of
builtins, however if they do you have to recognize that fact when
propagating new values out to the modules.  Updating an object in builtins
will look something like:

    def update_builtin_object(name, val):
        "update builtins with new value and update shadows"
        old = getattr(__builtins__, name)
        for modname in sys.modules:
            mod = sys.modules[modname]
            if hasattr(mod, name):
                modglob = getattr(mod, name)
                if modglob != old:
                    # module has a distinct shadow - don't mess w/ it
                    continue
                setattr(mod, name, val)
            else:
                # module doesn't have a copy of name - give it one
                setattr(mod, name, val)

This guarantees that a module's globals will have a copy of each builtin.
If the programmer wants to shadow the original value of a builtin object and
remain immune to changes to builtins, it suffices to copy it into a
different name and use that:

    mylen = __builtins__.len

This gets you halfway there.  The globals dict now has a copy of builtins.
It's necessary to implement something like Jeremy's dlict object to get full
performance boost.

Skip


From jim.jewett at eds.com  Wed Apr 21 16:39:10 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed Apr 21 16:40:09 2004
Subject: [Python-Dev] peps 329, 266, 267
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D48D@USAHM010.amer.corp.eds.com>

>> If this is really only about globals and builtins,
>> then you can just initialize each module's dictionary
>> with a copy of builtins.  (Or cache them in the module
>> __dict__ on the first lookup, since you know where it
>> would have gone.)

Phillip J. Eby:

> Interesting thought.  The same process that currently 
> loads the __builtins__ member could instead update the
> namespace directly.

> There's only one problem with this idea, and it's a big
> one: 'import *' would now include all the builtins,
> causing one module's builtins (or changes thereto) to
> propagate to other modules.  

Why is this bad?

The reason to import * is that you intend to use the
module's members as if they were your own.  If the
other module actually has modified a builtin, you'll
need to do the same, or the imported members won't 
work quite right.

If you're sure you don't want everything the other
module changed, then you shouldn't be using *.

If you absolutely have to use * but not get builtins
changed, then you can reupdate from builtins after
the import.

> ... declare that the any builtin used in a module
> that's known to be a builtin, is allowed to be
> optimized to the meaning of that builtin. 

> In effect, '__builtins__' should be considered an
> implementation detail, not part of the language,

Many builtins (None, True, KeyError) are effectively
keywords, and I would agree.

Others, like __debug__, are really used for 
intermodule communication, because there isn't
any other truly global namespace.  (Perhaps 
there should be a conventional place to look,
such as a settings module?)

-jJ

From pje at telecommunity.com  Wed Apr 21 16:54:34 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 21 16:54:50 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D48D@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.1.6.0.20040421164413.02df82e0@telecommunity.com>

At 04:39 PM 4/21/04 -0400, Jewett, Jim J wrote:
> >> If this is really only about globals and builtins,
> >> then you can just initialize each module's dictionary
> >> with a copy of builtins.  (Or cache them in the module
> >> __dict__ on the first lookup, since you know where it
> >> would have gone.)
>
>Phillip J. Eby:
>
> > Interesting thought.  The same process that currently
> > loads the __builtins__ member could instead update the
> > namespace directly.
>
> > There's only one problem with this idea, and it's a big
> > one: 'import *' would now include all the builtins,
> > causing one module's builtins (or changes thereto) to
> > propagate to other modules.
>
>Why is this bad?

Because some modules are examined by software, and only the expected names 
belong there.  For example, I believe if you run 'pydoc' on such a module, 
it will proceed to document all the builtins.


>The reason to import * is that you intend to use the
>module's members as if they were your own.  If the
>other module actually has modified a builtin, you'll
>need to do the same, or the imported members won't
>work quite right.

*shudder*  I'm glad the language really doesn't work in the way you just 
described.  :)  No, just because one module shadows a builtin, doesn't mean 
you have to follow suit.


> > ... declare that the any builtin used in a module
> > that's known to be a builtin, is allowed to be
> > optimized to the meaning of that builtin.
>
> > In effect, '__builtins__' should be considered an
> > implementation detail, not part of the language,
>
>Many builtins (None, True, KeyError) are effectively
>keywords, and I would agree.
>
>Others, like __debug__, are really used for
>intermodule communication, because there isn't
>any other truly global namespace.  (Perhaps
>there should be a conventional place to look,
>such as a settings module?)

__debug__ is also a builtin, in the sense of being optimizable by the 
compiler, so I don't see any reason to look at it differently.  In fact, 
isn't __debug__ *already* treated as a constant by the compilers?

Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help
 >>> __debug__
1
 >>> __debug__=0
SyntaxError: can not assign to __debug__ (<pyshell#1>, line 1)
 >>>

Yep, I guess so.


From pje at telecommunity.com  Wed Apr 21 17:04:43 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 21 17:04:57 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <16518.53916.22531.235406@montanaro.dyndns.org>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>
Message-ID: <5.1.1.6.0.20040421165458.02dfdec0@telecommunity.com>

At 02:59 PM 4/21/04 -0500, Skip Montanaro wrote:

>     Jim> If you really want to track changes to builtin, it is still faster
>     Jim> to echo builtin changes across each module than it would be to
>     Jim> track every name's referrers in every module (as in PEP 266.)
>
>This is an interesting idea but still has a problem (which I think can be
>worked around).  Most python apps will rarely create module-level shadows of
>builtins, however if they do you have to recognize that fact when
>propagating new values out to the modules.

Please note that:

1. None of this is needed in order to optimize builtins.  Names that are 
unambiguously builtins can and should be optimized at compile time.  Names 
that are ambiguous can use the existing semantics, and nothing further need 
be done.

2. Modifying builtins is a hack, that needs to be supported for backward 
compatibility, but can and should disappear in 3.0.

3. Having official rules for a Python compiler to decide whether a name 
should be considered builtin or global is much more broadly useful than 
having a trick to speed up CPython.  Several projects could make use of 
this, including Pyrex, Jython, PyPy, and IronPython at the least.



From skip at pobox.com  Wed Apr 21 17:36:25 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Apr 21 17:36:00 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <5.1.1.6.0.20040421165458.02dfdec0@telecommunity.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040421165458.02dfdec0@telecommunity.com>
Message-ID: <16518.59737.4238.284709@montanaro.dyndns.org>


    Phillip> Please note that:

    Phillip> 1. None of this is needed in order to optimize builtins.  Names
    Phillip>    that are unambiguously builtins can and should be optimized
    Phillip>    at compile time.  Names that are ambiguous can use the
    Phillip>    existing semantics, and nothing further need be done.

Because module A can modify module B's namespace at runtime the compiler
can't tell whether an access to len() will get the builtin or a shadow
version.  For the above to be true the semantics of modules will have to be
changed.

Skip

From pje at telecommunity.com  Wed Apr 21 18:35:39 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Apr 21 18:36:14 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <16518.59737.4238.284709@montanaro.dyndns.org>
References: <5.1.1.6.0.20040421165458.02dfdec0@telecommunity.com>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D48B@USAHM010.amer.corp.eds.com>
	<5.1.1.6.0.20040421165458.02dfdec0@telecommunity.com>
Message-ID: <5.1.1.6.0.20040421183253.02041660@telecommunity.com>

At 04:36 PM 4/21/04 -0500, Skip Montanaro wrote:

>     Phillip> Please note that:
>
>     Phillip> 1. None of this is needed in order to optimize builtins.  Names
>     Phillip>    that are unambiguously builtins can and should be optimized
>     Phillip>    at compile time.  Names that are ambiguous can use the
>     Phillip>    existing semantics, and nothing further need be done.
>
>Because module A can modify module B's namespace at runtime the compiler
>can't tell whether an access to len() will get the builtin or a shadow
>version.  For the above to be true the semantics of modules will have to be
>changed.

Yes, as I outlined earlier this week: we should disallow modifications of 
this kind if the text of the modified module doesn't explicitly allow it.  See:

http://mail.python.org/pipermail/python-dev/2004-April/044475.html

for details.


From bsder at allcaps.org  Wed Apr 21 19:57:23 2004
From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Wed Apr 21 19:52:53 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838DD@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838DD@escpl.tcp.com.ar>
Message-ID: <20040421163120.N21089@mail.allcaps.org>

On Wed, 21 Apr 2004, Batista, Facundo wrote:

> #- Actually, there is no round() operation in the spec.  I
> #- don't remember
> #- whether there used to be, but there definitely isn't now.
...
> Well, I think we must decide how it works

Careful.  Omissions from technical specifications can be as important as
inclusions.

Why is there no round operation in the spec?  Is it that the functionality
can be implemented using existing operations?  Is it that there is no good
definition of the operation (I find this hard to believe)?  Does it have
some strange side effect that causes problems?

I'm betting that the issue is that the performance of round() is dependent
upon the chosen internal representation (ie. rounding a digit
repesentation is O(1) but rounding a binary integer with scale could be
O(n)).  Removing something which is in use but has this kind of subtle
implication is far harder than adding a function later which got omitted.

For this pass, simply staying with the spec is more than sufficient work.
Work which I heartily thank Facundo for doing.

-a

From tim.one at comcast.net  Wed Apr 21 20:11:24 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 20:11:30 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838DD@escpl.tcp.com.ar>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEOAKGAB.tim.one@comcast.net>

[Batista, Facundo]
> ...
> Well, I think we must decide how it works
>
> With:
>
> >>> d = Decimal('12345.678')
> >>> d
> Decimal( (0, (1, 2, 3, 4, 5, 6, 7, 8), -3) )
> >>> str(d)
> '12345.678'
>
> And being the syntax Decimal.round(n), we have the
> following options:
>
>   a) n is the quantity of relevant digits of the
>      final number (must be non negative).
>
>      >>> d.round(4)
>      Decimal( (0, (1, 2, 3, 5), 1L) )
>      >>> str(d.round(4))
>      '1.235E+4'
>
>   b) n has the same behaviour that in the built in round().
>
>      >>> d.round(1)
>      Decimal( (0, (1, 2, 3, 4, 5, 7), -1L) )
>      >>> str(d.round(1))
>      '12345.7'
>      >>> d.round(-1)
>      Decimal( (0, (1, 2, 3, 5), 1L) )
>      >>> str(d.round(-1))
>      '1.235E+4'
>
> What option do you all like more?

I like (c) best:  drop round(), because it's redundant -- there are other
ways to do (a) and (b) already.  Perhaps (b) isn't obvious, so extending
your example with what already exists:

>>> dimes = Decimal.Decimal('0.1')
>>> print d.quantize(dimes)
12345.7
>>> print d.quantize(Decimal.Decimal('1e1'))
1.235E+4
>>>

As a bonus, the spec defines quantize in all cases (including infinite and
NaN inputs, the effects on the large mass of condition flags, and so on).

In general, we should be very reluctant to extend the spec at the start:
it's complicated, subtle and exacting, and plenty of big brains are working
hard on getting the spec exactly right.  Alas, it's also still a bit of a
moving target.


From tim.one at comcast.net  Wed Apr 21 20:44:34 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 20:44:39 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <brllhirv.fsf@yahoo.co.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEODKGAB.tim.one@comcast.net>

[Paul Moore]
> ...
> FWIW, my feeling now is that Kevin's requirement is something that can
> be handled by a subclass of Decimal, or a class which contains a
> Decimal.

I should emphasize that Decimal has always been intended to be an
implementation of IBM's proposed standard for decimal arithmetic, not to
emulate some arbitrary bag of behaviors picked from a particular database,
or other app (let alone to emulate the union of all schemes in current use).
I'm especially keen to keep it "pure" at the start of its life.  There are
extensions I'd like to see too, but I'm keeping quiet about them for now
(although adding a .copy() method to context objects is clearly harmless).

> I'm not convinced by Kevin's suggestion that the operations needed
> are "hard" - code complexity can (and should) be encapsulated
> inside the subclass, and I don't see the need for runtime
> inefficiency.

The current implementation is so inefficient in so many ways I'm not
concerned about efficiency arguments at all now.  The point is much more to
get the spec implemented correctly at first.

> Specifically, I can't see why, if you can first get an (effectively,
> according to whatever rules you want to apply) exact Decimal
> representation of your "number", you can't do any further
> scaling and changing of precision, etc, entirely with Decimal
> instances, and with minimal loss of runtime efficiency.

Indeed, the individual digits are stored as individual decimal digits right
now, so picking them apart is as cheap as it gets.

> Maybe a concrete example of what Kevin is after (I have a database
> background, so I'm happy if it's based around SQL NUMBER datatypes)
> would clarify his concerns.

That's fine -- although I don't expect it to influence the first release of
Decimal (which is aimed at implementing a specific standard).


From tim.one at comcast.net  Wed Apr 21 21:04:44 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 21:04:51 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <4086A7C3.90300@theopalgroup.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEOEKGAB.tim.one@comcast.net>

[Kevin Jacobs]
> Sorry for all of the unnecessary confusion. ...

No problem.

> First, I get decimal numbers from many database adapters, flat files,
> XML files, in a variety of string formats, mainly.  Virtually all are
> decimal string representations (i.e., a string of numbers with an option
> decimal point thrown in somewhere).

Then what problem are you trying to address when reading these numbers in?
Is it that you don't trust, e.g., that a column of a database declared with
some specific (precision, scale) pair enforced its own restrictions?  Using
Decimal.Decimal(string) exactly as-is today, you'll get exactly whatever
number a string-of-digits-possibly-with-a-decimal-point specifies.

> Not all of them encode scale explicitly by adding trailing zeros, though
> most of the time do they conform to a given maximum precision.  A few
> sources do provide decimals as an integer with an explicit decimal scale
> exponent.

The spec doesn't supply any shortcuts for changing the exponent, because
multiplication and division by powers of 10 are exact (barring underflow and
overflow).  Perhaps a shortcut for that would be handy, but it's not
semantically necessary.

...

> Clearly not.  The first example was supposed to have a precision of 5:
>
>    Decimal('20000.001', precision=5, scale=0) === Decimal('20000')

So you're really doing a data conversion step?  That is, you don't really
want the numbers your data source gives you, but want to transform them
first on input?  You *can*, of course, it just strikes me as an odd desire.


>> In any case, that's not what the IBM standard supports.  Context
>> must be respected in its abstract from-string operation, and maximum
>> precision is a component of context.  If context's precision is 4,
>> then
>>
>>    from-string('20000.001')
>>
>> would round to the most-significant 4 digits (according to the
>> rounding mode specified in context), and signal both the "inexact"
>> and "rounded" conditions.  What "signal" means:  if the trap-enable
>> flags are set in context for either or both of those conditions, an
>> exception will be raised; if the trap-enable flags for both of those
>> conditions are clear, then the inexact-happened and rounded-happened
>> status flags in context are set, and you can inspect them or not (as
>> you please).

> Yes -- this is what I would like to have happen, but with a short-cut
> to support this common operation.

What, exactly, is "this common operation"?  Everything I described in that
paragraph happens automatically as a result of a single from-string
operation.  Is it that you're reading in 10 numbers and want a unique
precision for each one?  That would surprise me.  For example, if you're
reading a column of a database, I'd expect that a single max precision would
apply to each number in that column.

> My previous comment about "great difficulty" was not in terms
> of the implementation, but rather the number of times it would have to
> be developed independently, if not readily available.

Well, I don't know what "it" is, exactly, so I'll shut up.

> However, I am still not aware of a trivial way to enforce a given
> scale when creating decimal instances.

Sorry, I don't even know what "enforce a given scale" *means*.  If, for
example, you want to round every input to the nearest penny, set context to
use the rounding method you mean by "the nearest", define a penny object:

    penny = Decimal("0.01")

and then pass that to the quantize() method on each number:

   to_pennies = [Decimal(n).quantize(penny) for n in input_strings]

Then every result will have exactly two digits after the decimal point.

If for some mysterious reason you actually want to raise an exception if any
information is lost during this step, set context's inexact-trap flag once
at the start.  If you want to raise an exception even if just a trailing 0
is lost, set context's rounding-trap flag once at the start.

> As you point out in a separate e-mail, there are many operations
> that in effect preserve scale due to unnormalized arithmetic
> operations.

Yes.

> However, this conversation is somewhat academic since there does not
> seem to be a consensus that adding support for construction with
> scale and precision parameters are of general use.

Except I'd probably oppose them at this stage even if they were of universal
use:  we're trying to implement a specific standard here at the start.

Note that a construction method that honors context is required by the spec,
and precision is part of context.

> So I will create my own decimal subclass and/or utility function and
> be on my merry way.

Don't forget to share it!  I suspect that's the only way I'll figure out
what you're after <wink>.


From tim.one at comcast.net  Wed Apr 21 21:14:51 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 21:14:57 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838D9@escpl.tcp.com.ar>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEOFKGAB.tim.one@comcast.net>

[Batista, Facundo]
> I didn't reviewed all the mails to write down the community will, but
> as far I recall, you *could* use context in creation time.
>
> I think that still is not clear if to use...
>
>   Decimal(number, [context])
>
> or
>
>   Decimal(number)
>   Decimal.using_context(number, [context])
>
> ...(I prefer the laters) but I think you could use it.

The second way of spelling it is fine, but there's no need to cater to an
optional context argument.  Decimal.using_context(input) should use the
current context object.  It's been a general rule so far that all operations
are available as methods *of* context objects too, so in the goofy case (I
say "goofy" because I predict it won't be used in real life) of wanting to
force use of a particular non-current context object c, the natural spelling
is (or should be)

    c.using_context(input)


From tim.one at comcast.net  Wed Apr 21 21:24:58 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed Apr 21 21:25:03 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <20040421163120.N21089@mail.allcaps.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEOFKGAB.tim.one@comcast.net>

[Andrew P. Lentvorski, Jr.]
> Careful.  Omissions from technical specifications can be as important
> as inclusions.
>
> Why is there no round operation in the spec?  Is it that the
> functionality can be implemented using existing operations?

Yes.  In particular, what most people will want most of the time is exactly
supplied by the spec's quantize() operation (although that may not be
obvious at first).

> Is it that there is no good definition of the operation (I find this hard
> to believe)?

No.

> Does it have some strange side effect that causes problems?

No, although the many revisions quantize()'s definition has been thru bear
witness to how subtle it is to cover every endcase.

> I'm betting that the issue is that the performance of round() is
> dependent upon the chosen internal representation (ie. rounding a
> digit repesentation is O(1) but rounding a binary integer with scale
> could be O(n)).  Removing something which is in use but has this kind
> of subtle implication is far harder than adding a function later
> which got omitted.

Yes, although it's pretty easy to trust that nothing truly essential is
missing from the spec (lots of numeric experts have reviewed it, and that's
still in progress).

> For this pass, simply staying with the spec is more than sufficient
> work.

Indeed, it's a near crushing amount of work!

> Work which I heartily thank Facundo for doing.

Hear, hear!  (For non-English readers, that means "garbanzo magnifico
blotski!" <wink>.)


From Paul.Moore at atosorigin.com  Thu Apr 22 06:00:23 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Thu Apr 22 06:00:28 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB070@UKDCX001.uk.int.atosorigin.com>

From: Tim Peters
>> For this pass, simply staying with the spec is more than sufficient
>> work.
>
> Indeed, it's a near crushing amount of work!
>
>> Work which I heartily thank Facundo for doing.
>
> Hear, hear!  (For non-English readers, that means "garbanzo magnifico
> blotski!" <wink>.)

I agree entirely.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________

From john at neggie.net  Thu Apr 22 08:15:22 2004
From: john at neggie.net (John Belmonte)
Date: Thu Apr 22 08:15:49 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <200404191510.i3JFAxX09635@guido.python.org>
References: <407C28E0.7090808@neggie.net>
	<4080C0C7.7070602@neggie.net>	<200404171733.i3HHX4B30513@guido.python.org>	<40827869.2030300@neggie.net>	<200404181721.i3IHLTp07934@guido.python.org>
	<40833974.10800@neggie.net>
	<200404191510.i3JFAxX09635@guido.python.org>
Message-ID: <4087B75A.3060405@neggie.net>

Guido van Rossum wrote:
> But that currently doesn't work.  Tbe most future-proof solution would
> be to put some kind of object in the dict values whose attributes can
> guide various metaclasses.  Perhaps:
> 
> class slotprop(object):
>     def __init__(self, **kwds):
>         self.__dict__.update(kwds)
> 
> class C(object):
>     __slots__ = {'slot1': slotprop(doc="this is the docstring"),
>                  'slit2': slotprop('doc="another one")}

Wouldn't it be just as future-proof to make __slots__ a list of bare 
slot names or objects?  One advantage is that slot names don't have to 
be carried around externally to the objects.  Moreover, using standard 
attribute names like __name__ and __doc__ will make the solution more 
future-proof.

The default metaclass will handle __slots__ along these lines:

     for item in obj.__slots__:
         if isinstance(item, StringTypes):
             slot_name, slot_doc = item, None
         else:
             slot_name = item.__name__
             slot_doc = getattr(item, '__doc__', None)

In that way, the default metaclass does not impose any restrictions on 
what the slot objects are.

Example usage:

     class MySlot(object):
         def __init__(self, name, doc=None):
             self.__name__ = name
             self.__doc__ = doc

     class C(object):
         __slots__ = ['slot1',
                      MySlot('slot2', 'this is the docstring')]


-John


-- 
http:// ift ile.org/

From goodger at python.org  Thu Apr 22 08:53:36 2004
From: goodger at python.org (David Goodger)
Date: Thu Apr 22 08:53:55 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternative to
	ConfigParser
In-Reply-To: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
References: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
Message-ID: <4087C050.5000106@python.org>

[Full text included for the benefit of python-dev]

Dan Gass wrote:
 > I have written a new module 'config.py' which can be used for
 > similar purposes as ConfigParser.  The biggest difference is that
 > this fetches user configured python objects.  The technique this
 > module supports is well thought out and provides a single solution
 > for the simplest to the most complex configuration problems without
 > sacrificing ease of use and simplicity.  For these reasons I think
 > it should be considered for inclusion into the Python distribution.
 > I work in the field of embedded software development and find this
 > module extremely useful for unit, integration, and functional
 > testing of our software as well as for controlling many other
 > utility scripts.
 >
 > To date I have:
 >
 > 1) provided documentation in the module doc string
 > 2) insured made sure module is well commented and follows the coding
 > guidelines
 > 3) written a test suite to insure functionality is correct (and
 > remains correct)
 >
 > I am willing to do further development of this module, support of
 > the module and take the steps necessary to see it through the PEP
 > process.  I am looking for feedback on whether this module should be
 > considered for inclusion in the standard Python distribution as well
 > as any technical suggestions.  I have placed the GNU license on the
 > module but would be more than happy (and legally able) to relabel it
 > with the license of your choice.
 >
 > Thanks in advance for your consideration!
 >
 > Dan Gass

Hi Dan,

Thanks for your interest.  I'm not sure why, but your message (sent
April 21 around 15:40 UTC) didn't make it to the python-dev list as
addressed.

Typically, a new module first has to prove itself in the real world
before being accepted into the standard library, especially for a
module replacing existing functionality.  It's usually easier to
accept additions/improvements to existing modules (i.e. patches) than
replacement modules.

I suggest you put the module (along with docs and the test suite) up
on the web and make an announcement on python-announce.  Then try to
generate community support.

In any case, if you'd like to write a PEP for it, please read through
PEP 1 and PEP 2.

-- 
David Goodger                    <http://starship.python.net/~goodger>
Python Enhancement Proposal (PEP) Editor <http://www.python.org/peps/>

(Please cc: all PEP correspondence to <peps@python.org>.)

From barry at python.org  Thu Apr 22 08:59:39 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 22 08:59:53 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternative to
	ConfigParser
In-Reply-To: <4087C050.5000106@python.org>
References: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
	<4087C050.5000106@python.org>
Message-ID: <1082638778.22227.27.camel@anthem.wooz.org>

On Thu, 2004-04-22 at 08:53, David Goodger wrote:

> I suggest you put the module (along with docs and the test suite) up
> on the web and make an announcement on python-announce.  Then try to
> generate community support.

You may also want to upload it to PyPi:

http://www.python.org/pypi

-Barry



From john at neggie.net  Thu Apr 22 09:02:30 2004
From: john at neggie.net (John Belmonte)
Date: Thu Apr 22 09:02:57 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternative
	to	ConfigParser
In-Reply-To: <4087C050.5000106@python.org>
References: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
	<4087C050.5000106@python.org>
Message-ID: <4087C266.8080503@neggie.net>

David Goodger wrote:
> Typically, a new module first has to prove itself in the real world
> before being accepted into the standard library, especially for a
> module replacing existing functionality.  It's usually easier to
> accept additions/improvements to existing modules (i.e. patches) than
> replacement modules.
> 
> I suggest you put the module (along with docs and the test suite) up
> on the web and make an announcement on python-announce.  Then try to
> generate community support.

It may also be useful to compare with other config alternatives, such as 
ZConfig.

-John


-- 
http:// ift ile.org/

From barry at python.org  Thu Apr 22 09:10:21 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 22 09:10:31 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternative
	to	ConfigParser
In-Reply-To: <4087C266.8080503@neggie.net>
References: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
	<4087C050.5000106@python.org>  <4087C266.8080503@neggie.net>
Message-ID: <1082639420.22227.41.camel@anthem.wooz.org>

On Thu, 2004-04-22 at 09:02, John Belmonte wrote:

> It may also be useful to compare with other config alternatives, such as 
> ZConfig.

ZConfig takes a completely different approach than ConfigParser; it
rocks but you do need to do more work (define a schema first, then a
configuration file, although I usually do that in the reverse order ;).

I'm hoping someday ZConfig itself (or something close to it) will make
it into the standard library.

-Barry



From lists at hlabs.spb.ru  Thu Apr 22 13:14:58 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Thu Apr 22 09:13:35 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternative
	to	ConfigParser
In-Reply-To: <4087C050.5000106@python.org>
References: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
	<4087C050.5000106@python.org>
Message-ID: <4087FD92.2070005@hlabs.spb.ru>

David Goodger wrote:
> Dan Gass wrote:
>  > I have written a new module 'config.py' which can be used for
>  > similar purposes as ConfigParser.  The biggest difference is that

[SKIP]

> 
> Typically, a new module first has to prove itself in the real world
> before being accepted into the standard library, especially for a
> module replacing existing functionality.  It's usually easier to
> accept additions/improvements to existing modules (i.e. patches) than
> replacement modules.
> 
> I suggest you put the module (along with docs and the test suite) up
> on the web and make an announcement on python-announce.  Then try to
> generate community support.
> 
> In any case, if you'd like to write a PEP for it, please read through
> PEP 1 and PEP 2.

IMO more powerful alternative to ConfigParser is ZConfig 
(http://www.zope.org/Members/fdrake/zconfig/)

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru

From dmgass at hotmail.com  Wed Apr 21 11:41:22 2004
From: dmgass at hotmail.com (Dan Gass)
Date: Thu Apr 22 10:08:00 2004
Subject: [Python-Dev] Proposal: A more powerful alternative to ConfigParser
Message-ID: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>

I have written a new module 'config.py' which can be used for similar 
purposes as ConfigParser.  The biggest difference is that this fetches user 
configured python objects.  The technique this module supports is well 
thought out and provides a single solution for the simplest to the most 
complex configuration problems without sacrificing ease of use and 
simplicity.  For these reasons I think it should be considered for inclusion 
into the Python distribution.  I work in the field of embedded software 
development and find this module extremely useful for unit, integration, and 
functional testing of our software as well as for controlling many other 
utility scripts.

To date I have:

1) provided documentation in the module doc string
2) insured made sure module is well commented and follows the coding 
guidelines
3) written a test suite to insure functionality is correct (and remains 
correct)

I am willing to do further development of this module, support of the module 
and take the steps necessary to  see it through the PEP process.  I am 
looking for feedback on whether this module should be considered for 
inclusion in the standard Python distribution as well as any technical 
suggestions.  I have placed the GNU license on the module but would be more 
than happy (and legally able) to relabel it with the license of your choice.

Thanks in advance for your consideration!

Dan Gass

_________________________________________________________________
MSN Toolbar provides one-click access to Hotmail from any Web page – FREE 
download! http://toolbar.msn.com/go/onm00200413ave/direct/01/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config.py
Type: application/octet-stream
Size: 45066 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040421/511863c2/config-0001.obj
From FBatista at uniFON.com.ar  Thu Apr 22 10:17:58 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 22 10:21:44 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838E5@escpl.tcp.com.ar>

[Tim Peters]

#- [Batista, Facundo]
#- > Well, I think we must decide how it works
#- >
#- > With:
#- >
#- > >>> d = Decimal('12345.678')
#- > >>> d
#- > Decimal( (0, (1, 2, 3, 4, 5, 6, 7, 8), -3) )
#- > >>> str(d)
#- > '12345.678'
#- >
#- > And being the syntax Decimal.round(n), we have the
#- > following options:
#- >
#- >   a) n is the quantity of relevant digits of the
#- >      final number (must be non negative).
#- ...
#- >   b) n has the same behaviour that in the built in round().
#- ...
#- > What option do you all like more?
#- 
#- I like (c) best:  drop round(), because it's redundant -- 
#- there are other
#- ways to do (a) and (b) already.  Perhaps (b) isn't obvious, 
#- so extending
#- your example with what already exists:
#- 
#- >>> dimes = Decimal.Decimal('0.1')
#- >>> print d.quantize(dimes)
#- 12345.7
#- >>> print d.quantize(Decimal.Decimal('1e1'))
#- 1.235E+4
#- >>>

The round() method has the appealing that it could has the same
name/meaning/function that the built in round().

But, the PEP is for implementing the Spec, so I'm +1 to drop round(), at
least in this stage.


#- In general, we should be very reluctant to extend the spec 
#- at the start:
#- it's complicated, subtle and exacting, and plenty of big 
#- brains are working
#- hard on getting the spec exactly right.  Alas, it's also 
#- still a bit of a
#- moving target.

Agree, agree.

.	Facundo

From FBatista at uniFON.com.ar  Thu Apr 22 10:20:52 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 22 10:23:23 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838E6@escpl.tcp.com.ar>

[Tim Peters]

#- The second way of spelling it is fine, but there's no need 
#- to cater to an
#- optional context argument.  Decimal.using_context(input) 
#- should use the
#- current context object.  It's been a general rule so far 
#- that all operations
#- are available as methods *of* context objects too, so in the 
#- goofy case (I
#- say "goofy" because I predict it won't be used in real life) 
#- of wanting to
#- force use of a particular non-current context object c, the 
#- natural spelling
#- is (or should be)
#- 
#-     c.using_context(input)

You mean adding a method to the context, to create a Decimal using itself as
context and not the one from the thread?

If yes, what about c.create_decimal(number) ?

And with floats? c.create_decimal_from_float(number)? Or the same method
that before?

.	Facundo

From FBatista at uniFON.com.ar  Thu Apr 22 10:23:02 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 22 10:25:32 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838E7@escpl.tcp.com.ar>

#- >> Work which I heartily thank Facundo for doing.
#- >
#- > Hear, hear!  (For non-English readers, that means 
#- "garbanzo magnifico
#- > blotski!" <wink>.)
#- 
#- I agree entirely.

Thank you all. Very much.

And specially to Tim, because of heart-defending the stickness of the PEP to
the Spec. As I am a non native english speaker and a number-newbie, you're
extremely helpful.

.	Facundo


From dmgass at hotmail.com  Thu Apr 22 10:19:12 2004
From: dmgass at hotmail.com (Dan Gass)
Date: Thu Apr 22 10:26:28 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternativeto
	ConfigParser
Message-ID: <BAY18-F838WjsifwXBg000065e3@hotmail.com>

>From: Barry Warsaw <barry@python.org>
>Date: Thu, 22 Apr 2004 09:10:21 -0400
>
>On Thu, 2004-04-22 at 09:02, John Belmonte wrote:
>
> > It may also be useful to compare with other config alternatives, such as
> > ZConfig.
>
>ZConfig takes a completely different approach than ConfigParser; it
>rocks but you do need to do more work (define a schema first, then a
>configuration file, although I usually do that in the reverse order ;).
>
>I'm hoping someday ZConfig itself (or something close to it) will make
>it into the standard library.
>
>-Barry

Anyone who has interest in ZConfig is strongly encouraged to look at my 
alternative method as it is alot easier to set up and use and provides more 
flexibility.  I'm hoping to see some cross-pollinating of ideas so your 
feedback would be greatly appreciated.

I will be making an open source project on sourceforge in the near future 
(unless I hear compelling arguments to not) and will make announcements to 
advertise it's existance (any suggestions on posting locations is 
appreciated as well).

Dan

_________________________________________________________________
Watch LIVE baseball games on your computer with MLB.TV, included with MSN 
Premium! 
http://join.msn.com/?page=features/mlb&pgmarket=en-us/go/onm00200439ave/direct/01/


From barry at python.org  Thu Apr 22 10:35:12 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 22 10:44:57 2004
Subject: [Python-Dev] Re: Proposal: A more powerful alternativeto
	ConfigParser
In-Reply-To: <BAY18-F838WjsifwXBg000065e3@hotmail.com>
References: <BAY18-F838WjsifwXBg000065e3@hotmail.com>
Message-ID: <1082644511.22805.7.camel@anthem.wooz.org>

On Thu, 2004-04-22 at 10:19, Dan Gass wrote:

> I will be making an open source project on sourceforge in the near future 
> (unless I hear compelling arguments to not) and will make announcements to 
> advertise it's existance (any suggestions on posting locations is 
> appreciated as well).

Sounds like a good plan!  I doubt anything will get into the stdlib for
Python 2.4, so it's probably a good time for a config bake-off.

-Barry



From fdrake at acm.org  Thu Apr 22 10:32:07 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Apr 22 10:49:35 2004
Subject: [Python-Dev] Proposal: A more powerful alternative to ConfigParser
In-Reply-To: <4087C266.8080503@neggie.net>
References: <BAY18-F96z34lXoUjpT00003dcb@hotmail.com>
	<4087C050.5000106@python.org> <4087C266.8080503@neggie.net>
Message-ID: <200404221032.07697.fdrake@acm.org>

On Thursday 22 April 2004 09:02 am, John Belmonte wrote:
 > It may also be useful to compare with other config alternatives, such as
 > ZConfig.

Hey, I've heard of that!  (Disclaimer: I wrote most of it.)  I just released a 
new version as well.  Information about ZConfig can be found at:

    http://www.zope.org/Members/fdrake/zconfig/

There's a Wiki page that discusses some future plans for it at:

    http://dev.zope.org/Zope3/ZConfig

In particular, I'd be very interested in seeing something like ZConfig in the 
standard library.  Some of my notes in the wiki relate specifically to what's 
needed to make this appropriate.  I'd be very interested in comments from 
people outside Zope Corporation who've tried the package.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From mcherm at mcherm.com  Thu Apr 22 11:47:04 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Apr 22 11:47:08 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <1082648824.4087e8f856292@mcherm.com>

Tim writes:
> But, the PEP is for implementing the Spec, so I'm +1 to drop
> round(), at
> least in this stage.

Me too. But let's promise that the docs will include an example
of how to use quantize() to do rounding, since the name will
not be obvious to the uninitiated.

-- Michael Chermside


From mcherm at mcherm.com  Thu Apr 22 11:50:18 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Apr 22 11:50:24 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <1082649018.4087e9ba25502@mcherm.com>

Facundo writes:
> > #-     c.using_context(input)
>
> You mean adding a method to the context, to create a Decimal
> using itself as
> context and not the one from the thread?
>
> If yes, what about c.create_decimal(number) ?
>
> And with floats? c.create_decimal_from_float(number)? Or the
> same method
> that before?

+1 on Context.create_decimal(float) and Context.create_decimal(string)
   (okay, so it's really just one method that takes two input
   types).

The name just fits my brain. The fact that it uses the context is
obvious from the fact that it's Context method.

-- Michael Chermside


From jeremy at alum.mit.edu  Thu Apr 22 11:56:57 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Apr 22 11:58:12 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <5.1.1.6.0.20040421104017.03c8aec0@mail.telecommunity.com>
References: <5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
	<5.1.1.6.0.20040421104017.03c8aec0@mail.telecommunity.com>
Message-ID: <1082649417.9092.13.camel@localhost.localdomain>

On Wed, 2004-04-21 at 10:50, Phillip J. Eby wrote:
> I could be wrong, but it seems to me that globals shouldn't be nearly as 
> bad for performance as builtins.  A global only does one dict lookup, while 
> builtins do two.  Also, builtins can potentially be optimized away 
> altogether (e.g. 'while True:') or converted to fast LOAD_CONST, or perhaps 
> even a new CALL_BUILTIN opcode, assuming that adding the opcode doesn't 
> blow the cacheability of the eval loop.

The coarse measurements I made a couple of years ago suggest that
LOAD_GLOBAL is still substantially slower than LOAD_FAST.  Less than 100
cycles for LOAD_FAST and about 400 cycles for LOAD_GLOBAL.

http://zope.org/Members/jeremy/CurrentAndFutureProjects/PerformanceMeasurements

It would be good to repeat the measurements with current Python.  I
suspect it's a lot harder to figure out where to measure the start and
stop times.  The timestamps would need to be integrated with PREDICT and
fast_next_opcode, for example.

Jeremy



From aahz at pythoncraft.com  Thu Apr 22 11:59:09 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 22 11:59:15 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838E6@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D033838E6@escpl.tcp.com.ar>
Message-ID: <20040422155909.GA13900@panix.com>

On Thu, Apr 22, 2004, Batista, Facundo wrote:
>
> If yes, what about c.create_decimal(number) ?

Sounds good.  If you're going this route, it seems to me that Decimal()
would work better if it ends up being a proxy class for an internal
implementation that's more closely linked to context, and that most of
the Decimal methods simply delegate to the internal implementation
through context.  (If you look at my original implementation, that's
similar to the direction I was going -- haven't actually looked at your
code to see what it's like currently.)  Make sure that Decimal has-a
internal implementation rather than is-a (i.e. use composition instead
of subclassing).

> And with floats? c.create_decimal_from_float(number)? Or the same method
> that before?

``from_float()`` probably is better.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From FBatista at uniFON.com.ar  Thu Apr 22 12:12:47 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 22 12:16:09 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838EB@escpl.tcp.com.ar>

[Michael Chermside]

#- Me too. But let's promise that the docs will include an example
#- of how to use quantize() to do rounding, since the name will
#- not be obvious to the uninitiated.

I'll do my best with the docs. I have in mind a concept explanation about
Decimal and Context, and syntax and examples of each method.

But let's promise that you'll give me feedback about them, ;)

.	Facundo

From pje at telecommunity.com  Thu Apr 22 12:18:08 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr 22 12:18:31 2004
Subject: [Python-Dev]  Re: [Python-checkins] python/nondist/peps
	pep-0329.txt, 1.2, 1.3
In-Reply-To: <1082649417.9092.13.camel@localhost.localdomain>
References: <5.1.1.6.0.20040421104017.03c8aec0@mail.telecommunity.com>
	<5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<Your message of "Tue, 20 Apr 2004 05:07:36 PDT."
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<E1BFu2K-0002jP-Pg@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040420120134.028b2a60@telecommunity.com>
	<5.1.1.6.0.20040420170833.02781dd0@telecommunity.com>
	<5.1.1.6.0.20040421104017.03c8aec0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040422120308.02aa39a0@telecommunity.com>

At 11:56 AM 4/22/04 -0400, Jeremy Hylton wrote:
>On Wed, 2004-04-21 at 10:50, Phillip J. Eby wrote:
> > I could be wrong, but it seems to me that globals shouldn't be nearly as
> > bad for performance as builtins.  A global only does one dict lookup, 
> while
> > builtins do two.  Also, builtins can potentially be optimized away
> > altogether (e.g. 'while True:') or converted to fast LOAD_CONST, or 
> perhaps
> > even a new CALL_BUILTIN opcode, assuming that adding the opcode doesn't
> > blow the cacheability of the eval loop.
>
>The coarse measurements I made a couple of years ago suggest that
>LOAD_GLOBAL is still substantially slower than LOAD_FAST.  Less than 100
>cycles for LOAD_FAST and about 400 cycles for LOAD_GLOBAL.
>
>http://zope.org/Members/jeremy/CurrentAndFutureProjects/PerformanceMeasurements

I notice the page says 400 cycles "on average" for LOAD_GLOBAL doing "one 
or two dictionary lookups", so I'm curious how many of those were for 
builtins, which in the current scheme are always two lookups.  If it was 
half globals and half builtins, and the dictionary lookup is half the time, 
then having opcodes that know whether to look in globals or builtins would 
drop the time to 266 cycles, which isn't spectacular but is still good at 
only about 3.5 times the bytecode fetch overhead.  If builtins are used 
more frequently than globals, the picture improves still further.

Still, it's very interesting to see that loading a global takes almost as 
much time as calling a function!  That's pretty surprising to me.  I guess 
that's why doing e.g. '_len=len' for code that does a tight loop makes such 
a big difference to performance.  I tend to do that with attribute lookups 
before a tight loop, e.g. 'bar = foo.bar', but I didn't realize that global 
and builtin lookups were almost as slow.


From jim.jewett at eds.com  Thu Apr 22 12:22:22 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu Apr 22 12:23:06 2004
Subject: [Python-Dev] peps 329, 266, 267
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D490@USAHM010.amer.corp.eds.com>

At 04:39 PM 4/21/04 -0400, Jewett, Jim J wrote:
>>>> If this is really only about globals and builtins,
>>>> then you can just initialize each module's dictionary
>>>> with a copy of builtins.

(Note that Raymond's original proposal was much stronger;
instead of just moving builtins to globals, it moved both
builtins and globals into locals.)

>>Phillip J. Eby:
>>> There's only one problem with this idea, and it's a big
>>> one: 'import *' would now include all the builtins,

>> Why is this bad?

> Because some modules are examined by software, and only
> the expected names belong there.  For example, I believe
> if you run 'pydoc' on such a module, it will proceed to
> document all the builtins.

Fair enough.  But that software could compare against
builtins, and do a set-minus.

>> ... If the other module actually has modified a 
>> builtin, you'll need to do the same ...

> ... just because one module shadows a builtin, doesn't
> mean you have to follow suit.

I did phrase that carelessly.  The objects you import
will still use their original module's globals.  You
only need to copy these when you want your own code to
act consistently with the other module.

But note that import * _already_ imports names which 
shadow builtins, so the only real change would be when 
the imported module does *not* shadow a builtin, but 
your own module does (and does so before the import,
perhaps because of an earlier import *).

If you want to protect even this obscure case, import
could be changed to special-case builtins.

By the way, what are people benchmarking to decide that 
builtin lookups are frequent enough to matter?  It seems
intuitively true (and probably for the same handful of
builtins throughout an entire program), but pystone is 
contrived enough not to show much difference compared
to its per-run variability.

-jJ

From FBatista at uniFON.com.ar  Thu Apr 22 12:21:29 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Apr 22 12:23:57 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <A128D751272CD411BC9200508BC2194D033838EC@escpl.tcp.com.ar>

[Aahz]

#- Sounds good.  If you're going this route, it seems to me 
#- that Decimal()
#- would work better if it ends up being a proxy class for an internal
#- implementation that's more closely linked to context, and 
#- that most of
#- the Decimal methods simply delegate to the internal implementation

I'm ok with that. But I'm kinda OO newbie, so: do you have a link to "formal
theory" about this? Some example of something you've done using this design?


#- through context.  (If you look at my original implementation, that's
#- similar to the direction I was going -- haven't actually 
#- looked at your
#- code to see what it's like currently.)  Make sure that Decimal has-a

That's what is WorkRep for?

Actually, I didn't messed up a lot with Decimal.py. I writed a lot of code
in test_Decimal.py, and then added from_float to Deicmal and fixed a lot of
small details: but just to comply with the test cases.

A fundamental redesign of Decimal will be delayed until have all the test
cases finished and working. But I think that will be needed in the seek of
performance.


#- internal implementation rather than is-a (i.e. use 
#- composition instead
#- of subclassing).
#- 
#- > And with floats? c.create_decimal_from_float(number)? Or 
#- the same method
#- > that before?
#- 
#- ``from_float()`` probably is better.

Michael proposed to use the same method. This is different from Decimal
(with all that that implies), but can you trust that the user that uses the
method from context is aware of binary float traps?

You propose another method. Its name is not clear to me:
context.from_float() ?

.	Facundo

  

From pje at telecommunity.com  Thu Apr 22 13:55:07 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Apr 22 13:56:44 2004
Subject: [Python-Dev] peps 329, 266, 267
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D490@USAHM010.amer.cor
	p.eds.com>
Message-ID: <5.1.1.6.0.20040422124756.02bb52c0@telecommunity.com>

At 12:22 PM 4/22/04 -0400, Jewett, Jim J wrote:
>At 04:39 PM 4/21/04 -0400, Jewett, Jim J wrote:
> >>>> If this is really only about globals and builtins,
> >>>> then you can just initialize each module's dictionary
> >>>> with a copy of builtins.
>
>(Note that Raymond's original proposal was much stronger;
>instead of just moving builtins to globals, it moved both
>builtins and globals into locals.)

But at least it was still backward-compatible with respect to what actually 
is found in the module's globals.


> >>Phillip J. Eby:
> >>> There's only one problem with this idea, and it's a big
> >>> one: 'import *' would now include all the builtins,
>
> >> Why is this bad?
>
> > Because some modules are examined by software, and only
> > the expected names belong there.  For example, I believe
> > if you run 'pydoc' on such a module, it will proceed to
> > document all the builtins.
>
>Fair enough.  But that software could compare against
>builtins, and do a set-minus.

It *could*, but it's wasteful to break such programs without necessity.


>But note that import * _already_ imports names which
>shadow builtins, so the only real change would be when
>the imported module does *not* shadow a builtin, but
>your own module does (and does so before the import,
>perhaps because of an earlier import *).
>
>If you want to protect even this obscure case, import
>could be changed to special-case builtins.

I think you're missing the part where builtins change from one release to 
the next.  I might have had a global named 'zip' or 'enumerate' in a Python 
1.5 program, and when I upgrade to 2.4 I am now "shadowing a builtin".


From jim.jewett at eds.com  Thu Apr 22 14:42:59 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu Apr 22 14:44:07 2004
Subject: [Python-Dev] peps 329, 266, 267
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D493@USAHM010.amer.corp.eds.com>

Jim:
>> note that import * _already_ imports names which
>> shadow builtins, so the only real change would be
>> when the imported module does *not* shadow a
>> builtin, but your own module does (and does so
>> before the import, perhaps because of an earlier
>> import *).

Phillip J. Eby:
> I think you're missing the part where builtins
> change from one release to the next.  I might have
> had a global named 'zip' or 'enumerate' in a Python 
> 1.5 program, and when I upgrade to 2.4 I am now 
> "shadowing a builtin".

How did you get that global?  You either defined it
or imported it (or had it added by another module).
Any of these actions will still work, and will still
replace the cached builtin.  

The only problem is if you then do an import * _after_ 
that definition.  Doing that was already risking a break; 
this change just makes the break a bit more likely.

-jJ

From tcdelaney at optusnet.com.au  Thu Apr 22 17:20:46 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Thu Apr 22 17:20:59 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <A128D751272CD411BC9200508BC2194D033838E5@escpl.tcp.com.ar>
Message-ID: <009f01c428af$ad5abf90$a100a8c0@ryoko>

Batista, Facundo wrote:

> The round() method has the appealing that it could has the same
> name/meaning/function that the built in round().

Perhaps the built-in round should be modified to work with Decimal ...

Tim Delaney



From tim.one at comcast.net  Thu Apr 22 19:12:07 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Apr 22 19:12:17 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838E5@escpl.tcp.com.ar>
Message-ID: <E1BGnMb-00027i-Rn@mail.python.org>

[Batista, Facundo]
> The round() method has the appealing that it could has the same
> name/meaning/function that the built in round().

As Tim Delaney said later, the best way to do that would be to change the
implementation of Python's builtin round().  I don't think changing Python
(beyond adding the new module) is in scope for the first release, though.



From tim.one at comcast.net  Thu Apr 22 19:20:22 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Apr 22 19:20:32 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <A128D751272CD411BC9200508BC2194D033838E6@escpl.tcp.com.ar>
Message-ID: <E1BGnUb-0001Rk-1S@mail.python.org>

[Batista, Facundo]
> You mean adding a method to the context, to create a Decimal using itself
> as context and not the one from the thread?

Yes.  This isn't novel:  all the operations are like that already.  For
example, Decimal.__abs__() is also available as Context.abs(decimal).  The
semantic difference is that the Decimal methods implicitly use the current
thread's current context object, while the Context methods (trivially) use
an explicitly specified context object.

> If yes, what about c.create_decimal(number) ?

I think the name's kinda wordy.  What else would a Decimal.Context object
want to create, other than a Decimal.Decimal?

> And with floats? c.create_decimal_from_float(number)? Or the same method
> that before?

I'd be happy with

    c.from_string(a_string)
    c.from_int(an_int_or_long) # and works identically to
                               # c.from_string(str(an_int_or_long))
    c.from_float(a_float)

I have no intention of creating Decimals from floats, so I didn't pay
attention to those arguments (other than to silently agree that the issues
involved are ugly enough that float->Decimal should have its own spelling --
if we have to have it at all).



From tim.one at comcast.net  Thu Apr 22 19:32:39 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Apr 22 19:32:51 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <1082648824.4087e8f856292@mcherm.com>
Message-ID: <E1BGngX-0007b6-2v@mail.python.org>

[Michael Chermside]
> Me too. But let's promise that the docs will include an example
> of how to use quantize() to do rounding, since the name will
> not be obvious to the uninitiated.

I agree it won't.  In some ways it's nicer than round(), because it works by
providing an example of the scale you want, not by making you count
positions from the decimal point.

BTW, note that the spec has nothing to say about how an implementation
spells anything, except for the results of the to-string operations.  That
is, there's no requirement to *call* the spec's quantize operation
"quantize"; the requirement is to provide something with the semantics of
the spec's quantize operation.  For example, round_like() or rescale() might
be friendlier names (in fact, the current quantize used to be called rescale
in the spec).



From tcdelaney at optusnet.com.au  Thu Apr 22 19:33:01 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Thu Apr 22 19:33:15 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <E1BGnMb-00027i-Rn@mail.python.org>
Message-ID: <00be01c428c2$29463550$a100a8c0@ryoko>

Tim Peters wrote:

> [Batista, Facundo]
>> The round() method has the appealing that it could has the same
>> name/meaning/function that the built in round().
>
> As Tim Delaney said later, the best way to do that would be to change
> the
> implementation of Python's builtin round().  I don't think changing
> Python (beyond adding the new module) is in scope for the first
> release, though.

I think we probably want a supplemental PEP for upgrading the `math` module
plus appropriate builtins (like round) to handle Decimal. A quick scan
through the docs reveals

- All the `math` module (thin wrappers for float, hand-written code for
Decimal I guess).
- abs
- divmod
- pow
- round

which look like candidates.

Tim Delaney




From tim.one at comcast.net  Thu Apr 22 19:42:43 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Apr 22 19:42:53 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <20040422155909.GA13900@panix.com>
Message-ID: <E1BGnqD-0007Ef-Jb@mail.python.org>

[Aahz]
> Sounds good.  If you're going this route, it seems to me that Decimal()
> would work better if it ends up being a proxy class for an internal
> implementation that's more closely linked to context, and that most of
> the Decimal methods simply delegate to the internal implementation
> through context.

The current implementation is approximately the opposite:  most of the
Decimal methods have an optional "context" argument, and most the Context
methods delegate to Decimal methods (passing context=self).  That's fine by
me, and I don't see a compelling reason to turn it inside out at this stage.
Both classes derive from (only) object.



From tim.one at comcast.net  Thu Apr 22 19:52:50 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Apr 22 19:53:00 2004
Subject: [Python-Dev] Re: Decimal data type issues
In-Reply-To: <009f01c428af$ad5abf90$a100a8c0@ryoko>
Message-ID: <E1BGo01-0005Pn-P8@mail.python.org>

[Tim Delaney]
> Perhaps the built-in round should be modified to work with Decimal ...

Eventually, it should -- and many other changes throughout Python, outside
of Decimal, should also be made.  That won't happen for the first release,
though, and possibly never if Decimal doesn't become popular.

Examples:

+ Float format codes, like %20.6g, make fine sense for Decimal, but
  feeding them Decimals won't do any good.

+ A Decimal version of the math module doesn't exist.

+ Looking up context all the time is expensive, and should eventually
  become part of CPython's C-level thread state (much as your FPU's
  context register(s) are part of your C's runtime thread state).



From edloper at gradient.cis.upenn.edu  Thu Apr 22 21:35:49 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Thu Apr 22 21:34:17 2004
Subject: [Python-Dev] __docformat__ scope
Message-ID: <408872F5.6030000@gradient.cis.upenn.edu>

Pep 258 [1] specifies that the __docformat__ variable should be used to 
define the markup language used by docstrings in a module.

Recently, I got an RFE for epydoc requesting that a __docformat__ in a 
package's __init__.py apply to the entire package [2].  My first 
reaction was that it was a good idea, but then I remembered a recent 
discussion on python-dev, where there seemed to be a strong sentiment 
that "from __future__ import absolute_import" should *not* have package 
scope [3]; and I wondered if some of the same arguments would go against 
  applying __docformat__ to the entire package.

So...  Should __docformat__ have package scope (with submodules/ 
subpackages possibly overriding it), or should it only apply to the 
actual module that defines it?

Either way, PEP 258 should be updated to reflect the decision.

-Edward

[1] pep 258 (Docutils Design Specification)
     http://www.python.org/peps/pep-0258.html

[2] epydoc RFE: __docformat__ for entire packages
 
http://sourceforge.net/tracker/index.php?func=detail&aid=928058&group_id=32455&atid=405621

[3] http://mail.python.org/pipermail/python-dev/2004-April/044201.html


From guido at python.org  Thu Apr 22 22:15:13 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr 22 22:15:56 2004
Subject: [Python-Dev] __docformat__ scope
In-Reply-To: Your message of "Thu, 22 Apr 2004 21:35:49 EDT."
	<408872F5.6030000@gradient.cis.upenn.edu> 
References: <408872F5.6030000@gradient.cis.upenn.edu> 
Message-ID: <200404230215.i3N2FDB17450@guido.python.org>

> Pep 258 [1] specifies that the __docformat__ variable should be used to 
> define the markup language used by docstrings in a module.
> 
> Recently, I got an RFE for epydoc requesting that a __docformat__ in a 
> package's __init__.py apply to the entire package [2].  My first 
> reaction was that it was a good idea, but then I remembered a recent 
> discussion on python-dev, where there seemed to be a strong sentiment 
> that "from __future__ import absolute_import" should *not* have package 
> scope [3]; and I wondered if some of the same arguments would go against 
>   applying __docformat__ to the entire package.
> 
> So...  Should __docformat__ have package scope (with submodules/ 
> subpackages possibly overriding it), or should it only apply to the 
> actual module that defines it?
> 
> Either way, PEP 258 should be updated to reflect the decision.

The case isn't quite the same, but still, I think it would be better
to be required to repeat this info in each module -- more explicit,
and we know that explicit is better than implicit.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From Paul.Moore at atosorigin.com  Fri Apr 23 05:42:31 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Fri Apr 23 05:42:33 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060E72@UKDCX001.uk.int.atosorigin.com>

From: Tim Delaney
> I think we probably want a supplemental PEP for upgrading the `math`
> module plus appropriate builtins (like round) to handle Decimal. A
> quick scan through the docs reveals

While I'm sure the numerical properties are iffy, just how bad is using
binary FP as an intermediate when calculating transcendental functions
for Decimal values?

I'm thinking of something like

    d = Decimal(...)
    f = math.sin(float(d))
    # Now convert back using exact float->Decimal, followed by
    # context-respecting "+" operation
    result = +Decimal.from_float(f)

I suspect that this is adequate for "casual" users. And I'm not at all
sure that the target audience for a Decimal type really cares about
transcendental functions...

(I think that boils down to claiming YAGNI, at least in the first
instance).

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________

From mcherm at mcherm.com  Fri Apr 23 09:10:10 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Apr 23 09:10:13 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <1082725810.408915b28649c@mcherm.com>

Tim Peters writes:
> BTW, note that the spec has nothing to say about how an implementation
> spells anything, except for the results of the to-string operations.  That
> is, there's no requirement to *call* the spec's quantize operation
> "quantize"; the requirement is to provide something with the semantics of
> the spec's quantize operation.

Without having given it a LOT of thought, I'm in favor of calling it
"quantize" just to match the name in the spec. Without deciding it yet,
there are good reasons to think we MIGHT want the behavior of "round"
to be somewhat different (eg: match the behavior of Python's round())
if that were supported in the future.

> For example, round_like() or rescale() might
> be friendlier names (in fact, the current quantize used to be called rescale
> in the spec).

Yes, but there are ALSO notes in the spec explaining that they changed
the name from rescale to quantize because people had misleading
preconceptions about what "rescale" would do. (Partly because it worked
differently in previous versions of the spec.)

-- Michael Chermside


From tcdelaney at optusnet.com.au  Fri Apr 23 09:27:37 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Fri Apr 23 09:27:46 2004
Subject: [Python-Dev] Re: Decimal data type issues
References: <16E1010E4581B049ABC51D4975CEDB8803060E72@UKDCX001.uk.int.atosorigin.com>
Message-ID: <002c01c42936$beaf2a60$a100a8c0@ryoko>

Moore, Paul wrote:

> While I'm sure the numerical properties are iffy, just how bad is
> using binary FP as an intermediate when calculating transcendental
> functions for Decimal values?

Ignoring other issues, what happens when you exceed the limits for float?
Throw an exception, or change to another code path (int/long)? If the
latter, might as well just use it in all cases - makes it more accurate and
less complicated.

> I suspect that this is adequate for "casual" users. And I'm not at all
> sure that the target audience for a Decimal type really cares about
> transcendental functions...

Might be surprised, but there's time to work on it. See next point.

> (I think that boils down to claiming YAGNI, at least in the first
> instance).

Most definitely not going to be in the first instance, as has been stated
before. Python 2.5+ - and then only after a PEP that's gone through as
rigorous a process as this one IMO.

Tim Delaney




From jim.jewett at eds.com  Fri Apr 23 11:50:28 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Fri Apr 23 11:51:32 2004
Subject: [Python-Dev] Re: Decimal data type issues
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D494@usahm010.exmi01.exch.eds.com>

Batista, Facundo:

> can you trust that the user that uses the
> method from context is aware of binary float traps?

If they are explicitly setting a context, I think
it is reasonable to assume that.  The problem case
is when they just use the default.

	inputcontext.Decimal(1.1)

has made an explicit request for specific handling.

	Decimal(1.1) 

has not indicated any awareness.  

The catch is that we don't want "just make another
context" to become such an idiom for floats that
people do it without knowing why.

Perhaps the Context.Decimal.__init__ could delegate 
to the raise function unless/until the user had set 
an attribute of approximate_float_ok = True.

-jJ

From guido at python.org  Fri Apr 23 14:26:58 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr 23 14:27:09 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
Message-ID: <200404231826.i3NIQxM18955@guido.python.org>

I'd like to get generator expressions checked into CVS.

Regarding the early-or-late binding issue, here's what I'd like to see
happen: I'd like the late-binding (i.e. non-capture) version checked
in and released with Python 2.4a1 and a2.  If we find there are real
problems with these semantics, we can switch to early-binding
semantics in 2.4b1; this would be almost completely backwards
compatible.  But if it works in the alpha release, we'll stick with
late binding.

My reasons for (still!) preferring late binding are:

(a) I find it hard to defend automatic variable capture given Python's
    general late-binding semantics

(b) I believe that use cases needing early binding are uncommon and
    strained: they all involve creating a list of generator
    expressions, which IMO is a pretty unusual thing to do

I cannot volunteer to do the final code review and checkin, and I
believe Raymond is still on vacation (he's asked me to relieve him
from the responsibility of shepherding this patch) so we need someone
to volunteer.

The patch is here:

http://sf.net/tracker/?func=detail&aid=872326&group_id=5470&atid=305470

--Guido van Rossum (home page: http://www.python.org/~guido/)

From judson at mcs.anl.gov  Fri Apr 23 14:43:10 2004
From: judson at mcs.anl.gov (Ivan R. Judson)
Date: Fri Apr 23 14:43:16 2004
Subject: [Python-Dev] PEP 754
Message-ID: <200404231843.i3NIhC0109140@mcs.anl.gov>


What's needs to be done to possibly get PEP754 (fpconst) to move ahead and
perhaps go into 2.4?

--Ivan


From mcherm at mcherm.com  Fri Apr 23 15:02:49 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Apr 23 15:02:50 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
Message-ID: <1082746969.40896859bcf09@mcherm.com>

> I'd like to get generator expressions checked into CVS.
>
> Regarding the early-or-late binding issue, here's what I'd like to see
> happen: I'd like the late-binding (i.e. non-capture) version checked
> in and released with Python 2.4a1 and a2.  If we find there are real
> problems with these semantics, we can switch to early-binding
> semantics in 2.4b1

Sounds like a good plan, but I have one question: Given the extent to
which alpha releases are tested (and subtracting out those who use
them only to check old code for compatibility and don't try out the
new features) are we going to be able to tell if there are problems
with the semantics?

My best idea is that those who currently favor the early-binding
approach (and I'm sure you've heard from plenty of them) should be
encouraged to try it out in the alpha and report any issues, along
with actual use cases. Someone collects these responses, then you
(Guido) look over the results (with a grain of salt since it's
mostly being tested by folks who object) and decide if it's a
problem.

Will that work? Anyone have a better idea? How do we go about asking
people to try it out (is a mention by Guido on comp.lang.python
sufficient)?

-- Michael Chermside


From jeremy at alum.mit.edu  Fri Apr 23 15:41:32 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Apr 23 15:42:35 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1082746969.40896859bcf09@mcherm.com>
References: <1082746969.40896859bcf09@mcherm.com>
Message-ID: <1082749292.9092.40.camel@localhost.localdomain>

On Fri, 2004-04-23 at 15:02, Michael Chermside wrote:
> > I'd like to get generator expressions checked into CVS.
> >
> > Regarding the early-or-late binding issue, here's what I'd like to see
> > happen: I'd like the late-binding (i.e. non-capture) version checked
> > in and released with Python 2.4a1 and a2.  If we find there are real
> > problems with these semantics, we can switch to early-binding
> > semantics in 2.4b1
> 
> Sounds like a good plan, but I have one question: Given the extent to
> which alpha releases are tested (and subtracting out those who use
> them only to check old code for compatibility and don't try out the
> new features) are we going to be able to tell if there are problems
> with the semantics?

We need reports from people writing real code with generator
expressions.  It's hard to guess if we'll get enough substantial
feedback in the alpha releases.  The kind of issues we're dealing with
usually manifest themselves in real programs rather than toy examples;
not sure that an alpha will get that kind of use.

On the other hand, a generator expression is nearly equivalent to a list
comprehension.  So it should be easy for people to experiment with
generator expressions, because they can swap them with list
comprehensions in many cases.

That leads me to wonder what exactly the rationale for generator
expressions is.  The PEP says that "time, clarity, and memory are
conserved by using an generator expression" but I can only see how
memory is conserved.  That is, I don't find them any easier to read than
list comprehensions and I don't understand the performance implications
very well.  It's not obvious to me that their faster.

> My best idea is that those who currently favor the early-binding
> approach (and I'm sure you've heard from plenty of them) should be
> encouraged to try it out in the alpha and report any issues, along

I'm not sure what you mean by "it." If you mean try the alpha with
late-binding semantics and see how it goes, I think that's a good plan.

> with actual use cases. Someone collects these responses, then you
> (Guido) look over the results (with a grain of salt since it's
> mostly being tested by folks who object) and decide if it's a
> problem.

Jeremy



From pje at telecommunity.com  Fri Apr 23 15:59:26 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Apr 23 15:59:47 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1082749292.9092.40.camel@localhost.localdomain>
References: <1082746969.40896859bcf09@mcherm.com>
	<1082746969.40896859bcf09@mcherm.com>
Message-ID: <5.1.1.6.0.20040423155826.03026480@telecommunity.com>

At 03:41 PM 4/23/04 -0400, Jeremy Hylton wrote:

>That leads me to wonder what exactly the rationale for generator
>expressions is.  The PEP says that "time, clarity, and memory are
>conserved by using an generator expression" but I can only see how
>memory is conserved.  That is, I don't find them any easier to read than
>list comprehensions and I don't understand the performance implications
>very well.  It's not obvious to me that their faster.

Alex Martelli previously posted some tests that showed them to be quite a 
bit faster for large lists.


From aahz at pythoncraft.com  Fri Apr 23 16:05:16 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Apr 23 16:06:03 2004
Subject: [Python-Dev] PEP 754
In-Reply-To: <200404231843.i3NIhC0109140@mcs.anl.gov>
References: <200404231843.i3NIhC0109140@mcs.anl.gov>
Message-ID: <20040423200516.GA28707@panix.com>

On Fri, Apr 23, 2004, Ivan R. Judson wrote:
> 
> What's needs to be done to possibly get PEP754 (fpconst) to move ahead and
> perhaps go into 2.4?

Nothing, because it ain't gonna happen.  In order for PEP754 to work, we
need to have a bunch of people volunteer to maintain the IEEE-754
semantics for a bunch of difference platforms.  Feel free to try to drum
up interest, but the PEP won't be accepted until there's evidence that
people are making long-term commitments.

</Guido channeling>
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From jeremy at alum.mit.edu  Fri Apr 23 16:10:39 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Apr 23 16:11:42 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <5.1.1.6.0.20040423155826.03026480@telecommunity.com>
References: <1082746969.40896859bcf09@mcherm.com>
	<1082746969.40896859bcf09@mcherm.com>
	<5.1.1.6.0.20040423155826.03026480@telecommunity.com>
Message-ID: <1082751039.9092.56.camel@localhost.localdomain>

On Fri, 2004-04-23 at 15:59, Phillip J. Eby wrote:
> At 03:41 PM 4/23/04 -0400, Jeremy Hylton wrote:
> 
> >That leads me to wonder what exactly the rationale for generator
> >expressions is.  The PEP says that "time, clarity, and memory are
> >conserved by using an generator expression" but I can only see how
> >memory is conserved.  That is, I don't find them any easier to read than
> >list comprehensions and I don't understand the performance implications
> >very well.  It's not obvious to me that their faster.
> 
> Alex Martelli previously posted some tests that showed them to be quite a 
> bit faster for large lists.

Anyone know where to find these numbers.  I've done some searching, but
I can't find them.  It would be good to include something concrete in
the PEP.

Jeremy



From tim.one at comcast.net  Fri Apr 23 16:32:27 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr 23 16:32:28 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1082749292.9092.40.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEKDKHAB.tim.one@comcast.net>

[Jeremy]
> ...
> That leads me to wonder what exactly the rationale for generator
> expressions is.  The PEP says that "time, clarity, and memory are
> conserved by using an generator expression" but I can only see how
> memory is conserved.  That is, I don't find them any easier to read
> than list comprehensions

They're not, although they can be more clear than code that defines helper
generating functions to get some of the same memory benefits.

> and I don't understand the performance implications very well.  It's
> not obvious to me that their faster.

When you've got an iterator producing a billion elements, it becomes obvious
at once <wink>.  Really, *when* they're faster than listcomps, it's mostly a
consequence of not creating in whole, then crawling over, a giant memory
object.

For short sequences, I expect listcomps are faster (and earlier timings have
shown that).  genexps require an additional frame suspend/resume per
element, and while cheap (esp. compared to a function call) it's not free.
For long sequences, avoiding the creation of a giant list becomes an
overwhelming advantage.

> Anyone know where to find these numbers.  I've done some searching, but
> I can't find them.  It would be good to include something concrete in
> the PEP.

This varies so wildly across platforms, timing procedure, and test case,
that I expect concrete numbers would do more harm than good.  The
qualitative argument is easy to grasp.

BTW, the most recent round of this was in the "genexps slow?" thread,
started last month and spilling into April.  Here's the start of it:

    http://mail.python.org/pipermail/python-dev/2004-March/043777.html


From pf_moore at yahoo.co.uk  Fri Apr 23 16:46:54 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Fri Apr 23 16:51:01 2004
Subject: [Python-Dev] Re: PEP 754
References: <200404231843.i3NIhC0109140@mcs.anl.gov>
	<20040423200516.GA28707@panix.com>
Message-ID: <vfjqpgld.fsf@yahoo.co.uk>

Aahz <aahz@pythoncraft.com> writes:

> On Fri, Apr 23, 2004, Ivan R. Judson wrote:
>> 
>> What's needs to be done to possibly get PEP754 (fpconst) to move ahead and
>> perhaps go into 2.4?
>
> Nothing, because it ain't gonna happen.  In order for PEP754 to work, we
> need to have a bunch of people volunteer to maintain the IEEE-754
> semantics for a bunch of difference platforms.

The PEP offers a reference implementation in pure Python. (Although
the link given is broken). The implementation is just about creating
and manipulating specific bit patterns stored in floats.

How useful the module is (given that Python doesn't mandate that
floats are stored using IEEE-754 format) I couldn't comment. But I'll
note that the standard library already contains modules which only
work on specific types of platform.

As to the inclusion of the module, I have no opinion...

Paul.
-- 
This signature intentionally left blank


From tim.one at comcast.net  Fri Apr 23 16:51:56 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr 23 16:52:04 2004
Subject: [Python-Dev] PEP 754
In-Reply-To: <200404231843.i3NIhC0109140@mcs.anl.gov>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEKFKHAB.tim.one@comcast.net>

[Ivan Judson]
> What's needs to be done to possibly get PEP754 (fpconst) to move
> ahead and perhaps go into 2.4?

+ The implementation URL in the PEP no longer resolves, so that
  needs to be fixed.

+ Last time I saw it, fpconst.py had a Pfizer copyright and
  license, which makes headaches for everyone.

+ It needs docs.

+ It needs a test suite.

+ Guido has to Pronounce on it.

[Aahz]
> In order for PEP754 to work, we need to have a bunch of people
> volunteer to maintain the IEEE-754 semantics for a bunch of
> difference platforms.  Feel free to try to drum up interest, but
> the PEP won't be accepted until there's evidence that people
> are making long-term commitments.

That isn't a good objection for this specific patch -- it's pure Python, and
picks apart bits using the struct module.  IIRC, it wasn't correctly written
for most 64-bit boxes, but that looked like a shallow flaw (I think it
assumed that a struct "l" format code, in native mode, always yields a
32-bit thingie).


From goodger at python.org  Fri Apr 23 19:43:09 2004
From: goodger at python.org (David Goodger)
Date: Fri Apr 23 19:43:19 2004
Subject: [Python-Dev] Re: PEP 754
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEKFKHAB.tim.one@comcast.net>
References: <200404231843.i3NIhC0109140@mcs.anl.gov>
	<LNBBLJKPBEHFEDALKOLCCEKFKHAB.tim.one@comcast.net>
Message-ID: <4089AA0D.4030205@python.org>

 > [Ivan Judson]
 >> What's needs to be done to possibly get PEP754 (fpconst) to move
 >> ahead and perhaps go into 2.4?

This one seems to have slipped through the cracks.  I haven't heard
from the author lately; hopefully he hasn't lost interest.  In any
case, an active champion is needed for progress.

Tim Peters wrote:
 > + The implementation URL in the PEP no longer resolves, so that
 >   needs to be fixed.

A Google search resulted in:
<http://www.analytics.washington.edu/statcomp/projects/rzope/fpconst/>
I've updated the PEP.

 > + Last time I saw it, fpconst.py had a Pfizer copyright and
 >   license, which makes headaches for everyone.
 >
 > + It needs docs.
 >
 > + It needs a test suite.
 >
 > + Guido has to Pronounce on it.

Is Guido's waiting for your (Tim's) recommendation?  Guido wrote,
"David, I'll leave the review of this PEP entirely to Tim, if you
don't mind".  At the time (2003-10-05) Tim wrote:

     Guido and I have discussed this in email today.  We're both
     favorable.  It may get buried under feature creep, though (that
     is, if the PEP is good so far as it goes, it may be necessary to
     make it go farther, addressing some of the other problems
     mentioned-- but not solved --by the PEP).

Gregory Warnes (the PEP author) had a few questions the next day
(2003-10-06); I wasn't copied on replies if any.  Shall I forward
Gregory's messages to you?  I'm copying Gregory on this message.

-- 
David Goodger                    <http://starship.python.net/~goodger>
Python Enhancement Proposal (PEP) Editor <http://www.python.org/peps/>

(Please cc: all PEP correspondence to <peps@python.org>.)

From paul at prescod.net  Sat Apr 24 05:51:36 2004
From: paul at prescod.net (Paul Prescod)
Date: Sat Apr 24 07:39:44 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1082749292.9092.40.camel@localhost.localdomain>
References: <1082746969.40896859bcf09@mcherm.com>
	<1082749292.9092.40.camel@localhost.localdomain>
Message-ID: <408A38A8.1030702@prescod.net>

Jeremy Hylton wrote:

>...
> 
> That leads me to wonder what exactly the rationale for generator
> expressions is.  The PEP says that "time, clarity, and memory are
> conserved by using an generator expression" but I can only see how
> memory is conserved.  That is, I don't find them any easier to read than
> list comprehensions and I don't understand the performance implications
> very well.  It's not obvious to me that their faster.

I think that there is a robustness argument to be made as well. It is 
very common to run into programs that work really well with small data 
sets and then completely run out of steam with large ones. Sometimes the 
algorithm is at fault. But it could also be that some Python programs 
fail to scale primarily because of the wastefulness of listcomps. 
Whether or not this is true in practice, it is certainly true that when 
you write programs that are meant to scale, you must consider every 
listcomp and think whether the dataset is going to get large or not. If 
so, you must switch to some more obfuscated syntax.

  Paul Prescod



From scav at blueyonder.co.uk  Sat Apr 24 10:58:14 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Sat Apr 24 10:49:46 2004
Subject: [Python-Dev] Re: generator expressions
In-Reply-To: <E1BHMOX-00081q-HP@mail.python.org>
References: <E1BHMOX-00081q-HP@mail.python.org>
Message-ID: <408A8086.3010005@blueyonder.co.uk>

Michael Chermside wrote:

>How do we go about asking
>people to try it out (is a mention by Guido on comp.lang.python
>sufficient)?
>
>-- Michael Chermside
>
>  
>
I'll play with generator expressions gladly. I have been using list 
comprehensions like mad, even in cases where I suspect a generator would 
be better. Also, I have a few uses of generators that might look clearer 
as generator expressions. So I'd be more than happy to make a test 
version of my current projects using generator expressions, just to see 
when they can make the code clearer, and whether there is anything about 
the semantics that trips me up too easily.

I don't think my data sets are going to be big enough to test 
performance, just usability.

Now, if someone checks in the implementation of PEP309 too <hint>, I 
would be as happy as a biscuit...

Peter Harris

From bob at redivi.com  Sat Apr 24 11:02:55 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Apr 24 10:58:46 2004
Subject: [Python-Dev] Re: generator expressions
In-Reply-To: <408A8086.3010005@blueyonder.co.uk>
References: <E1BHMOX-00081q-HP@mail.python.org>
	<408A8086.3010005@blueyonder.co.uk>
Message-ID: <777FCEB3-9600-11D8-BEAF-000A95686CD8@redivi.com>

On Apr 24, 2004, at 10:58 AM, Peter Harris wrote:

> Michael Chermside wrote:
>
>> How do we go about asking
>> people to try it out (is a mention by Guido on comp.lang.python
>> sufficient)?
>>
>> -- Michael Chermside
>>
>>
> I'll play with generator expressions gladly. I have been using list 
> comprehensions like mad, even in cases where I suspect a generator 
> would be better. Also, I have a few uses of generators that might look 
> clearer as generator expressions. So I'd be more than happy to make a 
> test version of my current projects using generator expressions, just 
> to see when they can make the code clearer, and whether there is 
> anything about the semantics that trips me up too easily.
>
> I don't think my data sets are going to be big enough to test 
> performance, just usability.
>
> Now, if someone checks in the implementation of PEP309 too <hint>, I 
> would be as happy as a biscuit...

Well, I'm +1 on PEP309.  I regularly wish I had partial function 
application in Python.  I also plan to experiment with generator 
expressions sooner than later.

-bob


From niemeyer at conectiva.com  Sun Apr 25 19:40:19 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Sun Apr 25 19:39:34 2004
Subject: [Python-Dev] Small issues in gettext support
Message-ID: <20040425234019.GA6436@burma.localdomain>

Hello folks,

I've been working with gettext support in Python, and found some
issues I'd like to discuss with you.

First, I've noticed that there's a difference in the internal
implementation of gettext and GNU gettext regarding the returned
encoding on non-unicode strings. Notice the difference in the
result of this code:

   import gettext
   import locale
   locale.setlocale(locale.LC_ALL, "")
   locale.textdomain("apt-cdrom-registry")
   gettext.textdomain("apt-cdrom-registry")
   print locale.gettext("Choose the available CDROMs from the list below")
   print gettext.gettext("Choose the available CDROMs from the list below")

This has shown the following:

   Escolha os CDROMs dispon?ves na lista abaixo
   Escolha os CDROMs dispon??ves na lista abaixo

The reason for this difference is clear: GNU gettext defaults to the
current locale when returning encoded strings, while gettext.py
returns strings in the encoding used in the .mo file. The fix is
simply changing the following code

   # Encode the Unicode tmsg back to an 8-bit string, if possible
   if self._charset:
	return tmsg.encode(self._charset)

to use the system encoding (sys.getdefaultencoding()) instead of
self._charset.

Regarding a similar issue, I've also noticed that we're currently
missing bind_textdomain_codeset() support. This function changes
the codeset used to return the translated strings.

So, I'd like to implement the following changes:

- Change the default codeset used by gettext.py in functions
  returning an encoded string to match the system encoding.
- Introduce bind_textdomain_codeset() in locale.
- Introduce bind_textdomain_codeset() in gettext.py implementing
  an equivalent functionality.

Comments?

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Mon Apr 26 08:38:26 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Apr 26 08:37:44 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1082749292.9092.40.camel@localhost.localdomain>
References: <1082746969.40896859bcf09@mcherm.com>
	<1082749292.9092.40.camel@localhost.localdomain>
Message-ID: <20040426123826.GA1922@burma.localdomain>

> We need reports from people writing real code with generator
> expressions.  It's hard to guess if we'll get enough substantial
> feedback in the alpha releases.  The kind of issues we're dealing with
> usually manifest themselves in real programs rather than toy examples;
> not sure that an alpha will get that kind of use.

An interesting idea might be to hack the standard library replacing
every occurrence of list comprehensions with generator expressions,
and check the respective test cases and/or applications using them.

-- 
Gustavo Niemeyer
http://niemeyer.net

From barry at python.org  Mon Apr 26 10:13:14 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Apr 26 10:13:20 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <20040425234019.GA6436@burma.localdomain>
References: <20040425234019.GA6436@burma.localdomain>
Message-ID: <1082988793.17933.59.camel@anthem.wooz.org>

On Sun, 2004-04-25 at 19:40, Gustavo Niemeyer wrote:
> Hello folks,
> 
> I've been working with gettext support in Python, and found some
> issues I'd like to discuss with you.
> 
> First, I've noticed that there's a difference in the internal
> implementation of gettext and GNU gettext regarding the returned
> encoding on non-unicode strings. Notice the difference in the
> result of this code:
> 
>    import gettext
>    import locale
>    locale.setlocale(locale.LC_ALL, "")
>    locale.textdomain("apt-cdrom-registry")
>    gettext.textdomain("apt-cdrom-registry")
>    print locale.gettext("Choose the available CDROMs from the list below")
>    print gettext.gettext("Choose the available CDROMs from the list below")
> 
> This has shown the following:
> 
>    Escolha os CDROMs dispon?ves na lista abaixo
>    Escolha os CDROMs dispon??ves na lista abaixo
> 
> The reason for this difference is clear: GNU gettext defaults to the
> current locale when returning encoded strings, while gettext.py
> returns strings in the encoding used in the .mo file. The fix is
> simply changing the following code
> 
>    # Encode the Unicode tmsg back to an 8-bit string, if possible
>    if self._charset:
> 	return tmsg.encode(self._charset)
> 
> to use the system encoding (sys.getdefaultencoding()) instead of
> self._charset.

I'd be worried most about backwards compatibility, since the module has
worked this way since its early days.  Also, wouldn't this be an
opportunity for getting lots of UnicodeErrors?  E.g. my system encoding
is 'ascii' so gettext() would fail for catalogs containing non-ascii
characters.  I shouldn't have to change my system encoding just to avoid
errors, but with your suggestion, wouldn't that make many catalogs
basically unusable for me?

> Regarding a similar issue, I've also noticed that we're currently
> missing bind_textdomain_codeset() support. This function changes
> the codeset used to return the translated strings.
> 
> So, I'd like to implement the following changes:
> 
> - Change the default codeset used by gettext.py in functions
>   returning an encoded string to match the system encoding.
> - Introduce bind_textdomain_codeset() in locale.
> - Introduce bind_textdomain_codeset() in gettext.py implementing
>   an equivalent functionality.

Would adding bind_textdomain_codeset() would provide a way for the
application to change the default encoding?

If so, I'd be in favor of adding bind_textdomain_codeset() but not
changing the default encoding for returned strings.  Then update the
documentation to describe current behavior and how to change it via that
function call.

-Barry



From niemeyer at conectiva.com  Mon Apr 26 11:16:07 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Apr 26 11:16:15 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <1082988793.17933.59.camel@anthem.wooz.org>
References: <20040425234019.GA6436@burma.localdomain>
	<1082988793.17933.59.camel@anthem.wooz.org>
Message-ID: <20040426151607.GA2361@burma.localdomain>

> I'd be worried most about backwards compatibility, since the module has
> worked this way since its early days.  Also, wouldn't this be an

IMO, the current behavior is wrong, so breaking backwards
compatibility in that case would be fixing something
important.

> opportunity for getting lots of UnicodeErrors?  E.g. my system encoding
> is 'ascii' so gettext() would fail for catalogs containing non-ascii
> characters.  I shouldn't have to change my system encoding just to avoid
> errors, but with your suggestion, wouldn't that make many catalogs
> basically unusable for me?

There are a few extra points to notice here:

- Different .mo files may have different encodings.

- The translation system is made in a way that the programmer should
  not have to worry about the encoding used by the translators.

- The current scheme may introduce a wrong practice: forcing
  translators to use some specific encoding to avoid breaking the
  program.

- We already have support for getting the "unicode" version of the
  string. This is currently the right way to get the translation in
  some specific encoding, since it uncouples the translation encoding
  from the expected encoding.

- In cases where you'd get the "UnicodeError", you'd see a mangled
  string which would be unreadable. To avoid the UnicodeError, we
  may also return the original string in cases where the
  UnicodeError is raised.

> Would adding bind_textdomain_codeset() would provide a way for the
> application to change the default encoding?

Yes, it changes the default encoding.

> If so, I'd be in favor of adding bind_textdomain_codeset() but not
> changing the default encoding for returned strings.  Then update the
> documentation to describe current behavior and how to change it via
> that function call.

Thanks for your suggestion!

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Mon Apr 26 11:33:03 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Apr 26 11:33:09 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <20040426151607.GA2361@burma.localdomain>
References: <20040425234019.GA6436@burma.localdomain>
	<1082988793.17933.59.camel@anthem.wooz.org>
	<20040426151607.GA2361@burma.localdomain>
Message-ID: <20040426153303.GA2676@burma.localdomain>

> > opportunity for getting lots of UnicodeErrors?  E.g. my system encoding
> > is 'ascii' so gettext() would fail for catalogs containing non-ascii
> > characters.  I shouldn't have to change my system encoding just to avoid
> > errors, but with your suggestion, wouldn't that make many catalogs
> > basically unusable for me?

Another interesting point, from the gettext documentation:

"""
The output character set is, by default, the value of `nl_langinfo
(CODESET)', which depends on the `LC_CTYPE' part of the current locale.
But programs which store strings in a locale independent way (e.g.
UTF-8) can request that `gettext' and related functions return the
translation in that encoding, by use of the `bind_textdomain_codeset'
function.
"""

So, we should use some variant of locale.nl_langinfo(locale.CODESET) to
get the default encoding. Since this should be changed together with
the language used when 'setlocale' is issued, the UnicodeError problems
you're afraid of would also go away.

-- 
Gustavo Niemeyer
http://niemeyer.net

From anthony at interlink.com.au  Mon Apr 26 11:59:37 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Apr 26 12:01:21 2004
Subject: [Python-Dev] today's pointless, yet slightly terrifying, factoid(s)
Message-ID: <408D31E9.80205@interlink.com.au>


I received some email about some _very_ old pages (no longer existing)
I put up about Python's internals. In 1996. This made me stop and
think 'bloody hell, how long _have_ I been doing this?'
cvs ann Misc/ACKS is slightly terrifying, saying at least 1994.

A teeny tiny sort script later, and this list follows. Note that it's
obviously not representative of anything at all, but may serve to
terrify others, as it terrified me, as to the sheer amount of their
life that they've wasted on Python <wink>

disclaimer: absolutely representative of nothing at all, not to be
taken internally, if pain persists see a doctor

Anthony

1.1          (guido    26-Jan-94): Mark Anacker
1.1          (guido    26-Jan-94): Jan-Hein B"uhrman
1.1          (guido    26-Jan-94): Anthony Baxter
1.1          (guido    26-Jan-94): Donald Beaudry
1.1          (guido    26-Jan-94): Eric Beser
1.1          (guido    26-Jan-94): Stephen Bevan
1.1          (guido    26-Jan-94): Peter Bosch
1.1          (guido    26-Jan-94): Terrence Brannon
1.1          (guido    26-Jan-94): Erik de Bueger
1.1          (guido    26-Jan-94): Dick Bulterman
1.1          (guido    26-Jan-94): David Chaum
1.1          (guido    26-Jan-94): Jonathan Dasteel
1.1          (guido    26-Jan-94): John DeGood
1.1          (guido    26-Jan-94): Roger Dev
1.1          (guido    26-Jan-94): Stoffel Erasmus
1.1          (guido    26-Jan-94): Niels Ferguson
1.1          (guido    26-Jan-94): Michael Guravage
1.1          (guido    26-Jan-94): Paul ten Hagen
1.1          (guido    26-Jan-94): Lynda Hardman
1.1          (guido    26-Jan-94): Ivan Herman
1.1          (guido    26-Jan-94): Chris Hoffman
1.1          (guido    26-Jan-94): Philip Homburg
1.1          (guido    26-Jan-94): Jack Jansen
1.1          (guido    26-Jan-94): Bill Janssen
1.1          (guido    26-Jan-94): Drew Jenkins
1.1          (guido    26-Jan-94): Lou Kates
1.1          (guido    26-Jan-94): Robert van Liere
1.1          (guido    26-Jan-94): Steve Majewski
1.1          (guido    26-Jan-94): Lambert Meertens
1.1          (guido    26-Jan-94): Steven Miale
1.1          (guido    26-Jan-94): Doug Moen
1.1          (guido    26-Jan-94): Sape Mullender
1.1          (guido    26-Jan-94): Sjoerd Mullender
1.1          (guido    26-Jan-94): George Neville-Neil
1.1          (guido    26-Jan-94): Randy Pausch
1.1          (guido    26-Jan-94): Marcel van der Peijl
1.1          (guido    26-Jan-94): Steven Pemberton
1.1          (guido    26-Jan-94): Tim Peters
1.1          (guido    26-Jan-94): John Redford
1.1          (guido    26-Jan-94): Timothy Roscoe
1.1          (guido    26-Jan-94): Kevin Samborn
1.1          (guido    26-Jan-94): Fred Sells
1.1          (guido    26-Jan-94): Denis Severson
1.1          (guido    26-Jan-94): Michael Shiplett
1.1          (guido    26-Jan-94): Paul Sijben
1.1          (guido    26-Jan-94): Dirk Soede
1.1          (guido    26-Jan-94): Per Spilling
1.1          (guido    26-Jan-94): Quentin Stafford-Fraser
1.1          (guido    26-Jan-94): Tracy Tims
1.1          (guido    26-Jan-94): Bennett Todd
1.1          (guido    26-Jan-94): Jaap Vermeulen
1.1          (guido    26-Jan-94): Dik Winter
1.2          (guido    26-Jan-94): Amrit Prem
1.3          (guido    14-Apr-94): Jurjen Bos
1.3          (guido    14-Apr-94): Stefan Esser
1.3          (guido    14-Apr-94): Rycharde Hawkes
1.3          (guido    14-Apr-94): Steve Kirsch
1.3          (guido    14-Apr-94): Mark Lutz
1.3          (guido    14-Apr-94): Jim Roskind
1.4          (guido    04-May-94): Adrian Phillips
1.5          (guido    04-May-94): Andy Bensky
1.6          (guido    23-May-94): Rickard Westman
1.7          (guido    14-Jul-94): Tommy Burnette
1.7          (guido    14-Jul-94): Simon Johnston
1.7          (guido    14-Jul-94): William Lewis
1.7          (guido    14-Jul-94): Richard Walker
1.8          (guido    05-Aug-94): Andrew Kuchling
1.8          (guido    05-Aug-94): John Tromp
1.9          (guido    30-Aug-94): Ulf Bartelt
1.10         (guido    14-Sep-94): Michael Scharf
1.11         (guido    06-Oct-94): Anne Lord
1.11         (guido    06-Oct-94): Skip Montanaro
1.12         (guido    10-Nov-94): Eric Bouck
1.12         (guido    10-Nov-94): Dave Brennan
1.12         (guido    10-Nov-94): Mike Carlton
1.12         (guido    10-Nov-94): John Cugini
1.12         (guido    10-Nov-94): Lance Ellinghaus
1.12         (guido    10-Nov-94): Paul Everitt
1.12         (guido    10-Nov-94): Robin Friedrich
1.12         (guido    10-Nov-94): Jim Fulton
1.12         (guido    10-Nov-94): Peter Funk
1.12         (guido    10-Nov-94): Mark Hammond
1.12         (guido    10-Nov-94): Barry Hantman
1.12         (guido    10-Nov-94): Jim Lynch
1.12         (guido    10-Nov-94): Ken Manheimer
1.12         (guido    10-Nov-94): Roger Masse
1.12         (guido    10-Nov-94): Michael McLay
1.12         (guido    10-Nov-94): Jim St. Pierre
1.12         (guido    10-Nov-94): Nick Seidenman
1.12         (guido    10-Nov-94): George Sipe
1.12         (guido    10-Nov-94): Richard Stoakley
1.12         (guido    10-Nov-94): Casper Stoel
1.12         (guido    10-Nov-94): Frank Visser
1.12         (guido    10-Nov-94): Barry Warsaw
1.12         (guido    10-Nov-94): Steve Waterbury
1.13         (guido    04-Jan-95): Matt Conway
1.13         (guido    04-Jan-95): Tom Culliton
1.13         (guido    04-Jan-95): Bob Watson
1.14         (guido    17-Jan-95): Dag Gruneau
1.14         (guido    17-Jan-95): Jun Hamano
1.14         (guido    17-Jan-95): Ken Howard
1.14         (guido    17-Jan-95): Eric Siegerman
1.14         (guido    17-Jan-95): R Lindsay Todd
1.15         (guido    14-Feb-95): Steve Clift
1.15         (guido    14-Feb-95): David Hobley
1.15         (guido    14-Feb-95): John Interrante
1.15         (guido    14-Feb-95): Bob Kras
1.16         (guido    09-Mar-95): Stig Bakken
1.16         (guido    09-Mar-95): Monty Brandenberg
1.16         (guido    09-Mar-95): Eric Daniel
1.16         (guido    09-Mar-95): Kevan Heydon
1.16         (guido    09-Mar-95): Soren Larsen
1.16         (guido    09-Mar-95): Steven Reiz
1.16         (guido    09-Mar-95): Ken Stox
1.17         (guido    07-Apr-95): Graham Matthews
1.18         (guido    28-Aug-95): David Ely
1.18         (guido    28-Aug-95): Gregor Schmid
1.19         (guido    11-Oct-95): Joel Shprentz
1.20         (guido    26-Jun-96): Mitch Chapman
1.20         (guido    26-Jun-96): Steve Cousins
1.20         (guido    26-Jun-96): Hans de Graaff
1.20         (guido    26-Jun-96): Rasmus Hahn
1.20         (guido    26-Jun-96): Manus Hand
1.20         (guido    26-Jun-96): Richard Jones
1.20         (guido    26-Jun-96): Harri Pasanen
1.20         (guido    26-Jun-96): Masazumi Yoshikawa
1.21         (guido    21-Jul-96): Jim Ahlstrom
1.21         (guido    21-Jul-96): Oliver Andrich
1.21         (guido    21-Jul-96): Ross Andrus
1.21         (guido    21-Jul-96): David Ascher
1.21         (guido    21-Jul-96): Neal Becker
1.21         (guido    21-Jul-96): Tom Christiansen
1.21         (guido    21-Jul-96): Sebastian Fernandez
1.21         (guido    21-Jul-96): Jim Hugunin
1.21         (guido    21-Jul-96): Just van Rossum
1.21         (guido    21-Jul-96): Amy Taylor
1.21         (guido    21-Jul-96): John Viega
1.21         (guido    21-Jul-96): Aaron Watters
1.21         (guido    21-Jul-96): Siebren van der Zee
1.22         (guido    30-Jul-96): Bill Tutt
1.23         (guido    20-Aug-96): Vladimir Marangozov
1.24         (guido    08-Oct-96): Nils Fischbeck
1.24         (guido    08-Oct-96): Tim Hochberg
1.24         (guido    08-Oct-96): Terry Reedy
1.24         (guido    08-Oct-96): Ka-Ping Yee
1.25         (guido    09-May-97): Jyrki Alakuijala
1.25         (guido    09-May-97): Andy Eskilsson
1.25         (guido    09-May-97): Lele Gaifax
1.25         (guido    09-May-97): Jeremy Hylton
1.25         (guido    09-May-97): Per Lindqvist
1.26         (guido    19-Jul-97): Thierry Bousch
1.26         (guido    19-Jul-97): Thomas Gellekum
1.26         (guido    19-Jul-97): Konrad Hinsen
1.26         (guido    19-Jul-97): Fredrik Lundh
1.26         (guido    19-Jul-97): Neil Schemenauer
1.27         (guido    15-Aug-97): Lars Wirzenius
1.28         (guido    07-Oct-97): David Beazley
1.28         (guido    07-Oct-97): Reimer Behrends
1.28         (guido    07-Oct-97): Donn Cave
1.28         (guido    07-Oct-97): Paul Dubois
1.28         (guido    07-Oct-97): Tim Everett
1.28         (guido    07-Oct-97): Brad Howes
1.28         (guido    07-Oct-97): Bob Kahn
1.28         (guido    07-Oct-97): Hannu Krosing
1.28         (guido    07-Oct-97): Chris Lawrence
1.28         (guido    07-Oct-97): Marc-Andre Lemburg
1.28         (guido    07-Oct-97): Craig McPheeters
1.28         (guido    07-Oct-97): Geoff Philbrick
1.28         (guido    07-Oct-97): Case Roole
1.28         (guido    07-Oct-97): Hugo van Rossum
1.28         (guido    07-Oct-97): Saskia van Rossum
1.28         (guido    07-Oct-97): Greg Stein
1.28         (guido    07-Oct-97): Al Vezza
1.28         (guido    07-Oct-97): Larry Wall
1.28         (guido    07-Oct-97): Truida Wiedijk
1.28         (guido    07-Oct-97): Sue Williams
1.28         (guido    07-Oct-97): Frank Willison
1.29         (guido    24-Nov-97): Matthias Klose
1.29         (guido    24-Nov-97): Bill Noon
1.29         (guido    24-Nov-97): Charles Waldman
1.30         (guido    26-Nov-97): Lee Busby
1.30         (guido    26-Nov-97): Ben Escoto
1.30         (guido    26-Nov-97): Jeff Rush
1.31         (guido    10-Dec-97): Anders Andersen
1.31         (guido    10-Dec-97): Raymund Galvin
1.31         (guido    10-Dec-97): Chris Herborth
1.32         (guido    11-Dec-97): Doug Marien
1.33         (guido    30-Dec-97): Roy Bixler
1.33         (guido    30-Dec-97): Drew Csillag
1.33         (guido    30-Dec-97): Joseph Koshy
1.33         (guido    30-Dec-97): Roman Milner
1.33         (guido    30-Dec-97): Piet van Oostrum
1.34         (guido    09-Apr-98): Des Barry
1.34         (guido    09-Apr-98): Paul Boddie
1.34         (guido    09-Apr-98): Roger Burnham
1.34         (guido    09-Apr-98): Jaromir Dolecek
1.34         (guido    09-Apr-98): Hans Eckardt
1.34         (guido    09-Apr-98): Rob Hooft
1.34         (guido    09-Apr-98): Ben Jackson
1.34         (guido    09-Apr-98): Tamito Kajiyama
1.34         (guido    09-Apr-98): Magnus Kessler
1.34         (guido    09-Apr-98): Ron Klatchko
1.34         (guido    09-Apr-98): Piers Lauder
1.34         (guido    09-Apr-98): Kip Lehman
1.34         (guido    09-Apr-98): Christopher Lindblad
1.34         (guido    09-Apr-98): Anthony Martin
1.34         (guido    09-Apr-98): Bill van Melle
1.34         (guido    09-Apr-98): The Dragon De Monsyne
1.34         (guido    09-Apr-98): Chad Netzer
1.34         (guido    09-Apr-98): Oscar Nierstrasz
1.34         (guido    09-Apr-98): Tim O'Malley
1.34         (guido    09-Apr-98): Jason Orendorff
1.34         (guido    09-Apr-98): Ty Sarna
1.34         (guido    09-Apr-98): Kurt Vile
1.35         (guido    10-Apr-98): Jeff Epler
1.36         (guido    13-Apr-98): Peter Haight
1.36         (guido    13-Apr-98): Gary Herron
1.37         (guido    30-Jun-98): Dominic Binks
1.37         (guido    30-Jun-98): Jesse Hallio
1.37         (guido    30-Jun-98): Jonathan Hendry
1.37         (guido    30-Jun-98): John Jorgensen
1.37         (guido    30-Jun-98): Ray Loyzaga
1.37         (guido    30-Jun-98): Nigel O'Brian
1.37         (guido    30-Jun-98): Eric Raymond
1.37         (guido    30-Jun-98): Sean Reifschneider
1.37         (guido    30-Jun-98): Ben Sayer
1.38         (guido    04-Aug-98): Ralph Butler
1.38         (guido    04-Aug-98): Nicolas Chauvat
1.38         (guido    04-Aug-98): Vadim Chugunov
1.38         (guido    04-Aug-98): Carey Evans
1.38         (guido    04-Aug-98): Dan Stromberg
1.38         (guido    04-Aug-98): Gerry Wiener
1.39         (guido    11-Aug-98): Jason Harper
1.40         (guido    02-Oct-98): Robin Dunn
1.40         (guido    02-Oct-98): Jonathan Giddy
1.40         (guido    02-Oct-98): Vladimir Kushnir
1.40         (guido    02-Oct-98): Richard Wolff
1.41         (guido    21-Dec-98): Jeff Bauer
1.41         (guido    21-Dec-98): Thomas Bellman
1.41         (guido    21-Dec-98): Jochen Hayek
1.41         (guido    21-Dec-98): Detlef Lannert
1.41         (guido    21-Dec-98): Fredrik Nehr
1.41         (guido    21-Dec-98): Chris Petrilli
1.41         (guido    21-Dec-98): RajGopal Srinivasan
1.41         (guido    21-Dec-98): Greg Ward
1.41         (guido    21-Dec-98): Stefan Witzel
1.42         (guido    25-Jan-99): Greg Couch
1.42         (guido    25-Jan-99): Nathan Sullivan
1.42         (guido    25-Jan-99): Blake Winton
1.43         (guido    08-Feb-99): Uwe Zessin
1.44         (guido    16-Feb-99): Finn Bock
1.45         (guido    18-Feb-99): Scott Cotton
1.45         (guido    18-Feb-99): Toby Dickenson
1.45         (guido    18-Feb-99): Larry Hastings
1.45         (guido    18-Feb-99): Grzegorz Makarewicz
1.45         (guido    18-Feb-99): Dieter Maurer
1.45         (guido    18-Feb-99): Mike Meyer
1.45         (guido    18-Feb-99): Rafal Smotrzyk
1.45         (guido    18-Feb-99): Jean-Claude Wippler
1.46         (guido    05-Apr-99): Per Cederqvist
1.46         (guido    05-Apr-99): Michael Ernst
1.46         (guido    05-Apr-99): Tadayoshi Funaba
1.46         (guido    05-Apr-99): Magnus L. Hetland
1.46         (guido    05-Apr-99): Albert Hofkamp
1.46         (guido    05-Apr-99): Jeffrey Honig
1.46         (guido    05-Apr-99): Donald Wallace Rouse II
1.46         (guido    05-Apr-99): Orjan Johansen
1.46         (guido    05-Apr-99): Kim Knapp
1.46         (guido    05-Apr-99): Nick Lockwood
1.46         (guido    05-Apr-99): Stephanie Lockwood
1.46         (guido    05-Apr-99): Takahiro Nakayama
1.46         (guido    05-Apr-99): Christopher J. Phoenix
1.46         (guido    05-Apr-99): Bernhard Reiter
1.46         (guido    05-Apr-99): Jim Robinson
1.46         (guido    05-Apr-99): Clay Spence
1.46         (guido    05-Apr-99): Tobias Thelen
1.46         (guido    05-Apr-99): Norman Vine
1.46         (guido    05-Apr-99): Edward Welbourne
1.46         (guido    05-Apr-99): Moshe Zadka
1.46         (guido    05-Apr-99): Milan Zamazal
1.47         (guido    10-Apr-99): Andrew Dalke
1.48         (guido    13-Apr-99): Jason Asbahr
1.48         (guido    13-Apr-99): Pablo Bleyer
1.48         (guido    13-Apr-99): Mark Favas
1.48         (guido    13-Apr-99): Pat Knight
1.48         (guido    13-Apr-99): Barry Scott
1.49         (guido    13-Apr-99): Andy Dustman
1.49         (guido    13-Apr-99): Cameron Laird
1.49         (guido    13-Apr-99): Michael P. Reilly
1.49         (guido    13-Apr-99): Mike Romberg
1.50         (jhylton  28-Mar-00): Greg Ewing
1.51         (guido    31-Mar-00): Jeffrey Chang
1.51         (guido    31-Mar-00): Albert Chin-A-Young
1.51         (guido    31-Mar-00): Lars Damerow
1.51         (guido    31-Mar-00): John DuBois
1.51         (guido    31-Mar-00): Ben Gertzfield
1.51         (guido    31-Mar-00): Duncan Grisby
1.51         (guido    31-Mar-00): Milton L. Hankins
1.51         (guido    31-Mar-00): Greg Humphreys
1.51         (guido    31-Mar-00): Eric Lindvall
1.51         (guido    31-Mar-00): Loren Luke
1.51         (guido    31-Mar-00): Mikael Lyngvig
1.51         (guido    31-Mar-00): Dom Mitchell
1.51         (guido    31-Mar-00): Ofir Reichenberg
1.51         (guido    31-Mar-00): Jan Pieter Riegel
1.51         (guido    31-Mar-00): Craig Rowland
1.51         (guido    31-Mar-00): Nathan Paul Simons
1.51         (guido    31-Mar-00): Paul Sokolovsky
1.51         (guido    31-Mar-00): Laurence Tratt
1.51         (guido    31-Mar-00): Kannan Vijayan
1.51         (guido    31-Mar-00): Klaus-Juergen Wolf
1.52         (guido    31-Mar-00): Ron Bickers
1.52         (guido    31-Mar-00): Gary S. Brown
1.52         (guido    31-Mar-00): Martijn Faassen
1.52         (guido    31-Mar-00): Fred Gansevles
1.52         (guido    31-Mar-00): Barry Haddow
1.52         (guido    31-Mar-00): Joerg-Cyril Hoehle
1.52         (guido    31-Mar-00): Gerrit Holl
1.52         (guido    31-Mar-00): Brian Hooper
1.52         (guido    31-Mar-00): Randall Hopper
1.52         (guido    31-Mar-00): Michael Hudson
1.52         (guido    31-Mar-00): Lucas de Jonge
1.52         (guido    31-Mar-00): Andreas Jung
1.52         (guido    31-Mar-00): Nick Maclaren
1.52         (guido    31-Mar-00): Andy Robinson
1.52         (guido    31-Mar-00): Nick Russo
1.52         (guido    31-Mar-00): Eric Tiedemann
1.52         (guido    31-Mar-00): Christian Tismer
1.52         (guido    31-Mar-00): Greg V. Wilson
1.53         (guido    31-Mar-00): Samuel L. Bayer
1.53         (guido    31-Mar-00): Gary Duzan
1.53         (guido    31-Mar-00): Eugene Dvurechenski
1.53         (guido    31-Mar-00): Andreas Faerber
1.53         (guido    31-Mar-00): Thomas Heller
1.53         (guido    31-Mar-00): Bernhard Herzog
1.53         (guido    31-Mar-00): Chih-Hao Huang
1.53         (guido    31-Mar-00): Vivek Khera
1.53         (guido    31-Mar-00): Mads Kiilerich
1.53         (guido    31-Mar-00): Greg McFarlane
1.53         (guido    31-Mar-00): Paul Prescod
1.53         (guido    31-Mar-00): Sam Rushing
1.53         (guido    31-Mar-00): John W. Shipman
1.53         (guido    31-Mar-00): Frank Stajano
1.53         (guido    31-Mar-00): Peter Stoehr
1.53         (guido    31-Mar-00): Christian Tanzer
1.53         (guido    31-Mar-00): Jason Trowbridge
1.53         (guido    31-Mar-00): Stephen Turner
1.53         (guido    31-Mar-00): Doug Wyatt
1.54         (guido    31-Mar-00): Bastian Kleineidam
1.54         (guido    31-Mar-00): Thomas Wouters
1.55         (guido    10-Apr-00): Stephen D Evans
1.55         (guido    10-Apr-00): Lars Marius Garshol
1.55         (guido    10-Apr-00): Lance Finn Helsten
1.55         (guido    10-Apr-00): Gregor Hoffleit
1.55         (guido    10-Apr-00): Nadav Horesh
1.55         (guido    10-Apr-00): Lawrence Kesteloot
1.55         (guido    10-Apr-00): Lenny Kneler
1.55         (guido    10-Apr-00): Greg Kochanski
1.55         (guido    10-Apr-00): Hrvoje Niksic
1.55         (guido    10-Apr-00): Nicholas Riley
1.55         (guido    10-Apr-00): David Scherer
1.55         (guido    10-Apr-00): Bruce Sherwood
1.55         (guido    10-Apr-00): Kragen Sitaker
1.56         (guido    11-May-00): Caolan McNamara
1.56         (guido    11-May-00): Trent Mick
1.57         (gvanross 29-Jun-00): Oleg Broytmann
1.57         (gvanross 29-Jun-00): Kalle Svensson
1.58         (gward    29-Jun-00): Harry Henry Gebel
1.59         (bwarsaw  30-Aug-00): James Henstridge
1.60         (gvanross 02-Sep-00): Martin von L?wis
1.60         (gvanross 02-Sep-00): Hajime Saitou
1.61         (gvanross 04-Sep-00): Audun S. Runde
1.62         (tim_one  16-Sep-00): Brad Chapman
1.63         (gvanross 25-Sep-00): Stefan Norberg
1.63         (gvanross 25-Sep-00): Todd R. Palmer
1.63         (gvanross 25-Sep-00): Neale Pickett
1.63         (gvanross 25-Sep-00): Dan Pierson
1.63         (gvanross 25-Sep-00): Fran?ois Pinard
1.63         (gvanross 25-Sep-00): Dan Wolfe
1.63         (gvanross 25-Sep-00): Bob Yodlowski
1.64         (fdrake   02-Oct-00): Daniel Dittmar
1.65         (jhylton  09-Oct-00): Stan Bubrouski
1.67         (gvanross 09-Oct-00): Tony Lownds
1.68         (gvanross 09-Oct-00): Jean-Claude Rimbault
1.69         (gvanross 03-Jan-01): Daniel Calvelo
1.69         (gvanross 03-Jan-01): Mike Clarkson
1.70         (tim_one  14-Jan-01): Jeffery Collins
1.71         (tim_one  18-Jan-01): Paul Jackson
1.72         (akuchlin 19-Jan-01): Jason Tishler
1.73         (gvanross 23-Jan-01): Greg Ball
1.73         (gvanross 23-Jan-01): Bill Bumgarner
1.73         (gvanross 23-Jan-01): Doug Fort
1.73         (gvanross 23-Jan-01): Luke Kenneth Casson Leighton
1.74         (tim_one  26-Jan-01): Ivan Frohne
1.74         (tim_one  26-Jan-01): Janne Sinkkonen
1.75         (tim_one  27-Jan-01): Dave Cole
1.76         (tim_one  28-Jan-01): Randall Kern
1.77         (jhylton  02-Feb-01): Grant Edwards
1.78         (jhylton  05-Feb-01): Nick Mathewson
1.79         (tim_one  09-Feb-01): Lawrence Hudson
1.80         (lemburg  09-Feb-01): Benjamin Collar
1.81         (nascheme 10-Feb-01): David M. Cooke
1.82         (akuchlin 21-Feb-01): Donovan Baarda
1.83         (gvanross 01-Mar-01): Samuele Pedroni
1.84         (tim_one  02-Mar-01): William Tanksley
1.85         (gvanross 02-Mar-01): Peter Schneider-Kamp
1.85         (gvanross 02-Mar-01): Sam Schulenburg
1.85         (gvanross 02-Mar-01): Dietmar Schwertberger
1.87         (gvanross 22-Mar-01): Gordon McMillan
1.88         (gvanross 27-Mar-01): Robin Thomas
1.89         (tim_one  08-Apr-01): Neal Norwitz
1.90         (moshez   09-Apr-01): Itamar Shtull-Trauring
1.91         (gvanross 10-Apr-01): Alastair Burt
1.91         (gvanross 10-Apr-01): Walter D?rwald
1.91         (gvanross 10-Apr-01): Christopher Lee
1.92         (gvanross 11-Apr-01): Billy G. Allie
1.93         (gvanross 14-Apr-01): Pete Shinners
1.94         (gvanross 15-Apr-01): Douglas Orr
1.94         (gvanross 15-Apr-01): Juan M. Bello Rivas
1.95         (tim_one  21-Apr-01): Stephen Hansen
1.96         (tim_one  24-Apr-01): Kevin Rodgers
1.97         (tim_one  09-Jun-01): Armin Rigo
1.98         (tim_one  17-Jun-01): Michael Chermside
1.99         (gvanross 23-Jul-01): Alex Coventry
1.100        (gvanross 31-Jul-01): John Popplewell
1.101        (tim_one  04-Aug-01): David Bolen
1.102        (tim_one  08-Aug-01): Erik van Blokland
1.103        (nascheme 09-Aug-01): Gregory P. Smith
1.104        (gvanross 09-Aug-01): Josh Cogliati
1.105        (tim_one  12-Aug-01): David Goodger
1.106        (akuchlin 13-Aug-01): Tarn Weisner Burton
1.107        (gvanross 17-Aug-01): Henrik Weber
1.109        (gvanross 02-Sep-01): Mike Zarnstorff
1.110        (gvanross 10-Sep-01): Lionel Ulmer
1.111        (gvanross 11-Sep-01): Gerhard H?ring
1.112        (jackjans 11-Sep-01): Donovan Preston
1.113        (tim_one  30-Sep-01): Tom Epperly
1.114        (gvanross 01-Oct-01): Travers Naran
1.115        (tim_one  01-Oct-01): Steven Scott
1.116        (gvanross 05-Oct-01): Doobee R. Tzeck
1.117        (fdrake   05-Oct-01): Fred L. Drake, Jr.
1.118        (loewis   09-Oct-01): Cesar Eduardo Barros
1.119        (gvanross 12-Oct-01): Jason Lowe
1.120        (gvanross 15-Oct-01): Alfonso Baciero
1.121        (gvanross 15-Oct-01): Jason Hildebrand
1.122        (gvanross 15-Oct-01): Richard Townsend
1.123        (gvanross 16-Oct-01): Shane Hathaway
1.124        (gvanross 17-Oct-01): Quinn Dunkan
1.125        (gvanross 17-Oct-01): Joe Norton
1.126        (gvanross 18-Oct-01): Roeland Rengelink
1.127        (gvanross 18-Oct-01): Max Neunh?ffer
1.128        (gvanross 19-Oct-01): Chris Gonnerman
1.129        (gvanross 20-Oct-01): Michael Piotrowski
1.130        (gvanross 23-Oct-01): Jay T. Miller
1.131        (gvanross 25-Oct-01): Dave Brueck
1.132        (gvanross 26-Oct-01): Noah Spurrier
1.133        (gvanross 30-Oct-01): David Cinege
1.134        (tim_one  30-Oct-01): Luigi Ballabio
1.135        (tim_one  31-Oct-01): Michael Muller
1.137        (tim_one  31-Oct-01): Rich Salz
1.138        (tim_one  27-Nov-01): Alex Martelli
1.139        (gvanross 03-Dec-01): Greg Chapman
1.140        (gvanross 04-Dec-01): Stefan Schwarzer
1.141        (gvanross 06-Dec-01): David Abrahams
1.142        (tim_one  07-Dec-01): Anthony Roach
1.143        (gvanross 08-Dec-01): Burton Radons
1.144        (gvanross 10-Dec-01): David Jacobs
1.145        (jhylton  13-Dec-01): Danny Yoo
1.146        (tim_one  19-Dec-01): David Costanzo
1.147        (gvanross 19-Dec-01): Dan Parisien
1.148        (gvanross 20-Dec-01): Mark Summerfield
1.149        (gvanross 20-Dec-01): Artur Zaprzala
1.150        (jackjans 27-Dec-01): Jens B. Jorgensen
1.151        (gvanross 28-Dec-01): Eduardo P?rez
1.152        (loewis   01-Jan-02): James A Morrison
1.153        (loewis   01-Jan-02): Brad Clements
1.154        (loewis   01-Jan-02): Gustavo Niemeyer
1.155        (loewis   12-Jan-02): Robert Kern
1.156        (theller  18-Jan-02): Martin Bless
1.157        (loewis   26-Jan-02): Eddy De Greef
1.158        (loewis   26-Jan-02): Bryce "Zooko" Wilcox-O'Hearn
1.159        (gvanross 25-Feb-02): Edward K. Ream
1.160        (tim_one  28-Feb-02): Gerald S. Williams
1.161        (gvanross 28-Feb-02): Derek Harland
1.162        (gvanross 01-Mar-02): Marc Recht
1.163        (gvanross 01-Mar-02): Naofumi Honda
1.164        (loewis   29-Mar-02): Ondrej Palkovsky
1.165        (loewis   08-Apr-02): Kirill Simonov
1.167        (nnorwitz 12-Apr-02): Raymond Hettinger
1.168        (gvanross 15-Apr-02): Ben Darnell
1.168        (gvanross 15-Apr-02): Vincent Fiack
1.168        (gvanross 15-Apr-02): Frank J. Tobin
1.168        (gvanross 15-Apr-02): Jacques A. Vidrine
1.169        (tim_one  19-Apr-02): Hern?n Mart?nez Foffani
1.170        (loewis   20-Apr-02): Michel Van den Bergh
1.171        (gvanross 23-Apr-02): John Williams
1.172        (loewis   08-May-02): Geert Jansen
1.173        (rhetting 16-May-02): Matthew Dixon Cowles
1.174        (gvanross 21-May-02): Christos Georgiou
1.175        (tim_one  23-May-02): John Aycock
1.176        (gvanross 30-May-02): Holger Krekel
1.177        (loewis   06-Jun-02): Peter ?strand
1.178        (gvanross 06-Jun-02): Michael Gilfix
1.179        (gvanross 07-Jun-02): Bernard Yue
1.180        (gvanross 10-Jun-02): Erik Anders?n
1.181        (loewis   11-Jun-02): Octavian Cerna
1.182        (gvanross 14-Jun-02): Oren Tirosh
1.183        (gvanross 18-Jun-02): Tim Northover
1.184        (rhetting 24-Jun-02): Kevin Jacobs
1.185        (jhylton  28-Jun-02): Bob Kline
1.186        (gvanross 12-Jul-02): Fernando P?rez
1.187        (tim_one  15-Jul-02): Jonathan Hogg
1.188        (tim_one  15-Jul-02): Andrew I MacIntyre
1.188        (tim_one  15-Jul-02): Tim MacKenzie
1.189        (nnorwitz 19-Jul-02): Brett Cannon
1.190        (gvanross 01-Aug-02): Nathan Srebro
1.191        (gvanross 02-Aug-02): Kevin O'Connor
1.192        (gvanross 05-Aug-02): Zack Weinberg
1.193        (tim_one  12-Aug-02): Christopher A. Craig
1.194        (gvanross 12-Aug-02): Cesar Douady
1.194        (gvanross 12-Aug-02): Jiba
1.195        (loewis   14-Aug-02): Hisao Suzuki
1.196        (rhetting 18-Aug-02): Steve Purcell
1.197        (jackjans 19-Aug-02): Michael J. Barber
1.197        (jackjans 19-Aug-02): Chris Barker
1.197        (jackjans 19-Aug-02): Bill Bedford
1.197        (jackjans 19-Aug-02): Erik van Blokland
1.197        (jackjans 19-Aug-02): Tom Bridgman
1.197        (jackjans 19-Aug-02): Daniel Brotsky
1.197        (jackjans 19-Aug-02): Dean Draayer
1.197        (jackjans 19-Aug-02): Russell Finn
1.197        (jackjans 19-Aug-02): Nitin Ganatra
1.197        (jackjans 19-Aug-02): Tony Ingraldi
1.197        (jackjans 19-Aug-02): Tattoo Mabonzo K.
1.197        (jackjans 19-Aug-02): Jacob Kaplan-Moss
1.197        (jackjans 19-Aug-02): Luc Lefebvre
1.197        (jackjans 19-Aug-02): Tom Loredo
1.197        (jackjans 19-Aug-02): Pascal Oberndoerfer
1.197        (jackjans 19-Aug-02): Russel Owen
1.197        (jackjans 19-Aug-02): M. Papillon
1.197        (jackjans 19-Aug-02): Alexandre Parenteau
1.197        (jackjans 19-Aug-02): Christopher Smith
1.197        (jackjans 19-Aug-02): Oliver Steele
1.197        (jackjans 19-Aug-02): Chris Stern
1.197        (jackjans 19-Aug-02): Corran Webster
1.197        (jackjans 19-Aug-02): Gordon Worley
1.198        (gvanross 20-Aug-02): Inyeol Lee
1.199        (jackjans 28-Aug-02): Bill Fancher
1.200        (jackjans 06-Sep-02): Dinu Gherman
1.201        (gvanross 11-Sep-02): Laura Creighton
1.201        (gvanross 11-Sep-02): Anders Qvist
1.202        (gvanross 11-Sep-02): Brian Quinlan
1.203        (gvanross 11-Sep-02): Mats Wichmann
1.204        (gvanross 18-Sep-02): Martijn Pieters
1.205        (gvanross 23-Sep-02): Attila Babo
1.206        (loewis   30-Sep-02): Tim Rice
1.207        (gvanross 09-Oct-02): Flemming Kj?r Jensen
1.207        (gvanross 09-Oct-02): Martin Ligr
1.207        (gvanross 09-Oct-02): Mark Perrego
1.207        (gvanross 09-Oct-02): Jody Winston
1.208        (loewis   10-Oct-02): Steve Reeves
1.209        (nnorwitz 10-Oct-02): Greg Copeland
1.210        (loewis   13-Oct-02): Martin Franklin
1.211        (nnorwitz 17-Oct-02): Maxim Dzumanenko
1.212        (loewis   26-Oct-02): Daniel Stutzbach
1.213        (loewis   26-Oct-02): Matthew Boedicker
1.214        (loewis   04-Nov-02): Ha Shao
1.215        (mwh      08-Nov-02): Richie Hindle
1.216        (loewis   21-Nov-02): Juan David Ib??ez Palomar
1.217        (loewis   22-Nov-02): J?rgen Hermann
1.218        (tim_one  25-Nov-02): Marius Gedminas
1.219        (loewis   06-Dec-02): Jean-Fran?ois Pi?ronne
1.220        (tim_one  28-Dec-02): Ben Laurie
1.221        (loewis   31-Dec-02): Paul Swartz
1.222        (nnorwitz 05-Jan-03): Lars Gust?bel
1.223        (gvanross 07-Jan-03): Michael Stone
1.224        (loewis   15-Jan-03): Gyro Funch
1.224        (loewis   15-Jan-03): Geoff Furnish
1.225        (gvanross 27-Jan-03): Geoff Talvola
1.226        (gvanross 03-Feb-03): Scott David Daniels
1.227        (nnorwitz 10-Feb-03): Jeffrey Ollie
1.227        (nnorwitz 10-Feb-03): Grant Olson
1.227        (nnorwitz 10-Feb-03): Denis S. Otkidach
1.228        (rhetting 09-Mar-03): Sebastien Keim
1.229        (gvanross 14-Mar-03): Stuart Bishop
1.230        (twouters 17-Mar-03): Hye-Shik Chang
1.231        (montanar 20-Mar-03): Kevin Altis
1.231        (montanar 20-Mar-03): Andrew McNamara
1.231        (montanar 20-Mar-03): Cliff Wells
1.232        (jackjans 25-Mar-03): Frank Vercruesse
1.233        (gvanross 19-Apr-03): Terry Carroll
1.234        (gvanross 25-Apr-03): Jp Calderone
1.235        (jhylton  05-May-03): Mihai Ibanescu
1.236        (jhylton  09-May-03): John Anderson
1.237        (bcannon  12-May-03): John J. Lee
1.238        (rhetting 20-May-03): Paul Moore
1.239        (rhetting 09-Jun-03): Stephen M. Gava
1.239        (rhetting 09-Jun-03): Kurt B. Kaiser
1.240        (rhetting 28-Jun-03): Steven Taschuk
1.241        (rhetting 29-Jun-03): Bob Halley
1.242        (jhylton  17-Jul-03): Vincent Delft
1.243        (jhylton  17-Jul-03): Gavrie Philipson
1.244        (akuchlin 13-Aug-03): Joerg Lehmann
1.245        (rhetting 01-Sep-03): George Yoshida
1.246        (rhetting 01-Sep-03): Andrew Gaul
1.247        (rhetting 02-Sep-03): Alexander Belopolsky
1.248        (gvanross 08-Oct-03): Wojtek Walczak
1.249        (bcannon  12-Oct-03): Michael Stone
1.251        (bcannon  14-Oct-03): Bjorn Pettersen
1.252        (loewis   31-Oct-03): Hector Urtubia
1.253        (gvanross 10-Nov-03): Iustin Pop
1.254        (loewis   10-Nov-03): Yves Dionne
1.257        (rhetting 05-Jan-04): Dmitry Vasiliev
1.258        (loewis   31-Jan-04): Frederik Fix
1.259        (rhetting 08-Feb-04): Mike Pall
1.260        (rhetting 04-Mar-04): Bob Ippolito
1.261        (mondrago 23-Mar-04): Nick Bastin

From kbk at shore.net  Mon Apr 26 13:59:29 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon Apr 26 13:59:34 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200404261759.i3QHxT14013977@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  270 open ( +5) /  2366 closed ( +1) /  2636 total ( +6)
Bugs    :  754 open ( +2) /  3993 closed (+10) /  4747 total (+12)
RFE     :  133 open ( -1) /   124 closed ( +0) /   257 total ( -1)

New / Reopened Patches
______________________

CodeContext - an extension to show you where you are  (2004-04-16)
CLOSED http://python.org/sf/936169  reopened by  noamr

Explain 'in' keyword as it is first used  (2004-04-22)
       http://python.org/sf/940026  opened by  Gerrit Holl 

Replace if/elif chain with dispatch pattern in sre_compile  (2004-04-23)
       http://python.org/sf/941071  opened by  Raymond Hettinger 

Allow any encodings other than latin-1 in interactive interp  (2004-04-24)
       http://python.org/sf/941229  opened by  Hye-Shik Chang 

Minimal fix for bug 940578 (glob.glob on broken symlinks)  (2004-04-24)
       http://python.org/sf/941486  opened by  Cherniavsky Beni 

PEP309 Partial implementation  (2004-04-26)
       http://python.org/sf/941881  opened by  Hye-Shik Chang 

Patches Closed
______________

CodeContext - an extension to show you where you are  (2004-04-16)
       http://python.org/sf/936169  closed by  kbk

Py_INCREF/DECREF available as macros only  (2004-04-20)
       http://python.org/sf/938302  closed by  theller

New / Reopened Bugs
___________________

message.as_string hoses headers  (2004-04-20)
CLOSED http://python.org/sf/938890  opened by  Carl Karsten 

cPickle.Pickler: in "list" mode, no way to set protocol  (2004-04-22)
       http://python.org/sf/939395  opened by  John Belmonte 

Building python 2.3.3 RPM on Mandrake 9.2. fails  (2004-04-22)
       http://python.org/sf/939699  opened by  fabien wahl 

Logging failes for rotating file log (during rollOver)  (2004-04-22)
       http://python.org/sf/939836  opened by  smeyers2002 

xml parser bug  (2004-04-22)
CLOSED http://python.org/sf/940263  opened by  David Tefft 

pydoc.Helper.help() ignores input/output init parameters  (2004-04-22)
       http://python.org/sf/940286  opened by  Maxim Krikun 

string.lstrip problem with '_' string  (2004-04-22)
CLOSED http://python.org/sf/940316  opened by  Arnold 

Python crashes with bus error on sparc64.  (2004-04-22)
       http://python.org/sf/940354  opened by  Generic Player 

misbehaviour with map / lambda / isdigit()  (2004-04-23)
CLOSED http://python.org/sf/940522  opened by  dreary 

glob.glob inconsistent about broken symlinks  (2004-04-23)
       http://python.org/sf/940578  opened by  Cherniavsky Beni 

section 5.10: 'not' returns boolean, not int  (2004-04-23)
CLOSED http://python.org/sf/940579  opened by  Jon Ashley 

List with Canvas.create_line Option arrow=LAST Broke  (2004-04-24)
       http://python.org/sf/941262  opened by  Brian Brooks 

AIX shared library fix  (2004-04-24)
       http://python.org/sf/941346  opened by  Dick Dunbar 

Bugs Closed
___________

Random.choice doesn't work with sets.  (2004-04-17)
       http://python.org/sf/936988  closed by  rhettinger

minidom.py writes TEXT_NODE wrong  (2004-04-18)
       http://python.org/sf/937282  closed by  loewis

re.sub on u'' gives NON-unicode ''  (2004-03-31)
       http://python.org/sf/926075  closed by  perky

message.as_string hoses headers  (2004-04-20)
       http://python.org/sf/938890  closed by  carlk

message.as_string hoses headers  (2004-04-20)
       http://python.org/sf/938890  closed by  tim_one

Shelve slow after 7/8000 key  (2004-01-21)
       http://python.org/sf/881522  closed by  tim_one

Slicing changes item-attributes of an instance of a UserList  (2004-04-19)
       http://python.org/sf/937700  closed by  hflori

xml parser bug  (2004-04-22)
       http://python.org/sf/940263  closed by  loewis

string.lstrip problem with '_' string  (2004-04-22)
       http://python.org/sf/940316  closed by  rhettinger

misbehaviour with map / lambda / isdigit()  (2004-04-23)
       http://python.org/sf/940522  closed by  tim_one

section 5.10: 'not' returns boolean, not int  (2004-04-23)
       http://python.org/sf/940579  closed by  rhettinger


From theller at python.net  Tue Apr 27 14:59:25 2004
From: theller at python.net (Thomas Heller)
Date: Tue Apr 27 14:59:34 2004
Subject: [Python-Dev] Need help with patch
Message-ID: <brldp7qq.fsf@python.net>

I would appreciate very much if someone can at least proofread the patch
that I posted to http://python.org/sf/876278.

Even better would be if users of modulefinder would try this with really
complicated applications.

I'm not sure if this should go into the 23-maintanance branch.

Thomas


From ndbecker2 at verizon.net  Tue Apr 27 15:55:46 2004
From: ndbecker2 at verizon.net (Neal D. Becker)
Date: Tue Apr 27 15:55:52 2004
Subject: [Python-Dev] libjit - looks interesting
Message-ID: <c6mds2$sqv$1@sea.gmane.org>

A free jit library.  Might be useful for Python.  Anyone looking at this?

http://www.southern-storm.com.au/libjit.html


From bob at redivi.com  Tue Apr 27 16:20:32 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Apr 27 16:16:14 2004
Subject: [Python-Dev] libjit - looks interesting
In-Reply-To: <c6mds2$sqv$1@sea.gmane.org>
References: <c6mds2$sqv$1@sea.gmane.org>
Message-ID: <55C12706-9888-11D8-B57C-000A95686CD8@redivi.com>


On Apr 27, 2004, at 3:55 PM, Neal D. Becker wrote:

> A free jit library.  Might be useful for Python.  Anyone looking at 
> this?
>
> http://www.southern-storm.com.au/libjit.html

I doubt it, it's GPL'ed.  There are other more mature GPL'ed JIT 
engines out there that we aren't looking at :)

-bob


From martin at v.loewis.de  Tue Apr 27 17:01:23 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Apr 27 17:01:52 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <20040426151607.GA2361@burma.localdomain>
References: <20040425234019.GA6436@burma.localdomain>	<1082988793.17933.59.camel@anthem.wooz.org>
	<20040426151607.GA2361@burma.localdomain>
Message-ID: <408ECA23.8000505@v.loewis.de>

Gustavo Niemeyer wrote:

> - The current scheme may introduce a wrong practice: forcing
>   translators to use some specific encoding to avoid breaking the
>   program.

That's not true. Applications should just always use ugettext.

Regards,
Martin


From martin at v.loewis.de  Tue Apr 27 17:07:27 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Apr 27 17:07:41 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <20040425234019.GA6436@burma.localdomain>
References: <20040425234019.GA6436@burma.localdomain>
Message-ID: <408ECB8F.3070707@v.loewis.de>

Gustavo Niemeyer wrote:

>    # Encode the Unicode tmsg back to an 8-bit string, if possible
>    if self._charset:
> 	return tmsg.encode(self._charset)
> 
> to use the system encoding (sys.getdefaultencoding()) instead of
> self._charset.

That shouldn't be sys.getdefaultencoding(), but
locale.getpreferredencoding().

However, I agree with Barry that the current behaviour should not
be changed. People may already rely on gettext returning byte
strings as-is.


> - Change the default codeset used by gettext.py in functions
>   returning an encoded string to match the system encoding.

No. Explicit is better that implicit; users desiring that
feature should write

_charset = locale.getpreferredencoding()
def _(msg):
   return dgettext("domain", msg).encode(_charset)

I advocate never to use gettext.install, in which case you
have a custom _ implementation *anyway*, which would then
also include the textual domain. It should not be too
much effort for that function to transcode if desired.

> - Introduce bind_textdomain_codeset() in locale.
> - Introduce bind_textdomain_codeset() in gettext.py implementing
>   an equivalent functionality.

That is ok. You could also try to provide that feature
consistently, e.g. inside .install.

Regards,
Martin


From niemeyer at conectiva.com  Tue Apr 27 17:08:56 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Apr 27 17:09:01 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <408ECA23.8000505@v.loewis.de>
References: <20040425234019.GA6436@burma.localdomain>
	<1082988793.17933.59.camel@anthem.wooz.org>
	<20040426151607.GA2361@burma.localdomain>
	<408ECA23.8000505@v.loewis.de>
Message-ID: <20040427210856.GA12923@burma.localdomain>

> >- The current scheme may introduce a wrong practice: forcing
> >  translators to use some specific encoding to avoid breaking the
> >  program.
> 
> That's not true. Applications should just always use ugettext.

As I said before, I agree that ugettext is the best way to get
translations in a generic way. OTOH, that's not what I'm
discussing here, as you may observe by the general discussion.

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Tue Apr 27 17:26:43 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Apr 27 17:26:44 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <408ECB8F.3070707@v.loewis.de>
References: <20040425234019.GA6436@burma.localdomain>
	<408ECB8F.3070707@v.loewis.de>
Message-ID: <20040427212643.GB12923@burma.localdomain>

> That shouldn't be sys.getdefaultencoding(), but
> locale.getpreferredencoding().

Agreed.

> However, I agree with Barry that the current behaviour should not
> be changed. People may already rely on gettext returning byte
> strings as-is.

Barry also said:

"""
Any sane app is going to use the class-based API an the ugettext()
method anyway, so maybe it does make sense to simply be API compatible
with GNU gettext for the old-style interface.
"""

> >- Change the default codeset used by gettext.py in functions
> >  returning an encoded string to match the system encoding.
> 
> No. Explicit is better that implicit; users desiring that
> feature should write
[...]

You belive that returning a string in some unpredictable
encoding used by the translator is explicit? I have to
disagree. We have something named 'gettext' with a different
behavior than the original project, in a way that breaks
the concepts used to decide the behavior of the classical
implementation.

> I advocate never to use gettext.install, in which case you
> have a custom _ implementation *anyway*, which would then
[...]

"You shouldn't be using it" is not a point I'm taking into
account.

> >- Introduce bind_textdomain_codeset() in locale.
> >- Introduce bind_textdomain_codeset() in gettext.py implementing
> >  an equivalent functionality.
> 
> That is ok. You could also try to provide that feature
> consistently, e.g. inside .install.

Agreed.

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Tue Apr 27 17:33:26 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Apr 27 17:33:27 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <408ECB8F.3070707@v.loewis.de>
References: <20040425234019.GA6436@burma.localdomain>
	<408ECB8F.3070707@v.loewis.de>
Message-ID: <20040427213326.GA13163@burma.localdomain>

> However, I agree with Barry that the current behaviour should not
> be changed. People may already rely on gettext returning byte
> strings as-is.

Btw, about people relying on the current implementation, we could
introduce this as a warning in 2.4, which could be disabled by
using "bind_textdomain_codeset()" for the expected translation, and
then turn the behavior into the default one on 2.4+1.

-- 
Gustavo Niemeyer
http://niemeyer.net

From martin at v.loewis.de  Wed Apr 28 14:15:50 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Apr 28 14:16:09 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <20040427212643.GB12923@burma.localdomain>
References: <20040425234019.GA6436@burma.localdomain>	<408ECB8F.3070707@v.loewis.de>
	<20040427212643.GB12923@burma.localdomain>
Message-ID: <408FF4D6.3000108@v.loewis.de>

Gustavo Niemeyer wrote:
> You belive that returning a string in some unpredictable
> encoding used by the translator is explicit? 

No. What is explicit here is that we return the bytes in
the .mo file; and conversion of these bytes should be
requested explicitly.

As for compatibility with GNU gettext: Older versions
of GNU gettext did not perform any conversion, either.
I think changing the behaviour of an existing function
was a bad thing to do for GNU gettext, as well, and I
don't think we should repeat that mistake.

It would be good to find out what users of Python
gettext think about a possible change.

Temporarily warning about this change of behaviour
is also unacceptable. Adding another function, e.g.
lgettext (for local charset), along with lngettext,
ldgettext, might be another solution.

Regards,
Martin


From niemeyer at conectiva.com  Wed Apr 28 14:20:36 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Wed Apr 28 14:20:46 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <408FF4D6.3000108@v.loewis.de>
References: <20040425234019.GA6436@burma.localdomain>
	<408ECB8F.3070707@v.loewis.de>
	<20040427212643.GB12923@burma.localdomain>
	<408FF4D6.3000108@v.loewis.de>
Message-ID: <20040428182036.GA8765@burma.localdomain>

> >You belive that returning a string in some unpredictable
> >encoding used by the translator is explicit? 
> 
> No. What is explicit here is that we return the bytes in
> the .mo file; and conversion of these bytes should be
> requested explicitly.
> 
> As for compatibility with GNU gettext: Older versions
> of GNU gettext did not perform any conversion, either.
> I think changing the behaviour of an existing function
> was a bad thing to do for GNU gettext, as well, and I
> don't think we should repeat that mistake.

I wasn't aware about this.

> It would be good to find out what users of Python
> gettext think about a possible change.

I'd be glad to hear from them as well. Anyone?

> Temporarily warning about this change of behaviour
> is also unacceptable. Adding another function, e.g.
> lgettext (for local charset), along with lngettext,
> ldgettext, might be another solution.

Ack. Thanks for discussing.

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Wed Apr 28 16:43:24 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Wed Apr 28 16:43:27 2004
Subject: [Python-Dev] PEP 754
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEKFKHAB.tim.one@comcast.net>
References: <200404231843.i3NIhC0109140@mcs.anl.gov>
	<LNBBLJKPBEHFEDALKOLCCEKFKHAB.tim.one@comcast.net>
Message-ID: <20040428204324.GA11446@burma.localdomain>

> That isn't a good objection for this specific patch -- it's pure
> Python, and picks apart bits using the struct module.  IIRC, it wasn't
> correctly written for most 64-bit boxes, but that looked like a
> shallow flaw (I think it assumed that a struct "l" format code, in
> native mode, always yields a 32-bit thingie).

I was just wondering, now that C99 supports part of IEEE754, should
we really get into the job of implementing our own mechanism?

As an example, the attached patch exports part of the C99 interface to
Python.

If we agree that this is something interesting to have in Python, I'll
be willing to prepare a patch including support for this interface and
hacking some bits to introduce a better IEEE754 support (e.g. checking
nan comparisons, etc).

-- 
Gustavo Niemeyer
http://niemeyer.net
-------------- next part --------------

#include <Python.h>
#include <math.h>

#define CHECK_FLOAT(o) \
	if (!PyFloat_Check(o)) { \
		PyErr_SetString(PyExc_TypeError, "float argument required"); \
		return NULL; \
	}

static PyObject *
ieee754_isnan(PyObject *self, PyObject *o)
{
	CHECK_FLOAT(o);
	return PyBool_FromLong(isnan(PyFloat_AsDouble(o)));
}

static PyObject *
ieee754_isfinite(PyObject *self, PyObject *o)
{
	CHECK_FLOAT(o);
	return PyBool_FromLong(isfinite(PyFloat_AsDouble(o)));
}

static PyMethodDef ieee754_methods[] = {
	{"isnan", ieee754_isnan, METH_O, ""},
	{"isfinite", ieee754_isfinite, METH_O, ""},
	{NULL, NULL}
};

DL_EXPORT(void)
initieee754(void)
{
	PyObject *m = Py_InitModule3("ieee754", ieee754_methods, "");
	PyModule_AddObject(m, "nan", PyFloat_FromDouble(NAN));
	PyModule_AddObject(m, "inf", PyFloat_FromDouble(INFINITY));
}
From edcjones at erols.com  Thu Apr 29 00:09:41 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Thu Apr 29 00:14:50 2004
Subject: [Python-Dev] Subtyping mutable and immutable built-in types
Message-ID: <40908005.20905@erols.com>

#! /usr/bin/env python

# According to
# http://www.cafepy.com/articles/python_attributes_and_methods/ch03s02.html
# the correct ways to subtype mutable and immutable built-in types are

# Mutable
class MyList(list):
    def __init__(self, itr): 
        list.__init__(self, [int(x) for x in itr])

# Immutable
class MyTuple(tuple):
    def __new__(typ, itr): 
        return tuple.__new__(typ, [int(x) for x in itr]) 

# This doesn't work.
class MyTuple(tuple):
    def __init__(self, itr): 
        tuple.__init__(self, [int(x) for x in itr])

# A variant is
class MyList(list):
    def __init__(self, *args): 
        list.__init__(self, args)

class MyTuple(tuple):
    def __new__(typ, *args): 
        return tuple.__new__(typ, args) 

# This doesn't work.
class MyTuple2(tuple):
    def __init__(self, *args):
        tuple.__init__(self, args)

print MyList(1, 2)
print MyTuple(1, 2)
print MyTuple2(1, 2)

# This warty stuff needs documenting.



From arigo at tunes.org  Thu Apr 29 07:59:11 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Apr 29 08:02:29 2004
Subject: [Python-Dev] Slides: how Psyco works
Message-ID: <20040429115911.GA15802@vicky.ecs.soton.ac.uk>

Hello,

I have put on-line some slides from the Python UK conference at ACCU 2004,
explaining how Psyco works.  It is actually a Pygame application...  As far as
I can tell, it is the first time someone in the room actually understood
something at the end :-)

   http://psyco.sourceforge.net/

I hope it should help to make Psyco a bit less mysterious, and also explain
why it is difficult to have a general idea about what kind of speed-up you can
expect for specific kinds of code: it is, after all, a pretty low-level
"local" process that Psyco does, and it sometimes pays off and sometimes not.
It also explains why this process is very much like the usual interpretation
that CPython does.  If you think about it you might see how useful it would be
for Psyco to build on top of an interpreter in a better language than C (i.e.
one that can be analysed, not just blindly run).


A bient?t,

Armin.


From mchermside at ingdirect.com  Thu Apr 29 08:15:27 2004
From: mchermside at ingdirect.com (Chermside, Michael)
Date: Thu Apr 29 08:15:31 2004
Subject: [Python-Dev] Subtyping mutable and immutable built-in types
Message-ID: <0CFFADBB825C6249A26FDF11C1772AE155096F@ingdexj1.ingdirect.com>

Edward C. Jones writes:
> # According to
> # http://www.cafepy.com/articles/python_attributes_and_methods/ch03s02.html
> # the correct ways to subtype mutable and immutable built-in types are
   [...]
> # This warty stuff needs documenting.

The use of __init__ and __new__ with immutable classes is already
documented at "http://www.python.org/2.2/descrintro.html#__new__".
If you have a suggestion of a another place this should be described,
let us know or better yet file a bug report at SourceForge.

-- Michael Chermside


This email may contain confidential or privileged information. If you believe you have received the message in error, please notify the sender and delete the message without copying or disclosing it.


From barry at python.org  Thu Apr 29 08:26:43 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 29 08:26:50 2004
Subject: [Python-Dev] Small issues in gettext support
In-Reply-To: <408FF4D6.3000108@v.loewis.de>
References: <20040425234019.GA6436@burma.localdomain>
	<408ECB8F.3070707@v.loewis.de>
	<20040427212643.GB12923@burma.localdomain>
	<408FF4D6.3000108@v.loewis.de>
Message-ID: <1083241602.20310.82.camel@anthem.wooz.org>

On Wed, 2004-04-28 at 14:15, "Martin v. L?wis" wrote:

> Temporarily warning about this change of behaviour
> is also unacceptable. Adding another function, e.g.
> lgettext (for local charset), along with lngettext,
> ldgettext, might be another solution.

That would work for me.  I'd much rather see a different interface to
support what Gustavo wants than to change an existing interface.

-Barry



From aahz at pythoncraft.com  Thu Apr 29 10:19:34 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 29 10:19:38 2004
Subject: [Python-Dev] Subtyping mutable and immutable built-in types
In-Reply-To: <0CFFADBB825C6249A26FDF11C1772AE155096F@ingdexj1.ingdirect.com>
References: <0CFFADBB825C6249A26FDF11C1772AE155096F@ingdexj1.ingdirect.com>
Message-ID: <20040429141934.GB29986@panix.com>

On Thu, Apr 29, 2004, Chermside, Michael wrote:
>
> The use of __init__ and __new__ with immutable classes is already
> documented at "http://www.python.org/2.2/descrintro.html#__new__".
> If you have a suggestion of a another place this should be described,
> let us know or better yet file a bug report at SourceForge.

It's also much easier to find now under "Documentation" | "New-style
classes".
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From guido at python.org  Thu Apr 29 10:20:16 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Apr 29 10:20:22 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
Message-ID: <200404291420.i3TEKGn05101@guido.python.org>

When I tried "cvs update" today, I got an error from ssh:

"""
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@       WARNING: POSSIBLE DNS SPOOFING DETECTED!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
The RSA1 host key for cvs.python.sourceforge.net has changed,
and the key for the according IP address 66.35.250.209
is unknown. This could either mean that
DNS SPOOFING is happening or the IP address for the host
and its host key have changed at the same time.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA1 host key has just been changed.
The fingerprint for the RSA1 key sent by the remote host is
33:50:33:91:b4:ec:7a:b6:fa:39:99:b0:ff:65:63:a7.
Please contact your system administrator.
Add correct host key in /home/guido/.ssh/known_hosts to get rid of this message.
Offending key in /home/guido/.ssh/known_hosts:1
RSA1 host key for cvs.python.sourceforge.net has changed and you have requested strict checking.
Host key verification failed.
cvs [update aborted]: end of file from server (consult above messages if any)
"""

After deleting the key from my known_hosts file, all I get is requests
for a password, which I refuse to give (since it shouldn't be
necessary and I don't want to rule out a man-in-the-middle attack
yet).  I've tried both ssh -1 and ssh -2.

Has this happened to anybody else?  The SF.net Site Status Page shows
nothing relevant.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry at python.org  Thu Apr 29 10:26:17 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 29 10:26:33 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
In-Reply-To: <200404291420.i3TEKGn05101@guido.python.org>
References: <200404291420.i3TEKGn05101@guido.python.org>
Message-ID: <1083248776.21368.74.camel@anthem.wooz.org>

On Thu, 2004-04-29 at 10:20, Guido van Rossum wrote:
> When I tried "cvs update" today, I got an error from ssh:

Same thing happens to me when I use cvs.python.sf.net:/cvsroot/python
but not when I use cvs.sf.net:/cvsroot/python.  I think the latter is
the current approved way of connecting to the cvs repository (I've been
using that form with Mailman for a while and also have no problems
there).

I'll attach a nice little cvs_chroot script that Greg Ward sent me a
long time ago.  I'm sure he won't mind.  cd to your python directory and
type:

% cvs_chroot cvs.sf.net:/cvsroot/python

Then do a cvs up and all should be golden.

-Barry

-------------- next part --------------
#!/usr/bin/env python
"""cvs_chroot

Change the repository directory of an existing CVS working directory
(actually, a whole working tree).  Very handy when you're moving
repositories around and don't want to re-check-out the whole world; nearly
essential when you have a heavily customized working directory (lots of
uncommited changes, a forest of symlinks to make everything work, etc.).

Using "cvs_chroot" is simple: just put yourself in the top of the working
directory that needs to be adjusted, and run

    cvs_chroot <new_repository_root>

For example, if you've just moved your CVS repository from /home/cvs to
/cvs, then you would go to some working directory and run

    cvs_chroot /cvs

Or if you've just uploaded a local project to a remote CVS server, so that
all the net may share in its development, you might do something like this:

    cvs_chroot :pserver:anonymous@cvs.python.sourceforge.net:/cvsroot/python

(assuming that the project in question is Python, and the remote CVS server
is one of SourceForge's).  Of course, this also applies if you just happen
to have a working tree of the project in question -- you don't have to be
the one who uploaded it to SourceForge (or wherever).

If you're paranoid and/or curious, just run "chroot" with the "-n" option
-- it'll tell you what it would do, without actually doing anything.
"""

# created 2000/05/12 (Greg Ward)
# documented 2000/05/17

__revision__ = "$Id: cvs_chroot,v 2.3 2002/09/28 16:06:08 barry Exp $"

import sys, os, string
import getopt
import errno

USAGE = "usage: %s [-n] newroot"


"""If you're curious about how it works, here's the whole story:
"cvs_chroot" walks a directory tree, finding all CVS directories under it,
and adjusting the "Root" and "Repository" file in each CVS directory.  The
terminology is a bit wonky here (blame CVS), so pay attention: the "Root"
files contain the location of the repository -- ie. the directory named in
the CVSROOT environment variable or specified in the CVS "-d" option.  For
example, if you check something out as follows:

    CVSROOT=/home/cvs cvs checkout project

(or cvs -d /home/cvs checkout project; the two are equivalent), then every
"CVS/Root" file under "project" will contain the sole line "/home/cvs".
Every "CVS/Repository" file will contain the full path to the repository
directory for that particular working directory, or a path which is
relative to the root (CVS supports both forms).  For example, if "project"
is laid out like:

    project --+ src ---+ module1
              |        | module2
              | doc

then the file "project/src/CVS/Repository" would contain either
"/home/cvs/project/src" or "project/src", and the file
"project/src/module1/CVS/Repository" would contain either
"/home/cvs/project/src/module1" or "project/src/module1".

If you're dealing with a remote repository, then things are a bit more
complex: the "Root" files are still all the same and contain the value of
CVSROOT or the "-d" option when you checked the project out, but the
Repository files leave off the remote bit (everything up to the last
colon).  For example, if you checkout "project" from a remote server:

   cvs -d :pserver:anon@cvs.somewhere.net:/home/cvs checkout project

then the "CVS/Root" files under "project" will all contain
":pserver:anon@cvs.somewhere.net:/home/cvs", but the "CVS/Repository" files
will be the same as before -- i.e., they will start with "/home/cvs".  This
time, of course, that's "/home/cvs" on the host cvs.somewhere.net.

Thus, cvs_chroot has to do the following for each CVS directory it finds:
  * replace the contents of "Root" with the new repository root
  * chop off the old repository root fragment (directory only, no
    host information) from the Repository file, and replace
    it with the new repository root fragment

If cvs_chroot finds any inconsistencies in the Root or Repository files of
your working directory, it prints a warning and skips the afflicted
directory.  (This can happen if you have a subtree of your working tree
checked out differently, eg. from a different repository.  In that case,
you'll have to run "cvs_chroot" on each such oddball subtree.)
"""

def warn (msg):
    lines = string.split(msg, "\n")
    sys.stderr.write("warning: " + lines[0] + "\n")
    for l in lines[1:]:
        sys.stderr.write ("         " + l + "\n")

def find_cvs_dirs (start_dir):
    def callback(arg, directory, files):
        for file in files:
	    if os.path.isdir(file) and file == "CVS":
                arg.append(os.path.join(directory, file))
    dirs = []
    os.path.walk(start_dir, callback, dirs)
    return dirs

def split_root (root):
    lastcolon = string.rfind(root, ":")
    if lastcolon != -1:
        root_host = root[0:lastcolon]
        root_dir = root[lastcolon+1:]
    else:
        root_host = None
        root_dir = root
    return (root_host, root_dir)

def repos_filenames (dir):
    return (os.path.join(dir, "Root"),
            os.path.join(dir, "Repository"))

def read_repos (dir):
    (root_fn, repos_fn) = repos_filenames(dir)
    root = open(root_fn).readline()[0:-1]
    repos = open(repos_fn).readline()[0:-1]
    return (root, repos)

def write_repos (dir, root, repos):
    (root_fn, repos_fn) = repos_filenames(dir)
    root_bk = root_fn + "~"
    repos_bk = repos_fn + "~"
    if os.path.exists(root_bk):
        os.remove(root_bk)
    if os.path.exists(repos_bk):
        os.remove(repos_bk)
    try:
        os.rename(root_fn, root_bk)
        os.rename(repos_fn, repos_bk)
        open(root_fn, "w").write(root + "\n")
        open(repos_fn, "w").write(repos + "\n")
    except (IOError, os.error):
        try:
            os.rename(root_bk, root_fn)
            os.rename(repos_bk, repos_fn)
        except (IOError, os.error):
            pass
        

def main (prog, args):
    usage = USAGE % os.path.basename(prog)

    dry_run = None
    help = None
    try:
        (opts, args) = getopt.getopt(args, "nh")
        for (opt, _) in opts:
            if opt == "-n":
                dry_run = 1
            elif opt == "-h":
                help = 1
    except getopt.error, msg:
        raise SystemExit, usage + "\n" + str(msg)

    if help:
        print __doc__
        print usage
        sys.exit(0)

    if len(args) != 1:
        raise SystemExit, \
              usage + "\nWrong number of arguments"

    new_root = args[0]
    (new_root_host, new_root_dir) = split_root(new_root)

    (top_root, top_repos) = read_repos ("CVS")

    sys.stdout.write("Finding CVS directories..."); sys.stdout.flush()
    cvsdirs = find_cvs_dirs (".")
    sys.stdout.write("found %d of them\n" % len(cvsdirs))
    
    for dir in cvsdirs:
        try:
            (root, repos) = read_repos(dir)
        except IOError, e:
            if e.errno <> errno.ENOTDIR: raise
            continue
        (root_host, root_dir) = split_root(root)
        orig_repos = repos              # so we can tweak 'repos' but still
                                        # report the original value

        # The CVS/Root file must be consistent throughout the entire
        # working tree; skip any directory where it's not the same as
        # the top-level CVS/Root file.
        if root != top_root:
            warn(("root in %s (%s) doesn't match\n" 
                  "top-level root (%s) (skipping)")
                  % (dir, root, top_root))
            continue

        # Checking the Repository file needs to know if it, and the
        # top-level repository, are absolute or relative.
        repos_abs = (repos[0] == '/')
        top_repos_abs = (top_repos[0] == '/')

        # If the CVS/Repository file is absolute (which happens under
        # older versions of RCS -- at least 1.10.6 has relative
        # Repository files), then the prefix of the repository directory
        # must match the directory portion of the root.
        if repos_abs and repos[0:len(root_dir)] != root_dir:
            warn(("repository at %s (%s) not under "
                  "root dir (%s) (skipping)")
                 % (dir, repos, root_dir))
            continue

        # If the top-level repository is absolute, but the current one
        # is not, then force the current repository into compliance with
        # the top-level repository -- i.e. make it absolute.
        if top_repos_abs and not repos_abs:
            repos = root_dir + '/' + repos
            repos_abs = 1

        # Other way round: make the current repository relative to match
        # the top-level repository.
        elif repos_abs and not top_repos_abs:
            repos = repos[len(root_dir)+1:]
            repos_abs = 0

        # Now we can make sure that the current repository is valid,
        # ie. is some descendant of the top-level repository.
        if repos[0:len(top_repos)] != top_repos:
            warn(("repository at %s (%s) not under\n"
                  "top-level repository (%s) (skipping)")
                  % (dir, repos, top_repos))
            continue

        # Now, at last we can generate a new repository directory, which
        # is the point of the whole exercise.  It will be absolute if
        # the top-level repository (not necessarily the current
        # repository) is absolute, and relative if the top-level
        # repository is relative.
        if repos[0] == '/':
            new_repos = new_root_dir + repos[len(root_dir):]
        else:
            new_repos = repos

        print dir + ":"
        print "  root: %s -> %s" % (root, new_root)
        print "  repos: %s -> %s" % (orig_repos, new_repos)

        if not dry_run:
            write_repos (dir, new_root, new_repos)

    # for dir in cvsdirs
    
# main ()    


if __name__ == "__main__":
    (prog, args) = (sys.argv[0], sys.argv[1:])
    main (prog, args)
From theller at python.net  Thu Apr 29 10:49:55 2004
From: theller at python.net (Thomas Heller)
Date: Thu Apr 29 10:50:02 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
In-Reply-To: <1083248776.21368.74.camel@anthem.wooz.org> (Barry Warsaw's
	message of "Thu, 29 Apr 2004 10:26:17 -0400")
References: <200404291420.i3TEKGn05101@guido.python.org>
	<1083248776.21368.74.camel@anthem.wooz.org>
Message-ID: <d65q7sa4.fsf@python.net>

Barry Warsaw <barry@python.org> writes:

> On Thu, 2004-04-29 at 10:20, Guido van Rossum wrote:
>> When I tried "cvs update" today, I got an error from ssh:
>
> Same thing happens to me when I use cvs.python.sf.net:/cvsroot/python
> but not when I use cvs.sf.net:/cvsroot/python.  I think the latter is
> the current approved way of connecting to the cvs repository (I've been
> using that form with Mailman for a while and also have no problems
> there).

Yes, this is also what they mention on the CVS pages.

> I'll attach a nice little cvs_chroot script that Greg Ward sent me a
> long time ago.  I'm sure he won't mind.  cd to your python directory and
> type:
>
> % cvs_chroot cvs.sf.net:/cvsroot/python
>
> Then do a cvs up and all should be golden.

I had to type

  cvs_chroot :ext:<name>@cvs.sourceforge.net:/cvsroot/python

where <name> is the sourceforge users name.

Greg's script even works on windows, as it seems.

Thomas


From aahz at pythoncraft.com  Thu Apr 29 10:50:38 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 29 10:50:57 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
In-Reply-To: <1083248776.21368.74.camel@anthem.wooz.org>
References: <200404291420.i3TEKGn05101@guido.python.org>
	<1083248776.21368.74.camel@anthem.wooz.org>
Message-ID: <20040429145038.GA20795@panix.com>

On Thu, Apr 29, 2004, Barry Warsaw wrote:
>
> Same thing happens to me when I use cvs.python.sf.net:/cvsroot/python
> but not when I use cvs.sf.net:/cvsroot/python.  I think the latter is
> the current approved way of connecting to the cvs repository (I've been
> using that form with Mailman for a while and also have no problems
> there).

Mind updating the following pages?

http://www.python.org/dev/devfaq.html
http://www.python.org/dev/tools.html

(I'd do it myself, but I think it'd be better for someone who actually
knows what zie's doing to handle it.  Fortunately,
http://sourceforge.net/cvs/?group_id=5470
already has the correct info.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From barry at python.org  Thu Apr 29 11:00:49 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 29 11:01:01 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
In-Reply-To: <d65q7sa4.fsf@python.net>
References: <200404291420.i3TEKGn05101@guido.python.org>
	<1083248776.21368.74.camel@anthem.wooz.org> <d65q7sa4.fsf@python.net>
Message-ID: <1083250848.21368.93.camel@anthem.wooz.org>

On Thu, 2004-04-29 at 10:49, Thomas Heller wrote:

>   cvs_chroot :ext:<name>@cvs.sourceforge.net:/cvsroot/python

Ah, I have stuff set up in my .ssh/config file so I don't need the
:ext:<name>@ prefix.

-Barry



From barry at python.org  Thu Apr 29 11:35:38 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Apr 29 11:35:47 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
In-Reply-To: <20040429145038.GA20795@panix.com>
References: <200404291420.i3TEKGn05101@guido.python.org>
	<1083248776.21368.74.camel@anthem.wooz.org>
	<20040429145038.GA20795@panix.com>
Message-ID: <1083252937.21368.111.camel@anthem.wooz.org>

On Thu, 2004-04-29 at 10:50, Aahz wrote:
> On Thu, Apr 29, 2004, Barry Warsaw wrote:
> >
> > Same thing happens to me when I use cvs.python.sf.net:/cvsroot/python
> > but not when I use cvs.sf.net:/cvsroot/python.  I think the latter is
> > the current approved way of connecting to the cvs repository (I've been
> > using that form with Mailman for a while and also have no problems
> > there).
> 
> Mind updating the following pages?
> 
> http://www.python.org/dev/devfaq.html
> http://www.python.org/dev/tools.html
> 
> (I'd do it myself, but I think it'd be better for someone who actually
> knows what zie's doing to handle it.  Fortunately,
> http://sourceforge.net/cvs/?group_id=5470
> already has the correct info.)

Done.
-Barry



From tismer at stackless.com  Thu Apr 29 11:45:46 2004
From: tismer at stackless.com (Christian Tismer)
Date: Thu Apr 29 11:44:43 2004
Subject: [Python-Dev] Re: [pypy-dev] Slides: how Psyco works
In-Reply-To: <20040429115911.GA15802@vicky.ecs.soton.ac.uk>
References: <20040429115911.GA15802@vicky.ecs.soton.ac.uk>
Message-ID: <4091232A.5020607@stackless.com>

Armin Rigo wrote:

> Hello,
> 
> I have put on-line some slides from the Python UK conference at ACCU 2004,
> explaining how Psyco works.  It is actually a Pygame application...  As far as
> I can tell, it is the first time someone in the room actually understood
> something at the end :-)
> 
>    http://psyco.sourceforge.net/

Hey, *this* is a nice animation, with my beloved Lemmings :-)

> I hope it should help to make Psyco a bit less mysterious, and also explain
> why it is difficult to have a general idea about what kind of speed-up you can
> expect for specific kinds of code: it is, after all, a pretty low-level
> "local" process that Psyco does, and it sometimes pays off and sometimes not.
> It also explains why this process is very much like the usual interpretation
> that CPython does.  If you think about it you might see how useful it would be
> for Psyco to build on top of an interpreter in a better language than C (i.e.
> one that can be analysed, not just blindly run).

Yes, this is very clear.
For PyPy, it will be interesting to see how efficient Psyco's
peephole-like optimization is compared to whole-program analysis.
There are probably programs simple enough to be completely
optimized through by WPA.
On the other hand, the possible optimization paths can grow
to very many, giving a huge code bloat. In that case, Psyco's
approach is probably more efficient, by creating new code
only when it is needed.
In the end, I guess both strategies should be combined to
get some optimum between analysis time, speed and memory
consumption.

Anyway, I can highly recommend to watch this nice movie
from Rigo Productions (C) :-))

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From aahz at pythoncraft.com  Thu Apr 29 12:47:37 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Apr 29 12:47:45 2004
Subject: [Python-Dev] SSH problems getting into SourceForge's CVS?
In-Reply-To: <1083252937.21368.111.camel@anthem.wooz.org>
References: <200404291420.i3TEKGn05101@guido.python.org>
	<1083248776.21368.74.camel@anthem.wooz.org>
	<20040429145038.GA20795@panix.com>
	<1083252937.21368.111.camel@anthem.wooz.org>
Message-ID: <20040429164737.GA27313@panix.com>

On Thu, Apr 29, 2004, Barry Warsaw wrote:
> On Thu, 2004-04-29 at 10:50, Aahz wrote:
>> On Thu, Apr 29, 2004, Barry Warsaw wrote:
>>>
>>> Same thing happens to me when I use cvs.python.sf.net:/cvsroot/python
>>> but not when I use cvs.sf.net:/cvsroot/python.  I think the latter is
>>> the current approved way of connecting to the cvs repository (I've been
>>> using that form with Mailman for a while and also have no problems
>>> there).
>> 
>> Mind updating the following pages?
>> 
>> http://www.python.org/dev/devfaq.html
>> http://www.python.org/dev/tools.html
> 
> Done.

Thanks!
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From niemeyer at conectiva.com  Thu Apr 29 13:01:43 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Apr 29 13:01:46 2004
Subject: [Python-Dev] Slides: how Psyco works
In-Reply-To: <20040429115911.GA15802@vicky.ecs.soton.ac.uk>
References: <20040429115911.GA15802@vicky.ecs.soton.ac.uk>
Message-ID: <20040429170142.GA4820@burma.localdomain>

> I have put on-line some slides from the Python UK conference at ACCU
> 2004, explaining how Psyco works.  It is actually a Pygame
> application...  As far as I can tell, it is the first time someone in
> the room actually understood something at the end :-)

Impressive presentation! Congratulations!

-- 
Gustavo Niemeyer
http://niemeyer.net

From jim.jewett at eds.com  Thu Apr 29 14:28:26 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu Apr 29 14:29:10 2004
Subject: [Python-Dev] new datatypes - should they subclasses (deque(list)?)
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A0@usahm010.exmi01.exch.eds.com>

The collections module will be offering 
new datatypes, including a double-ended
queue (deque).

Right now, these are new elemental types,
inheriting directly from object.  I believe
they should be subclassed from existing 
classes where this makes sense.  Even if
no actual methods are inherited, the API
is valuable.

For instance, should:

	list([1,2,3]) == deque([1,2,3])
	deque([1]) < list([1,2]) < deque([3])

If deque inherits directly from object,
then the only comparison available is
based on address.  If deque subclasses
list (or at least pretends to), then
sensible comparisons can be made.

All list operations make sense on a deque
(and vice versa).  Operations may be more 
(or less) efficient depending on the type, 
but they do make sense.  Even if the concern 
is to avoid encouraging inefficient code, 
the methods should still be available.

	d = deque(...)
	d[4:11] = ('x', 'y')

is inefficient, but still better than

	d = deque(...)
	temp = list(d)
	temp[4:11] = ('x', 'y')
	d = deque(temp)

As an additional benefit, making deque a 
subclass of list with the same API gives
later python implementations more freedom;
in theory, the compiler could determine
which internal representation made more
sense for each particular case.

-jJ

From niemeyer at conectiva.com  Thu Apr 29 15:47:35 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Apr 29 15:47:35 2004
Subject: [Python-Dev] Small additions to optparser
Message-ID: <20040429194735.GA7068@burma.localdomain>

Hi Folks, 

I have two small customizations I've used in most projects I've
plugged optparse into. Since they're simple and generic, I'd
like to integrate them back in optparser. These features are:

- Add a help formatter which uses capitalized headers.
- Add a "help" keyword in OptionParser, allowing straightforward
  addition of a custom help messages.

Comments?

Also, how are we currently handling changes in optparser? Do we
have an "official" maintainer (Greg?). IOW, should I:

- Send a proposed patch to SourceForge
- Contact the original project
- Commit the patch


FWIW, here is the proposed functionality:


class CapitalizedHelpFormatter(optparse.IndentedHelpFormatter):

    def format_usage(self, usage):
        return optparse.IndentedHelpFormatter \
                .format_usage(self, usage).capitalize()

    def format_heading(self, heading):
        return optparse.IndentedHelpFormatter \
                .format_heading(self, heading).capitalize()

class OptionParser(optparse.OptionParser):

    def __init__(self, *args, **kwargs):
        optparse.OptionParser.__init__(self, *args, **kwargs)
        self._override_help = kwargs.get("help")

    def format_help(self, formatter=None):
        if self._override_help:
            return self._override_help
        else:
            return optparse.OptionParser.format_help(self, formatter)

-- 
Gustavo Niemeyer
http://niemeyer.net

From captainjacksparrow at indiatimes.com  Thu Apr 29 23:39:41 2004
From: captainjacksparrow at indiatimes.com (captainjacksparrow)
Date: Thu Apr 29 23:45:08 2004
Subject: [Python-Dev] Mozpython help
Message-ID: <200404300305.IAA03360@WS0005.indiatimes.com>

hi,

i have tried to run Mozpython 0.1.1 on my i386 running Python 2.3.3 and Mozilla 1.6. The compilation is successful but when i try to run a python script in Mozilla it gives me a lost sys.stderr
ny comments r most welcome.

Jack
Get Your Private, Free E-mail from Indiatimes at http://email.indiatimes.com

 Buy The Best In BOOKS at http://www.bestsellers.indiatimes.com

Bid for for Air Tickets @ Re.1 on Air Sahara Flights. Just log on to http://airsahara.indiatimes.com and Bid Now!


From tjreedy at udel.edu  Fri Apr 30 01:22:25 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri Apr 30 01:31:15 2004
Subject: [Python-Dev] Re: Mozpython help
References: <200404300305.IAA03360@WS0005.indiatimes.com>
Message-ID: <c6snq9$7pd$1@sea.gmane.org>


"captainjacksparrow" <captainjacksparrow@indiatimes.com> wrote in message
news:200404300305.IAA03360@WS0005.indiatimes.com...
> i have tried to run Mozpython 0.1.1 on my i386 running Python 2.3.3 and
Mozilla 1.6. The compilation is successful but when i try to run a python
script in Mozilla it gives me a lost sys.stderr
> ny comments r most welcome.

This list is about developing future versions of Python.  Questions about
using current Python are best directed to comp.lang.python or specific
mailing lists when there are such.

tjr




From arigo at tunes.org  Fri Apr 30 06:41:58 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Apr 30 06:45:10 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200404231826.i3NIQxM18955@guido.python.org>
References: <200404231826.i3NIQxM18955@guido.python.org>
Message-ID: <20040430104158.GA31871@vicky.ecs.soton.ac.uk>

Hello Guido,

I did a quick review of the stdlib, including the tests, to see which list
comprehensions could be replaced with generator expressions.  Thought I admit
I am biased towards early binding, I ended up looking for cases with the
following properties:

* doesn't use the result immediately
* contains free variables

This is less that 10% of the cases.  However, for each of them, I had to check
if the free variables had a chance of being modified after the genexpr.  This
is pretty rare, admittedly, but there is an example in test/test_random.py
(test_avg_std).  There are a number of other examples that just don't happen
to modify the free variables, by chance; e.g. in optparse.py:

    metavar = option.metavar or option.dest.upper()
    short_opts = [sopt + metavar for sopt in option._short_opts]
    long_opts = [lopt + "=" + metavar for lopt in option._long_opts]

If we find out later that long_opts actually needs a slightly different value
for metavar, it would be tempting to do:

    metavar = option.metavar or option.dest.upper()
    short_opts = [sopt + metavar for sopt in option._short_opts]
    metavar = option.metavar or option.dest.capitalize()
    long_opts = [lopt + "=" + metavar for lopt in option._long_opts]

Replace these with genexprs and it doesn't work any more: the 2nd metavar is
unexpectedly used in the first genexpr as well.  In general I find it strange
to have to look if a given variable could be modified much later in the same
function to be sure of what a genexpr really means.  Early binding is closer
to the idea that turning a listcomp into a genexprs should just work if you
only iterate once on the result.


A bient?t,

Armin.


From guido at python.org  Fri Apr 30 08:16:32 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr 30 08:16:39 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: Your message of "Fri, 30 Apr 2004 11:41:58 BST."
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk> 
References: <200404231826.i3NIQxM18955@guido.python.org>  
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk> 
Message-ID: <200404301216.i3UCGXU07211@guido.python.org>

> I did a quick review of the stdlib, including the tests, to see
> which list comprehensions could be replaced with generator
> expressions.  Thought I admit I am biased towards early binding, I
> ended up looking for cases with the following properties:
> 
> * doesn't use the result immediately
> * contains free variables
> 
> This is less that 10% of the cases.  However, for each of them, I
> had to check if the free variables had a chance of being modified
> after the genexpr.  This is pretty rare, admittedly, but there is an
> example in test/test_random.py (test_avg_std).  There are a number
> of other examples that just don't happen to modify the free
> variables, by chance; e.g. in optparse.py:
> 
>     metavar = option.metavar or option.dest.upper()
>     short_opts = [sopt + metavar for sopt in option._short_opts]
>     long_opts = [lopt + "=" + metavar for lopt in option._long_opts]
> 
> If we find out later that long_opts actually needs a slightly
> different value for metavar, it would be tempting to do:
> 
>     metavar = option.metavar or option.dest.upper()
>     short_opts = [sopt + metavar for sopt in option._short_opts]
>     metavar = option.metavar or option.dest.capitalize()
>     long_opts = [lopt + "=" + metavar for lopt in option._long_opts]
> 
> Replace these with genexprs and it doesn't work any more: the 2nd
> metavar is unexpectedly used in the first genexpr as well.  In
> general I find it strange to have to look if a given variable could
> be modified much later in the same function to be sure of what a
> genexpr really means.  Early binding is closer to the idea that
> turning a listcomp into a genexprs should just work if you only
> iterate once on the result.

Thanks for the analysis.  My comment on this: why on earth would you
want to replace the perfectly sensible list comprehension with a
generator comprehension in this particular case?  (Note that the
remainder of that function proceeds to do list concatenation of the
two lists.)

A more general question would be: have you found any examples of list
comprehensions that weren't immediately used where there would be a
compelling reason to use a generator expression instead?  In this
example, I can't see any of the touted advantages -- the set of
command line options will never be long enough to have to worry about
the memory consumption.  How about the other examples you found?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis at bluewin.ch  Fri Apr 30 08:48:06 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Fri Apr 30 08:42:37 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200404301216.i3UCGXU07211@guido.python.org>
References: <Your message of "Fri, 30 Apr 2004 11:41:58 BST."
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
Message-ID: <5.2.1.1.0.20040430142620.030304e8@pop.bluewin.ch>

At 05:16 30.04.2004 -0700, Guido van Rossum wrote:


>A more general question would be: have you found any examples of list
>comprehensions that weren't immediately used where there would be a
>compelling reason to use a generator expression instead?  In this
>example, I can't see any of the touted advantages -- the set of
>command line options will never be long enough to have to worry about
>the memory consumption.  How about the other examples you found?

this assumes that people will choose either generator expressions or list 
comprehesions on a case-by-case basis. I'm not sure it's a reasonable 
assumption for the whole range of users - starting with newbies. Probably 
documentation and what is predicated on c.l.p will influence this.

OTOH both are probably good enough default choices, list comprehesions 
because if one gets insufficient performance she will learn to switch, 
generator expressions because for small numbers of elements things are a 
wash (unless someone is cretating a lot of small sequences :)).So the more 
they are interchangeable in all other respects the better.

"There should be one-- and preferably only one --obvious way to do it."
of course this is the case for generator expressions and list 
comprehsensions because they are separated
by performance characteristics and somehow by the fact that one directly 
produces a list. I suspect that we have to learn to live with the fact that 
they are distinguished by (just) that, both a nuisance (sometimes people 
don't want to bother to choose) and  a feature.

regards. 


From arigo at tunes.org  Fri Apr 30 09:07:00 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Apr 30 09:10:07 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200404301216.i3UCGXU07211@guido.python.org>
References: <200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404301216.i3UCGXU07211@guido.python.org>
Message-ID: <20040430130700.GA28030@vicky.ecs.soton.ac.uk>

Hello,

On Fri, Apr 30, 2004 at 05:16:32AM -0700, Guido van Rossum wrote:
> Thanks for the analysis.  My comment on this: why on earth would you
> want to replace the perfectly sensible list comprehension with a
> generator comprehension in this particular case?

I was using this example more generally to show that if you have a
late-binding genexpr, you have to worry about not changing the local variables
it uses, which looks like all uses of genexprs with free variables are subtle
bugs waiting to appear whenever someone modifies the function anywhere after
the genexpr.  The same argument applies to the other stdlib cases I mentioned
(which make somewhat less than 10% of all uses).

The alternative -- actively using the late-binding aspect -- would lead to
code in which you have to worry about which values the local variables have at
the moment the next element is fetched from the iterator.  For example:

    x_square = (x*x for i in itertools.repeat(None)).next
    ...
    x = 5
    print x_square()    # 25
    x = 6
    print x_square()    # 36

This looks (a) cool and (b) a complete hack that nobody should be allowed to
do without messing with sys._getframe().


A bient?t,

Armin.


From barry at python.org  Fri Apr 30 09:30:30 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr 30 09:30:43 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <20040430130700.GA28030@vicky.ecs.soton.ac.uk>
References: <200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404301216.i3UCGXU07211@guido.python.org>
	<20040430130700.GA28030@vicky.ecs.soton.ac.uk>
Message-ID: <1083331829.29603.14.camel@anthem.wooz.org>

On Fri, 2004-04-30 at 09:07, Armin Rigo wrote:

> This looks (a) cool and (b) a complete hack that nobody should be allowed to
> do without messing with sys._getframe().

It reminds me a little bit of the dynamic binding in Emacs lisp.  As
incredibly useful as that is sometimes, it's a disgusting hack. :)

-Barry



From dmgass at hotmail.com  Fri Apr 30 11:23:05 2004
From: dmgass at hotmail.com (Dan Gass)
Date: Fri Apr 30 11:23:10 2004
Subject: [Python-Dev] Small additions to optparser
Message-ID: <BAY18-F88JXOxJagXTi0002378c@hotmail.com>




>From: Gustavo Niemeyer <niemeyer@conectiva.com>
>To: python-dev@python.org
>CC: gward@python.net
>Subject: [Python-Dev] Small additions to optparser
>Date: Thu, 29 Apr 2004 16:47:35 -0300
>
>Hi Folks,
>
>I have two small customizations I've used in most projects I've
>plugged optparse into. Since they're simple and generic, I'd
>like to integrate them back in optparser. These features are:
>
>- Add a help formatter which uses capitalized headers.
>- Add a "help" keyword in OptionParser, allowing straightforward
>   addition of a custom help messages.
>
>Comments?
>

It would be nice to have a way for the white space to be cleaned up in the 
help string.  This would be useful when using multiline triple quoted 
strings to pass in help text to the option parser and you want to indent the 
help text on each line so the source code reads well.

_________________________________________________________________
Getting married? Find tips, tools and the latest trends at MSN Life Events. 
http://lifeevents.msn.com/category.aspx?cid=married


From edloper at gradient.cis.upenn.edu  Fri Apr 30 12:24:20 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Fri Apr 30 12:22:52 2004
Subject: [Python-Dev] Small additions to optparser
In-Reply-To: <E1BJaTm-00076C-PW@mail.python.org>
References: <E1BJaTm-00076C-PW@mail.python.org>
Message-ID: <40927DB4.9040508@gradient.cis.upenn.edu>

[Gustavo Niemeyer wrote]
>> Hi Folks,
>>
>> I have two small customizations I've used in most projects I've
>> plugged optparse into. [...] 

[Dan Gass wrote]
> It would be nice to have a way for the white space to be cleaned up in 
> the help string.  This would be useful when using multiline triple 
> quoted strings to pass in help text to the option parser and you want to 
> indent the help text on each line so the source code reads well.

There's already code to do this type of clean-up in inspect.getdoc(). 
Maybe it could be copied (or moved out of getdoc() into its own function 
and reused).

-Edward


From jim.jewett at eds.com  Fri Apr 30 12:26:13 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Fri Apr 30 12:27:06 2004
Subject: [Python-Dev] dict.merge()?
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A2@usahm010.exmi01.exch.eds.com>

At the C level, 

	PyDict_Update(old, new) 

is just a shortcut for

	override=1
	PyDict_Merge(old, new, override)

C extensions can use PyDict_Merge directly with override=0, 
which keeps the current value in case of duplicate keys.

Python code can only use update, which means that override
is always true.  Is there a good reason this?

-jJ

From edloper at gradient.cis.upenn.edu  Fri Apr 30 12:29:04 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Fri Apr 30 12:27:21 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <E1BJY73-00042j-Ql@mail.python.org>
References: <E1BJY73-00042j-Ql@mail.python.org>
Message-ID: <40927ED0.5020402@gradient.cis.upenn.edu>

> Hello Guido,
> 
> I did a quick review of the stdlib, including the tests, to see which list
> comprehensions could be replaced with generator expressions.  Thought I admit
> I am biased towards early binding, I ended up looking for cases with the
> following properties:  [...]

Even if we switched to early binding, won't these issues still bite you 
with mutable values?  E.g.:

     some_dict = {'x': 1, 'y': 2}
     iter1 = (some_dict.get(v, 3) for v in input1)
     some_dict['z'] = 5
     iter2 = (some_dict.get(v, 7) for v in input2)

It seems like it would be surprising (to me, anyway) if this gave a 
different result than:

     some_dict = {'x': 1, 'y': 2}
     iter1 = (some_dict.get(v, 3) for v in input1)
     some_dict = {'x': 1, 'y': 2, 'z': 5}
     iter2 = (some_dict.get(v, 7) for v in input2)

-Edward


From tim.one at comcast.net  Fri Apr 30 12:47:56 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr 30 12:47:55 2004
Subject: [Python-Dev] dict.merge()?
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A2@usahm010.exmi01.exch.eds.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEOCKJAB.tim.one@comcast.net>

[Jewett, Jim J]
> At the C level,
>
> 	PyDict_Update(old, new)
>
> is just a shortcut for
>
> 	override=1
> 	PyDict_Merge(old, new, override)

It is today, but that's a relatively new internal ability; IIRC, Guido added
it to help with the implementation of new-style classes.

> C extensions can use PyDict_Merge directly with override=0,
> which keeps the current value in case of duplicate keys.

Yes.

> Python code can only use update, which means that override
> is always true.

Yes.

> Is there a good reason this?

Dunno -- define "good" <0.5 wink>.


From jcarlson at uci.edu  Fri Apr 30 12:55:43 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Apr 30 13:01:30 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <40927ED0.5020402@gradient.cis.upenn.edu>
References: <E1BJY73-00042j-Ql@mail.python.org>
	<40927ED0.5020402@gradient.cis.upenn.edu>
Message-ID: <20040430093439.138D.JCARLSON@uci.edu>

> Even if we switched to early binding, won't these issues still bite you 
> with mutable values?  E.g.:
> 
>      some_dict = {'x': 1, 'y': 2}
>      iter1 = (some_dict.get(v, 3) for v in input1)
>      some_dict['z'] = 5
>      iter2 = (some_dict.get(v, 7) for v in input2)
> 
> It seems like it would be surprising (to me, anyway) if this gave a 
> different result than:
> 
>      some_dict = {'x': 1, 'y': 2}
>      iter1 = (some_dict.get(v, 3) for v in input1)
>      some_dict = {'x': 1, 'y': 2, 'z': 5}
>      iter2 = (some_dict.get(v, 7) for v in input2)

Semantically, in the first case you are modifying your dictionary, which
has consequences to anything that has a reference to the object. Mutable
modification during runtime should be something we are used to by now.

In the second case, you are rebinding the name some_dict to something
else.  Your earlier binding in the generator expression to some_dict (if
we had early-binding for generator expressions) doesn't change, so the
fact that the two iterators in the second case return different results,
to me (and others I'm sure), is expected.  Name rebinding should also be
something we are used to by now.

I think that it would be very convenient if generator expressions had
the exact same behavior as list comprehensions (people already know how
to do those). As such, early binding is essentially what list
comprehensions do, though they don't suffer from the "I modified my
dictionary before I used my generator expression" issue.  In that case,
the only way we can preserve behavior is through a copy of all the
objects that are bound in early binding, which is obviously a no-no, and
doesn't necessarily solve our problem.

I'm happy with early binding, even given the mutable object modification
issue shown above, and I think it would be relatively easy to document
with a few minor examples. What does everyone else think?


 - Josiah


From pedronis at bluewin.ch  Fri Apr 30 13:10:16 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Fri Apr 30 13:05:48 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <40927ED0.5020402@gradient.cis.upenn.edu>
References: <E1BJY73-00042j-Ql@mail.python.org>
	<E1BJY73-00042j-Ql@mail.python.org>
Message-ID: <5.2.1.1.0.20040430190249.0313a9c0@pop.bluewin.ch>

At 12:29 30.04.2004 -0400, Edward Loper wrote:
>>Hello Guido,
>>I did a quick review of the stdlib, including the tests, to see which list
>>comprehensions could be replaced with generator expressions.  Thought I admit
>>I am biased towards early binding, I ended up looking for cases with the
>>following properties:  [...]
>
>Even if we switched to early binding, won't these issues still bite you 
>with mutable values?  E.g.:
>
>     some_dict = {'x': 1, 'y': 2}
>     iter1 = (some_dict.get(v, 3) for v in input1)
>     some_dict['z'] = 5
>     iter2 = (some_dict.get(v, 7) for v in input2)
>
>It seems like it would be surprising (to me, anyway) if this gave a 
>different result than:
>
>     some_dict = {'x': 1, 'y': 2}
>     iter1 = (some_dict.get(v, 3) for v in input1)
>     some_dict = {'x': 1, 'y': 2, 'z': 5}
>     iter2 = (some_dict.get(v, 7) for v in input2)

well, the deep problem here is that lazinesss and side-effects (or 
mutability, if one wants, inclusive of bindings) do not mix well. There is 
so much that we can do about that.

It's really a matter of, depending on our choices, how often the casual 
user will
be bitten by this and what kind of questions we want to get.

Otherwise we would have to give up about the whole idea.







From barry at python.org  Fri Apr 30 13:30:37 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr 30 13:30:44 2004
Subject: [Python-Dev] Small additions to optparser
In-Reply-To: <BAY18-F88JXOxJagXTi0002378c@hotmail.com>
References: <BAY18-F88JXOxJagXTi0002378c@hotmail.com>
Message-ID: <1083346236.1048.72.camel@anthem.wooz.org>

On Fri, 2004-04-30 at 11:23, Dan Gass wrote:

> It would be nice to have a way for the white space to be cleaned up in the 
> help string.  This would be useful when using multiline triple quoted 
> strings to pass in help text to the option parser and you want to indent the 
> help text on each line so the source code reads well.

I use:

def parseargs():
    parser = OptionParser(...)
    parser.add_option('-a', '--age', action='store', default='1w',
                      help="""\
Confirmations older than this value are removed.  age is a float number
followed by one of the following unit specifier letters:

m - minutes
h - hours
d - days
w - weeks""")

-Barry



From ark-mlist at att.net  Fri Apr 30 13:59:12 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Apr 30 13:59:17 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1083331829.29603.14.camel@anthem.wooz.org>
Message-ID: <000601c42edc$d88e1710$6402a8c0@arkdesktop>

> > This looks (a) cool and (b) a complete hack that nobody should be
> allowed to
> > do without messing with sys._getframe().
> 
> It reminds me a little bit of the dynamic binding in Emacs lisp.  As
> incredibly useful as that is sometimes, it's a disgusting hack. :)

And, if I understand correctly, something from which the Lisp community is
moving away--preferring, instead, the notion of "lexical scoping" such as in
Scheme.


From barry at python.org  Fri Apr 30 14:01:32 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr 30 14:01:44 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <000601c42edc$d88e1710$6402a8c0@arkdesktop>
References: <000601c42edc$d88e1710$6402a8c0@arkdesktop>
Message-ID: <1083348092.1048.91.camel@anthem.wooz.org>

On Fri, 2004-04-30 at 13:59, Andrew Koenig wrote:
> > > This looks (a) cool and (b) a complete hack that nobody should be
> > allowed to
> > > do without messing with sys._getframe().
> > 
> > It reminds me a little bit of the dynamic binding in Emacs lisp.  As
> > incredibly useful as that is sometimes, it's a disgusting hack. :)
> 
> And, if I understand correctly, something from which the Lisp community is
> moving away--preferring, instead, the notion of "lexical scoping" such as in
> Scheme.

That's my understanding too.  Back when I followed such things, there
was a lot of discussion in the XEmacs lists about implementing lexical
scoping and getting rid of this ugly hack (eventually -- no doubt
there's tons of old elisp that would break as a result).

tim-would-have-fixed-python-mode-ly y'rs,
-Barry



From nbastin at opnet.com  Fri Apr 30 14:01:40 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Fri Apr 30 14:01:56 2004
Subject: [Python-Dev] Accessing base class C struct of subclass instance
Message-ID: <6E9B692A-9AD0-11D8-961D-000393CBDF94@opnet.com>

Ok, strange subject, but I have what seems like a simple question that 
the documentation doesn't seem to cover (and a quick perusal of the 
code doesn't answer, either).

If I define a new style class in C, and subclass it in python, how can 
I get the base class instance object from a subclass instance in C?

--
Nick


From nbastin at opnet.com  Fri Apr 30 14:15:15 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Fri Apr 30 14:15:42 2004
Subject: [Python-Dev] Accessing base class C struct of subclass instance
In-Reply-To: <6E9B692A-9AD0-11D8-961D-000393CBDF94@opnet.com>
References: <6E9B692A-9AD0-11D8-961D-000393CBDF94@opnet.com>
Message-ID: <5404E33E-9AD2-11D8-961D-000393CBDF94@opnet.com>


On Apr 30, 2004, at 2:01 PM, Nick Bastin wrote:

> Ok, strange subject, but I have what seems like a simple question that 
> the documentation doesn't seem to cover (and a quick perusal of the 
> code doesn't answer, either).
>
> If I define a new style class in C, and subclass it in python, how can 
> I get the base class instance object from a subclass instance in C?

Nevermind...the obvious solution works correctly.

--
Nick


From cm at leetspeak.org  Fri Apr 30 14:29:49 2004
From: cm at leetspeak.org (Michael Walter)
Date: Fri Apr 30 14:29:55 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <000601c42edc$d88e1710$6402a8c0@arkdesktop>
References: <000601c42edc$d88e1710$6402a8c0@arkdesktop>
Message-ID: <40929B1D.60608@leetspeak.org>

Andrew Koenig wrote:
>>>This looks (a) cool and (b) a complete hack that nobody should be
>>
>>allowed to
>>
>>>do without messing with sys._getframe().
>>
>>It reminds me a little bit of the dynamic binding in Emacs lisp.  As
>>incredibly useful as that is sometimes, it's a disgusting hack. :)
> 
> 
> And, if I understand correctly, something from which the Lisp community is
> moving away--preferring, instead, the notion of "lexical scoping" such as in
> Scheme.
Well, global variables in Common Lisp are still dynamically-scoped, 
which makes perfect sense IMO.

Also, I think there is a SRFI almost every Scheme implements (SRFI ~ 
PEP) which add dynamic bindings.

Cheers,
Michael

From jeremy at alum.mit.edu  Fri Apr 30 16:38:55 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Apr 30 16:40:43 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1083331829.29603.14.camel@anthem.wooz.org>
References: <200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404301216.i3UCGXU07211@guido.python.org>
	<20040430130700.GA28030@vicky.ecs.soton.ac.uk>
	<1083331829.29603.14.camel@anthem.wooz.org>
Message-ID: <1083357535.9092.148.camel@localhost.localdomain>

On Fri, 2004-04-30 at 09:30, Barry Warsaw wrote:
> On Fri, 2004-04-30 at 09:07, Armin Rigo wrote:
> 
> > This looks (a) cool and (b) a complete hack that nobody should be allowed to
> > do without messing with sys._getframe().
> 
> It reminds me a little bit of the dynamic binding in Emacs lisp.  As
> incredibly useful as that is sometimes, it's a disgusting hack. :)

The funny thing is that it's the result of a static scoping discipline
rather than dynamic scoping.  What's funny about it has more to do with
side-effects that scoping rules.  If the x_square function was returned
out of its defining block, there would be no way to rebind x and it
would not be possible to define a variable x local to the caller that
would affect it.

Put another way, it's possible to reason statically about what binding x
will use when x_square() is called.  It's the same technique you would
use to write an accumulator generator (that is, a function that returns
accumulator functions).  See the appendix of Paul Graham's essay:
http://www.paulgraham.com/icad.html

Jeremy

PS I do owe everyone a PEP on the subject of re-binding.



From rowen at cesmail.net  Fri Apr 30 17:00:08 2004
From: rowen at cesmail.net (Russell E. Owen)
Date: Fri Apr 30 17:00:22 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's Move
	Forward
References: <20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404301216.i3UCGXU07211@guido.python.org>
	<5.2.1.1.0.20040430142620.030304e8@pop.bluewin.ch>
Message-ID: <rowen-08D028.14000830042004@sea.gmane.org>

In article <5.2.1.1.0.20040430142620.030304e8@pop.bluewin.ch>,
 Samuele Pedroni <pedronis@bluewin.ch> wrote:

>...
> this assumes that people will choose either generator expressions or list 
> comprehesions on a case-by-case basis. I'm not sure it's a reasonable 
> assumption for the whole range of users - starting with newbies. Probably 
> documentation and what is predicated on c.l.p will influence this.

I completely agree. Generator expressions and list comprehensions are 
similar enough that it seems a wart to have to carefully differentiate 
between two in normal use. 

Personally, I'd only like to see generator expressions added to Python 
if they can allow list comprehensions to be deprecated -- to avoid 
language clutter.

One question, though: is "if" part of generator expressions or not? I 
read the current version of the PEP and I'm confused. It offers the 
example:

max(len(line)  for line in file  if line.strip())

But that is the only instance in the PEP that I noticed. The section 
"The Details" says almost nothing about "if". The only thing I found 
that might be relevant appeared at the end of: "2. The syntax 
requires...": "...where listmaker1 is almost the same as listmaker, but 
only allows a single test after 'for' ... 'in'." I suppose that might be 
saying that a single "if" condition is allowed. If so, great!

-- Russell


From tim.one at comcast.net  Fri Apr 30 17:28:28 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr 30 17:28:27 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's
	MoveForward
In-Reply-To: <rowen-08D028.14000830042004@sea.gmane.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPOKJAB.tim.one@comcast.net>

[Russell E. Owen]
> I completely agree. Generator expressions and list comprehensions are
> similar enough that it seems a wart to have to carefully differentiate
> between two in normal use.

In normal use you won't have to distinguish, where "normal use" is pretty
much defined by "cases where it doesn't matter" <wink>.

> Personally, I'd only like to see generator expressions added to Python
> if they can allow list comprehensions to be deprecated -- to avoid
> language clutter.

Listcomps can be deprecated, although not all existing uses of listcomps
will translate directly into genexps.

I think this has become a "practicality beats purity" feature.  When Python
had only 3 scopes, it was fine to ignore "explicit is better than implicit":
the scope model was so bare that explictness would have been overkill.  When
multiple nested scopes got introduced, and function bodies start appearing
in places other than lambda and def (which is the case for genexps), scope
gets much more mysterious (and so also potentially surprising) in the
absence of explicit scope declarations.  But simple genexps will work fine
anyway almost all the time, so it doesn't appear to matter much that devious
uses will have nightmarish semantics.

> One question, though: is "if" part of generator expressions or not?

Yes.  If the PEP isn't clear about this, it's a flaw in the PEP.  I don't
think the semantics of genexps are clear in the PEP either (unless that's
changed recently).


From eppstein at ics.uci.edu  Fri Apr 30 18:09:23 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Fri Apr 30 18:09:06 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's MoveForward
References: <rowen-08D028.14000830042004@sea.gmane.org>
	<LNBBLJKPBEHFEDALKOLCAEPOKJAB.tim.one@comcast.net>
Message-ID: <eppstein-E06EE9.15092230042004@sea.gmane.org>

In article <LNBBLJKPBEHFEDALKOLCAEPOKJAB.tim.one@comcast.net>,
 "Tim Peters" <tim.one@comcast.net> wrote:

> > Personally, I'd only like to see generator expressions added to Python
> > if they can allow list comprehensions to be deprecated -- to avoid
> > language clutter.
> 
> Listcomps can be deprecated, although not all existing uses of listcomps
> will translate directly into genexps.

Not all?  When would [listcomp] ever not be replaceable by list(gencomp)?

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From barry at python.org  Fri Apr 30 18:11:44 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr 30 18:11:52 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's
	MoveForward
In-Reply-To: <eppstein-E06EE9.15092230042004@sea.gmane.org>
References: <rowen-08D028.14000830042004@sea.gmane.org>
	<LNBBLJKPBEHFEDALKOLCAEPOKJAB.tim.one@comcast.net>
	<eppstein-E06EE9.15092230042004@sea.gmane.org>
Message-ID: <1083363103.3975.123.camel@anthem.wooz.org>

On Fri, 2004-04-30 at 18:09, David Eppstein wrote:

> Not all?  When would [listcomp] ever not be replaceable by list(gencomp)?

I dunno.  I happen to like [listcomp] syntax over list(genexpr).

TOOWTDI-be-damned-ly y'rs,
-Barry



From tim.one at comcast.net  Fri Apr 30 18:20:55 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri Apr 30 18:21:04 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's
	MoveForward
In-Reply-To: <eppstein-E06EE9.15092230042004@sea.gmane.org>
Message-ID: <E1BJgNS-0004bj-1d@mail.python.org>

>> Listcomps can be deprecated, although not all existing uses of listcomps
>> will translate directly into genexps.

[David Eppstein]
> Not all?  When would [listcomp] ever not be replaceable by list(gencomp)?

The point of Armin's original msg in this thread was that direct replacement
can have different semantics.  Applying a transformation (via list()) is a
different story, although I expect someone obsessed enough could concoct an
example that worked differently, because genexp guts truly live in a
different scope than listcomp guts (e.g., play games with locals()).



From guido at python.org  Fri Apr 30 19:12:59 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Apr 30 19:13:08 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: Your message of "Fri, 30 Apr 2004 14:07:00 BST."
	<20040430130700.GA28030@vicky.ecs.soton.ac.uk> 
References: <200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404301216.i3UCGXU07211@guido.python.org> 
	<20040430130700.GA28030@vicky.ecs.soton.ac.uk> 
Message-ID: <200404302313.i3UND0a08111@guido.python.org>

> I was using this example more generally to show that if you have a
> late-binding genexpr, you have to worry about not changing the local
> variables it uses, which looks like all uses of genexprs with free
> variables are subtle bugs waiting to appear whenever someone
> modifies the function anywhere after the genexpr.  The same argument
> applies to the other stdlib cases I mentioned (which make somewhat
> less than 10% of all uses).

That's not new, we all know that that's the downside of late binding.

My counter to that is that *any* use of genexps where the consumer
doesn't consume (or discard) the iterator before the next line is
reached is extremely advanced use.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Apr 30 19:27:37 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Apr 30 19:27:48 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's
	MoveForward
In-Reply-To: <4131456.1083338892@dhcp31-158.ics.uci.edu>
References: <rowen-08D028.14000830042004@sea.gmane.org>
	<LNBBLJKPBEHFEDALKOLCAEPOKJAB.tim.one@comcast.net>
	<eppstein-E06EE9.15092230042004@sea.gmane.org>
	<1083363103.3975.123.camel@anthem.wooz.org>
	<4131456.1083338892@dhcp31-158.ics.uci.edu>
Message-ID: <1083367654.3975.135.camel@anthem.wooz.org>

On Fri, 2004-04-30 at 18:28, David Eppstein wrote:

> Oh, me too (and I'm getting a little irritated with writing dict([...]) so 
> often instead of having a proper dictcomp syntax) but semantically I think 
> they are the same.

I'm easily convinced.

pep-274-ly y'rs,
-Barry



From eppstein at ics.uci.edu  Fri Apr 30 18:28:12 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Sat May  1 18:10:39 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's
	MoveForward
In-Reply-To: <1083363103.3975.123.camel@anthem.wooz.org>
References: <rowen-08D028.14000830042004@sea.gmane.org>
	<LNBBLJKPBEHFEDALKOLCAEPOKJAB.tim.one@comcast.net>
	<eppstein-E06EE9.15092230042004@sea.gmane.org>
	<1083363103.3975.123.camel@anthem.wooz.org>
Message-ID: <4131456.1083338892@dhcp31-158.ics.uci.edu>

On 4/30/04 6:11 PM -0400 Barry Warsaw <barry@python.org> wrote:
>> Not all?  When would [listcomp] ever not be replaceable by list(gencomp)?
>
> I dunno.  I happen to like [listcomp] syntax over list(genexpr).

Oh, me too (and I'm getting a little irritated with writing dict([...]) so 
often instead of having a proper dictcomp syntax) but semantically I think 
they are the same.
-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From peter at designtheory.org  Thu Apr 29 15:01:33 2004
From: peter at designtheory.org (Peter Dobcsanyi)
Date: Sat May  1 18:19:43 2004
Subject: [Python-Dev] bags in collections - PEP 320
Message-ID: <20040429190133.GA4568@designtheory.org>

Dear All,

I have just read PEP-320 and noticed the following comments regarding
the "collections" package:

    ...
    - ? bag (only if use cases established)
    ...

I would like to argue for such a use case. We at designtheory.org are
working on a (maths / stats / comp. sci.) project in the area of Design
Theory. Design theory intersects with many fields, to mention a few:
combinatorics, graph theory, finite geometry, coding theory, and the
design of statistical experiments (from which then name of the field
comes).  Most of our software development will be in Python, although
the released python software so far is very modest.

To show why bags are important for us and, in general, for discrete
mathematicians, here is the definition of the most important type of
designs. A binary block design is a multiset (that is a 'bag') of
subsets of a 'base' set; the elements of the base set are called
'points' and its subsets are called 'blocks'. (Personally, I prefer the
name bag but most mathematicians use multiset.) A non-binary block
design is one whose blocks can also be multisets.

The computations we are facing are very much combinatorial in nature so
we are happy to see that Sets became a builtin and, of course, would
like to see a C implementation for bags too. Our pydesign package (will)
heavily use C extensions. So far we use numarray to compute statistical
properties of designs, but many more combinatorial functionalities will
be implemented in C.

In particular, we are planning to implement a basic permutation group
package whose core will eventually be a C extension. We would like to
deal with automorphism groups and isomorphisms of designs on at least a
basic level within the pydesign package without having to resort to
specialized mathematical packages like GAP. These functionalities can be
useful not only for design theorist, but for anybody using Python to
deal with combinatorial structures, like graphs for example.

In all of these areas the use of multisets (bags) is pervasive.
Please, implement it  --  so we don't have to :-)

For more details on our project, please visit http://designtheory.org/

--             ,
    Peter Dobcsanyi